Proceedings of the 7th European Conference on Games Based Learning ECGBL 2013 Volume 2 by ACPIL

Proceedings of the 7th European Conference on Games-Based Learning Instituto Superior de Engenharia do Porto, Portugal 3-4 October 2013

Volume Two

Edited by Dr Carlos Vaz de Carvalho and Dr Paula Escudeiro Instituto Superior de Engenharia do Porto Portugal A conference managed by ACPI, UK www.academic-conferences.org

The Proceedings of The 7th European Conference on Games Based Learning Hosted by Instituto Superior de Engenharia do Porto (ISEP) Portugal Volume Two 3-4 October 2013 Edited by Dr Paula Escudeiro Programme Chair and Carlos Vaz de Carvalho Conference Chair

Copyright The Authors, 2013. All Rights Reserved. No reproduction, copy or transmission may be made without written permission from the individual authors. Papers have been double-blind peer reviewed before final submission to the conference. Initially, paper abstracts were read and selected by the conference panel for submission as possible papers for the conference. Many thanks to the reviewers who helped ensure the quality of the full papers. These Conference Proceedings have been submitted to Thomson ISI for indexing. Please note that the process of indexing can take up to a year to complete. Further copies of this book and previous yearâ&#x20AC;&#x2122;s proceedings can be purchased from http://academic-bookshop.com E-Book ISBN: 978-1-909507-65-4 E-Book ISSN: 2049-100X Book version ISBN: 978-1-909507-63-0 Book Version ISSN: 2049-0992 CD Version ISBN: 978-1-909507-66-1 CD Version ISSN: 2049-1018 The Electronic version of the Proceedings is available to download at ISSUU.com. You will need to sign up to become an ISSUU user (no cost involved) and follow the link to http://issuu.com Published by Academic Conferences and Publishing International Limited Reading UK 44-118-972-4148 www.academic-publishing.org

Contents Paper Title

Author(s)

Page No.

Preface

vii

Committee

viii

Biographys

Volume One Teachers’ Beliefs About Game Based Learning: A Comparative Study of Pedagogy, Curriculum and Practice in Italy, Turkey and the UK

Yasemin Allsop, Ebru Yeniman Yildirim and Marina Screpanti

Using Gamification to Animate a Virtual Community

António Andrade and Carlos Vaz de Carvalho

SIMaging the CITY: The Educational use of Simulation Video Games for Disadvantaged Youth

Massimiliano Andreoletti and Gianna Cappello

“The Chest That Longs to be Moved”: A Serious Game for the Greek Muslim Minority Children

Alexandra Androussou, Evangelia Kourti and Nelly Askouni

Transformational Play; Using 3D Game-Based Narratives to Immerse Students in Literacy Learning

Anna Arici and Sasha Barab

Approaches to Collaborative Game-Making for Fostering 21st Century Skills

Susan Bermingham, Nathalie Charlier, Francesca Dagnino, James Duggan, Jeffrey Earp, Kristian Kiili, Evelien Luts, Lien van der Stock and Nicola Whitton

Best Practices for Deploying Digital Games for Personal Empowerment and Social Inclusion

Lizzy Bleumers, Ilse Mariën, Jan Van Looy, James Stewart, Dana Schurmans and Anissa All

Investigating the Relationship Between School Performance and the Abilities to Play Mind Games

Rosa Maria Bottino, Michela Ott and Mauro Tavella

Experience With Digital Game-Based Embodied Learning: The Road to Create a Framework for Physically Interactive Digital Games

Carsten Busch, Florian Conrad, Robert Meyer and Martin Steinicke

Toward Improvement of Serious Game Reliability

Thibault Carron, Fabrice Kordon, Jean-Marc Labat, Isabelle Mounier and Amel Yessad

The Effects of Gamification on Student Attendance and Team Performance in a Third-Year Undergraduate Game Production Module

Hope Caton and Darrel Greenhill

Game-Based Learning in Health Sciences Education

Nathalie Charlier, Evelien Luts and Lien Van Der Stock

Specification and Design of a Generalized Assessment Engine for GBL Applications

Yaëlle Chaudy, Thomas Connolly and Thomas Hainey

105

Safer Internet: Enhancing Good Practices on the Internet Through Games Based Learning for Greek Elementary School Students

Vasiliki Choleva, Loukas Koutsikos Simeon Zourelidis, Vlassios Filis, Dimitris Metafas and Charalampos Patrikakis

115

Using Game Mechanics to Measure What Students Learn

Jill Denner, Linda Werner, Shannon Campe and Eloy Ortiz

123

Combining Game Based Learning With Content and Language Integrated Learning Approaches: A Case Study Utilizing QR Codes and Google Earth in a Geography-Based Game

Kyriaki Dourda, Tharrenos Bratitsis, Eleni Griva and Penelope Papadopoulou

130

Paper Title

Author(s)

Page No.

The Design and Evaluation of a Multiplayer Serious Game for Pharmacy Students

Maciej Dudzinski , Darrel Greenhill , Reem Kayyali , Shereen Nabhani , Nada Philip , Hope Caton , Sonya Ishtiaq and Francis Gatsinzi

140

Cheating and Creativity in Pervasive Games in Learning Contexts

Stine Ejsing-Duun, Thorkild Hanghøj and Helle Skovbjerg Karoff

149

Supporting Teachers in the Process of Adoption of Game Based Learning Pedagogy

Valérie Emin-Martinez and Muriel Ney

156

Cognitive Walkthrough for Learning Through Game Mechanics

David Farrell and David Moffat

163

Global Math: Development of Online Platform for Mathematical Thinking Games

Toru Fujimoto, Keiichi Nishimura, Kaoru Takahashi, Masahiro Yachi, Kiyoshi Takahashi and Yuhei Yamauchi

172

What Can Play Theory Tell us About Computer Games for Young Children?

Georgy Gerkushenko and Svetlana Sokolova

179

Role Game Playing as a Platform for Creative and Collaborative Learning

Lisa Gjedde

190

Development and Evaluation of a Generic E-CLIL Web2.0 Games Engine

Thomas Hainey and Thomas Connolly

198

Designing Games to Disseminate Research Findings

Claire Hamshire, Rachel Forsyth and Nicola Whitton

208

Facilitating Teacher Students’ Innovation Competence through Problem-Based Game Design Processes

Thorkild Hanghøj and Sia Hovmand Sørensen

216

Deploying Serious Games for Management in Higher Education: Lessons Learned and Good Practices

Jannicke Baalsrud Hauge, Francesco Bellotti, Rob Nadolski, Michael Kickmeier-Rust, Riccardo Berta and Maira Carvalho

225

Neuroeducational Research in the Design and use of Games-Based Teaching

Wayne Holmes, Paul Howard-Jones, Erico Tanimoto, Carol Jones, Skevi Demetriou, Owen Morgan, Philip Perkins and Neil Davies

235

Playing and Learning: An iPad Game Development Case Study

Jennifer Jenson and Rachel Muehrer

244

An Overview of Game Console Motion Sensor Technologies Exploited for Education

Marina Kandroudi and Tharrenos Bratitsis

252

Picking the Right Interface for Engaging Physical Activity Into Game Based Learning

Helle Skovbjerg Karoff, Gunver Majgaard, Lars Elbæk and Mona Have Sørensen

261

Playing and Gaming – Studied in an Informal Learning Setting

Helle Skovbjerg Karoff, Stine Ejsing-Duun and Thorkild Hanghøj

268

Game Based Learning in Mathematics: Teachers' Support by a Flexible Tool

Aikaterini Katmada, Apostolos Mavridis, Thrasyvoulos Tsiatsos

275

Learning Analytics with Games Based Learning

Harri Ketamo

284

Gamification and Intelligent Feedback Mechanisms for a Division Learning Tool

Michael Kickmeier-Rust and Dietrich Albert

290

Developing Games for Health Impact: Case Brains vs Zombies

Kristian Kiili, Manuel Ninaus, Mikko Koskela, M Tuomi and Antero Lindstedt

297

Meleon - a Casual Mobile Game Supporting Immersion and Reflection in Learning

Luise Klein

305

Paper Title

Author(s)

Page No.

The Literature Race - NFC Based Mixed Reality Game

Antti Koivisto, Harri Ketamo, Eero Hammais and Juho Salli

314

Bringing Game Achievements and Community Achievements Together

Johannes Konert, Nico Gerwien, Stefan Göbel and Ralf Steinmetz

319

Modeling the Player, Learner and Personality: Independency of the Models of Bartle, Kolb and NEO-FFI (Big5) and the Implications for Game Based Learning

Johannes Konert, Stefan Göbel and Ralf Steinmetz

329

Raising Awareness on Archaeology: A Multiplayer Game-Based Approach With Mixed Reality

Mathieu Loiseau, Élise Lavoué, Jean-Charles Marty and Sébastien George

336

Scientific Discovery Games for Authentic Science Education

Rikke Magnussen, Sidse Damgaard Hansen, Tilo Planke, Jacob Friis Sherson

344

Creating Games in the Classroom – From Native Gamers to Reflective Designers

Gunver Majgaard

352

A Holistic Framework for the Development of an Educational Game Aiming to Teach Computer Programming

Christos Malliarakis, Maya Satratzemi and Stelios Xinogalos

359

Examining Early Childhood Education Students’ Attitudes Toward Educational Computer Games in Kindergarten

Dionissios Manessis

369

Integrating Non-Virtual Electronic Activities in Game-Based Learning Environments

Jean-Charles Marty, Thibault Carron, Stéphane Talbot, Gregory Houzet and Philippe Pernelle

378

From « Haute-Couture » to « Ready-to-Wear »: Typology of Serious Games Implementation Strategies in Higher Education

Hélène Michel

386

Motivation and Manipulation: A Gamification Approach to Influencing Undergraduate Attitudes in Computing

Nicholas Mitchell, Nicky Danino and Lesley May

394

Sit Down to Table and Confess who you are! Design of an Epistemic Game for Nutritional Education at Secondary School

Réjane Monod-Ansaldi, Eric Sanchez, Daniel Devallois, Thomas Abad, Pierre Bénech, Anne Brondex, Isabelle Mazzella, Sandrine Miranda, Claudie Richet and Céline Recurt

401

Learning in Context Through Games: Towards a new Typology

Alex Moseley

409

Let the Students Construct Their own fun And Knowledge - Learning to Program by Building Computer Games

Peter Mozelius, Olga Shabalina, Christos Malliarakis, , Florica Tomos, Chris Miller and David Turner

418

Towards Understanding the Instructional Value of Real-Time Continuous Feedback From the use of Simulation Games

Mathews Nkhoma, Jaime Calbeto, Narumon Sriratanaviriyakul, Thu Yein Win, Quyen Ha Tran and Thanh Kim Cao

427

Learning Math as you Play: Comparing Arithmetic Performance Enhancement Induced by Game Play and Paper Exercises

Elena Patricia Nuñez Castellar, Anissa All and Jan Van Looy

434

Serious Game Adaptive Learning Systems

Chinedu Obikwelu and Janet Read

442

Combatting Social Isolation and Cognitive Decline: Play a Physical or Digital Game?

Daire Ó Broin and Ross Palmer

450

Volume Two

iii

Paper Title

Author(s)

Page No.

Sports Games’ Role for Learning Health Knowledge

Kelly O’Hara, Dulce Esteves, Rui Brás, Ricardo Rodrigues, Paulo Pinheiro and Marco Rodrigues

458

A Multi-Agent Architecture for Collaborative Serious Game Applied to Crisis Management Training: Improving Adaptability of non Played Characters

M’hammed Ali Oulhaci, Erwan Tranvouez, Sébastien Fournier and Bernard Espinasse

465

Nuclear Mayhem – a Pervasive Game Designed to Support Learning

Trygve Pløhn

475

StartUp_EU: Using Game-Based Learning and Web 2.0 Technologies to Teach Entrepreneurship to Secondary Education Students

Aristidis Protopsaltis, Thomas Hainey, Spiros Borosis, Thomas Connolly, Jesus Copado and Sonia Hezner

484

Measuring Effects of Reflection on Learning: A Physiological Study

Wen Qi, Dominique Verpoorten and Wim Westera

495

Evaluation of Adaptive Serious Games using Playtraces and Aggregated Play Data

Christian Reuter, Florian Mehm, Stefan Göbel and Ralf Steinmetz

504

Learning Effectiveness of Management Simulation Game Manahra

Petr Smutný, Jakub Procházka and Martin Vaculík

512

Using the Master Copy - Adding Educational Content to Commercial Video Games

Heinrich Söbke, Thomas Bröker and Oliver Kornadt

521

An Application of Adaptive Games-Based Learning Based on Learning Style to Teach SQL

Mario Soflano, Thomas Connolly and Thomas Hainey

531

Can Moral Sensitivity be Enhanced by Game Play?

Gunilla Svingby

539

Digital Educational Games: Adopting Pedagogical Agent to Infer Leaner‘s Motivation and Emotional State

Ogar Ofut Tumenayu and Olga Shabalina

546

Adapting the Complexity Level of a Serious Game to the Proficiency of Players

Herre van Oostendorp, Erik van der Spek and Jeroen Linssen

553

Designing Casual Serious Games in Science

Ayelet Weizman

561

Designing a Collaborative Serious Game for Team Building Using Minecraft

Viktor Wendel, Michael Gutjahr, Philipp Battenberg, Roman Ness, Sebastian Fahnenschreiber, Stefan Göbel and Ralf Steinmetz

569

Application of the Principles of Gamification to Facilitate Acquisition of Self-Management Skills in Young People With Long-Term Medical Conditions

Andrew Wilson and Janet McDonagh

579

Development of an Implementation Framework for Games-Based Construction Learning Using Scratch in Primary Education

Amanda Wilson, Thomas Hainey and Thomas Connolly

587

Game Literacy Revisited: Developing Critical Play in Schools

Rafael Marques de Albuquerque and Shaaron Ainsworth

599

A Systematic Literature Review of Methodology Used to Measure Effectiveness in Digital GameBased Learning

Anissa All, Elena Patricia Nuñez Castellar and Jan Van Looy

607

Investigating Collaborative Games to Teach Mathematics-Based Problem Solving in the Classroom

Reem Al-Washmi, Gail Hopkins and Peter Blanchfield

617

PHD papers

Paper Title

Author(s)

Page No.

Training Flexible and Adaptive Arithmetic Problem Solving Skills Through Exploration With Numbers: The Development of NumberNavigation Game

Boglárka Brezovszky, Erno Lehtinen, Jake McMullen, Gabriela Rodriguez and Koen Veermans

626

Trials to Assess Team-Based Mixed-reality (TBMR) Games in HE

John Denholm and Sara de Freitas

635

Understanding ‘Game-Ness’ Within the ® SCRABBLE Family of English Word Games

Paridhi Gupta

645

Interactive Story as a Motivator Element in an Educational Video Game

José Rafael López-Arcos, Francisco Luis Gutiérrez Vela, Natalia Padilla-Zea and Patricia Paderewski

656

A Domain Ontology of Game Theory Applied to Game Based Learning

Yemna Mejbri, Maha Khemaja and Rafik Braham

666

Puzzle-Based Games as a Metaphor for Designing in Situ Learning Activities

Javier Melero, Patricia Santos, Davinia HernándezLeo and Josep Blat

674

Supporting and Facilitating Collaborative Learning in Serious Games

Kimmo Oksanen and Raija Hämäläinen

683

Playing for the Future - Examining Gameplay, Narrative and fun in Games-Based Training

Mark O’Rourke

691

Towards Game Based Learning Design Process Based on Semantic Service Oriented Architecture (SSOA)

Kaouther Raies, Maha Khemaja and Rafik Brahamm

698

Using Games for Learning, From the Students’ Perspectives

Aishah Abdul Razak and Thomas Connolly

706

Incidental Learning in a World of Warcraft Guild, a Case Study

Gabriela Rodríguez

714

In Search for the Right Measure: Assessing Types of Developed Knowledge While Using a Gamified Web Toolkit

Martin Ruskov, Paul Ekblom and Angela Sasse

722

The Influence of Digital Games on Learning Reading: A Closer Look

Mas Idayu Md Sabri, Peter Blanchfield and Gail Hopkins

730

The Mediatization of Digital Games for Learning – a Dual Rub-Off Effect

Helga Sigurdardottir and Robin Munkvold

740

Efficacy of Reward Allotment on Children’s Motivation and Learning

Zhenhua Xu, Earl Woodruff and Bodong Chen

748

Applying Ideas From Intelligent Tutoring Systems for Teaching Programming in Game Based Learning

Matej Zapušek and Jože Rugelj

756

Cultivating Preschoolers Creativity Using Guided Interaction With Problem Solving Computer Games

Georgios Fessakis and Dimitrios Lappas

763

Haptic Physics Simulation

Luciano Santos and Carlos Vaz de Carvalho

771

Evaluating the Embedding of Games Based Learning in a Computing Subject at University

Emilia Todorova and David Moffat

776

Masters

WIP Papers

Paper Title

Author(s)

Page No.

A Design Approach for Implementing 3D Educational Collaborative Virtual Environments on Virtual World Platforms

Rosa Reis, Benjamin Fonsecaand Paula Escudeiro

785

EMOTE: Embodied-Perceptive Tutors for Empathy-Based Learning in a Game Environment

Sofia Serholt, Wolmet Barendregt, Tiago Ribeiro, Ginevra Castellano, Ana Paiva, Arvid Kappas, Ruth Aylett and Fernando Nabais

790

Exploring Learning Effects During Virtual Sports Through Biomechanical Analysis (a Work in Progress)

Pooya Soltani and Jo達o Paulo Vilas-Boas

793

siLang: Culturally Oriented Language Skill Development in Line With Workplace Needs

Hariklia Tsalapatas, Olivier Heidmann, Rene Alimisi and Elias Houstis

797

Developing Ethical Decision Making Skill of Novice Volunteers in Natural Disaster Response

Didin Wahyudin, Shinobu Hasegawa and Tina Dahlan

800

Preface These proceedings represent the work of researchers participating in the 7th European Conference on GamesBased Learning, which is being organised and hosted this year by the Instituto Superior de Engenharia do Porto (ISEP), Porto, Portugal. The Conference Chair is Dr Carlos Vaz de Carvalho and the Programme Chair is Dr Paula Escudeiro, both from the Instituto Superior de Engenharia do Porto, Porto, Portugal. The conference will be opened with a keynote from Dr. Baltasar Fernández-Manjón, from Complutense University of Madrid, Spain, on the topic of Learning Analytics Applied to Serious Games. The opening keynote address on the second day is by Paulo Gomes, Game Director and Producer at BIGMOON STUDIOS. The Conference is a valuable platform for individuals to present their research findings, display their work in progress and discuss conceptual advances in many different areas and specialties within Games-Based Learning. It also offers the opportunity for like minded individuals to meet, discuss and share knowledge. ECGBL continues to evolve and develop, and the wide range of papers and topics will ensure an interesting twoday confercence. In addition to the main streams of the conference, there are mini tracks focusing on the areas of Multi-User Virtual Environments, Content and Assessment Integration, User Profiling and Barriers and Opportunities for the introduction of GBL in Educational Settings. In addition to the presentations of research the conference this year has introduced a competition to provide an opportunity for educational game designers and creators to participate in the conference and demonstrate their game design and development skills in an international competition. This competition has been sponsored by SEGAN – Serious Games Network. With an initial submission of more than 50 games, 24 finalists will present their games at the conference. Prizes will be awarded to the three games judged to demonstrate the best quality and originality of game play itself and the positioning and articulation of the game’s contribution to the educational domain. With an initial submission of 179 abstracts, after the double blind peer review process, there are 71 research papers, 18 PhD research papers, 3 Masters research papers and 5 work-in-progress paperspublished in these Conference Proceedings. These papers represent research more than 30 countries, including Algeria, Australia, Austria, Belgium, Brazil, Canada, Czech Republic, Denmark, Finland, France, Germany, Greece, Hong Kong, India, Ireland, Israel, Italy, Japan, The Netherlands Norway, Portugal, Russia, Slovenia, Spain, Sweden, Switzerland, Tunisia, UK, USA and Vietnam. We hope that you have an enjoyable conference. Dr Paula Escudeiro Programme Chair and Carlos Vaz de Carvalho Conference Chair October 2013

vii

Conference Committee ECGBL Conference Director Professor Thomas M Connolly, University of the West of Scotland, UK Conference Executive: Dr Carlos Vaz de Carvalho, Instituto Superior de Engenharia do Porto, Portugal Dr Paula Escudeiro, Instituto Superior de Engenharia do Porto, Portugal Dulce Mota, Instituto Superior de Engenharia do Porto (ISEP), Porto, Portugal Isabel Azevedo, Instituto Superior de Engenharia do Porto (ISEP), Porto, Portugal António Castro, Instituto Superior de Engenharia do Porto (ISEP), Porto, Portugal Ana Barata, Instituto Superior de Engenharia do Porto (ISEP), Porto, Portugal Bertil Marques, Instituto Superior de Engenharia do Porto (ISEP), Porto, Portugal Rosa Reis, Instituto Superior de Engenharia do Porto (ISEP), Porto, Portugal Mini Track Chairs Prof. Dr Wilfried Admiraal, Leiden University, The Netherlands Dr Kristian Kiili, Tampere University of Technology, Finland Prof Konstantinos Kalemis, National and Kapodistrian University of Athens, Dr Thomas Hainey, University of the West of Scotland, UK Dr Jordi Sánchez-Navarro, Open University of Catalonia, Barcelona, Spain Dr Daniel Aranda, Open University of Catalonia, Barcelona, Spain Dr Stefan Göbel, Technical University of Darmstadt, Germany Viktor Wendel, Technical University of Darmstadt, Germany Johannes Konert, Technical University of Darmstadt, Germany Committee Members The 2013 conference programme committee consists of key people in the games based learning community, both from the UK and overseas. The following people have confirmed their participation: Dr Wilfried Admiraal (Leiden University, Leiden, The Netherlands); Dr. Minoo Alemi (Sharif University of Technology, Iran); Anissa All (iMinds-MICT-Ghent University, Belgi); Daniel Aranda (Universitat Oberta de Catalunya, Spain); Nikolaos Avouris (University of Patras, Greece); Isabel Azevedo ( Instituto Superior de Engenharia do Porto (ISEP), Portugal); Ana Barata ( Instituto Superior de Engenharia do Porto (ISEP), Portugal);Dr. Wolmet Barendregt (Gothenburg University, department of applied IT, Sweden); Francesco Bellotti (University of Genoa, Italy,); Mary Bendixen-Noe (Ohio State University, USA);Dr Tobias Bevc (University of Augsburg, Germany);Dr Bani Bhattacharya (IIT Kharagpur, India); Dr Peter Blanchfield (School of Computer Science, University of Nottingham, UK); Natasha Boskic (The University of British Columbia, Canada);Dr. Rosa Maria Bottino (Istituto Tecnologie Didattiche - Consiglio nazionale Ricerche, Italy); Hadya Boufera (University Of Mascara, Algeria); Philip Bourke (LIT-Tipperary, Ireland);Dr Liz Boyle (University of the West of Scotland, UK); Dr Tharrenos Bratitsis (University of Western Macedonia, Greece);Prof Anthony Brooks (Aalborg University, Denmark);Prof David Brown (Nottingham Trent University, UK);Prof. Dr.-Ing. Carsten Busch (University of Applied Sciences HTW-Berlin, Germany,); Dr George Caridakis (University of the Aegean / NTUA, Greece,); Dr Thibault Carron (Université de Savoie, Chambéry, France); Rommert Casimir (Tilburg University, The Netherlands); António Castro ( Instituto Superior de Engenharia do Porto (ISEP), Portugal);Dr Erik Champion (Massey University, New Zealand);Prof Maiga Chang (Athabasca University, Canada); Dimitris Charalambis ( University of Athens, Greece);Dr Darryl Charles (University of Ulster, UK); Nathalie Charlier (Catholic University of Leuven, Belgium);Dr Yam San Chee (Nanyang Technological University, Singapore);Dr. Ming-Puu Chen (National Taiwan Normal University, Taiwan,); Satyadhyan Chickerur (M S Ramaiah Institute of Technology, India);Professor Thomas Connolly (University of West of Scotland, UK); Tamer Darwish (Brunel University, UK); Ioannis Darzentas (University of Aegean, Greece);Dr Sara De Freitas (Birkbeck College University of london, UK);Dr Sophia Delidaki (Hellenic American Educational Foundation, Greece,);Dr Ioannis Deliyannis (Ionian University, Greece,);Dr. Muhammet Demirbilek (Suleyman Demirel University, Turkey);Dr David Edgar (Glasgow Caledonian University, UK); Patrick Felicia (Waterford Institute of Technology, Ireland); Georgios Fesakis (University of the Aegean, Greece);Dr. Brynjar Foss (University of Stavanger, Norway);Dr Christos Gatzidis (Bournemouth University, UK); Dr Sebastien George (INSA Lyon, France); Panagiotis Georgiadis (University of Athens, Greece); Andreas Giannakoulopoulos (Ionian University, Greece);Dr Stefan Goebel (Technical University Darmstadt, Germany); Pedro Pablo Gomez-Martin (Universidad Complutense, Madrid, Spain); Cleo Gougoulis (Peloponnesian Folklore Foundation, Greece);Dr Dimitris Gouscos (University of Athens, Greece); Maria Grigoriadou ( University of Athens, Greece);Dr David Guralnick (Kaleidoscope Learning, New York, USA);Dr Thomas Hainey (University of the West of Scotland, UK); Paul Hollins (The University of Bolton, United Kingdom);Dr Birgitte Holm Sorensen (Aalborg University, Copenhagen, Denmark);Professor Rozhan Idrus (Universiti Sains Malaysia, Malaysia);Dr Jose Ignacio Imaz (University of the Basque Country, UPV-EHU, Spain); Jeffrey Jacobson (Carnegie Museum viii

of Natural History, Pittsburgh, Pennsylvania, USA); Ruben Jans (Limburg Catholic University College, Belgium); Runa Jesmin (Global Heart Forum, UK);Dr Larry Jones Esan (London Academy Business School, UK); Alexandros Kakouris (University of Athens, Greece); Fragiskos Kalavassis (University of the Aegean, Greece);Prof Konstantinos Kalemis (National Centre of Local Government and Administration, Greece); Dr Michail Kalogiannakis (University of Crete, Faculty of Education, Crete); Dr Anastasios Karakostas (Aristotle University of Thessaloniki, Greece); Dr Elisabeth Katzlinger-Felhofer (Johannes Kepler University, Linz, Austria);Dr Harri Ketamo (Satakunta University of Applied Sciences, Finland);Dr Kristian Kiili (Tampere University of Technology, Pori, Finland); Evangelia Kourti (University of Athens, Greece); Rolf Kretschmann (University of stuttgart, Germany);Dr Timo Lainema (University of Turku, Finland);Prof Miguel Leitao (ISEP, Portugal);Dr. Ximena Lopez Campino (Initium, Italy,); Carrie Lui (James Cook University, Australia);Dr Hamish Macleod (University of Edinburgh, UK);Dr. Rikke Magnussen (Danish school of education, Aarhus university/Steno Health Promotion Center, Denmark); Emanuela Marchetti (Aalborg University Esbjerg, Denmark,); Bertil Marques (Instituto Superior de Engenharia do Porto (ISEP), Portugal);Dr Jean-Charles Marty (LIRIS lab, Lyon, France); Stephanos Mavromoustakos (European University Cyprus, Cyprus); Florian Mehm (Technische Universität Darmstadt, Germany); Michail Meimaris (University of Athens, Greece); Bente Meyer (The Danish University of Education, Denmark); Prof Florence Michau (Grenoble Institute of Technology, France);Dr Christine Michel (INSA-Lyon, France);Dr Jonathan Moizer (University of Plymouth, UK); Assoc Prof Begona Montero-Feta Universitat Politecnica de Valencia Dr Adam Moore (Trinity College, Ireland); Alexander Moseley (University of Leicester, UK); Dulce Mota ( Instituto Superior de Engenharia do Porto (ISEP), Portugal); Constantinos Mourlas (University of Athens, Greece); Peter Mozelius (Stockholm University, Department of Computer and Systems Sciences, Sweden); Karen Neville (University College Cork, Ireland);Dr Piotr Nowakowski (John Paul II Catholic University of Lublin, Poland); Kimmo Oksanen (Finnish Institute for Educational Research, University of Jyväskylä, Finland);Dr John O'Mullane (University College Cork, Ireland);Dr. Michela Ott (Institute Educational Technology, Italy); Dimitra Panagouli (Hellenic American Educational Foundation, Greece); George Papakonstantinou (University of Thessaly, Greece); Agis Papantoniou (Multimedia Laboratory of the School of Electrical and Computer Engineering (ECE) of the National Technical University of Athens (NTUA). , Greece);Dr Marina Papastergiou (University of Thessaly, Greece); Paul Peachey (University of Glamorgan, Treforest, UK); Gilbert Peffer (CIMNE, Spain);Dr Neil Peirce (Trinity College Dublin, Ireland);Dr Eva Petersson Brooks (Aalborg University Esbjerg, Denmark); Elias Pimenidis (University of East London, UK);Professor Selwyn Piramuthu (University of Florida, Gainesville, USA);Prof. Dr. Maja Pivec (FH JOANNEUM University of Applied Sciences, Austria); Angeliki Poylymenakou (Athens University of Economics & Business, Greece); Dr Aristidis Protopsaltis (Institut für Lern-Innovation Friedrich-Alexander-Universität, Germany); Rosa Reis ( Instituto Superior de Engenharia do Porto (ISEP), Portugal);Prof Dr Bernd Remmele (WHL Graduate School of Business and Economics Lahr, Germany); Vyzantinos Repantis (Psychico College, Hellenic-American Educational Foundation, Greece,); Simos Retalis (University of Piraeus, Greece);Dr Pauline Rooney (Dublin Institute of Technology, Ireland);Dr Eleni Rossiou (University of Macedonia,Thessaloniki, Greece);Dr Maria Roussou (makebelieve design & consulting, Greece);Dr Florin Salajan (North Dakota State University , Canada); Jordi Sanchez Navarro (Universitat Oberta de Catalunya, Spain); Manthos Santorineos (School of Fine Arts in Athens, Greece);Dr Olga Shabalina (Volograd State Technical University, Russia); Samir Shah (Penn State University, USA);Dr Markus Siepermann (Technische Universität Dortmund, Germany); Helga Sigurdardottir (Nord Trøndelag University College and the Norwegian University of Science and Technology, Norway);Dr Gavin Sim (University of Central Lancashire, England);Dr. JulieAnn Sime (Lancaster University, UK);Dr Chrysanthi Skoumpourdi (University of the Aegean, Greece,);Prof Venustiano Soancatl (Universidad del Istmo, Mexico); Elsebeth Sorensen (University of Aarhus, Denmark);Dr Mark Stansfield (University of West of Scotland, UK); Martin Steinicke (University of Applied Sciences HTW Berlin, Germany);Dr. Arnab Sylvester (Coventry University, UK);Dr Sabin Tabirca (University College Cork, Ireland, ); Uday Trivedi (R.C. Technical Institute, India);Dr. Thrasyvoulos Tsiatsos (Aristotle University of Thessaloniki, Greece,);Dr Chuang Tsung-Yen (National University of Tainan, Taiwan); Richard Tunstall (University of Leeds, UK);Dr Andrea Valente (Aalborg University Esbjerg, Denmark);Dr Peter Van Rosmalen (CELSTEC / Open University of the Netherlands, The Netherlands);Dr Linda Van Ryneveld (Tshwane University of Technology, Pretoria, South Africa);Dr Carlos Vaz de Carvalho ( Instituto Superior de Engenharia do Porto (ISEP), Portugal);Dr Ayelet Weizman (Snunit center for the advancement of web-based learning, the Hebrew University, Israel); Viktor Wendel (Technical University Darmstadt, Germany); Nicola Whitton (Manchester Metropolitan University, UK); Dorothy Williams (Robert Gordon University , UK); Andrew Wilson (Birmingham City University, United Kingdom); Amanda Wilson (University of the West of Scotland, Scotland);Dr. Aljona Zorina (ESCP Europe, France)

Biographies Conference Director Professor Thomas M Connolly is the original instigator of this conference in 2007, Thomas Connolly is a Professor in the School of Computing at the University of the West of Scotland, having managed the Department of Computing and Information Systems for several years. Thomas worked for over 15 years in industry as a Manager and Technical Director in international software houses before entering academia. His specialisms are games-based learning, online learning and database systems. He has developed three fully online MSc programmes and developed and leads the undergraduate BSc Computer Games Technology programme. He is co-author of the highly successful academic textbooks Database Systems (now in its 4th edition) and Database Solutions (in its 2nd edition). He is a reviewer for several international journals and has been on the committee for various international conferences. He is a member of CPHC (Council of Professors and Heads of Computing) and member of the Higher Education Academy.

Conference Chair Dr Carlos Vaz de Carvalho has a PhD in Information Systems and Technology. He is a Professor at the Engineering School of the Porto Polytechnic (ISEP) and the Director of the R&D group GILT (Graphics, Interaction and Learning Technologies). He was e-Learning Director (2001-2005) of ISEP and Director of the Distance Learning Unit of the Porto Polytechnic (1997-2000). He has published over 100 references on the subject including several books.

Programme Chair

Dr Paula Escudeiro is a Professor at IPP-ISEP with vast experience in project supervision and evaluation, accumulated for the past 21 years. She has a PhD on Informatics/Information Systems on Education and prior experience on software industry related to the development of educational software. She is the director of the Multimedia Laboratory at ISEP and belongs to the Graphics, Interaction and Learning Technologies research center (GILT).

Keynote Speakers Dr Baltasar Fernández-Manjón is a full professor in the Facultad de Informatica at the Complutense University of Madrid (2001-2006) and the Vice Dean of Research and Foreign Relationships at the Computer Science School at UCM (2006-2010). He is IEEE Senior Member and in 2010-2011 he has been Visiting Associate Professor at Harvard University and Visiting Scientist at LCS-MGH. He received a Bachelor in Physics (major in Computer Science) and a PhD in Physics from the UCM. He is member of the IFIP Working Group 3.3 "Research on the Educational uses of Communication and Information Tecnlogies" and of the Spanish Technical Committee for E-learning Standarization (AENOR CTN71/SC36 "Tecnologías de la información para el aprendizaje"). His main research interests are e-learning technologies, educational uses of serious games, application of educational standards, and user-modelling. Dr Paulo J. Gomes is the Game Director and Producer at BIGMOON STUDIOS and he produced games such ‘WRC3’, ‘MotoGP13’, ‘Jagged Alliance: Back in Action’, ‘Trapped Dead Lockdown’ and many others. Has more than 20 years of experience in software development and project management. He’s credited in more them 15 videogames published worldwide on PS3, Xbox360, Wii, PC, Linux, Mac and Mobile. Paulo has a PhD in Computer Science, MBA and he is a Multimedia Professor at Portucalense University.

Mini Track Chairs Prof. Dr Wilfried Admiraal is a full professor of Educational Sciences and chair of the research program Teaching and Teacher Learning of Leiden University Graduate School of Teaching. His research interest combines Educational Sciences, Social Psychology and technology. He published journal articles on mobile game-based learning and game creation by students. Dr Daniel Aranda is a Senior Lecturer in the Department of Information and Communication Studies at the Open University of Catalonia. He is researching on how young people use digital technologies. He works in the research group SPIDER (Smarter People through Interactive Digital Entertainment Resources), Communication & New Media (at the Internet Interdisciplinary Institute / IN3) and eCo (research and innovation in e-learning, information and communication), at the eLearn Center (UOC).

Dr Stefan Göbel holds a PhD in computer science from TUD and has long-term experience in Graphic Information Systems, Interactive Digital Storytelling, Edutainment applications and Serious Games. After five years work as researcher at Fraunhofer Institute for Computer Graphics, from 2002 to 2008 he was heading the Digital Storytelling group at the Computer Graphics Center in Darmstadt. In late 2008 he moved to TUD and is heading the prospering Serious Gaming group at the Multimedia Communications Lab. Dr. Göbel is author of numerous papers and member of different program committees such as ACM Multimedia, ICME, Edutainment, Foundations on Digital Games, Serious Games Conference and serves as jury member of the Serious Games Award. Dr. Thomas Hainey is a Lecturer in Computer Games Technology and Serious Games Researcher at the University of the West of Scotland. He teachers an honours level course in serious games and is primarily interested in the empirical evaluation of games-based learning applications and how to integrate assessment into games-based learning applications. He has a number of publications in this area. Dr Kristian Kiili works as a Senior Research Fellow and an Adjunct Professor at the Tampere University of Technology in which he heads the Advanced MultimediaCenter research laboratory. His current research focuses on game-based learning, user generated game content, game design, and educational exertion games. He is presently involved in two European initiatives: the Game and Learning Alliance (GALA NoE) and Making Games in Collaboration for Learning (MAGICAL). Results received from his research has been published in several scientific publications as well as applied in commercial products Johannes Konert finished his diploma in Computer Science and accompanying studies in cultural studies at the Karlsruhe Institute of Technology (KIT) with a thesis proposing a web-based knowledge management system for the integration of workflow and learnflow. After three years working on the foundation and development of the online social network friendcafe as CEO and senior developer he joined the research group at Multimedia Communication Lab (KOM) at Technische Universität Darmstadt to focus on Serious Games and Social Networks. He became a Ph.D. student of the DFG Research Training Group “Feedbackbased Quality Management in E-learning”. In his research he focusses on the development of solutions to use Social Media concepts for knowledge transfer between peers in Serious Games. Prof Konstantinos Kalemis is an Instructor at the National Centre for Public Administration and Local Government (E.K.D.D.A.) in Adult Education and Lifelong Learning and assigned at the Dept of Primary Education in National and Kapodistrian University of Athens. He has authored a large number of scientific articles, studies and papers in Educational Congress and Seminars. His interests focus on the introduction of New Technologies as an alternative teaching process and the design of new curriculum plans for the open and d-Learning. His research interests also include the education of immigrant ethnic minorities focusing on the gifted and talented students and aim to advance the theory and technology of natural language and knowledge processing, especially semantic analysis that bridges the gap between language and knowledge, by the novel use of both machine learning and inference methods. Dr Jordi Sánchez-Navarro is a Senior Lecturer in the Department of Information and Communication Studies at the Open University of Catalonia (Universitat Oberta de Catalunya).His research revolves around innovation in entertainment and how this interacts with the new practices of cultural consumption in the contemporary media landscape, and how they impact on education. Dr Thrasyvoulos Tsiatsos is currently Assistant Professor in the Department of Informatics of Aristotle University of Thessaloniki. He obtained his Diploma, his Master's Degree and his PhD from the Computer Engineering and Informatics Department of Patras University (Greece). His research interests include Networked Virtual Learning Environments, Computer Uses in Education, Evaluation methods of Internet Learning Environments and Open and Distance Education using Multimedia and Internet Technologies. He has published more than 120 papers in Journals and in well-known refereed conferences and he is coauthor in 3 books. He has been a PC member and referee in various international journals and conferences and participated in more than 20 R&D projects. Also he is member Technical Chamber of Greece. Viktor Wendel received his degree in Computer Science from the Julius-Maximilians-University of Würzburg in 2009. Since November 2009, he is working as a research assistant at the MultimediaCommunications-Lab at the Technical University of Darmstadt. Research topics are Game Mastering in Multiplayer Serious Games, and Collaborative Learning. Further, he is an editor for ACM SIGMM Records.

Biographies of Presenting Authors Aishah Abdul Razak received her MSc. in Information Technology from Multimedia University, Malaysia. She is now pursuing her PhD in the School of Computing at the University of the West of Scotland. Her research interest is in games-based learning for primary school children. Wilfried Admiraal is full professor of Educational Sciences and chair of the research program Teaching and Teacher Learning of Leiden University Graduate School of Teaching. His research interest combines Educational Sciences, Social Psychology and technology. He published journal articles on mobile game-based learning and game creation by students Rafael Marques de Albuquerque graduated as Bachelor (2008) and Master (2011) in Graphic Design in the Federal University of Santa Catarina (Brazil) and is currently carrying out his PhD in Education in the Learning Sciences Research Institute of the University of Nottingham (UK). His research interests are digital games, learning and school, especially game literacy. Rene Alimisi has a rich engineering background and thorough knowledge of the field of Information Communication Technology in Education. She holds a master with distinction in ICT in Education (IOE, University of London). She has more than 4 years experience within the area of Lifelong Learning European Projects and teaching experience in well- known institutions in Greece and UK. Anissa All works as a junior researcher at IBBT-MICT (Ghent University) since July 2011. Since January 2013, Anissa is working on a PhD through a IWT grant (Flemish agency for Innovation by Science and Technology). In this PhD research, she will develop a standard procedure to measure effectiveness of serious games aimed at cognitive learning outcomes. Yasemin Allsop has been working as an ICT coordinator in primary Schools across London for almost 10 years. MPhil/PhD student at Goldsmiths, University of London. Focus is on children’s learning and cognitive development through digital game making activities. Interested in the role of teachers when teaching with digital games and game design. Reem Al-Washmi is a second-year PhD student at the University of Nottingham, in the School of Computer Science. Her PhD is entitled “Collaborative games-based learning to support problem solving in UK KS2 Mathematics”. Her long term interests include the study of problems that children face in mathematics and the use of technology to overcome these. António Andrade has a Degree in Communication Design from the Faculty of Arts of the University of Porto. Currently he is working on his Computer Science MSc at the School of Engineering of the Porto Polytechnic (ISEP), researching on Virtual Communities of Practice. Massimiliano Andreoletti Professor of Educational gaming and animation at the Catholic University of the Sacred Heart of Milan. Researcher and author of several essays and articles in the Media Education and Educational Technologies (internet, video games, mobile, tablet/pad, cloud). He’s still a videogamer and father of a child of five years. Alexandra Androussou is a Associate Professor in Teaching Methodology at National and Kapodistrian University of Athens, Greece. Her research and writing focus on teaching practices, teacher education and education of minority groups. She also deals with the development of educational materials for children in electronic and conventional form and she produces educational materials for teachers such as the website www.kleidiakaiantikleidia.net Anna Arici is a Learning Scientist with the Center for Games & Impact, at Arizona State University, where she designs and researches game-based learning environments for educational and social challenges, change, and sustainable impact. Additionally, as director of Quest2Teach, she creates game-infused learning curricula and gamification systems for pre-service teachers to prepare and support highly effective educators. Jannicke Baalsrud Hauge is research scientist at Bremer Institut für Produktion und Logistik(BIBA). She is teaching decision making in SC at the University of Bremen and Jacobs University. Main interest: Serious games, TEL, use of ICT in logistics. Responsible for many WPs in EU and national projects on ICT applications, logistics and TEL. Authored 100+ papers. Sasha Barab is a Professor in the Teachers College at Arizona State University where he co-founded and serves as the Executive Director of the Center for Games & Impact. Dr. Barab is an internationally recognized learning scientist who has researched, designed, and published extensively on the challenges and opportunities of using games for impact.

xii

Lizzy Bleumers graduated as a Master in Psychology and obtained a postgraduate degree in user-centered design. She now conducts user research at iMinds-SMIT (Vrije Universiteit Brussel) within the area of play, learning and participation. She was recently involved in a European project, which explored how digital games can be part of empowerment and inclusion initiatives. Dr. Rosa Maria Bottino Director of the Institute for Educational Technology of the Italian National Research Centre (ITDCNR). Author of more than 100 scientific publications both in national and international journals, books and conference proceedings. Dr. Bottino participated in both national and European projects and Networks of Exellence in Technology Enhanced Learning, including GALA NoE on serious games Cyril Brom is an assistant professor at Faculty of Mathematics and Physics of Charles University in Prague. His research interest is in serious games, modelling behaviour and episodic memory of virtual human-like characters, in level of detail AI, and in computational biology. Hope Caton lectures Game Creation Processes at Kingston University, London, where she also leads inKUbator, a multidisciplinary games development studio Hope founded in 2010 with Dr Darrel Greenhill. In addition to investigating the effects of introducing gamification in the classroom, Hope’s areas of research include developing serious games for health and education. Nathalie Charlier is an assistant professor at the Faculty of Pharmaceutical Sciences and co-ordinator of the Teacher Training in Health Sciences Education at the KU Leuven, Belgium. She obtained a BSc and MSc in Pharmaceutical Sciences and her PhD in Medical Sciences. Her current research interests are (i) game-based learning in health science education and (ii) the use of new technologies in education. Yaëlle Chaudy researcher at the University of the West of Scotland. She obtained an MSc in computing from INSA Lyon (Institut National des Sciences Appliquées) and a bachelor in French as a Second Language from Stendhal University in Grenoble. Interested in both computing and education, she is now studying the integration of assessment in GBL applications. John Denholm In final stages of PhD at Serious Games Institute, Coventry, researching into value of educational games. MSc from Imperial College, London and held several senior positions in major UK companies, mainly corporate planning and development of strategic business models. He has lectured on Business, Project Management and Finance courses at Birmingham City and Coventry Universities and supervises Master’s students at Warwick and Manchester Universities. Jill Denner, PhD, is a Senior Research Scientist at Education, Training, Research, a non-profit organization in California where she studies how students learn while creating computer games. She has written numerous peer-reviewed articles, and coedited: “Beyond Barbie and Mortal Kombat: New Perspectives on Gender and Gaming,” published by MIT Press in 2008. Kyriaki Dourda is a post-graduate student at the Early Childhood Education Department, at the University of Western Macedonia, Greece. She has graduated from the School of English Language and Literature at the Aristotle University of Thessaloniki. Her research interests include: Learning and Teaching Modern languages, GBL, CLIL, Language learning strategies. Ronald Dyer has held senior positions in the area of technology strategy, transformation and change management for performance improvement in the US & Trinidad & Tobago. He is a final year candidate for the Doctorate in Business Administration at Grenoble Ecole de Management, France, focused on research on serious game. Stine Ejsing-Duun is interested in the relation between technology, perception and cognition. Her ambition is to describe how technologies allow us to transcend ourselves. Her research has been connected to games, play and playful processes in various areas. Her present studies are within the fields of learning and art. Lars Elbæk (Ph.D.) is associate professor at University of Southern Denmark, and director of the research group ‘PE Pedagogy and Sports Psychology’. He has worked with interaction design and use of IT in children and youth sport and physical education through a number of years. Lars Elbæk is currently working partly at the Play and Learning – Kids n’ Tweens Lifestyle EU founded project. www.kidsntweens.dk Valérie Emin, PhD, is a researcher at the Institut Français de l'Éducation, member of S2HEP Laboratory. She coordinates since 2008 a research project on pedagogical scenario design in science and technique discipline. Her current research topics are "Pedagogical scenarios design" and "Game based learning". She's an associate member of GALA european network of excellence.

xiii

David Farrell is a lecturer in game design at Glasgow Caledonian University with interests in player centric design and Serious Games. He was the lead designer and developer of the e-Bug Serious Games and his research focuses on improving the integration of good game design practice with well grounded pedagogy. Toru Fujimoto is an assistant professor at the Center for Research and Development of Higher Education, The University of Tokyo. He completed his graduate study at the Pennsylvania State University (Ph.D. in Instructional Systems). His research focus is on the design of learning environments using digital games and social media. Georgy Gerkushenko conducts research in the area of social informatics and e-learning since 2000. He gained PhD degree on creating electronic educational resources in 2004 and MBA degree on “Chief Information Officer” in 2009. Currently he is a senior lecturer of CAD department at Technical University in Volgograd, Russia. Main research interests are personal learning environment, education social networking and electronic educational resources. Dr. Lisa Gjedde is Professor with special responsibilities at Aalborg University in Copenhagen at the Dept. for Learning and Philosophy, director of the Research Center for Creative and Immersive Learning Environments: reCreate. Dr. Stefan Göbel holds a PhD in computer science from TUD and has long-term experience in Graphic Information Systems, Interactive Digital Storytelling, Edutainment applications and Serious Games. Since 2008 he is heading the Serious Gaming group at the Multimedia Communications Lab of Technische Universität Darmstadt. Paridhi Gupta is a PhD research student in the department of School of Design at the Hong Kong Polytechnic University (Hong Kong). Her research investigation focuses on interactive play and games within English Language subject classrooms in Hong Kong local Primary schools. She has a Master’s Degree in visual communication from IDC, IIT Mumbai. Dr. Thomas Hainey is a Lecturer in Computer Games Technology and Serious Games Researcher at the University of the West of Scotland. He teachers an honours level course in serious games and is primarily interested in the empirical evaluation of games-based learning applications and how to integrate assessment into games-based learning applications. He has a number of publications in this area. Claire Hamshire has worked at Manchester Metropolitan University (MMU) since 2003; initially as a Senior Lecturer in Physiotherapy and from 2008 as a Senior Learning and Teaching Fellow. This role combines faculty teaching with a cross institutional contribution to technology and games-based innovation. She was awarded a Higher Education Academy National Teaching Fellowship in 2012 Thorkild Hanghøj is an Associate Professor at the ResearchLab: IT, Learning and Design (ILD), Aalborg University, Copenhagen. He holds a PhD on the playful knowledge in educational gaming. Research areas include: game-based teaching, games and Mother Tongue Education, and problem-based game design. Shinobu Hasegawa received his B.S., M.S., and Ph.D. degrees in systems science from Osaka University in 1998, 2000, and 2002. He is now an associate professor in Center for Graduate Education Initiative, Japan Advanced Institute of Science and Technology. His research areas include support for Web-based learning, game-based learning, language learning, and community based learning. Wayne Holmes is currently Head of Education for the games-based learning company zondle. Previously, he was a teacher and the Head of Research for an education charity. He has an MA in Philosophy, an MSc (Oxon) in Education, and is completing his DPhil (doctorate) in Education (researching games-based learning) at the University of Oxford. Robyn Hromek is a practicing educational psychologist working in Australian schools and an Honorary Associate of the The University of Sydney, Australia. She has created a set of 15 board games to teach social and emotional skills to children and young people and has spoken at numerous international conferences on games-based learning. Jennifer Jenson is Professor and Director for the Institute of Research on Learning Technologies at York University, Canada. She has published on games, game design, gender and game play and digital games in education Helle Skovbjerg Karoff (PhD) is Assistant Professor at Aalborg University/Copenhagen and a member of ILD: IT and Learning Design. Helle´s main research field is play and interaction with technology, especially questions of the dynamics of play, for example through danger, movement and sociality. Harri Ketamo, PhD founder/chief scientist, Eedu ltd. and Adjunct Professor, Tampere University of Technology. Specialized in Complex Adaptive Systems, Cognitive Psychology of Learning, Neural Computing and Educational Technology. Was Direcxiv

tor of Education, Satakunta University of Applied Sciences and CEO & founder GameMiner ltd, company focusing on Data Mining/game AI. Published international/peer-reviewed research articles; presentations on studies in international forums. Several awards and nominations related to R&D activities. Michael Kickmeier-Rust holds a PhD in cognitive psychology and he is an experienced project manager and software developer. His research and development activities focus primarily on technology-enhanced learning, in particular intelligent, adaptive educational systems and human-computer interaction. Since 2010 Michael is with the Knowledge Management Institute at Graz University of Technology. Kristian Kiili works as an adjunct professor at Tampere University of Technology. His research focuses on game based learning, exergaming, and game design issues. Results received from his studies has been published in several scientific publications as well as applied in commercial e-learning products. Luise Klein obtained a MSc. degree in Digital Media from the University of Applied Sciences Bremerhaven. Her interests are in enabling learning with and about media and technology, especially in informal playful settings. She also develops her game-based learning and mobile learning applications. Antti Koivisto is a Ph.D. student at the Tampere University of Technology in Pori, Finland. He currently works at Satakunta University of Applied Sciences as a researcher and at Eedu Ltd as a game developer. His research interests are exergames. He is currently researching how games suit for elderly and mentally disabled people. Johannes Konert finished his diploma in Computer Science at the Karlsruhe Institute of Technology (KIT). After three years work on the foundation and development of the online social network friendcafe, in June 2010 he joined the research group at Multimedia Communication Lab (KOM) at Technische Universität Darmstadt to focus on Serious Games and Social Networks. Evangelia Kourti is an associate Professor of Social Psychology specializing in communication at the National and Kapodistrian University of Athens, Greece. Her research interests cover the scientific fields of communication, media and children and the psychology of cyberspace. Loukas Koutsikos holds a Master Degree (MSc) in "ICT for Education", from the National Kapodistrian University of Athens. He holds a Bsc of Electrical Engineering Educator from the Higher School of Pedagogical and Technological Education. He works in Secondary Vocational Education and has participated in various programs dealing with the implementation of Educational Technology. Dimitrios Lappas has graduated from the Hellenic Military Academy in 2005. He also has a Bachelor's Degree from Pre-school Education and Educational Design Department of the University of the Aegean. He is currently a postgraduate student in “Models of Designing and Planning Educational Units”, Master's and PhD Degree program at the University of the Aegean. José Rafael López-Arcos Member of the GEDES research group in the Department of Computer Languages and Systems at the University of Granada. His research focuses on the integration of storytelling into educational video games. Rikke Magnussen associate professor at ResearchLab: ICT and Design for Learning, Aalborg University Copenhagen. Main research interest is how game-based technology can open for innovation in science education. Part of numerous national and international learning game development and research projects for over ten years and published extensively on subject of game's potentials in science education. Gunver Majgaard (PhD) is Associate Professor at The Maersk Mc-Kinney Moller Institute, University of Southern Denmark. Her research interests are design of digital educational tools; human computer interaction; participatory design processes; learning processes; didactical design; program and curriculum development. She has developed the engineering program "Learning and Experience Technology". Christos Malliarakis is a teacher of Computer Science in Mandoulides Schools, a large private primary/secondary school in Thessaloniki, Greece. He holds a BSc and an MSc in Informatics from the Computer Science Department of the Aristotle University of Thessaloniki and he is undergoing his PhD research in Game Based Learning on Computer Programming since May 2011. Dionissios M. Manessis holds a M.Sc. in ICT for education, from the University of Athens, Greece. He is now a Ph.D. student at the department of Early Childhood Education of the University of Athens. His research interests include the use of digital games in Early Childhood Education and students’ attitudes towards Statistics. xv

Jean-Charles Marty associate professor at LIRIS lab in Lyon (France). Research interests are in observation of collaborative activities, through traces of these activities. Research results are applied to Technology Enhanced Learning, and in particular to learning game environments. Participates to several projects in this field (Learning Adventure, Learning Games Factory, Serious Lab for Innovation, Pegase). Organized an international school on Game-Based Learning in June 2011. Apostolos Mavridis is a PhD candidate on the subject of “Game Based Learning” in the Department of Informatics, Aristotle University of Thessaloniki, Greece. He holds a BSc in Computer Science and an MSc in ICTE (Information and Communication Technology in Education). Mas Idayu Md Sabri is a PhD student at the University of Nottingham. She is currently on study leave from her employment as a lecturer at the University of Malaya, Malaysia. She obtained her BComp Science from University of Malaya, majoring in Software Engineering and obtained her MSc Multimedia Technology from the University of Bath. Her research interests are multimedia technology, edutainment, and interactive learning. Emna Mejbri She obtained the master degree in computer sciences, from the University of Kairouan, Tunisia in 2011. Currently, she is a phD student at the National School of Computer Sciences of Mannouba, Tunisia. Her main research interest is in the area of Learning and Games development. Javier Melero received both his Engineering degree in Computer Science (2008) and Master in Information, Communication and Audiovisual Media Technologies (2009) from the Universitat Pompeu Fabra (UPF), Spain. Since 2006, he has been involved in European and National projects in the field of TEL. His main research focus is about designing technology-supported puzzle-based games. Hélène Michel is a Senior Professor in Grenoble Ecole de Management (France). With a specialization in Innovation Management, she started working on Serious Games in 2003. Her research focuses on the strategic approach of serious games and on their performance’s evaluation. Alex Moseley is an Educational Designer and University Teaching Fellow at the University of Leicester, with long experience of course design and development in higher education. His research areas are in games-based learning, student engagement and effective research skills, and he designs games for education and museum contexts. Peter Mozelius has since 1999 been employed as a teacher for the Stockholm University and the Royal Institute of Technology at the Department of Computer and Systems Sciences (DSV) in Kista, Sweden. He is currently working as an IT-Pedagogue and researcher. His research interests are in the fields of ICT4D, game-based learning and software engineering. Robin Munkvold has been teaching software design at Nord-Trondelag University College (Norway) since 1999. The last five years he has been Program Director within the field of Digital Games and Media Technology. He has published several papers on subjects regarding online learning and ICT as a tool for supporting different pedagogical approaches. Rob Nadolski is assistant professor in TEL at CELSTEC at the Open University of the Netherlands. Main interests: competencebased education, serious games, especially enhancing learner support facilities by exploiting newest technologies. Involved in design and project management of e-learning applications for acquiring complex cognitive skills and research on such applications. Has participated in European and national projects. Elena Núñez Castellar received her degree in Psychology in 2006. In 2011 she obtained her degree of Doctor in Psychology from the Ghent University. During her PhD she got broad experience with research methods in cognitive neuroscience, namely EEG (electro-encephalography) and reaction times research. Since 2012 she joined the research group for Media & ICT (iMinds-MICT). Chinedu Obikwelu is a PhD research student with the ChiCI research group, University of Central Lancashire, he is currently researching the scaffolding mechanism in serious games with emphasis on adaptive scaffolding. He has worked in both the educational and IT sector as a teacher and an IT Support Officer respectively. Daire Ó Broin holds a Ph.D. in Computer Science from TCD, which focussed on approaches to developing the conditions of flow. He has been a lecturer at IT Carlow since 2008, where he teaches on the Computer Games Development programme . His research interests include increasing engagement and intrinsic motivation in games and learning. Kelly O’Hara (Ph.D. Sport Science) is professor at Beira Interior University, Portugal, and a researcher at Sports Sciences, Health Sciences and Human Development Centre. Her research interests are develop game based-learning environments by

xvi

integrating, sports and health, and tennis training. She has published several papers, book chapters, and she is reviewer in international journals. Kimmo Oksanen (Lic.Ed.) is doctoral student at the Finnish Institute for Educational Research (FIER), University of Jyväskylä. He is working on his doctoral thesis about supporting and evaluating collaborative learning in a game context. His research interests include game experience, game design and collaborative learning. Mark O'Rourke is an Educational Advisor with the Curriculum Innovation Unit at Victoria University, Melbourne, Australia. He has worked as a VET researcher, Multimedia Lecturer, Program Manager, Head of School, and Chair Academic Board. Mark's research activity focuses on games-based learning and he is a Fulbright Professional Scholar having undertaken research at USC in LA. M’hamme Ali Oulhaci is a PhD student at LSIS laboratory Aix-Marseille University; his works include behaviors simulation, multi-agents systems, and learners’ assessment in serious games. He got his master at Paris Dauphine Unversity. Contact him at LSIS, Domaine universitaire de Saint Jérôme Avenue Escadrille Normandie Niemen 13397 MARSEILLE Cedex 20. Trygve Pløhn works as a lecturerer and a researcher at the Nord-Trøndelag University College. He obtained his MSc in Software Development, Information Technology from the IT University of Copenhagen in 2007. He is a PhD Candidate at the Norwegian University of Science and Technology. His main research interest is within Pervasive Games and Serious Games. Jakub Procházka, Ph.D. is an assistant professor at the Department of Corporate Economy and at the Department of Psychology, Masaryk University, Brno, Czech Republic. His current research focuses on psychology of leadership and leadership effectiveness. He teaches interactive courses in the field of organizational and work psychology. Dr Aristidis Protopsaltis is a Senior Researcher at the Institut für Lern-Innovation at Friedrich-Alexander-Universität ErlangenNürnberg. His background is in Cognitive Science, Serious Games, ICT and Education. He is involved in a number of European projects with focus on education, e-learning and Serious Games. He has published numerous peer-reviewed conference and journal papers. Wen Qi researcher at CELSTEC Open Universiteit. PhD in Men Machine Interaction. Research interests are in Virtual Environments, Serious Games (for learning) and Human Computer Interaction. He has worked in different research projects in both academia and industry. Those research projects were sponsored by US, European and national funding agencies. He is now active in gaming based learning. Rosa Reis teaches at IPP-ISEP, Computer Engineering Department. MSc on Information Systems in Education and PhD student on Informatics at University UTAD-Tras-os-Montes and Alto Douro, Vila Real. Researcher at GILT-Graphic Interaction & Learning Technologies. Researches application of techniques of software engineering in design of educational collaborative virtual environments. Involved in several National and European research projects, presently regular reviewer of several conferences and scientific journals. Christian Reuter studied Computer Science at TU Darmstadt and finished his Master Thesis about the “Development and Realization of Methods and Concepts for Multiplayer Adventures” in 2011 before he then joined the Multimedia Communication Lab. His research focus includes the Authoring-Platform “StoryTec”, especially its extension for multiplayer serious games. Tiago Ribeiro is an eclectic researcher seeking harmony between arts and technology. He has been collaborating internationally on research projects like LIREC and EMOTE, and also with CMU, focusing especially on non-verbal expression in robots both through animation and sound. He is currently in an early stage of obtaining his PhD, in which he pursues artist-oriented intelligent robot animation. Gabriela Rodríguez has a B.A. degree in Spanish Literature and a M.A. in Education. She is currently a PhD student at Turku University’s Faculty of Education, where she is part of a research group designing a mathematical Serious Game as a tool to develop flexible and adaptive use of arithmetic strategies amongst upper elementary children. Martin Ruskov is currently a PhD student at the UCL Deptartment of Computer Science. He has previously worked in the areas of interactive storytelling and authoring tools for multimedia publishing. For his PhD Martin explores how to develop effective serious games and measure the learning happening with their use.

xvii

Eric Sanchez is associate professor at the Ecole Normale Supérieure de Lyon, head of EducTice, a research team of the French Institute for Education and adjunct professor at the University of Sherbrooke, QC (Canada). His research work concerns the uses of digital technologies for educational purposes (e-learning, simulation, serious games). Luciano Santos has a graduate degree in informatics engineering from the Engineering School of the Porto Polytechnic (ISEP) in 2011, and is currently undertaking a Master’s degree in graphics and multimedia systems at ISEP. Ángel Serrano-Laguna, MSc, works for the Complutense University of Madrid as a researcher in the e-UCM e-learning group as well as being a PhD student. His current research interests are educational video games, learning analytics and the eAdventure project. He has published 8 academic papers related to these topics. Helga Sigurdardottir is a PhD candidate at the Nord-Trøndelag University College. She is attending the PhD program in Interdisciplinary Culture Studies at The Norwegian University of Science and Technology, Trondheim, Norway. Helga has a bachelor degree in Social Anthropology, a Master in Education (Program Evaluation) and a Teacher Certificate from The University of Iceland. Petr Smutný is assistant professor at the Department of Corporate Economy of Faculty of Economics and Administration, Masaryk University, Brno, Czech Rep. His current research focuses on managerial simulation games effectiveness and leadership effectiveness. He teaches several courses using game based approach. Currently he is vice-dean for external affairs of the faculty. Heinrich Söbke researcher in “Intelligent Learning” programme (www.intelligentes-lernen.de) at Bauhaus-Universität Weimar. Focuses on game based learning, where his background in computer science enables him to transfer software design principles into technical design of video games. Ws visiting scholar in Department of Curriculum & Instruction at University of Wisconsin, Madison, when worked on development of educational games at Morgridge Institute for Research. Mario Soflano is a researcher at University of the West of Scotland. His background education is in computer science. His main interests are computer games technology, games design, educational technology, web development, adaptive system and mobile games / software development. Pooya Soltani has a M.Sc. degree in Exercise Physiology from Shiraz University, Iran. He is now a PhD candidate of Sport Science at University of Porto. He’s interested in characterizing Exergames and their effects in three domains of physiology, biomechanics and psychology. He’s working at Porto Biomechanics Laboratory under supervision of Professor João Paulo Vilas-Boas. Narumon Sriratanaviriyakul (Cherry) is Senior Lecturer in Centre of Commerce and Management at RMIT University Vietnam and has 8 years of teaching experience in higher education in international universities. Her research interests include gamebased learning, online social network, case study methods, and technology in education. Martin Steinicke earned his BSc. and MSc. in Business Informatics at University of Applied Science HTW-Berlin. He works in the research project „Innovationsdramaturgie nach dem Heldenprinzip“ headed by Professor Carsten Busch and teaches DGBL. His work centers on game based learning in the business context and information & knowledge diffusion in social networks. Gunilla Svingby is a Professor of Education, at Malmö University, Sweden. I was professor at Lund University, Gothenburg, Oslo University, and Tromso University. Some research projects: A computer game on ethics as a learning environment, Continuous assessment and dynamic examination, Professional competence with simulations in teacher education, Learning by mobile games, Mathematics for the digital generation. Emilia Todorova has recently graduated in BSc Information Systems Development from Glasgow Caledonian University. She has worked on projects involving the Bologna Process, Quality Assurance and Education Policy. Her research interests are on improving learning and teaching, using technology within higher education and quality assurance in the EHEA. Ofut, Ogar Tumenayu: obtained his bachelor degree in Computer science from Cross River University of Technology, Calabar, Nigeria. He is currently studying for his master degree in Volgograd State Technical University Volgograd, Russian Federation. His scientific research is in field of Design and implementation of adaptive Educational Games. Herre van Oostendorp is Associate Professor Human-Media Interaction at the Institute of Information and Computing Sciences, Utrecht University. His research activities are on the domain of Human-Computer Interaction. He is a specialist on the areas of web navigation, hypertext comprehension, usability engineering and cognitive principles in serious game design xviii

Didin Wahyudin is a lecturer in Indonesia University of Education, Bandung. He received Master degree in Game Technology from Bandung Institute of Technology, Indonesia. He has experienced as a first responder in many disasters. Currently, He is a PhD student at School of Information Science JAIST Japan focusing on research of Mobile Game Based Learning. Ayelet Weizman Director of science education at Snunit center for the advancement of web-based learning, located at the Hebrew University of Jerusalem. Designing educational games and interactive learning and teaching materials in science on several websites. PhD in Planetary Sciences from Tel Aviv University and post doctorate studies in science education at Michigan State University. Viktor Wendel received his degree in Computer Science from the Julius-Maximilians-University of Würzburg in 2009. Since November 2009, he is working as a research assistant at the Multimedia-Communications-Lab at the Technical University of Darmstadt. Research topics are Game Mastering in Multiplayer Serious Games, and Collaborative Learning. Further, he is an editor for ACM SIGMM Records. Dr Nicola Whitton is a Research Fellow at Manchester Metropolitan University, specializing in the innovative use of learning technologies in Higher Education. Her particular interest is in the design and use of computer games with adult learners and she is the authors of Digital Games for Learning, a practical guide to educational game development. Amanda Wilson is a research student at the University of the West of Scotland. Her research focuses on how games based construction learning can be implemented into the curriculum within primary education in Scotland using Scratch. Andrew Sean Wilson worked in biomedical research for last twenty years. Interested in use of technology in medical research particularly in the management of musculoskeletal diseases. Designed and developed educational computer programs to help patients and practising doctors gain better understanding of how to manage these diseases. Sees game based learning as another way of assisting in this. Amel Yessad is PhD in computer science. Currently, she is an associate professor in the team MOCAH–LIP6 of the University Pierre et Marie Curie. Her research focuses on knowledge engineering, technology enhanced learning, and serious games. Dr Yessad is involved in several serious game projects. Ebru Yeniman Yildirim is a senior lecturer and Head of Computer Technology & Programming at Uludag University, Bursa, Turkey. She has written books on computer programming and managed many large-scale EU projects on Vocational Training. She is interested in e-learning, how new technologies impact on the teacher’s role in the classroom and game based learning. Matej Zapušek is employed as teaching assistant for computer science courses at University of Ljubljana, Faculty of Education. He is also a PhD student at University of Ljubljana, Faculty of computer and information science. His main field of interest is researching the possibilities for teaching introductory programming with intelligent tutoring systems. Symeon Zourelidis is a postgraduate student of the M.Sc. program "ICT in Education", in the National Kapodistrian University of Athens. He works as a director in primary school. He participates in teacher training course for the use and the application of ICT in the classroom. Research interest focuses on new technologies in education.

xix

Sit Down to Table and Confess who you are! Design of an Epistemic Game for Nutritional Education at Secondary School Réjane Monod‐Ansaldi1, Eric Sanchez1, Daniel Devallois1,2, Thomas Abad1,2, Pierre Bénech1, Anne Brondex1,2, Isabelle Mazzella1,2, Sandrine Miranda1,2, Claudie Richet1,2 and Céline Recurt1 1 French Institute of Education, Lyon, France 2 Lycée Madame de Staël, Saint Julien en Genevois, France rejane.monod‐ansaldi@ens‐lyon.fr eric.sanchez@ens‐lyon.fr Daniel.Devallois@ac‐grenoble.fr Abstract: Nutrition is a complex behaviour. The choices made depend on multiple dimensions as taste, physiological consequences, cultural aspects, or environmental and economic impacts. Rather than transmitting standards to follow, nutrition education might help people to take into account the complexity of the problem, to reflect on all the dimensions involved in nutrition and to make informed choices. The French‐Canadian research project Jouer pour apprendre en ligne aims at designing and studying digital epistemic games (Shaffer, 2007) that address such complex problems. Epistemic games are playful, complex, realistic, authentic and non‐deterministic learning situations that deal with ill‐structured problems (King & Kitchener, 1994). Rather than having a unique solution they have solutions that depend on the available information, knowledge and values. Within this context, we used the 3E model (Sanchez & al., 2012)‐ a tridimensional model for game design ‐ to design “Mets‐toi à table !” (“Sit down to table and confess who you are!”), a digital epistemic game about nutritional education. Our research methodology consists in a Design‐Based Research approach which allows identifying the key factors that have to be taken into account to enroll, entertain and educate the player/learner. Therefore, our work is based on a collaborative research which enables teachers and researchers to iteratively combine design and analysis within an ecological context. The results of this research are both pragmatic (the design of the game) and heuristic (understanding the impact of the game design on the learning process). The data collected encompasses the recording of the students during the game session and focus groups carried out with teachers and students. The analysis of the impact of the choices made for the game design permits to reconsider the design of the game. Indeed, the findings of the first iteration, which bring little play and little learning to the students, lead to modify the gameplay. In this paper we present the game, the methodology of the research and we discuss the key factors that should be taken into account to design a digital epistemic game. Keywords: digital epistemic game, ill‐structured problem, nutrition education, design‐based research

1. Introduction The nutritional choices made by each of us depend on multiple dimensions as taste, physiological consequences, cultural aspects, psychological dimensions or environmental and economic impacts (Vandenbroeck, Goossens, & Clemens, 2007). Rather than providing standards to follow, nutrition education might help people to take into account the complexity of the problem, to reflect on all the dimensions involved in nutrition and to make informed choices (Brixi, Gagnayre & Lamour, 2008). In France, the secondary school curriculum offers 15‐16‐year‐old students, as an optional course, the opportunity to carry out interdisciplinary approaches and to address complex scientific problems raised by our modern society (MPS, BO n°4, 29/04/2010). The course involves mathematic, physics and chemistry and biology and geology teachers. Science and food, is one of the themes of this course. We chose the context of this course to tackle the question of nutrition education, taking the opportunity to address the complexity of nutrition with an interdisciplinary approach. In this respect, complexity relates to what is woven together and there is complexity whenever the various elements that compose a whole are inseparable and interdependent (Morin, 2000). Regarding nutrition education, there is a stake to help students understand that their decisions are grounded on a complex set of values, cultural preferences and societal influences. They also need to understand that their choices concerning food can have a physiological impact (health) but also economic or environmental consequences. Therefore, our study aims at exploring the possibility to design a meaningful learning situation that offers students the opportunity to deal with complex ill‐structured problems (King & Kitchener, 1994). Our approach consists in designing a digital epistemic game to address such issue.

401

Réjane Monod‐Ansaldi et al. In this paper, we describe an empirical study which addresses the question of nutrition education though a Game‐Based Learning approach. This approach consists in designing a digital epistemic game. “Sit down to table and confess who you are!” offers the learner/player the opportunity to face the complexity of nutrition and to reflect about his/her choices and behavior. In the following we describe an empirical work dedicated to identifying the key factors that have to be taken into account to design such an epistemic game. We will present a three‐dimensional model as a background for the design of the game and the methodology of the study based on a collaborative work with teachers. Then, we will present the lessons learned after having tested two versions of the game with secondary students regarding our three‐dimensional model.

2. Digital epistemic games for learning complexity The French‐Canadian project « Jouer pour apprendre en ligne » (Gaming for online learning) (Sanchez et al., 2012) funded by the Canadian Social Sciences and Humanities Research Council aims at addressing two questions. (1) The relevance of digital epistemic games for designing playful learning situations adapted to the digital culture of a new generation of learners. (2) The impact of such learning situations on the personal epistemology of the learners regarding the nature of knowledge and the process of knowing. There is a growing body of research that focus on Digital Play‐Based Learning and there are arguments advocating the consideration of the contexts (Steinkuehler & Duncan, 2008) or the situations (Sanchez & Jouneau‐Sion, 2010) rather than the game as a digital artefact. From this stems the possibility of focusing on the interactions that emerge from the situation, rather than on the game itself (Mitgutsch, 2007). Therefore, following Shaffer (2006), we propose to avoid the widely used term serious game that focuses on the artefact used to play and we prefer the expression digital epistemic game to name playful situations designed with digital technologies that intend to foster epistemic interactions. Epistemic games are complex, realistic, authentic and non‐deterministic learning situations that address ill‐structured problems (King & Kitchener, 1994). Rather than having a unique solution they have solutions that depend on the available information, knowledge and values. Within this context, we designed “Mets‐toi à table !” (“Sit down to table and confess who you are!”), a digital epistemic game about nutritional education. The design of the game is based on a three‐dimensional model (3E) (Sanchez et al., 2012) which consists in a set of recommendations that permit to Enrol, Entertain and Educate the learner/player. The dimension Enrol encompasses the elements that are crucial to foster the students’ motivation to participate and to accept to take up the challenge. Enrolling the learner/player means fostering his/her motivation. Motivation is the point generally putted forward to underline the relevance of a Game‐Based learning approach. According to the self‐determination theory (Ryan & Deci, 2000), motivation results from different universal and innate needs, that include the need for competence, autonomy, and relatedness. The feeling of competence increases when students have to reach clear goals. Therefore, the devolution (Brousseau, 1986) of the problem becomes possible: the students accept to be responsible for solving the problem and take decisions that are motivated by their onsciousness of the situation rather than the teacher’s expectations (Ahuja, Mitra, Kumar, & Singh, 1995). The feeling of competence also depends on the difficulty that the learner player has to face. A too low or too high level of difficulty decreases the motivation. Enrolling the students also depends on the autonomy offered by the situation. Autonomy entails the freedom to make choices and to choose a strategy. Autonomy also depends on the resulting feedbacks of the decisions taken that enable the learner/player to assess the relevance of his/her strategy. Therefore feedbacks are crucial. At any time, they allow the learner/player to evaluate his/her decision and, therefore, to make changes or not. The need for relatedness entails competition as "game worlds are meritocracies" (Reeves, 2011) but also collaboration and the success may result from the ability to collaborate with other learners/players. The dimension Entertain relates the play factors of the game to make possible that the learner/gamer is involved in a playful experience. Entertaining the player is the first objective of a game and pleasure mainly results from the feeling of freedom as a core characteristic of a game (Brougère, 2005). Freedom means that the player/learner is allowed to take decisions and to shape his own strategy. This freedom should offer to the learner/player a high level of control and, despite chance is also a part of the game, what happens should be directly linked to his/her decisions. As a counterpart to freedom, there are arbitrary rules (ie. a set of norms and conventions shared by the players that limit freedom). Rules need to be acceptable, relevant (regarding

402

Réjane Monod‐Ansaldi et al. the model embedded in the game) and clear. Entertainment also results from earning points and getting rewards. They are important in order to maintain a player’s interest in a game’s challenges (Habgood & Overmars, 2006) by increasing the feeling of competence. Frivolity is another core dimension of games (Brougère, 2005). As actions or decisions do not impact the real world, play occurs in a safe space where the learner/player feels secure. The learner/player can make mistakes without real consequences and he is encouraged to continue trying (Gee, 2003). Games are also often based on the use of avatars that are projective identity. The avatar allows one to project one’s values and desires and endorses mistakes and failure. Entertainment also depends on humour and on the artistic value of the game. Educate, the third dimension of the model, encompasses more specifically the epistemic factors. As a meta‐ activity, a game is a model of a situation of reference and simulating a part of the world allows the player to explore a physical or human situation of this world (Egenfeldt‐Nielsen, 2006). Therefore, the relevance of game content is an important issue to deal with. Though that a game can be a metaphor, a "good” epistemic game is anchored in the real world. It should be authentic regarding the interactions enabled by the gameplay. The model of knowledge embedded into a game results from the transposition of a real situation (a situation of reference) into the context of the game. Therefore, this model ought to be relevant regarding the situation of reference (Kirriemuir & McFarlane, 2004). In addition, it is important that the game is adapted to the learning objectives and to the curriculum. An important issue that must be addressed is the link between the content and the gameplay (Habgood, 2007). Games are said to be intrinsic if core game mechanisms (gameplay) and learning content are integrated (i.e. the required knowledge to succeed in the game is the learning purpose). In an extrinsic game, content and gameplay are separated (i.e. the game entails alternate phases of play and school‐like exercises). Regarding learning, intrinsic games are known to be more efficient than extrinsic games (Ibid.). The role of the teacher during the game is recognized to be crucial as well. His/her role cannot be the role of a teacher anymore. Research in Digital‐Play‐Based Learning demonstrates that the knowledge developed within the game is mainly implicit and students do not develop declarative knowledge without reflection and debriefing (Garris, Ahlers, & Driskell, 2002; Habgood, 2007; Sanchez, 2011). Therefore the teacher is needed to help students become aware of the implicit knowledge that they gained. This step is called debriefing or "after action review" (Aldrich, 2009). It aims at bridging the game world and the real world in order to facilitate the transfer of the knowledge developed during the game experience. The issue of the assessment has been recognized to be paradoxical. On one hand, a game entails an embedded assessment used by the player/learner for recognizing achievements and failure. However, by assessing the learner/player with a typical educational point of view, there is a risk to kill the frivolity and the feeling of freedom and, therefore, to kill the game itself.

3. Methodology of the study: A design‐based and collaborative research Our research methodology consists in a Design‐Based Research approach (Design‐Based Research Collective, 2003), that aims to produce theories on learning and teaching, by interventionist iterative methods using design and experimentation in naturalistic contexts (Cobb et al, 2003). Design‐based research involves flexible design revision, multiple dependent variables, and capturing social interactions. Different participants take part in the design to bring their differing expertise into both producing and analyzing the design process and results. Our work is then based on a collaborative research which allows teachers and researchers to iteratively combine design and analysis within an ecological context. The research team includes six secondary teachers (2 mathematic teachers, 2 physics and chemistry teachers and 2 biology and geology teachers), two researchers and a pedagogical engineer. The collaborative work enabled to design of the game “Sit down to table and confess who you are!” and to discuss the key factors that have to be taken into account to enroll, entertain and educate the player/learner. The game design is based on the analysis of the knowledge and social practices in the field of nutrition and with respect to the gameplay from previous games. We drew a conceptual map of nutrition (Figure 1) to design the knowledge model of the game (ie. the concepts that will be included in the game).

403

Réjane Monod‐Ansaldi et al.

Figure 1: Simplified version of the conceptual map The skills related to this theme were also identified. Several focus groups were dedicated to discuss the knowledge model and the gameplay. This iterative and collaborative work enabled to explore different ideas while taking into account the numerous constraints of the challenge. A first version (first iteration design) appeared to be irrelevant regarding our objectives. The difficulties that we faced relate to the complexity of the theme that encompasses many different dimensions and the constraint to enable distant and asynchronous interactions between French and Canadian students on a digital platform (which is also used to record the digital traces of the learners’ activity). This failure led us to skip several constraints such as the use of the digital platform and the level of knowledge. Thus, the second version of the game being designed and tested is based on the lessons that we learnt from the first experimentation. Our methodology enabled to follow our pragmatic objectives (the game design) with more heuristic expectations (to understand the key factors involved that have to be taken into account to foster epistemic interactions) by organizing focus groups with teachers dedicated to discuss the design of the game or the observations that we made during the experimentations or information gained during focus groups with students. The two versions of the game are briefly described below. The two different versions of the game were experimented in three (2012) or four (2013) 75 minutes sessions, with four groups of students about age 15‐16. The data collected encompasses the recording and videotaping of the students during the game session, some students’ productions and focus groups carried out with teachers and students. Choices, actions and expressions of the learners/players were analysed referring to the three dimensions Enrol, Entertain and Educate of the game.

4. A first version of the game: A role playing game The first version of the game designed with the teachers was a role playing game. Each student (or pair of students) successively plays the role of the owner of a restaurant responsible for designing a menu, and the role of a customer choosing a restaurant. The owner of the restaurant whose menu is chosen by the largest number of clients wins the game. The game is driven by a teacher involved as the game master. The learners/players start by picking out "Restaurants cards" that indicate the type of restaurant and, therefore, specifies different constraints regarding the menus. Each team has 20 minutes to create the menu and to advertise it with a short text. Online resources on food and restaurants are available to help the owners of restaurants write their menus. Then, each player/learner shifts to the role of customer and votes individually for one menu except for her/his own. The master of the game scores and displays the results. He gives points to the three restaurants receiving the greatest number of votes. Then, the learner/player comes back to the role of the owners of a restaurant. A third step consists in a discussion: each team has to convince the customers regarding the quality of their menu, and, therefore, to get more votes. A new vote takes place and new scores are added to the previous ones. Then each team picks out a “Challenge card” among bonus, blank, or penalty and a new step, call challenge step, begins. Each team challenges another one by asking them to demonstrate that their menu matches the constraints. If the restaurant owner picked out a bonus card, he automatically wins the challenge. If he picked out a penalty card, he loses. A blank card leads to develop arguments in order to convince the other learners/players that the menu matches the constraints. The game master arbitrates the challenge and the new scores are added to the previous ones. The game ends with the announcement of the overall scores and the winner is declared to be the best restaurant.

404

Réjane Monod‐Ansaldi et al. We used the data collected during the game sessions, and some students and teachers focus groups conducted after the game sessions to analyse and assess this first version. Regarding the power of the game to enroll the students, the teachers noticed that the competition created a great motivation for students, but also generated problems. Some students are not comfortable in competing with their friends (probably due to the school context). Others developed avoidance strategies, completely unrelated to the learning objectives, as secret agreements to help each other or to make another team lose. These ways to cheat were also reported by students during the focus groups. For students, the game was somewhat funny, interesting and original, but they also identified different problems that occurred. They also deplored the academic dimension of the situation, considered by some of them to be more a school assignment than a game. Students asked for a game context different from the school context, for game material as game boards or hourglasses, for minimal intervention of teachers and for the possibility to choose their team. Types of restaurants and constraints were considered to be unbalanced: creating menus and finding arguments were more or less difficult depending on the restaurant. As a result, some students expressed a feeling of injustice. Student also pointed out the lack of action and the lack of diversity regarding the way of thinking during the game. In addition, the teachers also underlined the lack of feedback enabling the students to determine if the menu that they designed was relevant regarding the rules of the game and their autonomy was not totally insured. Therefore, the games failed to fulfil the requirements to foster entertainment. As a result, we all agreed to consider that this first version of the game had too little learning potential. The first result of our project was a game which was not adequate to enroll the learners/player, which failed to entertain them and with few educational potential. However, we learnt a lot and we gained new ideas for re‐ designing the game. Therefore, this first iteration was the source of many choices in the design of the second game version.

5. A second version of the game: Sit down to Table and Confess who you are! One of the main difficulties identified by the research team was to make the game intrinsic (Habgood, 2007) (ie: to foster interactions that mobilize all the targeted knowledge). Therefore, we decided to put our effort on the design of the gameplay. As a result, the gameplay was fully revised for the second version of the game. Each player is now represented by a character. The goal of the game consists in unmasking the other player and in finding his character. Each character is defined by his age, gender, weight, height, activity, personal choices and dietary restrictions. Using learning resources of the game, each player transcribed these features in puzzles, which must be unmasked by his partner. The last player unmasked win. The game material includes now "Characters cards", wildcards, and a game board with pawns, dice (Figure 2). Each "character card” specifies 13 features in four categories (physic, activity, choices and dietary restrictions), which are represented as boxes on the game board. Two other types of boxes are also present on the game board, leading to miss a turn or choose a category. Only a part of the game material is now available under a digital version and the computerization of the game is an ongoing process.

2a: Character card

2b: Game board

Figure 2: The game material

405

2c: Lexicon of pictograms

Réjane Monod‐Ansaldi et al. The game begins with the random drawing of a character card by each team. During the first thirty minutes, the teams mask their characters, by reformulating as far as possible its features in puzzles (Figure 3). Different resources are available for such a task (for example, the way to calculate the body mass index (BMI), information concerning proteins needs for different sports, or local fruits and vegetables production seasons…). Each team owns a wildcard that offers the possibility to ask once for teacher assistance. The two teams compete by throwing the dice, moving their pawns on the game board and asking the opponent to give a piece of the puzzle corresponding to a feature (according to the category referred to by the pawn on the game board). For example, if the pawn lands on a "physical box", a piece of the puzzle hiding weight, height, gender or age of the character must be provided. Information exchange is carried out through five game rounds, then a five minutes break allows each team to summarize and interpret the collected information using the game resources. If one team of students unmasks the opponent character, they win. If not, five new game rounds are played. A video summarizing the game steps is online (http://eductice.ens‐ lyon.fr/EducTice/recherche/jeux‐et‐apprentissage/mets‐toi‐a‐table/mets‐toi‐a‐table ).

Figure 3: Students masking their character’s features Taking into account the amount of resources (46 files of 1 to several pages), the use of computer is essential for simultaneous access to all resources for all teams and allows students to move easily and rapidly from one resource to another (hyperlinks). The results of this second experimentation permitted to state that this new version enables a better engagement of students. This engagement results from a greater level of autonomy offered to the students. They shaped their own strategies and they were able to describe these strategies after the game session. However, the problem which consists in providing the students with relevant feedbacks is not totally solved by this version of the game. We observed different feedbacks, provided by opponents, which were the opportunity for discussing concepts related to nutrition. Yet, the feedbacks are still, for a large part, under the responsibility of the teachers who decides what is relevant or not. The enrolment of students seems also to result from a good combination of collaboration (within pairs of players) and competition (with opponents). In addition, the level of difficulty can be adjusted depending on the resources that are used for the coding phase. This level of difficulty also depends on the system bend, namely the ability for a character to perform one or two deviations from his/her dietary habits. Overall, there was an atmosphere of play during the game sessions and this atmosphere results probably from the freedom offered to the students. However, the rules of the game need revisions. Some of them are simple and they have been respected but others are too complex and it is difficult to state if they have been upheld. Therefore, the autonomy of the students is not yet totally ensured.

406

Réjane Monod‐Ansaldi et al. The issue of the frivolity of the game appears now to have been adequately addressed. The character to be masked/unmasked is an avatar which does not reveal the dietary profile of the students and it is possible to play without consequences. Therefore the frivolity of the game is guaranteed. For this second version of the game, special attention has been paid to the graphical dimension of the game material and the students expressed their interest and also proposed new improvement. In a school context, this aspect has been recognized to be important as it enables to shape the game universe. According to the teachers, the educational properties of the game have improved with this second version. Indeed, during the game, some students made links between the different dimensions of nutrition. However, further investigations need to be carried out to appreciate to what extend the students learnt. Preliminary results show that the model of knowledge embedded into the game is still too simple. It also includes knowledge that is not directly linked to nutrition education. In addition, according to our observations, the students mainly considered independently the different dimensions of nutrition, which is totally opposite to our expectations. Furthermore, we observed that certain students managed to reveal the hidden characteristics of their opponents with a strategy based on a clever use of the dice and the game‐board but not on a relevant use of the knowledge related to nutrition. This aspect needs further improvements and the problem of educating the learner/player has not been totally solved yet. The role of the teacher varied during the game session. The beginning of the game was devoted to help the students use the resources. Later on, they have been more involved in the game itself by being responsible for providing scientific assistance according to the rule of the wildcard. The debriefing phase has not been analyzed yet. However, there is a common agreement for saying that this phase is crucial to help the students grasp the complexity of nutrition. Despite several weaknesses mentioned during the focus groups, this second version is now considered to be far more efficient to enrol, entertain and educate the students.

6. Conclusion The results of this empirical work confirm the complexity to design a digital epistemic game. There are many factors to take into account and, furthermore, they are not isolated elements but elements that are linked by complex relationships. Some elements are crucial. The autonomy of the learner/player is one of the most important issues to consider. Autonomy results from the possibility offered to the learner/player to assess his/her strategies. As a result, the design of a game should pay specific attention to the feedbacks provided during the play. Another difficult challenge that we faced is to combine a clear and easy to learn gameplay with a rich and authentic model of knowledge embedded into the game. A too complex gameplay leads to fail in enrolling the learner/player while a too simple model of knowledge leads to fails in educating him. Regarding this dimension, it is important to underline that designing interactions, both playful and epistemic, has been identified to be the most difficult challenge to tackle. As a result, making the game intrinsic should be one of the core challenges to tackle for the game designers. The visible improvement of the second version of the game that we experimented tends to demonstrate the relevance of a collaborative methodology. The involvement of teachers at the early stage of the design process offers some guarantees that the game will be adapted to the context. In addition, the capacity of researchers to record data offers the opportunity to get an objective idea of the impact of the game. Regarding this aspect, the ideas expressed by the students themselves appear to be particularly important to take into account and we now plan to involve students in the design process.

Acknowledgements References Ahuja, R, Mitra, S, Kumar, R, & Singh, M. (1995). Education Through Digital Entertainment ‐ A Structured Approach. Paper presented at the XXX Ann. Conv. Of CSI, New Delhi. Aldrich, C. (2009). Learning online with Games, simulations, and virtual worlds. Strategies for online learning. San Francisco, Ca: Jossey Bass. Brixi, O., Gagnayre, R. & Lamour, P (2008). Eduquer pour la santé autrement. Propositions en appui aux pratiques alternatives à l'œuvre. Editions Le Manuscrit, Paris. Brougère, G. (2005). Jouer/Apprendre. Paris: Economica.

407

Réjane Monod‐Ansaldi et al. Brousseau, G. (1986). Fondements et méthodes de la didactique des mathématiques. In J. Brun (Ed.), Recherches en didactique des mathématiques (Vol. 7). Grenoble: La Pensée Sauvage. Design‐Based Research Collective. (2003). Design‐based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5‐8. Garris, R, Ahlers, R, & Driskell, J. E. (2002). Games, motivation, and learning: A research and practice model. Simulation & Gaming, 33(4), 441‐467. Gee, JP. (2003). What Video Games Have to Teach us About Learning and Literacy? New York: Palgrave Macmillan. Habgood, J. (2007). The effective integration of digital games and learning content. PhD Thesis, University of Nottingham. Habgood, J, & Overmars, M. (2006). The Game Maker's Apprentice: Game Development for Beginners. Berkeley, Ca: APress. King, P M, & Kitchener, K S. (1994). Developping Reflective Judgement: understanding and Promoting Intellectual growth and Critical Thinking in Adolescents and Adults. San Francisco, CA: Jossey‐Bass Publishers. Kirriemuir, J, & McFarlane, C A. (2004). Literature Review in Games and Learning. FUTURELAB SERIES. Bristol. Mitgutsch, K. (2007). Digital play‐based learning; A philosophical‐pedagogical perspective on learning anew based on games. Paper presented at the Games in Action, Gothenburg, Sweden. Morin, E. (2000). Seven complex lessons in education for the future. Paris: Seuil. Reeves, T. (2011). Can Educational Research Be Both Rigorous and Relevant? Educational Designer(4). Ryan, R M, & Deci, E L. (2000). Self‐determination theory and the facilitation of intrinsic motivation, social development, and well‐being. American Psychologist 55, 68‐78. Sanchez, E. (2011). Usage d’un jeu sérieux dans l’enseignement secondaire : modélisation comportementale et épistémique de l'apprenant. Jeux sérieux, révolution pédagogique ou effet de mode ? Revue d’Intelligence Artificielle, 25(2), 203‐222. Sanchez, E, & Jouneau‐Sion, C. (2010). Les jeux, des espaces de réflexivité permettant la mise en œuvre de démarches d’investigation. Paper presented at the Ressources et travail collectif dans la mise en place des démarches d'investigation dans l'enseignement des sciences, Lyon, novembre. Sanchez, E, Jouneau‐Sion, C, L, Delorme, Young, S, Lison, C, & Kramar, N. (2012). Fostering Epistemic Interactions with a Digital Game. A Case Study about Sustainable Development for Secondary Education. Paper presented at the IOSTE XV International Symposium, Hammamet, Tunisia. Shaffer, D W. (2007). How computer games help children learn. NewYork: Palgrave. Steinkuehler, C, & Duncan, S. (2008). Scientific Habits of Mind in Virtual Worlds. Journal of Science Education and Technology, 17(6), 530‐543. Vandenbroeck, IP, Goossens, J, & Clemens, M. (2007). Foresight Tackling Obesities: Future Choices. Project Report. London: Government Office for Science.

408

Learning in Context Through Games: Towards a new Typology Alex Moseley University of Leicester, Leicester, UK am14@le.ac.uk Abstract: The use of real contexts in learning has been of central interest to educational developments such as experiential learning, case studies, work placements and simulations. They centre around the benefits of putting students in realistic situations, using real tools to solve real (or realistic) problems: immersing them in realistic activity rather than theoretical concepts. A growing number of learning games are making use of context in this way: initially focussed around serious games (many close in character to simulations) and most recently distilled into immersive and pervasive games, which mix real and imagined contexts for apparently deep learning experiences. This paper reviews the existing literature around the use of context in learning, considers the applicability to learning games, and proposes new theoretical developments through the consideration of four models for the integration of context into learning experiences (based on a typology of existing contextual learning experiences and games). To illustrate the models and begin to strengthen the typology, a number of existing case studies are referenced, and further research needs highlighted. Keywords: context, authentic, learning, games, models

1. Introduction The Latin origins of the word context give a strong clue to its strength within education: cotexere, to 'weave together'. As will be explored in this paper, context draws a number of facets together, and most importantly, gives meaning to the combined facets greater than the sum of their parts: weaving them together, adding colour and texture. This is not a new revelation, of course: in daily conversation one will hear a request to ‘give me some context’, or an example that begins ‘to put it in context...’. Context adds relevance; relates a concept to some familiar territory or setting. As such, it has enjoyed long and detailed study by psychologists, educators and social scientists across a number of domains and from a range of perspectives. By contrast, the role of context in learning games is a potential area of study that, to date, has not drawn focus, despite the fact that the game design process, and games themselves, use context to great effect (immersing players in other worlds, or in deep narratives, or realistic simulations). This paper aims to structure and catalyse research in this area through a study of the types of contextual learning possible within learning games and playful activities.

2. A study in context In their review of context across a number of domains, Edwards and Miller (2007, p266) identified five principle fields engaged in the debate and study of context: socio‐cultural psychology, applied linguistics, social anthropology, social studies of science, and organisational studies. Johnson (2002) adds the scientific sphere of neuroscience, drawing on principles from biology and physics. In addition to these core areas, study has been undertaken into context applied specifically to learners and study, known as learning context (and drawn from some of the principles from the core domains). Linguists study context from two perspectives: the linguistic context (what comes before and after in writing or speech) and the non‐linguistic (eg. social, temporal, locational aspects), and focus on how context is used (and generated) subconsciously during normal conversation: “whatever information listeners (or readers) use to make sense of what is said (or written)” (Mercer 2000, p20) – most importantly, anything not relevant to the conversation is left out: and this may include otherwise substantial elements such as location, time, physical appearance etc. These focussed, localised ‘contexts of use’ also vary continually, as each new piece of information within a conversation can add or alter the perceived context (Mercer 2000, p20‐21). Ethnographers have also considered context when studying cultural behaviour. Gilbert Ryle (1968) coined the terms thin and thick description to distinguish between an out‐of‐context and in‐context observation by ethnographers (he used the example of a twitch and a wink: whilst they may look the same, the cultural context will give them different meanings).

409

Alex Moseley In neuroscience, context is used to describe the relationship between different parts: the brain takes new content, “searches for meaning and when it finds it, it learns and remembers” (Johnson 2002, p23) – this searching for meaning is the process of applying a known context to new content (and making connections between them). This is an important principle for education: learning and teaching that helps the brain to make these connections will result in more effective learning. Biological connections of this nature are known as interdependence in science: there are strong parallels with the study of context in social anthropology, in particular in the way that groups in the same social or work environment or profession build meaning and development through their social connections (and interdependence on each other). This was observed by Wenger in his study of workplace practice (1998), and described in terms of a community of practice ‐ linked by tools, methods, place and other contextual aspects. Within this environment, new members of a group learn about the community by being immersed in the same context (using the same tools or methods in the same place as the existing members). Vygotsky’s earlier study of child development matches this model of learning in context, his zone of proximal development (Vygotsky 1978) describing this transfer region between beginners and experts. Sharing a strong context will see novice members form a zone of proximity with ‘elder’ members, and benefit from enhanced and more relevant learning as a result (Lave & Wenger 1991). The work of these social anthropologists in what came to be described as situated learning led to a branch of educational research focusing on learner contexts: looking in particular at things that might affect the learner around the learning activity itself; Hansman (2001, p44) neatly summarises: "learning in context is paying attention to the interaction and intersection among people, tools, and context within a learning situation". Edwards and Mercer (1987, pp68‐9) describe the process of education as the establishment of shared mental contexts: contexts that form during learning through communal experience of, and understanding of, an activity plus the context up to that point (previous experiences with the same people, or activities, or in the same space, etc.). Through these shared activities students might generate a common vocabulary and common methods; the context being the platform on which these developments can take place (p82). Johnson (2002) extends the context that needs to be considered when she notes that students "see meaning in schoolwork when they can connect new information with prior knowledge and their own experience" (p vii), and that such context "means much more... than events located in place and time. [It] also consists of unconscious assumptions we absorb growing up, of tenaciously held convictions we gain as if by osmosis, of a worldview than unobtrusively shapes our sense of reality" (p49). Various theoretical frameworks have been used to try to make sense of these wider learning contexts, the most notable of which are activity theory and actor‐network theory. Activity theory builds on Vygotsky's work around tool mediation ‐ how people are influenced not by an object, but by mediation through joint activity with shared cultural tools ‐ and considers tools, people and objects in an interconnected system. Russell (2002, p68) notes: "an activity system might be thought of as a context for behaviour and learning, but not in the sense of something that surrounds the individual's behaviour and learning... [it is] rather a weaving together of the learner with other people and tools into a web of sociocultural interactions and meanings that are integral to the learning". Actor network theory focuses closer on the actors, or linked individuals, with an emphasis on boundary objects that form the links between the actors (tools, activities, theories, etc.): the context is described by this overarching network of actors and the boundary objects that connect them. Whilst learner contexts focus on learners and their educational context, another branch of educational theory is concerned with contexts that are outside formal learning and academia, but that have an effect on that learning. As Lave (1996, p5) notes, we cannot isolate individual actions of learners from society and their relations with it. Situated learning, as Lave, Wenger and others have seen it, is learning situated in another ‐ possibly wider, possibly more directed ‐ context. Lave (1988) compared adult mathematical learning in a traditional school environment, with adults learning the same mathematical equations within a real grocery store setting. She found that learning in the grocery store context took place through handling items, special offers and sales whilst interacting with shop workers and customers; and that this method of learning was more effective than that in the traditional classroom setting. In her review of Lave's work, Hansman (2001, p45) concludes "real‐world contexts, where there are social relationships and tools, make the best learning environments".

410

Alex Moseley

3. Real‐world contexts Kolb (1984) looked at these real‐world contexts in a different way: seeing the same learning value in them as Lave, but using them as experiences within formal education; applying existing knowledge to a real‐life context. Drawing on earlier theorists, Kolb constructed an experiential learning cycle that describes a continual loop of testing out knowledge in a real context (applying it), then reflecting on the success or failure of this experience (which generates new knowledge), then re‐applying this new knowledge to a real context, and so on through the loop. It is easy to apply Kolb’s cycle to familiar experiences: playing computer games, for example, often involves trying to complete a level, or fight an opponent, or solve a puzzle. By trying one tack, failing, reflecting, then trying a new approach, and continuing to try new approaches each time (building on the previous attempts) until we succeed, we are engaging in experiential learning. Traditional apprenticeships, of course, are true experiential learning experiences that place the learner firmly within a real‐world community of practice; but such apprenticeships have now become rare except in certain areas (manual trades, mainly) due to the cost to employers, high numbers of students needing places, and incompatibility with modern curricula. Farmer et. al. (1992) describe a modern version: cognitive apprenticeships, through a five‐stage model to integrate real‐world experiences within education. Through modelling (observing experts in methods and tools), approximating (trying out the methods and tools themselves, then reflecting on their performance: both with support), fading (support for the doing‐reflecting is reduced), self‐directed learning (practicing doing‐reflecting alone) and generalising (to other related applications), students get to experience immersion in the real world context and its methods, whilst also benefitting from reflective, supported developmental activity within an educational setting. What if the whole learning process is embedded within a real‐world context? This is the aim of authentic education, which started to emerge in the early 1980s, but by the 1990s was gaining widespread interest. Shaffer and Resnick (1999) analysed this area to come up with four types of thickly authentic (from Ryle's earlier classifications) educational experience:

activities that are aligned with the outside world;

assessment that is aligned with instruction;

topics that are aligned with what learners want to know;

methods of enquiry that are aligned with the discipline.

(after Shaffer & Resnick, 1999:197‐199) For a learning experience to be thickly authentic, all of these conditions have to be met. Shaffer later combined this work with Collins and Ferguson’s (1993) ideas of epistemic forms and epistemic games. They distinguished these as “the difference between the squares that are filled out in tic‐tac‐toe and the game itself” (p25). The epistemic forms are the squares or the structure, the underlying context; whereas the epistemic games are played out upon it with particular “rules, strategies, and different moves that players master over a period of time” (p25). Shaffer took these ideas and, applying them to the theories of authenticity and communities of practice, suggested that an epistemic frame can be used to describe the “practice, identity, interest, understanding, and epistemology” (2005, p1) of particular communities. Taking one example, “Lawyers act like lawyers, identify themselves as lawyers, are interested in legal issues, and know about the law. These skills, affiliations, habits, and understandings are made possible by looking at the world in a particular way ‐ by thinking like a lawyer” (p1) – an epistemic frame for a lawyer would then feature all of these aspects. Epistemic frames for other professions, or interest groups, would be different – but all would have rich descriptions of the particular group. Epistemic frames therefore provide a very useful context in which to base educational activity: if the learning is designed within the epistemic frame, it will all be embedded within the context of the subject or profession the student is working in or aiming towards. Shaffer describes this type of learning by extending the definition of epistemic games – activities that use methods, tools and approaches from the profession or subject in order to solve problems that reflect those a professional would have to deal with: “they make it possible for students to learn through participation in authentic recreations of valued reflective practices” (2005, p4).

411

Alex Moseley Using the example of lawyers, students might be given a typical real world briefing and asked to run a moot court, being assessed on their professional performance – all activities and assessment fitting within the epistemic frame. It should be noted that Shaffer’s (and Collins and Ferguson before him) notion of an epistemic game is not intended entirely literally: the ‘game’ being merely a set of rules, methods, approaches and strategies. However, Shaffer in particular has applied his ideas to designing playful experiences, or games in the literal sense, to increase engagement and teach concepts within an authentic context (e.g. the game Madison 2200, described in Shaffer 2005, p4, although he admits this is strictly somewhat closer to a simulation than a game).

4. Context, learning games and learning activities The use of real‐world contexts to create authentic experiences has the potential to be used within learning environments in a variety of ways. Some methods exist already in tried‐and‐tested forms: traditional and cognitive apprenticeships and epistemic games as previously described, but also more commonly the use of case studies or fieldtrips ‐ core aspects of many subject domains. Much of this existing practice goes unnoticed or unrecognised, however, and as a result research into the effectiveness of context within an educational curriculum ‐ already limited ‐ is narrowed still further in looking at the newer forms described above. Is it more effective to immerse students in real‐world content at the start of a course, in the middle, or at the end? Should theoretical frameworks or factual information be provided before the immersion in context, after, or not at all? How should the curriculum integrate with the contextual elements: should assessment be within or outside the contextual later? Do game‐based approaches (a feature of many of the recent forms of contextual learning) provide particular affordances in this area? These and many other research questions are crucial if formal education is to equip students with the correct skills for modern professions; to ensure that graduate attributes are matched to industry requirements; and to help us understand how new forms of learning and teaching (including the continuing development of games for learning) might help. In order to bring structure to this area of study, to provide focus for research into particular approaches, and to help educators choose from a clearer range of options when developing new contextual learning programmes, a typology of approaches has been developed. Through an analysis of existing methods, four principle uses of context have been identified, and their key features described. In the case of each, current approaches that map to the model are identified (with selected case studies where available), and further development areas suggested.

5. Four contextual learning models Model A: Viewing Context The simplest method of introducing context, and one that is easily applied over existing fixed curricula, centres around the use of 'snapshots' of real context in the form of case studies or examples. Almost always brought in within a curriculum after theoretical or informational learning, to exemplify certain approaches (or provide a real example on which to analyse or reflect), the contextual element tends to be a discrete package: a different narrative or presentation style to the surrounding learning and teaching, and not connected directly to the assessment format. It is also usually brought into the classroom or learning environment, rather than taking the students to the real context.

Figure 1: Model A ‐ viewing context

412

Alex Moseley Existing Approaches The use of case studies from real (or realistic) contexts is the dominant form, used in a wide range of subject domains and in a variety of forms. They may be transcripts of conversation, reports of a particular event, audio‐visual material, company reports or case histories, etc. Usually the case studies are presented, and learners are then asked to reflect on and analyse the content, with teachers focusing or guiding discussion to match the intended learning outcomes. Further development work The effectiveness of case studies in "bridg[ing] the gap between theory and practice and between the academy and the workplace" (Barkley et al 2005, p182) has already been studied at length; however, the form of embedding and transfer activity are both in need of further research. Are videos more effective at generating the real context for the learner, than text transcripts or other forms? Should the case studies be presented in a different way, time or place to the rest of the curriculum? What are the most effective forms of transfer activity, to draw out lessons from the case studies into the wider curriculum? Model B: Using Contexts as Seeds This model describes a range of approaches that, for a certain portion of the curriculum, immerse learners within a real or realistic context for a period. Immersion goes beyond case studies to involve the students in real aspects of the context (locations, methods, tools, decisions, events etc.). In order to link the contextual element with the curriculum, theory or knowledge might be delivered before the contextual portion; or themes within the real(istic) context might be drawn out to link with the curriculum as 'lenses'. For example, a 'lens' of teamwork might be used to focus on that aspect of a group fieldwork project ‐ learners would be provided with the 'lens' details before entering the contextual portion, and would have it in mind as they perform in context, making it easier to reflect on teamwork aspects afterwards.

Figure 2: Model B ‐ using contexts as seeds Existing Approaches On a long timescale, work placements, fieldwork or years in industry/professions are an example of this model. Programmes in medicine (with periods in the real hospital context), teacher training (with a period of teaching practice in schools) and modern languages (with a year abroad immersed in the country of native language) commonly use this model, with training in theory and skills before the immersion, and reflection on practice and performance following it. Lenses are also commonly used (trainee teachers are asked to focus on particular planning and management skills, for example). Learning games that model particular contexts and simulations yet aren't developed for a specific curriculum conform to this model: they provide a realistic context at a certain point within the curriculum, provide

413

Alex Moseley immersion for the learner, yet weren't designed with specific learning outcomes or curriculum in mind. One of the first such games to be used within a training environment was the Monopologs inventory management simulation for the American Air Force (Renshaw and Heuston, 1957), but many off‐the‐shelf games and those involving detailed simulations of contexts are delivered in this way. The author ran a series of sandpits: rapid research project developments for creative professionals in the museum sector, in which a full research proposal was developed within two days. At the start of the sandpit, participants were given four visitor types to consider (their lenses) and immersed in a normal morning in a local museum, entering each gallery as 'visitors' and using the lenses to guide their movement and browsing. They then returned to the classroom to reflect on their experiences through the same lenses, before current theory and practice in visitor studies was introduced. Further development work Within this model are a range of possible orientations: the contextual element could take a number of forms in time, type and position within the curriculum: some may be better suited for particular domains or learning outcomes; some may provide more effective transfer of theory through applied practice to analysis and reflection. There is already some work in the area of embedding games within the curriculum (Whitton 2009, pp77‐88; Moseley and Jones 2012, p112) and growing evidence that the use of a reflective/analysis stage after playing a learning game or simulation provides more effective learning. Further experimentation of forms, and research into the many variables, is needed however. Model C: Applying Context In this model an activity or game is designed that adds an authentic layer to a significant portion of the curriculum: although principles or theories may be taught in the classroom as preparation for the contextual element, and assessment might take place outside of the authentic layer, back in the classroom. Within the authentic layer, the curriculum is delivered using contextual features: learning scenarios or activities are taken from those in the real context, matched to the learning outcomes; locations, objects, and real or simulated characters from the context might be used. Often, a detailed narrative or scenario will help to embed curriculum needs within a contextual frame. If assessment takes place in context, the assessment would be contextually‐relevant (for example, if the authentic layer is drawn from an engineering firm, the assessment might involve the creation of a design proposal and presentation to a commissioning panel). If out of context, it would reflect on the principles and frameworks defined earlier in the programme, as applied to the new and applied knowledge resulting from the contextual element.

Figure 3: Model C ‐ applying contexts Existing Approaches Learning games that have been designed specifically for a programme (around learning outcomes, or to cover specific aspects of the curriculum) fit into this model: contextual elements within the games can be used to generate authentic environments, yet the design parameters ensure a close fit with curriculum aims. This includes Schaffer's epistemic games (although some fuller implementations may fall into Model D). Where an

414

Alex Moseley existing, off‐the‐shelf, game matches the curriculum requirements and required context closely, that may also fit into this model: an example is the Civilisation commercial game that covers the development of societies over time, and has been used in a number of programmes covering history, sociology and politics (see eg. Squire and Barab, 2004). A growing number of board and card games that generate authentic contexts through simple design elements, and are designed and embedded into a programme, are another solid example of the model (Moseley and Whitton, 2013). Case studies that include an interactive component (where learners play a part in shaping the outcome of the case study, or create their own in‐context case study through study and modelling of others) would also fit into this model; as would roleplaying activities, where learners are given contextual roles and asked to play out events or activities from the real context. A range of other authentic‐learning approaches fit this model: in experiential learning, the use of metaphors or frame stories to structure contextual activities (and often to include reflection and assessment) are an effective example (see eg. Hildmann and Hildmann 2009). At the University of Leicester, final year Physics students take on the role of real‐world physicists and consider a research problem from literature or film (for example, how many seagulls would actually be needed to lift Roald Dahl's James's giant peach? ‐ Flood, 2013): their activity (working as research groups) and assessment (submit an academic paper to a journal) both occur within the authentic layer. Further development work Due to the potential range of approaches that might fit this model, further research and development would most usefully be focussed on the interfaces between the curriculum and the authentic layer: how are principles, theories or frameworks effectively set up before learners enter the authentic layer? Within the layer, what are the most effective methods for guiding or focussing learning (narratives, metaphors, theoretical lenses etc)? Is assessment more effective when embedded in the context, or as an out‐of‐context test of application/reflection/analysis? Model D: Designing In Context In this model the curriculum is developed with the real context firmly in mind: learning objectives are fully aligned to a real profession or role. Learners are therefore embedded in the context from the first day of the programme to the last, with theory, knowledge, practice, reflection, assessment and feedback all presented in an authentic form. The curriculum therefore tends to be delivered through a contextual narrative, or one or more scenarios ‐ designed to introduce the key theories, knowledge and skills through experiential learning methods. The real context will be evident in all aspects of the programme: learner roles, equipment, 'tools' (digital and non‐digital: eg. email, SMS text, letters, written reports, websites etc.: whatever is normal within the real context), events, etc. Assessment is likely to be designed into the real context, in line with the rest of the programme (and may be continuous or at fixed points); although learners might be brought out of context to apply theoretical or analysing lenses to their performance, and be assessed on these reflective elements.

Figure 4: Model D ‐ designing in context

415

Alex Moseley Existing Approaches The dominant traditional form of this model is apprenticeship: apprentices follow an (often unwritten) curriculum that will provide them with all learning opportunities and assessment within the authentic environment. This is where learning games have proved most interesting in the past decade, however. The development of immersive and pervasive gaming forms and their application to education has seen them used as authentic curriculum delivery forms in areas such as induction (eg. the ARGOSI project at Manchester Metropolitan University: Whitton 2009a), language learning (Connolly et. al. 2011), enterprise education (Brookes et. al. 2012) and transferable skills (Squire and Jan 2007). Based around alternate reality games (Whitton and Moseley 2012, p143) or emerging hybrid forms such as pervasive learning activities (Brookes and Moseley, 2012), such game forms embed learners in deep authentic contexts, mixing reality with fiction but using the same authentic tools, methods and activities throughout. Further development work There has been much consideration of the merits of apprenticeship, and this approach has its own strong area of interest and research. The area of immersive or pervasive games in education is, however, a relatively new one. Some research has already considered their effectiveness within education, particularly in the area of motivation (Moseley et. al. 2009), but this has highlighted a tendency for only a small proportion of learners to engage fully in the game and therefore learning context: and this is clearly an area for further study and development of the form. Aside from study of existing forms is the potential for the development of new forms, both games‐based and non‐games‐based, that provide effective ways to develop and deliver curricula entirely within a real or realistic context. For these reasons, this is certainly the most interesting of the four models in terms of further development work.

6. Conclusion The four models are presented here for two primary purposes:

to map existing contextual learning forms onto a framework, and encourage the study of elements within the model (in particular the methods of transfer into and out of contextual elements, the design frameworks that map curricula to real contexts, and the assessment forms) as well as overall effectiveness of particular approaches.

to provide a range of models for those teachers interested in adding contextual learning to their own programmes, ranging from simple non‐embedded approaches (model A) through to new or redesigned programmes that are developed in tandem with a real context (model D).

In particular, the role of learning games in models B through D is deserving of further study: why is it that games lend themselves well to the more embedded forms of contextual learning? Are they good starting points for teachers keen to embed context in their programmes, and if so, what are the key design steps for embedding such context? Finally, it is hoped that through application and analysis using these models, the models themselves will be tested, refined, and if necessary added to.

References Barkley, E. F, Cross, K. P. & Major, C. H. (2005). Collaborative Learning Techniques: A Handbook for College Faculty. San‐ Francisco: Jossey‐Bass. Brookes, S. & Moseley, A. (2012) Authentic Contextual Games for Learning. In N. Whitton & A. Moseley (eds) Using Games to Enhance Learning and Teaching: A Beginner’s Guide. New York: Routledge, 91‐107. Brookes, S., Moseley, A. & Underwood, S. (2012). Contextual Games: Pervasive Learning Activities in Enterprise Education (a case study). Presented at the ISBE 2012 Conference, Dublin, Ireland, 7‐8th November. [online] http://www.pla‐ academy.co.uk/home/wp‐content/uploads/2013/02/Universityof_957_Full_paper_final_2012.pdf [accessed 2/5/2013].

416

Alex Moseley Collins, A. & Ferguson, W. (1993). Epistemic forms and Epistemic Games: Structures and Strategies to Guide Inquiry. Educational Psychologist 28:1, 25‐42. Connolly, T.M., Stansfield, M.H. & Hainey, T. (2011). An Alternate Reality Game for Language Learning: ARGuing for Multilingual Motivation, Computers and Education, 57:1, 1389‐141. Edwards, D., & Mercer, N. (1987). Common Knowledge: The Development of Understanding in the Classroom. London: Methuen. Edwards, R. & Miller, K. (2007). Putting the Context Into Learning. Pedagogy, Culture & Society 15:3, 263–274. Farmer, J. A., Buckmaster, A., and LeGrand Brandt, B. (1992). Cognitive Apprenticeship: Implications for Continuing Professional Education. In H. K. Morris Baskett & V. Marsick (eds) Professionals’ Ways of Knowing: Findings on How to Improve Professional Education. New Directions for Adult and Continuing Education, 55. San Francisco: Jossey‐Bass. Flood, A. (2013). Think again, Roald Dahl: scientists hit on giant peach of a theory. The Guardian, 4 January 2013. [online] http://www.guardian.co.uk/books/2013/jan/04/roald‐dahl‐scientists‐giant‐peach [accessed: 1/5/2013]. Hansman, C.A. (2001). Context‐Based Adult Learning. New Directions for Adult and Continuing Education 89, 43–52. Hildmann, J. & Hildmann, H. (2009). Promoting Social Skills Through Initiative Games in the Classroom and Assessing Their Effects. In Proceedings of the 3rd European Conference on Games Based Learning: FH Joanneum University of Applied Sciences, Graz, Austria, 12 ‐ 13 October 2009. Reading: Academic Conferences, 180‐194. Johnson, E. B. (2002). Contextual Teaching and Learning. Thousand Oaks, CA: Corwin Press. Kolb, D.A. (1984). Experiential learning : experience as the source of learning and development. London: Prentice‐Hall. Lave, J. (1996). The Practice of Learning. In S. Chaiklin & J. Lave (eds) Understanding Practice : perspectives on activity and context. Cambridge: Cambridge University Press, 3‐34. Lave, J. (1998). Cognition in Practice: Mind, Mathematics, and Culture in Everyday Life. Cambridge: Cambridge University Press. Lave, J. & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: University of Cambridge Press. Mercer, N. (2000). Words and Minds: How we use language to think together and get things done. New York: Routledge. Moseley, A. & Jones, R. (2012) Mapping Games to Curricula. In N. Whitton & A. Moseley (eds) Using Games to Enhance Learning and Teaching: A Beginner’s Guide. New York: Routledge, 108‐123. Moseley, A. & Whitton, N. (eds) (2013). New Traditional Games for Learning: A case book. New York: Routledge. Moseley, A., Whitton, N., Culver, J. & Piatt, K. (2009). Motivation in Alternate Reality Gaming Environments and Implications for Learning. In Proceedings of the 3rd European Conference on Games Based Learning. Graz: Academic Conferences. Renshaw, J. R. & Heuston, A. (1957). The Game Monopologs. Santa Monica, CA: RAND Corporation, 1957. [online] http://www.rand.org/pubs/research_memoranda/RM1917‐1 [accessed: 1/5/2013]. Russell, R. R. (2002) Looking beyond the interface: activity theory and distributed learning. In M. Lea & K. Nicholl (eds) Distributed Learning: social and cultural approaches to practice, London, Routledge with The Open University, 64–82. Ryle, G. (1968). The Thinking of Thoughts. In University Lectures, The University of Saskatchewan, Vol 18. Shaffer, D.W. (2005). Epistemic games. Innovate 1/6. [online] http://www.innovateonline.info/index.php?view=article&id=79 [accessed 20/4/2013]. Shaffer, D.W. & Resnick, M. (1999). ‘Thick’ Authenticity: New Media and Authentic Learning. Journal of Interactive Learning Research 10:2, 195‐215. Squire, K. & Barab, S. (2004). Replaying history: engaging urban underserved students in learning world history through computer simulation games. In Proceedings of the 6th international conference on Learning sciences (ICLS '04). International Society of the Learning Sciences 505‐512. Squire, K. & Jan, M. (2007). Mad City Mystery: Developing Scientific Argumentation Skills with a Place‐based Augmented Reality Game on Handheld Computers. Journal of Science Education and Technology, 16:1, 5‐29. Vygotsky, L.S. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press. Wenger, E. (1998). Communities of Practice: Learning, Meaning, and Identity. Cambridge, UK: Cambridge University Press. Whitton, N. (2009). Learning with digital games. A practical guide to engaging students in higher education. New York: Routledge. Whitton, N. (2009a). Alternate Reality Games for Orientation, Socialisation and Induction. [online] http://argosi.playthinklearn.net/ [accessed 28/4/2013]. Whitton, N. & Moseley, A. (eds) (2012). Using Games to Enhance Learning and Teaching: A Beginner’s Guide. New York: Routledge.

417

Let the Students Contruct Their own fun And Knowledge ‐ Learning to Program by Building Computer Games Peter Mozelius1, Olga Shabalina2, Christos Malliarakis3, Florica Tomos4, Chris Miller4 and David Turner4 1 Stockholm University, Sweden 2 Volgograd State Technical University, Russia 3 University of Macedonia, Greece 4 Glamorgan University, Wales mozelius@dsv.su.se Abstract: Computer programming is a core subject in most Computer science programmes at university level but many students have difficulties with both the understanding of theoretical concepts and with the learning of practical programming skills. Several studies have pointed out that there exist pedagogical and motivational problems resulting in high drop‐out rates and low learning outcomes. On the other hand today’s students have grown up in a digital and internet connected world where playing computer games is a common and appreciated spare time activity. It has been discussed during many years amongst teachers and researchers which important features a programming language should have to support learning and which the appropriate programming paradigms are for introductory programming courses. Less has been discussed and written about modifying the actual course content and use the recognised attraction and catalytic effect of computer games. The aim of this case study is to describe, analyze and discuss the concept of learning to program by game construction. In this case study two course analyses have been combined with a literature review on pedagogy for Game Based Learning (GBL Findings from the literature review show that the idea of students constructing knowledge in their interactions with their environment is not a new one and that games have been used in educational contexts long before the introduction of computers. The idea of Game based learning has support in the constructivist learning theory concept that was introduced theoretically at university level in the 1970s by Jean Piaget and Lev Vygotsky. Results from the study also indicate that the practical application of the game construction concept has been successful in the two investigated programming courses both when it comes to student motivation and learning outcomes. Furthermore, this game based pedagogy can motivate students not only to develop themselves as future programmers but also to become more innovative and entrepreneurial by improving the quality and performance of the games in order to deploy, promote and sell them. Keywords: game‐based learning, GBL, computer games, programming education, constructivism

1. Introduction In most Computer science programmes at university level programming is still a core subject but several studies on programming education show that many students are facing difficulties even in the understanding of basic programming techniques (Guzdial & Soloway 2002) (Järvinen, Ala‐Mutka and Lahtinen, 2005) (Eckerdal, 2009). This is not only related to theoretical programming concepts and research studies show that there are problems with the practical programming and code construction as well (Eckerdal, 2009). Some researchers see this mainly as a pedagogical problem but maybe as a psychological and motivational issue as well (Wiedenbeck, Labelle & Kain, 2004). Drop‐out rates from programming courses are high in general (Guzdial & Soloway 2002) and there is no consensus on how programming should be introduced. Some researchers have strongly argued for an early introduction of object‐oriented techniques and concepts (Kölling & Rosenberg, 1996) (Cooper, Dann & Pausch, 2003) meanwhile others completely reject the idea of object‐orientation for introductory courses (Decker & Hirshfield, 1994). It has been widely discussed amongst teachers and researchers which important features a programming language should have to support learning (Mannila & De Raad, 2006), and which programming paradigms that are appropriate for introductory programming courses (Vujusevic‐Janicic & Tosic, 2008). Less has been discussed and written about modifying the actual course content and use the recognised attraction of computer games. Many students find programming education difficult and at the same time boring. Programming courses often have a focus on syntax and algorithms without considering differences in students’ learning styles. (Jenkins, 2002) A solution recommended in studies is to redesign courses and give them a more flexible outline to allow different students to learn in different ways. According to (Fasli et al, 2009) a successful learning environment

418

Peter Mozelius et al. should provide a combination of challenge and entertainment for the students. What effect the playing of computer games could have on students’ ability to successfully execute given tasks has been studied studied by Pillay (2003). He claims that the completion of tasks through educational‐based computer games can increase the students’ cognitive abilities Playing games as a pedagogical approach was introduced at university level in the 1970s by Jean Piaget (1973) and Lev Vygotsky (1978) but a broader discussion on playing started earlier in 1938 when the Dutch historian and cultural theorist Johan Huizinga published his ideas on man as a Homo Ludens (Huizinga, 1938) where play is described as a basic cultural phenomenon. There is no model that successfully integrates educational theory and game design aspects but idea use of integrating flow theory with learning theory in computer games have been investigated by the Finnish researcher Kristian Kiili (2005). He has also studied how students can improve their learning process when they produce their own multimedia learning objects (Kiili, 2006). The use of Game Based Learning (GBL) in education has generally increased during the last decade but there is a risk for negative consequences with a stereotypical gamification (Kapp, 2012). There are two different approaches in GBL for programming education. In the first one the main idea is that students learn to program by constructing digital games. By using imagination and creativity in order to build their own executable computer games they will at the same time learn traditional programming techniques. In the second approach, they learn programming by playing educational or serious games where programming skills and knowledge can be learnt from the actual gaming (Ljungkvist, & Mozelius, 2012). This study will have a focus on the first approach: Learning to program by game construction.

1.1 Problem A dilemma in many Computer science programmes today is how to engage novice students in programming courses (Järvinen, Ala‐Mutka and Lahtinen, 2005) (Eckerdal, 2009) and that the courses at the same time must provide sound instruction in traditional programming techniques (Guzdial & Soloway 2002) (Haden, 2006). There is no clear answer to how programming courses should be designed to obtain a higher student engagement and at the same time keep the learning outcomes at a high level but research indicates that students find the programming courses boring (Jenkins, 2002) and that the generally low pass rates and learning outcomes might depend on students’ lack of motivation (Wiedenbeck, Labelle & Kain, 2004).

1.2 Aim The aim of this study is to describe and discuss the design and outcome of some programming courses based on the concept of learning to program by game construction.

2. A constructivist view of learning and teaching This study will consider the construction of computer games as an educational method to gain knowledge about basic programming by the novice students with different levels of basic knowledge and a manner to increase their motivation. For this reason, in 1970 Piaget emphasised the role of “knowing how” within the evolution of cognitive functions. Piaget (1970; 1980) argued that the intelligence and the cognitive development are in fact two different and concurrent processes, although related phases. Furthermore, Piaget (1970) argued that the constructivism of knowledge is an act of “creation of novelty”. Based on the above justification, this research suggests that the knowledge is determined by students’ creativity in learning programming by games construction. Piaget’s theory about the construction of knowledge supported also substantial criticism. Thus, referring to Piaget’s opinion on learning and child internal representation of things, Fox (n. d.) criticised the idea of general and synchronized development of all areas of the mind and indicated that recent studies demonstrated that knowledge develops independently in different areas of experience. However, Piaget (1970) argued that the pressure of the environment upon individuals requires the need of accommodation. Furthermore, an increase in knowledge happens when the individual ensures an equilibrium between accommodation and assimilation, in other words, adaptation (Piaget, 1970). Moreover, Piaget (1970) defined the intelligence as the assimilation, or the process of including his/her experience within a framework. Additionally, Fox (n. d.) recommended Vygotsky’s research on individual mind. Vygotsky (1978: 24) discovered that “...practical and abstract intelligence, occurs when speech and practical activity...converge”. Nevertheless, was Piaget (1970:

419

Peter Mozelius et al. 398) who defined intelligence as “...the development of an assimilatory activity... [whose] ...structures are elaborated by the interaction between itself and the external environment”. As a consequence, student’s mind develops independently according to their own experience. However, their intelligence progresses when social interaction by communication and practice takes place at once. Additionally, Piaget (1970) emphasised the role of active experimentation within the discovery process. This indicates that students’ active experimentation and communication emerge into knowledge by discovery and innovation. Piaget (1970) argued that within the sensori‐motor intelligence phase, a child explores the new situation. Similarly, a student explores a new task, in order to assimilate the knowledge. However, the mature student requires a period o concrete operation in order to be able to face “the formal operation period”. When the student deals with with a new situation, he/she has to find appropriate means to achieve his/her target. Thus, students will search for well‐known means as initial basis of knowledge, acquired previously and which they can use successfully to solve a new problem (Piaget, 1970). Vygotsky’s theory on games to support learning The use of games to support learning was introduced theoretically at university level in the 1970s by Jean Piaget (1973) and Lev Vygotsky (1978). Vygosky’s (1978) highlighted the existence of three main theoretical positions regarding the relations between child developments and learning process. The first theoretical viewpoint assumes that the learning process is an independent process. The second presumes that the learning and development are mixed, and the third opinion saw the process of learning as embedded within the development process. Further, Vygotsky (1978) recognised the relationship between learning and development and differentiated between the actual development and the possible development under guidance. This study will use construction of computer games in order to learn programming and will build on students’ basic information and knowledge, ensuring differentiation to reach the student potential development in the real world (Piaget, 1970,1980; Vygotsky, 1978). Vygotsky (1978) approached the importance of games and motivation these can give. Vygotsky also defined the meaning of action and play and explained the games as the prediction of future situations. Consequently, Vygotsky tried to present the role of imaginary situation for further learning and development. This, Vygotsky said, will assist student imagination, creativity and innovation. According to Kaffka (Vygotsky, 1978) the play is an imaginary world that can be transferred into life. It has practical consequences for student development and trust in approaching more confident situations within the real world. Methodology and Methods The overall approach for our research is the case study strategy where a case study is defined as an empirical inquiry investigating a real world contemporary phenomenon (Yin, 1989). Case studies are a strategy where the researchers explore and evaluates a programme, process or activity in depth using a combination of methods for the data collection (Creswell, 2009) in a setup where the different sources of evidence together should generate a deeper understanding of the investigated phenomenon (Remenyi, 2012). Case study research is more than any other kind of research relying on both primary and secondary data (Remenyi, 2012) and in this study the primary data from the two embedded case units are completed with secondary data from a literature study on constructivist pedagogy and game‐based learning. Case Study Unit 1 – Volgograd The experiment involved 40 students in total with an age of 18‐22, who studied software engineering during four years and got a bachelor degree after finishing their study. All the students studied development process using project‐oriented approach. 13 of them wished to choose educational games as their study projects. The results of students’ study were collected and analysed. All the courses that cover software development were included into the analysis (Programming Languages, Data Bases, Computer Graphics, Theory Translation and others, 25 in total). Students obtained scores in the range [61, 100] according to the Russian grading standard for every course they studied. Data has been collected from the teachers of corresponding subjects. Average improvement of scores during study period was measured. According to accomplished ANOVA analysis, the average improvement in a group of students involved into educational games development was higher (+3.81 versus +1.38) and this increase was statistically significant. The probability that all the students belong to the

420

Peter Mozelius et al. same group and this increase is accidental was computed with T‐test t=0.0483 is less than 0.05, which means that these two evaluated groups are significantly different with p=0.95. Case Study Unit 2 – Stockholm The focus in this unit has been on the course Multimedia programming in Python that is described in detail under 2.3. Data has been collected in a combination of semi‐structured interviews, document analyses and an analysis of student’s postings in on‐line forums. Two students that have taken the course at different time periods were selected for interviews. One of them had earlier knowledge of Python and the other had to learn Python from the beginning. The student that had to learn Python from scratch in now working in a company that develops and sells computer games, the other student is using Python to build an artefact for his Bachelor thesis. Both informants were committed when they took the course but they were selected because of their enthusiasm and not because they were top students in their course batches. The conducted interviews are semi‐structured and the informants will in this article be referred to as Student 1 and Student 2. Interviews were both recorded as mp3‐files with a successful result but as a backup notes were taken by pen and paper as well. Document analyses are based on the course syllabus and course statistics from 2009 ‐ 2012 retrieved from DSV’s internal information and course management system Daisy (Daisy, 2012). Student’s course evaluations in the Daisy system have been compared with student discussions in the Moodle virtual learning environment at: https://ilearn.dsv.su.se

3. Learning to program by game construction Case Study Unit 1 – Volgograd State Technical University Computer‐Aided Design department at Volgograd State Technical University graduates specialists in computer‐ aided systems design and development. The most part of special courses starting from the third year require a high level of programming skills. So thus students start learning programming languages from the very beginning: they study programming languages C, C++ during the spring semester of the first year and the autumn semester of the second one. After finishing the second year studying and passing the exams all the students are required to pass a month training course (normally in July). The main objective of this course is to gain practical skills in programming by doing some simple but real‐world tasks. Each student (or a small team of students) gets an individual task and works on it during one month under supervision of a teacher. During the third and the fourth year students use their skills while fulfilling different projects included in corresponding courses dealing with software development components like data bases, artificial intelligence and computer graphics. Over the last several years educational games have been used for teaching programming (Shabalina, Chickerur, 2010). A game‐based approach is integrated to the all stages of the learning process. The teaching methodology is based on a two‐fold use of the games. Games for teaching programing developed by students are used for teaching the next generation of students. After gaining basic skills in programming these students are again involved in the development of new educational games. Learning programming language by the use of games Existing games for training programing skills such as Colobot and its modification CeeBot (Epsitec Games, 2001), Robocode (Robocode, 2001), Robot Battle (Garage Games, 2002) and others are based on an idea of writing programming code and using it for controlling the behavior of some game objects. Thus the game process includes two types of action – training in programming and game actions that occur sequentially. It means that training programming in such games is almost the same as in traditional learning. In order to train skills in a game context an approach of fulfilling learning and gaming task in parallel (Shabalina et al., 2012) is applied in games for training programming skills The idea is based on writing programming code as step‐by‐step composition of a sequence of lexemes. To do this what a player must do at each step is to choose a lexeme from a set of choices which include at least on correct lexeme (from the perspective of fitting the possible task decision). Each lexeme is matched to a game object, and the rules of game are interpreted in such a way that a player could compose programming code from the lexemes by controlling the game objects. Step‐by‐step composition of programming code helps students to learn programming language syntax and understand the code structure and to keep their interest in playing a game.

421

Peter Mozelius et al. Examples of games for training basic programming skills The set of games for training basic programming skills was developed by second and third year students and present different realizations of the idea of composing programming code from separate lexemes in a game context. Popular mini‐games were used as game prototypes as these mini‐games are familiar to almost all players. Then the players do not have to learn the game rules because they already know them, they are ready to learn through that game immediately. In order for the player to compose the code for a task one step at a time a set of alternative lexemes is presented to the player at each step. The set includes some correct lexemes (one or more depending on the game options) and a set of lexemes that can be chosen randomly from a database of possibilities. The total number of lexemes depends on certain game rules. The set of alternative lexemes is dynamically generated as the set of correct lexemes on the next step depends on the lexeme chosen by a player on the previous step. After a player has chosen the next lexeme to add to the program code it is checked as a partial task solution. If the lexeme chosen by the player matches some correct solution it displays on the output window, if it doesn’t, the player is penalised, and gets a new set of lexemes to choose from. Screenshots presented in Figure 1 show some games for training programming skills developed by the students.

1a) The CSnake game

1b) The CRace game

Figure 1: Screenshots of games for training programming skills developed by students Case Study Unit 2 – Stockholm University At the Department of Computer and Systems Sciences at the Stockholm University there is a tradition of building computer games in programming courses, and after the establishment of a Computer Game Development Bachelor’s programme games are now seen as more constructive that distractive. But in a standard programming course on a Bachelor’s or Master’s programme there is not that much of games or game construction and programming techniques are taught and learned by more traditional exercises and assignments. University programmes in Sweden are a combination of spring and autumn semester, but in the last decades the number of summer courses given in a shorter and more informal summer trimester has increased. A summer course is seldom part of a curriculum or aligned to any programme and can for that reason be designed a bit different and more experimental. The course that will be described here is Multimedia programming in Python, a course where game construction with multimedia techniques is the way for students to learn important basic imperative programming techniques and how they should be implemented in the Python programming language. The is a distance course given in the Moodle environment where streaming lectures and tutorials combined with 2 course books provides the information for the students independent work with four assignments and a game project. Course Design Practically everything in the course is based on analysing and building games with two course books where all code examples are runnable computer games. The first book Python Programming for the Absolute Beginner (Dawson, 2010) introduces basic programming concepts like variables, selection, iteration, Boolean condition

422

Peter Mozelius et al. and data structures in commented code examples of classical computer games like Hangman and Tic Tac Toe. The involved programming techniques are also explained in streaming lectures and exemplified in recorded online lessons and tutorials. As in many other game‐themed introductory programming courses the first start‐ up assignment is relatively simple and on basic integer and floating‐point number arithmetic (Sung, 2008). Assignment 1 can later in the students’ project be extended to a mathematical educational game. In Assignment 2 the students should, with help from the course literature write code for the well‐known computer game Guess my Number (BFGL, 2010, a kid’s game where the player have to use the Computer science principle of ‘Divide and Conquer’ to get a good score. In Assignment 3 this should be enhanced with a graphical user interface built by the use of the Tkinter library (Lundh, 1999) that is an integrated part of the Python language. Assignment 4 introduces core Python multimedia features for the construction of a slideshow where images combined with sound stored in a data structure should be combined with a navigation system. For students that find the final project a difficult task Assignment 4 can later, for a grade of D or C, be extended to a multimedia quiz. For the grades of A and B the game idea must be more complex or involve techniques from outside the course syllabus.

Figure 2: Flag quiz (screen shot from a student project) To provide techniques and ideas for more complex game construction in the final project there is a second course book More Python Programming for the Absolute Beginner (Harbour, 2012) that introduces multimedia features in the add‐on library pygame. Like in the main book by Michael Dawson (2010) practically all chapters in the book are built around the Python code for complete and executable games. Most games in the book are classical computer games like Snake and Block Breaker. Compared to programming education in general the described course outline could be classified as more constructivist but not as pure constructivism. The frequent use of games and game construction is meant to support the Vygotskian ideas of stimulating learners’ imagination, creativity and innovation by games and play. Assignments are composed with a mix of closed and open exercises where the more closed assignments with a given game design should provide training of traditional programming techniques. The final open project assignment where students’ should design their own games is on the other hand given to stimulate students’ active experimentation and creation of knowledge by discovery and innovation in the spirit of Jean Piaget.

4. Findings and discussions The tradition of using games in educational contexts is much older than Computer science and several well described examples exist on how games and computer games can be used in teaching sessions and self

423

Peter Mozelius et al. learning. Theoretically, the concept of using games at university level was introduced in the 1970s by Jean Piaget and Lev Vygotsky. Game Based Learning for programming education can be divided into two main approaches. In the first one, students learn by constructing computer games when they use their creativity to design and implement. The second approach is based on students learning to program by playing games classified as educational games or as serious games. Students that participated in the experiment at Volgograd State Technical University and developed educational games expressed their opinion about the project. They consider game development as a very inspiring thing to do. The development of educational games helped them a lot in practicing skills they learned during studying. Also, most of the students noticed that they were motivated to learn many new things that are not included in the standard curriculum. Analysis of using game development for teaching programming shows that students gained knowledge and skills that to a considerable extent extend the educational program, and raised their qualifications. Students who participated in game development projects gained important professional skills such as dealing with real projects, software design, testing, debugging and development work with open libraries, Version Control Systems (VCS) and other modern tools, and also working with somebody else's code. Students who participated in game projects gained soft skills such as team working, project management, priority assignment and conflict resolution. Students and graduates have created two small enterprises and obtained financial support from the Russian government. The companies are actively developing and produce regular and educational games for different platforms – PCs, mobile devices and games for social nets. As a result, students have been able to gain work placements during their study. Compared to other summer courses given in distance mode at the Department of Computer and System Sciences Multimedia, programming in Python has the highest past rate ever. The course has been given four times with pass rates between 54 – 66 percent which is a good result even when the course is compared with other courses than just programming courses. Most students are males (around 90%) and between 20 to 35 years but in the last course batch the oldest participant was 69 years old. Most students classify the difficulty level as average, and a bit easier than programming courses in general. But with variations since many students have chosen to develop complex games in the final project. What the interviewed students liked most about the course was the freedom to design and develop their own game idea in the final project. Student 1 mentioned that some of his classmates chose the “easy way out” and completed the final project quite quickly by building some of the given alternatives for less complex games. On the other hand, he and other students spent several months on the design and implementation of games that had features that were not replicated by other games. Student 2 said that he appreciated the freedom to plan and implement his own unique game project in a way that has similarities with how he is now working for a software company where his main occupation is to construct computer games that sometimes are less complex than the game he built in the course project. Both students said that what might be missing in the course are assignments on the Python model for object‐orientation but, as an afterthought, they mentioned that this might be something for another separate course. On the final question regarding what parts of courses that teach Computer Science programs can be gamified, both students were a bit hesitant. However, after a while they both said that this would be an appreciated element in several courses, not necessarily in all.

5. Conclusions The use of games to support learning is an old concept, far older than the history of Computer science and programming education. As several pedagogues and researchers has found, games and gaming can be used in education to raise engagement. In the 21st century where students have grown up with computers, virtual environments and with digital game playing as part of their everyday life, the use of computer games in education seems to be this century’s learning through entertainment. Game Based Learning for programming education can be divided into two main categories. The first category is a pedagogical approach based on students learning by playing educational and serious games. In the other category discussed in this article students develop their programming skills by designing, implementing and testing their own computer games. Both described programming courses contain components that have roots in other pedagogical theories but the main common approach is constructivism. Courses on computer science and programming are like most courses at university level, built by a mix of pedagogical ideas, but to improve

424

Peter Mozelius et al. the students’ theoretical knowledge as well as their practical skills we believe that making is thinking and effective learning. The discussed concept of learning to program by game construction seems promising and our recommendation is that the described approach is worth to implement in more programming courses.

References Creswell, J. W. (2009) “Research Design, Qualitative, Quantitative and Mixed Methods Approaches”, Sage Publications Inc, ISBN: 978‐1‐4129‐6557‐6 Daisy (2012) ”DSV's internal information system” (retrieved 20/04/2013) http://dsv.su.se/en/education/comp/daisy/ Dawson, M. (2010) "Python Programming for the Absolute Beginner" Course Technology, CENGAGE Learning, ISBN10: 1435455002 Decker, R. & Hirshfield, S. (1994) "The top 10 reasons why object‐oriented programming can't be taught in CS 1" ACM SIGCSE Bulletin, 26 (1) Eckerdal A. (2009) “Novice Programming Students' Learning of Concepts and Practise”, PhD Thesis Uppsala University, available at http://www.avhandlingar.se/avhandling/6809751ebf/ Epsitec Games (2001) “Colobot” http://www.ceebot.com/colobot (retrieved 18/04/2013) Fasli, M. & Michalakopoulos, M. (2006) “Interactive Game based Learning”, Association for Learning Technology Newsletter, 1(6). Guzdial, M. & Soloway, E. (2002) “Log on education: teaching the Nintendo generation to program” Communications of the ACM, 45(4), pp. 17‐21. Haden, P. (2006) “The incredible rainbow spitting chicken: teaching traditional programming skills through games programming", ACE '06: Australian conference on Computing education Harbour, J. S. M. (2012) "More Python Programming for the Absolute Beginner" Course Technology, CENGAGE Learning, ISBN10: 1435459809 Huizinga, J. (1955, originally published in 1938) “Homo Ludens: A Study of the Play Element in Culture” Beacon Press, Boston Jenkins, T. (2002) “On the difficulty of learning to program”, In Proceedings of the 3rd Annual LTSN_ICS Conference, The Higher Education Academy, pp. 53‐58. Kapp, K. M. (2012) ”The Gamification of Learning and Instruction: Game‐based Methods and Strategies for Training and Education”, John Wiley & Sons, ISBN: 978‐1‐118‐09634‐5 Kiili, K. (2005). “Digital game‐based learning: Towards an experiential gaming model” The Internet and Higher Education, 8, 13–24. Kiili, K. (2006) ”Towards a participatory multimedia learning model”, Education and Information Technologies, 11(1), 21‐32. Kölling, M. & Rosenberg, J. (1996) "An object‐oriented program development environment for the first programming course", SIGCSE '96 Proceedings of the twenty‐seventh SIGCSE technical symposium on Computer science education, 83‐87 Lahtinen, E., Ala‐Mutka, K., Jarvinen, H. (2005) “A Study of Difficulties of Novice Programmers” In: Innovation and Technology in Computer Science Education 2005, pp. 14–18. Ljungkvist, P., & Mozelius, P. (2012) ”Educational Games for Self Learning in Introductory Programming Courses‐a Straightforward Design Approach with Progression Mechanisms” , Proceedings Of The 6th European Conference On Games Based Learning, ECGBL 2012 (pp. 285‐293) Lundh, F. (1999) “An Introduction to Tkinter”, (retrieved 15/03/2013) http://www.pythonware.com/library/tkinter/introduction/ Mannila, L. & De Raad, M. (2006) “An Objective Comparison of Languages for Teaching Introductory Programming” Baltic Sea '06 Proceedings of the 6th Baltic Sea conference on computing education research: Koli Calling 2006 Petley, R., Parker G. & Attewell, J. (2011) “The Mobile Learning Network: Getting Serious about Games Technologies for Learning", International Journal of Game‐Based Learning (IJGBL), Volume 1, Issue 4 Piaget, J. (1970) “The Principles of Genetic Epistemology”, Routledge & Kegan Paul Ltd., London Piaget, J. (1973) "To understand is to invent”, Grossman, New York (Original work published 1948) Piaget, J. (1980) “The origin of Intelligence in the Child”, Penguin Education, UK Pillay, H. (2003) “An investigation of cognitive processes engaged in by recreational computer game players: Implications for skills of the future.” Journal of Research on Computing in Education, 34(3) Remenyi, D. (2012) “Case Study Research”, Academic Publishing International Limited, Reading, United Kingdom ISBN: 978‐ 1‐908272‐40‐9 Robocode (2001) “Robocode” http://robocoderepository.com/ (retrieved 04/01/2011) Shabalina, O. & Chickerur S. (2012) "Competence‐based approach for effectively teaching software development curricula", Izvestiya VolgGTU, " Chapter Actual problems of control, cvomputing and informatics in technical systems", Volume 1, Issue 15 (102), p. 102‐111. Shabalina O., Vorobkalov P., Kataev A., Davtyan A. & Blanchfield P (2012) "Development of Computer games for Training Programming Skills", Proceedings of the 6th European Conference on Games Based Learning, Cork, Ireland , 4‐5 October 2012 / The University College Cork And Waterford Institute of technology, Ireland.‐ Cork, 2012.‐ P. 460‐471.

425

Peter Mozelius et al. Sung, K., Panitz, M., Wallace, S., Anderson, R. & Nordlinger, J. (2008) ”Game‐themed programming assignments: the faculty perspective”, 'SIGCSE '08: Proceedings of the 39th SIGCSE technical symposium on Computer science education' , ACM, New York, NY, USA Vujusevic‐Janicic, M. & Tosic, D. (2008) “The Role Of Programming Paradigms In The First Programming Courses” The Teaching Of Mathematics, XI (2) Vygotsky, L.S. (1978) "Mind in society: The development of higher psychological processes." Cambridge, Harvard University Press Wiedenbeck, S., Labelle, D. & Kain, V. N. R. (2004) "Factors affecting course outcomes in introductory programming", 16th Annual Workshop of the Psychology of Programming Interest Group Vygotsky, L.S. (1978) "Mind in society: The development of higher psychological processes”, Harvard University Press, Cambridge Yin, R. K. (1989 ‐ 2008) ”Case study research: Design and Methods”, Sage, Thousand Oaks

426

Towards Understanding the Instructional Value of Real‐Time Continuous Feedback From the use of Simulation Games Mathews Nkhoma, Jaime Calbeto, Narumon Sriratanaviriyakul, Thu Yein Win, Quyen Ha Tran and Thanh Kim Cao RMIT University Vietnam, Ho Chi Minh City, Vietnam mathews.nkhoma@rmit.edu.vn jaime.calbeto@rmit.edu.vn narumon@rmit.edu.vn thuyein.win@rmit.edu.vn quyen.tran@rmit.edu.vn thanh.cao@rmit.edu.vn Abstract: Simulation games have long been used as a teaching tool in the classroom environment mainly due to the high level of participation and engagement that students are able to generate from these, making the learning process more enjoyable and capable to replicate real‐life scenarios. When all is said and done, students are rewarded with a more authentic and complete learning experience. Feedback given during the simulation helps to motivate students to find better solutions to the problems being presented throughout the games and thus enhance their hands‐on knowledge on particular subjects. The purpose of this research is to provide empirical evidence of interrelations and impacts that exist between real‐time continuous feedback and simulation game performance as well as the interrelations and impacts that exist between real‐time continuous feedback and both students’ attitude and engagement towards learning. The research comprised 60 undergraduate students enrolled at the Centre of Commerce who had undergone at least three semesters of studying at various programmes at RMIT University Vietnam. For test purposes, the research employed a 3D IBM Business Process Management (BPM) simulation game, INNOV8 (more information is available at http://www‐ 01.ibm.com/software/solutions/soa/innov8/index.html) developed by IBM Academic Initiative. Students in the sample were asked to play this simulation game. A web‐based survey followed at the conclusion of the simulation game for the collection of data. The findings of the research concluded that students showed a favourable attitude towards learning through the simulation game. In addition, the real‐time continuous feedback given during the simulation game had a positive impact on the students’ cognitive learning outcomes. The originality of this research stems from the nature of the feedback being given to students in a real‐time continuous basis during the gameplay of a computer‐based simulation game to examine how this impacts students’ learning outcomes. Keywords: real‐time continuous feedback, simulation games, game‐based learning, serious games

1. Overview of simulation games Games are a form of entertainment and enjoyment, which have long had a negative connotation due to its richness in storyline, graphics, engagement, and interactivity, with a potential to cause serious addictions leading to health exhaustion in young adults, one case taking place in South Korea (Sutter, 2012). Nevertheless, a significant amount of research has also been carried out to examine the positive impacts in playing computer‐based games, one of which is the educational value (Connolly et al., 2012). A study on game‐ based learning by de Freitas (2006) shows that when there are clear educational objectives and goals, then games can be used effectively as a tool for teaching and learning, particularly Serious Games. This key point is supported by Zyda (2005), saying that Serious Games, with the implementation of theories and principles in the gameplay, break away from the conventional definition of games by adding to it an academic dimension. Michael and Chen (2006) as cited in Susi et al. (2007) reaffirm this pedagogic aspect of Serious Games emphasizing that the outcome of the game‐playing process is to have students actually learn from the game‐ play experience. Simulations games were known as the earliest application of educational games (de Freitas, 2006). With the inclusion of the entertainment aspect, simulation games offer users hands‐on experience from its open‐ended replication of real‐world scenarios and tasks which ask users to tackle the challenges given during the game play and find solutions that can improve the outcomes in the end (Susi et al., 2007; Gredler, 2004). Gredler (2004) raises three reasons why simulation games provide educational value and strengthen students’ learning experience, making it not only a popular teaching tool in academic but also in professional working environments. Firstly, simulation games require players to solve real‐life issues, thus helping to bridge theories with reality; secondly, simulation games help to spot out students’ misunderstanding and confusion about the

427

Mathews Nkhoma et al. abstract theories learnt in class leading to clarification or further explanation; and lastly, simulation games give students guidance to find better solutions when they face the problems in real life.

2. Positive impacts on learning outcomes A lot of researchers have examined the effectiveness of simulation games and how they impact on students’ learning outcomes. Interaction with simulation games provides a student‐centered learning environment because of its active and pragmatic approach to obtaining new knowledge and enhancing skills (Lainema, 2009, Mayer et al., 2010). An experiment carried out by Tan and Seng (2010) using computer games to teach difficult‐to‐understand IT concepts and algorithms was able to discover that students favored learning from interacting with simulation games and that they could comprehend and perceive the abstract concepts better. Ke (2009), on the contrary, failed to find a relationship between students’ engagement and attitudes towards simulation games with the learning outcomes when employing mathematical games to compare the effectiveness of computer games with the traditional teaching method when it came to boosting students’ math learning outcomes. Specifically, findings showed that although students had favorable attitudes towards learning through simulation games, there was no significant difference in the learning outcomes when comparing computer games and the traditional teaching method. The lack of objectives when evaluating the knowledge, skills that players obtain from simulation games and whether these meet the learning outcomes is pointed out by Anderson and Lawton (2007, 2009). Consequently, there is a need to examine and find out how and to what extent students’ attitudes towards simulation games could impact their learning outcomes. From what has been stated, the first hypothesis is proposed: H1: A positive correlation exists between the students’ performance on the simulation game and the students’ attitude and engagement toward learning through the simulation experience. Wouters et al. (2009) proposed a taxonomy of learning outcomes based on the authors’ literature review examining students’ interaction with Serious Games and its impact on learning experience with the following categories: cognitive skills, motor skills, affective skills, and communicative skills; with cognitive skills being studied the most by researchers (Connolly et al., 2012). The process of employing the knowledge and theories learnt in class to tackle problems given during the simulation makes the participants more likely to achieve desired learning outcomes. Anderson and Lawton (2007) conducted a research to examine the connection between students’ performance on the simulation games and their attitudes towards the use of simulation games as a learning tool along with their cognitive learning outcomes. Despite the success of the research, the sample size of only 25 students was too small to find correlations between students’ achievement in the games and their learning outcomes. Furthermore, the simulation game used in Anderson and Lawton’s research is designed in a way that students could easily achieve high results by making guesses without basing answers on knowledge. As a result, it is worth investigating whether a true level of knowledge learnt by the application of theories can be reflected through the students’ performance at the conclusion of the simulation games. In this research, a more feedback‐driven simulation called INNOV8, a 3D IBM Business Process Management (BPM) simulation game was incorporated. In this simulation, players are presented with business model problems and clues on how to solve them. When players make the wrong decisions, feedback and directions are provided to assist them in choosing the right ones in order to move forward with the game. The simulation is designed in a way that students will have to apply the theories they have learnt so as to achieve high scores rather than through guesswork. Furthermore, with 60 students participating in the research, the sample size was larger than the one from Anderson and Lawton’s (2010) making the quantitative findings more representative and accurate to the larger population (Vogt, 2007). From what has been stated, a second hypothesis is proposed:H2: A positive correlation exists between the students’ performance on the simulation game and the students’ perception of how much they learned from the simulation experience.

3. Feedback in simulation games It is emphasized by my researches that an effective framework and design for simulation games should incorporate a supportive feedback system. Garris et al. (2002) identifies the motivation for learning stems from

428

Mathews Nkhoma et al. the feedback provided while interacting with the simulation game, making the flow of the gameplay engaging and challenging and thus encouraging participants to perform better in order to achieve higher scores. Gobet et al. (2004), as cited in Dunwell et al. (2010), and Gredler (2004) also agree with this point saying that feedback motivates players to follow the flow of the game until the end and helps them achieve the desired learning outcomes. There are several justifications for feedback being crucial to the students’ achievement of the intended learning outcomes. Firstly, feedback helps students realize how far they are from their current performance to their desired learning outcomes and thus motivates them to learn harder in order to bridge that gap (Song & Keller, 2001). Secondly, students can be ambiguous when trying to comprehend new concepts, but with the timely guidance given by feedback, it makes students feel more confident and motivated in tackling problems given during the simulation (Paas et al., 2003; Moreno, 2004). Finally, feedback helps fine‐tuning students’ understanding of the new theories they learn in class to the right direction and thus, reducing students’ misconceptions while boosting their learning outcomes and performance (Mason & Bruning, 2001; Goodman et al., 2004; Mory, 2004; Narciss & Huth, 2004). One aspect of feedback that many researchers have vested time in examining is the timing feedback being given and its relationship to the learning experience and efficiency in learning (e.g., Gentry, 1990; Black and William, 1998; Clariana et al., 2000; Dihoff et al., 2004; Schooler et al., 2008; Epstein et al., 2010; Dihoff et al., 2012). Nonetheless, there is still an ongoing debate as to which type of feedback is more effective: immediate feedback which is given instantly after an action is made or delayed feedback which is given only after all the tasks have been completed for some time. Despite the inconsistent findings on feedback timing, it is argued that immediate feedback is more effective in the classroom context than delayed feedback (e.g., Dihoff et al., 2004; Jarvis and De Freitas, 2009; Brosvic et al., 2010). This is mainly because when struggling or when introduced to new theories, students are likely to have misconceptions about ideas introduced from the new lessons and thus less confidently examine problems with the the right understanding in how to solve them. Therefore, immediate feedback will adjust their understandings timely and improve the ability of students to give correct responses after making the initial incorrect ones (Mory, 2004). In this research, INNOV8 was used to examine students’ learning performance. INNOV8 is a simulation game that provides immediate feedback, or as referred in this research as ‘real‐time continuous feedback’. During the simulation, students will be presented with business problems and every time they submit their answers, they will be given feedback throughout the simulation until they reach the conclusion of the game. Consequently, it is essential to examine how using Serious Game in providing real‐time continuous feedback as a teaching method can affect students’ final game performance. From what has been stated, a third hypothesis is proposed: H3: A positive correlation exists between the students’ performance and the students’ perception of how much real‐time continuous feedback they receive from interacting with the simulation experience.

4. Research methodology Quantitative research method was employed in this research. The sample included 60 students who volunteered in teams of two to four players each to play INNOV8, a 3D IBM Business Process Management (BPM) simulation game, for a period of about 30 minutes. Participants were then requested to answer an online survey adapted from Rowe and Wood consisting of twelve closed‐ended questions as well as one optional open‐ended question (Rowe and Wood 2008). After the survey, data was collected and analyzed via SPSS Platform version 20. Linear regression is used to model the value of a dependent variable based on its linear relationship to one or more predictors in order to test the three hypotheses. This technique requires the input of the following quantitative data (as indicated in Table 1): One dependent variable, i.e. perceived performance from playing the game, represented as Measure 1, and various single independent variables (students’ attitude and engagement in learning from playing the simulation game represented as Measure 2; students’ perception of how well the simulation game reflected the disciplines of courses studied at RMIT Vietnam represented as Measure 3; students’ perception of how much real time continuous feedback they are receiving from the

429

Mathews Nkhoma et al. simulation game represented as Measure 4; and students’ preferred form of feedback when interacting with the game represented as Measure 5). The measures mentioned above are multi‐dimensional, thus they need to be grouped by factor analysis. To do so, Cronbach’s alpha coefficient was used to evaluate the scale reliability and also provide information about the relationships between individual items in the scale. Kaiser‐Meyer‐Olkin (KMO) and Bartlett test were used to measure sampling adequacy. Factor analysis was used in data reduction to identify a small number of factors explaining most of the variance been observed in a much larger number of observed variables. These factors were then used for hypothesis testing through regression analysis.

5. Data analysis Cronbach’s alpha coefficient was generated to evaluate the scale’s reliability and validate the consistency of items within the scale. According to Hair et al., the reliability coefficient is considered acceptable if it meets the value of 0.7. The Cronbach’s alpha coefficients shown in Table 1 reveal that the reliability of the scales to be acceptable since all of the coefficients are over 0.7. Table 1: Cronbach’s alpha coefficients Measure / Factor 1 2 3

Description Students’ perceived performance Students’ attitude and engagement toward learning based on 6‐scale Students’ perception of how well the simulation game reflected the discipline of courses studied at RMIT Vietnam

Number of Items 7 6 7

Cronbach’s Alpha .773 .775 .817

Students’ perception towards how much real‐time continuous feedback they received while playing the game

.730

Students’ preferred forms of feedbacks when interacting with the game

.798

Kaiser‐Meyer‐Olkin (KMO) and Bartlett test were used to measure sampling adequacy in order to determine whether the sample size is sufficient for factor analysis. Achieving a KMO value of 0.682 and a Bartlett’s Test Significance value below 0.001 (as seen in Table 2) consents for the dataset to be used for factor analysis. Table 2: KMO and Bartlett’s test Kaiser‐Meyer‐Olkin Measure of Sampling Adequacy.

.682

Bartlett's Test of Sphericity

Sig.

.000

The average number was then used to perform factor analysis. According to Hair et al., “recalling the concept of a summated scale, which is formed by combining several individual variables into a single composite measure” and “more commonly the average score of the variables is used as a replacement variable” because of two specific benefits that this approach provides: (1) "means for overcoming to some extent the measurement error", and (2) “ability to represent the multiple aspects of a concept in a single measure". Therefore, the factor analysis rendered 5 factors:

Factor 1: Students’ perceived performance (Feedback)

Factor 2: Students’ attitude and engagement toward learning based on 6‐scale (Anderson)

Factor 3: Students’ perception of how well the simulation game reflected the discipline of courses studied at RMIT Vietnam

Factor 4: Students’ perception towards how much real‐time continuous feedback they received while playing the game

Factor 5: Students’ preferred forms of feedbacks when interacting with the game

6. Hypothesis testing Five measures in terms of the five factors were used in this study. To drive toward the research outcomes, testing of the three hypotheses would follow the model below:

430

Mathews Nkhoma et al.

Figure 1: Research model Four regression analyses were employed in order to test the hypotheses proposed in this research, with results being summarized in Table 3. The first regression examined the relationship between students’ performance and their attitudes towards learning; the second between students’ performance and their perception of how much they learn; the third between students’ performance and their perception of real‐time continuous feedback received; and the last one between students’ performance and their preferred form of feedback when interacting with the game. Table 3: Regression analyses Hypothesis

Beta t‐value p‐value R ‐square

.327 .383

3.158

.003

.147

.454 .419

3.514

.001

.176

.541 .449

3.794

.000

.202

.317 .274

2.155

.035

.075

All of the four regressions are significant at the p<0.05 level (F‐value = 9.974; 12.350; 14.394; 4.642), confirming the integrity of the models. The coefficients of R square reveals that, ‘Attitude towards learning’ explains 14.7%, ‘Perception of how much they learn’ explains 17.6%, ‘Perception towards how much real‐time continuous feedback provided by interacting with the game’ explains 20.2%, and ‘Preferred forms of feedbacks when interacting with the game’ explains 7.5% in the variation for performance. All predictors are significant in explaining the positive relationships with the dependent variable which is the measure for performance, confirming H1, H2, and H3. As indicated in the regression analysis outcomes in Table 3, there is adequate statistical evidence to prove the fact that all the p‐values are under 0.05 and all the Beta coefficients are positive (0.383 for Measure 2; 0.419 for Measure 3; 0.449 for Measure 4; 0.274 for Measure 5) . Amongst these factors, it can be seen that ‘Perception towards real‐time continuous feedback provided by interacting with the game’ has the biggest positive effect on Performance. Of lesser but significant importance are the ‘Perception of how much they learn’ and the ‘Attitude towards learning’. Least significant is the ‘Preferred forms of feedbacks when interacting with the game’.

7. Discussion This research is carried out with the purpose to give the exploratory evidence of the correlations and the impacts of feedback, specifically real‐time continuous feedback, and simulation games along with the correlations of real‐time continuous feedback on students’ attitude and participation towards learning through the use of simulation game. As mentioned, Anderson and Lawton in 2007 also conducted a similar research but failed to find out the effectiveness of teaching and learning through the use of simulation games, chiefly because of the small sample size of only 25 students and the predictability of the questions employed in their survey, which allowed students to achieve high scores easily through guesswork. With a larger sample size of 60 students and a different simulation game that is more feedback‐driven in which the students are required to apply what they learn in class so as to obtain good performance, the results of this research statistically prove that there is a positive relationship between students’ game performance and their favorable attitudes

431

Mathews Nkhoma et al. towards the use of simulation games as a teaching method. Among 60 participants, 62% say that they had a good time playing the game and 55% state that the game was engaging and stimulating. The study’s results also confirm that students perceive stimulation games as a valuable method to help them improve their learning performance. Moreover, responses from the participants show that when students were presented with business problems during the simulation, they had to apply the theories they had learnt in order to move on to the next level and thus, the theories became clearer to them. With 93% of the respondents stating that they felt motivated to find better ways to tackle the business problems in order to achieve higher scores on their second play, it is a good sign that students’ cognitive learning outcomes can be positively driven by interaction with the learning medium of simulation games. As previously mentioned, the preference for using immediate feedback in the classroom context to encourage and positively influence students’ learning experience as well as findings from this research demonstrating the importance of feedback to students specifically via the incorporation of real‐time continuous feedback, not only confirms the hypotheses proposed in this research but also supports findings from previous studies about the preference of using immediate feedback in the classroom context to encourage students’ learning experience (e.g., Dihoff et al., 2004; Jarvis and De Freitas, 2009; Brosvic et al., 2010) as well as the reasons for the importance of feedback to students as discussed (Locke & Latham, 1990; Song & Keller, 2001; Mason & Bruning, 2001; Paas et al., 2003; Ashford et al., 2003; Mory, 2004; Narciss & Huth, 2004). The hypothesis testing has proven that students’ performance is most positively affected by ‘Students’ perception towards how much real‐time continuous feedback students are receiving from playing the game’. Statistics show that 65% of the respondents felt motivated to study more when they received feedback, and 85% reported that they could achieve better results thanks to the feedback provided. Undoubtedly, feedback was perceived as important by most of the participants and thus, they were obliged to take notice and study all the feedback provided during the gameplay. Furthermore, students’ ambiguity and anxiety during the simulation were lessened thanks to the feedback given which also motivated students to learn more. As a matter of fact, 87% of participants reported that feedback helped them to be aware of their current performance and therefore make better decisions in order to reach higher scores.

8. Conclusion The primary limitation consisted principally in the relatively small sample size. The research outcome is based on data collected from a total of 60 students participating in this study, and therefore the research outcomes might not be a true representative sample of the total student population studying at RMIT University Vietnam. A second limitation, also involving the demographics of the sample, consisted in permitting students from a variety of majors to take part in this research. Since this research did not use the game to test the knowledge learnt in the classroom but rather exposed them to a totally unfamiliar environment, perhaps, the results could be more significant if students who played the game had taken courses related to Business Process Management. Understanding with more precision the value that students can gain from the application of real‐time continuous feedback in the classroom, specifically in the form of computer‐based simulation games, requires more exploration. A follow‐up project to this research could incorporate a more defined sample of students enrolled in a Business Process Management course. From findings uncovered from this type of sample, we can then assess more clearly the value gained by students from real‐time continuous feedback toward better comprehension of the theory of the course content and hence the achievement of the learning outcomes in a course that has more to do with Business Process Management.

References Anderson, P. H. & Lawton, L. 2007. Simulation Performance and Its Effectiveness as A PBL Problem: A Follow‐up Study. Developments in Business Simulation and Experiential Learning, 34. Anderson, P. H. & Lawton, L. 2009. Business Simulations and Cognitive Learning : Developments, Desires and Future Directions. Simulation & Gaming, 40, 193‐216. Black, P. & Wiliam, D. 1998.Assessment and classroom learning.Assessment in education, 5, 7‐74. Brosvic, G. M., Epstein, M. L., Dihoff, R. E. & Cook, M. J. 2010. Acquisition and retention of Esperanto: The case for error correction and immediate feedback. The Psychological Record, 56, 4. Clariana, R. B. 1999. Differential Memory Effects for Immediate and Delayed Feedback: A Delta Rule Explanation of Feedback Timing Effects

432

Mathews Nkhoma et al. Connolly, T. M., Boyle, E. A., Macarthur, E., Hainey, T. & Boyle, J. M. 2012.A systematic literature review of empirical evidence on computer games and serious games.Computers & Education, 59, 661‐686. De Freitas, S. 2006. Learning in immersive worlds.London: Joint Information Systems Committee. Dihoff, R. E., Brosvic, G. M. & Epstein, M. L. 2012. The role of feedback during academic testing: The delay retention effect revisited. The Psychological Record, 53, 2. Dihoff, R. E., Brosvic, G. M., Epstein, M. L. & Cook, M. J. 2004. Provision of feedback during preparation for academic testing: Learning is enhanced by immediate but not delayed feedback. Psychological Record, 54, 207‐232. Epstein, B. B. & Brosvic, G. M. 2010. Immediate feedback assessment technique promotes learning and corrects inaccurate first responses. The Psychological Record, 52, 5. Garris, R., Ahlers, R. & Driskell, J. E. 2002. Games, Motivation, and Learning: A Research and Practice Model. Simulation & Gaming, 33, 441‐467 Gentry, J. W. 1990. What is Experiential Learning.Guide to business gaming and experiential learning, 9‐20. Goodman, J. S., Wood, R. E. & Hendrickx, M. 2004. Feedback specificity, exploration, and learning.Journal of Applied Psychology; Journal of Applied Psychology, 89, 248. Gredler, M. E. 2004. Games and Simulations and Their Relationships to Learning.Handbook of research on educational communications and technology, 2, 571‐581 Hair, J.F., Black, W.C., Babin, B.J., Anderson, R.E. 2010.Multivariate data analysis 7th edition. Upper Saddle River, NJ : Prentice Hall. Ke, F. 2008. Computer games application within alternative classroom goal structures: cognitive, metacognitive, and affective evaluation. Educational Technology Research and Development, 56, 539‐556. Lainema, T. 2009. Perspective Making Constructivism as a Maning‐Making Structure for Simulation Gaming. Simulation & Gaming, 40, 48‐67. Mason, B. J. & Bruning, R. 2001. Providing feedback in computer‐based instruction: What the research tells us. 15. Available: http://dwb.unl.edu/Edit/MB/MasonBruning.html [Accessed 15 November 2012]. Mayer, B. W., Dale, K. M., Fraccastoro, K. A. & Moss, G. 2010.Improving Transfer of Learning: Relationship to Methods of Using Business Simulation.Simulation & Gaming, 42, 64‐84. Moreno, R. 2004. Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discovery‐based multimedia.Instructional science, 32, 99‐113. Mory, E. H. 2004. Feedback research revisited. Handbook of research on educational communications and technology, 745‐ 783. Narciss, S. & Huth, K. 2004.How to design informative tutoring feedback for multimedia learning.Instructional design for multimedia learning, 181‐195. Paas, F., Renkl, A. & Sweller, J. 2003. Cognitive load theory and instructional design: Recent developments. Educational psychologist, 38, 1‐4. Rowe, A. D. & Wood, L. N. 2008.Student Perceptions and Preferences for Feedback.Asian Social Science, 4, 78‐88. Schooler, L. J. & Anderson, J. R. 2008.The disruptive potential of immediate feedback. Shute, V. J. 2008. Focus on formative feedback. Review of educational research, 78, 153‐189. Song, S. H. & Keller, J. M. 2001. Effectiveness of motivationally adaptive computer‐assisted instruction on the dynamic aspects of motivation.Educational technology research and development, 49, 5‐22. Susi, T., Johannesson, M. & Backlund, P. 2007. Serious Games ‐ An Overview. Sutter, J. D. 2012. Wired for success or destruction? [Online].CNN. Available: http://edition.cnn.com/interactive/2012/08/tech/gaming.series/korea.html [Accessed 15 November 2012]. Tan, B. & Seng, J. L. K. 2010. Game‐based Learning for Data Structures: A Case Study. International Conference on Computer Engineering and Technology.IEEE. Vogt, W. P. 2007. Quantitative research methods for professionals, Pearson/Allyn and Bacon. Wouters, P., Van Der Spek, E. & Van Oostendorp, H. 2009. Current practices in serious game research: A review from a learning outcomes perspective. Games‐based learning advancements for multi‐sensory human computer interfaces: techniques and effective practices, 232‐250. Zyda, M. 2005. From visual simulation to virtual reality to games.Computer, 38, 25‐32.

433

Learning Math as you Play: Comparing Arithmetic Performance Enhancement Induced by Game Play and Paper Exercises Elena Patricia Nuñez Castellar, Anissa All and Jan Van Looy Department of Communication Sciences, iMinds‐MICT‐Ghent University, Belgium ElenaPatricia.NunezCastellar@Ugent.be Anissa.All@Ugent.be J.Vanlooy@Ugent.be Abstract: One of the promises of video game training is that, compared to traditional training, it can be more engaging and entertaining (Boot et.al., 2008). However, besides entertainment, games have shown to have the potential to impact a larger variety of cognitive abilities. Previous research has consistently shown that several aspects in cognition such as visual short‐memory, multitasking and spatial cognition can be enhanced by game play. In a previous study, we found that playing Monkey Tales, a commercial game aimed at training arithmetic skills in children, helped second grade pupils to increase their accuracy in mental calculation as compared to paper exercises or no exercises. The present study aimed to explore how arithmetic performance enhancement induced by game play and paper exercises differs. In order to do this, we compared the performance gains that second graders achieved in a computer test made for assessing their math skills. We performed a combined analysis of the changes in two behavioral measurements: accuracy and reaction times. Children were tested at two points in time: before and after the three week period. We compared the reaction times and the accuracy improvements between these two moments and compared different items types (e.g. understanding tenths, understanding hundreds, even or odd up to 100 among other types). We found indirect evidence suggesting that arithmetic performance enhancement induced by game play and paper exercises might rely on slightly different cognitive mechanisms. Keywords: arithmetic training, mental calculation, educational game, traditional training, reaction times, accuracy

1. Introduction Video games are one of the more interesting and promising means to improve cognitive abilities, particularly with children. One of its promises is that, compared to traditional training, can be more engaging and entertaining (Boot et al., 2008), and effective (Wouters et al., 2013 ). Moreover, besides entertainment, games have the potential to impact a larger variety of cognitive abilities. Recently, research has consistently shown that several aspects in cognition such as visual short‐memory, multitasking and spatial cognition can be enhanced by game play (for a complete review, see Bavelier et al., 2012). In a previous study (Nuñez Castellar et al., submitted), we reported that playing Monkey Tales, a commercial game aimed at training arithmetic skills in children, helped second grade pupils to increase their accuracy in mental calculation as compared to paper exercises or no exercises.. However, the extent to which the positive changes induced by gaming or by paper exercises differ in its nature and characteristics is an issue that has not yet been explored. Specifically, based on previous research showing that video game playing can enhance working memory capacities and attention (Bavelier et al., 2012), in the present paper we explore whether by a detailed, combined analysis of the changes in accuracy and reaction times after game training and traditional training by means of math paper exercises, we can provide more informed description of how arithmetic performance enhancement induced by these two methods might differ. Specifically, there are reasons to believe that arithmetic performance enhancement induced by game play might be modulated by improvements in the domains of attention and working memory. Working memory is the ability to explicitly maintain a mental representation of some amount of information, while being engaged simultaneously in other mental processes (Baddeley, 2000). Research has demonstrated that working memory capacity increases from preschool through the elementary school years. Preschool children can hold three to four items of information, such as numbers, in working memory, whereas a typical fourth grader can hold five to six items (Kail, 1990). Although during the past decades it was traditionally assumed that working memory is highly heritable and unlikely to be influenced by environmental experience and opportunity (see Campbell et al., 1997), recent findings have provided evidence suggesting that children’s working memory can be enhanced by means of training (Klingberg et al., 2005; Holmes, Gathercole and Dunning, 2009; Turley‐Ames and Whitfield, 2004).

434

Elena Patricia Nuñez Castellar, Anissa All and Jan Van Looy Remarkably the study of Holmes, Gathercole and Dunning (2009) has demonstrated that attention training can lead to a significant boost of the academic mathematics performance of children. This study showed that IQ scores (both verbal IQ and performance IQ scores) did not show a comparable boost after working memory training, suggesting that, rather than leading to global performance enhancement, improvements in working memory seem to act locally, boosting arithmetical performance. Moreover, studies with clinical populations indicate the existence of a close relationship between working memory capacities and mathematical skills. For instance, studies investigating children with a mathematics learning disability(MD), have shown that they receive diminished scores on a variety of working memory tasks when compared with their same age pairs (Hitch and McAuley, 1991; McLean and Hitch, 1999; Siegel and Ryan, 1989; Swanson, 1993). Furthermore, recent studies have reported evidence suggesting that working memory and attention can be trained in normal adults by means of video gaming. For instance, it has been found that video game players are faster and more accurate in the monitoring and updating of working memory than non‐ video game players (Colzato et al., 2012). Green and Bavelier (2003) conducted a series of experiments on the effects of video game playing on visual attention comparing action video game players and non‐video game players, and found that video game playing experience enhances the capacity of the players’ visual attention system. Likewise several recent studies have demonstrated that action video game players have the ability to switch faster between tasks compared with non‐video game players (Karle, Water and Shedden, 2010, and Boot et al., 2008). Finally, a recent study has shown that performance gains are not restricted to the action game genre, but that playing Tetris, a casual puzzle game, can also improve attention, working memory and visuo‐spatial ability in young adults (Nuchi et al., 2013). Taken together, the results mentioned above suggest the existence of a close link between, working memory, attention and arithmetic skills, and that, remarkably, these cognitive abilities can be trained by means of game play. This creates important opportunities for using games for mathematics training but also questions as to how these different performance gains are related and how they compare with traditional methods for practicing mathematics. Hence, in the present study, we explore whether traditional methods and game training differ in terms of the cognitive processes that both are able to impact. In order to do this, we compared the results that second graders achieved in a test made for assessing their math skills. We conducted a combined analysis of the changes in accuracy and reaction times whilst considering different item types (e.g. understanding tenths, understanding hundreds, even or odd up to 100, etc.). Moreover, we explored whether the type of item that showed the largest improvements differed between game training and the traditional training group. Finally, results are compared with the ones of a group that did not receive any assignment (control group).

2. Methods 2.1 Participants Participants were drawn from a previous study. Overall arithmetic performance in a math test, as well as subjective measures like math anxiety, enjoyment and perceived competence from this sample have already been reported (Nuñez Castellar et al., submitted). However, that report did not examine individual reaction times and accuracy rates per type of arithmetical problem or item type. Children were recruited by sending letters to schools in the area of Ghent, Belgium. The parents interested in participating, registered via the Computer‐Aided Registration Tool for Experiments (CORTEX) (Elson and Bente, 2009). Parents gave written informed consent for their child’s participation. Children were tested at two points in time: before and after the three‐week period (Pre‐ and Post‐testing). During the first evaluation 88 second graders were tested. At the second moment of measurement, 84 were assessed (one child could not participate because of illness and three parents did not react to the repeated calls for post‐testing). From this sample some participants were excluded from the analyses because they were clinically diagnosed with disorders listed in the Diagnostic and Statistical manual of Mental disorders (DSM‐IV) (American Psychiatric Association, 2000), namely learning disability, ADHD, and dyslexia. Also participants who could not complete the task assignment and all participants who performed the computer math test at chance level or below, either in the pre‐ or the post‐test, were excluded from the analyses. In the present study data of 74 children are reported. As can be seen in Table 1, the groups did not differ significantly in terms of age, gender or game and study habits (see Table 1).

435

Elena Patricia Nuñez Castellar, Anissa All and Jan Van Looy Table 1: Socio‐demographic data and study and game habits by group

Educational Game (N=25)

Paper Exercises (N=23)

Control (N=26)

Male gender Age

n 18 Mean 7.52

n 15 Mean 7.26

n 18 Mean 7.35

Chi² 0.26 F 1.33

p .88 p .27

Median 4 4

Chi² 1.30 4.51

p** .52 .10

Mean 2h 02min 0h 53min 3h 41min

Mean 2h 10min 1h 08min 3h 42min

Mean 1h 30min 1h 05min 3h 11min

F 0.87 0.49 0.48

p .42 .61 .62

2h 48min

2h 23min

2h 21min

0.53

.59

Level education parents Education level father* Education level mother* Study and game habits Homework hours per week Math homework hours per week Gaming hours during the week Gaming hours during the weekend

*Four levels: Primary = 1, Junior High School/Middle School = 2, High School = 3, College/University = 4. ** Independent Sample Kruskall‐Wallis Test

2.2 Design Children were randomly assigned to three groups. One group was instructed to play through the entire educational game Monkey Tales in three weeks’ time (gaming group). A second group was instructed to complete a set of math drill exercises in the same period, equivalent in quantity and basic level of difficulty to the exercises in Monkey Tales (paper exercises group). Additionally we included a group that did not receive any assignment (control group).

2.3 Stimulus material 2.3.1 Educational game We used the 3D video game Monkey Tales (Larian studios, 2011), which exists in different versions for second to sixth grade and is used to support the learning of math. The main goal of this educational game is not to instruct but to improve mental arithmetic of children by motivating them to engage in drill exercises with increasing time pressure. Only by being faster than a monkey (artificial intelligence) they can go through all the game levels. Importantly, the game uses an algorithm that tries to establish where a child is on the learning curve, and then stimulates the child to make progress by progressively augmenting the difficulty of the exercises. For the present study we selected the Museum of Anything, which is meant for children in the 3rd grade (ages 8+) to repeat what they have learned in the 2nd. The educational game is divided into chapters and levels in which the player has to solve 3D puzzles (moving something that blocks the way or neutralize a laser for instance) and is challenged by a monkey to take part in a minigame (an educational math exercise in classic game format, e.g. 2D shoot ‘em up) which the player has to win to get to the next level (see figure 1). The game contains 42 basic and one final level. In order to complete all the levels of the game, children need to finish 322 math exercises whereby the exact number depends on how many times they need to replay a minigame because of mistakes.

436

Elena Patricia Nuñez Castellar, Anissa All and Jan Van Looy

Figure 1: Screenshot Monkey Tales 2.3.2 Paper exercises As one of the goals of the present study was to compare Monkey Tales with paper exercises, the latter needed to be as similar as possible to the former. Therefore, the educational publisher of the game, Die Keure, provided us with exercises based on their educational method that are equivalent in basic level of difficulty to the exercises included in Monkey Tales. Of around 1000 exercises we received, a sample of 340 exercises that were representative of the Belgium math curriculum for second graders were selected. The exercises were organized in ascendant order of difficulty (as it is done in the educational game), and were given to the parents of the children in a folder that they gave back to us at the post‐test.

2.4 Measurement of math performance: accuracy and speed Two equivalent versions of exams (test A and test B) for assessing the math skills of children of the second grade were provided by publisher Die Keure. These test were based on the academic curriculum for second grade in Belgium. We used the questions of these two tests to program a computerized version that allowed us to automatically measure not only the accuracy rates but also the reaction times of each item of the test in milliseconds. We programmed this computerized version using Tscope. Tscope is a C/C++ experiment programming library for cognitive scientists. It provides functions for graphics, sound, timing, randomization and response registration (Stevens et al., 2006). After a number of practice trials to become familiar with the multiple choice task, all children performed the computer math test in the pre‐ and post‐session. In each group, half of the children performed test A as pre‐test measurement and test B as post‐test measurement. The other half performed the tests in the opposite order.

2.5 Procedure The participants were tested at the beginning of May 2012 for the pre‐test session and at the end of May for the post‐test session. As previously described, children were randomly assigned to three groups. One group of children was instructed to finish the educational game Monkey Tales in three weeks’ time (Gaming group). Parents were instructed to help with the software installation and support the children while playing the game tutorial. However, they were explicitly asked not to help children with the math exercises. Moreover, the parents were asked to monitor on a weekly basis how far the children had progressed in the game, and to motivate them to play if needed. Importantly they were briefed about how to check the progress and detect when children had completed all the levels of the game. Finally, one week before the post‐test an e‐mail was sent as a reminder that, by the end of the week, the children should have completed the game. A second group of children was instructed to complete a set of math drill exercises on paper in three weeks’ time of equivalent quantity and basic level of difficulty as the exercises in Monkey Tales (Paper exercises group). Similarly to the parents of the group of children that was asked to play the educational game, the parents of this group were instructed to check and motivate the children to do the math drill exercises, but not to help them. One week before the post‐test, an e‐mail was sent as a reminder that by the end of the week children should have completed the math drill exercises. Additionally, we included a group that did not receive any assignment (Control group) but served as a comparison. The parents of this group were asked not to change anything from their normal routine and specifically they were asked not to let their children play any educational math games.

437

Elena Patricia Nuñez Castellar, Anissa All and Jan Van Looy All parents received the instruction to let the children continue to do their math homework as usual. The group that completed the paper exercises and the control group received the educational game at the end of the post‐test as a reward. All the parents were rewarded with 15 euros for their participation.

2.6 Data analysis 2.6.1 Accuracy improvements per item type In order to investigate whether the items in which the largest improvement was observed differed between groups (e.g. understanding tenths, understanding hundreds, even or odd up to 100, divide into equal parts, multiplication tables from 2 up to 10, we compared the accuracy in the pre‐test and the post‐test measurements for each item type. All the items of the Math test were included in the analyses except one for which in the pre‐test session 100% of the children gave a correct answer and therefore no improvement was possible. Afterwards we made a ranking based on the percentage of improvement including all the items to identify the ones in which the largest accuracy improvement was registered. 2.6.2 Correlation between accuracy and reaction time improvements Similarly to the accuracy improvements analyses, we compared the reaction times in the pre‐test and the post‐ test measurements for each item type. Only reaction times for correct responses were included in the analyses. Additionally, too fast reaction times ( < 300ms) and reaction times slower than 60 seconds were excluded. After calculating the reaction time improvements, we investigated the relationship between the accuracy improvements and the reaction time improvements. Therefore we performed a correlational analysis separate for each group: Monkey Tales, paper exercises and control. A significance level of 0.05 was used.

3. Results 3.1 Accuracy improvements Figure 2 shows the accuracy improvements per item type ordered by percentage of improvement for the three groups whereby the improvement of the control group can be considered as a baseline to interpret the improvement of the two experimental groups. As can be observed in the figure, when considering only the items ranked in the first positions, the graph shows that the largest accuracy improvement is observed for the group that made the paper exercises. For instance, the first ranked items on that group show an accuracy improvement above 30%. However, interestingly, when considering the pattern more globally, the results show that overall playing Monkey Tales lead to a larger sustained accuracy improvement in most of the items included in the math test. When we look at the number of items scoring accuracy improvements of 6% and above, we see that there are 18 for the Monkey Tales group as compared to only 13 in the math exercises group.

Figure 2: Accuracy improvements per item type

438

Elena Patricia Nuñez Castellar, Anissa All and Jan Van Looy

3.2 Correlation between accuracy and reaction time improvements Since not only accuracy but also reaction times can be analyzed to investigate performance enhancement we conducted correlation analysis between these two behavioral measures. The results revealed that for the group that played monkey tales, accuracy and reaction time improvements were strongly correlated r(37) = .46, p < .001. Likewise, for the group that made the paper exercises, reaction time improvements were correlated r(37) = .33, p < .05, but less strongly than for the game group. Additionally, when analyzing the relationship between accuracy and reaction time improvements for the control group, the association between these two measurements was found to be non‐significant r(37) = .08, p = 61. Based on the correlation analysis we selected for each group the 5 items in which the largest accuracy and reaction time improvement were observed. The results of this ranking can be found in table 1. As can be observed, there are important differences between the types of items that are ranked for each of the three groups. The results revealed that doing the paper exercises predominantly lead to improvements on items where second graders were required to solve addition problems. Playing Monkey Tales, on the other hand, lead to improvement in a variety of items including items that require second graders to make parity judgments and rehearse the multiplication tables of 7 and 8. The graph also shows the ranking for the group that got no assignment (control group). Table 2: Ranking items with the largest accuracy and reaction time improvements T = tens and U = units Ranking

MONKEY TALES Even or odd up to 100

PAPER EXERCISES CONTROL Divide into equal parts Multiplication table 7 (e.g. Find the correct number: (e.g. select the odd number) 12= . + . + . ) (e.g. 7 x 5) Tens and units Addition TU+TU up to 100 with Addition TU+TU up to 100 regrouping with regrouping (e.g. Which number has 7 (e.g. 55 + 29 = ) (e.g. 55 + 29 = ) units?) Multiplication table 8 Addition TU+TU up to 100 with Subtraction with 1 multiple of regrouping 10 up to 100 (e.g. 8 x 8) (e.g. 55 + 29 = ) (e.g. 86 – 10 = ) Multiplication table 7 Addition TU+TU up to 100 with Subtraction TU+TU = T regrouping (e.g. 7 x 5) (e.g. 55 + 29 = ) (e.g. 45 + 25 = ) Subtraction with up to 3 Subtraction TU‐U up to 100 Multiplication table 6 multiples of 10 up to 100 with regrouping (e.g. 75 – 30 = ) (e.g. 63 – 6 = ) (e.g. 6 x 5)

4. Discussion and conclusions In spite of the fact that the present study was explorative in nature, our results point to three interesting findings. First, our accuracy improvement analysis showed that, when considering only the items ranked in the top positions, the largest accuracy improvement can be observed in the group that made the paper exercises. Playing Monkey Tales, however, lead to a larger sustained accuracy improvement in terms of number of items included in the math test. In other words, rather than acting locally (leading to the improvement in few item types), playing the game had a global impact on the accuracy performance in a large variety of items. Although these results should be interpreted with caution given the exploratory nature of this study, an interesting venue for future research would be to investigate whether this global impact might be the result of secondary effects of the arithmetical training in other cognitive domains like working memory and attention. Since the content of the paper exercises and the game exercises were carefully matched ‐ not only regarding content but also in number ‐ it is unlikely that the differences we observed between groups relies on the type of arithmetical exercises. Rather, we suggest that there might be other mediating factors associated with game play that could better explain this pattern. For instance, an increased capacity to hold information items, such as numbers, in working memory could explain the broader positive impact for the game group in terms of variety of arithmetical problems for which improvement occurred.

439

Elena Patricia Nuñez Castellar, Anissa All and Jan Van Looy Secondly, the correlation analysis performed between accuracy and reaction time improvements, revealed that, although for both groups, Monkey Tales and paper exercise, the correlation between these two behavioral markers was significant, this association was stronger for the gaming group. Moreover, this relationship was absent for the children who only went to school and did not receive any kind of repetition exercises during the three‐week period. This is an important finding considering that, in order to be effective, a game aimed to train mathematical skills in children would be expected to have a positive impact for both behavioral measurements. The present study shows that, similar to the traditional method of paper math drills, Monkey Tales can lead to performance enhancement by making children perform faster and more accurate in mental calculations. Thirdly, a ranking of the five items in which the largest accuracy and reaction time improvement were observed, revealed that doing the paper exercises lead to improvements predominantly on items where children were required to solve addition problems while playing Monkey Tales lead to improvement in a broad range of item types. These findings provide indirect evidence for the idea that arithmetic performance enhancement induced by game play and paper exercises might rely on slightly different mechanisms. For instance, previous research in the field of cognitive psychology has shown that information about the parity of numbers is associated with their arabic representation and directly retrieved from long‐term memory when needed (Dehaene, Bossini, and Giraux, 1993). Thus, the parity judgment task involves memory retrieval as well as the selection of the response induced by the retrieved information. Interestingly, it was a parity judgment item that showed the largest accuracy improvement after playing through Monkey Tales. This suggests that videogame training boosted cognitive processes like memory retrieval and response selection ‐ according to Miyake (2000) one of the three major cognitive control functions ‐ being both crucial for parity judgment. Whilst listing the cognitive processes and the working memory load involved in the arithmetical problems reported in the present study is not at the core of this study, the previous example shows the utility of this kind of exhaustive analysis. Since previous research has shown that arithmetic, and more specifically numerical calculation, involves the use of working memory (i.e., keeping information available in the cognitive system) and attention (Rubinsten and Henik., 2009), we are convinced that detailed analyses of improvement considering different categories of arithmetical problems can be informative about the underlying mechanisms of performance enhancement. Finally, some limitations of the present study have to be acknowledged. Given its explorative nature, it provides a very first examination about how, in terms of cognitive processes, arithmetic performance enhancement induced by game play and paper exercises might be different, but also that there are several ways this study could be improved. First of all, our results strongly suggest that learning mathematics through game play might be mediated by improvements in the use of working memory resources and attention. This partly remains speculative, however, because we did not apply tests that measured working memory and attention directly and thus have only presented indirect evidence. Future research could incorporate such standardized measurements to further investigate this issue. Additionally, rather than aiming to be conclusive, the present study explored whether we could find indications that performance gains differ between traditional learning methods and game training. Consequently, several questions could be now experimentally investigated, for instance the extent to which are results are generalizable to other games and whether similar positive effects can be found with children of different ages. Finally, future research could investigate the impact of arithmetical training through games for children disadvantaged by learning difficulties or socioeconomic status.

Acknowledgements This work was performed in the context of the iMinds‐ICON Games at School project, which is cofunded by iMinds, a research institute founded by the Flemish Government. We thank Vicky Vermeulen and Nic Pappijn (Die Keure) for their support with the materials and information about the educational game.

References Association, A.P., 2000. Diagnostic and statistical manual of mental disorders: DSM‐IV‐TR: American Psychiatric Publishing, Inc. Baddeley, A., 2000. The episodic buffer: a new component of working memory? Trends in Cognitive Sciences, 4, 417‐423. Bavelier, D., Green, C.S., Pouget, A. & Schrater, P., 2012. Brain Plasticity Through the Life Span: Learning to Learn and Action Video Games. In S.E. Hyman (ed.) Annual Review of Neuroscience, Vol 35. 391‐416.

440

Elena Patricia Nuñez Castellar, Anissa All and Jan Van Looy Boot, W.R., Kramer, A.F., Simons, D.J., Fabiani, M. & Gratton, G., 2008. The effects of video game playing on attention, memory, and executive control. Acta Psychologica, 129, 387‐398. Campbell, T., Dollaghan, C., Needleman, H. & Janosky, J., 1997. Reducing bias in language assessment: Processing‐ dependent measures. Journal of Speech Language and Hearing Research, 40, 519‐525. Colzato, L.S., Van Den Wildenberg, W.P., Zmigrod, S. & Hommel, B., 2012. Action video gaming and cognitive control: playing first person shooter games is associated with improvement in working memory but not action inhibition. Psychological research, 1‐6. Dehaene, S., Bossini, S. & Giraux, P., 1993. The mental representation of parity and number magnitude. Journal of Experimental Psychology General, 122, 371‐371. Elson, M., & Bente, G, 2009. CORTEX ‐ Computer‐Aided Registration Tool for Experiments [online]. University of Cologne. Available from: http://cortex.uni‐koeln.de [Accessed Access Date Green, C.S. & Bavelier, D., 2003. Action video game modifies visual selective attention. Nature, 423, 534‐537. Holmes, J., Gathercole, S.E. & Dunning, D.L., 2009. Adaptive training leads to sustained enhancement of poor working memory in children. Developmental Science, 12, F9‐F15. Kail, R. & Park, Y.S., 1990. Impact of practice on speed of mental rotation. Journal of Experimental Child Psychology, 49, 227‐244. Klingberg, T., Fernell, E., Olesen, P.J., Johnson, M., Gustafsson, P., Dahlstrom, K., Gillberg, C.G., Forssberg, H. & Westerberg, H., 2005. Computerized training of working memory in children with ADHD ‐ A randomized, controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 177‐186. Mclean, J.F. & Hitch, G.J., 1999. Working memory impairments in children with specific arithmetic learning difficulties. Journal of Experimental Child Psychology, 74, 240‐260. Miyake, A., Friedman, N.P., Emerson, M.J., Witzki, A.H., Howerter, A. & Wager, T.D., 2000. The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive psychology, 41, 49‐100. Siegel, L.S. & Ryan, E.B., 1989. The development of working memory in normally achieving and subtypes of learning disabled children. Child development, 973‐980. Stevens, M., Lammertyn, J., Verbruggen, F. & Vandierendonck, A., 2006. Tscope: AC library for programming cognitive experiments on the MS Windows platform. Behavior Research Methods, 38, 280‐286. Studios, L., 2011. Monkey Tales. Gent, Belgium: Die Keure Larian Studios. Swanson, H.L., 1993. Working memory in learning disability subgroups. Journal of experimental child psychology, 56, 87. Turley‐Ames, K.J. & Whitfield, M.M., 2003. Strategy training and working memory task performance. Journal of Memory and Language, 49, 446‐468. Wouters, P., Van Nimwegen, C., Van Oostendorp, H. & Van Der Spek, E.D., 2013. A meta‐analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105, 249‐265.

441

Serious Game Adaptive Learning Systems Chinedu Obikwelu and Janet Read University of Central Lancashire, UK coobikwelu@uclan.ac.uk JCRead@uclan.ac.uk Abstract: Serious games have evolved from the traditional one‐size‐fits‐all mode to a Dynamic Difficulty Adjustment (DDA) mode. DDA is an individualized approach that is based on the principle of adaptivity. For serious games, there is an emphasis on adapting hints and feedbacks according to the changing learner’s competence by using set rules. Adaptive Learning Systems (ALS) are sometimes referred to as Personalised Learning Systems (PLS).According to Karagiannidis and Sampson every PLS should answer the following questions ‐ What is the learning content being adapted? Which aspects of the learning experience ‘drive’ adaptations? What is the basis for adaptation? These questions make up the adaptation logic which differs from game to game and from model to model. This paper reviews the current literature by investigating different adaptation logics embodied in existing serious game Adaptive Learning Systems. These different Adaptive Learning Systems have been adopted by and proposed for serious games. The Adaptive Learning Systems investigated in this paper include the ‘NUCLEO’ framework which emphasizes collaboration in a Multi‐User Virtual Environment (MUVE) with role assignment and team formation adapted to learners; S.M.I.L.E which is more of an accessibility model accommodating player‐learner’s with handicaps by adapting quests with the uniqueness of allowing teachers to define educational games with stored educational materials; ‘Framework for Adaptive Game Presenters with Emotions and Social Comments’ which adapts emotions and social feedback; the Fine‐Tuning System (FTS) which is based on adaptive fading ‐ adapting scaffolds including feedbacks and hints based on the fading principle; and ALIGN which separates the game logic from the adaptation logic by creating reusable adaptation abstractions. The trend in assessment generation which is drifting from the traditional After Action Review (AAR) to assessment generation for adaptive interventions is also highlighted in this paper. Keywords: serious game; learning; adaptivity; personalised learning; feedback; adaptive learning system; adaptation logic

1. Introduction A possible means for increasing the effectiveness and enjoyment of an educational game is the introduction of adaptivity, allowing a game to be customized for a specific player based on an assessment of their state of learning or other characteristic(Mehm et al., 2012) these characteristics could include motivational states, gaming preferences and psycho‐pedagogical implications (M.D Kickmeier‐Rust & Albert, 2010). In current forms of technology‐enhanced learning, concepts of adaptivity, adaptability and personalization have increasingly become important (M.D Kickmeier‐Rust & Albert, 2010). User‐adaptation in e‐learning can be characterized as the ability of a system to personalize the learning experience to different individual conditions over time (Sancho, Moreno‐Ger, Ruben, & Baltasar, 2009). An interesting concept is Dynamic Difficulty Adjustment (DDA) which (Bailey & Katchabaw, 2005) described as the ability of a game to automatically adapt the difficulty level of gameplay to match the skills and tolerances of a player. In their work, they specified that gameplay including player character attributes; non‐player character (NPC) attributes; Game World and level attributes; puzzle and obstacle attributes could all be made adjustable. The success of adaptation is addressed at two distinct layers – the user modelling and adaptation decision‐ making layers (Brusilovsky & Sampson, 2004). The primary task for game‐based adaptive educational mechanisms is to guide and support the learner in acquiring knowledge by, for example, informing the learner, intervening when misconceptions occur or when the learning progress is unsatisfactory, and hinting or providing the learner with appropriate feedback (Kickmeier‐Rust & Albert, 2010). In addition tasks are required to be motivating, maintaining immersion and personalizing the game according to the preferences and needs of the learner (Kickmeier‐Rust & Albert, 2010). Accomplishing this goal requires a theoretical and technological approach that enables the game to assess cognitive states (e.g. competence states or motivational states), learning progress, possible misconceptions or undirected/ unsuccessful problem‐solving strategies (Kickmeier‐ Rust & Albert, 2010).

2. Stages in adaptation in serious games (Ismailovic, Haladjian, Kohler, Pagano, & Brugge, 2012) describe adaptivity in serious games as follows ‐ Adaptivity in serious games is an approach that enables a serious game to (A) learn from learner’s behaviour by (A1) intelligent monitoring and (A2) interpreting learner’s actions in the game’s world and (B) to intervene

442

Chinedu Obikwelu and Janet Read in the game’s world by (B1) automatically adjusting the learning content and (B2) adjusting the game elements according to (C) the students individual ZPD and using the principles of (D) the More Knowledgeable Other ‐ MKO, where ADAPTIVITY is a MKO for the learner according to the Social Development Theory. “According to this definition, the adaptivity process in a serious game consists of four stages:

Monitoring players (A1)

Learner characterization (A2)

Assessment generation (B1)

Adaptive intervention (B2)” (Ismailovic et al., 2012)

These adaptation stages are discussed in further detail in the following sections.

2.1 Monitoring players The proposed NUCLEO framework which is deeply rooted in socio‐constructive pedagogical theories (Sancho et al., 2009) emphasizes intelligent monitoring in a Multi‐User Virtual Environment. In this framework students are classified into four different types in relation to their attitude to learning Vermunt’s Meaning Directed (MD), Application Directed (AD), Reproduction Directed (RD), and Undirected (U) describe the types. Each player is given a role in relation to their attitude to learning ‐ based on the aforementioned student types (Vermunt, 1987). According to (Sancho et al., 2009) this classification of learners help to distinguish the students who need more intensive guidance through the learning process from those who are more capable of guiding their own learning experience.

2.2 Learner characterization This is often referred to as the user model. Game logs recording the players’ performance are used to create models of players’ actions, preferences or personality (Lopes & Bidarra, 2011). Given a game state, these models assess and predict the player’s desired experience for the next game state (Lopes & Bidarra, 2011). Models for the player experience and performance are then used to steer an adaptation and generation engine, which adjusts the appropriate game components to better fit both (Lopes & Bidarra, 2011).

2.3 Assessment generation The traditional approach to assessment generation is the After Action Review (AAR), which typically allows instructors and students to critically review the decisions made and actions taken during game‐play (Raybourn, 2007).This AAR has evolved to assessment generations in games required for adaptive interventions. A plausible example is that utilized in the S.M.I.L.E model (Divéky & Jurnecka, 2007). The model utilizes the Item Response Theory to predict how players react to tasks and questions they are given while solving quests, and thus is able to measure their level of forgetfulness by testing them on tasks and questions they have previously successfully completed and answered.

2.4 Adaptive intervention Pedagogical interventions and feedback are an integral element of the educational process (M. Kickmeier‐Rust & Steiner, 2010). Dynamically adjusting game elements according to individual player performance can contribute to make the game experience more unique and personal (Lopes & Bidarra, 2011). In serious games, adapting to specific skills is more important than to the global notion of difficulty or challenge (Lopes & Bidarra, 2011). Adaptive games have specialized (and usually ad‐hoc) approaches, where the game components are adjusted to encourage training a specific skill (Lopes & Bidarra, 2011). Individual learners/gamers are provided with appropriate personalized interventions, either in the form of personalized feedback or hinting or in the form of smooth alterations of the game (e.g. Its storyline, its pace, or its difficulty) (M. Kickmeier‐Rust & Steiner, 2010). The condition under which a certain adaptive intervention is given is developed on the basis of pedagogical and didactic rules while considering a strong integration in the game‐ play context (Michael D Kickmeier‐Rust, Hockemeyer, Albert, & Augustin, 2008). (M.D Kickmeier‐Rust & Albert, 2010) have classified intervention into the following

443

Chinedu Obikwelu and Janet Read

“Competence activation interventions: may be applied if a learner becomes stuck in some area of the problem space and some competencies are not used even though the system assumes that the learner possesses them.

Competence acquisition interventions: may be applied in situations when the system concludes that the learner lacks certain competencies

Motivational interventions: may be applied, for example, if the learner unexpectedly fails to act for a certain long period of time.

Feedback: may be utilized to provide the learner with information about the learning progress or the game.

Assessment clarification interventions: may be applied, for example in the form of a query, if the learner’s actions provide contradicting support for the assumption of a certain competence state.”

3. The adaptive learning systems The adaptation logic of any personalized learning system can be defined in the following terms (Karagiannidis & Sampson, 2004).

The constituents (what is being adapted?)

The determinants (what the adaptation logic is based on) This information is normally stored in the student/user model. Brusilovsky & Milan (2007) characterize the typical content of user models as “the user’s knowledge, interests, goals, background and individual traits”.

The rules (the logic that defines which constituents are affected by the determinants and how?).

In this section, some existing frameworks are discussed in relation to their adaptation logic in terms of constituents, determinants and rules

3.1 NUCLEO framework (Sancho et al., 2009) The learning strategy in NUCLEO is deeply grounded in a socio‐constructive pedagogical stream (Sancho et al., 2009). It combines Problem‐Based Learning (PBL) and Computer Supported Collaborative Learning (CSCL) in a framework that uses a multiplayer role‐playing video‐game as the delivery format (Sancho et al., 2009). 3.1.1 Constituents Team formation and individual role assignments linked to learning experience are being adapted 3.1.2 Determinant This information includes the following

The mark obtained by group in the mission

Individual mark in peer‐to‐peer evaluation

Frequency of use of specific role tools

3.1.3 Rul After every mission, the student model is updated collecting information from three different sources: the mark obtained by the group in the mission, the individual mark obtained by the student in the peer‐to‐peer evaluation (every student evaluates his/her teammates) and the frequency of use of the specific role tools. According to this information, students’ roles and teams can be reassigned, for instance, in order to reduce intra‐team conflicts or to give students the possibility of experiencing new social contexts.

3.2 Smart multipurpose interactive learning environment S.M.I.L.E (Divéky & Jurnecka, 2007)(Kajan, Bieliková, Divéky, Omelina, & Jurnecka, 2010) S.M.I.L.E combines the advantages from interactive educational materials and popular computer games by giving teachers (i.e. authors of educational materials) the ability to transform study materials into exciting educational games that can be played by users with disabilities including those with visual and hearing

444

Chinedu Obikwelu and Janet Read impairments(Divéky & Jurnecka, 2007). The system is based on an automatic adaptive game generation concept. For adaptivity, the system attempts to be different for different learners and groups of learners by taking into account information accumulated in the learner’s’ or group’s’ profiles. In order to deal with the challenge of uses with disabilities, for adaptability, the system can be customised by every user according to his needs and abilities. The discussion in this work is restricted to adaptivity. 3.2.1 The constituents Games and quests are adapted. The game is a persistent virtual world made up of quests based on all available educational materials, and therefore practically resembles one vast game that contains all possible educational quests. 3.2.2 The determinants This is the subject‐specific skill level of the student. 3.2.3 The rules Players are dynamically navigated into solving quests most suitable for their estimated knowledge level. Additionally, while solving quests, options of appropriate difficulty are chosen for players to solve (e.g., if a player has his estimated knowledge of physics at a low level, he will need to answer a simpler physics question in order to solve the particular quest – and vice versa). The framework uses Item Response Theory to predict how players will react to both tasks and questions they are given while solving quests, and thus to measure their level of forgetfulness by testing them on tasks and questions they have previously successfully completed and answered (Divéky & Jurnecka, 2007).

3.3 Framework for adaptive game presenters with emotions and social comments (Karouzaki & Savidis, 2012) This is a framework for building artificial game presenter characters with emotions, capable of delivering knowledgeable social comments, adapted to individual profiles and game progress for table‐top multiplayer computer games (Karouzaki & Savidis, 2012). The framework supports emotional facial expressions for the presenters, allowing them to convey their emotions and thus be more expressive than the majority of the commentary systems today. Presenters provoke social interaction in order to keep the players and the audience constantly motivated and alerted about the game progress (Karouzaki & Savidis, 2012). 3.3.1 3.3.1 The Constituents: Agents with adaptive social feedback including adaptive emotions. The social feedback includes humour, reward, sympathy, surprise, disappoint, enthusiasm, agony or anticipation. 3.3.2 The determinants Performance, progress and profile of each individual player 3.3.3 The rule The framework uses inputs such as game events and player profiles and keeps track of incoming information about game progress to decide and deliver player‐adapted and context‐sensitive social feedback. Dynamic Game Difficulty (DGD) balancing is employed where the player’s performance is monitored and used to adjust its difficulty accordingly (Karouzaki & Savidis, 2012).

3.4 Adaptive learning in games through non‐invasion ALIGN (Peirce, Conlan, & Wade, 2008) The idea behind the ALIGN system is the separation of the game logic and adaptation logic with flow experience overlapping. In this system, the adaptation is authored at a level abstracted from a particular game, thus could be reused within multiple games (Peirce et al., 2008). The separation of the adaptation and the game features is at the core of the ALIGN system (Peirce et al., 2008). This feature aims to reduce the cost of

445

Chinedu Obikwelu and Janet Read implementing adaptation through maximizing the reuse of domain expert authored adaptation (Peirce et al., 2008). 3.4.1 The constituents In this system, a diverse range of Adaptive Elements (AEs) can be adapted depending on the game. The Adaptive Elements (AEs) include Cognitive feedback; Instant meta‐cognitive feedback; Tendency meta‐ cognitive feedback; NPC Confidence/Prudence feedback; Affective/ motivational feedback; Knowledge based hinting; Progression hinting etc. 3.4.2 The determinants The information contained in the user/student model of the game for which an adaptation abstraction is used. 3.4.3 TherRules The adaptation logic for a particular game focuses solely on selecting desirable AEs as the AEs are already constrained by feasibility and appropriateness.

3.5 The fine‐tuning system (FTS) (Obikwelu, Read, & Sim, 2013) The FTS is based on faded scaffolding levels. According to (Merrill, 2011) faded scaffolding levels are of interest because they offer an interaction with the student in which the working memory load is not as heavy as for totally faded scaffolding levels – so there is a gradual transition to ‘full’ exercises “Fading can be used to anchor and stimulate Limited problem solving; Reflection; Self‐explanation” (Melis & Goguadze, 2004). The structure of the full scaffolding level, determines the possibilities of fading (Melis & Goguadze, 2004).Static fading and Adaptive fading are the two types of fading (Reisslein, Reisslein, & Seeling, 2006)

Static fading ‐ as the scaffolding levels are faded, the learners are required to attempt each scaffolding level at a predetermined rate. No attention is paid to whether or not the learner could keep up with the increasing problem‐solving demands (Reisslein et al., 2006).

Adaptive fading ‐ adapts the fading of scaffolding levels to the learner’s successes and failures in the problem‐solving process (Reisslein et al., 2006).Adaptive fading is the process of reducing (fading) the scaffolding level with every new problem continues until all the scaffolding levels are faded away and the learner has to independently solve the entire problem (Reisslein et al., 2006).

3.5.1 Constituent Learning support (scaffolds) including feedbacks and hints 3.5.2 Determinant Successful attempts, number of attempts and attempts per scaffolding level 3.5.3 Rule The scaffold is reduced whenever the player‐learner successfully solves a particular task at a particular scaffolding level – adaptive fading. The lesser the number of attempts at a particular task on a given scaffolding level, the faster the fading. An interesting measure elucidated from this framework is ‘attempts per scaffolding level’ being inversely proportional to the fading rate. The fading rate is expected to be a direct representation of the player‐learner’s learning rate.

4. Student/user modelling: Challenges and limitations Adaptive interventions in serious games are based on the determinants which are mainly the current state of the player‐learner’s knowledge – i.e. the Student Model. The representation of the player‐learner’s current state of knowledge is rather problematic as the determinants used in most ALS could expectedly be influenced

446

Chinedu Obikwelu and Janet Read by the player‐learner’s game preference, background and game‐play expertise. Questions that have arisen to this effect include

How do you show that the students are learning what you claim they are learning?

How do you know you are measuring what you think you are measuring? (Chen & Michael, 2005)

The solution to these problems could be in the initialization of the student model based on game‐related player‐learner’s characteristics – game preference, background and game‐play expertise. This initial classification of the player‐learners would help minimize the effect of the ephemeral data elucidated via game‐ play (showing the state of knowledge) upon which adaptive intervention is based. “The simplest kind of assessment classifies the student in terms of behaviour, preferences or background”. “Such a stereotype user‐ model (Rich 1979) is useful for ‘priming’ the modelling process when there is no prior experience on which to base expectations (Carr & Goldstein 1977, Clancey 1982a)”(Clancey, 1986). There is also the challenge of setting clear student modelling goals. According to (AILE, 2006) the goals of student modelling could include

Finding out what the student knows, believes, can do

Looking for evidence the user fails to exploit some knowledge

Looking for inconsistent beliefs, differences between student and domain model

It is important to identify a number of relevant criteria associated with the goal of student modelling. For example “Looking for inconsistent beliefs between student and domain model” has been described by (Brown, Burton, & Larkin, 1977) as the bug model. “In general a bug is some structural flaw (faulty part) manifested in faulty behaviour (a process). Thus the term bug is used to refer to the incorrect part of a constructed procedure” (Clancey, 1986).

4.1 Student modelling based on the Bug Model (Brown et al., 1977):

List possible inconsistent beliefs that can be made (code bugs) – bug library with ratings based on criticality of misconception

Describe the bug: The reason about what student would have to believe in order to exhibit behaviour indicating the bug.

Single out the bugs – This can be done through direct access to real‐time behaviours in unobtrusive ways within the context of the system (Knight, Buckingham Shum, & Littleton, 2013)

Measure by means of a formula: The list of captured inconsistent beliefs are combined with a formula for measurement purposes “The combining formula may be simple, and require only the addition of weighted or unweighted component scores or ratings. On the other hand, the formula may be complicated (taking, for example, conjunctive and disjunctive form)” (D.Royce, 1989)

4.2 Student modelling based on the Overlay Model A user’s knowledge of a subject is most often represented by an overlay model which is based on the structural model of the subject domain (Brusilovsky, 1996). The idea of the overlay model is to represent an individual user’s knowledge of the subject as an “overlay” (subset) of the domain model which reflects the expert level knowledge of the subject (Brusilovsky & Millan, 2007).

Break the knowledge for the given domain into elementary pieces (concepts) and show the relationship. These could include the learning activities.

Assign values to these elementary pieces for a representation of the user knowledge of the concept. “This can be just a binary value (known‐not known), a qualitative measure (good‐average‐poor), or a quantitative measure, such as a probability that the user knows the concept” (Brusilovsky, 1996). “An overlay model of user knowledge can be represented as a set of pairs “concept – value”, one pair for each domain concept”. (Brusilovsky, 1996)

The overlay model is the student model associated with most serious game ALS. It is envisaged that the student modelling for any serious game ALS would be more effective if the overlay model is rightly combined

447

Chinedu Obikwelu and Janet Read with the bug model. Further research into the appropriate combination of existing student models for effective student modelling is required.

5. Conclusion This work summarizes several Adaptive Learning Systems that are being used in serious games. Serious game Adaptive Learning Systems are expected to protect and enhance the immersion element of the game whilst directing the player‐learner to the game’s learning objective. Adaptivity whilst protecting and enhancing immersion is the prime challenge of adaptive scaffolding. Another challenge is deducing learning analytics (“learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.”(1st International Conference on Learning Analytics and Knowledge, 2011) (George & Long, 2011)) that could be associated with a particular ALS. At present, serious game ALS is focused on the adaptation of adaptive elements to the player‐learner’s competence models based on set rules. The extent to which they protect immersion and elucidate useful learning analytics from the student modelling is unclear at the moment.

References 1st International Conference on Learning Analytics and Knowledge. (2011). Learning Analytics & Knowledge. Retrieved June 2, 2013, from https://tekri.athabascau.ca/analytics/ AILE. (2006). Introduction to Student Modelling. Retrieved from http://www.inf.ed.ac.uk/teaching/courses/aile/lect6studmhoout.pdf Azevedo, R., Gromley, J. G., & Seibert, D. (2004). Does Adaptive Scaffolding Facilitate Students’ Ability to Regulate Their Learning with Hypermedia? Contemporary Educational Psychology, 29(3), 344–370. Bailey, C., & Katchabaw, M. (2005). An experimental testbed to enable auto‐dynamic difficulty in modern video games. Proceedings of the 2005 GameOn North …. Brown, J. S., Burton, R. R., & Larkin, K. M. (1977). Representing and using procedural bugs for educational purposes. ACM ’77 Proceedings of the 1977 annual conference (pp. 247–255). New York,NY, USA. Brusilovsky, P. (1996). Methods and techniques of adaptive hypermedia. User Modeling and User Adapted Interaction, 6(2‐ 3), 87–129. Brusilovsky, P., & Millan, E. (2007). User models for adaptive hypermedia and adaptive educational systems. The Adaptive Web (pp. 3–53). Berlin: Springer‐Verlag. Brusilovsky, P., & Sampson, D. (2004). Layered evaluation of adaptive learning systems Charalampos Karagiannidis. Engineering Education, 14, 402–421. Chen, S., & Michael, D. (2005). Proof of Learning: Assessment in Serious Games. Gamasutra. Clancey, W. J. (1986). Qualitative Student Models. Ann. Rev. Comput. Sci., 1, 381–450. D.Royce, S. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18, 119–144. Divéky, M., & Jurnecka, P. (2007). Adaptive Educational Gameplay within Smart Multipurpose Interactive Learning Environment. Semantic Media …. George, S., & Long, P. (2011). Penetrating the Fog: Analytics in Learning and Education. Educause Review. Ismailovic, D., Haladjian, J., Kohler, B., Pagano, D., & Brugge, B. (2012). Adaptive Serious Game Development. Games and Software Engineering (GAS), 2nd International Workshop on. IEEE. Kajan, R., Bieliková, M., Divéky, M., Omelina, L., & Jurnecka, P. (2010). Learning with Smart Multipurpose Interactive Learning Environment. International Federation for …. Karagiannidis, C., & Sampson, D. (2004). Adaptation Rules Relating Learning Styles Research and Learning Objects Meta‐ data. Workshop on Individual Differences in Adaptive Hypermedia, 3rd International Conference on Adaptive Hypermedia and Adaptive Web‐based Systems (AH2004), Eidhoven, Netherlands. Karouzaki, E., & Savidis, A. (2012). A Framework for Adaptive Game Presenters with Emotions and Social Comments. International Journal of Computer Games Technology, 2012, 1–18. doi:10.1155/2012/929814 Kickmeier‐Rust, M., & Steiner, C. (2010). A glissade on the learning curve: multi‐adaptive immersive educational games. … for Education. Digital …, 361–366. Kickmeier‐Rust, M.D, & Albert, D. (2010). Micro‐adaptivity: protecting immersion in didactically adaptive digital educational games. Journal of Computer Assisted Learning, 26, 95–105. Kickmeier‐Rust, Michael D, Hockemeyer, C., Albert, D., & Augustin, T. (2008). Micro adaptive, non‐invasive assessment in educational games. In M. Eisenberg, Kinshuk, M. Chang, & R. McGreal (Eds.), Proceedings of the second IEEE International Conference on Digital Game and Intelligent Toy Enhanced Learning (pp. 135–137). Banff, Canada. Knight, S., Buckingham Shum, S., & Littleton, K. (2013). Epistemology, pedagogy, assessment and learning analytics. Proceedings of the Third International Conference on Learning Analytics and Knowledge ‐ LAK ’13, 75. doi:10.1145/2460296.2460312 Lopes, R., & Bidarra, R. (2011). Adaptivity Challenges in Games and Simulations: A Survey. IEEE Transactions on Computational Intelligence and AI in Games, 3(2), 85–99. doi:10.1109/TCIAIG.2011.2152841 Mehm, F., Konert, J., Göbel, S., Steinmetz, R., Authoring, A., Educational, D., In, G., et al. (2012). An Authoring Tool for Adaptive Digital Educational Games State of the Art, (September), 236–249.

448

Chinedu Obikwelu and Janet Read Melis, E., & Goguadze, G. (2004). Towards adaptive generation of faded examples. Intelligent Tutoring Systems, 1–18. Merrill, D. (2011). Systematic Development of an E‐learning Module for Teaching Writing. World Conference on Educational Multimedia, Hypermedia and Telecommunications (Vol. 2011, pp. 1132–1137). Obikwelu, C., Read, J., Sim, G.(2013). Children ’ s Problem‐Solving in Serious Games : The “ Fine‐ Tuning System ( FTS )” Elaborated, 11(1), 49–60. Peirce, N., Conlan, O., & Wade, V. (2008). Adaptive Educational Games: Providing Non‐invasive Personalised Learning Experiences. 2008 Second IEEE International Conference on Digital Game and Intelligent Toy Enhanced Learning, 28– 35. doi:10.1109/DIGITEL.2008.30 Raybourn, E. M. (2007). Applying simulation experience design methods to creating serious game‐based adaptive training systems. Interacting with Computers, 19(2), 206–214. doi:10.1016/j.intcom.2006.08.001 Reisslein, J., Reisslein, M., & Seeling, P. (2006). Comparing static fading with adaptive fading to independent problem solving: The impact on the achievement and attitudes of high school students learning. JOURNAL OF ENGINEERING …, (July). Sancho, P., Moreno‐Ger, P., Ruben, F.‐F., & Baltasar, F.‐M. (2009). Adaptive role playing games: an immersive approach for problem based learning. … Technology & Society, 12, 110–124. Vermunt, J. D. (1987). Learning styles and self‐regulation. Annual Meeting of the American Educational Research Association. doi:ERIC Document 285 900

449

Combatting Social Isolation and Cognitive Decline: Play a Physical or Digital Game? Daire Ó Broin and Ross Palmer Institute of Technology Carlow, Ireland daire.obroin@itcarlow.ie ross.palmer@itcarlow.ie Abstract: Two significant problems among the elderly community are those of social isolation and the cognitive decline that occurs as we age. These problems can be readily compounded (for example, reduced levels of attention are associated with an increased risk of falling). Keeping active is commonly advised to reduce these problems ‐ many senior groups exist that meet regularly and engage in activities that encourage social interaction. However, mobility and other problems mean that elderly people may not be able to attend such groups very often, if at all. To address this problem, we set out to create an experience that confers the same benefits (increasing social connectedness and inhibiting cognitive decline) without users having to leave their homes. Our user group selected one of the games they play together, Memory, a game that requires players to closely attend to their experience. We designed and developed a digital multiplayer version of Memory for browser and tablet targeted at elderly users. The game is integrated into a custom social network, and enables users to talk and play without having to be in the same physical space. It also incorporates user‐generated content from the social network. A pilot study has been carried out with two user groups across two European countries. It investigates whether the experience of playing a physical game between players in present the same physical space differs significantly from playing in different locations using the digital version of the game. In order to uncover differences, the study compared how intrinsically motivating each experience is. In the study, which uses within‐subjects design, the users played Memory with real cards together in the physical space and afterwards completed the Intrinsic Motivation Inventory (IMI). The users also played the digital version of the game via the social network, this time where users are in different locations using a PC, Smart TV, or Tablet, and communicate using VOIP, and afterwards completed the IMI. The data collected was analysed and used to inform semi‐structured interviews. This paper presents findings from the pilot study, and from these findings outlines the main study and future work in investigating and improving the game’s benefits. Keywords: games for seniors, social connectedness, cognitive decline, intrinsic motivation

1. Introduction Two of the most significant problems affecting elderly people are those of social isolation and the cognitive decline that occurs with age. Social isolation and loneliness in older people are problems that are becoming more and more widely recognised in international policy (Cattan et al., 2005). Numerous studies have shown the extremely negative effects social isolation can have on elderly people, such as cognitive decline (Zunzunegui et al. 2003, Bassuk, 1999), risk of dementia (Fratiglioni, 2000), and increased mortality (Steptoe et al., 2013). These problems can lead to further problems leading to further decline. For example, reduced levels of attention are associated with an increased risk of falling (O’Halloran et al., 2011). Keeping active is advised to alleviate these problems, and there are many senior groups that meet regularly and engage in activities aimed at encouraging social interaction. As a result of problems, such as that of mobility, elderly people may not be able to attend such groups frequently, or indeed at all. This paper describes work that forms part of a larger project, Join‐In (Join‐In, 2013), whose aim is to address the issues of social isolation, cognitive decline, and physical decline in elderly people using a social network portal customised for elderly people from which multiplayer games can be initiated. The use of games to promote well‐being in elderly people has been demonstrated, for example, by Goldstein et al, whose work showed that elderly people playing digital games for five hours a week for five weeks show improvements in reaction times, self‐esteem and sense of well‐being (Goldstein et al, 1997). Gaming also has the advantage of being multigenerational and facilitates the elder generation to socialise with the newer generations (e.g. their children and grandchildren). This paper describes the design of one of the multiplayer games developed, based on the card game Memory, which aims to address the issues of social isolation and cognitive decline. As this is a game selected by the user groups, it was necessary, before rolling out the main study, to investigate whether the experience of playing a physical game between players in the same physical space differs significantly from playing in different locations using the digital version of the game.

450

Daire Ó Broin and Ross Palmer A pilot study of this game has been carried out with two user groups in two European countries (Ireland and Germany). In order to uncover differences, the study compares how intrinsically motivating each experience is. In the study, participants played both the physical and digital versions of Memory and afterwards completed a subset of the Intrinsic Motivation Inventory (IMI) (Ryan et al., 1983). In the physical version, pairs of participants played together with real cards in the same physical space. In the digital version of the game, participants accessed the game via the social network portal using an iPad. This paper begins by describing the design of the Memory game; it then presents the design and findings from the pilot study. A discussion follows, and the main study is outlined, informed by the findings of the pilot study. The paper concludes by outlining future work.

2. Design of the game This section describes the requirements of the game gathered from the user groups and how some of the main requirements were met.

2.1 Requirement gathering and selection of a game The initial stage of the Join‐In project was to develop a social networking methodology for elderly persons who are little inclined to seek new contacts themselves. This target group were involved in activities to help them build and establish new social networks. User requirements were gathered from interviews, focus groups, questionnaires, and consultations with experts in elderly care. These requirements were ranked, and included:

Support for cooperative and competitive play

Possibility of communicating with other players

Simple to follow rules (that do not cause trouble for elderly users)

Beneficial for mental fitness

Means of tracking progress (for users and health professionals)

A simple and intuitive interface

Familiar elements (to ease access to the game)

Must be perceived as useful

Fun to play

Allow for multiple users at different locations to play simultaneously

Give positive feedback

Follow “design for all” principles, including

Adjustable speed/different levels of difficulty.

This can be used to accommodate users (including friends and family) with different ability levels and allow them to play together in a meaningful way.

Possible to play with limited fine motor skills

Both visual and aural feedback

Accessible from multiple platforms to enable wide target audience to play from their existing devices

In order to meet the needs and interests of the target group, the activities and games that are the favourites of the elderly user groups were elicited. Both board games and card games were frequently mentioned by the different user groups, and a multiplayer card game based on the game ‘Memory’ was finally settled on. The memory card game, also commonly known as the game of ‘concentration’, is a turn based card game. The game consists of ten pairs of cards arranged in four rows of five cards. Initially, the cards are all placed face down. Each player takes turns to try and match pairs of cards by selecting the two cards they wish to turn over. When two cards are matched, they remain drawn in a face up position and form no further part of the game. A player is rewarded for a successful match by getting another turn. The game ends when all ten pairs have been matched and the winner is the player with the most matches.

451

Daire Ó Broin and Ross Palmer

2.2 Meeting the requirements This section describes how some of the main requirements of the game were met, illustrating with some screenshots from the game. 2.2.1 Accessibility In order to maximise accessibility to the target audience, the game needed to be platform‐agnostic as far as possible. This was achieved by creating a browser‐based game using HTML5. This enables users to use most devices including tablets, Smart TVs, and PCs. The game is readily accessed from the customised social network portal, based on Elgg (http://elgg.org/), which has a simplified user interface suitable for elderly users. This is shown in Figure 1 – the coloured circle beside a friend’s name indicates if they are online and can be invited to play.

Figure 1: The social portal with simplified user interface suitable for elderly users; the coloured circle beside a friend’s name indicates whether they are online and can be invited to play 2.2.2 Adjustable speed/different levels of difficulty The game has adjustable levels of difficulty and allows players of different skills to play together by means of bonuses and handicaps. The simplest of these is beginning the game with some of the cards already upturned (Figure 2).

Figure 2: The main gameplay screen of the memory game; bonuses and handicaps enable players of different skills to play together; this example shows some cards already upturned for the player

452

Daire Ó Broin and Ross Palmer 2.2.3 Simple rules The memory game employs simple rules and already familiar game mechanics. It also opened the possibility of inter‐generational play between seniors and youngsters (such as grandparents and grandchildren) (Abeele and De Schutter, 2010).

Figure 3. Pop‐up notifications are used to give positive reinforcement messages 2.2.4 Positive feedback During play, a simple feedback system displays short‐lived notifications to the players. These notifications appear occasionally and praise successful card matches or offer encouragement when no matches are made (Figure 3). 2.2.5 Replayability The memory game has a number of selectable card ‘decks’ based on different themes (for example, nature, cartoon faces, national food dishes) as illustrated in Figure 4. To enhance replayability, users may create their own picture albums on the social portal and designate which albums they would like to use as card decks in the memory game. It is anticipated that this feature will offer opportunities for storytelling and offer a compelling reason for seniors to engage with the game on a recurring basis.

Figure 4. Players can choose which set of images to use as the in‐game card deck

453

Daire Ó Broin and Ross Palmer 2.2.6 Measure player performance A recent study (Grégoire et al. 2012) provides empirical support for the relationship between mindfulness and wellbeing where this relationship is partially mediated by personal goal setting. To enable players set personal goals, it is necessary to measure and record game performance. A set of metrics were devised to capture the player performance in terms of (i) skill level and (ii) engagement by the player. Skill level The computation of the skill level is derived from a player’s performance based upon a notional working model of how a player might perform. Each time a card is turned (by either player), it enters the players working memory, where the player remembers both the card and its position in the game. Cowan (Cowan, 2010) suggests that young adults have a working memory capacity of about 3 to 5 chunks of information (letters, digits or words) without rehersal. A baseline value of four cards (and their associated positions) is assumed as the maximum number of items a player can hold in their working memory. When a pair of cards is matched, we assume that both cards are removed from the player’s working memory. If the outcome of a turn is a matching pair of cards, a turn score is calculated using a devised algorithm. Player engagement A simple summation of the game score is used to set the experience level of a player. The more often players engage in games of memory, the more rapidly their experience level will grow (with an expectation that their skill level increases too). An approach similar to that used in commercial games is employed by not placing a cap on the experience level. The frequency of engagement is also important and this metric can be computed easily from the datetime stamp recorded after each game is completed. After each multiplayer game, the game score, the duration of play and a datetime stamp are recorded, and from this, the total duration for any given day and “streaks” (number of consecutive days) may be calculated.

2.3 Technical design The memory game server is written in Node.js (http://nodejs.org), a server‐side JavaScript framework for developing scalable network applications and uses socket.io to implement an asynchronous messaging system with the game clients, and provides an abstraction layer over Web Sockets and other communication schemes, depending on the browser capabilities.

3. The study This study is a pilot study of a larger study that is currently being rolled out to a much larger sample and rather than taking place in a lab environment will be used by the participants in their homes for about three months in four European countries (Ireland, Germany, Hungary, and Norway).

3.1 Objective The main objective of the study was to determine whether there is a significant difference in the participants’ experience between playing a physical game of Memory with co‐located players to playing the digital multiplayer version in different locations. A secondary purpose was to identify and address usability issues.

3.2 Study design The study employed within‐subjects design in which each subject is observed under each condition. In the study, participants played both the physical and digital versions of Memory and afterwards complete a subset of the Intrinsic Motivation Inventory (IMI) (Ryan et al., 1983). In the physical version, pairs of participants played together with real cards in the same physical space. In the digital version of the game, participants accessed the game via the social network portal using an iPad.

3.3 Measures The players’ experiences were assessed using the Intrinsic Motivation Inventory (IMI), a multidimensional instrument that measures the interest/enjoyment‚ perceived competence‚ effort‚ value/usefulness‚ felt pressure and tension‚ and perceived choice of participants when they were performing a particular activity

454

Daire Ó Broin and Ross Palmer (Ryan et al., 1983). Usability was also measured with a custom questionnaire and by observing the ease with which the participant completes the tasks she needed to do.

3.4 Setting The study took place in a number of laboratory rooms in the Institute of Technology Carlow, and also in a similar setting in Helmholtz in Germany. The participants were in different rooms and couldn’t see each other to simulate, as closely as possible, playing alone from home. Assistants were available to help them get setup.

3.5 Sample The participants, aged between 66 and 80, were recruited by contacting elderly groups, and had very little experience with technology. 18 people, roughly balanced by gender, participated in the study, although 2 of these made some omissions in the questionnaires and their data was not used as a result.

3.6 Procedure Participants are given a short introduction to the project. They are told that further refinement and input from the players is needed before the portal and games would be ready for playing at home. In pairs, they play both digital and physical versions of Memory; pairs alternated between playing the physical version or the digital version first. After each session (physical and digital), participants fill out the IMI, either with paper and pencil or digitally on a Google form. For the physical version, participants play a game of Memory using cards with the same pictures as the digital game and in the same format (a 4 x 5 grid). For the digital version, each participant is given an iPad (connected to Wifi), from which the social portal and subsequently the game can be readily accessed. One of the players invites the other to a game of Memory (players can see a list of their friends and a coloured circle indicating whether they are currently online as shown in Figure 1). Her partner gets a pop up notification (‘Daire invites you to a game of memory accept/reject’). She taps accept and the game begins. At this point the VOIP component starts up, and the participants can chat. The inviter is allowed to choose the deck of cards that they wish to play the game with. They then play for about 15 or 20 minutes (about two or three games) and are then given the IMI to fill in.

3.7 Results The results of the pilot study are depicted in Figure 5. The IMI scores are from 1 to 7, the higher the number, the higher the magnitude of the variable. In summary, the results indicate that the participants found the digital game to be more enjoyable than the physical one. They found the activity to be highly valuable and useful, and all participants noted they would be most willing to try it again.

Figure 5: IMI Scores are from 1 to 7; the results indicate that the participants found the digital game to be more enjoyable, and perceived competence was around the same in both physical and digital versions, suggesting no issues caused by the technology or the participants playing against each other from different locations

455

Daire Ó Broin and Ross Palmer

4. Discussion Prior to the study, there were two principal concerns. First, would the digital version of the game, where players were not in the same physical location as their opponent, pale in comparison with the experience of playing a physical version with the other player physically present? And second, would the lack of experience the participants had with technology be an issue (that is, negatively affect their experience?) In short, would there a significant difference between the physical and digital games (would the digital game simply not stack up against the physical game)? The results are encouraging as they suggest that participants enjoyed the digital version – indeed they enjoyed it more than the physical game. That is, though the participants were in separate locations, this did not have a negative effect on the participants’ enjoyment of the game. As this is the context for which the game was designed (users playing at home by themselves playing with other users also at home by themselves), this is an extremely positive result. Also, while there were some minor usability issues, such as the indicator for knowing whose turn it is not being clear to several participants (see Figure 2 – the player whose turn it is has a large dark blue rectangle around his name, and the other player has a small light blue rectangle around her name), the results suggest that usability or the technology were not a cause of major issues to the elderly users. This is suggested by the observation that the ratings of perceived competence, effort, and pressure/tension for the digital game closely approximated the corresponding ratings for the physical game. Moreover, all the usability issues identified in the study were minor and can be readily improved upon. Finally, the participants found the activity to be highly valuable and useful (in particular for memory and concentration) ‐ high ratings for value/usefulness were recorded for both versions of the game, and indeed somewhat higher for the digital version. Also the perceived choice (that is the degree to which the players felt they were choosing to do the activity) was high in both and again somewhat higher for the digital version. These two factors suggest a willingness of the participants to try the experience again.

5. Conclusion and future work This paper has described the design of a game for elderly users whose goal is to increase social connectedness and inhibit cognitive decline. A pilot study of the game with participants from the target group yielded positive results, and in particular that although the participants were in separate locations, this did not have had a negative effect on the participants’ enjoyment of the game compared to playing the physical version of the game in the same physical space as the other player. Moreover, usability and inexperience with technology did not negatively affect the experience the players had. The pilot study has alleviated the initial concerns, the questions now are: will this game and the other social portal games continue to be enjoyable over weeks and months, will people fit it into their lives, into their existing patterns, and will it lead to increasing social connectedness? These are the questions that the main study aims to answer. The main study is currently being rolled out in four European countries and aims to measure player engagement over a longer period (about three months) as well as assessing the players’ intrinsic motivation over time and the effect playing has on the participants, including well being measured pre and post game as in (Ryan et al., 2006), and whether over this time it will cause an increase in a player’s social connectedness, measured by an adaptation of Cohen’s Social Network Index (Cohen, 1997).

References AAL Joint Programme (2013), Join‐in. Online; http://www.aal‐europe. eu/projects/join‐in/. Abeele, V. V. and De Schutter, B. (2010) Designing intergenerational play via enactive interaction, competition and acceleration. Personal and Ubiquitous Computing, 14, 425–433. Bassuk, S. S., Glass, T. A., and Berkman, L. F. (1999) Social disengagement and incident cognitive decline in community‐ dwelling elderly persons. Annals of internal medicine, 131, 165–173. Cattan, M., White, M., Bond, J., and Learmouth, A. (2005) Preventing social isolation and loneliness among older people: a systematic review of health promotion interventions. Ageing & Society, 25, 41–67. Cohen, S., Doyle, W. J., Skoner, D. P., Rabin, B. S., and Gwaltney Jr, J. M. (1997) Social ties and susceptibility to the common cold. JAMA: the journal of the American Medical Association, 277, 1940–1944.

456

Daire Ó Broin and Ross Palmer Cowan, N. (2010) The magical mystery four how is working memory capacity limited, and why? Current Directions in Psychological Science, 19, 51–57. Fratiglioni, L., Wang, H.‐X., Ericsson, K., Maytan, M., Winblad, B., et al. (2000) Influence of social network on occurrence of dementia: a community‐ based longitudinal study. Lancet, 355, 1315. Goldstein, J., Cajko, L., Oosterbroek, M., Michielsen, M., Van Houten, O., and Salverda, F. (1997) Video games and the elderly. Social Behavior and Personality: an international journal, 25, 345–352. Gregoire, S., Bouffard, T., and Vezeau, C. (2012) Personal goal setting as a mediator of the relationship between mindfulness and wellbeing. International Journal of Wellbeing, 2. O’Halloran, A. M., P ́enard, N., Galli, A., Fan, C. W., Robertson, I. H., and Kenny, R. A. (2011) Falls and falls efficacy: the role of sustained attention in older adults. BMC geriatrics, 11, 85. Ryan, R. M., Mims, V., and Koestner, R. (1983) Relation of reward contingency and interpersonal context to intrinsic motivation: A review and test using cognitive evaluation theory. Journal of Personality and Social Psychology, 45, 736–750. Ryan, R. M., Rigby, C. S., and Przybylski, A. (2006) The motivational pull of video games: A self‐determination theory approach. Motivation and Emotion, 30, 344–360. Steptoe, A., Shankar, A., Demakakos, P., and Wardle, J. (2013) Social isolation, loneliness, and all‐cause mortality in older men and women. Proceedings of the National Academy of Sciences, 110, 5797–5801. Zunzunegui, M.‐V., Alvarado, B. E., Del Ser, T., and Otero, A. (2003) Social networks, social integration, and social engagement determine cognitive decline in community‐dwelling spanish older adults. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, 58, S93–S100.

457

Sports Games’ Role for Learning Health Knowledge Kelly O’Hara1,2, Dulce Esteves1,2, Rui Brás1,2, Ricardo Rodrigues3, Paulo Pinheiro3 and Marco Rodrigues1 1 Sport Science Department, University of Beira Interior, Covilhã, Portugal 2 Research Centre in Sport Science, Health Science and Human Development, Vila Real, Portugal 3 Business and Economic Department, University of Beira Interior, Covilhã, Portugal NECE Research Centre ohara@ubi.pt desteves@ubi.pt rmmb@ubi.pt rgrodrigues@ubi.pt pgp@ubi.pt Abstract: Design learning environments in order to develop 21st century skills is crucial to create in learners “adaptive expertise” or “adaptive competence”. Educational/serious games, accomplished with appropriate learning environments, facilitate teaching through experience by offering immediate feedback and engaging the learner’s attention. The aim of this paper is to develop and applied serious games in health education, using physical exercise and sports as a learning environment. Methods: 201 high school students (15‐22 years, 17.56±1.37) were exposed, during eleven weeks, six hours/week to a learning sport‐game environment in other to promote physical activity and increase the awareness of its benefits. To evaluate the process effectiveness, two groups were formed, the control group (CG) and practical group (PG) that was exposed to serious games. To evaluate the learning process, a survey was developed and tested, resulting in 23 items distributed across the contents under investigation: Caloric Balance (CB); Posture (P), Heart Rate (HR). Student’s knowledge perception about each concept and their need for acquiring more knowledge related to health behaviors were also observed. A Pre and Post‐ intervention test was made. Results: Significant differences were observed between CG and PG in post‐intervention assessment (CB= p<0.000; HR p<0.000; P<0.000) where PG achieved better performance. Assessing the students’ knowledge perception about the contents (KP) and students searching for more knowledge information (SKI), for KP there is no significant difference between pre and post‐test (pre: p<.321 and post: p<.051). However for SKI after intervention results show that students of PG present significant difference (pre: p<0.172 and post: p<0.005) for searching knowledge information. Conclusions: Results show that a learning environment based on serious games can be very useful to motivate and promote students success, therefore should be integrated in school curriculum as a strategy of increasing school health and students’ wellbeing and quality of live. Keywords: learning environment; sport; serious games, health promotion, school‐based approach

1. Introduction Society has suffered a deep transformation from reliance on an industrial to a knowledge base. Design learning environments in other to develop 21st century skills is crucial to create in learners “adaptive expertise” or “adaptive competence”, i.e. the ability to apply meaningfully‐learned knowledge and skills flexibly and creatively in different situations (Dumont and Istance 2010, Spire 2008). To develop those competences, student’s need to learn how to generate, process and sort complex information; to think systematically and critically; to make decisions by weighing different forms of evidence; to ask meaningful questions about different subjects; to be adaptable and flexible to new information; to be creative; and to be able to identify and solve real‐world problems. The learning outcome can be implicit or explicit and is dependent upon the underlying game mechanics and how the content is integrated into the game so the learning is intrinsic to play. For Gee and Shaffer (2010) in formal education there are examples where games used with sufficient support are shown to be motivational and an aid to learning high level or complex skills. The authors argue that games are good for teaching and assessing because they provide appropriate challenges since game design is based on the deep human inclination to play games as a source for highly motivated learning (Gee, 2003). Other authors (Rodrigues et al 2012) consider that if built on previous information, serious games require problem solving and critical thinking. The use of serious games has not yet transferred to the classroom (Gee

458

Kelly O’Hara et al. and Shaffer 2010, Rodrigues et al 2012) because games teach and assess 21st century skills, such as problem solving, collaboration, negotiation etc., that are not the foundation of the current education system. Games with a direct link to the curriculum are more likely to be used in the classroom, especially if the game can provide appropriate assessment and fits into existing lesson structures (Rodrigues et al 2012, Popescu et al 2013). To Rodrigues et al (2012) the criterion for using a game is often whether it will make the teacher’s life easier and not developing learning skills. Serious games to health education are a growing domain, not only with simulators in medical schools, but also in games address to general population. For example the Wii Fit has been recognized as a way of training players in certain appropriate behaviors that will benefit their health (Ulicsak and Wright 2010). The use of serious games in high‐school context should comprehend not only the development of intellectual capacities (retained knowledge of concepts, memory, process and sort complex information; think systematically and critically, etc.) but also social skills (Wouters et al 2009) so, the learning environment should be based on complex problems with different information sources (virtual and real) and different tasks that require different abilities from the student. Those additional tasks must maintain the motivational aspect that serious games achieve. The use of physical exercise and sport is a possible strategy, since the majority of high school students willingly participate in physical activity, especially in the form of team games (Bak‐Sosnowska and Skrzypulec‐Plinta 2012). The creation of a learning environment that includes problem solving and team exercise tasks is particularly important for health education and exercise promotion. However it tends to decrease between ages 11 to 15 for most European countries (OECD 2012). If the learning outcome is understand the benefits of an active lifestyle for health and understand the caloric balance, in order to reversing the overweight and obesity prevalence in young’s and to promote an active lifestyle. Learning environment must comprehend movement and exercise. So, this investigation concerns the development and application of serious games for health education, using physical exercise and sports as a learning environment, concerning three major concepts: Caloric Balance (CB) – obesity is one of the biggest health problems and has substantially increased over the past 20 years (Howard‐Jones 2010). Between 1990 and 2009, the increase in adult population was 9 % to 17% in Organization for Economic Co‐operation and Development (OECD) members, 23% to 34% in US. Much adult health behavior, such as eating habits and physical activity patterns are established during childhood and adolescence (Magarey 2003, Singh 2008, Garber 2011). Understanding the caloric balance allows students to evaluate the amount of energy ingested and the consequent energy that need to be wasted (in physical exercise, for instance), in order to maintain the caloric intake equilibrated with energy spending. Heart Rate (HR) ‐ understanding cardiovascular system and it relationship with physical activity can fight sedentary behaviors. In 2001, only 32% of high school students participated in daily physical education classes (Centers for Disease Control and Prevention 2002). Understanding cardiovascular system and it relationship with physical activity can fight sedentary behavior’ and adequate the exercise intensity to individual capacities. Posture (P) ‐ in Europe 20 or 30% of adults are affected by muscle skeleton pains. In Portugal about 1.2 million of persons have back pain every week. Postural problems become evident in the course of growing. Therefore preventive measures, education habits and right life styles are needed (Jones 2003). The contents tasks considered meaningful real‐life problems since it has a key role to play in bolstering the relevance of the learning being undertaken, supporting both engagement and motivation. The learning environment comprehends the use of different tools, technological devices and information systems, in order to create different solicitations on students.

459

Kelly O’Hara et al. Since studies in high school (Yazzie‐Mintz 2007, National Research Council and the Institute of Medicine 2004) found that almost a quarter of students surveyed indicated more dissatisfaction than satisfaction with their school experience, our aim is to applied those games situations to young people who are unmotivated and disinterested with school activities.

2. Methods 2.1 Participants 201 high school students, aged between 15 and 22 years (17.56±1.37), both genders (Female=80, Male =121), were selected and randomly assigned to two groups, control group (CG) n= 99, and Practical Group (PG) n=102. The protocol was approved by the University of Beira Interior Research Ethics Boards, Portuguese Government Educational Ministry and School board. Parents were informed and gave their permission for the student’s participation.

2.2 Procedure All research interventions were conducted by the investigators. Before and after intervention both groups performed a test questionnaire to assess their basal knowledge contents (CB, HR and P). it was also assess students’ knowledge perception about the contents (KP) and students searching for more knowledge information(SKI). To ensure content and face validity, all questions were reviewed by an expert panel of professors and researchers not involved in the study. The board consisted of 3 sport scientists (with research experience), one expert researcher on market studies survey development and one expert researcher on knowledge management. To guarantee clarity, comprehension and time to complete all items, the survey was reviewed by 2 experts and 40 students of same age interval that did not participate in the investigation. The experimentation occurred during eleven weeks, for 6 hours. Game description The game develop by the research team uses physical activity to promote not only learning goals but also social, psychological, personal development and teamwork. The game conceptions began with concept selection that we intend to transmit to students (CB, HR and P). For each concept a game learning situations was created. The Game stated by introducing the student to the concept (CB or HR or P), the members of their team and the games rules. Students were divided in teams, since teamwork as the potentiality to promote cooperation, cohesiveness over competition and each team had to explore the problem– situations for solving the game. Through the course of the game they have information that they need to consult, reflect upon and debate to solve the mystery. The game‐situation environment are autonomous, players need team work, develop communication skills, perform a broad range of actions including performing self – experiments, interacting with other teams, reading and gathering background information on the concepts. In the current games, there are goals that players can achieve but the way they choose to play determines in which the actions can be performed. As a centered learning environment, criteria of challenge, curiosity, control; learning includes competence and direction in the face of novelty, complexity, and ambiguity. A Game example: Game 1 ‐ “Eating without being fat”. The aim was to show that calories are fundamental in supplying energy to the organism but if they are not completely used up they contribute to weight increase. Each pair of students received a card with a food type. First they had to identify the quantity of calories of this food type (using video information) and then define the effort that they have to make to burn the excess calories, in proportion to the amount of effort needed to complete the game. They were then asked to complete a circuit with more or less difficulties (different types of obstacles, travel distance, time do accomplishes the game), depending on the number of calories per effort.

460

Kelly O’Hara et al. Game 2 ‐ “Looking for the good fat cell”. The aim was to show that there are various types of fat cells (good and bad) and their consequent implications on the functioning of the organism. To transmit this knowledge, researchers created a circuit that combined speed, travel distance, balance and skill accuracy. To transmit the notion of bad fat cells, the student’s ability to complete the circuit with mobility was decreased by adding weight to their limbs and by using uncomfortable clothes. Motivation, social and collaborative learning, communication, knowledge about food, space / time and intensity of the situations were key variables to achieve the goals. The experimental design considerer the fundamental principles of learning environment [Dumont and Istance 2010, OECD 2007, Danish, 1996) trough:

encourage curiosity (exercises led to seeking and integrate information about body functioning);

perceived by students as relevant work to their own personal goals (exercises with important concepts for their own health, to improve performance, controlling anxiety a placement test);

challenging situations (exercises with progressive and competitive goals);

encourages teamwork (with scores exercises, need to communicate and exchange ideas);

the use of technological equipment for student motivation (exercises with Polar heart rate monitors);

demonstrate and explain how simple scientific concepts can improve everyday activities (exercises for postural awareness of the issues and implications on the day‐to‐day).

Considering the key Physical Activity and Health recommendations, games were performed with moderate‐to‐ vigorous intensity (O'Donovan et al 2010, Pangrazi 2003) controlled by heart rate monitoring (Polar Rs100 Basic Heart Rate and Timing).

2.3 Statistic analysis The data was analyzed using t‐Student test for independent variables. A p value of.05 or less was considered as significant. The data were analyzed using the software Statistical Package for the Social Sciences (SPSS 19 for Windows). A 5‐ point Likert‐type scale was used (1=Strongly disagree, 2=Disagree, 3=Neither agree nor disagree, 4=Agree and 5=Strongly agree).

3. Results At baseline, no significant differences were found between the CG and PG in terms of background concerning knowledge about the learning contents (CB= p<.22, HR=p<.525). After intervention, results show (Table 1) that for all contents there were significant difference (p<.000), between groups (CG and PG). Table 1: Comparisons between CG and PG for each knowledge content in post‐test PG.

Sig ta

CG. 56,7%

87,9%

‐8,235

.000

40,2%

53,9%

‐6,281

.000

57,7%

77,8%

‐11,188

.000

Content

Comparing pre‐ and post‐test (Table 2) it is clear that PG had significative better results for all Knowledge contents. Table 2: Comparisons between pre and post‐ test for each group (CG or PG)

HR pre_pos t CB pre_pos t P pre_pos t

CG.

PG.

Sig ta

‐1.9p.p.

‐27.9 p.p

‐6.864

.000

‐.6p.p.

8.4 p.p

‐3.694

.001

‐.6p.p

14.7p.p

‐7.963

.000

461

Kelly O’Hara et al. Assessing the students’ knowledge perception about the contents (KP) and students searching for more knowledge information(SKI), for students’ knowledge perception there is no significant difference between pre and post‐test (pre: p<.321 and post: p<.051). In respect to SKI after intervention results show that students of PG were searching for more knowledge information that students from CG with significant difference (pre: p<0.172 and post: p<0.005).

4. Discussion The main purpose of this research study was to examine high school unmotivated students’ achievement in health concepts learning through an educational games, and if game based learning can be effective in this kind of students population. This study showed that games can be utilized in formal learning environments to support students learning and simultaneously practice physical activity. Students achieved statistically significant learning gains when learning about health concepts. Considering the results about knowledge contents, our findings are in agreement with literatures (Jennifer et al. 2005, OECD 2007, Spires 2008, Dumont and Istance 2010, Howard‐Jones et al 2011) were learning can be optimized if students a) were involved in real life and meaningful problem‐based activities, b) if is based on situated practice, c) extended engagement of self as an extension of an identity to which the player is committee, d) participate in practical situations (games) that operates at a player’s “outer regime of competence” e) are challenging and f) participate in a cooperative team work. Our results suggest that the exposure to new content, new ways of presenting knowledge, context and real situations of interest (Dumont and Istance 2010), special is this kind of population. Create motivation and engagement in the learning process by exploring, interacting and collaborating are crucial for its exit (Yazzie‐ Mintz 2007). These results are in agreement with studies (Huebner, et al. 2000, Jennifer et al 2005) that suggest that students who feel connected to school are more likely to commit themselves to the learning process and to exhibit more positive behaviors and attitudes. Those, who perceive school as positive and relevant, are more likely to exhibit participatory classroom, such as seeking for new information about the learning contents. According to Zyda (2005), games based learning activities are designed to help achieving a balance between fun and educational value since it is one of active learning methodology. The use of games with learning purposes has also benefitted from the use of educational technologies (Padrós et al. 2011). Our experimental combine not only technology but also physical activity in other to develop problem solving competences, decision making. If school activity were treated in terms of learning, playing, then students could be more thoroughly engaged in the learning process (Popescu et al. 2013). Considering the obtain result for SKI, the proposed learning games had a positive effects in this problematic student’s (as already mentioned with poor motivation and performance, lack of participation in school activities (National Research Council and the Institute of Medicine 2004, Hafen et al. 2012). During and after intervention they were motivated and interested for acquire more information about the learned knowledge, as SKI results showed. This finding can be a contribution to the proposed holistic interventions (Larson 2000, McNeely 2002, Haraldssona 2008, Douglas et al. 2010) in other to promote well‐ being related to stress. For example, one of the HR games situations was to develop student’s ability to control the HR frequency in several contexts (stress, anxious, fear). This findings toward with research, that learning would be optimized if student’s were involved in real‐life and meaningful problem‐based activities. The proposed games situations contribute to the development of the new educational challenge on the relationship between physical activity and health promotion, showing how it can be possible to learn but also develop others skills in a motivating way.

462

Kelly O’Hara et al.

5. Conclusions The present study allowed us to help students to develop team work, seeking changes in lifestyle and health promotion behavior’s, learn new information, experimenting, discussing, taking decisions in different situations. This paper prepossesses a School‐Based Interventions that increases both physical activity level and knowledge about some fundamental health knowledge concepts in a motivated, playing and learning way, even for difficult student population. Considering the results, this kind of learning environment should be integrated in school curriculum as a strategy of increasing school health and students’ wellbeing and quality of live. Since learning content should be an ongoing part of playing the game, emerging from game play as a consequence of game play rather than an antecedent (Charsky and Ressler 2011), sport or physical activities games can be used as an effective tool because “game” is it essence. However more longitudinal studies are also recommended to consolidate results. New games situations should be develop and applied for a longer period of time in other to identified changes healthy behaviors.

Acknowledgements Thanks to teachers, students and school staff from Campos Melo High School, Portugal, who took part in this study.

References Bak‐Sosnowska, M., and Skrzypulec‐Plinta, V. (2012) ”Eating habits and physical activity of adolescents in Katowice ‐ the teenagers' declarations vs. their parents' beliefs”, Journal of Clinical Nursing, Vol. 21( 17‐18), pp. 2461‐2468, July. Centers for Disease Control and Prevention (2002) “Youth Risk Behavior Surveillance—United States, 2001”, MMWR Surveill Summ, Vol. 51, pp 1–64. Charsky, D. and Ressler, W. (2011). “Games are made for fun”: Lessons on the effects of concept maps in the classroom use of computer games. Computers & Education, Vol. 56, No 3, pp 604‐615. Danish, S.J., Nellen, V.C., and Owens, S.S., (1996) “Teaching life skills through sport: Community based programs for adolescents”. In J. L. Van Raalte, & B. Brewer (Eds.), Exploring Sport and Exercise Psychology. (American Psychological Association, Ed.) (pp. 205–225). Washington; DC. Douglas C. S., Ito, A., Gruenewald J. and YehH. (2010) “Promoting School Engagement: Attitudes Toward School Among American and Japanese Youth” Journal of School Violence Vol. 9, No 4, pp 392‐406. Dumont H. and Istance, D. (2010) “The Nature of Learning Using Research to Inspire Practice”, Edited by Hanna Dumont, David Istance and Francisco Benavides, OCDE. Garber, et al, (2011) "Quantity and Quality of Exercise for Developing and Maintaining Cardiorespiratory, Musculoskeletal, and Neuromotor Fitness in Apparently Healthy Adults: Guidance for Prescribing Exercise." Medicine & Science in Sports & Exercise, Vol. 43, No 7 pp 1334‐59. Gee, J. P. (2003). What video games have to teach us about learning and literacy. Palgrave MacMillan, New York. Gee, J. and Shaffer, D. (2010) “ Looking where the light is bad: video games and the future of assessment”, [online] Madison: university of Wisconsin‐Madison. URL: http://epistemicgames.org/eg/looking‐where‐the‐light‐is‐bad. Hafen, C., Mikami, J.P., Gregory, A., Hamre, B. and Pianta, R.C. (2012) “The pivotal role of adolescent autonomy in secondary school classrooms,” Journal of Youth and Adolescence, Vol. 41, No 3, pp 245‐255, December. Haraldssona, K., Lindgrena, E., Fridlundb, B., Baigia, A., Lydella, M. and Marklunda, B. (2008) “Evaluation of a school‐based health promotion programme for adolescents aged 12–15 years with focus on well‐being related to stress,” Journal of the Royal Institute of Public Health, Vol.122, No.1, pp 25–33, April. Howard‐Jones, P., Skevi D., Rafal B., Jee H.Y. and Ute L. (2011) "Toward a Science of Learning Games." Mind, Brain, and Education 5, no. 1, pp 33‐41. Chapter 8. Learning with technology— Richard E. Mayer The Nature of Learning Using Research to Inspire Practice, Edited by Hanna Dumont, David Istance and Francisco Benavides, OCDE. Huebner, E.S., Drane, W. and Valois, R.F. (2000) “ Levels and demographic correlates of adolescent life satisfaction reports”. School Psychology International, Vol. 21, No 3, pp 281–292. Jennifer,F., Blumenfeld, P., Friedel, J. and Paris, A. (2005) "School Engagement." Chap. 19 In What Do Children Need to Flourish?, edited by KristinAnderson Moore and LauraH Lippman. The Search Institute Series on Developmentally Attentive Community and Society, 305‐21: Springer US, Jones, G. T., Watson, K. D., Silman, A. J., Symmons, D. P., and Macfarlane, G. J. (2003) “Predictors of low back pain in British schoolchildren: a population‐based prospective cohort study”, Pediatrics, Vol. 111, No 4 , pp 822–828, April. Larson, R.(2000) “Toward a psychology of positive youth development,” American Psychologist, Vol.55, No.1, pp 170–183. McNeely, C., Nonnemaker, J. and Blum, R. W. (2002) “Promoting school connectedness: Evidence from the National Longitudinal Study of Adolescent Health,” Journal of School Health, Vol. 72, No.4 pp.138–146.

463

Kelly O’Hara et al. Magarey, A.M., Daniels, L.A., Boulton, T. and Cockington, R.A. (2003) “Predicting obesity in early adulthood from childhood and parental obesity”, Journal of the International Association for the Study of Obesity, Vol. 27, No.4, pp 505–513. National Research Council and the Institute of Medicine “Engaging schools,” The National Academies Press , Washington, DC. O'Donovan, G., Blazevich,A., Boreham, C., Cooper, A., Crank, H., Ekelund,U., Fox,K., et al. (2010)."The Abc of Physical Activity for Health: A Consensus Statement from the British Association of Sport and Exercise Sciences." Journal of Sports Sciences Vol. 28, No 6,pp. 573‐91. OECD (2007) “Understanding the Brain: The Birth of a Learning Science”. (O. For Economic Co‐operation & Development, Eds.) Learning pp. 13–18, OECD Publishing. OECD (2012) Health at a Glance: Europe 2012, OECD Publishing. Padrós, A. Romero, M. and Usart, M. (2011). “Developing serious Games: Form Face‐to‐Face to a Computer‐based Modality”, E‐learning papers, Vol 25. Pangrazi, R.P., Beighle, A., Vehige, T., and Vack, C. (2003) “Impact of Promoting Lifestyle Activity for Youth (PLAY) on children’s physical activity”. The Journal of school health, Vol.73, No 8, pp. 317–321. Popescu, M. M., Romero, M., & Usart, M. (2013). Serious Games for Serious Learning Using SG for Business, Management and Defence Education. International Journal of Computer Science Research and Application, Vol 3, No 1, pp 5‐15. Rodrigues, R.G., Pinheiro, P. and Barbosa,J. (2012)”Online Playability – the social dimension to the virtual world”. In M. M. Cruz‐Cunha (Ed.), Handbook of Research on Serious Games as Educational, Business and Research Tools, pp. 391– 421, Hershey, PA, USA: IGI Global. Singh, A.S., Mulder, C., Twisk, J.W., Mechelen, W. and Chinapaw M.J. (2008) "Tracking of Childhood Overweight into Adulthood: A Systematic Review of the Literature." Obesity Reviews, Vol. 9, No 5, pp. 474‐88. Spires, H.A., (2008) “Twenty‐first century skills and serious games: Preparing the N Generation” In Serious educational games: From theory to practice, ed. L.A. Annetta, 13–24., The Netherlands: Sense Publishers, Amsterdam. Ulicsak, M. and Wright, M. “Games in Education: Serious Games. A Futurelab literature review” in Serious games in education. Tech. rep. (2010). [online] URL http://media.futurelab.org.uk//resources/documents/lit reviews/Serious‐ Games Review.pdf Wouters, P., Van der Spek, E.D. and Van Oostendorp, H. (2009) “Current Practices in Serious Game Research: A Review from a Learning Outcomes Perspective”. In T. Connolly, M. Stansfield, & L. Boyle (Eds.), Games‐Based Learning Advancements for Multi‐Sensory Human Computer Interfaces: Techniques and Effective Practices (pp. 232‐250). Hershey, PA: Information Science Reference. Yazzie‐Mintz, E. (2007),” Voices of students on engagement: A report on the High School Survey of Student Engagement. Bloomington Center for Evaluation & Education Policy. Indiana University. Zyda, M. (2005) From visual simulation to virtual reality to games. IEEE computer, Vol. 38, No 9, pp 30–34.

464

A Multi‐Agent Architecture for Collaborative Serious Game Applied to Crisis Management Training: Improving Adaptability of non Played Characters M’hammed Ali Oulhaci1, 2, Erwan Tranvouez1, Sébastien Fournier1 and Bernard Espinasse1 1 Aix‐Marseille Université (AMU), LSIS UMR CNRS 7296, Marseille, France 2 Groupe SII, Société d’Ingénieur et de Conseil en Technologies, Aix‐en‐Provence, France ali.oulhaci@lsis.org erwan.tranvouez@lsis.org sebastien.fournier@lsis.org bernard.espinasse@lsis.org Abstract: Serious Games (SG) are more and more used for training in various domains, and notably in the crisis management domain. Crisis management can gather several hundred stakeholders, which can present various difficulties when organizing field exercises. Serious Game constitutes a more practical alternative with specific benefits concerning detailed players’ actions tracking during a simulated exercise. Moreover, Non Played Characters (NPC) can be used to adapt the crisis management exercise perimeter to the available stakeholders as well as specific training objectives. In this paper we present a general Multi‐Agent System (MAS) architecture providing support to the behavioral simulation as well as the monitoring and assessment of human players. To each NPC is associated a so called Game Agent designed to reproduce the behavior of the actor simulated. The Game Agents are based on a deliberative model (Belief Desire Intention) with added editing features to facilitate the scenario design phase. Thus an Agent editor allows a designer to configure agents’ behaviors illustrated in this paper with the case of crisis management scenario. The behavior simulation was implemented within the preexisting SIMFOR project, a serious game for training in crisis management. Keywords: serious game, multi‐agents system, multi‐agent simulation, crisis management

1. Introduction Serious Games (SG) are more and more used for training in various domains, and notably in the crisis management domain. Crisis management can gather several hundred stakeholders, which can present various difficulties when organizing field exercises. Serious Game constitutes a more practical alternative with specific benefits concerning detailed players’ actions tracking during a simulated exercise. Moreover, Non Played Characters (NPC) can be used to adapt the crisis management exercise perimeter to the available stakeholders as well as specific training objectives. The work presented in this paper focuses on adding NPC capabilities to the pre‐existing SIMFOR Serious Game (simulation and training, SIMulation & FORmation in French) dedicated to training actors (with various level of expertise) involved in a Crisis Management situation. NPC capabilities implies being able to simulate humans’ behaviors with whom human players interact. This paper addresses the modeling and software requirements needed to support these objectives. A general Multi‐Agent System (MAS) architecture has thus been proposed providing support to the behavioral simulation as well as the monitoring and assessment of human players. To each NPC is associated a so called Game Agent designed to reproduce the behavior of the actor simulated (role incarnated). The Game Agents are based on a deliberative model (Belief Desire Intention) which is quite usual in Multi‐Agent Systems for complex behavior modeling (and simulation). This feature can be summarized in describing an agent as pursuing multiple goals (possibly with different priorities), which can be attained by plans composed of a sequence of actions either applied in the virtual world (3D environment) or resulting in interactions between agents/human players (with a self‐evolutionary response). To facilitate the scenario design, we have implemented an agent editor which will allow a designer to configure agents’ behaviors (as well as dialogues) for a SG scenario, applied here to Crisis Management training. The next section presents the SIMFOR project, a serious game for training crisis management. Section (3) discuss the NPC issue and relates to different works in the field of multi‐agent systems and behaviors simulation. In section (4), we define our Game Agent model for the SIMFOR project and in section (5), we present a short game scenario in the SIMFOR project to illustrate how the implementation phase is supported. Finally we conclude and present future works for the SIMFOR project.

465

M’hammed Ali Oulhaci et al.

2. The SIMFOR project In this section, we briefly present SIMFOR project, a serious game for crisis management as well as the general architecture combining Intelligent Tutoring System (ITS) and Serious Game (SG) elements.

2.1 SIMFOR context SIMFOR (figure 1) is a serious game developed by SII 1 company in partnership with Pixxim 2 company, in response to serious gaming call for project launched by the French Secretary of State for Forward Planning and Development of the digital economy. SIMFOR provides a fun and original approach for learning crisis management as a serious game. SIMFOR is adapted to actors’ needs and enables learners to train for major crisis management by integrating multi‐stakeholder aspect (i.e. heterogeneous learning profiles). The project objective is to create a training environment that immerse users in a crisis management situation in real‐time context and realistic in terms of environment, self‐evolving scenarios and actors.

Figure 1: Screenshot from SIMFOR project SIMFOR is a multi‐player game and allows different people to learn skill (shared or specific) in the same game. This is possible because SIMFOR does not target only the specialists in the field of crisis management, but rather the non‐professional. Managing a major crisis can mobilize several hundred stakeholders, from the regional Prefect in his office to the firefighter in the field. These stakeholders are required to communicate and work together in order to restore a normal situation.

2.2 The general architecture of the system The SIMFOR architecture combines elements from the Intelligent Tutoring System and Serious Game domains (see (Oulhaci et al., 2013a) for a detailed presentation). Our goal is to associate the playful learning of SG and the different modules of an ITS (domain model, learner model, pedagogical model) to get the optimal learning environment. The SIMFOR architecture is composed of the following components: The SG module (SIMFOR): this module includes the 3D models, user interface (as a communication channel between the learner and the system), simulation module (for natural phenomena such as fire propagation), and data models. This module constitutes the former ”perimeter” of the Simfor SG.

http ://www.groupe‐sii.com http ://www.pixxim.fr

466

M’hammed Ali Oulhaci et al. The Behaviors Simulation module: which allows simulating humans behaviors to replace absent players with ”artificial” actors (Game Agent). The Evaluation module: the evaluation module aims to provide skills assessment of players in real time to the pedagogical module. The Pedagogical module: which plays the role of a virtual tutor accompanying the learners by providing support and help during (and after) their training. Knowledge representation module: All knowledge used or produced by the previous modules of our proposed architecture is stored in the following models:

The Domain model: the domain model represents the different concepts of crisis management and it’s segmented into parts representing a role or a skill to learn.

The learner model: for each learner or agent, a learner model is associated. This model represents the mental state of actors at a time t.

As this paper focuses on simulation of human actors in a SG, the following section exposes the scientific issue of NPCs in the SIMFOR project and some relevant work in this field.

3. Adaptive NPC for SIMFOR The SIMFOR project faces two issues:

The simulation of human behavior of NPC players.

The monitoring and evaluation of learners during their training.

The learner assessment was discussed in (Oulhaci et al., 2013b) and (Oulhaci et al., 2013a). To deal the heterogeneous aspect of the learner assessment (assess different skills and trades), we have proposed the concept of the ”Evaluation Space”. The guiding idea is to consider a SG scenario through different view, each corresponding to a particular evaluation objective. An ”Evaluation Space” thus gathers (homogeneous) information and primitives to manipulate these information in order to produce assessments, such as a Behavioral Space (for evaluating procedural knowledge) or Social Space (for evaluating actors interaction during a game scenario). Adding NPC capabilities to the SIMFOR SG implies being able to simulate actors’ behaviors with whom human players interact (in the best case scenario without knowing the virtual nature or not of other players). This step requires extracting from domain experts nominal behaviors which players are expected to follow, and express them in suitable format. The challenge of behaviors simulation is how to transform an expert domain nominal behavior to a SG NPC behavior? Given the number of stakeholders’ as well as their skill heterogeneity, designing a scenario to establish crisis management exercises is a complex task. We present a scenario example in section 3.1 to discuss this issue.

3.1 NPC and complex behavior simulation As a SG, SIMFOR aims at immersing players in a virtual world enabling them to pretend acting as they would (and should) do in a real emergency situation. Knowledge and skills involved in such situation are various in nature as well as in terms of evaluation means, but nonetheless must be all assessed in order to certify (or not) that players know their part of the job on which many lives may depend. To better understand the heterogeneous aspect of the behaviors simulation as well as the assessment needs, let’s consider a simplified example of emerging situation scenario. This scenario starts with a TDM (Transport of Dangerous Material) truck overturned after a traffic accident. The tank is damaged and hydrocarbon is spreading over the road. A witness to the accident gives the alarm by calling the CODIS (Departmental Center for Operational Fire and Rescue Services in French) which in turn must perform four missions consequently to the alert. First, CODIS has to send a Firefighter on the scene to retrieve information about the accident (”send firefighter”). Once information on the accident is received (transmitted by the Firefighter in the ground), confirming a TDM accident has occurred, the CODIS must secondly gives instructions to an Officer (firefighter) on the measures to be taken. In a TDM accident the Officer must give the intervention order (send another Firefighter with a

467

M’hammed Ali Oulhaci et al. fire truck). Then thirdly, the CODIS must complete an information sheet on the disaster that passes later through a fax to the Mayor, Prefect and the Sub‐Prefect (sending order is not important). Finally the last mission is to inform the OCP officer (Operational Command Post) once it is sent by the Prefect. This scenario excerpt illustrates the needs of the domain, for behaviors simulation (define nominal agent behavior), as well as for the assessment process (if the CODIS is played by a human). Actors can play the same role (Firefighter), but enact different behaviors (one collect disaster information, and the other should intervene on the disaster). Moreover, we have a trades which do not belong to a sole organization (given the large number of stakeholders in crisis management), like School Principal, Mayor, media, etc. In addition, the actors’ behaviors may differ from a scenario to another (depending of the disaster nature i.e. fire, earthquake...). Therefore, the scenario designer (domain expert) must specify the actors involved their associated behaviors, disasters consequences, etc, for each scenario exercise. In the next section we present some works in the field of behavior simulation and multi‐agents architecture and how our work relates to these works.

3.2 Related work In this section, we present some representative works related to behaviors simulation in Serious Games (SG). The Artificial Intelligence (AI) has always been present in video games, more or less elaborated (depending on the video game objective) (Bakkes et al., 2009). SGs borrow much from classic video games, but SGs are more than game, a SG is a game that is used to learn something. The tools and mechanisms borrowed to the video game must be more elaborated, like AI. The AI is used to simulate natural phenomena or human behaviors to get a realistic virtual word for training (Zyda, 2005). In (Buche et al., 2003), Buche propose MASCARET, a pedagogical multi‐agent system for virtual environment for training. The MASCARET model is proposed to organize the interactions between agents and to endow them with reactive, cognitive and social abilities to simulate the physical and social environment. The physical environment represents, in a realistic way, natural phenomena. The social environment is simulated by agents executing collaborative and adaptive tasks. The MASCARET model was applied to SECUREVI, an application for fire‐fighters training. The MASCARET model allows designing a complex organization (by defining role, organization, behaviors ...), but is difficult to transpose in situations where many and heterogeneous organizations interact. For each organization, we must define roles and procedures etc, which are not adapted for larger scale crisis management exercises. In addition, the SECUREVI project does not include any assessment solution. More specifically on multi‐agent organizational modeling, the MOISE+ model (Hübner et al., 2002) (Model of Organization for multi‐agent SystEms) considers organizational structure and dynamics of a Multi‐Agent System or MAS (for example for simulation purpose). This model adds an explicit deontic relation (to structure and dynamics) to make the (artificial) agents able to reason on the fulfillment of their obligations or not. In these models, obligations and permissions are entitled to roles (such as a firefighter has to extinguish a fire and may use a fire hose which a bystander may not). As training may require actors to detect wrong behaviors (during a collaborative task between a simulated actor and human player), we would also need to allow (voluntary) erroneous behaviors which is not covered by MOISE+.

To simulate NPCs in the SIMFOR project, we have opted for BDI architecture. The BDI (Beliefs, Desires, and Intentions) model is an agent modeling standard in the field of agent behavior modeling, inspired from the human reasoning process (Rao et al., 1995), and has been widely applied. A BDI model is based on the notion of capacity, skills, beliefs, purpose, desire, and intention‐plan. Agents aim at achieving their goals by executing plans depending of their current knowledge (beliefs). These concepts allow designing and programming agents with complex behaviors. Our goal in the SIMFOR project is to provide an agent architecture helping the main expert to design NPCs for a specific scenario. These NPCs can have a nominal behaviors (perform the expected behaviors) or intentionally erroneous behaviors. The NPCs must also adapt their behaviors related to other players (learners) actions and interaction (social abilities) as well as events from the environment (3D world) that can occur during the game.

4. A BDI architecture for SIMFOR The proposed model is composed of a set of agent, actions and facts. Each SIMFOR scenario is associated with several agents model that reflect the NPCs behaviors. An agent model is represented as follows:

Model(role) = {Goals; Plans; Facts; Dialogues}

(1)

A Game Agent will play a role in a scenario, and as such tries to achieve Goals (activated i.e. evaluated as reachable,

468

M’hammed Ali Oulhaci et al. by the context) by enacting its associated plans depending on its knowledge of the situation (defined as a list of declarative Facts). Each plan is composed of actions either directed toward the environment or other agents/actors (causing different type of ”effects” on the scenario). In the former case, interactions between Game Agent/human actors (or GA/GA to fully simulate a scenario) are codified by adaptive Dialogues as a set of Sentences (see section 4.2). Figure 2 synthesizes the general structure of the GA model with an UML metamodel.

Figure 2: The SIMFOR Game Agent metamodel On an architectural or software level, an agent Engine is defined in order enliven the agents (i.e. enact the agents life cycle). A Game Agent (GA) begins to update its Facts Base through its perception of the environment (events) and messages from other agents (or players), and will select the appropriate goal according to the situation (role, ...) then select the intention (plan) that will achieve this goal and finally execute the plan. For goal selection, the GA selects first realizable goals by studying the plans feasibility for each goal. Once the realizable goals known, the GA selects the highest priority goal. If there are several goals with the same priority, the GA selects a goal randomly between the highest priority goals. This process enables an agent to adapt its behavior to the state of the environment in a broad sense (i.e. information about the virtual world and the other actors whether incarnated by agents or not). The objective of this model is to provide adaptive NPC behaviors, but we also help the scenario de‐ signer(s) with a user‐friendly and efficient tool to configure these NPCs behaviors. Thus, a graphical editor tool has been developed as illustrated in section 5.1 applied to the risk management case study.

4.1 Action modeling Actions characterize what an actor can do during a crisis management situation. These actions thus serve as model to design agents’ behaviors as well as assessment data when comparing what is done to what should have been done by human actors. Regarding the Game Agent, an action has Preconditions (expressed by a set of facts

supposed to be present in the agent Facts Base) and Effects (see fig. 2). An action can be performed in several ways: one shot (action undertaken only once), cyclic (repetitive action such as ”check fire progression”) or performed at a time t. For each action the designer can specify the number of attempts (if the number is reached without success, the agent goes to the next action). Actions influence the environment through three kinds of Effects:

Physical effects (PE): such action influences/impacts the SIMFOR 3D environment (as would a human actor through its user interface).

Knowledge modification effect (KE): direct consequence of an action is facts modification (i.e. knowledge update) which in turn can abort a goal or validate other goals or actions preconditions.

469

M’hammed Ali Oulhaci et al.

Message Effect (ME): such effect reflects the social nature of agent and actors as in the SIMFOR SG, they communicate in order to carry out the collaborative task of managing a crisis situation.

Table 1: Example of crisis management actions and their effects Action

Effect type KE, ME

Phone Fax

KE, ME Radio

KE, ME Talk

Move

KE, ME PE

Daybook

Description This action allows joining a player by phone. If both interlocutors are human player, the communi cation will be oral via VOIP (Voice over IP). If one of the interlocutors is a GA, the communication will be done as a dialogue with textual phrases exchange (see section 5.2) This action allows sending a fax to one or more recipients. Faxes are represented by preformatted HTML documents related to crisis management (to be filled in with the right informations) This action allows joining a players by radio. The player must then select a channel and press the talk button to communicate with all actors listening to this channel. To add a radio action into a plan, the designer must specify the channel and the id of the dialogue that will be used (by GA). This action allows talking with the nearest players (with a defined perimeter) This action allows to move in the virtual environment. The players can move using the mouse or automatically by selecting a chosen address in address book (either on foot or using a vehicle). the daybook simulates a web portal that allows stakeholders to relay information on the disaster The player writes the information to be shared and will be available to all stakeholders..

Table 2 illustrates actions with various effects as defined in the SIMFOR SG. Some actions may only carried out by GA (NPCs). These actions help to enrich the simulation and make it more realistic to human actors. For example, a (virtual) mayor assistant may prepare a room for a press conference. This action will take some time during which the (human) mayor can not start the press conference. These kinds of actions can also trigger some physical change in the SIMFOR 3D environment, for example activate an emergency siren, and put in place a foam pad (firefighter action). These virtual actions are defined by the scenario designer detailing their execution time, preconditions and effects.

4.2 Dialogue modeling There are several ways to interact in SIMFOR (phone, fax, radio...) but if one of the interlocutors is a GA, the communication is done in the form of textual dialogue. Partial automated interaction may result in fixed interaction lacking flexibility to reflect the various situations actors and agent may face. In order to avoid that, we have defined a dialogue process as a set of possible Sentences (see fig. 2 ), each one characterized with a context pertaining to the agents’ perception of their environment. A dialogue is thus designed as sentence

470

M’hammed Ali Oulhaci et al. tree where each node is represented by a sentence. A sentence is characterized by a list of properties described in table 2. Table 2: Structure of a sentence Sentence Attribute Id parent Children Content Preconditions Transmitted information Display type

Roles list

Description identify the sentence in the Dialogue tree list of (possibly) preceding sentences (parent nodes in the Dialogue tree) list of (possibly) following sentences (children nodes) represent the displayed text in the dialogue box contains the preconditions (set of facts) required for the GA to answer this sentence. represents the fact transmitted when the phrase is answered (by GA or human player) defines if the phrase will be displayed for all roles, only roles in the role list or all roles except the roles in the role list a list of roles involved in the sentence, used for display

In section 5.2, we will see a dialogue example and how the dialogue can be adaptive and interactive.

5. Implementation In this section we present how we carried out the agent model and the corresponding editor tool, on a software level.

5.1 The agent editor The agent editor allows the designer to set up the NPCs behaviors. The SIMFOR scenario specifies the actors, means and disasters involved during the game as well as the associated events (fire accident for example). For each SIMFOR actors (i.e. roles), the designer must associate an NPC. If the actor is not played by a human participant, it will be simulated by a GA, and conversely the GA will be disabled. Moreover, if the game begins with a NPC and a learner want to join in an ongoing game and play the role of the NPC, the GA will be disabled. Also, if a learner plays a role and for some reason leaves the game, the NPC takes over and plays his role.

Figure 3: The Game Agent edition For each SIMFOR scenario, the designer (domain expert) uses the agent editor to specify facts, actions for each agent (GA) involved in the scenario (figure 3). Facts are used for actions’ preconditions and effects, for goal preconditions and for dialogues. The Facts base (Base de faits zone) represents the facts known by the GA at the beginning of the game. Action(s) represent the actions that can be realized by the GA (these actions will be used to define plans). The trigger events (Evénements déclencheurs) are facts which can make Goals realizable (cf. agent engine in section 4). A goal is defined by its name, a priority, preconditions, previous goal (goals that must be previously completed) and finally a set of plans (permitting to achieve this goal). Each plan is

471

M’hammed Ali Oulhaci et al. represented by a set of (ordered) actions. We will see in more details the dialogues between human players and NPCs in section 5.2.

5.2 The dialogue editor In crisis management scenario, the different stakeholders must collaborate to restore the normal situation. To do this, there are several interaction processes between actors during the game. In accordance with the Dialogue and Sentence concepts defined in section 4.2, the designer can make rich and interactive dialogue with adaptive response. Within the agent editor, a dialogue editor helps the dialogue and sentence design (figure 4) and saves it in a XML format. We can also import dialogues (of phrase) to create and reuse more complex dialogues. Figure 5 presents an example of dialogue. This dialogue represents an interaction between the actor Codis and the actor Fireman1. The Codis must inform the fireman1 of the accident, and the fireman1 can answer with two choices: Ask for the road closures or not. The response of Codis actor will depend on its fact base (precondition road blocked). In figure 4 we can see the tree structure of the dialogue as well as the different sentences properties (id, content, precondition etc as described in table 2).

5.3 The agent/environment interaction The Agent model designed for behaviors simulation is implemented as a library and is completely generic. As agent actions may influence/modify the 3D environment, a communication interface with the SIMFOR 3D environment is required. This communication is based on Commands, which drive the behaviors simulation model (GA), but are also exploited as learner traces for the learner support and skill assessment, For example, when the GA wants to perform a move action, the Agent engine sends a Move command with the necessary parameters (destination, means of transport used). SIMFOR processes the request: if the 3D avatar of the GA is near the vehicle and the vehicle is available, SIMFOR carries out the trip (as a 3D animation), otherwise, SIMFOR sends an error command. This process can relate to the MASQ (Multi‐Agent Systems based on Quadrants) model (Stratulat et al., 2009). The MASQ model separates the agent mind (decisions) and the agent body (actions). A mind corresponds to the internal structure of an agent or to the decision‐making component. The body, either physical or social is parts of the environment and is connected to minds. As with SIMFOR, the mind of the agent is represented by the GA, and the body of the agent is represented by the 3D avatar in the virtual environment. In the ext section, based on the example presented in section 3.1, we illustrate how GA behaviors as well as interactions between players and GA can be edited.

Figure 4: The dialogue and phrase edition

5.4 Case study To illustrate the behaviors simulation, we present a scenario example defined by a domain expert, which describes the interaction between GA and human player. For this, we resume the scenario presented in section 3.1. Considering the CODIS role, figure 6 shows an UML activity diagram of the GA. With the given scenario (defined in section 3.1), we have defined and attributed five goals to the GA that play the role of the CODIS.

472

M’hammed Ali Oulhaci et al.

Figure 5: A dialogue example from the TDM scenario

Figure 6: Codis activity diagram The first goal ”send firefighter” is triggered when the CODIS receives the alert (from a witness of the accident), who then sends a firefighter at the disaster scene. This goal is reached by calling the firefighter and ordering him to go to the disaster scene (dialogue process). The second goal ”give instruction to the officer” consists in informing the officer of the initial actions to undertake concerning the TDM. The CODIS calls the officer and with a dialogue process gives the procedures to follow. The third goal ”Follow up the fire‐fighter for disaster information” will be triggered if the firefighter is slow to transmit information. This goal is achieved by a cyclic action repeated until the disaster information is received. The fourth goal ”warn municipal officials” consists in warning municipal officials by filling in the information sheet of the disaster and faxing it to the Prefect, Mayor and Sub‐prefect. If the CODIS is played by a human player, the information sheet will be filled in manually by the player, if not, the GA will use a pre‐filled sheet based on its Facts base content (disaster information). Finally the last goal ”inform officer for PCO place”, aims at informing the officer of the PCO location, by selecting the right address (found in the Facts base) and calling the officer. This Behavior model can be used for behaviors simulation, but also for learner assessment, as it is part of the domain model (Anderson, 1988) and is used as a reference (overlay model) for the learner model (VanLehn, 1988) (learner’s actions and knowledge). During the game, we can follow each GA actions (figure 7), its current goal, current plan and the content of its facts base. We can also modify the GA behavior through the interface (figure 7), for example we can reset a goal, add new fact (to trigger some goal), etc. This flexibility in the control of the NPCs is very useful because crisis management is a collaborative process, and the GA behaviors can influence the learner performance. In (Oulhaci et al., 2013b), we have presented different kinds of learner assess‐ment, for example the collective assessment assesses the global performance of all the stakeholders. This performance takes into account the human players (learner) as well as the NPCs.

473

M’hammed Ali Oulhaci et al.

Figure 7: GA monitoring

6. Conclusion With the growing interest in SG for training purpose, the behaviors simulation of NPCs is increasingly relevant. In this paper, we have presented how behaviors simulation could challenge SG into better training simulation. Consequently, in order to address this challenge, we have proposed a BDI‐like Game Agent architecture to simulate the NPCs. The goal of this implementation is to cover all trades of crisis management stakeholders and facilitate agent programming for better design of crisis management scenario. This Game Agent model is integrated into SIMFOR project, a serious game for crisis management. The GAs interacts with the SIMFOR environment through command system as well as human players through a dialogues system. This integration is used for behaviors simulation but also plays a role in the learners’ assessment. The crisis management is collaborative process, and the learners’ player and GA must collaborate to restore the situation. The GAs behaviors can influence learners’ performance and the GAs behaviors can be intentionally erroneous to evaluate the learners’ behavior in reaction to these errors. Our immediate work in the SIMFOR project is to focus on the collaborative aspects in the field of cri‐sis management, based on an analysis of the interaction graph permitting real‐time interpretation for better pedagogical support. On a more medium‐term perspective, primary feedback on our conceptual and architectural proposition reveals sufficient genericity to consider applying our approach to other SG training situation.

References Anderson. J, (1988). ‘The expert module’. Foundations of intelligent tutoring systems pp. 21–53. Bakkes, S. et al. (2009). ‘Rapid and reliable adaptation of video game AI’. Computational Intelligence and AI in Games, IEEE Transactions on 1(2) :93–104.

Buche, C. et al. (2003). ‘MASCARET : pedagogical multi‐agents systems for virtual environment for training’. In Cyberworlds, 2003. Proceedings. 2003 International Conference on, pp. 423–430. IEEE. Hübner, J. F. et al. (2002). ‘MOISE+ : towards a structural, functional, and deontic model for MAS organization’. In Proceedings of the first international joint conference on Autonomous agents and multiagent systems : part 1, pp. 501–502. ACM. Oulhaci, et al. (2013a). ‘Intelligent Tutoring Systems and Serious Game for Crisis Management : a Multi‐Agents Integration Architecture’. In IEEE International conference on state‐of‐the‐art research in enabling technologies for collaboration in CT2CM Track. IEEE. Oulhaci, et al. (2013b). ‘A Multi‐Agent System for Learner Assessment in Serious Games : Appli‐cation to Learning processes in Crisis Management’. In Seventh IEEE International Conference on Research Challenges in Information Science. IEEE. Rao, et al. (1995). ‘BDI agents, From theory to practice’. In Proceedings of the first international conference on multi‐agent systems (ICMAS‐95), pp. 312–319. San Francisco. Stratulat, et al. (2009). ‘MASQ : towards an integral approach to interaction’. In Proceedings of The 8th International Conference on Autonomous Agents and Multiagent Systems ‐ Volume 2, AAMAS ’09, pp. 813–820, Richland, SC. International Foundation for Autonomous Agents and Multiagent Systems. VanLehn K. (1988). ‘Student modeling’. Foundations of intelligent tutoring systems pp. 55–78. Zyda M. (2005). ‘From visual simulation to virtual reality to games’. Computer 38(9) :25–32.

474

Nuclear Mayhem – a Pervasive Game Designed to Support Learning Trygve Pløhn Nord‐Trøndelag University College, Steinkjer, Norway trygve.plohn@hint.no Abstract: Pervasive gaming is a new and emerging gaming genre where the games are not confined to the virtual domain of the computer, but integrates the physical and social aspects of the real world into the game and blends into the player’s everyday life. Games have in general proven to be useful in different types of learning situations. Given the nature of pervasive games, it may be possible to use that type of game as a tool to support learning in a university course by providing a gameplay where the students, by playing the game, expands the area of learning beyond the lecture hall and lectures and into the students everyday life. If this is possible, the area for learning will also become pervasive and be everywhere and anywhere at any time. To address this research area, a prototype of a playable pervasive game to support learning in university studies has been designed. The name of this game is Nuclear Mayhem and it is designed to support university studies in the development of Flash based Web‐games at the Nord‐Trøndelag University College, Norway. The game Nuclear Mayhem ran parallel with the course and started same day as the course began and ended when the students had completed the exam nine weeks later. The only mandatory activity the students had to do during the course was to participate in the game and to be allowed to attend the exam they had to complete the game within a given time limit. This paper presents the experimental pervasive game Nuclear Mayhem and how the game was designed to be pervasive and support the curriculum of the course. Analysis of log files showed that 87% of the logins in the game client was done outside of the time period that was allocated to lectures and lab exercises. Although most of the logins occurred during daytime, logins where registered in all the 24 hours of a day. These numbers indicate that the game became pervasive and a part of the students/players everyday life. The log files however also suggest that there is a need for a stronger link between the progress of the game and the course to get a better learning outcome. Interviews with the players indicate that they found the game exciting and fun to play, but that the academic tasks and riddles that they had to solve during the game was too easy to solve. The paper concludes that games such as Nuclear Mayhem are promising tools to support learning and transform the area for learning to become pervasive according to the players everyday life and suggest improvements in the game for the next versions. Keywords: pervasive games, education, serious gaming

1. Introduction The terms pervasive game and pervasive gaming is widely used on a lot of different types of games, toys and experiences (Magerkurth et al. 2005). Pervasive gaming is a gaming genre where the game is not confined to the virtual domain of the computer but extend the gaming experience out into the real world ‐ be it on city streets, in the remote wilderness, or a living room (Benford et al. 2005). The players must interact with the environment and with real objects to achieve certain goals (game objectives and missions). In contrast to traditional computer games, which take place in limited and well‐defined settings, pervasive games erase the boundaries between spatial, temporal, and social expansion (Lindt et al. 2007). Pervasive games are staged in reality and their main attractiveness is generated by using reality as a resource in the game (Waern et al. 2009). It can be difficult to motivate students to devote enough time working with the academic material in the curriculum throughout the courses. This leads to students not having the necessary academic maturity and understanding of the course material when the exam is approaching, and in spite of "pressure reading" the last week(s) before the exam it is ‐ for most of the students ‐ not possible to obtain a sufficient understanding of the subject to be able to get a good grade in the course. It is desirable that students work much more smoothly with the subject matter throughout the course instead of just "burst reading" when the exam is approaching. A solution to this problem might be to expand the area for learning outside of the lecture hall to become pervasive and enable learning to happen anytime and anywhere, for the duration of the course. Given the nature of pervasive games, being games where the players are in the game everywhere all the time for the whole duration of the game, such games may be useful as a tool and a platform to extend the area for learning beyond the lecture room and into the students everyday life in such a way that the area for learning becomes pervasive. The research question is therefore as follows: Can a pervasive game be used to expand the area for learning and awareness beyond the university classroom and into the students’ everyday life and enable learning to be anytime and anywhere?

475

Trygve Pløhn To address this research question a playable prototype of a pervasive game to support university studies was designed (Nuclear Mayhem) and the game went parallel with the course it was designed to support. The duration of the course, and thereby of Nuclear Mayhem, was nine weeks. The students had to complete the game within a given time limit to be allowed to attend the exam, and participation in the game was the only mandatory activity during the course. Apart from participation in the game, everything else was voluntary, including attending the lectures.

2. Related work There has been a lot of research on the use of games in education. Research has shown that games can be used to support teaching and learning (Jenkins et al. 2003) and that the use of games in education can improve skills in many different types of areas. Research done on how to use community‐building mechanics in games to achieve learning in education suggests that social gaming has the potential to revolutionize the way students learn (Hicks 2010). In higher education, games can be used in three areas (Wang and Wu 2009):

Games can be used instead of the mandatory/traditional assignments

Games can be used to increase participation and motivation of the students

Students may, by developing a game, learn about other topics such as for example game development, mathematics, physics, programming, game design and software development

Nuclear Mayhem tries to encompass all the three points above. There are no mandatory assignments or requirements in the course the game is designed to support except that the students must participate in the game and complete it within a specified deadline. Participation in the game is also intended to motivate students to spend more time on the subject, and as a part of the game, the player/student must develop a web‐based game and as a result of this learn programming and game development. Pervasive gaming is a research area that is becoming increasingly popular and more and more scientific articles are being published on this topic. Currently, pervasive gaming is mainly taking place within the research community and is not yet being widely used commercially. Research in pervasive gaming has so far been largely technology oriented where the motivation behind the development and design of the games has been to create games that are suitable as a platform for research on the technology one wants to explore. These include games like Mobio Threat (Segatto et al. 2008), The Drop (Smith et al. 2005), Epidemic Menace (Lindt et al. 2007) and Capture the Flag (Cheok et al. 2006) where the motivation mainly is to test and explore technologies and how technology can be applied to move games out in the real world. Research on the use of pervasive games to support learning or education (serious gaming) is a very interesting research area where some research has been done. The pervasive game The search for the professor (Spikol et al. 2009) was designed to introduce social web technologies and to support team building for a university course to beginning media technology students. The search for the professor shows promise to be a useful tool but needs a clearer integration into the course work. Some research has been done on how pervasive games can benefit from being game‐mastered rather than be fully automatic (Jonsson and Waern 2008). When a game is to be used in education it is critical that the game is aligned with the curriculum of the courses (Monroy et al. 2011). To ensure the best possible coordination with the topics of the course and the course progress Nuclear Mayhem had to be game‐mastered.

3. Nuclear Mayhem Nuclear Mayhem is a prototype of a pervasive game developed to support university studies in Multimedia and Web‐game technology at Nord‐Trøndelag University College, Norway. The pervasive game, Nuclear Mayhem, supports the course by providing a gameplay that is strongly related to the course syllabus. To be successful in the game, the players have to understand and master the topics in the syllabus. The game starts on the first day of the course and ends when the examination is conducted. During this period the players (students) are in the game 24/7 everywhere and anywhere, both when they are at the university taking classes, when they are at home in bed, out partying with friends or doing whatever students are doing in their leisure time.

476

Trygve Pløhn The game engages in the player's everyday life in many ways. Clues and tasks are found on Facebook, are sent by SMS to the player's phone in the middle of the night, is a part of the cityscape in Steinkjer (for example, tags that are placed in shop windows ‐ Figure 1), is a part of student life (some of the lecturers appear to have secret messages that players must obtain) and on the Internet (clues and tasks are spread across different websites that players need to find).

Figure 1: A paper note with a game clue is placed in the window at a hair salon in the main street of Steinkjer Several of the tasks that are carried out by the players during the game give points. For those players who are keen to win the game, it is an important strategy to be the first to perform the various tasks. The players can at any time see how many points they have in the game high score list. When the game is finished, the three players with the most points will be awarded.

4. Game story The game story was constructed from the ability to support the story by referring to real life events that had already occurred (reality hack), and the likelihood that something would happen related to the story that would be referred to by the news media (newspapers, television) and would be possible to implement as a part of the game story or game plot while the game was in progress. The strategy behind this design choice was that the use of real life events would help to create awareness about the game and make the game more pervasive and the game story more "real". At the time, Iran’s alleged nuclear weapons program was often mentioned in the news. There had already been a number of different events that could be used to substantiate the game story, and the issue seemed to be so relevant that it was highly probable that one or more events could happen during the game, that would be featured in the news media and thereafter could be implemented in the game. On the basis of these considerations the background theme chosen for the game story was Iran's nuclear program. Nuclear Mayhem and the academic connection The relationship between Nuclear Mayhem and the teaching of the curriculum of the course is illustrated in Figure 2. The top line represents the game Nuclear Mayhem and the bottom line represents the university course. The game and the course both start and end exactly at the same dates. The course starts with a presentation of the curriculum of the course and information on how to register as a player in the game Nuclear Mayhem. The students are told that participation in the game and completing the

477

Trygve Pløhn game within a given time limit is mandatory to be allowed to take the exam. After this information, the teaching of the course begins with the curriculum of week 1.

Figure 2: The alignment between the curriculum of the course and the pervasive game Nuclear Mayhem. To be able to complete level 1 in the game and register the secret code in the game client at the end of level, the student/player has to learn the topics that are taught in week 1 since the academic challenge in level 1 is directly related to this. Furthermore, the academic challenge at level 2 is directly related to what is discussed in lectures in week 2, the same for level 3/week 3, level 4/week 4 and so on. The last two weeks of the course, the students will conduct the exam project, which is to develop a functioning web game. At the same time period the players in Nuclear Mayhem have been assigned a mission where they shall create a web game that will be used to distract a guard, and that they – when the game is ready – will inform the saboteur group about this by registering the game URL in the Nuclear Mayhem game client. Registering the URL address in the game client completes the game. To succeed with Nuclear Mayhems main mission, the URL address must be registered within a given deadline, a deadline that coincides exactly with the deadline for submission of the exam project in the course.

Figure 3: The game client shows the countdown towards the deadline for both the game and the course This is the only time limit that exists in the game and the game client is reminding the players about this by displaying a timer that is counting down towards the deadline second by second.

478

Trygve Pløhn

5. Participants 17 students were attending the University Course that Nuclear Mayhem is designed to support, and all of the students were male. The age distribution of the students was from 20 years to 45 years. 17 students started Nuclear Mayhem and of those students, 15 live in the city where the game takes place and two students had to commute. A total of 16 students completed the game and of those, 14 students took the exam (included the two students that had to commute). None of the students had any previous experience with pervasive gaming.

6. Methods and procedure The nature of a pervasive game such as Nuclear Mayhem that is designed to be played anywhere at any time in the players everyday setting, is such that it is practically impossible to use an ethnographic approach where we study and observe the players while they are playing the game. An ethnographic approach to register and capture the players’ interactions with the game and all the potential situations of game play, would require that the players were observed 24 hours a day, both in their private and professional life, for the whole duration of the game. Furthermore it is not possible to study the players’ interactions with the environment and the environments ubiquitous artefacts since those are not directly accessible (Jegers and Wiberg 2006). Since an ethnographic method for evaluation is unsuitable, four other methods were used to overcome the methodological challenges:

a questionnaire

interview of selected individual players

system logs of user activities

observations made by game master during the game

The game was played during a 9 week period. In this period, all the activities that were done by the players via email, Web or SMS to interact with the game were logged. A questionnaire, with both open‐ended questions with free‐text answering and multiple‐choice questions, was used to capture the player’s subjective opinions of the game. The data collection was done via a Web‐based questionnaire that the players had to complete immediately after the game was completed. The questionnaire dealt with topics such as participation in the teaching program, previous programming experience, the use of mobile devices, previous gaming experience, the types of games you usually play, how you played Nuclear Mayhem, what you liked or disliked, what motivated or demotivated you to participate in the game and how the game managed to support the course. There are too few participants in the survey to have statistically significant results, but the answers still provide an indication of the players' attitudes and opinions. Based on the responses in the questionnaire five people were chosen for in‐depth interviews. The subjects were chosen based on their attitude towards the game. Of those who were selected for interview, there were two persons with a positive attitude, one person who was neutral and two persons who were negative to the game. The in depth interviews were conducted after the exam grade was set, and the interviewees were informed about this fact and that nothing they would say in the interview, whether it was positive or negative, would make any difference to their final grade. The interviews dealt with the respondents' general attitude towards games and the game Nuclear Mayhem in particular, how they felt about the game, the experience of the individual game plots, suggestions for improvements, the use of reality and the real world as part of the game and specific and detailed questions about what they perceived as good or bad during the game.

7. Results and discussion In the duration of Nuclear Mayhem the 16 players logged in at the game client a total of 610 times. Of those logins 80 of them where in the same time period as the lectures in the course (in the classroom or at the

479

Trygve Pløhn computer lab) and 530 logins were in the period where there were no lectures in the course (before or after lectures or at dates there were no lectures at all).

Figure 4: The percentage distribution of logins in the game client within and outside of the time period allocated to lectures in the course 87% of the logins that were made in the game client was outside the time period devoted to teaching the course. The fact that so much of the game client activity occurred outside the time allocated for lectures in the course, suggests that the game managed to expand the area for learning beyond the boundary of the lecture hall. Analysis of log files also show that although the majority of logins occurred in the period from 0800‐1600, players logged into the game client most of the 24 hours a day.

Figure 5: Number of logins in the game client during day and night time The fact that there was activity in the game client around the clock suggests that the game succeeded in becoming pervasive in relation to the player's everyday life in the period the game lasted. There was only one deadline in the game that players had to comply with and to get the best possible match between the teaching of the course and the game, the players had to play and complete the game as shown in Figure 3. Analysis of the log files indicate that not all players played the game as intended and this is illustrated in Figure 6. The figure above shows when the seven level‐codes were registered by each player in the game client. To achieve the best possible match between the game and the lectures in the course the line with the actual registrations and the desired registrations should match exactly. The reason that the lines do not match is that some of the players have completed the tasks in the game later than it was intended. The fact that some players did not play the game according to the planned schedule is unfortunate because they will then not be working with the academic challenges in the game at the same time that this is being taught in the course, and those students do not get the same learning outcome of playing the game as those who complete the game as planned. The use of a game story that is designed based on real events has certainly helped to reinforce the game story and make it more exciting and real. The use of real events, that was featured in newspapers and in news

480

Trygve Pløhn broadcast on TV while the game was in progress, as a part of the game, reinforced the game story, the game plot and created more awareness about the game.

Figure 6: Registration of the seven secret codes (level codes) in the game client Table 1: Quotes from the interviewed players/students on the topic of the use of the real world and real world events as part of the game and the game plot It makes it all the more realistic ... or more real ... For my part, I think it made it more real ... when you connect it to more realistic stuff so ... it increases the tension as well. ‐ Subject A It was a good plot, there was a lot of work put in it. It was good. It made it the more exciting ... ‐ Subject B The story itself was well supported, it was well made, and the details were good. It was exciting, it was. It increased the atmosphere of the game. ‐ Subject C I think that was good because it made the story more believable. Being able to read it in the newspapers made it a bit more credible. Real. ‐ Subject D ... story was perhaps a little too serious ... and then it might be a bit difficult to take it seriously … I think it might be good ... then it becomes a little more realistic ... ‐ Subject E

All the interviewed students were positive to the use of real events in the game, even those who did not like this type of game or were negative to the game. The only objection that was mentioned was a player who thought the game might be a bit too serious, but the same player also emphasized that this made the game more realistic. Whether the students/players experienced the game as fun to play or not, seems to depend on what kind of game they preferred initially. Those who are positive to this type of games experienced the game as very funny, while those who do not like RPGs were negative to the game even if they thought the game was well made. However, most of the players had some experiences in the game which they thought were interesting or fun regardless of whether they were positive or negative to the game. Table 2: Quotes from the interviewed players/students on the topic whether they experienced the game as fun game to play I think it was awesome ‐ Subject A ... I have never had a taste for RPG games, and maybe this game it's more like an RPG. ... It was very well made, but it was not for me. ‐ Subject B ... when you entered the code in the game client ...that was very exciting. It really was the highlight. That was fun. ‐ Subject C ... it was exciting to enter the code ... I knew it was correct but it was still ... hehe ‐ Subject D ... my biggest motivation was to win the game. ‐ Subject E

481

Trygve Pløhn Another factor that may have affect on the gaming experience is the fact that the prototype only had one way through the game that all the players had to follow, and that this could have lead to an experience of railroading (Jonsson and Waern, 2008), but none of those interviewed mentioned this as a problem. All interviewees mentioned that they got a learning benefit of participating in the game, but several of them pointed out that they felt they would have had a greater benefit with an ordinary arrangement of compulsory exercises. At the same time, several of the interviewees mentioned that their participation in the game resultet in them using more time trying to understand and review the programming code using the textbook than if they had not participated in the game. Table 3: Quotes from the interviewed players/students on the topic of the learning benefits of participating in the game ... the academic assignments were too easy ‐ Subject A ... to solve the problems in the game I used the textbook to compare the programming code and I learned something by doing this ‐ Subject B ... participation in the game was a motivating factor for me to sit down and study the programming code. ‐ Subject C ... I have learned about ActionScript 3.0 by participating in the game. I did not know how to program before. ‐ Subject D ... I would say that I have learned some ActionScript 3.0 and also something about pervasive gaming by participating in the game. ‐ Subject E

The learning benefit each player got from participating in the game also apperars to be dependent on their level of expertice. Hence it can be that each player would have a larger learning benefit with a more dynamic model that adapts the professional challenges to the player's skills. All players who finished the game managed to pass the exam.

8. Conclusion This paper has presented the concept and the prototype of the pervasive experimental game Nuclear Mayhem and showed how the game is designed to support learning in university studies. Participants in the game reported that they believe they gained a learning benefit from participating in the game but that there is a need for a stronger connection between the game and the course and a more dynamic academic challenge in the game that is better adapted to the players’ academic level. An instrument in ensuring a better match between the progress of the game and the lectures in the course would be to attach a deadline to each of the seven codes and a requirement for when each code has to be registered in the game client. A deadline on each of the seven secret codes will be easy to implement in the game. All of the players, including those who were negative to the game, mentioned that they experienced all or parts of the game as fun or motivational. Nuclear Mayhem has shown that this type of game has the potential to expand the learning space towards being pervasive and support and help learning, but to better evaluate the academic benefits of participation in the game there is a need to develop a model to measure the academic benefits of participation in this type of game in terms of the learning objectives in the course.

References Benford, S., Magerkurth, C. and Ljungstrand, P. (2005) 'Bridging the physical and digital in pervasive gaming', Commun. ACM, 48(3), 54‐57. Cheok, A. D., Anuroop, S., Lei, C. and Thang, L. N. (2006) 'Capture the flag: mixed‐reality social gaming with smart phones', Pervasive Computing, IEEE, 5(2), 62‐69.

482

Trygve Pløhn Hicks, A. (2010) 'Towards social gaming methods for improving game‐based computer science education', in Proceedings of the Fifth International Conference on the Foundations of Digital Games, Monterey, California, 1822386: ACM, 259‐ 261. Jegers, K. and Wiberg, M. (2006) 'Pervasive gaming in the everyday world', Pervasive Computing, IEEE, 5(1), 78‐85. Jenkins, H., Klopfer, E., Squire, K. and Tan, P. (2003) 'Entering the education arcade', Comput. Entertain., 1(1), 1‐11. Jonsson, S. and Waern, A. (2008) 'The art of game‐mastering pervasive games', in Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology, Yokohama, Japan, 1501803: ACM, 224‐231. Lindt, I., Ohlenburg, J., Pankoke‐Babatz, U. and Ghellal, S. (2007) 'A report on the crossmedia game epidemic menace', Comput. Entertain., 5(1), 8. Magerkurth, C., Cheok, A. D., Mandryk, R. L. and Nilsen, T. (2005) 'Pervasive games: bringing computer entertainment back to the real world', Comput. Entertain., 3(3), 4‐4. Monroy, C., Klisch, Y. and Miller, L. M. (2011) 'Emerging contexts for science education: embedding a forensic science game in a virtual world', in Proceedings of the 2011 iConference, Seattle, Washington, 1940845: ACM, 622‐629. Segatto, W., Herzer, E., Mazzotti, C. L., Bittencourt, J. R. and Barbosa, J. (2008) 'Mobio threat: A mobile game based on the integration of wireless technologies', Comput. Entertain., 6(3), 1‐14. Smith, I., Consolvo, S. and LaMarca, A. (2005) 'The Drop: pragmatic problems in the design of a compelling, pervasive game', Comput. Entertain., 3(3), 4‐4. Spikol, D., Pettersson, O. and Gerestrand, A. (2009) Designing Pervasive Games to Support University Studies in Media Technology, translated by 261‐263. Waern, A., Montola, M. and Stenros, J. (2009) 'The three‐sixty illusion: designing for immersion in pervasive games', in Proceedings of the 27th international conference on Human factors in computing systems, Boston, MA, USA, 1518939: ACM, 1549‐1558. Wang, A. I. and Wu, B. (2009) 'An application of a game development framework in higher education', Int. J. Comput. Games Technol., 2009, 1‐12.

483

StartUp_EU: Using Game‐Based Learning and Web 2.0 Technologies to Teach Entrepreneurship to Secondary Education Students Aristidis Protopsaltis1, Thomas Hainey2, Spiros Borosis3, Thomas Connolly2, Jesus Copado1 and Sonia Hezner1 1 Institut für Lern‐Innovation, Friedrich‐Alexander‐Universität, Erlangen‐Nürnberg, Germany 2 University of the West of Scotland, Paisley, Scotland, UK 3 Menon Network EEIG, Brussels, Belgium aristidis.protopsaltis@fim.uni‐erlangen.de thomas.hainey@uws.ac.uk thomas.connolly@uws.ac.uk spiros.borotis@menon.org.gr jesus.copado@fim.uni‐erlangen.de sonia.hetzner@fim.uni‐erlangen.de Abstract: The present paper reports on the pre‐piloting of the StartUp_EU Lifelong Learning European Project. StartUp_EU is a project for secondary schools that simulating the excitement and creative innovation of starting up a new company. The project developed a series of mini‐games, an encompassing Web2.0 platform and a collaborative competition for secondary school students, between 14 and 18 years of age, to inspire high tech entrepreneurship. The platform was based on the open source ILIAS Learning Content Management (LCM) platform and the mini‐games were developed in 3D Flash. Each mini‐game focused on a specific challenge/task associated with business start‐up. Students participated in a competition completing a series of 8 tasks simulating the process of arriving at an idea for a business to a real product and creating an elevator pitch. Students had to fill in a pre‐competition questionnaire at the beginning and a post‐competition questionnaire at the end of the competition. The questionnaires were completed online and the pre‐piloting lasted eight weeks, from the middle of January 2013 until the middle of March 2013. Forty seven (47) students took part in the pre‐ piloting with the vast majority of them never taken part in a competition about entrepreneurship and had limited or no experience with game‐based learning. This paper presents the results of the pre and post questionnaires. Keywords: game‐based learning, serious games, entrepreneurship, Web 2.0 technologies, education

1. Introduction Europe faces a number of challenges that can only be met if it has innovative, well‐educated, and entrepreneurial citizens who, whatever their walk of life, have the spirit and inquisitiveness to think in new ways, and the courage to meet and adapt to the challenges facing them (EACEA, 2012). It is critical for Europe to maintain a knowledge‐based economy and be at the forefront of technological, innovative entrepreneurship to maintain its competiveness. To meet these targets young people should study and seek careers in the scientific and technological fields and understand entrepreneurship. In a discussion entitled: ‘Educating the Next Wave of Entrepreneurs’ at the World Economic Forum 2011, it was concluded that the earlier an entrepreneurial spirit is encouraged the better the results for society. This project field was chosen to directly address the motivation of entrepreneurship and innovation within European secondary school students. StartUp_EU is designed to motivate secondary school students by simulating the excitement and creative innovation of a start‐up company. The project has created a series of educational games to foster the development of entrepreneurial skills on a Web2.0 technology platform where secondary school students will learn about entrepreneurship through inspiring and thought‐provoking videos, online workbooks covering business and marketing plans, and presentation skills. Students are then supported to develop their own business ideas collaboratively, and across Europe if possible. The process mirrors the idea creation, barriers and problems in developing new technology and building a company. The goal is to enable students to understand the problems and rewards of working in the exciting high tech area and inspire students to seek out careers in this vital European sector. Through reflection activities students will have the opportunity to understand what factors influenced their success or failure.

484

Aristidis Protopsaltis et al. According to the Key Competence Framework, the entrepreneurship key competence refers to an individual’s ability to turn ideas into action. It includes creativity, innovation and risk taking, as well as the ability to plan and manage projects to achieve objectives (Brag & Henry, 2011). The Web2.0 platform will provide a mechanism for students to turn their ideas into action and includes: communications, facilities for partner finding, inspirational videos, online materials, competition submission, and online presentation tools. The reality of creating a new idea, analysing its potentials and the real need, will be set against the problems and costs of developing the idea into a virtual product for a market. The students will have the option and ability to play the mini‐games autonomously. Digital computer games have now been around for over three decades and the term games‐based learning (GBL) has been attributed to the use of computer games that are thought to have educational value, however there has been much debate surrounding this theory (Pivec, 2009). Hainey, Connolly, Boyle and Stansfield (2011) show that GBL has been applied in a number of different fields such as medicine, languages and software engineering. Further research (de Freitas, 2006; de Freitas & Neumann, 2009; Egenfeldt‐Nielsen, 2005; Prensky, 2006; Squire, 2004; Squire & Jenkins, 2003) has shown that serious games can be a very effective as an instructional tool and can assist learning by providing an alternative way of presenting instructions and content on a supplementary level. GBL and Serious Games can promote student motivation and interest in subject matter resulting in enhanced learning effectiveness. Learning through games offers increased motivation and interest to learners through introducing fun into the learning process. Adding fun into the learning process makes learning not only more enjoyable and compelling, but more effective as well (Prensky, 2002, p. 4). One of the main characteristics of GBL is the fact that the instructional content is presented together with fun elements. A game that is motivating makes learners become personally involved with playing it in an emotional and cognitive way. By engaging in a dual level, learner attention and motivation is increased (Protopsaltis et al., 2011). Systematic literature reviews (Boyle et al., 2012; Connolly et al., 2012) have indicated that playing computer games confers a range of perceptual, cognitive, behavioural and affective, motivational impacts and outcomes where the most frequently occurring outcomes and impacts were affective and motivational followed by knowledge acquisition/content understanding. Computer games have also had an effect of students learning styles. Beck and Wade (2004) examined a large number of young professionals and found that their approach to learning was deliberately overlooking the structure and format of formal education. Young professionals extensively used trial and error, welcoming contribution and instruction from peers, and emphasising 'just in time' learning to fulfil their needs and complete their tasks. All of these skills are considered essential in the modern world and GBL can assist towards developing and practicing them. Furthermore, the next generation of jobs will be characterized by increased technology use, extensive problem solving, and complex communication (Levy & Murnane, 2004). These are skills that go beyond typical reading, writing, and arithmetic of years past. It is not only what students need to learn that is shifting, but also how and when they learn. Students of today are growing up with laptops, tablets, cell phones, and video calls, and they expect to use this technology in their daily interactions (NCREL & Metiri, 2003). Additionally, the skills required for success in games such as thinking, planning, learning, and technical skills are also sought by employers (Federation of American Scientists, 2006). Games are frequently cited as important mechanisms for teaching 21st century skills because they can accommodate a wide variety of learning styles within a complex decision‐making context (Squire, 2006), foster collaboration, problem‐solving, and procedural thinking (Johnson et al., 2011) which are important 21st century skills and important aspect of entrepreneurship.

2. Method 2.1 Participants Sixty three (63) participants, between 14 and 18 years of age, were registered to the StartUp_EU platform to participate in the initial piloting, forming 13 groups from 4 schools. The participants were recruited via their schools. The consortium partners contacted schools affiliated with their institutions to invite them to participate to the StartUp_EU pre‐piloting. 49 participants filled out the pre‐piloting questionnaire and after

485

Aristidis Protopsaltis et al. data refinement, 47 questionnaires were used for the analysis. 14 participants (29.8%) were female and 33 participants (70.2%) were male.

2.2 Material Different types of material were available to the students and teachers through the Web 2.0 platform. The materials were videos, mini‐games, how to guides and examples. Videos are videos of 2 to 3 minutes that set the task within the narrative framework and declare the task's goals and expected outcome. Mini‐games are self‐contained games to be played for stimulating certain skills to be applied in the challenge at hand and save the scores within the game. How to guides are short practical guides that assist in solving the tasks. There are three how to guides, one explaining the StartUp_EU competition, the second explaining the challenges and the third explaining how to use the platform. Examples are useful to enhance the quality of student work by modelling the expected output. The examples were taken from real IT companies (Apple, Dell, Microsoft, etc.). The game rules were presented in written documents describing the rules, roles, deadlines and assets. All the materials were made available through the StartUp_EU platform.

2.3 Apparatus Two types of apparatus were used in this study a Web 2.0 Learning Content Management System (LCMS) and five independent mini‐games. 2.3.1 Web 2.0 LCMS The StartUP_EU platform was based on the ILIAS 4.3.0 version. ILIAS is a SCORM compliant LCMS developed at the University of Cologne/Germany and is available as Open Source Software (OSS). The technical characteristics available to the users were the following:

Integration/upload of text, images, audio, video in various file formats including: (jpg, png, gif, mp3, wav, mp4, avi, pdf, doc).

File upload, material collection and creation of containers such as folders for topics. Uploads were restricted to a defined maximum size to prevent server overload.

Group communication enabled by the following facilities: messages that could be forwarded to a personal email account, forum, chat, communication with tutor/mentor/discussion board for individuals or teams to ask questions that could be answered by mentors or peers, a star system allowing students to rate each other’s responses and allowing mentors to provide qualitative feedback, group collaboration tools (wiki), links to Google Docs documents to allow collaboration, surveys to decide on questions within groups and a calendar to set deadlines for the submission of documents.

Group brokerage function ‐ groups can form online or offline, rights and roles are defined to support different users on the platform, groups are able to use a “notice board” to post “job offers” to recruit team members with particular skill sets and mentors/tutors can change the status of an individual’s group membership.

2.3.2 Mini‐games A set of educational mini‐games have been developed that allows students to practice and enhance their entrepreneurial skills as they develop their own business ideas collaboratively and autonomously across Europe. Mini‐games have been created to support a number of challenges including:

Sparking creativity

Building your company team

Understanding your clients

Marketing your product

How to develop your IT product.

Assessment in each mini‐game is through the use of quests and how well an individual or team performs determines their final score. Mini‐games can be played more than once to increase a player’s score (that is, increase their learning). For example, in the mini‐game ‘Understanding your clients’ the player plays the role of

486

Aristidis Protopsaltis et al. a salesman who sells a number of products. By driving a car through part of a city, the player has to find potential clients and match them to a suitable product. On finding a client, the player is allowed to ask a number of questions to help select the most appropriate product. Figure 1 provides an illustration of this mini‐ game.

Figure 1: StartUp mini‐game “Understanding your clients” As a second example, in the mini‐game ‘Marketing your product’, the player is given a marketing budget and has to identify characteristics of his clients that may indicate how the budget should be spent. For example, one client may travel by train and watch TV, while another might travel by train and use the Internet frequently. Given that two clients travel by train, this might suggest that some of the marketing budget should be allocated to train advertising with a lower spend on TV adverts and Internet adverts. Figure 2 provides an illustration of two screens for this game, one where the player is identifying characteristics of a client and the second screen showing an example of how the marketing budget is being allocated.

(a)

(b)

Figure 2: StartUp mini‐game ‘Marketing your product’: (a) finding out about the clients; (b) allocating the marketing budget

2.4 Procedure The students were set a series of 8 challenges, a preparation stage and a final stage which replicated the process of arriving at an idea for a business. The process involved building a business plan for taking an idea or pitch to the reality of a real product. Each challenge was related to a real problem or information needed to build the business plan and was introduced by a thought‐provoking video accompanied by a mini‐game and supportive material which explained in detailed how each challenge should be completed. All the material was uploaded onto the StartUp_EU platform and was accessible online. The pre‐piloting assessed three of the mini‐ games and the StartUp_EU platform.

487

Aristidis Protopsaltis et al. Students had to fill in a pre‐competition questionnaire at the beginning and a post‐competition questionnaire at the end of the competition. The pre‐competition questionnaire assessed their own experiences, attitudes, knowledge and skills, and their expectations of the StartUp_EU tools, while the post‐competition questionnaire focussed on assessing the usability of the tools, students’ motivation, pedagogy and the skills developed by the students. Additionally, both questionnaires assessed their perceptions towards the broad dimensions of entrepreneurship (EACEA, 2012). The questionnaires were completed online and the pre‐ piloting lasted eight weeks, from the middle of January 2013 until the middle of March 2013.

3. Results 3.1 Pre‐competition questionnaire 3.1.1 Experience with entrepreneurship education Students were asked to state to what degree they had had previous experience with entrepreneurship education. The metric used included the adjectives “never”, “1‐2 times per year”, “twice in a school term”, “once a month”, and “more than once a month”. There was also the option available not to declare anything as a separate option. The majority of participants (23, 48.94%) stated that they never had any experience of entrepreneurship education and 11 participants (23.40%) stated that they only had entrepreneurship education one or two times in a year. The results are as follows: Table 2: Experience with entrepreneurship education Past experience with entrepreneurship education More than once a month Once a month Twice in a school term 1‐2 times per year Never Not Applicable Total

Number 2 3 3 11 23 5 47

Percentage 4.26% 6.38% 6.38% 23.40% 48.94% 10.64% 100%

Students were also asked to describe briefly their previous experiences with entrepreneurship education programs. Some interesting comments presented by the Belgian students included the following: “For economy, we had to create a little company and create our own product.” “For an assignment for the economy class, we had to create a self‐made product. We also had to sum up all the costs and profits during the production.” “'Kinderen van dewindt' game, it was a fun game. You had to buy a building and start a business. Last year we played a game were we had to make our own company (buy a building hiring persons ...).” “We made a little virtual company and we made a business plan.” Students were also asked whether they have participated in a competition on entrepreneurship in the past, and if yes, whether they would do that again. Unfortunately only the 8,5% (4) of them had done that in the past, and these four people did not provide additional valuable information concerning their future intentions. 3.1.2 Experience with game‐based learning Students were asked to state to what extent they previous experience with GBL. The metric used included the adjectives “never”, “1‐2 times per year”, “twice in a school term”, “once a month”, and “more than once a month”. There was also the option available not to declare anything as a separate option. The results in Figure 3 indicate that the 40% of the students have no experience at all with GBL while another 31,91% only had experience once or twice per year.

488

Aristidis Protopsaltis et al.

Figure 3: Students past experience with game‐based learning 3.1.3 Self‐assessment of entrepreneurship own characteristics Based on research from the EACEA (2012) students were asked to self‐assess their own attitudes, knowledge and skills related to entrepreneurship. The skills included were: self‐awareness, self‐confidence, sense of initiative, risk‐taking, creativity, problem‐solving, knowledge on career opportunities and the world of work, knowledge of economic and financial literacy, knowledge on business organisation and processes, communication, presentation, planning, team work, exploring entrepreneurial opportunities, and design business projects. The cumulative results in percentages can be seen are shown in Table 3. Table 3: Students self‐assessment of their entrepreneurship related attitudes, knowledge, and skills. Self‐awareness Self‐confidence Sense of initiative Risk‐taking Creativity Problem‐solving Knowledge ‐ career opportunities and the world of work Knowledge of economic and financial literacy Knowledge on business organisation and processes Communication Presentation Planning Team work Exploring entrepreneurial opportunities Design business projects

Very low 0.00 0.00 0.00 0.00 0.00 4.26 0.00 2.13 4.26 0.00 0.00 0.00 0.00 6.38 2.13

Low Medium High 4.26 53.19 25.53 2.13 46.81 31.91 4.26 34.04 46.81 12.77 29.79 38.30 10.64 36.17 31.91 2.13 29.79 53.19 17.02 48.94 19.15 29.79 40.43 17.02 29.79 42.55 14.89 2.13 25.53 51.06 6.38 29.79 46.81 8.51 40.43 36.17 0.00 27.66 31.91 10.64 38.30 21.28 12.77 46.81 23.40

Very high 14.89 14.89 12.77 14.89 19.15 8.51 12.77 6.38 4.26 19.15 14.89 12.77 38.30 10.64 12.77

NA 1 2.13 4.26 2.13 4.26 2.13 2.13 2.13 4.26 4.26 2.13 2.13 2.13 2.13 12.77 2.13

The results revealed that students generally self‐assessed their entrepreneurship related attitudes, knowledge and skills from medium to high, with the only two exceptions being: knowledge of economic and financial literacy and knowledge on business organisation and processes which were rated medium to low. 3.1.4 Expectations from StartUp_EU Students were asked also to declare their expectations towards their participation in the StartUp_EU “exercise”. In particular, they were asked to declare which attitudes, knowledge and skills related to entrepreneurship they were expecting to develop through their participation in the StartUp_EU competition. The results are presented in Figure 3. Interestingly, the results showed that students expect – or would like – to develop mostly team working, creativity and problem solving skills through their participation in the StartUp_EU competition. Presentation skills, knowledge of business organization and processes, as well as their sense of initiative are close to the aforementioned first “group” of expectations with the rest of the skills, attitudes and knowledge. 1

NA: Not applicable OR I don’t answer OR I don’t want to answer

489

Aristidis Protopsaltis et al.

Figure 3: Students expectations in developing skills, knowledge and attitudes

3.2 Post‐test questionnaire Although a total of 63 students declared their participation initially to the StartUp_EU competition, only the 31 of them finally submitted the elevator pitch. Moreover, only the 27 of them completed the pre and post‐tests. This means that the analysis can be performed using the group who had filled in the pre and post‐test (27 participants) and the group with the additional 4 participants who had only completed the post‐test including that 27 giving 31 participants could be used for further exploration. Participants were requested to self‐ evaluate their participation in the StartUp_EU competition, filling in a questionnaire that included Likert‐type questions with the objectives of assessing the following:

Their intention to participate again in the StartUp_EU competition

Their usability assessment of the StartUp_EU platform

Their motivational usability to use StartUp_EU

The usefulness of StartUp_EU for learning entrepreneurship

The evaluation of the pedagogical model of StartUp_EU in its contribution of developing their entrepreneurial‐related knowledge, skills and competences

The separate evaluation of learning objects included into the StartUp_EU (videos, mini‐games)

Strongly, the assessment of the entrepreneurship‐related expectations was only a sub‐set of the complete study (analysis), which by nature was and should be more generic. The reason unfolds from the fact that, StartUp_EU employs a new approach to learning in the school‐level settings, which is currently applied to entrepreneurship learning. In line, and if effective, this approach could be used also to other key competences, like the social and civic competences. The group of 31 participants consisted of 19 males (61.29%) and 11 females (38.71%) and the group of 27 participants consisted of 11 females and 16 males (59.26%). 3.2.1 Intention to use StartUp_EU The students were asked whether they intended to use the StartUp_EU service to participate in the competition in the following year, given that they have access to the service for this period. The results were generally positive with the majority of the students from both groups specifying either strongly agree, agree or neutral. The results are shown in Table 4. 3.2.2 Usability assessment The usability assessment of the service was conducted based on the System Usability Scale (SUS) of Brooke (1996). SUS is a ten‐item attitudinal Likert scale that provides a global view of subjective assessments on usability. It is typically used after the respondent had had an opportunity to use the system being evaluated

490

Aristidis Protopsaltis et al. and yields to a single score representing a composite measure of the overall usability of the system being evaluated. SUS scores have a range of 0 to 100. In the case of StartUp_EU, the respective usability score was estimated to 63.87 for the 27 participant group and 63.15 for the 31 participant group out of 100 respectively, which is a generally positive result. Table 4: Intention to use StartUp_EU to participate in the competition in the next 12 months Answer NA or Empty Strongly Disagree Not Agree Neutral Agree Strongly Agree

27 participant group % 0.00% 0.00% 16.13% 48.39% 32.26% 3.23%

31 participant group % 0.00% 0.00% 14.81% 51.85% 29.63% 3.70%

The constituents of the usability score were also looked at to identify potential areas of improvement for the platform. Some of the key findings were the following:

6.45% in the 27 participant group and 7.41% in the 31 participant group found the service to be complex and that the functions were not very well integrated.

12.90% in the 27 participant group and 14.81% in the 31 participant group did not find the platform easy to use, stated that they did not feel confident using the service and thought that they had to learn a lot of things before they could proceed. Approximately 10% in each group stated that it was difficult to use and that they would like a technical support person to use StartUp_EU.

16.13% in the 27 participant group and 18.51% in the 31 participant group believed that most people would not learn to use StartUp_EU platform very quickly.

The results have shown that overall, minor changes are required to make the platform more appealing, less complex and to improve overall usability. 3.2.3 Motivational usability The participants were asked to assess the motivational usability of the StartUp_EU service. The respondents were asked to assess whether StartUp_EU service incorporates novel characteristics, whether it stimulated further inquiry and whether it is enjoyable and interesting. The results are generally positive showing that the StartUp_EU platform is generally enjoyable and interesting. The results are shown in Table 5. Table 5: Motivational usability of the StartUp_EU service

n=27

n=31

Question

Strongly Agree 7,41% 7,41% 3,23% 6,45% 6,45% 3,70%

Incorporates novel characteristics Stimulates further inquiry Is enjoyable and interesting Incorporates novel characteristics Stimulates further inquiry Is enjoyable and interesting

Agree

Neutral

18,52% 37,04% 58,06% 16,13% 38,71% 55,56%

59,26% 44,44% 29,03% 58,06% 45,16% 29,63%

Not Agree 11,11% 7,41% 9,68% 16,13% 6,45% 11,11%

Strongly Disagree 3,70% 3,70% 0,00% 3,23% 3,23% 0,00%

3.2.4 Usefulness of StartUp_EU The results indicate that approximately 77% of the students from both groups believed that the service is useful for learning entrepreneurship. Overall, the 51.62% from the 27 participant group and 48.15% from the 31 participant group believe that StartUp_EU is identical for entrepreneurship education. 3.2.5 Evaluation of entrepreneurship‐related characteristics The students were also asked to assess particular characteristics of the pedagogical model used that are related to entrepreneurship. The reader may remember that these characteristics were also assessed in the pre‐pilot phase (pre‐test questionnaire). The researchers decided to use the same measures so as to identify potential tensions in increase of these self‐assessment measures. To create more coherent results and identify the potential tensions, only the data from students having completed both the pre‐competition and the post‐ competition questionnaires are presented at this stage. The results are shown in Table 6.

491

Aristidis Protopsaltis et al. Table 6: Ratings of self‐assessed entrepreneurship characteristics in the pre and post‐competition Attribute Self‐awareness Self‐confidence Sense of initiative Risk‐taking Creativity Problem‐solving Knowledge of career opportunities/world of work Knowledge of economic and financial literacy Knowledge on business organisation and processes Communication Presentation Planning Team work Exploring entrepreneurial opportunities Design business projects

M before 3.59 3.81 3.70 3.63 3.37 3.81 3.30 3.11 2.89 3.78 3.74 3.70 4.19 3.19 3.30

M after 3.19 3.00 3.33 2.85 3.44 3.56 3.19 3.19 3.56 3.11 3.15 3.04 3.56 3.33 3.44

Difference ‐11.14% ‐21.26% ‐10.00% ‐21.40% +2.07% ‐6.56% ‐3.33% +2.57% +23.18% ‐17.72% ‐15.78% ‐ 17.83% ‐ 15.03% + 4.39% + 4.24%

The results suggest that participants became more hesitant towards entrepreneurship, however there is an increase in creativity, knowledgeable of economic and financial literacy, business organization and processes, inclination to explore entrepreneurial opportunities and design business projects. The decrease in scores of some of the characteristics was possibly due to lack of entrepreneurial experience of participants. 3.2.6 Evaluation of learning objects The participants were asked to assess the learning objects on the StartUp_EU platform, particularly, to assess whether the videos introducing each challenge were highly motivational, whether the mini‐games improved their learning of entrepreneurship, whether they were complicated, and whether they were useful in learning entrepreneurship. In terms of the videos, results indicate that participants did not agree whether the videos accompanying StartUp_EU were highly motivational or not, however slightly more of the participants indicated that they strongly disagreed or disagreed that the videos were motivational. The results are shown in Table 7. Table 7: Ratings of the introductory videos Rating Videos are highly motivational (n = 27) Videos are highly motivational (n = 31)

Strongly disagree ‐ disagree 45.16 % 44.44 %

Neutral 16.13 % 14.81 %

Agree ‐ strongly agree 38.71 % 40.74 %

Overall, there is a positive attitude towards the contribution and usefulness of mini‐games towards learning entrepreneurship and a quite clear opinion that the mini‐games are not complicated. The results are shown in Table 8. Table 8: Ratings of the attributes of the mini‐games Group Question The mini‐games improved my learning of entrepreneurship The mini‐games are useful for learning entrepreneurship The mini‐games are complicated

27 participant group SA A N 25.92 % 37.04 % 37.03 %

31 participant group SA A N 32.35 % 32.26 % 35.49 %

33.33 %

29.63 %

37.03 %

29.04 %

32.26 %

38.71 %

40.74 %

18.51 %

41.94 %

16.13 %

4. Discussion and conclusion This study focused on the assessment of the pre‐piloting use of the StartUp_EU service in school education. The objective was to acquire early input from the students so as to refine appropriately both the service and the evaluation tools. Interestingly, no major amendments to both questionnaires proved necessary. Though, interesting findings were surfaced for the StartUp_EU approach, despite the small sample (which hinders the generalization of the results).

492

Aristidis Protopsaltis et al.

4.1 Pre‐competition questionnaire The majority of the participants in the pre‐test (88.94%) stated that they had never had any experience of entrepreneurship education before or only experienced it once or twice a year. This education seemed to take place primarily in the context of an economy class, in a game to start a business or company and constructing a business plan. Only 8.5% of the participants had taken part in an entrepreneurship competition before suggesting that the StartUp_EU competition presents a novel learning opportunity for the participants. The majority of the participants (72.34%) also had never experienced or had only experienced GBL once or twice a year. This again indicates that the StartUp_EU platform presents a number of novel learning opportunities. Student self‐assessment of their entrepreneurship related attitudes indicated that they rated their skills from medium to high. Knowledge of economic and financial literacy and knowledge on business organisation and processes were the only exceptions to this and were rated medium to low. The results indicated that the students were relatively confident in terms of their knowledge and skills and had positive attitudes towards entrepreneurship. The participants expected to develop a number of skills from participating in the StartUp_EU competition including: team work, creativity, problem solving skills, presentation skills, knowledge of business organisation and processes and sense of initiative.

4.2 Post‐test questionnaire Students were generally positive about using the StartUp_EU service to participate in the competition in the following 12 months. The majority of the participants stated that they strongly agreed, agreed or were neutral indicating that the StartUp_EU service is appealing enough to get participants to use it in future competitions. In terms of usability of the service, the System Usability Scale score for both groups was approximately 64%, which is a generally positive result. Approximately 7% of the participants found the service to be complex with functions that were not well integrated. Approximately 14% did not find the platform easy to use and stated that they did not feel confident using the service and thought that they had to learn a lot of things before they could proceed. Approximately 10% in each group stated that it was difficult to use and that they would like a technical support person to assist to use StartUp_EU. 17% of participants believed that people would not learn to use the StartUp_EU platform very quickly. Overall, minor changes are required to make the platform more appealing, less complex and to improve overall usability. These results will feed into the next phase of platform development iteration. Participants generally found the platform to be interesting and enjoyable with regards to motivational usability. 77% of the participants believed that the service was useful for entrepreneurship education and approximately 50% of the participants believed that was identical for entrepreneurship education. With regards to participant self‐evaluation of entrepreneurship related characteristics, the results indicated that the participants became more hesitant however there was an increase in creativity, knowledge of economic and financial literacy, business organisation and processes, inclination to explore entrepreneurial opportunities and design business projects. The decrease in scores of some of the characteristics is somewhat alarming but that the consortium will look closely into it and will try to identify its causes. One possible explanation for this decrease might possible be the lack of previous entrepreneurial experience of participants which might led the participants to realize that entrepreneurial education and/or entrepreneurship was more complex than originally anticipated. However, the increase in self confidence regarding other entrepreneurship related aspects can offer a light of optimism since these characteristics are vital in today’s working environment and an integral part of entrepreneurship. In terms of the video learning objects, the results indicate that participants did not agree whether the videos accompanying StartUp_EU were highly motivational or not, however slightly more of the participants indicated that they strongly disagreed or disagreed that the videos were motivational. These results show us that additional or alternative videos are required. The Web 2.0 platform will be updated with more relevant videos in an effort to increase motivation and inspiration. These videos can be for example videos of well known successful entrepreneurs or motivational couches. Overall, there was a positive attitude towards the contribution and usefulness of mini‐games towards learning entrepreneurship and a quite clear opinion that the mini‐games are not complicated and the majority of participants believed that the mini‐games improved learning of entrepreneurship and that they were useful for learning about entrepreneurship.

493

Aristidis Protopsaltis et al. This paper has presented some of the results of a pilot study of a competition run on the StartUp_EU platform along with the learning objects presented on that platform in the form of accompanying material and mini‐ games. The results of the mini‐pilot have been analysed and used to improve the platform and learning objects for the main‐pilot which is currently in progress. Future research directions will involve completing the main pilot and analysing the results on a much larger scale to ascertain if the initial findings are consistent.

Acknowledgements This work has been co‐funded by the EU Lifelong Learning Programme under contract 518060‐LLP‐I‐2011‐1‐ UK‐COMENIUS‐CMP (StartUp_EU ‐ Be a High Tech Entrepreneur).

References Beck, J. C., and Wade, M. (2004). Got game: How the gamer generation is reshaping business forever: Harvard business school press. Boston, MA: Harvard Business School Press. Boyle, E., Connolly, T.M, Hainey, T., Hancock, F. and Boyle, J. (2012). "Engagement in digital entertainment games: a systematic review", Computers and Human Behaviour, Vol. 28, Issue 3, pp. 771‐780. Bragg, S., & Henry, N. (2011). Order 121 ‐ Study on Support to Indicators on Entrepreneurship Education: Final Report. Birmingham: GHK Brooke, J. (1996). "SUS: a "quick and dirty" usability scale". In P. W. Jordan, B. Thomas, B. A. Weerdmeester, & A. L. McClelland. Usability Evaluation in Industry. London: Taylor and Francis. Connolly, T.M., Boyle, E. A., MacArthur, E., Hainey, T. and Boyle, J.M. (2012). "A systematic literature review of the empirical evidence on computer games and serious games". Computers and Education, 59, 661 – 686. de Freitas, S. (2006). Using games and simulations for supporting learning. Learning, Media and Technology Special Issue on Gaming, 31(4), 343‐358. de Freitas, S. and Neumann, T. (2009). The use of ‘exploratory learning’ for supporting immersive learning in virtual environments. Computers and Education, 52(2), 343‐352. Education, Audiovisual and Culture Executive Agency. (2012). Entrepreneurship Education at School in Europe: National Strategies, Curricula and Learning Outcomes. Brussels: EACEA P9 Eurydice and Policy Support. Egenfeldt‐Nielsen, S. (2005). Beyond edutainment: Exploring the educational potential of computer games. University of Copenhagen, Copenhagen. Federation of American Scientists. (2006). Summit on educational games: Harnessing the power of video games for learning. Retrieved from http://www.fas.org/programs/ltp/policy_and_publications/summit/Summit%20on%20Educational%20Games.pdf Hainey, T., Connolly, T.M., Stansfield, M.H., and Boyle, E.A. (2011). The Differences in Motivations of Online Game Players and Offline Game Players: A Combined Analysis of Three Studies at Higher Education Level, Computers and Education, Vol. 57, Issue 4, pp. 2197‐2211. Levy, F. and Murnane, R. J. (2004). The new division of labor: How computers are creating the next job market. Princeton, NJ: Princeton University Press. North Central Regional Education Laboratory [NCREL], & Metiri Group. (2003). enGuage 21st century skills: Literacy in the digital age. Retrieved from http://pict.sdsu.edu/engauge21st.pdf Pivec, P. (2009). Game‐based Learning or Game‐based Teaching? Becta report. Prensky, M. (2002). The motivation of gameplay. On the Horizon, 10(1). Prensky, M. (2006). Don’t bother me mom, i’m learning. St. Paul, MN: Paragon House. Protopsaltis, A., Pannese, L., Pappa, D., and Hetzner, S. (2011). Serious Games and Formal and Informal Learning, eLearning Papers, elearningeuropa.info, n 25, (July 2011). Squire, K. (2004). Replaying history: Learning world history through playing civilization iii. Indiana University, Indiana, USA. Squire, K. and Jenkins, H. (2003). Harnessing the power of games in education. Insight, 3, 5‐33. Squire, K. (2006). From content to context: Video games as designed experiences. Educational Researcher, 35(8), 19‐29.

494

Measuring Effects of Reflection on Learning: A Physiological Study Wen Qi, Dominique Verpoorten and Wim Westera CELSTEC, Open University in the Netherlands, The Netherlands Wen.Qi@ou.nl Abstract: As an economical and feasible intervention, reflection demands learners using critical thinking to examine presented information, questioning its validity, and drawing conclusions based on the resulting ideas during a learning process. The aim of this study is to gain insight into the effects of practicing short, frequent and structured reflective breaks that are interspersed with the reading process of a learning material. It tries to reveal whether physiological signals can be used as appropriate indicators to reveal the actual changes of cognitive states while introducing different reflective breaks during learning. The recorded physiological signals include skin temperature, blood volume pulse, pulse volume amplitude, and pulse rate. The results show that while these embedded “reflection rituals” did not affect learners’ performance they had significantly impact on time on task, perceived learning and those learners' physiological (cognitive) states. Physiological data returned significant differences between the reading and reflection activity. Measurements of temperature and pulse rate are lower when covering the course equipped with additional reflection affordances while blood volume pulse and pulse volume amplitude are higher. In addition, applying statistics analysis to the physiological data exhumes significant differences between different types of reflection activities for those measurements including skin temperature, pulse volume amplitude and pulse. Keywords: reflective break, physiological signals, learning, reflection

1. Introduction Today’s teaching practice aims to educate the knowledge workers of the future how to master domain knowledge and develop transversal (domain‐independent) skills. The latter enables individuals to cope with requests for acquiring new knowledge and ongoing personal development. As an economical and feasible intervention, reflection is often embedded into a learning process and demands learners using critical thinking to examine presented information, questioning its validity, and drawing conclusions based on the resulting ideas. At the same time, learning through gaming format starts to gain popularity since it makes good uses of a competitive mechanism that pits the learners against each other or provides challenges for learners in order to motivate them to learn better. This paper probes two research topics in Gamed Based Learning: the effectiveness of physiological signals as indicators to monitor and correlate with the cognitive state during reflection and non‐reflection period and the potential benefits of tidy frequent and structured reflective breaks interspersed with the reading process of learning material to advance such a student/professional development.

1.1 Physiological signals and learning Scales or items referring to student workload or stress have commonly been recognized as an important variable in designing and proposing learning techniques. The major reason for measuring workload is to quantify the mental cost of performing learning tasks in order to predict learner performance. Mental workload can be affected by numerous factors that make it difficult to have definitive measurement. Physiological signals have been used by researchers as indicators of mental workload and stress for some time [Vicente, Thornton & Moray 1987; Wilson 2001]. Psychologists use physiological measurements as special identifiers of human emotions such as anger, nervousness, and sadness [Ekman, Levenson & Friesen 1983]. However, physiological data have not been employed widely to identify learners’ experience states, such as engagement and reflection.

1.2 Reflective breaks John Dewey [Dewey 1966] has stated” We do not learn from experience. We learn from reflecting on experience.” Reflective breaks have received attention from research when applied to face‐to‐face lectures [Di Vesta, Smith 1979; Ruhl, Hughes & Schloss 1987; Simpson 2004]. Despite the availability of theoretical models of reflection [Boud, Keogh & Walker 1985; Le Cornu 2009; Moon 1999] and a clear drift towards the promotion of thinking skills [Romainville 2007; Rychen & Salganik 2003], finding practical means to introduce the reflective habits to learners remains a challenge for researchers [Claxton 2006; Csap 1999] and

495

Wen Qi, Dominique Verpoorten and Wim Westera practitioners [Jaschnik 2011; Joseph 2003]. Researchers and practitioners have identified that there are several types of reflection breaks:

questioning: previous research highlights the importance of encouraging students to generate questions about the study material [Logtenberg, van Boxtel & van Hout‐Wolters 2011; Marbach‐Ad & Sokolove 2000; Pedrosa & Moreira 2009]. In this study, students deliberately and systematically exerted a questioning strategy called “student set the test”.

evoking: an evocation brings or recalls to the conscious mind what has been previously read. Conceptual works of the ”mind management” theory [Brown‐Frossard 2012; De la Garanderie 1989] suggest that this process of mental imaging allows readers to somehow transform what they have read into a mental object [Seel 2001; Vermersch 2009] and so doing to anchor it in their mind.

self‐assessing: research shows that self‐assessment can lead to significant enhancements in learning [Taras 2002] by developing students habit to evaluate the strengths and weaknesses in their own study.

2. Experiment design 2.1 Research questions In this comparative study, we have formulated two research questions that guided the experiment design.

We assume that the biofeedback measurement can bring extra information about possible contrasts between distinct activities performed within the learning process. Therefore, the first question was whether the absence/presence of reflection amplifiers impacts upon the physiological measurements of the control/treatment group. Reflection amplifiers refer to deliberate prompting approaches offering learners brief episodes of thinking while studying [Verpoorten, westera & Sprecht 2011].

In this study, we incorporate all three types of reflection breaks to establish learning as an object of attention and reflection and, so doing, to introduce students to essential components of academic literacy. For the questioning type of reflection in this study, students deliberately and systematically exerted a student set the test strategy. Therefore, the second research question was whether different types of reflective amplifiers do not have the same impact upon the physiological measurements.

Figure 1: The page design bundles content (purple frame) and affordances to develop thinking habits: reflective breaks (light green) and learning dashboard (dark green)

2.2 Method and materials The course designed for this study was a shortened version (1H) of the 4‐hour online course Seks en de evolutie (Sex and the evolution) created [Eshuis & Goltstein 2007] and offered in Dutch by the Open University the Netherlands. The course covered non trivial and interrelated notions and mechanisms as defined by Darwin and his followers: reproductive value, paternity uncertainty, mating strategies, differential investment in parenthood, etc. The course invited learners to use this theory as an interpretation grid of gender‐related behaviors observable in everyday life. The course consists of 5 chapters of 5 pages each, which contained about 200 words and one or two illustrations (Figure 1). In order not to bias the use of the different reflective breaks (see next section) by uneven levels of difficulty in the content, special attention was paid to ensure equivalence between all chapters. Each of them underwent the Flesch reading ease test [Flesch 1948] which returned an average comprehension difficulty level of 52 (SD = 4), which is comparable to the level of the Time news magazine. In addition, a systematic concept mapping procedure of each chapter ensured that they presented an even level of complexity regarding the number of new concepts introduced.

496

Wen Qi, Dominique Verpoorten and Wim Westera

Figure 2: The reflection portlets: question (2a), evocation (2b), self‐assessment (2c) The online course was delivered in the study at 2 different conditions: with and without reflective breaks (RBs). The study exposed participants to 3 types of RBs: questioning, evoking and self‐assessing. To support and condense the reflective processes of questioning, evocating and self‐assessing, 3 miniature Web applications (called portlets on the Liferay platform) were developed (Figure 2). They displayed, in a clear and graphical style.

The Question break portlet offered a note‐taking tool where the students wrote down their questions (Figure 2a).

The Evocation break portlet combined a “starts the evocation” button and a “stop the evocation” button (Figure 2b).

The Self‐assessment break portlet presented as a 5‐star visual scale (Figure 2c) that the students used to indicate their current level of mastery of a defined portion of content (for each level a standardized explanation was given).

Table 1: Compact view of the course chapters with offered reflective breaks Course chapter

Question breaks

Evocation breaks

Self‐assessment breaks

‐

Yes

‐

Yes

‐

Yes

During the study, the treatment group studied chapter 1 just like the control group: without any reflective break. This arrangement opened to participants a possibility of contrast within the learning experience and provided an internal yardstick to the chapters studied with support tools. In chapters 2‐3‐4, students got acquainted with one reflective technique (see Table 1). In chapter 5, all techniques were available. Based on their experience in the previous chapters, students could decide which one to use after each visited page. The students had to deliberately practice the offered RBs after each page visited or re‐visited. In order to consolidate this systematic reflective approach, a learning dashboard (Figure 3) was set up. It contained a built‐in reminder of the importance to practice the reflective breaks. A color scheme indicated whether or not the number of (re‐)visited page matched the number of use of the RBs. In case of match, the number appeared in green and in case of discrepancy in red.

Figure 3: The learning dashboard for chapter 4; In green, the number 4 mirrors that the student practiced self‐ assessment each time he/she visited a page of this chapter

497

Wen Qi, Dominique Verpoorten and Wim Westera

2.3 Physiological measures Physiological data was collected with the appliance Biofeedback 2000 x‐pert from SCHUHFRIED (Figure 4). This non‐invasive biofeedback system recorded the following physiological signals: a) skin temperature (TEMP), b) blood volume pulse, viz. the pulse component of the surface blood flow (BVP), c) pulse volume amplitude, viz. the amplitude of the blood volume pulse (PVA), and d) pulse rate (PR). The sampling pace was one measure every 25 milliseconds (Figure 5). The learning sessions of the students were also screen‐recorded with the software Camtasia in order to grab supplementary information about the sequencing of reading and reflecting periods.

Figure 4: During the experiment, the module is fastened with Velcro strap to the index finger of the non‐ dominant hand

Figure 5: Visualization of the 4 measured physiological signals for one subject (treatment condition)

2.4 Procedure Participants received a 15 euro iTunes voucher for their participation and were debriefed before leaving. Taken prior to the course, the background questionnaire evaluated the students’ pre‐knowledge of the course topic with 6 multiple‐choice questions. Meta‐cognitive ability was assessed for each student by their teacher on a 3‐item Likert scale. After a pre‐test, participants individually studied in one version of the course (with or without reflection breaks) according to a random distribution. Both groups were evenly invited to practice a thoughtful study freed from time pressure in order to gain as much mastery as possible of the learning material. The tracked data was the time in the course (total and per chapter), the number of pages visited (total and per chapter) and the number of time a reflective break was used. The logs also stored the choices made by learners in chapter 5 regarding the reflective breaks. After the course completion, students filled in a post‐test. This questionnaire gathered:

Evaluative feed‐back: open and closed questions collected students’ perceptions of overall satisfaction, sense of control, and feeling of learning. Questions relating to the instructional intervention were added for the participants to the treatment group.

Performance measures: a test assessed the knowledge and comprehension. Ten multiple choice questions were selected among a pool of questions tested by 137 students in a previous experiment based

498

Wen Qi, Dominique Verpoorten and Wim Westera on the same study material. The discrimination index was of .67 in average. For this index, values above 0.4 are desirable [McAlpine 2002], which located the test at a medium‐high level of difficulty. Three open questions asked students to comment on pictures with what they learned in the course. This was consistent with the design of the course that displayed carefully selected pictures on each page. A follow‐up questionnaire was administered one month after the experiment in an attempt to evaluate possible persistent effects. The follow‐up questionnaire asked students to give to an imaginary friend who ought to take the same course some advice regarding 8 study strategies, including the 3 reflective breaks. The perceived relevance of the strategies was rated with sliders on 100‐point scales, an asset available on the survey software Qualtrix.

3. Results The experiment convened 42 subjects at the same time. The sample population consisted of secondary‐school students physically present in computer rooms during the experiment. In that sense, the context of this study was close to regular schooling practice. It sought to provide more stable experimental conditions, more homogeneity in the sample and a contrast regarding the target audience of the reflective breaks. The data collected were the returns from the questionnaires (pre, post, follow‐up), the logging data and the physiological measures. Students who missed either the pre or the post questionnaire were removed from the analysis. It was the case for 2 participants in the treatment group. 40 test persons (mean age = 17 years old, 37% female, 63% male) composed the final sample: 21 participants in condition 1 (control) and 19 in condition 2 (reflective breaks). Table 2: Results for the physiological signals in with/without reflection amplifiers condition between the control and treatment groups) Results TEMP with RBs TEMP without RBs BVP with RBs BVP without RBs

Mean 33.65 30.98 49.37 49.52

Standard deviation (SD) 1.05 3.12 12.35 13.04

PVA with RBs PVA without RBs

31.18 34.53

19.57 24.8

PR with RBs

68.69

12.74

PR without RBs

60.11

12.85

p< 0.02 p<0.02 p<0.02 p<0.02

3.1 Physiological data Physiological data was collected from 4 students (2 at the control group without reflection breaks and 2 at the treatment group with reflection breaks) because limited amount of physiological sensors are available. T‐tests were conducted on 130773 paired sampled measures to compare TEMP, BVP, PVA and PR in the “with and without reflection breaks” conditions. This returned significant differences for the 4 physiological signals (Table 2). TEMP and PR are lower when covering the course equipped with additional reflection affordances while BVP and PVA are higher. The accuracy of the measure is acceptable (the observed difference in temperature is far above the variations that could be imputed to the recording system (0.01C) and the other measures embed compensations for interference and automatic averaging of data at baseline). Table 3: Mean and SD of the physiological signals with different reflection amplifiers TEMP (mean) (SD) BVP (mean) (SD) PVA (mean) (SD) PR (mean) (SD)

reading 33.350 1.193 49.334 11.184 28.415 17.928 66.893 11.553

questioning 34.055 0.590 49.441 17.524 45.817 25.509 68.721 17.096

499

evoking 33.722 0.717 49.389 11.276 26.190 15.747 70.912 12.383

self‐assessing 33.634 0.641 49.503 9.442 24.146 11.702 64.874 8.562

Wen Qi, Dominique Verpoorten and Wim Westera In order to refine the analyses, the screen recordings of the learning session from the subjects in the treatment group were analyzed to identify reading versus reflection periods. According to this timing information, the sampled physiological measures matching respectively each category were put together. Applying One‐Way ANOVA also exhumed significant differences (p<.0005) for the 4 physiological signals but with a slightly different pattern: in the periods of structured reflection (use of the reflection amplifiers), TEMP, PR and PVA are higher while BVP is lower (Table 2). Following the same process, the sampled measures corresponding to the periods of use of the different types of reflection amplifiers were contrasted against each other and with the reading activity (Table 3). Applying One‐Way ANOVA exhumed significant differences for three signals except BVP (p < .002).

3.2 Time on task Total time on task (Figure. 6) was descriptively higher in the group prompted to reflect (M = 52 min, SD = 9 min) than in the group without prompting (M = 26 min, SD = 12 min), and the difference was significant, t (38) = 7.46, p < .0001, d = 2.45.

Figure 6: Average time (in minutes) per chapter for the control and the treatment group

3.3 Performance Scores for the multiple‐choice questions revealed no significant differences between the control group (X = 4.5, SD = 2.24) and the treatment group (X = 4.7, SD = 1.59), t (38) = .41, p = .67, d = .08. A 3‐level scoring rubric was used to control the quality of the answers to the open questions:

trivial explanation of the picture;

explanation invocating the correct Darwinian concept;

explanation contextualizing the correct Darwinian concept in the overarching evolution theory.

The treatment group (X = 4.5, SD = 1.6) did not perform differently from the control group either (X = 3.7, SD = 1.7), t (38) = 1.54, p = .13, d = .05.

3.4 Feedback from learners 73% of subjects in the treatment group claimed that their learning experience in the course differed from their daily experience against 61% of subjects in the control group (relative percentages). Results showed that each of the 3 reflective breaks was foreign to about half of the sample population (Figure 7). 16% of the respondents answered almost never for all 3 reflective techniques.

500

Wen Qi, Dominique Verpoorten and Wim Westera

Figure 7: Familiarity level with the reflective breaks prior to the experiment

3.5 Perceived effect on time and mindfulness The reflective breaks were rated by the students on a 3‐point Likert‐type scale for their contribution to their study result and study time (1 = decreased the quality of my study/my study time, 2 = did not affect the quality of my study/my study time, 3 = increased the quality of my study time). Figure 8 illustrates the results.

Figure 8: Perceived contributions of the reflective breaks to study quality and time

4. Discussions In this study, we observed that physiological measurements differ between the conditions both at the global level of the course and when reading/reflection periods are contrasted. The observed variations indeed bring extra information to the study of reflection in formal learning. So far, reflective activity attached to this context has usually been inferred from performance changes or claims of students (scales, open questions) or think‐ aloud protocols [Veenman, Van Hout‐Wolters & Afflerbach 2006]. What we have learned here is that biofeedback measurements can be another dimension to the study of the phenomenon of reflection during learning. The findings of this study suggest that the cognitive states associated to different learning activities may be detected and recognized from physiological parameters. For instance, the externally‐imposed reflection remits seems to trigger internal responses traceable in physiological data. However, interpretation of these findings is not straightforward. It is not clear whether reflection is assimilated to some forms of

501

Wen Qi, Dominique Verpoorten and Wim Westera mediations, which can slow down certain body activities. Or on the contrary, reflection reverberates as a form of stress [McCraty, et al. 1999] because of its compulsory (this is an assignment) or/and perhaps unfamiliar character, which has effects on some physiological signals. Answering these questions goes beyond the scope of this study. It would require further interdisciplinary discussions combining both pedagogical and psycho‐physiological expertise. But, the findings related to the physiological measures in this paper should nevertheless be taken with prudence for the following reasons:

the results bear on a limited number of subjects with physiological measurements,

the huge amount of sampled observations can partly cause the significance.

It calls for further investigation into the relationships between reflection and learning performance through physiological measures that would be carried out with larger samples and with contrasted audiences of low and high performers in order to confront the way they study and practice reflection to their respective physiological coherence.

5. Conclusions Reflective break means to induce regular mental tingling for evaluating ones own learning, nurturing internal feedback [Butler & Winne 1995] and maintaining active commitment to the tasks at hand. This study has explored the potential benefits of reflective breaks for learning activities. The study also investigates whether physiological signals can be used as appropriate indicators to detect the actual changes of cognitive states while introducing reflection breaks during learning. The pattern of findings suggests that the benefit of a one‐ hour hand‐on session embedded with the reflective strategies is not significant in terms of enhancing cognitive performance. However, these reflective practices did bring an increased awareness and intensified their presence to the learning process. This study has demonstrated a physiological method that can be used to study the effects of reflection on learning. The study points out the future challenges faced by researchers while studying reflective break with physiological signals.

Reference Brown‐Frossard, R. (2012). Memorization Isnt A Lot Of Hocus Pocus. The International TEYL (Teaching English to Young Learners) Journal Retrieved June 1, 2012, from http://www.teyl.org/article14.html Boud, D., Keogh, R., & Walker, D. (1985). Reflection,Turning Experience into Learning. London: Kogan Page. pp. 100‐116. Butler, D. L., & Winne, P. H. (1995). Feedback and self‐regulated learning: A theoretical synthesis. Review of Educational Research Review, Vol. 65, No. 3, pp. 245‐281. Claxton, G. (2006). Expanding the capacity to learn: a new end for education? keynote speech, British Educational Research Association Annual Conference, University of Warwick, 6‐9 September 2005. Csap, B. (1999) Improving thinking through the content of teaching. In H. Hamers,J. van Luit & B. Csap (Eds.), Teaching and learning thinking skills (pp. 37‐62). Lisse: Swets and Zeitlinger. De la Garanderie, A. (1989). Defense et illustration de l’introspection [Defense and illustration of introspection]: Le Centurion. Dewey, J. (1966). Democracy and Education: an introduction to the philosophy of education. Free Press, New York. Di Vesta, F. J., & Smith, D. A. (1979). The pausing principle: Increasing the efficiency of memory for ongoing events. Contemporary Educational Psychology, 4(3), pp.288‐296. Ekman, P., Levenson, R.W. & Friesen, W.V. (1983). Autonomic Nervous System Activity Distinguishes among Emotions. Science, 221(4616): p. 1208‐1210. Eshuis, J. H., & Goltstein, G. P. H. (2007). Seks en de evolutie. Retrieved 18‐02, 2011, from http://www.ou.nl/eCache/DEF/2/22/850.html Flesch, R. (1948). A new readability yardstick. The Journal of applied psychology, 32(3), pp.221‐233. Jaschsik. (2011). Can students learn to learn? Inside Higher ED Retrieved June 1, 2012, from http://www.insidehighered.com/news/2011/01/31/ colleges try to use metacognition to improve student learning Joseph, N. (2003). Metacognitive awareness: investigating theory and practice, Academic Exchange Quarterly Retrieved June 1, 2012, from http://findarticles.com/p/articles/mi hb3325/is 4 7/ai n29059143/ Le Cornu, A. (2009). Meaning, Internalization, and Externalization. Adult Education Quarterly, 59(4), pp.279‐297. Logtenberg, A., van Boxtel, C.,& van Hout‐Wolters, B. (2011). Stimulating Situational Interest and Student Questioning through Three Types of Historical Introductory Texts. European Journal of Psychology of Education, 26(2), pp.179‐198. Marbach‐Ad, G., & Sokolove, P. G. (2000). Can undergraduate biology students learn to ask higher level questions? Journal of Research in Science Teaching, 37(8), pp.854‐870. Moon, J. (1999). Reflection in Learning and Professional Development. London: Kogan Page. McAlpine, M. (2002). A summary of methods of Item analysis . Bluepaper Number 2, University of Glasgow. ISBN 1‐904020‐ 02‐X.

502

Wen Qi, Dominique Verpoorten and Wim Westera McCraty, R., Atkinson, M., Tomasino, D., Goelitz, J., & Mayrovitz, H. (1999). The impact of an emotional self‐management skills course on psychosocial functioning and autonomic recovery to stress in middle school children. Integrative Physiological and Behavioral Science, 34(4), pp.246‐268. Pedrosa de Jesus, H.,& Moreira, A. C. (2009). The role of students questions in aligning teaching, learning and assessment: a case study from undergraduate sciences. Assessment & Evaluation in Higher Education, 34(2), pp.193‐208. Romainville, M. (2007). Conscience, metacognition, apprentissage : le cas des competences methodologiques [Consciousness, meta‐cognition, learning: the case of the methodological skills]. In F. Pons & P.‐A. Doudin (Eds.), La conscience chez l’enfant et chez l’lve [Consciousness in the child and in the student] (pp. 108‐130). Qubec: Presses de l’Universit du Qubec. Rychen, D. S., & Salganik, L. H. (2003). Key Competencies for a Successful Life and a Well‐Functioning Society: Hogrefe Publishing Ruhl, K. L., Hughes, C. A., & Schloss, P. J. (1987). Using the Pause Procedure to Enhance Lecture Recall. Teacher Education and Special Education: The Journal of the Teacher Education Division of the Council for Exceptional Children, 10(1), pp.14‐18. Simpson, W. (2004). The pause method in undergraduate auditing: An analysis of student assessments and relative effectiveness. In J. Ketz (Ed.), Advances in Accounting Education Teaching and Curriculum Innovations (Vol. 6, pp. 69‐ 85): Emerald Group Publishing Limited. Seel, N. M. (2001). Epistemology, situated cognition, and mental models: Like a bridge over troubled water. Instructional Science, 29(4), pp.403‐427. Taras, M. (2002). Using Assessment for Learning and Learning from Assessment. Assessment & Evaluation in Higher Education, 27(6), pp.501‐510. Vicente, K.J., Thornton, D.C. & Moray, N. (1987). Spectral Analysis of Sinus Arrhythmia: A Measure of Mental Effort. Human Factors, 29(2): pp.171‐182. Verpoorten, D., Westera, W., & Specht, M. (2011). Reflection Amplifiers in Online Courses: A Classification Framework. Journal of Interactive Learning Research, 22(2), pp.167‐190. Veenman, M., Van Hout‐Wolters, B., & Aﬄerbach, P. (2006). Metacognition and learning: conceptual and methodological considerations. Metacognition and Learning, 1(1), pp.3‐14. Vermersch, P. (2009). Describing the Practice of Introspection. Journal of Consciousness Studies, 16(10‐12), pp.20‐57. Wilson, G.M. (2001). Psychophysiological Indicators of the Impact of Media Quality on Users. In Proceedings of CHI 2001 Doctoral Consortium. Seattle, WA, USA. ACM Press. pp.95‐96.

503

Evaluation of Adaptive Serious Games using Playtraces and Aggregated Play Data Christian Reuter, Florian Mehm, Stefan Göbel and Ralf Steinmetz TU Darmstadt, Darmstadt, Germany christian.reuter@kom.tu‐darmstadt.de florian.mehm@kom.tu‐darmstadt.de stefan.goebel@kom.tu‐darmstadt.de ralf.steinmetz@kom.tu‐darmstadt.de Abstract: Adaptive Serious Games often feature complex algorithms and models, which influence the player’s progression through the game. These models include properties like pre‐existing knowledge or preferred playstyle and are matched with a pool of appropriately annotated parts of the game, such as assignments or scenes, during runtime. While being transparent for players, these models must be visualized for testing and evaluation purposes. In order to allow authors the retrospective interpretation of playtraces generated by a gaming session, we developed a replay component for adaptive serious games created with the authoring tool “StoryTec”. This method removes the need for continuous observation of individual players while retaining the same level of detail and being much more understandable compared to log files, especially for the non‐programming audience addressed by StoryTec. In addition to showing the player’s view, the state of the internal models and the progression through the story structure are also visualized. Sharing the same models and data structures as the authoring tool and making their runtime behaviour visible to the author, the replay component is therefore able to offer additional benefits compared to more generic methods like screen capturing or key recording tools. A complementary tool which is able to aggregate a large number of playtraces into one comprehensive spreadsheet for statistical analysis was also implemented. This allows authors to gain an overview over a great number of players in a shorter time compared to investigating them individually. In order to reduce the complexity of the result, the table contains aggregated information like the total time the players spent in each scene or the final value of variables at the end of their sessions. If authors detect an anomaly, they can then access more detailed information by loading the original traces into the replay component, which uses the same data format. Together these two components support the evaluation of adaptive serious games by means of user studies with the intended target audience, for example pupils. By combining them with our testbed for rapid prototyping named “StoryPlay”, we were able to provide a set of tools covering a broad range of evaluation tasks based on the same underlying models and data formats. Using these tools, it is possible to gain insights on how the adaption algorithms behave over a large number of players, e.g. which paths were taken by how many players or whether the time to solve a task as estimated by the author was matched. Keywords: Serious Games, evaluation, adaptation, playtrace, testbed

1. Introduction While the idea of Serious Games, i.e. games which serve an additional purpose besides entertainment like education or health, becomes more and more popular, there are still critical voices who question the effects of these kinds of games. Therefore it is necessary to provide hard evidence in this regard in order to fully establish this idea, especially when those games are meant to provide game‐based learning and training. This is usually done by means of evaluation studies, where the game is played by a large number of players and their results and / or gameplay experiences are recorded. Besides assessing the outcome of the games, user studies also allow the game’s creators to identify technical as well as design problems and measure the acceptance by the intended target audience. There are already many generic tools and methods available that support user studies of software in general as well as games and Serious Games in particular. Adaptive Serious Games are games that contain adaption mechanism that allow the game to alter itself depending on the player using it. This can happen by simply personalizing the presentation based on the user’s preferences or by taking his pre‐existing knowledge into account and restructuring the order and overall number of assignments / game scenes accordingly. When evaluating adaptive games, it is important to not only record the player’s view, but also the state of the underlying adaption models and algorithms, giving insight into how these behave over a wide range of users. An obvious benefit of this approach would be a case, where an adaptive game with multiple paths through its story is evaluated, showing that some paths are never chosen by the adaption engine. In order to assess whether this is an error and how to fix it, a close observation of the internal states is necessary.

504

Christian Reuter et al. The current state‐of‐the‐art regarding evaluations can be separated into two groups of methods. Generic methods and tools like observation, screen capturing or questionnaires only record the “outsiders view” and therefore work with arbitrary games. In contrast, specific approaches must be tailored towards a concrete game, but allow the recording of internal information. To fully evaluate adaptive games, it is therefore necessary to employ specific methods, which are closely linked to their internal models. A drawback of specific methods however is that they offer limited reusability and often must be adjusted for each game they are applied to, creating a trade‐off between the amounts of information an evaluation tool is able to provide and the costs associated with its use. In order to address this problem we therefore developed two tools based on the adaption models of our authoring tool “StoryTec” (Göbel et al. 2010). Since the games created with it are based on the same adaptation models, it is possible to evaluate all of them using the same set of tools. The tools also aim at reducing the effort for evaluating complex games with many players and are intended to be easily used by single non‐programming authors. In this paper we first discuss the current state‐of‐the‐art in regards to evaluation methods and tools for adaptive Serious Games. We then give a short overview over the authoring tool “StoryTec” and the associated “StoryPlay” Testbed, which build the foundation for this work. After that we describe our evaluation framework, consisting of a replay component and an aggregation tool for playtraces. We also show how our tool is used in the ongoing evaluation of a learning game for mathematics, focussing on explorative results like the overall playtime or anomalies in player behaviour.

2. Related work A good overview of software evaluation in general can be found in (Hilbert & Redmiles 2000). They argue that user interfaces automatically generate events that can be recorded and describe different types of evaluation goals and event data. Based on this the authors compared tools and techniques, which are able to analyse these recordings and can extract higher level information from them. Examples include the synchronization with other data sources such as video recordings, transformation (e.g. filtering), summarization (to decrease the amount of data) and the detection / comparison of subsequences of events. Another overview aimed specifically at games was done by (Nacke et al. 2009), differentiating between playability, which focusses on the game itself, and player experience, which concerns the player’s interaction with the game. They argue that good playability is necessary to conduct studies on player experience. The work then lists a number of methods for both types of evaluations. While playability is measured against heuristics, player experience can be assessed by a number of different objective (like biofeedback, gameplay data) and subjective methods (questionnaires). It is concluded that a combination of several methods yields more information and is therefore advisable. (Nacke et al. 2010) follows up on the topic of player experience in the regards to Serious Games. They also list several evaluation methods, grouped in individual (e.g. psychophysiological, gameplay data, player modelling and questionnaires) and context oriented (playability heuristics, also questionnaires) ones that take the environment in which the game is played into account. (Bruder et al. 2004) describes a certificate for computer‐based learning environments, which also takes the learning effects and outcomes into account – a criterion for Serious Games in particular. Individual approaches that work with games in general include work by (Ketkar & Youngblood 2010), where the movement of players in a 3D world was recorded. After that graph based algorithms were used in order to build player profiles. (Liu et al. 2011) used heatmaps to visualize players’ progress in games with discrete states, needing only the states and transitions between them as an input information. They also proposed the mapping of continuous spaces to state features, making their approach viable for games without discrete states while also defining higher level information. This mapping however must be done manually and the results of the visualization are highly dependent on the mapping. (Kim et al. 2008) developed a testing framework which can be used to interpret user triggered events, which are logged by a game. It is able to visualize anomalies on different levels of granularity. When linked to a video recording, they were able to find the reason behind sudden difficulty spikes and changed their game accordingly. They noted that it is important to decide which events to record, that the sequence of events is very important and that only the number of occurrences is often insufficient.

505

Christian Reuter et al. One example where data analysis was conducted for a commercial game can be found in (Medler et al. 2011). They used a graph based visualization of events to balance two heterogeneous teams in a multiplayer setting. Their server‐based system was directly integrated into the development process. Practical insights for the development of such tools were noted and clustered in the categories production (for example to build the tool parallel to the game), functionality (enabling faster analysis by aggregation) and game team integration (involving the game team in the tool development process). Other work by (Hullett et al. 2012) gathered game data from real players spanning 3 years after release of a game via the internet, requiring no additional collection setup. Their analysis produced insights for future games, such as popular or unused game modes and elements – to pick their focus for future releases to decrease development costs. Another visualization tool for game data is “StoryPlay”, formerly “Bat Cave” (Mehm et al. 2010). This tool is tailored for adaptive Games created with the authoring tool “StoryTec” and that visualizes the internal game state. Since the tool is closely linked to this work, it will be discussed in the next chapter.

3. Background The evaluation framework we describe in this paper is based on the internal models of the StoryTec‐authoring environment on the one hand and on the visualizations of the StoryPlay‐testbed on the other hand.

3.1 Adaptive games in StoryTec Games – or stories – build in StoryTec offer adaptation along three dimensions: the learner, player and story model (Göbel et al. 2010). The learner model takes the pre‐existing knowledge and dependencies between individual skills into account. Its main task is to make sure that the players do learn all skills taught by the game while guaranteeing that the learner has the right perquisites to understand this knowledge. To represent the dependencies between individual skills, the learning context is modelled as a graph based on the “Competence based Knowledge Space” (Korossy 1999). The player model is based on (Bartle 1996) and maps the players behaviour to four playstyles, “killer”, “achiever”, “socializer” and “explorer”. This allows different representations of same content, fitting the players preferred style of play. Each of these styles is modelled by a number in the interval [0, 1], based on how well the style fits the player. Lastly the story model uses a modified version of the Hero’s Journey (Göbel et al. 2009) to balance between adaptation and an interesting story as defined by the author. The fixed order of event types (like “call to adventure” or “return”) from the Hero’s Journey is preserved while allowing the events themselves to play out differently. During game creation, the author is able to annotate every scene along these three dimensions, for example describing which learning content they provide and thereby making them “Narrative Game‐based Learning Objects”. When the game is played, these models are constantly updated based on the players progress and behaviour by the Story Engine. Whenever a free transition to another scene is triggered, the engine calculates a score for every potential successor based on their annotation and the current state of the models as well as individual weights for each dimension. The scene which fits the player best is then selected.

3.2 Testbed StoryPlay When prototyping adaptive games created with StoryTec, it was necessary to visualize the state of these internal models. This made it possible to understand decisions of the Story Engine and, if the results seemed undesirable, change the game or the adaption algorithms accordingly. Since the models may seem very complex, especially for single untrained authors, we developed a rapid prototyping environment which is able to visualize the state of the models and the decisions of the Story Engine in an understandable way (Mehm et al. 2010). The testbed works directly on the story files created by StoryTec and is linked to the same models, reducing effort for rapid prototyping and eliminating the risk of mismatches or loss of information between the two tools.

506

Christian Reuter et al.

4. Evaluation framework Most evaluation methods described in the related work require internal play data from the game, which should include the state of the internal models in adaptive Serious Games. Since this data is already visualized by the testbed, our first approach used StoryPlay for small user studies. When working with practitioners it became clear however, that a rapid prototyping session by an author has different requirements than a general evaluation tool:

Authors usually work alone or in small groups, while evaluations ideally feature large groups of players.

Authors need lots of information regarding the internal models, while players participating in an evaluation get distracted by them.

Authors want to see the state of the models in real‐time, but during evaluations the models will be analysed retrospectively.

For rapid prototyping the amount of information is limited to that generated by one gameplay session at a time, while in evaluations an overview over a number of sessions is needed.

For rapid prototyping information has to be presented during play, while in an evaluation setting the information can be transformed before it is presented.

In order to address these requirements, we developed two additional tools that complement the authoring tool and testbed. The first one is a replay component, which was directly integrated into the testbed to give it the ability to review individual game sessions after they were played. This extension is accompanied by an aggregation tool, which combines the data from a large number of individual sessions into one spreadsheet for further analysis. Both new tools work with playtraces that are written to log files during a normal play session, where the internal models are hidden from the player. They use the same format / timestamp and are linked to the original story files as well as the models used by StoryTec, so there is no mismatch between these new components either. Therefore it is possible to analyse the aggregated view of the spreadsheet and then investigate anomalies in detail with the replay component without losing information. Based on the research questions asked in our previous evaluations of adaptive Serious Games, we decided that the playtraces had to contain at least the following information:

Change of the internal adaptation models along each dimension.

Time taken per scene (for example explanation, tasks, help‐screen).

If a scene was visited and how often.

User input (multiple choice, text, minigames and mouse movement).

We also decided to log every stimulus, which notes the execution of an event in the game. This will allow us to always reconstruct the whole playthrough in combination with the original story files. This way there will never be a loss of information, even if a feature was added without logging its information explicitly.

4.1 Replay component The replay component allows the testbed to replay game sessions recorded in playtraces retrospectively. It offers the same level of detail like a direct observation of the player, including mouse movements (Figure 1, left). In contrast to the original play session however, the state of the internal models is also visualized (Figure 1, right). The advantages compared to a direct observation are that the replays can be watched independently from the original game session and more than once. The internal game information and the evaluation setup is invisible for the player and does not require any additional effort besides the collection of the playtraces, which could even happen automatically while the game is played on a website. Compared to generic approaches like capturing a video feed, the playtraces can be much smaller by referencing simple events and having the replay component calculate the same result as the original game – which is only possible when tailoring the component to a specific game or game type.

507

Christian Reuter et al. Additional features of the tool include remote observation during live play and a check whether the story used during generation of a particular playtrace matches the version used during the original play, so no misleading results based on minor variations can be generated.

Figure 1: Screenshot of the replay component, showing the players view (including his mouse cursor symbolized by a white box) on the left and the internal state of some of the models (scene history, player model and parts of the knowledge space) on the right

4.2 Aggregation tool The aggregation tool in contrast is designed to provide an overview over a large group of players. Its task is to parse a number of playtraces created by a game and compile this data into one spreadsheet, which can be further analysed using common office tools. Calculating an average score for example does not require opening each playtrace separately, saving time and enabling large scale evaluations for single authors. By specialising the tool in reading our playtrace format, we were able to exploit the semantic of certain information, for example for aggregation. It is also able to distinguish between temporary and final values of the internal models and variables, although the event might technically be the same. The same differentiation can be done for answers a player has given. Important “higher level” questions that can be investigated using the tool include how the internal models evolve and if they converge over time. It can detect how often scenes were reached, giving the author feedback whether the adaptation produced different results for his players. It is also possible to detect unnecessary scenes, which are seldom reached, giving him a clue on where to focus when refining the game. Based on completion time, he can also estimate the difficulty of the game and gather information on the expected time for certain types of tasks or minigames, providing data for the creation of a game designed around a certain play time (e.g. during a school lesson). However certain information cannot be extracted automatically, because there are universal concepts which might be used differently in structurally similar games. One example showed us that an author used generic buttons to model multiple answers to task and to skip the task altogether. Since these buttons can be used in many different ways, we could not differentiate between solved and skipped tasks in a way that could be used over every game created with our authoring environment. When using a predefined template with a fixed semantic for such tasks however, the tool is even able to detect whether a task was solved correctly or not. In order to support very large games, we implemented a function which is able to filter certain events based on annotations. The aggregation tool also displays a warning, if multiple variations of the same story were used and splits the results accordingly to reflect the differences. We also added anonymous user IDs, which allowed the matching of play sessions to external data source like accompanying questionnaires. These IDs are

508

Christian Reuter et al. checked for uniqueness to prevent duplicate entries while complying with privacy regulations by being arbitrary identifiers.

5. Use Case: Evaluation of learning game for mathematics The first applications where our evaluation framework was used was a game called “Der Wechsel” (“The Switchover”), created for teaching about mathematic functions in school. The game combines the everyday use of math with elements of a crime story and optional minigames like puzzles or hidden object games. It was evaluated with multiple school classes of about 30 pupils each (age 14‐15, 55% male), where the pupils had roughly one hour to solve the game on their own. Afterwards the playtraces were collected and analysed using the aggregation tool. We will focus on explorative results in this paper, noting some interesting findings in regards to the overall gameplay, which we were able to quickly isolate by combining the aggregated play data with the replay tool. Since the game’s design did not use the adaptive features of the authoring framework, the adaptation models stayed constant over the course of the game. We could however verify that the evaluation workflow worked smoothly while including non‐expert authors and that even the analysis of non‐adaptive variables could produce helpful information, some of which will be discussed in the following paragraphs.

Figure 2: Graphs displaying the total playtime of each pupil (left) and the time it took them to solve a puzzle minigame (right) While the game was supposed to last one hour, analysis of the total playtime showed that most pupils needed about ten additional minutes to finish the game (Figure 2, left). However on outlier was obvious, who took less than 45 minutes. Further analysis with the replay tool then revealed that the pupil skipped all optional minigames and took lucky guesses at a lot of tasks instead of solving them, indicated by quick and often wrong answers when others paused to calculate their answers. Analysis of the time the players took to solve a puzzle minigame also yielded valuable feedback (Figure 2, right). Some players took up to twelve minutes to solve the minigame alone, although the game’s authors estimated that completing the puzzle would only take up to one minute (it only consisted of seven parts). Again the replay tool helped to answer this question by revealing that those players had supposedly solved the puzzle much earlier (all parts were in the right order), but were held back by a usability problem: The game would only acknowledge that the puzzle was solved when they aligned the combined parts with the upper left corner of the screen, but they centred them in the middle. In contrast, almost half of the players skipped the optional puzzle altogether in order to focus on the math questions, which is an interesting information when designing similar games in the future. Other important information is how well the pupils performed. The analysis tool allowed us to view the answers given for each task. We then grouped them into categories manually (Figure 3, left), for example revealing that the majority of players solved task 4.6 correctly. About a third made a mistake regarding the decimal digit (7200 or 7.2 instead of 720) and only one of ten pupils was completely wrong. This information then could be used by the teacher to discuss common mistakes in a subsequent lecture. In order to get a hint for the design of similar games the authors also check whether the pupils used a help function summarizing the learning content and giving the pupils hints. This was the case for almost every pupil; a majority even used the function more than once (Figure 3, right).

509

Christian Reuter et al.

Figure 3: How well a specific task was answered (left) and the number of times the help was used (right)

6. Conclusion The use case showed us that making internal game information visible to the authors in an easily understandable way is crucial for evaluating the intended effects (e.g. learning or training) and usability of serious games. If the data collection is tailored towards a specific game or technology, it is possible to automatically extract semantic information and reduce the overhead of the study at the same time. Offering aggregated data is great to get a quick overview, while providing a more detailed view for further analysis is crucial in the case of anomalies. When these different views are based on the same data, it is beneficial if they are provided by a suite of closely linked tools or even a single application. Using a model shared between the games created with a specific authoring tool, our evaluation environment is providing a middle ground between general purpose and highly specialized evaluation tools, balancing information detail and associated costs. While it is tailored to games created with the authoring tool, the general approach could be adapted for other models and environments as well. Future work will include the analysis of games that use the adaptive features more extensively, allowing us to evaluate the adaptation engine and algorithms themselves. We are also planning to offer the possibility to visualize aggregated data directly without using an external spreadsheet application. And since the evaluation framework is using the same models and data structures as our authoring tool, it would be interesting to couple them even further – for example by feeding the average completion time for each task back into the authoring tool. In a case where the players took more time than estimated, the author could then use this data to decide which tasks should be omitted in order to reach the expected completion time in a future version of the game.

Acknowledgements Parts of this work where funded and supported by the “Forum für interdisziplinäre Forschung” at Technische Universität Darmstadt in the project “Effekte mathematischer Lern‐ und Diagnoseumgebungen mit spielerischen Elementen” (May 2011 – September 2012).

References Bartle, R., 1996. Hearts, Clubs, Diamonds, Spades: Players Who Suit MUDS. Journal of Virtual Environments, 1(1), p.19. Available at: http://www.mud.co.uk/richard/hcds.htm. Bruder, R. et al., 2004. Third party certification of computer‐based learning environments. In EISTA: Int. Conf. on Education and Information Systems: Technologies and Applications. Orlando, Florida. Göbel, S. et al., 2009. 80Days: Adaptive Digital Storytelling for Digital Educational Games. In Y. Cao et al., eds. Proceedings of the 2nd International Workshop on Story‐Telling and Educational Games (STEG’09). CEUR Workshop Proceedings. Göbel, S. et al., 2010. Personalized, Adaptive Digital Educational Games using Narrative, Game‐based Learning Objects. In Proceedings of Edutainment 2010. pp. 438–445. Hilbert, D.M. & Redmiles, D.F., 2000. Extracting usability information from user interface events. ACM Comput. Surv., 32(4), pp.384–421. Available at: http://doi.acm.org/10.1145/371578.371593. Hullett, K. et al., 2012. Empirical analysis of user data in game software development. Proceedings of the ACM‐IEEE international symposium on Empirical software engineering and measurement ‐ ESEM ’12, p.89. Available at: http://dl.acm.org/citation.cfm?doid=2372251.2372265. Ketkar, N.S. & Youngblood, G.M., 2010. Graph‐Based Data Mining for Player Trace Analysis in MMORPGs. In Game Programming Gems 8 (AI Section). Charles River Media, pp. 335–352.

510

Christian Reuter et al. Kim, J.H. et al., 2008. Tracking real‐time user experience (TRUE): a comprehensive instrumentation solution for complex systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY, USA: ACM, pp. 443–452. Available at: http://doi.acm.org/10.1145/1357054.1357126. Korossy, K., 1999. Modeling Knowledge as Competence and Performance. In D. Albert & J. Lukas, eds. Knowledge Spaces: Theories, Empirical Research, and Applications. Psychology Press, pp. 103–123. Liu, Y.‐E. et al., 2011. Feature‐Based Projections for Effective Playtrace Analysis. In Proceedings of the 6th International Conference on Foundations of Digital Games. New York, NY, USA: ACM, pp. 69–76. Available at: http://doi.acm.org/10.1145/2159365.2159375. Medler, B., John, M. & Lane, J., 2011. Data Cracker : Developing a Visual Game Analytic Tool for Analyzing Online Gameplay. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY, USA: ACM, pp. 2365–2374. Available at: http://doi.acm.org/10.1145/1978942.1979288. Mehm, F. et al., 2010. Bat Cave: A Testing and Evaluation Platform for Digital Educational Games. In Proceedings of the 4th European Conference on Games Based Learning. pp. 251–260. Nacke, L.E. et al., 2009. Playability and Player Experience Research. In Proceedings of DiGRA 2009: Breaking New Ground: Innovation in Games, Play, Practice and Theory. DiGRA. Available at: http://www.digra.org/dl/display_html?chid=http://www.digra.org/dl/db/09287.44170.pdf. Nacke, L.E., Drachen, A. & Göbel, S., 2010. Methods for Evaluating Gameplay Experience in a Serious Gaming Context. International Journal of Computer Science in Sport, 9, p.2010.

511

Learning Effectiveness of Management Simulation Game Manahra Petr Smutný1, Jakub Procházka1, 2 and Martin Vaculík2 1 Faculty of Economics and Administration, Masaryk University, Brno, Czech Republic 2 Faculty of Social Studies, Masaryk University, Brno, Czech Republic psmutny@econ.muni.cz jak.prochazka@mail.muni.cz vaculik@fss.muni.cz Abstract: The subject of the study was to examine the learning effectiveness of a management simulation game called Manahra. The observed management game was a long‐term competition among several 20‐member teams. The team members acted in the roles of fictitious business managers and performed complex management tasks associated with the production and sales of passenger cars. The economic competition among the individual businesses took place in a simulated commercial market and the individual players were remunerated for their work with fictitious money. The amount of accumulated money was the criterion of their evaluation at the end of the game. The study consists of two relatively autonomous parts that worked together to answer the key question: does completion of management game lead to the development of managerial skills? The first part of the study presents the results of an original, yet unpublished research work. The research was conducted through a questionnaire survey. In the questionnaire, participants had to assess the level of their managerial skills. Individual managerial skills were included into the questionnaire based on an a priori created model of managerial competence. Self‐assessment was done using rating scales for each managerial competence separately. Players filled out questionnaires at the beginning and at the end of the game. The questionnaire distributed at the end of the game was extended by including a second scale in which players rated the magnitude of the change regarding their managerial competencies. The second part of the study presents the follow‐up research focused on obtaining qualitative responses from the players. They were again asked to fill out a questionnaire designed to obtain spontaneous reports about the development of their managerial competencies due to their participation in the game. This assessment was then compared with the initially established competency model and to the results of the original research. Keywords: gaming simulation, managerial skills, training, management, game, educational effectiveness

1. Introduction Numerous research studies (Bigelow 1991, Clark 2003) show that managerial skills can be acquired and developed, and that good results can be achieved by various training activities outside of the work environment (off‐site training). Large number of training activities is based on the theory of experiential learning (Paglis 2012). Management simulation games are primarily used to develop skills, therefore learning effectiveness is the main measure of their quality. Previous research, however, show rather ambiguous results. Some authors have documented the benefits of using this type of teaching method as being: practice in a quasi‐realistic environments with limited risks, increased creativity, faster decision making, better targeted analysis of competition (competitive analysis) and a better understanding of the relationships among business functions (cross‐functional understanding) (Chapman & Sorge 1999; Reibstein & Chussil 1997). A number of studies concluded that evidence is insufficient to show that the game experience can be potential learning (Faria & Wellington 2004, Wolfe & Jackson 1989). Factors that influence the effectiveness of simulation games can be divided into two main groups: a) the quality of the application (simulation design); and b) the quality of the game play (quality of implementation). The main feature of simulation games is the participation of real people in the role play. In this regard, Peters (1998) talks about the psychological validity as an important factor influencing effectiveness. The objective is to create a game environment which seems realistic to the participants. Furthermore, the additional goals are meant to affirm a high degree of similarity through objective parameters, and also to ensure a high degree of similarity of the subjective perception of reality among the players (Norris 1986). When creating a credible simulation game, it is therefore necessary to consider the experience and competencies of the participants.

512

Petr Smutný, Jakub Procházka and Martin Vaculík

2. Management simulation game Manahra A group of about 20 students make up the management of a fictitious car manufacturer, aiming to maximize the company’s accumulated profit during seven gaming rounds. Students learn the rules of the game in introductory seminars and, in a selection procedure, choose the managing director. The managing director appoints top management, divides the company into departments, and assigns management members to them. During the game they can release any member of their team or headhunt employees from another company and recruit them. For their work players receive remuneration in form of fictitious money and the sum they receive determines their final mark at the semester’s end. A successful company on the market can generate larger funds and its employees have a better chance of getting a better mark. The market computer simulation model takes the decisions of the carmaker’s management into account and then determines the overall demand for cars and the market share of individual car makers accordingly. At the beginning of the game the position of all car making companies is identical. During the game, students have a number of options for enhancing the performance of their company. They decide how many cars to produce in each round, set production costs, invest in research, add accessories to the standard equipment of a car to meet target customer group’s specifications, run advertising campaigns, and negotiate credit lines with banks. In every round, each car maker must complete financial statements, analyze the results of other companies, and pay out salaries. The entire workload is insurmountable for one person, or even for a small group. A successful company will involve all, or virtually all of the students in its tasks and functions. The managing director is the company’s main manager.

3. Study 1 3.1 Methodology and research sample The methodology of Study 1 is based on Kirkpatrick's model (Kirkpatrick, 1959), the presented results are consistent with findings on second level ‐ learning. The observed management simulation game allows the development of all components of managerial competencies ‐ knowledge (declarative, procedural or knowledge of type "know why") as well as skills. This has consequences also for the research methodology. Whereas knowledge can simply be measured using quizzes, such as tests, these are inappropriate for skills assessment (Boud 1992). Skills can be measured through case studies, evaluation of others (e.g., subordinates, colleagues etc.) or self‐assessment. Agut et al. (2003) reported the results that have confirmed that the method of self‐assessment of management competencies is, in terms of the quality of the data, equivalent to evaluation by subordinates. Also Hansson (2001), concludes that self‐assessment is sufficiently accurate and can be used to measure competencies. Therefore, we have also used the self‐assessment to evaluate skill levels. To collect data, a questionnaire was developed in which participants of the game evaluated their own level of managerial skills. Self‐assessments were carried out using rating scales individually for each managerial competence. The participants answered the questions concerning their own level of managerial skills listed. We used managerial skills that are part of a verified model of managerial competencies (Smutný 2007). Participants marked responses on rating scales, each varying from 0 to 100 points. Players completed the questionnaires before and after the game. The questionnaire distributed at the end of the game was extended by a second scale in which players assessed the magnitude of change in their managerial competencies. This enabled an extension of the classic two‐level "before and after" testing methodology to three‐level methodology. Sadri and Snyder (1995), referring to the series of experiments, state that such a procedure reliably leads to revealing progress due to changes in perception and evaluation of the factors. The participants were full‐time undergraduate students of Faculty of Economics and Administration at Masaryk University in Brno, Czech Republic. It was a relatively homogeneous group of students of the first three years of full‐time study. All participants completed the management simulation game as a compulsory part of their coursework. Only the data of those who filled out all questionnaires were considered for the data analysis. In total, data were obtained from the 195 participants, i.e., 60.19% of all participants. With the exception of bankers, all playing roles were represented, while owners were represented significantly less proportionately (in 41.67% of all owners in Manahra). Other roles otherwise were adequately represented.

513

Petr Smutný, Jakub Procházka and Martin Vaculík

3.2 Results Hypothesis 1: Completion of Manahra leads to positive changes in managerial skills evaluated through self‐reports. The data for verification of the hypothesis was obtained from a questionnaire filled out by players after completing the game. The questions concerned: What changes in managerial skills have you experienced as a result of completing Manahra? Table 1: Changes in managerial skills

Test Value = 0

t 18.49

Sig. (2‐ tailed) 0.00

Communication skills

N 195

Mean 8.73

Std. Dev. 6.59

Cooperativeness

195

11.73

8.50

19.27

Motivational skills

195

4.35

6.44

Cognitive skills

195

7.18

Organizational Skills

195

Flexibility / Adaptability

195

95% Confidence Interval

Lower 7.80

Upper 9.66

0.00

10.53

12.93

9.43

0.00

3.44

5.26

7.48

13.42

0.00

6.13

8.24

7.82

8.66

12.62

0.00

6.60

9.05

9.43

7.75

16.98

0.00

8.33

10.52

The results (see Table 1) show a slight improvement in all observed competencies, mainly between 5 and 10 points. The greatest improvement is reflected in cooperation, flexibility and communication skills. The fact that the measured progress is not zero is also confirmed by the results of the statistical analysis with the one‐ sample t‐test. The value of the test criterion for all competencies lies within critical values, always at a confidence interval of α <0.05. We therefore confirm the hypothesis that completion of Manahra leads to improvement in all monitored managerial skills. These results were validated through analysis, in which we observed differences in the evaluation of initial and final level of managerial skills of participants. Data for verification of the analysis were obtained from questionnaires filled out by the players before and after completing the game and answered the question: How do you rate your current level of managerial skills? Table 2: Changes in managerial skills – alternative analysis Paired Differences

Communication skills

initial 67.51 195

Std. Dev. Mean 12.71 6.51 16.41

Cooperativeness

final 78.79 195

12.06

initial 77.74 195

15.91

final 65.38 195

15.76

initial 58.80 195

18.87

final 71.21 195

14.48

initial 66.90 195

15.79

final 70.14 195

16.35

initial 64.69 195

19.74

final 76.16 195

13.52

initial 72.31 195

16.22

Motivational skills Cognitive skills Organizational Skills Flexibility / Adaptability

Mean N final 74.02 195

95% Confidence Interval

Std. Dev.

Lower

Upper

Sig. (2‐ tailed)

14.73

4.43

8.59

6.17

0.00

1.05

14.87

‐1.05

3.15

0.99

0.32

6.58

18.93

3.91

9.25

4.85

0.00

4.31

14.18

2.31

6.32

4.25

0.00

5.45

18.35

2.85

8.04

4.14

0.00

3.86

15.08

1.73

5.99

3.57

0.00

514

Petr Smutný, Jakub Procházka and Martin Vaculík The results (see Table 2) comparing the initial and the final evaluation of managerial skills show that the final level of all investigated managerial skills reported via self‐reports is higher than those at the initial stage. The statistical significance of differences was also assessed at significance levels for individual values of t‐test criterion. Differences can be declared significant for almost all studied competencies. Only in terms of cooperativeness, the significance level was α> 0.05 . According to the results, it can be concluded that completion of Manahra leads to positive changes in managerial skills. These changes are for the sake of convenience in the further text referred to as the effect of learning. Given the generally low level of practical experience in management of our participants, we examined whether, in addition to the effect of learning, completing the game has other effects as well. We tested the hypothesis that experience with managerial work gained in the game will lead to a more realistic assessment of participants' own managerial skills. Hypothesis 2: The reported final level of managerial skills does not exactly correspond to the initial level and magnitude of the change (i.e., there is an effect of self‐reflection). Data for the verification of this hypothesis was obtained from questionnaires filled out by the players before and after completion of Manahra. Of these, we obtained three kinds of reports: assessing the level of skills at the beginning, evaluating skills at the end of the game and reports concerning the perceived change in skills (assessed at the end of the game). With these values it was possible to calculate the quasi‐value of the initial skills and compare it with the value that players originally reported before the start of the game. Table 3: Correspondence of initial and ex‐post calculated quasi‐initial levels of managerial skills Paired Differences 95% Confidence Interval

Communication skills Cooperativeness

Std. Std. Mean N Dev. Mean Dev. initial 67.51 195 16.41 2.22 15.40 computed 65.29 195 13.56 initial

initial

Flexibility / Adaptability

Sig. (2‐ tailed)

0.04

4.39

2.01

0.05

10.68 16.05

8.42

12.95

9.30

0.00

‐2.23 19.00

‐4.91

0.46

‐1.64

0.10

2.87 14.62

0.81

4.94

2.74

0.01

2.38 19.12

‐0.32

5.08

1.74

0.08

5.57 14.41

3.54

7.61

5.40

0.00

66.90 195 15.79

computed 64.03 195 14.57

Organizational Skills

58.80 195 18.87

computed 61.03 195 15.70

Cognitive skills

Upper

77.74 195 15.91

computed 67.06 195 13.50

Motivational skills

Lower

initial

64.69 195 19.74

computed 62.32 195 15.68 initial

72.31 195 16.22

computed 66.74 195 13.87

Comparing the calculated values to the initial self‐reported level of managerial skills shows that there are differences in all observed managerial skills. These differences could be examined using the significance levels for each value of t‐test criterion. Communication and cognitive skills, cooperativeness and flexibility show the statistical significance of the differences. The remaining two show insignificant results. The results of the statistical analyses confirm the examined hypothesis. The final level of managerial skills does not correspond to the initial level and perceived magnitude of change. We call this difference the effect of self‐ reflection. Furthermore, it was demonstrated that the effect of self‐reflection is generally negative, thus there is a negatively directed re‐evaluation of the initial level of one’s own skills.

515

Petr Smutný, Jakub Procházka and Martin Vaculík The last step of the analysis was to examine the relationship of both effects. Examining the relationship of the two effects is logical, in particular with regard to the conclusions of Kolb‘s theory of learning. Hypothesis 3: The magnitude of the effect of self‐reflection varies differently among students who experience various degrees of the effect of learning. The participants were divided into two groups according to the magnitude of the effect of learning: group of students with above‐average value of the effect of learning (group 1) and a group of other students (group 0). The average value of the effect of self‐reflection in both groups of respondents was compared. Table 4: The relationship between differences and changes in managerial skills

Group Statistics

t‐test for Equality of Means 95% Confidence Interval

Communication skills

group N 0 75

Mean 2.57

Std. Dev. 16.05 15.97

120

‐2.34

124

‐15.50

18.62

‐18.83

20.47

6.47

20.72

2.45

18.64

105

‐2.05

16.47

‐7.53

18.17

Organizational Skills

105

2.93

18.28

‐4.53

18.74

Flexibility / Adaptability

‐67.01

17.53

119

‐61.39

16.76

Cooperativeness Motivational skills Cognitive skills

Sig. (2‐ tailed)

Lower

Upper

2.09

0.04

0.27

9.56

1.16

0.25

‐2.34

9.00

1.42

0.16

‐1.55

9.58

2.21

0.03

0.59

10.38

2.81

0.01

2.23

12.71

‐2.25

0.03

‐10.57

‐0.68

The results show that changes in self‐assessment (self‐reflection effect) due to the completion of Manahra are higher among respondents with higher levels of learning. By comparing the relevant values of the calculated significance level of p with the selected significance level α = 0.05, the null hypothesis can be rejected for all observed characters except for cooperativeness and motivational skills.

4. Study 2 4.1 Methodology and research sample In the first study, participants assessed their managerial skills and their development based on a checklist and description of individual skills. It is possible that while, under other circumstances, the participants did not think of their newly developed skills as being the result of Manahra; after seeing the checklist of skills, they might decide to indicate the development of some of them. Therefore, we conducted a second study, which identified whether players perceive the development of managerial skills even when asked an open‐ended and not close‐ended question about their skills development. New players were approached again through a questionnaire which had three parts. In the first part, participants answered questions concerning the degree of the development of soft and hard skills as a consequence of participating and completing Manahra. Participants answered on a five‐point scale ranging from "not at all“ to “very much." In the second part of the questionnaire, open‐ended questions were posed, the aim of which was to obtain spontaneous answers about the skills are being developed as a result of their participation in the game. The third part of the questionnaire included questions about the participants themselves ‐ we investigated their age, gender and degree and type of participation in the management game. The participants were full‐time undergraduate students at the University of Economics in Prague, Czech Republic. It was a relatively homogeneous group of students in their first three years of full‐time studies. The questionnaire was completed by a total of 126 players of Manahra, about 45% of all participants in the game.

516

Petr Smutný, Jakub Procházka and Martin Vaculík Slightly more men (53%) than women were represented in the sample. The average age of the participants was 21.44 years (SD = 2,08). Management positions (general manager, department managers, deputy manager) were held by 56 participants, 58 participants were regular employees of the company. The remaining participants had another position (banker, business owner) or non‐specific position. None of the participants in Study 2 was a participant in the first study.

4.2 Results According to the majority of the players, Manahra helps develop soft skills to an average degree. Only 5 (4%) of the players reported that Manahra did not help develop their skills at all. 28 players (22.2%) believe that Manahra developed their soft skills above average. At least one specific managerial skill was indicated by 86 players of Manahra as being developed (68.25%), while these players indicated an average of 1.47 (SD = 0.66) in terms of the development of different managerial skills. The most commonly reported skills as being developed were organizational skills, communication skills, cooperativeness and cognitive skills (Table 5). Table 5: Managerial skills and the number of players that reported them as developed

Sum

Rel.

Communication skills

126

22.22%

Cooperativeness

126

22.22%

Motivational skills

126

5.56%

Cognitive skills

126

18.25%

Organizational Skills

126

26.19%

Flexibility / Adaptability

126

1.59%

The players holding a managerial position indicated a higher level of development of soft skills than the players in non ‐ managerial positions (Table 6). We investigated whether this difference was demonstrated not only in terms of the perceived overall level of development of soft skills, but also in light of the idea that managers might be more likely to indicate particular skills as being developed. We focused on the four most frequently mentioned skills. The results of the analysis of variance shows that there is no difference between the position of manager and employee as to which skills were developed (Table 7). Therefore, we conclude that the player's role in the game affects the magnitude of the development of soft skills, but not other expected‐to‐ be‐developed skills. Table 6: The magnitude of development of soft skills for managers and non‐managers

Mean

Std. Deviation

Employee

2.64

Manager

Total

Confidence interval Lower

Upper

0.79

2.43

2.85

3.20

0.80

2.98

3.41

114 2.91

0.84

2.76

3.07

F (1, 112)

14.17

0.00

Table 7: Development of skills for managers and non‐managers Mean

Std. Deviation

Employee

0.17

Manager

0.30

Total

Communication skills

Cooperation Cognitive skills

Confidence interval LLCI

ULCI

0.38

0.07

0.27

0.46

0.18

0.43

0.24

0.43

0.16

0.32

Employee

0.28

0.45

0.16

0.39

Manager

0.14

0.35

0.05

0.24

Total

0.21

0.41

0.13

0.29

Employee

0.17

0.38

0.07

0.27

517

F (1, 112)

2.73

0.10

3.06

0.08

0.03

0.87

Petr Smutný, Jakub Procházka and Martin Vaculík Mean

Std. Deviation

Manager

0.16

Total Organizational skills

Confidence interval LLCI

ULCI

0.37

0.06

0.26

0.17

0.37

0.10

0.24

Employee

0.21

0.41

0.10

0.31

Manager

0.36

0.48

0.23

0.49

Total

0.28

0.45

0.20

0.36

F (1, 112)

3.22

0.08

Skill development was not affected by the department in which the employee in Manahra works. Players from different departments indicated a similar degree of development of soft skills and differences between various departments are not statistically significant (Table 8). Even if we focus on the four most frequently mentioned skills, one cannot say that some of them were developed to a greater extent in one department but not in another (Table 9) Table 8: The magnitude of development of soft skills based on the department in Manahra

Mean

Std. Deviation

Production

2.52

0.83

Confidence interval LLCI

ULCI

2.20

2.83

Marketing

3.04

0.90

2.68

3.39

Finances

3.03

0.80

2.75

3.31

Human Resources

2.85

0.67

2.54

3.16

110 2.86

0.83

2.71

3.02

Total

F (3, 106)

2.65

0.05

Table 9: The development of individual skills based on the department in Manahra N

Mean

Std. Deviation

Production

0.17

Marketing

Communication skills

Cooperation

Cognitive skills

Organizational skills

Confidence interval LLCI

ULCI

0.38

0.03

0.32

0.11

0.32

‐0.02

0.24

Finances

0.24

0.43

0.09

0.39

Human Resources

0.30

0.47

0.08

0.52

Total

110

0.20

0.40

0.12

0.28

Production

0.34

0.48

0.16

0.53

Marketing

0.22

0.42

0.05

0.39

Finances

0.26

0.45

0.11

0.42

Human Resources

0.10

0.31

‐0.04

0.24

Total

110

0.25

0.43

0.16

0.33

Production

0.10

0.31

‐0.01

0.22

Marketing

0.22

0.42

0.05

0.39

Finances

0.29

0.46

0.13

0.46

Human Resources

0.15

0.37

‐0.02

0.32

Total

110

0.20

0.40

0.12

0.28

Production

0.24

0.44

0.08

0.41

Marketing

0.19

0.40

0.03

0.34

Finances

0.32

0.47

0.16

0.49

Human Resources

0.25

0.44

0.04

0.46

Total

110

0.25

0.44

0.17

0.34

518

F (3, 106)

0.99

0.40

1.33

0.27

1.32

0.27

0.51

0.68

Petr Smutný, Jakub Procházka and Martin Vaculík

5. Discussion Methodology is important when reflecting the results. Various experiential learning courses and programs have been assessed in a variety of ways: objective learning tests, perceived learning measures, behavioral measures and others (Gosen and Washbush, 2004). The self‐assessment methodology used in our studies is compatible with measuring perceived learning. Our data represent respondents’ perceptions. The effectiveness of the game was not tested in any alternative way. Previous studies in the area of managerial skills (Agut, Grau and Peiro, 2001, 2003) have proven self‐assessment to be equivalent to evaluation by others in terms of reliability and validity of the data collected. The results of both studies show the development of the participants' managerial skills and they are complementary. In the case of organizational skills, communication skills, cooperativeness and cognitive skills, a fifth to a quarter of players spontaneously stated these skills as being developed. At the same time, by comparing the self‐assessment at the beginning and at the end of Manahra, the statistical significance of the development of these skills was confirmed. The results show an improvement of all the examined participants' skills. The results of the study support the conclusions of other authors on the applicability of simulation games to develop managerial skills. Those authors arrived to similar results while using self‐assessment (perceived learning measures) (Dedeke, 1999; James, 2000) and other research methods (Premi and Shannon, 2001). In particular, the results of Study 2 can be interpreted as a positive indicator of the structural quality of the analyzed management game. Although it is a complex management game (total‐enterprise game) in which players hold different positions, the same skills are being developed for the various positions. The players in leadership positions indicated a greater degree of development of their skills. This may not be the result of imbalances of Manahra, but rather that more proactive players, who are willing to invest more time and energy into Manahra, took those leadership positions. Manahra can therefore be described as a balanced simulation game. These results confirmed the findings of the previous studies of Smutný (2007). An important factor in the effective use of simulation games is the method of its implementation (Stainton, Johnson and Borodzicz, 2010). The simulation game used in our study is a part of mandatory class, as it is a part of seminars of ‘The Basics of Management‘ course. It is not linked to the lecture part of the course. Implementation of the game follows the standard division of the preparatory phase (lecturers present the basic aspects of the game, players are studying the rules), and the actual playing of the game (1 training round and 7 actual game rounds) and phase reflection (evaluation of play by the players and teachers). Faria (2001), based on extensive research, identified other important factors influencing the effectiveness of management games: students' exposure to management games (attitude of students), and the applicability of the players' game results to their own evaluation. The players participated in the management game as a part of university course, which was mandatory for most of them. This fact may have influenced their approach to the game and the motivation with which they come into the game. It can be assumed that some students would, if given a chance, have possibly preferred not to participate in the game. However, the players had strong external motives to participate in the game‐ credits gained for the course participation, the obligation to complete the game in order to get a higher education. One can also assume that some students would have, in the absence of external motivation, chose not to participate in the game. The effectiveness of simulation games could be even higher in individuals with prevailing internal motivation. The results that players achieve are dependent on their individual drive and on the success of the team as a whole. Players received a fictitious salary (which could have been increased through bonuses by the more successful teams), and the acquired money they could invest individually in a simulated stock market. The total value of the acquired assets is then the main criterion for evaluating players. The obtained result is then counted as the one third of their overall evaluation in ‘The Basics of Management‘ course. An important aspect of the beginning of the game is the fact that on the basis of information from previous studies and practice, the majority of our participants can be characterized as relatively inexperienced (Smutný, 2007). The game is, in terms of its content and difficulty of the tasks, adapted to the players. It is likely that the effect of learning would be lower if the players had more experience at the beginning and if they had had

519

Petr Smutný, Jakub Procházka and Martin Vaculík higher entry level managerial skills (see Washbush and Gosepund, 1994). The inexperience of the participants at the beginning of the game probably affected the existence and size of the effect of self‐reflection. The game creates an environment in which students recognize the complexity and nature of managerial work, and, after its completion, they are better able to evaluate the real level of their managerial skills. The reason may be that the completion of the game leads to increased cross‐functional knowledge (Scherpereel, 2013). This assumption was confirmed by the finding that players who achieved a higher level of learning, are more critical to the initial level of their skills after the game. We dubbed this phenomenon the effect of self‐reflection, and although it was observed statistically only in regards to some of the examined skills, it is important that it was demonstrated at those competencies that the participants spontaneously reported as being the most developed. This supports the hypothesis that magnitude of learning affects the degree of self‐reflection of the game participants.

Acknowledgements This article is part of a research „Effective leadership: Integrative approach“. The research has been funded by Czech Science Foundation (P403/12/0249).

References Agut, S., Grau, R., Peiro, J.M., 2001. Los sesgos de la autoevaluaci on de competencias. El caso de los gerentes de organizaciones turisticas (The biases of competences self‐assessment. The case of tourism industry managers). Revista de Psicologia Social Aplicada, Vol. 11, No. 1, pp 5–20. Agut, S. Grau, R., Peiró, J. M. (2003) ‘Competency needs among managers from Spanish hotels and restaurants and their training demands’. Hospitality Management, vol. 22, pp 281‐295. Bigelow, J. D., ed. (1991) Managerial Skills: Explorations in applied knowledge, Sage, Newbury park. Boud, D. (1992) The use of self‐assessment schedule in negotiated learning. Studies in Higher Education, Vol. 17, No. 2, pp 185 – 201. Chapman, K. J. and Sorge, C. L., (1999) ‘Can a simulation help achieve course objectives? An exploratory study investigating differences among instructional tools’, Journal of Education for Business, Vol. 74, No. 4, pp 225‐230. Clark, S. C., Callister, R. and Wallace, R. (2003) 'Undergraduate management skills courses and students' emotional intelligence', Journal of Management Education, Vol. 27, No. 1, pp 3‐23. Simulation & Experiential Exercises, Vol. 14, pp 43‐46. Dedeke, A. (1999) Design, integration and student evaluation of response papers in an introductory management course. Journal for Education of Business, Vol. 77, pp 211 – 214. Faria, A. J. (2001) ‘The Changing Nature of Business Simulation/Gaming Research: A Brief History.’ Simulation & Gaming. Vol. 32, No. 1, pp 97‐110. Faria, A. J. & Wellington, William (2004). ‘A Survey of Simulation Game Users, Former‐Users, and Never‐Users.’ Simulation & Gaming. Vol. 35, No. 2, pp 178‐207. Gossen, J.,Washbush, J. ‘A review of Scholarship on Assessing Experiential Learning Effectiveness. ’ Simulation & Gaming. Vol. 35, No. 2, pp 270‐293. Hansson, B. (2001) ‘Competency models: are self‐perceptions accurate enough?’ Journal of European Industrial Training, Vol. 25, No. 9, pp 428‐441. James, P. (2000) ‘The influence of a period of environment oriented work on students’ perception of their learning style. Environmental Education Research, Vol. 6, 157 – 165. Kirkpatrick, D. L. (1959). ‘Techniques for evaluating training programs.’ Journal of the American Society of Training Directors, 13, pp 3–9. Norris, D.R. (1986). ‘External validity of business games.’ Simulation & Gaming, 1Vol. 7, No. 4, pp 447‐459. Paglis, L. L. (2012) 'A review of managerial skills training in the classroom', Journal of Management Education. Vol. 36, No. Peters, V. (1998) ‘The Validity of Games.’ Simulation and Gaming, Vol. 29, No. 1, pp 20 – 30. Premi, J. and Shannon, S. I. (2001) Randomized controlled trial of an educational program for individualized learning. Journal of continuing Education in the Health Professions, Vol. 17, pp 245 – 249. Reibsein, D. J. and Chussil, M. J. (1997) Putting the Lessons before the Test: Using Simulation to Analyze and Develop Competitive Strategies, Wiley and Sons. Sadri, G. and Snyder, P. (1995) ‘Methodological issues in assessing training effectiveness.’ Journal of Managerial Psychology, Vol. 10, No. 4, pp 30 – 32. Smutný, P. (2007) Simulační hry jako nástroj zvyšování kvality lidského kapitálu podniku, unpublished thesis (doctoral dissertation), Masaryk University. Stainton, A. J., Johnson, J. E., Borodzicz, E. P. (2010) ‘Educational Validity of Business Gaming Simulation: A Research Methodology Framework‘ Simulation & Gaming, Vol. 41, No. 5, pp 705–723 Scherpereel, C. M. (2013) ‘Changing mental models: Business simulation exercises’, Simulation Gaming Vol. 36, No. 3, pp 388 – 403. Wolfe, J, and R. Jackson (1989). ‘An Investigation of the Need for Algorithmic Validity.’ Simulation & Games Vol. 20, No. 3 , pp 272‐291.

520

Using the Master Copy ‐ Adding Educational Content to Commercial Video Games Heinrich Söbke1, Thomas Bröker1 and Oliver Kornadt2 1 Bauhaus‐Universität Weimar, Faculty of Civil Engineering, InnoProfile: Intelligentes Lernen (Intelligent Learning), Weimar, Germany 2 Technische Universität Kaiserslautern, Faculty of Civil Engineering, Kaiserslautern, Ger‐ many heinrich.soebke@uni‐weimar.de thomas.broeker@uni‐weimar.de oliver.kornadt@bauing.uni‐kl.de Abstract: Successful development of educational video games has to overcome plenty of challenges. In addition to the requirements of a successful software development project, an attractive game experience has to be designed and imple‐ mented. However a failure in one of the most ambitious tasks of developing an educational game is known as "chocolate‐ dipped broccoli". This term was used by Bruckman (1999) to describe the unsuccessful integration of game mechanics and learning content. An alternative approach to avoid these problems is not new but still seems to be underrepresented in the field of Digital Game Based Learning: the use of existing, popular video games. Recent research has acknowledged several video games as educational media for the training of professional skills. Besides that video games can be used to spread domain knowledge. Such video games either already contain specific domain knowledge or they can be extended easily to embed domain knowledge. This paper presents case studies of four commercial video games, which are potentially eligible for educational extension. Each game represents a distinct category: Fliplife stands for simple, community enhancing Social Network Game (SNG) Triviador as a SNG is the synthesis of two well known board games (Risk and Trivial Pursuit). Ju‐ raShooter StGB ‐ already designed as an educational game ‐ exemplifies mobile device games. Lastly SimCity 5 is a repre‐ sentative of a classic strategy and simulation game with focus on systems. Each game is shortly introduced, our relevant gameplay experiences and the potential learning content is described. Finally we identify potential extensions regarding additional learning content for each game. Having these examples in mind we suggest a more systematic approach to use commercial video games as learning tools: if a game has been proven as a viable source for transfer of domain knowledge, we suggest categorizing it due to two characteristics: complexity of learning content and mainly attracted player types. Based on such a categorization a directory of games can be compiled. This directory then can be used in a concrete educa‐ tional scenario to identify appropriate games. As a categorization for learning content we suggest Bloom’s revised taxon‐ omy. A well known categorization for player types was proposed by Bartle. Although by far not every commercial video game can be enriched with educational content, the effort to customize existing games seems to be much smaller than the effort to create educational games from scratch. This paper argues for a systematic approach to facilitate commercial video games in educational settings as an additional option apart from individual implementations of educational games. Keywords: digital game based learning; COTS; commercial video games; SimCity, JuraShooter StGB, Fliplife, Triviador

1. Introduction Video games are a relatively new type of media. Originally just created for entertainment purposes they are now also used in educational settings. Gee (2005) refers to video games as learning machines. In the light of the huge impact on players learning sophisticated models to master a game, educators started to develop educational games. However this approach has seen a lot of failures. Egenfeldt‐Nielsen (2007) states that “edutainment started as a serious attempt to create computer games that taught children different subjects. Arguably, it ended up as a caricature of computer games and a reactionary use of learning theory.” Papert (1998) uses the picture of a Shavian reversal: “Shavian reversals — offspring that keep the bad features of each parent and lose the good ones — are visible in most software products that claim to come from a mating of education and entertainment”. Bruckman (1999) used the expression chocolate‐dipped broccoli for her observation that “fun is often treated like a sugar coating to be added to an educational core”. All these perceptions mirror at least partially the ultimate complexity of educational game development: Build‐ ing software is already a risky venture: Ambler (2010) found that only around half of IT projects are successful. The next hurdle to take is to design an attractive game: a commonly acknowledge method uses effort consum‐ ing cycles of development and play testing (Fullerton 2008). For educational games this process has to be ad‐ justed to certain learning targets and complemented with an appropriate context and content. Habgood & Ainsworth (2011) call a seamless interlacing of game and educational content “intrinsic integration”: learning

521

Heinrich Söbke, Thomas Bröker and Oliver Kornadt content should not appear as a foreign object in the game. Not being all enough, these complex tasks have to be covered with limited financial resources, because normally educational games do not reach those high sales revenues as their commercial counterparts do (Bröker et al. 2011). As one solution to this dilemma Young et al. (2012) “believe that commercial gaming companies and educa‐ tional researchers could mutually benefit by bringing academic content into the fictitious worlds originally created without educational content objectives in mind”. Commercial video games have been already used as educational tools: The Futurelab Project (Sandford et al. 2006) investigated the usage of “commercial off the shelf [(COTS)] computer games in formal education”. The Sims 2, Roller Coaster Tycoon and Knights of Honor have been those COTS games looked at in this study. The Sims 2 as a medium in formal educational settings have been studied by Peterson (2011) and Panoutsopoulos & Sampson (2012). A. N. Foster (2011) chose Roller Coaster Tycoon 3 as a tool for the transfer of disciplinary knowledge in economics and social studies. 1 Looking at all these examples at least three areas of game usage for educational purposes can be identified : First, games can be used as a medium to spur reflection. At Wabash College Portal was set on the booklist of an compulsory seminar, which “is devoted to engaging students with fundamental questions of humanity from multiple perspectives and fostering a sense of community” (Abbott 2010). Second games are used to teach professional skills. Poling (2010) facilitated StarCraft in a course about 21st Century Skills. Steinkuehler & Dun‐ can (2008) found that successful play of World of Warcraft requires “scientific habits of mind”. And third games are employed for construction of disciplinary knowledge. Squire (2003) taught history with a modified version of Civilization III. A case study of Moshirnia (2007) done with Civilization IV lead to similar results. So far we have argued that building successful educational video games is a highly challenging task. Then again there exist a lot of intriguing commercial video games. In this paper we want to propose the systematic inves‐ tigation of COTS games for either existing learning content or the possibility to add and embed appropriate content. We use a comprehensive definition of COTS video game: It is any digital game which could be used as 2 host for potential educational content but which currently is not holding that content . In addition to dedi‐ cated educational games this would be a further, effort saving approach to establish video games as educa‐ tional tools.

2. Case studies The proposed approach to use COTS games for educational purposes is based on a few assumptions: It may be possible that a game can embed educational content. For a systematic approach to classify a game its content should be graded according to a knowledge taxonomy – the knowledge dimension of the game. A well‐known taxonomy has been introduced by Bloom (1956) and modified by Anderson et al. (2000). This is not the only valid categorization – e.g. the already mentioned overview of Prensky (2007) classifies content specifically found in video games, but it is a taxonomy commonly utilized in educational contexts. Players are categorized by player type taxonomies, e.g. Bartle's player types (1996). In addition to these as‐ sumptions we hypothesize that the learning objectives are given as well as the structure of the player type distribution. Then it should be possible to choose an appropriate game, enrich it with content and use it in the considered educational setting. Such an approach would need a catalogue of COTS games capable of including educational content. Our case studies introduce four video games as potential elements for such a catalogue. We describe each game in four categories. Game description is a rough summary of the games main character‐ istics. It conveys a short impression of the game. In the section Gameplay experiences we summarize our per‐ sonal impressions of the gameplay as we think they are relevant to use that game in an educational context. Learning content is the caption for an analysis of the structure and type of potential educational content. The part Possible adoptions proposes appropriate changes of the game software to support the game’s usage as an educational tool.

Prensky (2007) gives a more comprehensive overview of different types of learning content including facts, skills, behaviour and creativ‐ ity. 2 This definition implies also educational games themselves ‐ as the example JuraShooter StGB demonstrates. The important characteristic we want to point out is the combination of an existing video game and additional educational content

522

Heinrich Söbke, Thomas Bröker and Oliver Kornadt

2.1 Fliplife Game description: Fliplife is a multiplayer, HTML5 based browser game (Fliplife 2012; Fliplife.com 2012). Unlike a typical social network game the relationship between two players is not fixed, but defined according to the amount of common actions: The more interactions two players have, the nearer becomes their relation‐ ship. Main objective of Fliplife is a simulation of the player’s life: professional career, education and free time. The player pursues a career by taking part in a well‐ defined set of projects. Spare time activities are sports and parties. These activities are rewarded with energy, which is needed for doing the projects. Education is pro‐ vided by multiple choice question based quizzes. Fliplife got special attention when it was said to be an as‐ sessment tool for a German trust (Söbke, Hadlich, et al. 2012). Gameplay experiences: The basic game mechanics of Fliplife seem to be very simple, although they can build the foundation for complex problems (Söbke, Bröker, et al. 2012). In general Fliplife shows the characteristics of an online third place: players meet online in their spare time, make acquaintances and friends (Soukup 2006). The game mechanics create a frame for communication with co‐players. Fliplife may be a low cost ver‐ sion of an online game which serves as a third place (Steinkuehler & Williams 2006).

Figure 1: Fliplife: project description Learning content: Fliplife contains domain knowledge: specialist’s expertise is represented by career paths and projects. Also the game objects are partly domain specific. However to play Fliplife successfully this knowledge is not mandatory. The so‐called university is another possibility to learn: The quizzes are quite demanding and require broad general knowledge. Possible adoptions: Müller (2012) has investigated possibilities to integrate domain knowledge in Fliplife using the domain of building physics, a discipline in civil engineering concerning the comfort in buildings. Her sugges‐ tions include a domain specific career path, a building physics institute as employer and projects from the field of building physics. Apart from descriptive texts, striking graphics could emphasize technical details. Beyond that the university can support learning by field specific quizzes and issuing field specific rewards and badges. Furthermore she suggests assignments which require the player to do a technical analysis of the projects (e.g. “Execute 3 heat protection projects.” where heat protection is not a shown attribute of a project, but derivable from the project description). For the integration of building physics learning objectives it seems appropriate to upgrade the player’s building with more technical details as it is done in the current version. An additional mini game could deal with wall structures: The player has to build wall structures from a given set of materials to meet certain requirements. Crowdsourcing is also an option mentioned by Müller: certain could cause play‐ ers to deal with technical details. Her main approach to complement Fliplife with domain specific know‐how is integration of specific information, graphics and items. As already mentioned succeeding in the game is not connected to any technical knowledge. Nevertheless it could bring domain affiliated persons together. If those persons are involved in the game, the game’s strength

523

Heinrich Söbke, Thomas Bröker and Oliver Kornadt as a virtual place could induce community building processes (“Community of Practice” (Lave & Wenger 1991)).

2.2 JuraShooter StGB ‐ Jagd nach dem Katzenkönig Game description: JuraShooter StGB ‐ Jagd nach dem Katzenkönig is an educational iOS game to memorize the crucial legal terms of criminal law (Lernfreak GbR 2012). This drill & practice game was produced 2011 by a group inspired and led by the german lawyer Raban von Buttlar. Its title refers to a well‐known and widely discussed criminal cause (Pötters 2009). Objective of the game is to unmask the alien cats of the Katzenkönig (“cat king”). This is done by instantly tapping the cats that carry parts of the correct answer to a law question. The player is rewarded for correct answer and gets extra points for faster answers and for answering in a cor‐ rect order. Questions itself are collected in packages. Each package focuses on a certain topic. Currently all content relates to the domain of law. Gameplay experiences: Our personal gameplay experiences have been dominated by audiovisual feedback caused by tapping the monsters. It seems to act both as a reward and a short cycled feedback. This feedback and the additional entry in the high score list are the main motivational elements of this game. Extrinsic moti‐ vation for learning the content vanishes during the play. Answering questions becomes a tool for reaching a high score – at least for a certain type of player. This conclusion is not yet backed up by scientific data. How‐ ever it is derived from the entries in high score lists: Top entries can only be reached when the player instantly taps the first appearance of each answer. That is when the correct answer is perfectly memorized and can be answered without delay (Buttlar et al. 2012).

Figure 2: JuraShooter StGB: Tapping definition elements Learning content: The learning content of this game is obvious. Additional packages of learning content can be added with an existing content editor. The editor does not simply support the text of the question and wrong and correct answers. It also allows determining structural attributes of a question, e.g. to take account of the order of the answer’s parts. Possible adoptions: Possible adoptions of the game include ‐ besides knowledge of other domains ‐ different types and structures of questions, e.g. matching or estimation questions. It could also be necessary to change the narrative (and accordingly graphics) because the main figure “Katzenkönig” is closely related to the law domain.

2.3 Triviador Game description: Triviador is a Social Network Game (SNG) available on the Social Network Service (SNS) Facebook (THX_Games_PLC 2011). It is an adoption of the well‐known board game Risk. While Risk uses dice for random decisions, Triviador replaces dice by questions in several forms. At the first stage a multiple choice

524

Heinrich Söbke, Thomas Bröker and Oliver Kornadt question is asked. If both players choose the right answer, an estimation question with a numeric answer is issued. The player with the nearest answer wins. In case of a tie the fastest answer wins. As a further game element players can use jokers which increase the probability of finding the correct answer in different ways. Gameplay experiences: Our gameplay experiences contained a similar trait as those already mentioned for JuraShooter StGB: Especially in the war mode the desire to win a duel outshines the awareness of probable learning. In the context of this game questions become a tool to win a match. They are not the primary pur‐ pose of playing the game, but an integrated game mechanic.

Figure 3: Triviador: Selecting bases Learning content: The learning content of the game is given by questions and their answers. Questions are taken randomly from a pool of more than 10.000 questions. So far there are no mechanics for enhancing learning: a player barely can choose the topic of questions. Also the game determines the rhythm of play – a player cannot reflect at her own pace about a question nor can she review a completed question. Possible adoptions: In general the displayed questions are chosen randomly from a large pool of questions. A so‐called Targeted Subject Booster allows the attacking player to choose the domain of the question. Further‐ more there are some country specific versions of Triviador that use a map of the country and country related questions. To enhance further Triviador’s characteristics of a learning tool the player should be able to choose preferred topics in his personal game options. During the matching phase of the game preferably two other players with same topic preferences are assigned. Furthermore the selection of questions could adhere to an algorithm of flashcard systems (e.g. Leitner (2011)). This approach could improve the educational benefit. Another feature of Triviador is the crowd sourcing of questions: When a player has reached a certain level she is allowed to enter own questions in several categories. These questions undergo a review by other players. After logging in they could opt for answering a certain amount of crowd sourced questions. They have to judge the question in several categories, e.g. if the spelling and grammar is correct and if they want to answer that question in the game. Both activities have educational value. The creation of questions means producing arte‐ facts and judging questions spurs reflection. We suggest tagging the questions as a further improvement: until now there is only a fixed set of domain cate‐ gories. Tags would introduce more and specialized categories. A question could bear more than one tag and belong to more than one (sub‐)category (Peters 2009).

525

Heinrich Söbke, Thomas Bröker and Oliver Kornadt

2.4 SimCity Game description: SimCity is a well‐known and popular simulation game. The first version has been published in 1989. Earlier this year (2013) SimCity 5 3 has been issued on a new technical foundation. SimCity is about building, developing and managing cities. Thus a player takes the role of an engineer when he interacts with the underlying systems. Will Wright, who originally designed SimCity, considers simulation games as tools for experimenting with their base systems (Wright 2007). SimCity has been used and investigated as educational tool (Adams 1998; Gaber 2007; Minnery & Searle 2012). Gameplay experiences: The attractiveness of SimCity has already been proven by millions of sold copies of every single version of the game. SimCity 5 now supports online multiplayer gameplay, e.g. it connects cities of neighboured players as sources and sinks of (material) streams. From the technical viewpoint of an engineer of 4 urban hydrology we examined the model elements of this discipline. We only found a rudimentary and sche‐ matic implementation of an urban hydrology system. D’Artista & Hellweger (2007) have examined SimCity 4 and marked its compliance with current urban hydrology models as insufficient. Compared to their findings the last version of SimCity contains more detailed models 5 . However, they still do not meet the requirements of realistic simulation models 6 . Learning content: An important learning goal connected to simulation games is the understanding of the un‐ derlying systems. Every simulation game is based on a model. Players reveal and learn the game’s model dur‐ ing gameplay. As our experiences suggest game models often do not meet requirements of reality‐like models, i.e. the structure of model does not reflect important real world elements. Devisch marks blurred, incorrect models as one argument against usage of simulation games for learning about real systems (2008). In general the genre Simulation Game has often been used as a pedagogical tool, which is demonstrated by different examples: Foster (2011) examined the knowledge construction using Roller Coaster Tycoon, Squire has used mods of Civilization III to teach social science. Mobility was built as an educational simulation game based on scientifically validated models for transport systems (Brannolte et al. 2000), but it has also reached attractive‐ 7 ness as a video game . Possible adoptions: The rule driven simulation engine GlassBox of Simcity 5 is highly configurable. It could be the base for other simulation games as well (Willmott 2012). Until now the API of GlassBox has not been dis‐ closed. If that would happen, adjusting model elements of SimCity could lead to realistic scenarios.

2.5 Questions: A valid game element for learning? Multiple choice questions (MCQ) are a main pedagogical element of three of the four above presented case studies. MCQs are often criticized. One argument is that they just contain knowledge at the recall level. And students learn to choose the right answer from a set of options instead of knowing all alternatives they have 8 . Scouller (1998) attributes surface learning to MCQ based examinations. Nicol (2007) presents a framework for MCQs as an educational tool. In general MCQs are considered as a valid assessment tool for the understanding of abstract concepts (Hopkins 1998). Iz & Fok (2007) gave an example for the domain of geomatics ‐ showing that it is possible to cover all levels of the cognitive domain of Bloom’s taxonomy with MCQs. Higher levels of complexity may require activities outside the test, e.g. doing calcula‐ tions to determine the correct answer. Furthermore they found a relation between the complexity of the question and the time students spent on answering that question. Also Simkin & Kuechler (2005) experienced difficulties in constructing MCQs of higher knowledge levels, but they confirmed that this is an practicable approach. 3

We refer to the current version of SimCity as SimCity 5, although it is officially named SimCity. Urban hydrology is – besides building physics and law ‐ one of the disciplines we are using for exemplifying usability of commercial video games as educational tools. 5 For example water pumps can be upgraded to filtration pumps. This allows the reuse of dirty water as drinking water. 6 As an example it is not possible to use surface water for drinking water catchment. Also storm water management is nonexistent in the game. 7 This was witnessed by a dialogue partner: when he heard about the authors’ affiliation to Bauhaus‐Universität, he mentioned the game Mobility which production has been scientifically accompanied by members of that University. He named this game a cause for many short nights of his former life. 8 As an example: the examination regulations of our faculty exclude MCQ based examinations explicitly as a valid test tool th (http://www.uni‐weimar.de/cms/fileadmin/uni/files/ka/mdu_akad/07/30_2007.pdf, last accessed April, 12 2013) 4

526

Heinrich Söbke, Thomas Bröker and Oliver Kornadt Palmer & Devitt (2007) stated, based on a study in the domain of medicine, that MCQ tests “are capable of withstanding the intellectual and statistical scrutiny imposed by a high stakes exit examination”. Mukhopadhyay et al. (2010) made a similar conclusion also for MCQs in medical education. MCQ may not be fun causing game elements. They are neither part of a problem solving, constructive game‐ play. But they are a valid educational instrument. And as far as they can be integrated in gameplay without affecting intrinsic motivation negatively, they can be used as an educational tool for the transfer of knowledge. Prensky (2011) defines an appropriate measurement for fun and learning in an educational game: “1. Is the game fun enough that someone who is not in its target audience would want to play it [...]? 2. Do people using it think of themselves as ‘players’ [...]? 3. Is the experience addictive? [...] 4. Are the players’ skills in the subject matter and learning content of the game [...] significantly improving at a rapid rate, and getting better the longer he or she plays? [...].” Due to our experiences these questions can be answered positively for all of our MCQ “enriched” examples – mostly without any restriction, but always to a certain grade.

2.6 Levels of knowledge complexity Game structure and game mechanics form the frame for knowledge that can be included in the game. So the complexity of the learning objectives depends on the game. The following discussion of an example uses Bloom’s revised taxonomy: JuraShooter StGB’s current learning content covers level 1 to 3 in the Knowledge Dimension: Factual Knowledge is included in the form of domain terminology. The game also contains Concep‐ tual Knowledge: legal structures and models are object of the questions, e.g. a player has to be familiar with the “legal model” of a second degree murder. An example for Procedural Knowledge is a list of necessary elements of an offense to constitute a punishable crime (“Knowledge of criteria for determining when to use appropriate procedures”). The Cognitive Process dimension is only supported in the first level: The player just needs to remember facts. The use of additional types of questions requires a change of the game software. However it would enable the game to include content of other Cognitive Dimension’s levels than level 1 “Re‐ member”. As an example ordering questions would require comparisons, which is located in level 2 “Under‐ stand”. As Triviador does support only a smaller set of question types (Selection question and Estimation ques‐ tion), the complexity of the knowledge is lower. Iz & Fok demonstrated that all levels of the Cognitive Dimension can be reached by answering questions. How‐ ever the attractivity of questions may be low when answering needs a lot of work outside of the game. Simula‐ tions games ‐ like SimCity ‐ allow to experiment with potential answers in the game itself. Because of versatile possibilities of interaction between player and software simulation games can include also higher levels of the taxonomy – at the cost of much more effort for development of the game.

3. Conclusions and discussion Educational games often do not meet the promise of a sensed effortless learning. Habgood (2007) observes "chocolate covered brocoli"‐like edutainment applications. Missing “intrinsic integration”, i.e. intrinsic motiva‐ tion cannot be consistently perpetuated during gameplay, is considered as an important reason (Habgood & Ainsworth 2011). These failures may result from the enormous complexity to create an “intrinsically inte‐ grated” educational video game. We propose an additional approach to employ video games as educational media for domain knowledge trans‐ fer: the systematic review of existing commercial off‐the‐shelf (COTS) video games. By using existing games the only challenge to overcome is to embed educational content while the game software has been produced and 9 the game itself has been proven as attractive . Furthermore successful games are considered to be learning machines teaching their content effectively (Gee 2005; Becker 2006). In a larger framework suitable video games could be classified according to two criteria: a first criterion is the potential learning content the game is capable to embed. We propose Bloom’s revised Taxonomy (Anderson et al. 2000) as an appropriate categorization. The second criterion is the predominant player type of the game. These classifications would allow selecting appropriate games for a defined learning content and a known set of learners. 9

For example often these games outperform their educational “counterparts” in terms of audiovisual presentation (Breuer 2012).

527

Heinrich Söbke, Thomas Bröker and Oliver Kornadt Of course this approach has to fight some imponderables. One of them is the question if the suggested tax‐ onomies are resilient in this context. Bartle’s taxonomy created for players of a Multi User Dungeon (MUD) may not be directly applicable to players of other game genres. Another item needing further research is the transfer to other context. Gee (2003) postulates that successful learning in games depends on context, i.e. the constructed knowledge is not transferred to other contexts. So players learn to master the game, but are not able to apply their knowledge in real world contexts (Rehm 2013; Niegemann 2009). This may become a prob‐ lem, as contexts given by commercial games cannot be adjusted easily. So the proposed classification may need to be extended by a context dimension. Also it may be questionable if necessary game related changes are feasible – the developer of a game may pursue other approaches. Concerning our sample there is a mixed situation: SimCity has been announced to be open for new content, but the GlassBox API has not yet been revealed. Triviador is open in parts: new questions can be added, but no new categories can be established and selected during gameplay. Fliplife is extensible theoretically, but de facto it is not yet clarified if the devel‐ oper company would support customized versions. This has already been done for JuraShooter StGB, which is the only ready for reuse game among our case studies. Another challenge of simulation games is the problem of simplified and blurred models, which do not suffi‐ ciently mirror domain knowledge. FarmVille (Zynga 2009) can be taken as an example: It is build for being extended with additional content (Mahajan 2010). But for crops and animals almost the only fact which is connected to reality is their name – unless they are not pure fantasy elements. Other important simulation model characteristics – as for example maturation periods – are only game specific. The proposed approach has still a lot of challenges to overcome. Games which can be enriched with educa‐ tional content are admittedly a rare species. On the other hand the possible gains of employing (modified) COTS games are compelling; even small achievements might be worthwhile. It is an additional option in the field of game based learning. A next step on our agenda is to extend JuraShooter StGB to the field of urban hydrology. While the extension of the other three examples is more hypothetical and still has to be substanti‐ ated, in the case of JuraShooter StGB the financial effort has been calculated: it is about one‐tenth of the origi‐ nal application.

References Abbott, M., 2010. Portal on the booklist. Available at: http://www.brainygamer.com/the_brainy_gamer/2010/08/portal‐ booklist.html [Accessed January 18, 2011]. Adams, P.C., 1998. Teaching and Learning with SimCity 2000. Journal of Geography, 97(2), pp.47–55. Ambler, S.W., 2010. 2010 IT Project Success Rates. Dr.Dobbs. Available at: http://www.drdobbs.com/architecture‐and‐ design/2010‐it‐project‐success‐rates/226500046# [Accessed May 10, 2012]. Anderson, L.W. et al., 2000. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educa‐ tional Objectives, Abridged Edition L. W. Anderson & D. R. Krathwohl, eds., Allyn & Bacon. Bartle, R.A., 1996. Hearts, Clubs, Diamonds, Spades: Players Who Suit MUDs. The Journal of Virtual Environments, 1(1). Available at: http://www.citeulike.org/user/drakkos69/article/3474752. Becker, K., 2006. Classifying Learning Objectives in Commercial Video Games. In Summer 2006 Institute ‐ Linking Research to Professional Practice. University of Calgary. Bloom, B.S., 1956. Taxonomy of Educational Objectives J. P. Keeves, ed., Longman London. Brannolte, U. et al., 2000. Aktualisierung und Erweiterung von Planspielansätzen im Verkehrswesen im Hinblick auf die Erstellung von Mobilitätsspielen, Weimar. Breuer, J., 2012. Broccoli‐coated chocolate? The educational potential of entertainment games. In W. Kaminski & M. Lor‐ ber, eds. Gamebased Learning: Clash of Realities 2012. München: kopaed, pp. 87–96. Bröker, T., Söbke, H. & Kornadt, O., 2011. Close the gap — Obstacles and solutions for the missing educational games in graduate education. In 5th European Conference on Games Based Learning. pp. 74–80. Bruckman, A., 1999. Can educational be fun? In Game Developer’s Conference, San Jose, California. pp. 75–79. Buttlar, R. von et al., 2012. Die Jagd nach dem Katzenkönig. In W. Kaminski & M. Lorber, eds. Gamebased Learning: Clash of Realities 2012. Kopäd, p. 384. D’Artista, B.R. & Hellweger, F.L., 2007. Urban hydrology in a computer game? Environmental Modelling & Software, 22(11), pp.1679–1684. Devisch, O., 2008. Should Planners Start Playing Computer Games? Arguments from SimCity and Second Life. Planning Theory & Practice, 9(2), pp.209–226. Egenfeldt‐Nielsen, S., 2007. Educational Potential of Computer Games (Continuum Studies in Education), Continuum. Fliplife, 2012. Fliplife. Available at: http://fliplife.com/ [Accessed April 26, 2012]. Fliplife.com, 2012. Was ist Fliplife? Fliplife for Business. Available at: http://business.fliplife.com/lerne‐fliplife‐kennen/was‐ ist‐fliplife/ [Accessed May 21, 2012].

528

Heinrich Söbke, Thomas Bröker and Oliver Kornadt Foster, A.N., 2011. The process of learning in a simulation strategy game: Disciplinary knowledge construction. Journal of Educational Computing Research, 45(1), pp.1–27. Fullerton, T., 2008. Game Design Workshop: A Playcentric Approach to Creating Innovative Games 2nd Revise., Morgan Kaufmann. Gaber, J., 2007. Simulating Planning: SimCity as a Pedagogical Tool. Journal of Planning Education and Research, 27(2), pp.113–121. Gee, J.P., 2005. Learning by design: Games as learning machines. E‐Learning and Digital Media, 2(1), pp.5–16. Gee, J.P., 2003. What Video Games Have to Teach Us About Learning and Literacy. Comput. Entertain., 1(1), pp.20–20. Habgood, M.P.J., 2007. THE EFFECTIVE INTEGRATION OF DIGITAL GAMES AND LEARNING CONTENT. University of Notting‐ ham. Habgood, M.P.J. & Ainsworth, S.E., 2011. Motivating Children to Learn Effectively: Exploring the Value of Intrinsic Integra‐ tion in Educational Games. Journal of the Learning Sciences, 20(2), pp.169–206. Hopkins, K.D., 1998. Educational and psychological measurement and evaluation 8. ed., Boston: Allyn and Bacon. Iz, H.B. & Fok, H.S., 2007. Use of Bloom’s taxonomic complexity in online multiple choice tests in Geomatics education. Survey Review, 39(305), pp.226–237. Lave, J. & Wenger, E., 1991. Situated learning: legitimate peripheral participation, Cambridge University Press. Leitner, S., 2011. So lernt man lernen. Der Weg zum Erfolg., Herder, Freiburg. Lernfreak GbR, 2012. Jura Shooter: Jagd nach dem Katzenkönig. Lernfreak. Available at: http://lernfreak.de/produkte [Ac‐ cessed July 16, 2012]. Mahajan, A., 2010. Rapidly Developing FarmVille. GDC 2010. Available at: http://de.slideshare.net/amittmahajan/rapidly‐ building‐farmville‐how‐we‐built‐and‐scaled‐a‐1‐facebook‐game‐in‐5‐weeks [Accessed September 10, 2011]. Minnery, J. & Searle, G., 2012. The effectiveness of computer games for planning education: A SimCity case study. In AN‐ ZAPS 2012. pp. 142–154. Moshirnia, A., 2007. The educational potential of modified video games. Issues in informing science and information tech‐ nology, 4, pp.511–521. Mukhopadhyay, M. et al., 2010. EVALUATION OF MCQs FOR JUDGMENT OF HIGHER LEVELS OF COGNITIVE LEARNING. Gomal Journal of Medical Sciences, 8(2), pp.112–116. Müller, N., 2012. Erweiterung von Fliplife mit bauphysikalischen Inhalten. Bauhaus‐Universität Weimar. Nicol, D., 2007. E‐assessment by design: using multiple‐choice tests to good effect. Journal of Further and Higher Education, 31(1), pp.53–64. Niegemann, H.M., 2009. Lernen durch Computerspiele : Das spielende Klassenzimmer? [Vortrag im Rahmen der Ringvorle‐ sung der Universität Erfurt “Spielemedien ‐ Medienspiele” am 08.12.2009]. , p.49. Available at: http://www.db‐ thueringen.de/servlets/DerivateServlet/Derivate‐19610/niegemann.pdf [Accessed October 9, 2012]. Palmer, E.J. & Devitt, P.G., 2007. Assessment of higher order cognitive skills in undergraduate education: modified essay or multiple choice questions? BMC medical education, 7, p.49. Panoutsopoulos, H. & Sampson, D.G., 2012. A Study on Exploiting Commercial Digital Games into School Context. Journal of Educational Technology and Society, 15(1), pp.15–27. Papert, S., 1998. Does Easy Do It? Children, Games, and Learning. Game Developer, 5(6). Peters, I., 2009. Folksonomies : Indexing and retrieval in Web 2.0, De Gruyter/Saur. Peterson, L., 2011. Values in Play–Interactional Life with the Sims, Poling, N., 2010. 21st Century Skills in Starcraft. Available at: http://www.honors.ufl.edu/courses/coursesfall10.html [Ac‐ cessed January 18, 2011]. Pötters, S., 2009. Strafrecht Classics – Der Katzenkönig ( BGHSt 35 , 347 ). JuraExamen, pp.1–7. Available at: http://www.juraexamen.info/strafrecht‐classics‐der‐katzenkonig‐bghst‐35‐347/ [Accessed April 10, 2013]. Prensky, M., 2011. Computer games and learning: Digital game based learning. In J. Raessens & J. Goldstein, eds. Handbook of Computer Game Studies. MIT Press, pp. 97–124. Prensky, M., 2007. Digital Game‐Based Learning, Paragon House, St. Paul, Minnesota, USA. Rehm, M., 2013. Do gamers change attitudes towards economics through playing manager games? Zeitschrift für Ökono‐ mische Bildung, (01), pp.162–176. Sandford, R. et al., 2006. Teaching with Games Using commercial off‐the‐shelf computer games in formal education. Fu‐ turelab. Scouller, K., 1998. The influence of assessment method on students’ learning approaches: Multiple choice question exami‐ nation versus assignment essay. Higher Education, 35, pp.453–472. Simkin, M.G. & Kuechler, W.L., 2005. Multiple‐Choice Tests and Student Understanding: What Is the Connection? Decision Sciences Journal of Innovative Education, 3(1), pp.73–98. Söbke, H., Hadlich, C., et al., 2012. Social Game Fliplife : Digging for talent – an analysis. In P. Felicia, ed. Proceedings of the 6th European Conference on Games Based Learning. Academic Publishing Limited, pp. 487–494. Söbke, H., Bröker, T. & Kornadt, O., 2012. Social Gaming – Just Click and Reward? In P. Felicia, ed. Proceedings of the 6th European Conference on Games Based Learning. Academic Publishing Limited, pp. 478–486. Soukup, C., 2006. Computer‐mediated communication as a virtual third place: building Oldenburg’s great good places on the world wide web. New Media & Society, 8(3), pp.421–440. Squire, K., 2003. Replaying History: Learning World History through playing Civilization III. Indiana University.

529

Heinrich Söbke, Thomas Bröker and Oliver Kornadt Steinkuehler, C.A. & Duncan, S., 2008. Scientific Habits of Mind in Virtual Worlds. Journal of Science Education and Tech‐ nology, 17(6), pp.530–543. Steinkuehler, C.A. & Williams, D., 2006. Where Everybody Knows Your (Screen) Name: Online Games as “Third Places”. Journal of Computer‐Mediated Communication, 11(4), pp.885–909. THX_Games_PLC, 2011. Triviador on Facebook. Facebook. Available at: http://apps.facebook.com/triviador [Accessed October 10, 2012]. Willmott, A., 2012. Inside GlassBox. In GDC Conference 2012. Wright, W., 2007. Will Wright makes toys that make worlds. Available at: http://www.ted.com/talks/will_wright_makes_toys_that_make_worlds.html [Accessed December 23, 2010]. Young, M.F. et al., 2012. Our Princess Is in Another Castle: A Review of Trends in Serious Gaming for Education. Review of Educational Research, 82(1), pp.61–89. Zynga, 2009. FarmVille. Available at: http://apps.facebook.com/onthefarm/.

530

An Application of Adaptive Games‐Based Learning Based on Learning Style to Teach SQL Mario Soflano, Thomas Connolly and Thomas Hainey University of the West of Scotland, UK mario.soflano@uws.ac.uk thomas.connolly@uws.ac.uk thomas.hainey@uws.ac.uk Abstract: The fact that each student has a different way of learning and processing information has long been recognised by educationalists. In the classroom, the benefits derived from delivering learning content in ways that match the student’s learning style have also been identified. As new modes of delivery of learning content such as computer‐assisted learning systems (e.g. eLearning) have become increasingly popular, research into these has also identified the benefits of tailoring learning content to learning styles. However, in games‐based learning (GBL), the adaptation based on learning style to enhance the educational experience has not been well researched. For the purpose of this research, a game with three game modes has been developed. 1) non‐adaptivity mode; 2) a mode that customises the game according to the student’s learning style identified by using a learning style questionnaire; and 3) a mode that has an in‐game adaptive system that dynamically and continuously adapts its content according to the student’s interactions in the game. This paper discusses the term adaptivity in a GBL context and presents the results of an empirical experimental study investigating the differences in learning effectiveness of different learning groups. The study was performed between three different game groups and a paper‐based learning group with 120 students in Higher Education learning SQL (Structured Query Language). The results show that the game developed, regardless of mode, produced better learning outcomes than those who learned from a textbook. Particularly for adaptive GBL, the learning effectiveness was identified to be higher with lower completion time compared to the other modes of the game. Keywords: adaptive GBL, adaptivity, learning style, SQL, NeverWinter Nights, RPG

1. Introduction The fact that each student has a different way of learning and processing information has long been recognised by educationalists (Kolb, 1984; Felder and Brent, 2005). In the classroom, the benefits derived from delivering learning content in ways that match the student’s learning style have also been identified (Smith and Renzulli, 1984; Price, 2004). As new modes of delivery of learning content such as computer‐assisted learning systems (e.g. eLearning) have become increasingly popular, research into these has also identified the benefits of tailoring learning content to learning styles (Miller, 2005). However, as Connolly and Stansfield (2006) have suggested, eLearning simply replicates the traditional education system (classroom style) and may be overly focussed on method of delivery, i.e. delivering materials over the web rather than on actual teaching and learning, and indeed motivating and engaging the students in the learning process. In contrast, games, particularly video games, appear to be able to engage people over extensive periods of time and also motivate them to re‐play the game repeatedly until they have mastered it (Kirriemuir and McFarlane, 2004). Therefore, some educationalists (for example, Prensky, 2006) have considered games to be a potential platform in supporting student learning and have turned their attention to what is now called games‐based learning (GBL). While many GBL applications have been developed in the last two decades, there remains a lack of empirical evidence to support the use of GBL for learning purposes (Connolly et al., 2012). Given that there appears to be genuine advantages for learning outcomes to be derived from the adaptation of teaching materials to learning styles in the classroom and remotely through eLearning, it may also be possible that GBL applications that are adapted to the individual’s learning style would improve learning outcomes. Kirriemuir and McFarlane (2004) have suggested that games, unlike classroom learning or eLearning, provide a different type of engagement as they demand constant interaction and generate a ‘flow’ that could assist in engaging students. It is therefore possible for students to adopt different leaning styles in GBL than they adopt in other learning settings.

531

Mario Soflano, Thomas Connolly and Thomas Hainey In the next section, adaptivity is presented followed by a review of previous empirical work in adaptive GBL. Section 3 discusses the evaluation of an adaptive GBL application on Section 3. The evaluation includes analysis in the difference in learning effectiveness and completion time between experimental groups. Section 4 provides conclusions from the study and discusses future research directions

2. Previous research 2.1 Definition of adaptivity In computing, there are two types of adaptation process: adaptability and adaptivity (Jameson, 2003). Adaptability refers to the ability of the student to ‘adapt’ to the system by explicitly customising the system according to their preferences (Bontcheva, 2002). On the other hand, adaptivity, which is usually used in the context of a user‐adaptive system, refers to the ability of the system to identify the student’s preferences or characteristics and customise the system accordingly; that is, the student implicitly influences the adaptation process (Mulwa et al., 2010). In modern computer systems, adaptability is usually implemented by providing customisation options that allow the student to customise the system according to their preferences. For example, in eLearning the student can choose a font size and font style associated with the learning materials. Conversely, adaptivity does not explicitly require input from the students and it is usually hidden from them. The students simply see the result of the customisation process provided by the system. Although both types can exist in a computer system, each type has differences in terms of its usage. Nowadays, adaptability through student customisation is widely used and exists in many computer systems. Whilst it is certainly useful in some circumstances, adaptability requires direct manipulation from the student that can result in an increase in the student’s cognitive load, especially if there are many options the student needs to choose (Oppermann, Rashev and Kinshuk, 1997). Conversely, adaptivity can capture the interactions between the student and the learning system and the adaptivity analyses the historical interactions before making an automatic adjustment. Adaptivity is considered to be less intrusive compared to adaptability as it does not require the student to make any changes and, as a result, the interaction between the student and the system can be maintained. This is useful especially in a system that has a considerable amount of elements that the student would need to manipulate. In addition, adaptivity can be used for a behavioural pattern analysis of the student’s interaction with the system. Such an analysis may be used to create different student experiences. The disadvantage of adaptivity is that the student does not have direct control in customising the system. For this particular research, adaptivity is used to refer to the system’s ability to automatically customise certain elements of the system based on a series of the student’s interactions with the system.

2.2 Adaptivity in GBL Reflecting back to the benefits of accommodating students’ learning styles on the learning outcomes in classroom‐style learning and in eLearning, the same benefits may also apply in GBL. However, not all elements of learning style theory can be adopted in GBL. According to Becker (2007) one of the main characteristic of games, regardless of the genre, is their interactivity, which requires players to actively interact with the game, and indicates that the ‘reflection’ element of the Felder‐Silverman learning style may not be relevant to GBL. This statement is also supported by Boyle, Connolly and Hainey (2011) who indicated that games provided an active, experiential, situated and problem‐based learning environment. According to Charles et al. (2005) and Melis and Monthienvichienchai (2004) adaptation can be incorporated into games through:

A player’s character: all actions undertaken by the character have implications; for example, if the character is wounded, the movement of the character is slower.

Non‐player character (NPC): the player can access this feedback by ‘talking’ to the NPC. Besides providing feedback, the conversation itself may be used to alter the story based on the selections the player has made in the conversation.

532

Mario Soflano, Thomas Connolly and Thomas Hainey

The game environment: adaptation through the game environment can be categorised into: customisation, contextualisation and personalisation (Melis and Monthienvichienchai, 2004). In the context of GBL, customisation is related to the functionalities of the learning system; for example, when the student reaches a particular level, a new control will appear. Contextualisation means that the content that is going to be delivered is adaptable according to the student’s performance, learning history or response to certain missions. Personalisation relates more to the student’s preference, for example, the font size.

Feedback/scaffolding: can be used to inform the student about their status in the game and to help the student to achieve the game and educational objectives (Jackson, Krajcik and Soloway, 1998).

2.3 Adaptive GBL applications ‐ empirical evidence A systematic literature search was conducted on electronic databases including: Association for Computing Machinery (ACM), Cambridge Journals Online, Institute of Electrical and Electronic Engineering (IEEE), Index to theses, IGI Global, Ingenta Connect, Science Direct, Springer Link, Wiley Online Library, Extended Academic asap, simulation and gaming and emerald. the following search terms were used: (adaptive or adaptivity or personalisation or personalization or “learning style”) and (“serious games” OR “games‐based learning”) The search returned 978 papers, however, after detailed analysis only 8 papers were identified as being relevant to adaptive GBL applications with empirical evidence and these are summarised in Table 1. Table 1: Existing empirical evidence in adaptive GBL Authors

Area

Felicia and Pitt (2007, 2008, 2009)

Maths

Conati and Zhou (2002), Conati and Zhao (2004)

Maths

Peirce, Conlan and Wade (2008)

Physics of optics

Lynch, Steen, Pritchard, Buzzell and Pintauro (2008) Hwang, Sung, Hung, Huang and Tsai (2012) Lee and Ko (2011) Yongyuth, Prada, Nakasone, Kawtrakul and Prendinger (2010) Demmel, Kohler, Krusche and Schubert (2011)

Food safety

Adaptivity Learning style and personality traits (The Big‐5 model). Cognitive theory of emotions (joy, distress, pride, shame, admiration and reproach). Number of interactions between the student and certain elements of the game. Learning style based on Dunn and Dunn’s theory. Felder‐Silverman (sequential ‐ global).

Plantation Logic programming Agriculture Languages

Successful rate of the code execution. Changes occurring in the environment caused by the student’s actions. Number of mistakes the student has made.

MathQuest was developed to teach maths (algebra) at secondary school level. Felicia and Pitt (2007, 2008, 2009) proposed a learning style and personality traits concept based on the MBTI and the Big‐5 model known as PLEASE (Personality, Learning styles, Emotions, Autonomy, Systematic approach and Evaluation). Using this model, Felicia and Pitt (2007) developed strategies that could be incorporated and implemented to the game design and game play. For example, for students with a high level of competitiveness, ranks and scores were displayed and for students with a high level of extraversion, frequent rewards were given. Felicia and Pitt (2009) conducted an experiment involving 80 secondary school students that investigated the effect of personality in learning through MathQuest. The experiment used a questionnaire on game preferences, an International Personality Item Pool (IPIP) personality questionnaire, pre‐test and post‐test of the subject matter and a questionnaire about the game. They found that adaptivity towards elements of the Big‐5 such as neuroticism and conscientiousness did not have a significant impact in learning outcomes, whilst adaptivity towards agreeableness, openness and extraversion seemed to benefit the students. They concluded that although “not all students will benefit from a video game but that when settings and options match their preferences and learning style, improvements can be obtained” (Felicia and Pitt, 2009, p.151). Prime Climb was developed to teach maths (factorisation) for grade 6 and 7 students (Conati and Zhao, 2004). To analyse the student’s performance and actions with the game, two layers of dynamic Bayesian networks were developed. The first layer, the short‐term student model, was used to capture changes in the student’s behaviour from one interface action to the next whilst climbing a specific mountain. Conversely, the second

533

Mario Soflano, Thomas Connolly and Thomas Hainey layer was used to record the student’s performance for each finished mountain. The game used cognitive theory of emotions (the OCC theory) as the basis for the student’s profile (Conati and Zhou, 2002). The OCC theory consists of 22 emotion types and in Prime Climb the emotional types that were used were joy, distress, pride, shame, admiration and reproach. To evaluate the game, Conati and Zhao (2004) used a control group (who played the original game) and an experimental group (who play the game with adaptivity) with ten participants for each group. The results showed that the experimental group had significantly higher gain in learning effectiveness compared to the control group and the control group hardly improved without any external guidance. However, in terms of the correlation between agent hints and learning effectiveness, there was no conclusive evidence. ELECTRA is an RPG game with micro‐adaptivity implemented through feedback and hints. The game uses adaptivity logs that record the interaction between the student and the game. Peirce, Conlan and Wade (2008) investigated the effectiveness of the ELECTRA adaptivity procedure and its impact on the student’s motivation. The experiment involved 49 participants split into four experimental groups (no hint, neutral hint, adaptive hint and counter‐adaptive hint). The study conducted pre‐tests and post‐tests of the physics of optics to measure the participant’s knowledge of the particular material. The post‐test also included game evaluation to measure the gaming experience. The results showed that the learning outcomes and game experience for the game with adaptive hints was better than the other groups although the difference was not statistically significant. Lynch et al. (2008) developed Ootle‐U to teach aspects of food safety. The game is adapted to the student’s learning style, based on Dunn and Dunn’s theory: motivation/learning enjoyment (M); persistence towards task completion (P); sense of responsibility (R); structure (S); alone versus peer (AP); auditory (A); visual (V); tactile (T); and kinesthetic (K). The authors suggested that M, P, R and S were related to emotional preferences while A, V, T and K could be categorised as perceptual preferences. Their experiment involved 217 participants to measure the learning effectiveness of the game. The effectiveness was measured by comparing the pre‐test and post‐test scores. The experiment also assessed the impact of learning style on learning effectiveness and the correlation between the learning achievement and the game. The results showed that the participants’ knowledge in food safety was improved although the results indicated that the improvement was not significant because the students had prior knowledge of the subject matter. With respect to learning style, the experiment indicated that a greater number of participants preferred to mix their learning methods rather than consistently use a single learning method. The authors suggested that perhaps the participants, who were considered as digital natives, interacted with various different media in their everyday life. Hwang et al. (2012) developed an adaptive GBL application based on learning style to teach about plantation. Their evaluation of the game involved 46 participants and they investigated whether learning in a game that could match the student’s learning style was better than a game without any consideration of the student’s learning style in terms of learning effectiveness and motivation. The experiment also evaluated the easiness and usefulness of the game. A pre‐test/post‐test experimental design was used with an equal number of participants in the control group (who played the game without any adaptivity) and the experimental group (who played the game with adaptivity based on the learning style identified before the game). The learning style adopted in this research was the ‘sequential/global’ element of the Felder‐Silverman learning style, identified using their learning style questionnaire, while a test sheet was used in the pre‐test and post‐test to measure the knowledge of the participant on the subject matter. The results showed that the adaptive version had a significantly higher learning effectiveness. Gidget is an autonomous agent in a GBL application developed by Lee and Ko (2011). The game itself teaches logic programming to novice programmers. Gidget has an adaptivity that is reflected in its expression based on the execution of the code (smile when the code is successful and look sad otherwise). The experiment involved 116 participants comprising 50 females and 66 males with an average age of 27.5 ranging from 18 to 59 years of age. The level of education of the participants were pre‐high school (<1%), high school (13%), college (23%), associate degree (3%), bachelor degree (38%), master degree (14%) and doctoral degrees (6%). The experiment used two groups: a control group that played the game with non‐adaptive feedback and an experimental group that played the game with adaptive feedback. The results indicated that Gidget with its expression had a positive effect on motivating the participants and the experimental group completed more levels compared to the control group in a similar completion time.

534

Mario Soflano, Thomas Connolly and Thomas Hainey Agrivillage is a GBL application based on Second Life and its objective is to teach agriculture. The game itself was developed by Yongyuth et al. (2010) and it has an adaptivity implemented through NPCs based on changes that occur in the environment caused by the student’s actions. The game was evaluated by 20 university students through the use of a questionnaire. While there was no pre‐test and post‐test of agriculture knowledge, the results suggested that the game had a positive impact on learning and raising awareness about the impact of agriculture on the environment. Demmel et al. (2011) developed a collaborative and adaptive game called weMakeWords that teaches alphabetical words and symbols related to Chinese, German or English. The adaptivity is implemented through scaffolding based on the number of mistakes the student makes. Their preliminary research, involving children aged between 4 and 8 years of age, investigated the effectiveness of the adaptive game. The data gathered in this research was analysed by looking at how many characters the students could draw after the game. The results showed that children remembered on average four Chinese symbols and their meanings and some could draw the symbols after playing the game for 15 to 30 minutes. From the empirical evidence generated from the adaptive GBL that has been discussed above, adaptivity has the potential for improving learning effectiveness in GBL. Moreover, it has also shown:

The current trend in adaptive GBL research is the investigation of the effect of theories used as the basis for the adaptivity. Such theories may be personality theories as researched by Felicia and Pitt (2009) or learning styles as used by Lynch et al. (2008) and Hwang et al. (2012).

The majority of research in adaptive GBL uses a pre‐test/post‐test experimental design. This is mostly used when measuring learning effectiveness as reflected in the differences between pre‐test and post‐test either within or between control and experimental groups. The analysis methods adopted in the studies are mainly quantitative.

The adaptive systems mostly use log files or a database to collect information about interactions between the games and the students.

3. Learning‐style‐based adaptive GBL For the purpose of this research, an adaptive GBL application based on learning style was developed. The game was intended to teach the basics of the database programming language SQL (Structured Query Language) while the learning style adopted in this game was the Felder‐Silverman learning style model, particularly the presentation elements (picture‐text). Felder‐Silverman learning style model has been widely used in eLearning and GBL and its reliability and validity have been tested. When compared to other learning style model, Felder‐Silverman model represents elements from most models which indicate the generalisability of the model. The selected genre of the game was role‐playing games and it was developed by using NeverWinter Nights 2 engine. In this study, there were three modes of the same game designed and developed:

a non‐adaptive mode of the game. This mode treats all students the same and takes no account of the student’s learning style.

an out‐of‐game adaptive mode. The characteristics of a student are identified by means of the Felder‐ Silverman learning style questionnaire completed in advance of playing the game and the game is then customised according to the student’s learning style.

an in‐game adaptive mode. In this mode, the student’s characteristics are identified during the gameplay. As it is possible for the student to change learning style in the course of the game, the game will have an adaptive system that can automatically customise the game in real‐time according to the student’s current learning style.

The difference between the modes concerns the nature of the adaptive approach adopted while the rest of the game elements such as storyline, game environment, controls and game interface, are identical. The adaptive approach itself was implemented through the presentation of the learning materials presented by the conversation system of the game.

535

Mario Soflano, Thomas Connolly and Thomas Hainey

3.1 Methodology To investigate the differences in the learning effectiveness, 120 university students with no knowledge of SQL voluntarily participated in the study. Only the participants who scored 0 in the pre‐test SQL test were selected in this study. The participants were 83 undergraduate students (69.2%) and 37 postgraduate students (30.8%) with 43 students from the University of the West of Scotland (35.8%), 46 students from Heriot‐Watt University (38.3%), 19 students from Napier University (15.8%), 9 students from University of Edinburgh (7.5%), 2 students from Glasgow University (1.7%) and 1 student from Strathclyde University (0.8%). The participants were also selected from various programmes. There were 78 participants (65%) from a computing programme, 1 students from accounting (0.8%), 3 students from business (2.5%), 10 students from engineering (8.3%), 1 student from finance (0.8%), 3 students from languages (2.5%), 1 student from law (0.8%), 7 students from life science (5.8%), 9 students from management (7.5%), 2 students from mathematics (1.8%), 1 student from medicine (0.8%), 1 student from nursing (0.8%), 1 student from psychology (0.8%) and 2 students from social science programmes (1.8%). For the purposes of analysis, the participants were categorised into two groups, computing and non‐computing students, because the number of participants in each of the non‐computing programmes were too low to be analysed individually. There were three experimental groups which represent each mode of the game and one control group which the students allocated in this group would learn SQL by using a textbook. Each participant was allocated randomly to one of the four groups. In this section, the groups will be referred to as: the paper‐based group, the non‐adaptive game group, the out‐of‐game LS group and the in‐game adaptive group. The participants were required to answer the SQL tests and those aspects of the game relating to the questionnaire. The SQL tests were conducted twice: in both the pre‐test and post‐test. In the pre‐test, the SQL test was used to identify the participants’ knowledge of SQL. Those who had no knowledge of SQL were selected for the experiment. The SQL post‐test was used to test the students’ knowledge of SQL after learning SQL through the game. The SQL pre‐test had similar questions to the post‐test. Both had 11 questions in total. The maximum points achievable for the SQL pre‐test and post‐test was 16. The analysis of the learning effectiveness to be conducted is a non‐parametric test according to the Levene statistic = 8.280 (p<0.001), while the analysis for the completion time is also a non‐parametric test according to the Levene statistic = 3.879 (p<0.024). A Levene statistic with p<0.05 means the homogeneity cannot be assumed. In this thesis, the confidence value used is 95% (p<0.05).

3.2 Results This section describes the results of the experiment starting from differences in learning effectiveness followed by the completion time analysis between the experimental groups. 3.2.1 Learning effectiveness Before learning effectiveness can be measured, the reliability of the data must be tested by an independent rater. To analyse the reliability of the data analysis the Cronbach Alpha test (Kinnear and Gray, 2008) was used. The test showed that the Cronbach Alpha value of the data is 0.990, which means the post‐test scores marked by the experimenter were consistent with the scores marked by the independent rater. The learning effectiveness of the paper‐based SQL learning for 30 university students has a mean of 2.6 (SD=3.8065). The mean score of the learning effectiveness from the non‐adaptivity game group, which consists of 30 university students, is 12.517 (SD = 4.690). The learning effectiveness of the out‐of‐game LS group, consisting of 30 university students, has a mean score of 12.35 (SD = 4.065). The learning effectiveness of the in‐game adaptive group, which also consists of 30 university students, has a mean score of 14.042 (SD = 2.354). When compared to any mode of the game, there is a significant difference in learning effectiveness between the groups. By using the Mann‐Whitney U tests, the learning effectiveness for the non‐adaptivity game group is significantly higher when compared to the paper‐based learning group (Z = ‐5.903, p<0.001). The same results also apply when comparing the paper‐based group and the out‐of‐game LS game group (Z = ‐5.995, p<0.001) and the in‐game adaptive group (Z = ‐6.495, p<0.001).

536

Mario Soflano, Thomas Connolly and Thomas Hainey However, the Mann‐Whitney U test shows there is no significant difference between the non‐adaptivity group and the out‐of‐game LS group (Z = ‐0.872, p<0.389). A similar result is found when comparing the learning effectiveness of the non‐adaptivity group to the game with the in‐game adaptivity (Z = ‐0.378, p<0.711) and between the out‐of game learning LS group and the in‐game adaptive group (Z = ‐1.546, p<0.124). So far, the tests have proven that the learning effectiveness for the modes of the game are significantly higher than those for paper‐based learning. Another statistical test is performed using the Kruskal‐Wallis test to compare all these groups (paper‐based, game without adaptivity, game with out‐of‐game LS and game with in‐ game adaptivity). The Kruskal‐Wallis test is significant beyond the 0.01 level (χ2 = 58.153, p<0.01). A summary of the comparison analysis is presented in Table 2. Table 2: Comparison of Learning effectiveness Groups

Learning Effectiveness between Groups

Paper‐based Group and Non‐adaptive Group

Non‐adaptive Group and Out‐of‐game LS Group

Z = ‐5.903, p<0.001 (Non‐adaptive Group > Paper‐ based Group) Z = ‐5.995, p<0.001 (Out‐of‐game LS Group >Paper‐ based Group) Z = ‐6.495, p<0.001 (In‐game Adaptive Group > Paper‐ based Group) Z = ‐0.872, p<0.389

Non‐adaptive Group and In‐game Adaptive Group

Z = ‐0.378, p<0.711

Out‐of‐game LS Group and In‐game Adaptive Group

Z = ‐1.546, p<0.124

All Groups

χ2 = 58.153, p<0.01

Paper‐based Group and Out‐of‐game LS Group Paper‐based Group and In‐game Adaptive Group

4. Conclusions and future directions Adapting the learning contents according to the student’s learning style has been identified to have a positive impact. An experiment was conducted to investigate the differences in learning effectiveness between different adaptive modes of a game and paper‐based learning and the difference in the completion time of the game groups. The results show that the game groups have significantly higher learning effectiveness compared to the paper‐based group. The in‐game adaptive game also produced higher learning effectiveness compared to other game groups. These findings are similar to the studies described in section 2.3. The results also indicate the potential of GBL as the supplement in teaching SQL. One of the considerations in designing an educational game lies in how to teach the learning materials without losing the ‘fun’ part of the game so the student can remain motivated to learn the materials. One way of maintaining motivation is to blend the materials and the game story in such a way that the learning materials are part of the challenge of the game. The findings here may be specific to the integration of these particular learning materials and the mechanics of the game therefore modifications in the game’s specification could give rise to different learning outcomes. However, the conversation system used to present the learning materials in the game is highly customisable and can provide a useful platform for exploring the game’s potential for teaching different disciplines or accommodating different learning styles. Further research based on this game and its variants may contribute empirical evidence of the beneficial effects of GBL and adaptive GBL in particular. The adaptivity may also be improved in future research particularly to address multiple learning styles in a game and to create more complex adaptivity in various elements of the game.

Acknowledgements This work has been co‐funded by the EU Lifelong Learning Programme under contract 519057‐LLP‐1‐2011‐1‐ UK‐KA3‐KA3NW (Ed2.0Work – European Network for the integration of Web2.0 in education and work) and as part of the Games and Learning Alliance (GaLA) Network of Excellence on ‘serious games’ funded by the Eurpean Union in FP7 – IST ICT, Technology Enhanced Learning (see http://www.galanoe.eu).

References Becker, K. (2007). Digital Game‐Based Learning Once Removed: Teaching Teachers. British Journal of Educational Technology, 38(3), 478‐488.

537

Mario Soflano, Thomas Connolly and Thomas Hainey Boyle, E.A., Connolly, T.M. Hainey, T. (2011). The Role of Psychology in Understanding the Impact of Computer Games, Entertainment Computing, 2(2), 69‐74. Bontcheva, K. (2002). Adaptivity, Adaptability, and Reading Behaviour: Some Results from the Evaluation of a Dynamic Hypertext System, AH’2002, 69‐78 Charles, D., McNeill, M., McAlister, M., Black, M., Moore, A., Stringer, K., Kerr, A., Kucklich, J. (2005). Player‐centred game design: Player modelling and adaptive digital games. Proceedings of DiGRA 2005 Conference: Changing Views – Worlds in Play, 285‐298. Conati, C., Zhou, X. (2002). Modelling Students’ Emotions from Cognitive Appraisal in Educational Games. Proceedings of th the 6 International Conference on Intelligent Tutoring Systems, London, UK, Springer‐Verlag, ACM Conati, C., Zhao, X. (2004). Building and Evaluating an Intelligent Pedagogical Agent to Improve the Effectivenesss of an Education Game. IUI'04, International Conference on Intelligent User Interfaces, Funchal, Madeira, Portugal, 6‐13. Connolly, T.M. Stansfield, M.H. (2006). Enhancing eLearning: Using Computer Games to Teach Requirements Collection and Analysis. Second Symposium of the WG HCI & UE of the Austrian Computer Society, 23 November 2006, Vienna, Austria. Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., Boyle, J. M. (2012). A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59, 661–686. th Demmel, R. B., Kohler, B., Krusche, S., Schubert, L. (2011). The serious game: wemakewords. Proceedings of the 10 SIGPLAN symposium on New ideas, new paradigms, and reflections on programming and software, New York, USA, ACM, 109‐110 Felder, R. M., Brent, R. (2005). Understanding Student Differences. Journal of Engineering Education, 94(1), 57‐72. Felicia, P., Pitt, I.J. (2007). Evaluating the Effect of Personalities on the Design of Educational Games. Proceedings of the ECGBL Conference, Paisley, Scotland Felicia, P., Pitt, I.J. (2008). Personalising Educational Games to Students' Learning Styles. Proceedings of the International Technology in Education and Development Conference, Valencia, Spain Felicia, P. Pitt, I.J. (2009). Profiling Users in Educational Games. In Games‐based Learning Advancement for Multisensory Human Computer Interfaces: Techniques and Effective Practices (Eds: T.M. Connolly, M.H. Stansfield, E Boyle), Idea‐ Group, Hershey. Hwang, G. J., Sung, H. Y., Hung, C. M., Huang, I., Tsai, C. C. (2012). Development of a Personalized Educational Computer Game Based on Students’ Learning Styles. Special Issue on Personalized Learning, 60(4), Springer, 623‐638 Jackson, S. L., Krajcik, J. Soloway, E. (1998). The Design of Learner‐adaptable Scaffolding in Interactive Learning Environments. Proceedings of ACM CHI 1998 Conference on Human Factors in Computing Systems, 197‐194. Jameson, A. (2003). Adaptive Interfaces and Agents. In Human–Computer Interface Handbook. (Eds: Jacko, J. A., Sears, A.). Lawrence Erlbaum, Mahwah, 305–330 Lee, M. J., Ko, A. J. (2011). Personifying Programming Tool Feedback Improves Novice Programmers’ Learning. Conference on International Computing Education Research (ICER), Providence, Rhode Island, USA, ACM, 109‐116 Lynch, R. A., Steen, M. D., Pritchard, T. J., Buzzell, P. R., Pintauro, S. J. (2008). Delivering Food Safety Education to Middle School Students Using a Web‐Based, Interactive, Multimedia, Computer Program. Journal of Food Science Education, 7, Wiley, 35–42 Kirriemuir, J., McFarlane, A. (2004). Literature Review in Games and Learning. Report 8, NESTA, Futurelab, Bristol. Kolb, D. (1984). Experiential Learning. Englewood Cliffs, New Jersey. Prentice‐Hall Inc. Melis, E., Monthienvichienchai, R. (2004). They call it learning style but it's so much more. Proceedings of World Conference on e‐learning in Corporate, Government, Healthcare, and Higher Education 2004, Chesapeake, VA, 1383‐1390 Miller, L. M. (2005). Using learning styles to evaluate computer‐based instruction. Computers in Human Behavior, 21, 287‐ 306 Mulwa, C., Lawless, S., Sharp, M., Arnedillo‐Sanchez, I., Wade, V. (2010). Adaptive Educational Hypermedia Systems in Technology Enhanced Learning: A Literature Review. Proceedings of the 2010 ACM Conference on Information Technology Education, ACM Oppermann, R., Rashev, R., Kinshuk. (1997). Adaptability and Adaptivity in Learning Systems. in Knowledge Transfer, (Eds: nd Behrooz, A), pAce, London, 2 ed, 173‐179. Peirce, N., Conlan, O., Wade, V. (2008). Adaptive Educational Games: Providing Non‐invasive Personalised Learning Experiences. Second IEEE International Conference on Digital Games and Intelligent Toys Based Education; IEEE, 28‐ 35 Prensky, M. (2006). Don’t Bother Me, Mom, I'm Learning!: How Computer and Video Games Are Preparing Your Kids for 21st Century Success and How You Can Help. St Paul, Paragon House. Price, L. (2004). Individual differences in learning: Cognitive control, cognitive style, and learning style. Educational Psychology, 24, 681‐698. Smith, L. H., Renzulli, J. S. (1984). Learning Style Preferences: A Practical Approach for Classroom Teachers. Theory into Practice, 23, 44‐50. Yongyuth, P., Prada, R., Nakasone, A., Kawtrakul, A., Prendinger, H. (2010). AgriVillage: 3D Multi‐Language Internet Game for Fostering Agriculture Environmental Awareness. Proceeding of the International Conference on Management of Emergent Digital Ecosystems, NY, ACM, 145‐152

538

Can Moral Sensitivity be Enhanced by Game Play? Gunilla Svingby Malmö University, Malmö, Sweden Gunilla.svingby@mah.se Abstract: The question of the title may seem senseless. Computer games presumably do not foster moral sensitivity. On the contrary, the player often takes the role of soldier, thief, monster etc. with the aim of winning at any cost. And yet, digital games carry great potentials for learning. The possibility to act and engage in virtual situations may be used in many other contexts with the aim of handling difficult moral dilemmas, in which the “practicing of goodness” may lead to understanding and empathy. The study seeks an answer to the question of the title by means of the game, Men and animals. Moral development of the young generation is recognized as a growing problem by western countries. As the lives of young people are characterized by moral ambivalence, and competing moral discourses, values seem to be increasingly uncertain and fragmented. Researchers describe such morality as “situational” in contrast to rule based moral values. To develop a more coherent and reflected morality, people have to meet, discuss, and act in a variety of complex situation, leading to deepened “moral sensitivity”. The digital game Men and animals was developed to meet such demands. Embedded in a narrative, the game presents the students to authentic dilemmas that can be “solved” in more than one way. The solutions represent different moral values. The game is played in pairs allowing for discussion. To solve the dilemmas a range of opinions and information is presented. The game can be played several times allowing students to test the consequences of polar value positions. Playing takes 45 minutes. Effects and experiences of playing the game was studied involving 100 students aged 16‐19. By interviews and a questionnaire students’ moral reasoning was tested in various situations before and after playing the game. Observation of players’ discussions and arguments while playing was included. The choices made by each pair were registered and positioned on a scale from “Absolute animal rights” to “Animals have no rights at all”. At the end of the play students saw all their choices, and were invited to reflect on them in relation to the value scale. This gave room for reflection on the situational character of the choices, which for most pairs mirrored an ambivalent moral position. A majority of the students experienced the continuous demand for decision‐ making as highly engaging – and difficult. The game made the students question their initial general idea of the relations between men and animals. Before playing, a majority ascribed the same value to animals as to men. Having played, students’ decisions became more relational and situational, mirroring a more reflected position. Students themselves commented on the problem of understanding the moral components of a dilemma, as well as their own position. They articulated a feeling of not having thought of such things in depth, and interest in playing more games that challenged their presumptions. Keywords: game, learning, moral sensitivity

1. Background The question of the title may seem senseless. Computer games presumably do not foster moral sensitivity. On the contrary, the player often takes the role of soldier, thief, monster etc. with the aim of winning at any cost. And yet, digital games carry great potentials for learning. The possibility of engaging and acting in virtual situations may be used in other contexts, even with the aim of handling difficult moral dilemmas. Research shows that games offering simulations of real dilemmas have the power of supporting learning (Jenkins, 2005, Kirriemuir & Mc Farlane, 2003, Barab et al. 2007). This paper explores the potentials of a digital game Men and animals to enhance students’ moral sensitivity. Why is the enhancing of moral development by a game an interesting issue? As part of growing up, young people have to build their values in interaction with the world they live in. As life in the “post‐modern society” is characterized by moral ambivalence and competing moral discourses values seem to be increasingly uncertain and fragmented. Researchers describe the morality as “situational” in contrast to rule based moral values (Bagnall, 1998; Nussbaum, 1990). Accordingly, the moral development of the young generation is recognized as a growing problem in many western countries. Schools have been assigned an important role in helping students to develop in this respect. The Swedish curriculum, for example, states that it is essential for the school to make students embrace the values of the Declaration of Human Rights, to recognize the existence of conflicting values, and to stimulate the discussion of ethical dilemmas.

539

Gunilla Svingby Young people have to develop the ability to reflect on and articulate the morality of their own actions. They have to meet, discuss, and act in a variety of complex situations. By developing situational sensitivity education may contribute to enhancing a truly situational morality (Bagnall, 1998). What sort of educational experiences may contribute to moral sensitivity in a world of conflicting moralities and life styles? The traditional method of transmitting of rules from teacher to students is not effective enough, which is confirmed by teachers, who express difficulties in dealing with moral issues (Oscarsson & Svingby, 2005). The shortcomings of formal schooling in this area is, further, shown by the results of four successive Swedish national evaluation studies in which 2 ‐ 4000 students answered questionnaires that contained both open and closed questions, and simulated real‐life situations. On the whole, students expressed a positive interest in discussing complex moral dilemmas, and an overall positive attitude to the general curriculum values (Svingby et al, 1990; Svingby, 1993, 1998; Oscarsson & Svingby, 2005; Jönsson & Persson 2006). When students had to act in life‐like situations many, changed choice of action and the underlying value of their choice. Age and gender influenced the answers, and the tendency to change when confronted with a tangible situation. At the age of 12, both girls and boys answered in line with the established values, but neither group could explain their answers. At the age of 16, girls still – on a general level – agreed with the curriculum values, while about one third of the boys did not. When the values of the Declaration of Human Rights were tested in “real” situations, many boys shifted from the altruistic values to an egoistic position. The studies further, indicated that boys lacked the competence to explain the reasons of their choices, whereas girls to a higher degree were able to give explanations to their choice of action. It seems as if the boys, neither in the school nor in the world outside of the school, had met the challenges of exploring and discussing the moral basis of an action or the consequences of alternative actions.

2. Games and learning As computer games are part of the every‐day life of many European young persons, and they already use them to act out various types of morality, it can be argued that today the experience of digital gaming is part of the construction of a student’s identity and morality (Svingby, 2005). The importance of experience ‐ lived or simulated ‐ in forming values is established which means that gaming experiences might be used for formal learning. Researchers argue that games have a great potential for learning by linking virtual world problems to problems in the real world, and by allowing players to test different identities and roles (Jenkins, 2005; Gee, 2006). Being involved in simulated situations which are interpretations of the situations‐as‐lived, the player can test out consequences in a virtual world before s/he acts in the real world. Games may, thus, be a means to develop alternative moral actions (Nussbaum, 1990, Gee, 2006). The question is, however, what type of moral values are practiced in games. While commercial games often offer the involvement with characters like soldiers or thieves, there is no reason why a game could not let players act in the moral dilemmas that ordinary people encounter in their daily life, and thus experience a match between the game world and the way ordinary men sees and acts in the real‐life world. Such a game may offer the possibilities to confront their own values in the game world. Gee (2006) argues in line with this: Since video games are “action‐and goal‐directed preparations for, and simulations of, embodied experience” they allow language to be put into the context of dialogue, experience, images, and actions. Furthermore, good video games give verbal information “just in time” – near the time it can actually be used – or “on demand” – when the player feels a need for it and is ready for it (p. 17)

3. A prototype of a digital game on moral issues The digital game Men and animals was developed to meet such demands. The game is seen as a prototype of a type of digital games that may be developed and used for the enhancement of moral sensitivity in a row of life situations. The intent of the game is to let players aged 15‐18 explore the complex interrelationships between men and animals. The game is based on the assumption that to build a morality of his/her own a student has to be engaged in authentic/virtual problems that can be “solved” in more than one way. When playing the game, students will meet with and explore different value positions. In each situation the players take action according to one of five different moral values. The consequences of each choice will be mirrored on the

540

Gunilla Svingby health‐and happiness scale of the “game inhabitants”. In order to make the decisions more informed and reflected the game presents relevant factual information offers time for discussion and reflection. The game narrative portrays a small virtual city. The players act the character of Mayor of the city. A range of realistic situations are presented which forces the Mayor (the players) to take decisions on if and how to use animals for medical research, for tests of cosmetics, and as food. Before taking a decision, players can ask for advice from the five members of the ‘Citizenship committee’. These are virtual characters designed to represent the five positions on a scale from animal liberationist – free all animals – to the position that man can use all animals without any restrictions (Orlans, 1993). Additional facts are presented to players if they ask for it. Various long term health problems representing different degrees of self‐infliction are presented to the Mayor, who chooses among five different long‐term actions. The demand for ‘fast action’ is met, when unforeseen things occur, which demand immediate action. The game is constructed to give the players enough time to reflect on the situations before taking a decision with enduring effects. Quite substantial texts are presented, and players are given time to read, reflect, and discuss. At any time, the players can get an indication of how satisfied the villagers are and of their health status. To use animals for medical research will, as an example, give credit from those positive to man’s use of animals at large, but will create hostility from ‘animal liberationists’. Continuously, players can also observe their decisions in relation to the five value positions referred to above. The recommendation to play the game in pairs is in line with the request for discussion and reflection. Playing takes about an hour and the game can be played several times allowing students to test the consequences of polar value positions.

4. The issue of men and animals The specific context of the game is man’s use of animals. This issue has grown controversial during the last decades, which is illustrated by questions like ‘For what reasons and to what extent can humans use animals? How can you distinguish use from abuse?’ There are no simple answers to these and similar questions. Since Singer published his book Animal liberation (1975) the debate has been intense on man’s right to use animals for his own good. The debate has focused on a range of situations where animals are used (and abused) by man. The positions range from on one hand seeing man first of all as an animal amongst animals to on the other hand stressing man’s superiority to the rest of the creation, with an absolute right to use animals in his own interests. One background to the study was a representative Swedish study including about 2000 students aged 15‐16, who among other questions about morality and action, answered questions about men’s use of animals (Jönsson & Persson, 2006). Half of the students answered yes on the general question Is a human’s life worth more than the life of an animal? implying that the Human rights declaration should also apply to the animal world (Ibid, 2006). When confronted with specific, authentic situations the majority, however, gave preference to men. About sixty percent of the students, for example, accepted medical tests on animals to find cures to severe illnesses. But there were restrictions. All animals were not ascribed the same value, and human illnesses were graded. Banana‐flies and rats were accepted for research, whereas dogs and cats were not, and few accepted to use animals to find a cure to self‐inflicted illnesses. Boys and girls as groups answered differently. While a majority of the boys in the authentic situations said that a man’s life has a higher value than that of an animal, only a third of the girls supported the same opinion. The dominant argument forwarded by the girls is that animals should not suffer because of human’s disastrous ways of life. The answers, further, revealed that students’ decisions to a large extent were based on emotions and not on factual knowledge or on reflected values. For the majority of the students, the answers, thus, did not mirror a reflected moral attitude on the use of animals, and the majority of the students seemed unable to articulate the rationale behind their moral choices (Oscarsson & Svingby, 2005). Empirical studies The game was designed and researched according to an iterative model called Design Based Research (Barab & Squire, 2004). Four field studies were conducted with in all 151 students (Svingby, 2005; Bergman & Svingby,

541

Gunilla Svingby 2006; Svingby & Jönsson, 2007). The game was played in pairs for an hour as part of the ordinary teaching of Social Studies. Before playing, students answered a questionnaire about their attitudes to animals, and about their gaming experiences. Some of the questions were identical to the ones used by the National Evaluation Study of 2003 (ibid). The questions were a mix of open and multiple‐choice. Some questions portrayed authentic dilemmas. After having played students answered the questionnaire again. Students were also observed while playing, and in two of the empirical studies they were interviewed.

5. Research questions and data collection The overall aim of the studies was to investigate if the game, when used in a school setting, can help young people develop a more reflected, and informed moral attitude to the issue of man’s use of animals. As the game is seen as a prototype of a type of game designed to promote the experience of the reflection on moral issues of various kinds, the aim was secondly to get information about how students played the game, and how they reacted to this specific type of digital game. The following more specific questions were explored:

1. Do students take time to discuss before they take a decision? What type of discussions do students engage in?

2. Do students use the factual information offered by the game? Do they express a wish for more information?

3. Which moral positions do players choose? Is there a pattern?

4. Do students change their moral position towards man’s use of animals?

6. Design Research data was collected as follows: before playing the game, during play and after play from 148 students aged 15‐18, who played in pairs (78 women and 70 men) as is shown below: Questionnaire 1 Gameplay + Observation Questionnaire 2 (+Interview) Before playing: Students’ attitudes to man’s use of animals were collected by a questionnaire with multiple‐choice and open questions. Some of the questions portrayed authentic dilemmas. During playing: Students’ discussions when playing were gathered by observation/ field notes. In three of the studies, one researcher who was present in the classroom observed all discussions between the players while playing. In one study, each pair of players was individually observed. Students’ choices were registered. The choices made by each pair in the various situations were registered and positioned on a scale from Absolute animal rights to Animals have no rights at all. At the end of the game session, students were presented to all their choices, and were invited to reflect on them in relation to the value scale. After playing: Students’ experiences of playing the game were gathered by multiple‐choice and open questions and by interviews. Students’ attitudes to man’s use of animals were collected by a questionnaire with multiple‐choice and open questions. Some of the questions portrayed authentic dilemmas.

7. Results Playing the game

542

Gunilla Svingby The game was very well received by the students, who found it interesting and provoking. A majority of the students experienced the continuous demand for decision‐making as highly engaging – and difficult. In contrast, a group of experienced game‐players, mainly males, mentioned the striving for credits as the most engaging. The observations confirmed that this group played the game differently. They did not discuss the situations as moral dilemmas but rather how to get high scores on the health‐and‐happiness scale. Except for the male students mentioned above, most of the students were observed to be intensely engaged in discussions both when playing the ‘slow’ situations and the situations demanding immediate action. According to most students the embedded realistic situations where you have to make difficult decisions, and can see the effects of on people in the virtual community, was the most interesting aspect of the game. Comments like the following were frequent: ‘It makes you aware of how society has to deal with problems that its inhabitants evaluate differently’. ‘How difficult it must be to be a politician!’, and ‘It was an interesting game that made us think on ethics and animals right in a way we had never done before’. The fact that the game was played in the classroom added to its value. Use of the inbuilt and external information Students appreciated both that different opinions were present in each situation, and the factual information offered in the game. Many expressed an interest in acquiring more information on themes such as ‘how animals are treated in research laboratories’, ‘national and international laws on animal rights’, and ‘the causes of various types of human illnesses’. This information can be found via internet links connected to the game. Few students, however, used the links while playing, but they may be used by teachers in a follow up of the game. An example lesson is offered as a complement to the game.

8. Values On the general question of the worth of men and animals, the students answered in line with earlier studies (Oscarsson & Svingby, 2005). About half of the students, thus, agreed that ’A human’s life is worth more than the life of an animal.’ Playing the game seemed to have an effect of students’ reasoning about men and animals. Even if the initial attitude did not change much, answers to the open questions showed that playing had challenged students’ presumptions. The answers to the second questionnaire, thus, showed to be more situational, which was mirrored in the choices made in the game. Depending of both the problem and the specific animals and men involved, students’ choices varied on the scale, mirroring both situational morality and the fact that the situations and dilemmas were new to the students. Students, thus, to a greater extent differentiated between various types of animals ‐ and men. For example, more students accepted animal testing for medical research using rats and pigs, but not to cure ‘self‐inflicted’ illnesses: ‘Animals should not suffer because people got ill by drinking alcohol’. Further, after playing, the answers from female students, who initially were more restrictive to the use of animals differed less from those of the men. The majority of the students said that playing the game may have an impact on a player’s attitudes, by making you more observant, more conscious, and more sensible on the issue. That students changed quite little as measured by the questionnaire may be explained by the fact that the problems encountered by the game were new to them. A majority said that they had seldom thought of moral issues in depth, and that the issue of men and animals had not been discussed either at school or at home. The majority expressed an interest in playing more games that challenged their presumptions.

9. Conclusions The ambition with this project was to explore the potentials of a digital game to enhance moral sensitivity on the issue of men and animals. The results are limited to the specific content area, but the ambition is to see the game as a prototype of games for learning with the aim of offering virtual experiences of value laden situations to promote greater sensitivity, and competence in various moral dilemmas (Svingby 2009). In order to do so, the specific game was built on the assumption that when exposed to virtual but authentic situations which can be solved in several ways, young people are helped in the building of reflected values. Accordingly, the situations offered have to be difficult, mirroring the multitude of value positions possible, and the underlying moralities. The game, further, must be built so that the players will be informed as well as emotionally involved. The game design gave room for reflection on the situational character of the choices.

543

Gunilla Svingby The game content is very close to real life. The problems that the players are confronted with in the virtual world have their equivalents in the real world. The game brings questions of similarities and differences between men and animals, of human versus animal rights, of research ethics to the fore, questions that can be followed up in the classroom. As the players take the virtual identity of a decision maker, being constantly confronted with problematic situations that demand a choice between alternative solutions, most students are prompted into discussions of the ethical basis of a solution. The discussions were both general and specific: Shall animals have the same rights as men? In all situations? All animals‐ or only the ‘cute and curly’? Have some animals have more rights than others, and do some men have less rights than others? The impact of a game will always depend on to what extent the players engage in the moral problems of the game, or in the striving for the highest possible credits. As Gee (2003) argues, students’ projective identity, the identity they find most appropriate in the play situation, either reflects the players own values independent of the credit given, or it reflects the values that the player estimates will give the best credit. The studies show that most students did engage in the moral problems, but that a group of experienced males mainly looked for a way to gain the most credit. A revised game must adapt also to these players by making reflections on the dilemmas necessary to gain high credits. An hour is a short time to engage in difficult dilemmas. To really contribute to students’ moral sensitivity, the issues need to be discussed further in the classroom and outside of it, and maybe the game can be played more than once with the aim of exploring what will happen if you adapt all choices to a definite moral position. The study supports the assumption that a good simulation game on a central issue may be a valuable tool to stimulate students’ engagement in and learning of values, and indicates that the actual game can be a rewarding tool both for engaging students in moral problems, and for the deepening of the understanding of such problems. Further studies of actual learning and of the integration of games in the school learning system are needed to realize the full learning potential of games in education. The educational use of electronic simulation games works the same way‐‐not as a replacement for good teaching or tried‐and‐true methods, but as a tool that good teachers can use to spark learning and to provide a context for a range of other related experiences. More and more teachers are bringing games into their classrooms on these terms” (Jenkins, 2005).

References Bagnall, R. (1998) Moral education in a Postmodern World: Continuing Professional Education, in Journal of Moral Education, 27, pp 313‐331. Barab, S. A.,& Squire, K. (2004). Design‐Based Research. Putting a Stake in the Ground. Journal of the Learning Sciences. Volume 13, Issue 1, 2004. Barab, S. A., Sadler, T. D., Heiselt, C., Hickey, D. & Zuiker, S. (2007). Relating Narrative, Inquiry, and Inscriptions: Supporting Consequential Play. Journal of Science Education and Technology, 16 (1), 59‐82. Bergman, P. & Svingby, G. (2006). Utveckling och utprövning av ett elektroniskt spel om människors förhållningssätt till djur. Interimsrapport 2006‐02‐23. Malmö: Malmö University. Gee, J.P. (2003). What Video games Have to Teach Us About Learning and Literacy. USA: Palgrave Mac Millan. Gee, J.P. (2006). Why are Video Games Good for Learning? Madison, USA: University of Wisconsin‐Madison, Department of Curriculum and Instruction, Department of Educational Psychology. Jenkins, H. (2005). Getting Into the Game. Educational Leadership, Apr.2005, Vol. 62, Issue 7. Virginia, USA: Association for Supervision & Curriculum Development. Jönsson, R. & Liljefors Persson, B. (2006). Religionskunskap i årskurs 9: Rapport från den nationella utvärderingen av grundskolan 2003 (NU03). (Religious Studies in Year 9: The report of the national evaluation of the compulsory school 2003), Malmö: Malmö University: Educare. Kirriemuir, J. & McFarlane, A.E. (2003). Literature Review in Games and Learning, Report 7. Bristol: Nesta Futurelab. Nussbaum, M. (1990). Love´s Knowledge: essays on philosophy and literature. Oxford: Oxford University Press. Oscarsson, V. & Svingby, G. (2005). Sanhällsorienterande ämnen. Ämnesrapport (Citizenship education. Main report. The national evaluation, 2003) Stockholm: Skolverket. Orlans, F.B. (1993). In the Name of Science. Issues in responsible Animal Experimentation. New York: Oxford University Press. Singer, P. (1975). Animal Liberation: a new ethics for our treatment of animals. New York. Svingby, G., Lehndals, B.& Ekbom, D. (1990). Omvärldskunskap:SO. Bakgrund, Beskrivning av undervisningen. Fostran till demokrati. (Social Studies: Background, The teaching, Democratic values). Gothenburg: University of Gothenburg. Department of Education, 1990:02.

544

Gunilla Svingby Svingby. G. (1993). The result of the National evaluation of Social Studies, 1992. (Samhällsorienterande ämnen. Den nationella utvärderingen av grundskolan 1992). Stockholm: Skolverket, nr 17. 1993. Svingby, G. (1998). Vad är rätt och rättvist? Ungdomars val i etiska situationer som utryck för demokratisk kompetens. Samhällsorienterande ämnen. (What is right and what is just? The National Evaluation of Social Studies, 1995). Stockholm: Skolverket. Nr.125. Svingby, G. (2005). Teaching citizenship with a collaborative computer game. In Ross, A. (Ed.). Teaching citizenship, pp487 – 493. CiCe publication, London Svingby, G. & Jönsson, R. (2007). What is Use and What is Abuse of Animals? Malmö: Malmö University. Working paper. Svingby, G. (1997). Utvärdering av de samhällsorienterande ämnena. (The 1995 National Evaluation of the Compulsory School: Social Studies). Utbildning och Demokrati, tidskrift för didaktik och utbildningspolitik, 2/1997. Göteborg. Svingby, G. (2009). Practicing goodness by a computer game. A model for enhancing children’s pro‐social behavior. Scientific Project Plan. Malmö: Malmö University, Working paper.

545

Digital Educational Games: Adopting Pedagogical Agent to Infer Leaner‘s Motivation and Emotional State Ogar Ofut Tumenayu and Olga Shabalina Volgograd State Technical University, Volgograd, Russia ofuriti4u@yahoo.com O.A.Shabalina@gmail.com Abstract: Digital educational games (DEGs) possess the ability of providing an attractively and essentially motivating learning context. However, an adaptive learning game would increase the probability of a DEG being actually motivating and emotionally appealing. A pedagogical agent‐based environment suggests a new opportunity for computer mediated learning emphasizing virtual social relations between learners and computers. The overall goal of this work is to provide pedagogical agents with social intelligence, so that they can decide when is an appropriate time to interact with the learner, be sensitive to the motivational and emotional state of the learner, and try to develop a positive social relationship with the learner. The study examine the effects of the competency (low vs. high) and interaction type (proactive vs. responsive) of pedagogical agents as learning companions (PALs) on learning, self‐efficacy, and attitudes. Interactions are been analyzed as a series of interaction tactics, where the speaker seeks to address one or more informational, motivational, or social goals, and monitors the listener’s response to ensure that these goals are achieved. This is followed by brief look at Affective Human Agent Interaction to have Existing agent systems typically infer human affect by sensing and reasoning about the state of a game or an outcome related to an action taken by the user within the learning environment. Keywords: pedagogical agent, pedagogical agents as learning companions, interaction type, motivational and emotional state

1. Introduction Animated pedagogical agents can promote effective learning in computer‐based learning environments (Johnson, Rickel, and Lester 2000). Learning materials incorporating interactive agents engender a higher degree of interest than similar materials that lack animated agents (Moreno, Mayer, Spires, & Lester 2001). Animated agents can produce a positive affective response in the viewer, sometimes referred to as the persona effect (Lester, Converse, Kahler, Barlow, Stone, and Bhogal 1997). This effects would go long way to improve the motivation and emotional state of the learner. They exploit the computer human tendency to tread computer systems as if they are social actors (Reeves and Nass 1996), capable of expressing emotions and attitudes. It has also been acknowledge by researchers the important in tutoring of recognizing the learner’s affective and motivation state ( Lepper, Woolverton, Mumme, & Gurtner 1993). Inferring and tracking computation techniques are being developed for such state (De Vicente and Pain 2002). (Conati, C. & Zhou, X. 2003) and for influencing them positively e.g. to try promote a positive atti+tude about learning ( Del Soldato and du Boulay 1995). In a nutshell, this paper would report the idea of social interaction procedures between pedagogical agent and the learner in a DEG in order to improve motivations and emotions in game based learning. It main purpose is to discuss how pedagogical agents can help to make better learners’ levels of interaction with the course content.

2. How pedagogical agents (PAs) heighten effective motivations and emotions to foster learning 2.1 Pedagogical agents (PAs) in DEG Pedagogical agents are human‐like characters that are included in educational and instructional materials in order to somehow stimulate learning (Eliseo, Eunice, and Letícia 2007). Pedagogical agents can be defined as entities that can perform a task or a set of tasks in addition to performing pedagogical roles. Animated agents can mimic human gestures and emotion‐driven expressions, which can help engage students’ more than static avatars (Elizabeth & Debbie). When provided with shape, personality and abilities of human agents can play the role of human‐like characters.

546

Ogar Ofut Tumenayu and Olga Shabalina With the help of Artificial Intelligent Pedagogical agents can be autonomous with a way of thinking through knowledge representation and the use of rules. They can have abilities like moving, talking, feeling and reacting to user within their environment. “a learner can learn content through interacting with one or more pedagogical agents, who may provide information or encouragement, share menial tasks, or collaborate with the learner” (Kim and Baylor, 2006, pp. 224). Pedagogical agents allow communication and interaction in learning environments (Girara & Viccari 1998). Animated pedagogical agents, especially life like characters, have significant motivational benefits; play an important pedagogical role by acting as virtual learning companions (Maragos & Grigoriadou 2005). Agents can also have different roles to support education, for example observing the students' actions and assessing them, in addition to providing feedback, explanations and demonstrations to the learner (Hospers et al. 2003; Abbas & Sawamura 2009). In a study conducted by Kim and Baylor (Kim and Baylor, 2006, pp. 224) it was found that students who worked with high‐competency Animated Pedagogical Agents – in both proactive and responsive conditions – achieved higher scores in applying what they had learned and showed more positive attitudes toward the animated pedagogical agents. According to Atkinson (Atkinson 2002) voice animated pedagogical agents environments have better results in measures of learning than text‐only or audio‐only environments. Moreover, emotional agents can be used to support the student system interactions and provide human‐like tutoring (Nkambou 2006; Neji et al. 2008). In this way, the activities of pedagogical agents can create a situation where by learners’ engaged themselves in a more meaningful interaction in which can infer the learners’ motivation and emotional state (see Figure 1).

2.2 Pedagogical agents (PAs) can create emotion and motivation in learning process According to Scherer (Scherer 2000), emotion is the relatively brief episode on synchronized responses for most or all organic systems for the evaluation of an external or internal event as being of major significance. Some examples of emotions are anger, sadness, joy, fear, shame, pride, and desperation. According to the approach of Peter and Herbon (2006) emotions are represented by the circumflex model of emotion (e.g. Russell, 1980; Larsen and Diener, 1992). The circumflex model is divided into two continuous dimensions of pleasantness and activation. Pleasantness or valence is considered as a bipolar dimension with the two poles pleasantness and unpleasantness. Activation, also known as arousal, is considered as a unipolar dimension with the poles of low and high activation (e.g. Harcourt and Lang, 1995). Each emotional or affective state can be described in terms of these two independent dimensions. For example, the emotional state excitement could be characterized as highly activated and pleasant (Larsen and Diener, 1992). Studies on the effect of emotional states on learning outcomes and efficiency (in particular with respect to valence) have yielded ambiguous results (Bower, 1992). However, with respect to activation, research indicates unambiguously that a medium level of activation leads to a superior learning process in terms of efficiency and sustainability in comparison too high or too low activation levels (Revelle and Loftus, 1992). Frijda (1994)considers that the emotion is an intentional mental state, because it is “directed toward” an object, its intentional object. For instance: I am angry with John, but I admire Nicholas. Integrating pedagogical agents with emotional features into a learning environment, aim at capturing and managing the emotions expressed by the learner during a learning session. The inclusion of emotional features in pedagogical agents is with high importance for improving the level of interaction in man machine communication system. The works on affectivity and education have also considered the motivation of the student. Student motivation deals with the student’s desire to participate in the learning process (Ames, 1990).Formal studies show that motivation was seen as a personality trait, as something people had in higher or lower degrees, a depending part on their genetic nature and on their childhood experiences (Meece &, 2001). However, current researchers have established that motivation is sensitive to contexts and it can be fostered in digital educational game. This way, a lot of works have developed in the sense of fostering student’s motivation to learn in educational computing systems (Del Soldato & De Boulay,1995; Bercht & Viccari, 2000; De Vicent & Pain, 2002). Motivational aspects in Digital Educational Game can be addressed from two perspectives, the perspective of the learning game design (as a combination of typical game design with instructional design, which can be considered being more than the sum of its parts) and the perspective of the game’s dynamic and intelligent adaptation to the motivational state of the learner. The approach to motivational aspects in game‐ based learning can be utilized for both, design and adaptive features of the game (Elke Mattheiss 2009).

547

Ogar Ofut Tumenayu and Olga Shabalina

Figure1: Pedagogical agent techniques: DEG hosting PA Now pedagogical agents with a high sense of motivation and emotion state in digital educational games are likely to optimize interactivity and further enhance a better flow of the game, which can promote a focused attention of the player: Firstly, by making the aim to be achieved understandable at all point through direct interaction with the agents with a clear provided feedback without the player bordering to think on what to do. Secondly, the challenges would be address to suit the ability of the player in order to avoid negative experience like boredom or anxiety in order to encourage a better flow. And the player can now be in control of the situation. Thirdly, the game would be more instructional to play with defined procedure of actions, which would promote fluentness. A better usage would avoid the player engaging on inappropriate actions.

3. The competency of a pedagogical agent and how it interacts between peer 3.1 Impact of pedagogical agents interactivity By simulating human instructional roles, pedagogical agents may provide learners with similar social contexts. Given that human computer interaction is consistent with human‐to‐human interaction (Reeves & Nass, 1996), learners might become more engaged in learning tasks through social interaction with pedagogical agents. Taking into concentration the potential of pedagogical agents for learning, several studies have examined the instructional impact of pedagogical agent‐based learning environments. Learners exposed to an environment with a pedagogical agent demonstrated deeper learning and higher motivation than learners without an agent (Moreno et al., 2001). Students in a voice‐plus‐agent environment outperformed those in a text‐only environment and those in a voice‐only environment on both process and product measures of learning. Similarly, students in the voice‐plus‐agent environment perceived worked‐out examples as being less difficult than did their counterparts (Atkinson, 2002).

3.2 Pedagogical agents as learning companions (PALs) Baylor and Kim (2003c) effectively simulated agents as an expert, a motivator, and a mentor who served distinct instructional purposes. The expert agent provided learners with relevant information in the professional manner; the motivator agent provided verbal persuasion and encouragement, emphasizing affective affiliation with learners; and the mentor agent incorporated both qualities to simulate an ideal instructor. In contrast to the other agent roles, pedagogical agents as learning companions (PALs) adopt a peer

548

Ogar Ofut Tumenayu and Olga Shabalina metaphor, where the agent serves to learn “with” the learner, acting as a simulated peer. In this manner, PALs have the potential to strengthen social relations and to facilitate social interaction during the learning process. PALs are pedagogical is descript to be an agents with the specific role of peers. PALs are different from other learning companion systems. PALs with peer‐like appearances and behaviors interact with learners to simulate traditional peer mediated learning. PALs may be able to eliminate some limitations like pairing logistics and distractions. Bandura’s social cognitive theory (2001) supports the benefits of human peer partners in intellectual and social development. According to Bandura, a great deal of psychological modeling occurs when learners see their everyday associates as similar to themselves. Learners tend to enhance their self‐efficacy beliefs based on perceptions of peer models. Peer interaction can provide a free and open forum to facilitate a more active, and thus a more productive, exchange of ideas (Driscoll, 2000). The main function of PALs is to serve similar functions as human peers; and also, the functions of PALs can be designed to meet specific instructional goals. Here, the potential functions of PALs for learning and motivation are suggested in terms of social cognitive theories. According to distributed cognition, PALs can share learners’ cognition and flexibly function as artifacts (cognitive tools) or collaborating partners (social tools). PALs as cognitive tools can bring knowledge and skills that learners intend to learn and mechanical tasks to preserve the ample cognitive capabilities of the learners for higher mental activities (person‐plus cognition). PALs as social cognitive tools can provide learners with social contexts by discussing and sharing ideas and perspectives with the learners. In these ways, PALs could play significant roles as social intellectual partners for learning. Similarly, Salomon (1990) suggests two types of outcomes in technology‐mediated learning, effects of technology and effects with technology. Effects of technology include the acquisition of cognitive skills resulting from the use of technology. Effects with technology are cognitive changes occurring “while” the learner is working with computer software. PALs as both artifacts and collaborating partners support learners in attaining both outcomes.

3.3 PALs competency PALs can be designed to achieve different levels of competency depending on the theoretical perspective. A PAL could be designed to demonstrate knowledge and skills maximizing the effect of technology (Salomon, 1990) or to advance the learner’s knowledge in the zone of proximal development. Likewise, the concept of proxy agency (social modeling) supports PAL competency that is high, but not too high, to serve as a desirable social model for learning. As Bandura (2001) warned, a highly competent model could sometimes impede the cultivation of personal competencies because the learner might unduly rely on the model’s competency. In a review of literature on peer modeling, Schunk (1987) noted that similarity in competency between a learner and a model might serve as an important source of self‐efficacy information, especially in unfamiliar tasks, where the learner had little information on which to base self‐efficacy judgments. It would be apparently reasonable for learners, for especially probationer learners, to increase their self‐ efficacy in the task more by working with a low competency PAL than with a high Competency PAL. The educational advantage of this approach is worth using. There would be need for further research to confirm the findings of studies and to identify if the optimal level of PAL competency would interact with learners competency.

3.4 Interactive type of PAL The interaction that occurs among students is extremely dissimilar between a Digital Educational Game and the traditional classroom course. The agents’ format excludes physical interaction, which may contribute positively on learning. The interaction type suitable for PAL in these contexts is learner –learner Interaction. Learner‐learner interaction can be between one learner and a tutee or between several tutees. Goodman and his colleagues (1998) suggested a learning triangle that illustrates the types of interaction exchanges between a learner and a partner: clarify, critique, explain, question, evaluate, articulate, reinforce, and justify. Related to the content of interaction, Craig, Gholson, Ventura, Graesser, and Group (2000) examined the effect of learning companion dialogue versus monologue with 48 college students in a domain of computer literacy. Results showed that students who overheard the dialogue in which a virtual tutee asked a virtual tutor questions wrote significantly more propositions in free recall and asked significantly more questions in the transfer task than did students who overheard a monologue. Also, students who overheard the dialogue showed deeper‐level reasoning. In order to promote effective learning by using learner ‐learner, four types of peer behavior are necessary in a computer mediated environment: (a) participation, (b) response, (c) provision of affective feedback, and (d)

549

Ogar Ofut Tumenayu and Olga Shabalina short, focused messaging. Team work, or collaborative learning, involves students working together in groups to complete academic assignments (Alavi, 1994; Palloff & Pratt, 2001). Learner – learner interaction form would increase understanding of the course content and encourage critical thinking among learners. Also collaborative work may eliminate feeling of isolation and promote team work spirit within learners in digital educational games.

4. Improving PA interactivity: From PA techniques to tutor and tutee agents’ techniques This paper has argued strongly that pedagogical agents can provide or create motivations and emotions in DEG in order to infer learning. In order to achieve this we suggest an approach that considers Pedagogical agent techniques as Tutor and Tutee agent techniques to create a high social and collaborative DEG environment. The Tutor Agent (TrA) is engineered to interact directly with a learner and explicitly guide him/her through the domain. These pedagogical agents are applicable in teaching components and user interfaces. They encourage the learner by providing feedback within the learning environment. The Tutee Agent (TeA), similar to Pedagogical Agent as Learning Companions (PALs), serves as a Peer‐Mediated Learning Agent, this type of agents are integrated into the user interface to act as interactive partners in a learning his process. For a long time, educators have observed the benefits of peer interaction and implemented systematic peer‐mediated interventions to increase achievements and motivation. Peer‐ mediated learning (also called peer‐assisted learning) refers to learning through interactive help and support among peers (K. Topping & Ehly, 1998). The peer agent competency level is model in such a way that the peer agent intellectuality is a little bit higher than that of the learner, with these they can share information and collaborate with each other in a game based learning environment. With the combination of TrA and TeA in DEG would not only affected leaner‘s motivation and emotional state but would provide a fruitful and powerful way of enhancing learning practices by eliminating the problem of isolation and promoting team work spirit among learner’s in a digital learning environment (see Figure 2).

Figure 2: Improved pedagogical agent techniques: DEG hosting TrA and TeA

5. Discussion When we critical consider already existing DEG with Pedagogical agent functions we have every reason to believe that they are still some limitations. The most common among them is the absence of learner’s companion and lack of social interaction in the games. This limitation could promote isolation in game based

550

Ogar Ofut Tumenayu and Olga Shabalina learning. We have decided to diversify the functionality of a Pedagogical agent by introducing a Sub‐agent (TeA) as a learner’s companion. While the main agent (TrA) is engage in tracking the learner’s activities and inferring learner state, the TeA initiate interactions with the learner at the appropriate times and in appropriate manners. For instances, we want to enable an agent to intervene proactively when the learner appears to be confused or stuck. This kind of intervention would be better handle by TeA because of the type of interaction (Learner‐learner interaction) that exists between them. A proactive intervention is very important because sometimes the learner may feel reluctant to ask for help directly from the TrA. If the learner in any point in time have difficulty understanding the game, it may be appropriate for the TeA to interrupt and offer assistance to the learner. The competence of the TeA is in every level of the game higher than that of the learner, in order to continuously monitor the learner’s activities. If the TeA does not track the learner’s activities accurately it could interrupt the learner with assistance when it is not really needed, which could disrupt the flow of the game. The social interaction between TeA and the learner would be able to influence the learner motivation and emotional state.

6. Conclusion and future work In this paper, we highlighted the importance of boosting interactivity content in DEGs by adopting pedagogical agent to heighten motivation and emotional state for game‐based learning to become widely acceptable in classroom teaching and learning. DEGs have long proved to enhance learning in a digital environment. By creating a real‐classroom situation in a safe challenging environment, a DEG with learning companion agent would contribute immensely to knowledge, and skills acquisition through an attractive and motivational learning interface. The introduction of tutee agent when combined with the social interaction abilities of a pedagogical agent in DEG would promote more than just individual acquisition of knowledge: they would provide some learning affiliations and encouragement to the learner by sharing menial tasks, and also collaborate with the learner. Furthermore, we are currently developing a game prototype to demonstrate the effectiveness of our approach. There would be need for further research to confirm the findings of studies and to identify if the optimal level of tutee agent competency would interact with learners competency. If the two modules are integrated successfully, the games would be used for self‐learning.

References Atkinson, R.K.(2002) Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology, v. 94, n.2, pp 416‐427. Ames, C. (1990) Motivation: What Teachers Need to Know. Teachers College. 91, 3, pp 409‐21. Alavi, M. (1994).Computer‐mediated collaborative learning: An empirical evaluation.MIS Quarterly, 18, pp 159‐174. Baylor, A. L., & Ryu, J. (2003). The API (Agent Persona Instrument) for assessing pedagogical agent persona. Paper presented at the International Conference of Ed‐Media, Honolulu, Hawaii. Bower, G.H. (1992) ´How might emotions affect learning?´ in Christianson, S.A. (ed.) The handbook of emotion and memory. Hillsdale, NJ: Erlbaum, pp.3‐31. Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of Psychology, pp 52, 1‐26. Craig, S. D., Gholson, B., Ventura, M., Graesser, A. C., & Group, T. R.(2000). Listening in on dialogues and monologues in a virtual tutoring session: Learning and questioning. International Journal of Artificial Intelligence in Education, 11, pp 242–253. Conati, C. & Zhou, X. (2003). A probabilistic model to assess user affect from goals, personality, and context. Proc. of the Intl. Conf. On Intelligent User Interfaces. New York: ACM Press. Driscoll, M. P. (2000). Psychology of learning for instruction. Boston, MA: Allyn & Bacon. De Vicente, A. and Pain, H. (2002). Informing the detection of the students’ motivational state: An empirical study. In S.A. Cerri, G. Gouardères, & F. Paraguaçu (Eds.), Intelligent Tutoring Systems: 6th International Conference, ITS 2002, pp 933‐943. Berlin: Springer‐Verlag. Del Soldato, T. and du Boulay, B. (1995). Implementation of motivational tactics in tutoring systems. J. of Artificial Intelligence in Education 6(4), pp 337‐378. Eliseo Reategui, Eunice Polonia, and Letícia Roland (2007). The role of animated pedagogical agents in scenario‐based language e‐learning: a case‐study, Author manuscript, published in "Conference ICL2007, September pp 26 ‐28, 2007, Villach : Austria (2007)" Frijda, N. (1994).Varieties of affect: emotions and episodes, moods, and sentiments. In Ekman, P. & Davidson, R. J. (Ed.). The Nature of Emotion. New York: Oxford Univ. Press, pp 59‐67.

551

Ogar Ofut Tumenayu and Olga Shabalina Gira, L. & Viccari, R. (1998). The use of agents techniques on intelligent tutoring systems. In XVIII International Conference of the Chilean Computer Science Society, pp.76. Goodman, B., Soller, A., Linton, F., & Gaimari, R. (1998). Encouraging student reflection and articulation using a learning companion. Paper presented at the 8th International Conference on Artificial Intelligence in Education, Kobe, Japan. 9 (3‐4), pp 237‐255. Hospers, M., Kroezen, E., Nijholt, A., op den Akker, R., & Heylen, D. (2003). Developing a generic agent‐based intelligent tutoring system. In Third IEEE International Conference on Advanced Learning Technologies (ICALT'03), pp 443. Harcourt, A.H and Lang, P.J. (1995) ´The emotion probe. Studies of motivation and attention´, American Psychologist, vol. 50, no. 5, pp 372‐385. Johnson, W.L., Rickel, J.W., and Lester, J.C. (2000). Animated Pedagogical Agents: Face‐to‐Face Interaction in Interactive Learning Environments. International Journal of Artificial Intelligence in Education 11, 47‐78. Kim, Y.; Baylor, A.L. Pedagogical Agents as Learning Companions: The role of agent competency and type of interaction. ETR&D, v. 54, n. 3, pp 223‐243, 2006. Larsen, R.J. and Diener, E. (1992) ´Promises and problems with the circumplex model of emotion´, in Clark, M.S. (ed.) Review of personality and social psychology: Emotion. Newbury Park: Sage, pp 25‐59. Lester, J. C., Converse, S. A., Kahler, S. E., Barlow, S. T., Stone, B. A., and Bhogal, R. S. (1997). The persona effect: Affective impact of animated pedagogical agents. In Proceedings of CHI '97, 359‐366. Lepper, M.R., Woolverton, M., Mumme, D., & Gurtner, J. (1993). Motivational techniques of expert human tutors: Lessons for the design of computer‐based tutors. In S.P. Lajoie and S.J. Derry (Eds.), Hillsdale, NJ: Lawrence Erlbaum Associates. Computers as cognitive tools, pp 75‐105. Maragos, K. & Grigoriadou, M. (2005). Towards the design of intelligent educational gaming systems. In proceedings of AIED workshop5, at the 12th International Conference on Artificial Intelligence in education, Amsterdam, pp 35‐38. Meece, J. & McColskey, W. (2001) Improving Student Motivation. Serve. Available at: http://www.serve.org/Products/ProdPub.php. Moreno, R., Mayer, R. E., Spires, H. A., & Lester, J. C. (2001). The case for social agency in computer‐based teaching: Do students learn more deeply when they interact with animated pedagogical agents? Cognition and Instruction, 19(2), pp 177‐213. Nkambou, R. (2006). Towards affective intelligent tutoring system. In Workshop on Motivational and Affective Issues in ITS. 8th International Conference on ITS 2006, pp. 5‐12. Palloff, R. M., & Pratt, K. (2001). Lessons from the cyberspace classroom: The realities of online teaching. San Francisco: Jossey‐Bass. Peter, C. and Herbon, A. (2006) ´Emotion representation and physiology assignments in digital systems, Interacting with Computers, vol. 18, no. 2, pp 139‐170. Russell, J.A. (1980) A circumplex model of affect, Journal of Personality and Social Psychology, vol. 39, no. 6, pp 1161‐1179. Revelle, W. and Loftus, D. (1992) The implications of arousal effects for the study of affect and memory, in Christianson, S.A. (ed.) Handbook of emotion and memory. Hillsdale, NJ: Erlbaum, pp 113‐150. Reeves, B., & Nass, C. (1996). The media equation: How people treat computers, television, and new media like real people and places. Cambridge: Cambridge University Press. Scherer, K. (2000) Psychological models of emotion. In J. Borod (Ed.) The neuropsychology of emotion. Oxford/New York: Oxford University Press, pp 137‐162 Salomon, G. (1990). Cognitive effects with and of computer technology. Communication Research, 17(1), pp 26‐44. Schunk, D. H. (1987). Peer models and children's behavioral change. Review of Educational Research, 57(2), pp 149‐174. Topping, K., & Ehly, S. (Eds.). (1998). Peer‐assisted learning. Mahwah, NJ: Lawrence Erlbaum Associates.

552

Adapting the Complexity Level of a Serious Game to the Proficiency of Players Herre van Oostendorp1, Erik van der Spek2 and Jeroen Linssen3 1 Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands 2 Department of Industrial Design, Eindhoven University of Technology, Eindhoven, The Netherlands 3 Department of Human Media Interaction, University of Twente, Enschede, The Netherlands H.vanOostendorp@uu.nl e.d.v.d.spek@tue.nl j.m.linssen@utwente.nl Abstract: As games are continuously assessing the player, this assessment can be used to adapt the complexity of a game to the proficiency of the player in real time. We performed an experiment to examine the role of dynamic adaptation. In one condition, participants played a version of our serious game for triage training that automatically adapted the complexity level of the presented cases to how well the participant scored previously. Participants in the control condition played a version of the game with no adaptation. The adapted version was significantly more efficient and resulted in higher learning gains per instructional case, but did not lead to a difference in engagement. Adapting games to the proficiency of the player could make serious games more efficient learning tools. Keywords: serious game, proficiency, dynamic adaptation, learning efficiency, engagement

1. Introduction Serious games can be used to engender learning in a player, and two recent meta‐analyses have shown that the usage of serious games may even lead to superior learning compared with traditional (but passive) instructional methods (Sitzmann, 2011; Wouters et al, 2013). However a serious game is found to be primarily efficacious if a person is allowed to play the game multiple times (Sitzmann, 2011), a result that Wouters and Van Oostendorp (2013) argue underlines the notion that games are complex environments in which the player first has to learn how to control the game and the way in which it conveys the instructional material, before this material itself can be learned. Games in turn are products that have to be made beforehand and have a preset pace, and often do not take into account the individual learning rate. People learn at different speeds, which may lead to a number of problems. Firstly, the rich multimodal information of a game may overload the limited working memory capacity of a player, leading to incorrect learning (Moreno & Mayer, 2007), and some learners will therefore benefit from a slower pace in the presentation of instructional material in order to correctly organize all the new information that is coming in. Conversely, efficient learning may also be hindered by cognitive underload, where the learner is stimulated too little, for instance when a quick learner plays a game that has a slow pace in order to accommodate slow learners. Cognitive underload can lead to (passive) fatigue, which has been shown to result in disengagement from the task and higher distractibility and can subsequently degrade performance (Saxby et al, 2007; Paas, Renkl & Sweller, 2004). If a game were to actively prevent the player from becoming cognitively overloaded or underloaded, it could therefore both be more efficient. Secondly and closely related to this, Csikszentmihalyi (1988) posited that one can experience the feeling of flow, which is a feeling where someone is completely engaged in an activity to the point of losing self‐ consciousness and the activity becomes rewarding in its own right, and that this leads to the individual functioning at his or her fullest capacity (Shernoff et al, 2003). This is achieved when the provided challenge is optimally suited to the skills of the user; and as videogames are often stated to be engaging, with players reporting an experience of being completely absorbed in the game, they seem to be ideally suited to produce flow (Sweetser & Wyeth, 2005; Cowley et al, 2008). Flow has been shown to be positively correlated to learning (Webster, Trevino & Ryan, 1993); therefore, keeping players in a sense of flow by adjusting the challenge to their skills could improve learning (Liu et al, 2011).

553

Herre van Oostendorp, Erik van der Spek and Jeroen Linssen

Summarizing, if quick learners were able to progress in the game at a faster pace, for instance because the game recognizes their proficiency and adapts the game accordingly, engagement in performing the task could be enhanced which in turn results in a higher efficiency of the game. Similarly, a slower pace for slower learners would also improve engagement and efficiency for them. In this paper we will examine in an experimental study whether adapting a serious game to the proficiency of players improves learning and engagement. But first we will discuss in next section different aspects of adaptivity in general, how we monitored or assessed proficiency of players and how we implemented adaptivity in a dynamic way in the serious game Code Red Triage.

2. Adaptivity 2.1 Aspects of adaptivity In line with Lopes and Bidarra (2011), we can distinguish several components of adaptation: 1) the game world and its objects can be varied; e.g. the layout of the game world can be made simpler for underachieving players (Walker, 2009); 2) the game play mechanics; e.g. adjusting shooting difficulty by providing player aim assistance, according to individual skills (Brathwaite & Schreiber, 2009); 3) adapting the attributes of the non‐ player characters in the game; e.g. when domain knowledge is automatically gathered by the game, based on Artificial Intelligence‐techniques, in order to offer more challenging behavior of the non‐player characters (Bakkes, Spronck & van den Herik, 2009); 4) game narratives; e.g. adapting the sequence of events to the pace or behavior of the player (Roberts & Isbell, 2008); and 5) game scenarios ‐ more or less similar to the previous one; adapting the flow of events and actions within a game. For instance, monitoring the players actions and based on that certain points in the plot are included in the game (or not) (Niehaus & Riedl, 2009). A next issue in creating adaptive games is to decide on the method of generating the content. Lopes and Bidarra distinguish two general methods. First, offline adaptivity (or customized content generation); adjustments are made considering player‐dependent data, but prior to initiating the gameplay. Secondly, online adaptivity, i.e. adjusting the game to its players, in real time, as they play. A further discussion on the way adaptation can be implemented in games and the associated challenges can, for instance, be found in Lopes and Bidarra (2011). Though in the (game) industry and academia now many different adaptive (serious) games are developed, and progress has been made, empirical research to effects of adaptivity in terms of learning and engagement are still scarce. In this paper we will remedy this and present results of an empirical study on the learning and affective effects of a game with dynamic adaptivity. That is, a game where the challenges of, or difficulties caused by, the game are increasing, and at a rate dependent on the proficiency of the player (online adaptation). We will mainly be concerned with varying the attributes of the non‐player characters.

2.2 Assessing the proficiency of players For the principle of fitting the instruction to the learner's proficiency level to be implemented in serious games effectively, it is important first that the proficiency should be assessed and secondly that the challenge should be adapted to the player automatically in a non‐obtrusive way. Automatically assessing and adapting the challenge or difficulty of a game to the proficiency of a player is slowly becoming commonplace in entertainment games. For instance in Guitar Hero 5, a musical instrument simulation game, the player needs to hit the correct notes of a song with good timing. The game adds more notes and places a greater emphasis on timing when the player performs well, or vice versa when the player performs badly. Racing games like Mario Kart and Need for Speed, implement a simple adaptation known as ‘rubber banding’: when the player lags behind the other racing contestants, they will slow down in order to let the player catch up with them – when the player is up front, his opponents will become faster and try to keep up with him. Here, we will elaborate on two modes of assessing that are most relevant to our research. Firstly, one interesting avenue in which a game can be adapted to the player was undertaken by Yun et al (2009), who used an infrared camera that was mounted on a TV displaying the game. This camera (overtly) recorded the faces of the participants while they were playing a game that revolved around shooting enemy robots. Looking at the heat signatures from the supra‐orbital region of the face, they were able to derive how much apparent stress the game exerted on the player during game play. At the same time, the player reported at set intervals whether they found the game too easy, just right or too difficult, and whether they were enjoying the game or

554

Herre van Oostendorp, Erik van der Spek and Jeroen Linssen

would like to quit. This research is relevant to our own for two reasons. One, they discovered that people who found the game too difficult and wanted to quit actually had lower stress levels than when the game was moderately difficult. They argued that this is due to the player becoming disengaged with the game, thereby corroborating the previously made assertion that too high a challenge leads to cognitive overload and is detrimental to the engagement or flow experience. Two, a version of the game where the game automatically assesses and adapts to the stress level of the player was shown to lead to higher engagement and better in‐ game performance (in terms of how many robots were defeated) than in conditions with preset difficulty levels, even for the easy difficulty level. Another interesting example of how to adapt the game to the player is the entertainment game The Elder Scrolls 4: Oblivion. Here, the player roleplays a character in a large and open medieval fantasy world. As the player encounters new locales, performs quests and defeats monsters, his or her character will gradually become stronger and gain better weapons and items (see further Shute et al, 2009). Because the game features an open world for the player to explore freely, this traditionally leads to problems where the player may encounter monsters that are far too strong for his or her avatar to defeat at that point in time. To counter this and provide the optimal experience for everyone, the player’s adversaries in the game also progress in power at the same rate as the skill level of the player. Contrarily to what would be expected, many gamers criticized this feature, as it made them feel that their actions were largely inconsequential (Bostan & Ogut, 2009); they were not getting stronger than their enemies and therefore they didn’t feel like they were mastering the game. Above we mentioned two different techniques of assessing the player proficiency within the game. The first was a more overt technique, where in real life settings the player would have to install an infrared camera for it to work; the second example featured so‐called ‘stealth’ assessment (Shute et al, 2009), that is, a more covert assessment that is coupled to the naturally occurring moves of the player in the game. In essence, all games are an assessment device, in that progressing past an obstacle is contingent on acquiring the needed knowledge of how to do so. As digital games are played on computers, which require that every game rule and in‐game problem encountered is computable, determining whether the player succeeded is often easily quantifiable.

2.3 Dynamic adaptivity in the serious game Code Red Triage As indicated we want to study whether the online adaptation of the challenge or difficulty of a learning experience to the proficiency of players, improves learning and enhances engagement. Following Bailey and Katchabaw (2005) we use the term dynamic adaptivity to designate online adaptation of game experiences in terms of complexity and matching that to the proficiency of players. In order to test this hypothesis we used the serious game Code Red Triage, a total conversion mod of Half‐Life 2 (Van der Spek, Wouters & Van Oostendorp, 2011). The game is designed to teach the triage procedure, a procedure for medical first responders to prioritize the victims of a mass casualty event according to how urgently the victim needs medical attention. The mobility (sieve) triage taught here is a relatively simple procedure, where it takes the first responder between one and five steps to determine the severity of the victim’s injuries. When the game starts, the player finds himself in an empty train station with signs of recent panic. Here, he learns that he is a medical first responder who has received a call that a bomb has gone off on a subway platform. The player is then told to find the subway platform and perform the triage procedure on the victims. Upon reaching the subway platform, a visible timer starts counting down from seventeen minutes. When the timer reaches zero, the game ends. This timer was added to instill a sense of immediacy and stress; in practice almost every participant is able to triage all victims comfortably within this time. At the subway platform, the player can then walk up to a victim and press a button to enter the triage menu, which consists of eight buttons for triage actions, and four buttons for the four different triage categories (see Figure 1). Pressing a triage button will give a few lines of general information on what the action entails and approximately at what stage in the procedure it should be used, and a line with specific information on how the action affected the victim the player’s looking at. After choosing a few triage actions the player should be able to have an idea how heavily injured the victim is and assign a triage category. Once this is done, the victim changes color to depict the chosen category and the player receives a score showing how well he did, as well as a few lines telling him whether or not he forgot to take procedure steps, took steps in the wrong order, took unnecessary steps and whether it was done within the allotted time (between 10 and 55 seconds), see Figure 2 for a screenshot. The in‐game score that can be obtained per victim ranges from 0 to 100 and is based on the previous four criteria.

555

Herre van Oostendorp, Erik van der Spek and Jeroen Linssen

Figure 1: Triage menu in the game Figure 2: Feedback after categorizing a victim in a triage category In the case of Code Red Triage, we already have a measure to assess how well the player is performing in the game, namely the in‐game score, which provides us with an objective measure of whether the player is able to correctly apply the procedure to a given victim case. The player’s performance can therefore be seen as an indication of their proficiency level. We can thus use the above mentioned covert method to assess the proficiency of players here. We used this in‐game score to adapt the difficulty of the game to the proficiency of the player. In Code Red Triage, there are a total of six paths with an increasing number of steps in the triage procedure that are taught with the game, but there are multiple victims for any given path. As the victims are encountered in increasing order of complexity (i.e. the number of steps needed to come to a correct categorization), these groups of victims are called ‘victim tiers’. In the set of victims 6 tiers or levels of complexity were distinguished. In other words, the attributes of the non‐player characters were varied in complexity. If a player scores above a preset threshold, he or she has proven to have a certain level of proficiency and can move on to a more complex victim tier. In the adaptive condition of Code Red Triage this was operationalized as the game deleting all remaining victim cases within the same tier, if the player scored higher than a threshold value for that victim. The threshold was determined with the data from a pilot experiment, by rounding up the average score per victim tier. A player who was unable to triage a victim case and scored below the threshold, received one or more of the remaining cases of that tier before going to the next level of complexity. In other words, more successful players could attain the most complex case in less cases, and consequently learn to perform the triage more efficiently. In the control version of the game all (19) cases were presented in a gradually increasing complexity. We hypothesize that players feel more engaged by the dynamic adaptive version, because the game always remains challenging (compared to a control version), and secondly we expect in the dynamic adaptive version of the game that players are able to learn more efficiently, because redundant learning experiences (triage cases) can be skipped.

3. Method 3.1 Participants In total 28 individuals of university‐level education, 19 male and 9 female, participated in the experiment, and were randomly assigned to the adaptive game condition (n=14), and the control condition (n=14). Average age was 22.86 with a standard deviation of 5.68.

556

Herre van Oostendorp, Erik van der Spek and Jeroen Linssen

3.2 Materials To measure the learning of players, three types of instruments were used. The in‐game score formed the first measure: an indication of the progression of the player in the game. Statistics from the game that were logged furthermore included triaged victims, number of triaged victims, tier of victim, time per victim, total time, score per victim and total score. Second and third, we measured how much a participant learned in the game with two measures: a pen‐and‐paper knowledge test and a structural knowledge assessment. The knowledge test was in the form of eight verbal and eight pictorial multiple choice questions where the player had to answer questions related to the triage procedure by choosing one of four alternatives (total score range 0‐16). Whereas the knowledge test measured how well the participant could reproduce declarative knowledge, the structural knowledge assessment determined how the information was organized on a deeper, more structural level. Here, a computer program called PCKNOT was used, that let participants rate the degree of relatedness of pairs of concepts from the triage procedure. These ratings could subsequently be used to elicit a participant’s knowledge structure with the Pathfinder metric (Schvaneveldt, Durso & Dearholt, 1985) and compared to the knowledge structure of experts; resulting in a similarity measure that indicated how well the participant had organized the information of the triage procedure structurally (Gomez, Hadfield & Housner, 1996). The score range varies from ‐1 through 0 to +1. Pathfinder has been successfully by Day, Arthur Jr. and Gettman (1994) to measure learning from a complex videogame, and found that it was also predictive of skill retention and skill transfer. For further information see Wouters, Van der Spek and Van Oostendorp (2011). In our case we focused on 8 important concepts from the triage procedure and consequently 28 pairs were presented for the related judgments. The created networks were compared with the referent structure that was derived by averaging the elicited knowledge structures of the current researchers. The engagement of players was measured by using the subscale of the ITC Sense of Presence Inventory (ITC‐ SOPI), which indicates the participant’s feelings of engagement with a twelve item five‐point Likert scale (Lessiter et al, 2001). If the challenge of the game is better adjusted to the abilities of the player, one would expect the player to be drawn into the game more, which we hoped to see expressed in the scores on this subscale. The reliability of the ITC‐SOPI Engagement questionnaire appeared to be relatively low, Cronbach's coefficient α = 0.59.

3.3 Apparatus and procedure The game was played on a 17” laptop at a resolution of 1920 x 1200 with circum‐aural headphones in a room with the lights turned off. The graphics settings were set at their maximum and the game ran at a constant 60 frames per second. The participants were asked to perform the structural knowledge assessment with the PCKNOT software. Then, the knowledge test was administered. Before playing the game, the participants were given instructions about Code Red Triage and were informed about its goal. Nothing was revealed to them about the condition they took part in. Playing the game from start to finish took each participant at most 25 minutes: a few minutes for the entry level, a few more for the hallway part and a maximum of 17 minutes was allowed for the metro platform part, in which the triages took place. The scores participants reached in the game gave information about their performance (see also section 2.3). Directly after the participants finished playing the game, they were asked to fill out the engagement questionnaire. They were then asked to do the structural knowledge assessment and knowledge test as before, but with the questions in a different order. Finally, the participants were thanked for their cooperation and they received a coupon for their work. An overview of the procedure can be seen in Figure 3.

Figure 3: Procedure of the experiment

4. Results Engagement The mean scores and standard deviations of the engagement questionnaire are mentioned in Table 1. An ANOVA showed no significant effect of condition on the ITC‐SOPI engagement questionnaire, F(1,26) < 1.

557

Herre van Oostendorp, Erik van der Spek and Jeroen Linssen

Learning Efficiency There are several ways to determine whether learning was more efficient in the adaptive condition. A reliable measure for efficiency is to divide the posttest scores of the participants by the number of victim cases triaged, giving us an indication of how much the participant has learned per unit of instruction, and whether this would be higher in a game that adapts the information presentation to the player’s proficiency. Another way would be to divide learning performance by total time spent playing the game. However some players navigate more efficiently than others towards the platforms etc, which blurs what we want to measure. We therefore decided to use learning performance divided by the number of cases triaged, as a purer measure of learning efficiency. An ANCOVA with the pretest as covariate, condition as fixed factor and posttest score divided by the total number of victims triaged as dependent variable showed that condition had a significant effect on both the knowledge test (F(1,25) = 21.98, p < .001, d = 1.81) and the structural knowledge assessment (F(1,25) = 5.05, p < .05, d = .89). The means on these relative measures and standard deviations of these tests are listed in Table 1. Table 1: Mean engagement and efficiency scores on knowledge test and structural knowledge assessment (sd).

Control Condition

Adaptive Condition

Engagement (1‐5)

3.63 (.33)

3.66 (.45)

Knowledge test

.57 (.19)

1.02 (.30)

Structural knowledge assessment

.015 (.004)

.028 (.019)

In‐game score The total in‐game score was significantly higher for the control condition (M = 777.7, SD = 321.2) than for the adaptive condition (M = 316.4, SD = 107.8), F(1,26) = 25.95, p < .001, however this more or less follows from the result that participants triaged significantly less victims in the adaptive condition.

5. Conclusion and discussion We hypothesized that a serious game that dynamically adapts its challenge, or complexity presentation, to quick learners could make a serious game more engaging and more efficient. The first part of the hypothesis was not confirmed, while the second part was confirmed; participants in the adaptive game version learned significantly more per victim case than in the control condition, and were therefore more efficient. We found no difference in the engagement ratings. If the improved learning per unit of instruction was due to less disengagement from the task, one would expect this to appear from the results of the engagement questionnaire. We propose four explanations why we did not find a difference in engagement. Firstly, when participants had to appraise their engagement just after playing the game, they lacked knowledge of the other condition and thereby a reference point. The intervention itself may be too small next to all the other determinants of engagement, such as the game’s setting, world, expectations, control interface, et cetera, to show up as a difference on the rating scale, but the adaptive version may still be preferred when the conditions were placed side by side. A second explanation could be related to the fact that we only asked participants to appraise their engagement after the game. It is unclear whether a continuous measurement of a participant’s engagement, for instance with an infrared camera as in the research by Yun et al (2009), as we mentioned in the introduction, would have resulted in higher ratings throughout the game in the adaptive version. Thirdly, people may play games for different reasons; a higher challenge could lead to higher engagement in some players, whereas it has the opposite effect on others. Lastly, and perhaps as a result of the previous explanation, we found that the homogeneity of the engagement questionnaire (Cronbach’s alpha) was low. Perhaps this measurement problem contributed to the fact that we did not find an effect of engagement. We saw that participants learned more per victim case in the adaptive condition compared to the control condition. It could be that the moment a participant grasps the procedure to resolve a victim case pertaining to a certain tier, the information presented in the following victims in that tier is redundant, at least to a point that it does not improve learning of the procedure anymore, making the adaptive version more efficient.

558

Herre van Oostendorp, Erik van der Spek and Jeroen Linssen

In order to determine whether the adaptive condition not only made learning the instructional material more efficient, but also leads to deeper learning (Graesser et al, 2009), other experiments should be set up such as e.g. a study where learning is also measured after a longer delay or with transfer tasks. One last observation concerns the relation between engagement and learning; the results found indicate that an increase in engagement does not seem necessary to enhance learning efficiency. Also the correlation between engagement and learning efficiency appeared to be low and not significant (p > .05) for both groups of participants. However for this finding too, the same remarks as before should be made concerning the measured engagement of players. All in all, a rather simple alteration of a serious game where it dynamically adapts the presentation of complexity to the player’s performance and thereby its challenge has been shown to markedly improve the efficiency thereof. This is a promising result for serious games developers that worry about the comparative efficiency of their game, as well as for researchers interested in improving games with the aid of more sophisticated adaptation engines. It can also be a useful result for entertainment game developers, as many games need to incorporate tutorial levels that are necessary for players to understand the game, but are not a lot of fun to play, especially upon repeated playthroughs. A dynamic adaptive version that adapts to the player's proficiency could greatly speed up these mandatory instructional sequences and (possibly) make them more challenging.

5.1 Future research Above we already mentioned two limitations to our study, viz. that it is impossible to conclusively state whether dynamically adapting to the player’s performance only resulted in more efficient instruction, or also in deeper learning, and that it is unclear whether participants differed in engagement during gameplay. In addition, another limitation of our experimental setup that warrants future research is that we did not measure retention over longer time periods. Participants in the adaptive game version received less practice and consequently less opportunity to internalize the information. Therefore there is a real possibility ‐ or even danger ‐ that the participants in the dynamic adaptive condition remember less of the instruction after several weeks. Regarding dynamic adaptation itself, in this study we did make some specific choices during the design and implementation process. We focused on the nature of the non‐player characters and let them vary in number of steps needed to perform a correct triage. Several alternatives are open for continued research to the role of dynamic adaptation. For instance, the set of buttons for executing the triage actions could be adapted, that is, starting simple and increasing over time, depending on performance. Or the feedback given to players could be adapted, e.g. stating more or less explicitly what went right and what went wrong during performing the triage. Finally, another option for making training procedures adapt themselves to participants is the notion of adaptability. In this form of offline adaptation participants indicate themselves what direction they want to practice and what part of the procedure they want to repeat. These are questions that still need to be examined in the future.

Acknowledgements This research has been supported by the GATE project (http://gate.gameresearch.nl/), funded by the Netherlands Organization for Scientific Research (NWO).

References Bailey, C. and Katchabaw, M. (2005) An experimental testbed to enable auto‐dynamical difficulty in modern video games. In Proceedings of the 2005 GameOn North America Conference (pp.18‐22). Bakkes, S. Spronck, P. and van den Herik, J. (2009) Rapid and reliable adaptation of videogame AI. IEEE Transactions Computational Intelligence and AI in Games, 1(2), 93‐104. Bostan, B. and Ogut, S. (2009) Game challenges and difficulty levels: lessons learned From RPGs. In International Simulation and Gaming Association Conference. Retrieved from http://www.silentblade.com/presentations/Bostan_Ogut_Full_Paper.pdf Brathwaite, B. and Schreiber, I. (2009) Challenges for game designers. Boston, MA: Charles River media. Cowley, B., Charles, D., Black, M. and Hickey, R. (2008) Toward an understanding of flow in video games. Computers in Entertainment 6(2), article 20, 1‐27

559

Herre van Oostendorp, Erik van der Spek and Jeroen Linssen

Csikszentmihalyi, M. (1988) The flow experience and its significance for human psychology. In M. Csikszentmihalyi & I.S. Csikszentmihalyi (Eds.), Optimal experience: Psychological studies of flow in consciousness (pp. 15‐35). Cambridge, MA: Cambridge University Press. Day, E. A., Arthur Jr., W. and Gettman, D. (2001) Structural knowledge assessment: Comparison of referent structures. Journal of Educational Psychology, 86(5), 1022‐1033. Gomez, R. L., Hadfield, O. D. and Housner, L. D. (1996) Conceptual maps and simulated teaching episodes as indicators of competence in teaching elementary mathematics. Journal of Educational Psychology, 88(3), 572‐585. Graesser, A.C., Chipman, P., Leeming, F. and Biedenbach, S. (2009) Deep learning and emotion in serious games. In U. Ritterfeld, M. Cody & P. Vorderer (Eds.), Serious Games: Mechanisms and Effects (pp. 81‐100). New York: Routledge. Guitar Hero 5 [Computer software]. Santa Monica, CA: Activision Blizzard. Half‐Life 2 [Computer software]. Bellevue, WA: Valve Corporation. Lessiter, J., Freeman, J., Keogh, E. and Davidoff, J. (2001) A cross‐media presence questionnaire: The ITC‐sense of presence inventory. Presence: Teleoperators and Virtual Environments, 10(3), 282‐297. Liu, M., Horton, L., Olmanson, J. and Toprac, P. (2011) A study of learning and motivation in a new media enriched environment for middle school science. Education Technology Research Development, 59, 249‐265. Lopes, R. and Bidarra, R. (2011) Adaptivity challenges in games and simulations: a survey. IEEE Transactions on Computational Intelligence and AI in Games 3(2), 85‐99. Mario Kart [Computer software]. Kyoto, Japan: Nintendo. Moreno, R. and Mayer, R. (2007) Interactive multimodal learning environments. Educational Psychology Review, 19(3), 309‐326. Need for Speed [Computer software]. Redwood City, CA: Electronic Arts. Niehaus, J. and Riedl, M.O. (2009) Scenario adaptation: an approach to customizing computer‐based training games and simulation. In Proceedings AIED Workshop Intell Edu. Game. Brighton, UK (pp. 89‐98). Paas, F., Renkl, A. and Sweller, J. (2004) Cognitive load theory: Instructional implications of the interaction between information structures and cognitive architecture. Instructional Science, 32(1), 1‐8. PCKNOT [Computer Software]. Las Cruces, NM: Interlink Inc. Roberts, D.L. and Isbell, C.L. (2008) A survey and qualitative analysis of recent advances in drama management. Int. Trans. Sci. Appli., 3(2), 61‐75. Saxby, D. J., Matthews, G., Hitchcock, E. and Warm, J. S. (2007) Development of active and passive fatigue manipulations using a driving simulator. In Proceedings of the Human Factors and Ergonomics Society 51st Annual Meeting (pp. 1237‐1241). Santa Monica, CA: Human Factors and Ergonomics Society. Schvaneveldt, R. W., Durso, F. T. and Dearholt, D. W. (1985) Pathfinder: Scaling with network structures (Memorandum in Computer and Cognitive Science, MCCS‐85‐9, Computing Research Laboratory). Las Cruses: New Mexico State University. Shernoff, D. J., Csikszentmihalyi, M., Shneider, B. and Shernoff, E. S. (2003) Student engagement in high school classrooms from the perspective of flow theory. School Psychology Quarterly, 18(2), 158‐176. Shute, V. J., Ventura, M., Bauer, M. I. and Zapata‐Rivera, D. (2009) Melding the power of serious games and embedded assessment to monitor and foster learning: Flow and grow. In U. Ritterfeld, M. Cody, & P. Vorderer (Eds.), Serious games: Mechanisms and effects (pp. 295‐321). New York, NY: Routledge. Sitzmann, T. (2011) A meta‐analytic examination of the instructional effectiveness of computer‐based simulation games. Personnel Psychology, 489‐528. Sweetser P. and Wyeth, P. (2005) GameFlow: a model for evaluating player enjoyment in games. Computers in Entertainment, 3(3), 1‐24. The Elder Scrolls 4: Oblivion [Computer software]. Rockville, MD: Bethesda. Van der Spek, E. D., Wouters, P. and Van Oostendorp, H. (2011) Code Red: Triage Or COgnition‐based DEsign Rules Enhancing Decisionmaking TRaining In A Game Environment. British Journal of Educational Technology, 42(3), 441‐ 455. Walker, J. (2009) Left 4 Dead 2: Exclusive RPS Hand on Preview. Rock Paper Shotgun, PCGaming since 1873. Online available: http://www.rockpapershotgun.com/2009/06/01left‐4‐dead‐2‐exclusive‐rps‐preview Webster, J., Trevino, L. K. and Ryan, L. (1993) The dimensionality and correlates of flow in human‐computer interactions. Computers in Human Behavior, 9(4), 411‐426. Wouters, P.J.M., Van der Spek, E.D. and Van Oostendorp, H. (2011) Measuring learning in serious games: a case study with structural assessment. Educational technology research and development, 59(6), 741‐763. Wouters, P.J.M., & Van Oostendorp, H. (2013) A meta‐analytic review of the role of instructional support in game‐based learning. Computers & Education, 60, 412‐425 Wouters, P.J.M., Van Nimwegen, C., Van Oostendorp, H. and Van der Spek, E.D. (2013) A Meta‐analysis of the Cognitive and Motivational Effects of Serious Games. Journal of Educational Psychology, 105, 249‐265. Yun, C., Shastri, D., Pavlidis, I. and Deng, Z. (2009) O' game, can you feel my frustration?: Improving user's gaming experience via stresscam. In Proceedings of the 27th international conference on Human factors in computing systems (CHI '09) (pp. 2195‐2204). New York: ACM.

560

Designing Casual Serious Games in Science Ayelet Weizman Snunit center for the advancement of web‐based learning, The Hebrew University, Jerusalem, Israel ayeletw@snunit.org.il Abstract: One of the problems in science education is students’ low motivation and difficulty to see the connection between science learning and their daily life. In addition, many science concepts are not intuitive, and involve complicated relations between parameters. We previously presented (Weizman & Broza, 2012) evaluation of a model integrating digital educational games with interactive learning units, where we found that students reported experiencing both learning and enjoyment. The current study focused on games in science, and examined the relation between the game design parameters and the learning experience based on the Flow model. We define these games as casual‐serious, since they require short playing time and relatively low budget to develop, while based on educational purposes. The goal of educational games is to give players the right balance between skill and challenge that will not be too difficult or too easy, enabling a flow experience. The research goal was to investigate the relation between the game design parameters and the learning experience. We assumed that by changing the game parameters the relation between challenge and skill will be influenced, as expressed by the flow experience, which is related to the learning experience. We studied the design process of two games in science, aiming to teach complicated concepts, like the idea that good health is based on balanced nutrition combined with physical activity. The influence of game parameters on students’ learning experience was investigated through a pilot process in which each game was piloted in a primary school class, calibrated and piloted again. Flow was evaluated using online questionnaires. We found a significant change in the challenge to boredom rate between the three phases of the pilot. Students’ responds indicate assimilation of social practices and understanding of the relation between parameters. Findings show that careful design of game parameters influences the flow experience as well as the learning experience and understanding of content goals and social values. We conclude that casual‐serious games designed according to the described design principles may be useful in science teaching, since they provide an enjoyable way to deal with complicated concepts involving relations between parameters. By balancing game parameters it is possible to create a flow experience and lead to meaningful learning of science content and skills. Keywords: serious‐games, educational‐games, flow, science‐learning, game‐based learning, casual‐games

1. Introduction 1.1 Games and motivation According to OECD data, about 60% of the students in the USA, Canada and many countries in Europe think that school is boring. Moreover, studies show that one of the reasons for low achievements in science is negative attitudes towards science in school (Osborne, Simon, & Collins, 2003). One of the reasons for low motivation in school is that students seldom see the relation of school science to everyday life, especially when the topics are abstract or complicated (Foster, 2008). Computer games, on the other hand, are one of the favorite activities that teenagers choose to spend their time on. Moreover, while playing digital games youths often seem to be fully engaged in the experience and unaware of their environment as in the flow theory (Csikszentmihalys, 1975).

1.2 Flow and game‐based learning in science The state of flow was defined by Csikszentmihalys (1975) as a balance between challenge and skills, or anxiety and boredom. Originally it referred to activities like rock climbing, music composing or dancing, but later the model was extended to other contexts including digital environments. One of the implications of flow experience in digital environments is increased learning (Webster, Trevino, & Ryan 1993). A recent report (Clark et al., 2013) evaluated three aspects of learning: cognitive, intra‐personal and inter‐personal in previous studies of game‐based learning, and found that using games in science learning may improve cognitive competencies compared to learning in traditional methods. Another finding shows that the effectiveness of game influence on learning depends on game design. The challenge for game developers is to design the game parameters in a way that will enhance flow, while increasing the level of challenge according to the increase in skills, in a way that will keep the player in the flow zone all along the playing time.

561

Ayelet Weizman Kiili (2005) suggested a framework for flow in computer‐mediated environments referring to three stages of flow: Flow antecedents, Flow state and Flow consequences (Table 1). Educational‐game developers should take into account all the components comprising the flow framework. Table 1: Kiili’s (2005) three stage game flow model Game flow antecedents Game flow state Flow consequences Challenge to Skill balance Concentration Content learning gains Playability Sense of control Exploratory behaviors Gamefulness Time distortion Clear Goals Loss of self‐consciousness Immediate Feedback Autotelic experience Frame story

1.3 Design principles for educational games The main design principle for educational games was formulated by Gee (2003, 2005): in order to achieve meaningful learning game goals must be aligned with learning goals. This principle is the key for designing a game that is educational and enjoying at the same time. When goals are truly aligned the motivation to learn will be internal and unite with the motivation to play. Another principle for educational games is integrating the learning content in the game (and not stopping the game flow or taking a break in order to teach something). In a simulation‐game, which is a common genre in science games, the factors in the game must correlate with a scientific model, and represent the relations between them. In addition, success in the game cannot be coincidental, but should depend on knowledge of content or skills that are relevant to the learning goals. The potential in the influence of digital educational games on motivation, engagement and learning of science content and skills was investigated in several studies (e.g Kiili, 2005, Zheng, 2012, Pavlas, 2010), that found a th positive relation in specific contexts. Zheng (2012) found that 5 class students who played a digital game in science reported on flow experience and showed significant achievements in content learning. Pavlas (2010) found significant relation between flow experience and declarative knowledge in a science game.

1.4 Casual‐serious games Serious games are usually defined as games with a goal besides entertainment (for example: PeaceMaker), and recently many serious games have educational goals. Usually these games are complicated, expensive, and require extended playing time. On the other hand, casual games are usually not identified with educational games. They are characterized as simple, easy to learn, and easy to play, and attract a different audience than serious games (Orji, Vassileva & Mandryk, 2012). But actually, if we aim to integrate digital games in school learning, isn’t it more realistic to develop casual‐serious games that will be easy for teachers to adopt, will be possible to play within a lesson time‐frame, but at the same time will be based on educational learning goals?

2. Methodology 2.1 Game design In the following sections we describe the design process of two casual‐serious games in science, dealing with the subject of “healthy and active lifestyle”, which is included in the science curriculum. The games were developed by Snunit center for the advancement of web‐based learning, in cooperation with the ministry of health, the ministry of education and the ministry of sport and culture in Israel. The games are part of a website dealing with health in the kids‐governmental portal. The games in this portal are digital educational games that can be defined as casual‐serious games since they are low‐budget and require a relatively short playing time. In previous studies (Weizman & Broza 2012) we described our model for integrating learning with enjoyment in digital games in “Al‐Hagova” website. The games were shown to create positive learning experience for children playing after school hours. The model incorporates the games with separate learning units in the

562

Ayelet Weizman website, in a way that the free‐choice learning from the game is supported by explicit learning in the interactive units. In formal contexts the mediation may be done by the teacher using the learning units or any other teaching strategies. In the current study we explore the connection between game parameters and the learning experience in two casual‐serious games in science. The research question is: How can we influence flow experience and learning experience through the design of game parameters? 2.1.1 The game “Healthy and Happy” The game deals with the balance between nutrition and sportive activities in everyday life. The learning goals include:

Developing awareness to the significance of healthy lifestyle

Developing awareness to the connection between nutrition and daily activities

Enabling decision making and problem solving in this context.

These learning goals require high order thinking skills in addition to understanding complicated concepts like the food pyramid. The game is based on the genre of caring games. The player chooses a character (boy or girl with several choices of appearance characteristics), and has to care for its nutrition according to the food pyramid, to give it enough food and drink and at least 60 minutes a day of physical activities.

Figure 1: First screen of the game “Healthy and Happy” Figure 1 shows the game’s first screen including the game main parameters: the food pyramid (according to the Israeli ministry of health), food, drink and activity parameters. The game’s timeline is one day from morning to evening, divided to five sections representing five meals. Food items can be chosen from four categories – as shown in Figure 2. Each item has a nutritious combination determined by true values of the food groups it contains. The levels of the food pyramid parameter are filled according to the player’s choices of food items. The design principle of aligning game goals with learning goals is applied here in the choice of parameters, the relations between them, and the conditions for success and failure in the game. The feedback (Figure 3) refers to specific choices of the player in the game, and describes the achieved level of each parameter in the game, so that the player can improve her gameplay next time.

563

Ayelet Weizman

Figure 2: The main screen of the game “Healthy and Happy”

Figure 3: Feedback for the first level in the game 2.1.2 The game “Couch Potatoes Defense (60 minutes)” This game deals with the significance of physical activity during daily life, as part of a healthy lifestyle. The message of the game is that you have to be active at least 60 minutes a day in order to be healthy and happy. The game aims to clearly convey that these 60 minutes can be easy and fun, and there is a large variety of activities you can do without changing your everyday routine. The game is based on the genre of defense games, and specifically inspired by the long established Tower Defense game. The "enemies" are lazy characters advancing on a conveyor belt, whose aim is to go to sleep as soon as possible, while avoiding physical activities. The player's goal is preventing them from being lazy by placing various activities in specific positions along the conveyor belt.

564

Ayelet Weizman

Figure 4: The main screen of the game “60 minutes” The principle of aligning game’s goals with learning goals is applied in this game as well, and expressed in the choice of parameters (e.g. accumulation of minutes to reach 60 minutes a day for each character, and setting defense categories that represent physical activity categories), in the relations between parameters and in the conditions of success and failure in the game (e.g. in order to win you have to combine various activity categories).

2.2 Research framework Each game was piloted and revised in a process that lasted about a month: The game was piloted in a primary school class (4th to 6th grade, about 20 students in each class), game parameters were calibrated according to the feedback, and then it was piloted again in another class. The pilots were conducted in a computer room, with children siting in couples or individuals near computers. After a short explanation they were asked to play the game about 20 minutes, and then fill an online questionnaire. Finally there was a general discussion that we recorded. Game parameters that we could calibrate include the conditions for failure in each level, the number of challenges appearing simultaneously in the game and the time intervals between them. For example, in the game “Healthy and Happy” we could change the decrease pace of the drinking parameter, thus increasing or decreasing the time pressure in the game. In the second game parameters that were varied include time spans between the appearance of characters, their speed and number. The first game “Healthy and Happy” was published (free) online on June 2012, and we continued to collect data through the online questionnaire for several months. 60 kids aged 8‐14 filled the health game questionnaires, while data from Google Analytics for the same time period shows 12,646 unique visitors who played the game 6.5 minutes on average. Therefore, for this game there are three phases: A‐ first pilot, B‐ second pilot, C‐ after publication. We used Zheng (2012) game flow survey to evaluate the flow experience. The learning experience was evaluated using the Weizman & Broza (2012) questionnaire, combined with students’ responses in the oral discussion regarding the level of learning and description of the game’s goal and what they understood from playing it.

565

Ayelet Weizman

3. Findings Figure 5 presents players’ learning experience in the first game in each of the three phases (A, B, C). The learning experience was estimated through a measure of attitudes towards four categories: Level of understanding, Interest in the game, Enjoyment and Challenge. In the challenge category there is a significant change between the three phases‐ in each phase the game is appreciated as more challenging.

Figure 5: Players’ attitudes towards the game “Healthy and Happy” Presenting the data in a “flow tunnel” graph (Figure 6) emphasizes the development in the challenge to skill rate. In phases A and B the level of challenge is similar, but the level of interest is different. It seems that the required skill level in phase B was too low in relation to the students’ skills. In phase C the challenge level increased, but not to a level of anxiety. The level of enjoyment was high in all three phases. For the second game, “60 minutes” we had only two phases of pilot so far, but there is already evidence for the influence of game parameters and their calibration on students’ flow and learning experience: In figure 7 we compare the number of students who reached each of the five levels of the game. In an ideal game we would expect a normal distribution to account for high level of Gamefulness. The results show an improvement in this aspect in the second phase. Other components of flow are compared in Table 2. All of them show improvement from phase A to phase B, yet, these are still preliminary findings and we intend to continue the analyses in the following months. Further evidence for learning can be found in students’ answers to open questions and during the open discussion. Already in the first pilots it was clear that the main message of the game was understood. In the game “Healthy and Happy” students connected between varied nutrition and physical activity to happy and healthy lifestyle. For example: “I learned in the game about nutrition, that it is important to eat varied food from all the levels of the food pyramid. I also learned it is important to be active in order to feel better.” Examples for learning from the second game: “After playing this game I know more possibilities of physical activity to do during the day”, “I learned about the importance of doing at least one hour of physical activities every day”.

4. Summary and conclusions Previous studies found a positive influence of digital games on students’ learning of science concepts in certain conditions related to game design and flow experience. We assumed that by careful design of the game parameters it will be possible to use the game environment for significance learning combined with a positive experience.

566

Ayelet Weizman

Figure 6: Challenge/skill rate in the game “Happy and Healthy” for each of the three phases, compared to the “flow tunnel” diagram (Csikszentmihalys, 1975) Table 2: Comparison of flow components in 2 phases of the pilot for the game “60 minutes” Flow component Challenge to skill balance Clear goals playability Attention focus Sense of control Loss of time and self‐ consciousness Learning new content Deepening knowledge Change of attitudes

Phase A 68% 91% 66% 77% 69% 73%

Phase B 94% 100% 88% 94% 87% 81%

29% 61% 43%

44% 69% 57%

Figure 7: The number of students who reached each level in the game “60 minutes”

567

Ayelet Weizman In this study we examined the relation between game parameters to the learning experience from the game in two games in science dealing with health that can be defined as casual serious games. We followed the design principles of aligning game goals with learning goals, and calibrated the game parameters to achieve a flow experience. In a series of pilots for the two games we found an improvement in the flow experience of primary school students that was associated with evidence for content learning and change of attitudes. This is a preliminary study that requires further research mainly in clarification of learning expectations and outcomes. These findings have implications for integrating digital games in science teaching, which we believe have a great potential for improving science learning experience in schools. Educational games enable positive experience of dealing with complicated concepts, understanding relations between parameters, training problem solving in relevant contexts, and practicing scientific skills and high order thinking skills. Using effective digital games in science teaching may develop students’ internal motivation for learning, and help in clarifying the relevance of science learning to their daily life.

References Clark, D., Tanner‐Smith, E., Killingsworth, S & Bellamy, S. (2013). Digital Games for Learning: A Systematic Review and Meta‐Analysis (Executive Summary). Menlo Park, CA: SRI International. Csikszentmihaly, M. (1975). Beyond boredom and anxiety. San Francisco: Jossey‐Bass. Csikszentmihaly, M. (1991). Flow: The psychology of optimal experience. New York: Harper Perennial. Foster A. (2008). Games and motivation to learn science: Personal identity, applicability, relevance and meaningfulness. Journal of Interactive Learning Research. 19(4): 597‐614. Gee, J. P. (2003). What video games have to teach us about learning and literacy? Palgrave Macmillian Publishing, New York. Gee, J. P. (2005). Learning by Design: good video games as learning machines. E‐Learning, Vol 2 (Number 1), p. 5‐16. Gee, J. P. (2007). Good video games + good learning: Collected essays on video games learning and literacy. PETERLANG Publishing, New York. Kiili, K. (2005a). On Educational Game Design: building blocks of flow experience. PhD thesis, Tampere University of technology, Pori, Finland. Kiili, K. (2005b). Digital game‐based learning: Towards an experiential gaming model. The Internet and Higher Education, 8 (1), 13‐24. Orji, R., Vassileva, J. & Mandryk, R.L. (2012). LunchTime: a slow‐casual game for long‐term dietary behavior change. Personal and Ubiquitous Computing DOI 10.1‐11. Osborne, J, Simon, S. & Collins, S. (2003). Attitudes towards science: a review of the literature and its implications, International Journal of Science Education, 25:9, 1049 – 1079. Pavlas, D. (2010). A Model of Flow and Play in Game‐based Learning: The Impact of Game Characteristics, Player Traits, and Player States. PhD thesis, University of Central Florida, Orlando, Florida. Prensky (2001). Digital game‐based learning. USA7 McGraw‐Hill. Shelton, B. & Scoresby, J. (2011). Aligning Game Activity with Educational Goals: Following a Constrained Design Approach to Instructional Computer Games. Educational Technology Research and Development. Vilas (Ed.), Education in a Technological World: Communicating Current and Emerging Research and Technological Efforts, Formatex Research Center. Webster, J., Trevino, L. K., & Ryan, L. (1993). The dimensionality and correlates of flow in human‐computer interaction. Computers in Human Behavior, 9, 411–426. Weizman, A. & Broza, O. (2012) Learning and enjoyment during after‐school hours; Evaluation of an integrative model. ECGBL 2012, Cork, Ireland. Zheng, M. (2012) Fifth Graders’ Flow Experience in a Digital Game‐Based Science Learning Environment, PhD thesis, North Carolina State University. Zheng, M. (2011), Examining upper elementary students’ gameplay experience: A flow theory perspective, in A. Mendez‐

568

Designing a Collaborative Serious Game for Team Building Using Minecraft Viktor Wendel, Michael Gutjahr, Philipp Battenberg, Roman Ness, Sebastian Fahnenschreiber, Stefan Göbel and Ralf Steinmetz Technische Universität Darmstadt, Darmstadt, Germany viktor.wendel@kom.tu‐darmstadt.de gutjahr@psychologie.tu‐darmstadt.de p.battenberg@googlemail.com us93buza@rbg.informatik.tu‐darmstadt.de uh93ozoc@rbg.informatik.tu‐darmstadt.de stefan.goebel@kom.tu‐darmstadt.de ralf.steinmetz@kom.tu‐darmstadt.de Abstract: For collaborative learning scenarios to be successful many factors are necessary, like group formation, the setup of the group task, and the team members' ability and willingness to work or collaborate in a team. With easily moddable popular sandbox games like Minecraft being available today, new opportunities for Serious Games arise, especially in the field of multiplayer games. In this paper, we propose an approach for a game‐based solution of collaborative learning. This approach focuses on soft skills, especially communication as well as on improving both the motivation and the ability to collaborate and work in a team. We created a Minecraft mod for a collaborative gaming experience focusing on solidarity and teamwork. We designed a special obstacle course for a set of four players using especially designed game mechanics to improve collaboration skills. Communication skills are required as well as the ability to work in a team in order to win the game. Our hypothesis is that the mod can be used as a team forming and motivational tool in the context of collaborative learning by increasing the willingness to collaborate with other people by playing the game. We performed an evaluation in which seven groups of four random players (total 28 participants) played our game. Before and after the gaming session, the players played a version of the prisoner’s dilemma game in order to test their willingness to cooperate before and after playing the game. In a control group, participants worked together at solving a jigsaw puzzle instead. Results show that our Minecraft mod provides a better game experience and group experience than a non‐digital game with cooperative aspects. It also shows the impact of the participants’ working and social background in terms of an initial willingness to cooperate. Keywords: Serious Games, collaborative learning, Minecraft

1. Motivation In recent years a new form of creating Serious Games has emerged. Modding APIs (Minecraft) or game/level editors of successful games (e.g. Neverwinter Nights, Starcraft 2) enable easy and rapid development of Serious Games. One prominent example is MinecraftEdu 1 , which was created as a modification to Minecraft for use in the classroom. The concept of collaborative learning is being used in schools and institutes of higher learning as well as in various training scenarios today, ranging from mere group works over concepts like mutual teaching (learning by teaching) to collaborative working on complex projects. Whereas soft skills like the ability to work in teams and to communicate with group members are vital, they can be trained specifically by using collaborative learning principles. So, it seems promising to combine the concept of Serious Games with the concept of collaborative learning. Games, especially multiplayer games, inherently offer many of the features which are necessary for collaborative learning to be successful, like common goals, or the necessity to communicate with fellow players. In previous work, we described first approaches for collaborative multiplayer Serious Games focusing on either collaboration itself (Wendel et al. 2013) or for using such a game for learning specific contents (Wendel et al. 2010). In this paper, we propose a new approach for designing collaborative multiplayer Serious Games with the purpose of training soft skills, focusing on the ability and the willingness to collaborate and to work in a team. 1

minecraftedu.com/

569

Viktor Wendel et al. This concept is based both on previous work, and on concepts for collaborative learning (Johnson and Johnson 1994) and for collaborative gaming (Zagal et al. 2006). Our hypothesis is that our game design can be used to create games which encourage collaborative and team focused behavior and subsequently for a training of those soft skills. We decided to implement these concepts by creating a Minecraft mod. The basic functionality is identical with Minecraft, thus enabling an easy access. Our hypotheses are as follows: A digital serious multiplayer game for cooperation...

provides better user experience for the player ...

leads to more trust in the other group members ...

leads to more cooperative behavior ...

than a common game that includes aspects of multiplayer cooperation (e.g. a puzzle). We performed an evaluation with 28 participants. One group played our game for 25 minutes. A second group of participants worked together at solving a puzzle game for 25 minutes instead of playing our game. Results showed, that the participants were much more willing to collaborate using our game.

2. Related work 2.1 Digital educational games In the field of Digital Educational Games, or game‐based learning, the research is mainly focusing two aspects: Motivation for the use of Serious Games for learning, and Serious Game design. The use and potential of Serious Games has been argued by (Gee 2003), (Prensky 2003), (Squire 2003) and (Van Eck 2006) in terms of learning, or (Delwiche 2006), (Steinkühler 2004), or (Mansour and El‐Said 2008) in terms of Serious Games design. (Harteveld 2011) provides useful guidelines looking at the design process from the three dimensions reality, meaning, and play. A Serious Game design document is provided in (Bergeron 2006).

2.2 Collaborative learning This work is mainly based on concepts derived from (Johnson and Johnson 1994) which identified five essential elements which foster cooperative work in face‐to‐face groups. (Zea et al. 2009) provide first concepts for introducing collaborative learning techniques into educational video games. Apart from that approach, Computer‐supported Collaborative Learning (CSCL) research mainly focuses on e‐learning tools, like shown in (Onrubia and Engel 2009), or (Larusson and Altermann 2009). Concepts from those traditional CSCL fields of application might be transferable to Serious Games, too.

2.3 Motivation in games (Prensky 2002) argues that games can provide the necessary motivation for people who otherwise are not properly motivated to learn. However, in order for (Serious) games to be motivational, they need to fulfill several requirements. The game needs to provide the right amount of challenge (Gee 2003), (Lepper and Henderlong 2000). The concept of 'Flow' as first proposed by (Csikszentmihalyi 2000) and later refined especially for games by (Sweetser and Wyeth 2005) becomes important in this context.

2.4 User experience and collaboration User experience is a versatile construct that describes the whole experience a user has by playing a game (Nacke 2009). This includes aspects of cognition, emotion, physiology, etc. This is to say many aspects like positive and negative emotion, cognitive load and arousal, usability, immersion and flow (and many more) are part of this experience. To measure trust and cooperative behavior we used the prisoner's dilemma game approach of (Sheese and Graziano 2005). This approach allows differing between trust and cooperative behavior.

570

Viktor Wendel et al.

2.5 Creating games with Minecraft Today, Minecraft is already being used in various projects in schools (Duncan 2011), (Bayliss 2012), (Levin 2011), (Shifter 2012). The most prominent example is MinecraftEdu, a modification of Minecraft especially for a use in the classroom. As an official modding API is not yet released, the most promising alternative is the use frameworks Bukkit 2 and Spout 3 . As tutorials and documentation are very rare, the most important resources are the respective APIs. Using the Spoutcraft client instead of the original Minecraft client, client‐sided modifications are also possible without touching the client itself. The complete modification is done inside a server‐sided plugin similar to Bukkit.

3. Approach 3.1 Game design foundations Our game design is based on the five essential elements which according to (Johnson and Johnson 1994) foster cooperative work in face‐to‐face groups:

Positive Interdependence: knowing to be linked with other players in a way so that one cannot succeed alone

Individual Accountability: individual assessment of each student's performance and giving back the results to both the group and the individual

Face‐to‐Face Promotive Interaction: Promoting each other's success by e.g. helping, encouraging and praising

Social Skills: Interpersonal and small group skills are vital for the success of a cooperative effort

Group Processing: Group members discussing their progress and working relationships together

We also take into account the rules and pitfalls as to regard when designing collaborative games as stated by (Zagal et al 2006):

Lesson 1: To highlight problems of competitiveness, a collaborative game should introduce tension between perceived individual utility and team utility.

Lesson 2: To further highlight problems of competitiveness, individual players should be allowed to make decisions and take actions without the consent of the team.

Lesson 3: Players must be able to trace payoffs back to their decisions.

Lesson 4: To encourage team members to make selfless decisions, a collaborative game should bestow different abilities or responsibilities upon the players.

Pitfall 1: To avoid the game degenerating into one player making the decisions for the team, collaborative games have to provide a sufficient rationale for collaboration.

Pitfall 2: For a game to be engaging, players need to care about the outcome and that outcome should have a satisfying result.

Pitfall 3: For a collaborative game to be enjoyable multiple times, the experience needs to be different each time and the presented challenge needs to evolve.

The use of these fundamental elements for a game design for collaborative multiplayer Serious Games is described in more detail in (Wendel et al. 2013). In this paper, we address how a game can be designed based on these rules and guidelines in order to create a collaborative multiplayer game specifically for enhancing teamwork and collaboration.

3.2 Game design for team building Based on the foundations stated in the previous section, we derived the following game elements for creating a necessity for collaboration inside a team‐based multiplayer game. Our design is for a team of four players.

http://bukkit.org/ http://get.spout.org/

571

Viktor Wendel et al. 3.2.1 Minecraft (sandbox) mechanic By the nature of the game, players are able to control their character in the way they want. They cannot be forced to certain decisions. At several points throughout the game, players will have to choose between selflessly helping the team (by getting themselves into danger) or staying safe. However, the team can never force one player to do something. This implements Lesson 2. 3.2.2 Creation of a common goal The players are provided with a background story telling them what they need to do in order to win the game and what failure means. The players' task is to save the 'last gnome on earth' (idea from the Left For Dead 2 gameplay mutation). If they are not able to save that gnome in time, the game will be lost. There are only two possible outcomes: victory and failure. By providing the players with such a narrative background, a foundation for the fact that players need to care about the outcome is created. This way we address Pitfall 2. Moreover, it becomes clear that the game cannot be won alone. If the players achieve their goal, they all won, if someone fails, the whole team fails. This refers to Positive Interdependence. 3.2.3 Player separation According to (Reuter et al. 2012), ‘Player Separation’ is a concept for multiplayer puzzles. In our concept, we use player separation to prevent one player from being able to solve all tasks (Pitfall 1). When players are physically divided only those players being present at position x can solve the respective puzzle there. Thus, we also address Lesson 4; we bestow players with different abilities, whereas the abilities are based on their location. 3.2.4 Gnome handling Only one player at a time can carry the gnome. This player will continuously slow down until he/she cannot move at all. Furthermore, this player cannot jump. This mechanic forces players to hand over the gnome between each other in order to be able to move the gnome forward and to overcome certain obstacles. This mechanic implements Positive Interdependence as well as Lesson 1. Players might always decide to take over the gnome (team utility), getting into danger of being caught by enemies following them or just to try to stay safe by not carrying the gnome. Here, it is important for players to learn about the consequences of their choices (Lesson 3). Staying safe while the fellow player, which is carrying the gnome, becomes too slow, thus getting caught by enemies might lose the game. 3.2.5 Team highscore At the beginning and at the end of a game session, the players can see the highscore including their own score provided they won the game. This rather simple method prevents Pitfall 3, as it creates a motivation to beat the highscore, thus to play the game again. As the collaborative puzzles are changing from game session to game session, the challenge is renewed each game, making another game session interesting, again. 3.2.6 Need for communication Resulting from the player separation, players often possess information which the other part of the team needs (see Section 3.3). This makes communication inevitable. Also, telling another player that he/she needs to take over the gnome is necessary. Several puzzles require the players to coordinate their actions (see Section 3.3), thus making communication essential. This requires Social Skills and Group Processing. During this procedure, also Face‐to‐Face Promotive Interaction takes place. We did, however, not directly incorporate Individual accountability into our design, as is seems rather counter‐ intuitive to reward selfish actions. Furthermore, it should not be important which player pressed a button but rather that the team figured out which button to press together.

572

Viktor Wendel et al.

Figure 1: Colored lever puzzle

3.3 Collaborative puzzles The collaborative puzzles we designed are based on the concepts for multiplayer puzzles as described in (Reuter et al. 2012). We make use of the concepts of 'Player Separation' and 'Heterogeneous Resources'. Using these, we designed the following puzzle types:

Lever Puzzle: This puzzle is based on player separation. Players of both parts need to pull levers in order to open doors, etc. This simple puzzle prevents single players from being able to advance alone.

Lever Color Puzzle: An extension to the previous puzzles is coloring levers. Players need to know which levers the have to pull. The information is given to the other part of the team such that they need to find that information and provide their teammates with it. The information is a resource which at first is only available to one part of the team (see Figure 1).

Gnome Button: We introduce a special button which can only be pressed by the player possessing the gnome. This supports the 'Heterogeneous Resources' feature.

Simple Math Puzzle: Again, using division of information, we designed math puzzles in the form of: 'Find d'. Hints are given in the form of 'd=c+5', ' c=b‐3', 'b=a+1', and 'a=8' with one part of the team having the first and the third hint and the other part having the other hints. This puzzle can only be solved if the team can give information back and forth.

Heavy Block: At some points, the players need to place a block in order to create a step to climb. Therefore, a heavy block needs to be moved from a starting point to its destination. The player carrying the block cannot move so that the players need to form a human chain to pass the block on. This requires all four players to move and work together.

Difficult Terrain: This puzzles type requires the gnome to be carried to several locations to press the gnome button there. The terrain is designed in a way such that one has to move a long way alone. However, handing over the gnome at special locations shortens the distance drastically. This requires all four players to move and work together.

4. Implementation 4.1 Spout and Bukkit As it seemed reasonable to be able to use our Mod with future Minecraft versions, we decided not to mod the Minecraft client itself, but instead to use the Bukkit framework. This way, it is possible to create a Mod which can be used together with other available mods due to the modularized use of mods in the Bukkit framework. The access to the client is limited without modding the client itself. Therefore, we decided to use spout, which is a framework enabling multiplayer Bukkit plugins with an access to the client when using the SpoutCraft client. The SpoutCraft client is a modified version of the Minecraft client. Thus, using the SpoutCraft client and SpoutPlugin as a plugin for Bukkit, a server‐side API for client side changes is available.

4.2 Level design All players start the game together in one room. From there they can decide which two players will walk through the hallway carrying the gnome. The remaining two players will help them from upstairs. Once two players used the one‐way teleporters towards the gnome hall, and they pick up the gnome, the game actually

573

Viktor Wendel et al. starts. The players then move along the hallway as shown in Figure 2. Throughout the game, the players will solve colored‐levers‐puzzles (colored circles), or math puzzles (numbers). At the points with the gnome symbol, the gnome is handed over to the other two players, effectively switching roles. The icon with the four players means that the ‘Player Separation’ is revoked and all four players meet to solve a task. The pyramid is a puzzle where all players need to build stairs with heavy steps by forming a human chain. During the red arrow parts, the players are under time pressure, as they are chased by zombies. This is in the beginning during the colored‐lever‐puzzles and the math puzzles and in the last part as to create a thrilling finale. Players need to run while being chased by zombies. They need to hand over the gnome frequently while pulling a lot of levers in order to open doors. Moreover, the terrain is difficult in this part so that handing over the gnome becomes more vital.

Figure 2: Schematic level overview

4.3 Modded features Following, we describe in detail the main features which were created for this mod in order to implement our game design. 4.3.1 Building restrictions Free placement of blocks as well as destroying blocks was forbidden. This was necessary in order to prevent players from creating shortcuts.

574

Viktor Wendel et al.

Figure 3: Gnome Sockets 4.3.2 Gnome handling To prevent deadlocks due to Minecraft game mechanics (i.e. items despawn (disappear) after a few minutes when dropped), it was forbidden to throw or drop items. The gnome and the heavy blocks are passed from one player to another by clicking on the receiving player if he/she is close enough. In addition to that, special gnome related blocks (sockets) have been created. In Figure 3, those blocks are shown: from left to right:

StartSocket: Players receive the gnome from there

SleepSocket: The gnome can be placed there (this is used for puzzles where all four players need to be able to move)

SleepSocket with gnome: The gnome can be taken back from there

EndSocket: The players need to bring the gnome there to win the game

4.3.3 Logic elements The most important part of our mod is the logic‐packet. It enables the freely definable combination of levers and buttons with doors and triggering of events without the 'redstone'‐related delay 4 . Our logic system is based on two elements: triggers and responses. They are connected via a context. A context can have more than one trigger or response. They can be connected via logical ANDs and ORs. Whenever a trigger is fired, it sends an event to its context which checks the logical condition and sends a signal to the responses if the condition becomes true (see Figure 4). 4.3.4 Zombie control Our game design requires some sort of pressure at some points in the game. We create that pressure in form of time pressure. The players need to solve puzzles while being chased by zombies. Zombies are entities which are originally available in Minecraft. Therefore, a ZombieManager package was created providing functionalities for spawning zombies at desired locations, despawning zombies, setting targets and movement speed.

Figure 4: Logic control 'Redstone' is part of the vanilla Minecraft used to create logical circuits

575

Viktor Wendel et al.

5. Evaluation 5.1 Participants 28 participants attended the study. Their age ranged from 21 to 45 years (m=25.81; sd=5.16). Two were female, 24 were male, two more did not name their age and gender. The mean time of playing video‐games a week was 9.04 hours (sd=10.64). One group was excluded due to one member of the group having major problems using the Minecraft game controls, which made meaningful play impossible for the group. So 12 participants joined the experimental and 12 participants joined the control group.

5.2 Design The study has a 2‐factorial design between subjects. The independent variable is the kind of cooperative game (puzzle/ Minecraft). As dependent measurement a user experience questionnaire, a questionnaire asking for the quality of group‐cooperation and a version of the prisoner's dilemma game was used (see Table 1). Table 1: Prisoner’s dilemma game Decision of other player Share Do not share

Own decision Share Small candy nothing

Do not share Choice of big candy Small random candy

5.3 Stimuli The two games used were our Minecraft mod and a jigsaw puzzle consisting of 2000 parts. Each participant got about a quarter of the parts, including some curbs. A jigsaw puzzle was selected, because it is a common, well known game that includes aspects of cooperation (like communication, division of cognizance and work), can be done for an adjustable time and presents the success of the cooperation to the player all the time. The user experience questionnaire includes 7 sub‐scales of user experience (negative emotion, positive emotion, cognitive load, motivation, immersion, flow and arousal). Each sub‐scale includes three items (e.g. frustration, anger, boredom for negative emotion). Each of these items has to be answered on a 10 point scale. An evaluation of 145 of these user experience questionnaires using different games and settings showed a Cronbach's Alpha of .93, this is to say the questionnaire is measuring one homogeneous construct. The questionnaires used to calculate reliability came from different studies conducted or overseen by the second author.The group‐cooperation questionnaire is based on the design elements for collaborative multiplayer Serious Games presented in (Wendel et al. 2012). It includes 15 items like 'the communication in the group was good'. The prisoner's dilemma game has been conducted with the reward system shown in Table 1. The game includes two questions: At first, in respect to (Sheese and Graziano 2005): 'please mark with a cross the answer you think your opponent will choose'. This enables to differ between uncooperative behavior that is caused by fear (of the opponent’s choice) and hostility. The second question was 'please mark with a cross the answer you like to choose'. In contrast to common versions of the prisoner's dilemma game the opponent was unknown (chosen randomly from the group members) because it is aimed to measure trust in the group and not in a specific person.

5.4 Aggregation For the evaluation the data has been aggregated as followed. For each participant the mean of the items of the UX and the group cooperation questionnaire was built. For the prisoner's dilemma game the deviation between the two rounds was build. Therefore cooperate was rated with a value of '+1' and not cooperate was rated with a value of '0'. The value of the second round was subtracted from the value of the first round. So, participants who did not change their behavior had a value of '0', those how chanced from cooperate to not cooperate have a value of '‐1' and those changing from not cooperate to cooperate have a value of '+1'.

576

Viktor Wendel et al.

5.5 Analysis To analyze the data a two tailed independent samples t‐test with the kind of game as independent measurement has been conducted for the UX questionnaire and the group‐cooperation questionnaire. The prisoner’s dilemma game was tested by a two tailed ANOVA (between subjects).

5.6 A Priori It was more fun (t(22)= ‐1.93; p=.066) to play the Minecraft Mod (m=6.70; sd=0.90) than to play the puzzle (m=5.85; sd=1.23). The group‐interaction has been rated more positive (t(22)=1.99; p=.060) by playing the Minecraft Mod (m=8.15; sd=0.70) than by playing the puzzle (m=7.42; sd=1.05). Neither the expectation of the behavior of the other nor the own chooses of the prisoner's dilemma game are different in dependency of the kind of game (p>.20).

5.7 Discussion 5.7.1 Hypotheses We hypothesized that our digital multiplayer game will lead to a better user experience than a common non‐ digital game with cooperative aspects. In fact, the user experience questionnaire as well as the group‐ cooperation questionnaire showed that the experience was better while playing the digital multiplayer game. So this hypothesis seems to be true. We also hypothesized that the digital multiplayer game will lead to more trust in the group members as well as to more cooperative behavior. But trust and behavior did not differ between the two games ‐ so this hypothesis has to be rejected. 5.7.2 Shortcoming One reason for the not significant differences in the prisoner's dilemma game may be the sample of the participants. Students and lecturers were accustomed to cooperative work. In this case they were even accustomed to each other. So they mostly started the prisoner's dilemma game with ‘cooperate’ and ended with ‘cooperate’ as well. In fact, 20 of the 24 participants did so. This assumption is also based on the cooperation values showing a mean value of 7.43 for the puzzle and a mean value of 8.15 for the game (on a 10 point scale). One could conclude that initial group‐cooperation and trust was very good throughout all groups ‐ perhaps too good to be improved by a 25 minute lasting game.

6. Conclusions In this paper, we presented an approach for a multiplayer Serious Game for enhancing teamwork abilities. Our game design is based on design guidelines found in literature. It is designed in a way to support collaborative behavior in a game through the overall setup, special collaborative puzzles and the use of mechanics like player separation and heterogeneous resources. We implemented our design as a Minecraft Mod. We decided to create a plugin for Bukkit using Spout‐ Plugin. Thus, we created a Minecraft level based to our game design. We also created an editor in order to enable further level creation for non‐programmers. We evaluated our concept using our Minecraft Mod prototype. Results showed that our Minecraft Mod provides a better game experience and group experience as a non‐ digital game with cooperative aspects. But to show a significant increase of trust and cooperative behavior a sample of participants might be required, that is not familiar to each other and therefore will not be so trustful to each other right from the start.

References J. Bayliss, J. (2012) Teaching game AI through Minecraft mods. In Games Innovation Conference (IGIC), 2012 IEEE International, pages 1‐4, Rochester, NY, USA. Bergeron, B. (2006) Developing Serious Games (Game Development Series). Charles River Media. Csikszentmihalyi, M. (2000) Beyond Boredom and Anxiety. Jossey‐Bass. Delwiche, A. (2006) Massively Multiplayer Online Games (MMOs) in the New Media Classroom. Educational Technology & Society, 9(3):160‐172. Duncan, S.C. (2011) Minecraft, Beyond Construction and Survival. Well Played, 1(1):1‐22. Gee, J.P. (2003) What Video Games Have to Teach Us About Learning and Literacy. Computers in Entertainment (CIE), 1:20. Harteveld, C. (2011) Triadic Game Design. Springer‐Verlag New York Inc.

577

Viktor Wendel et al. Johnson, D. and Johnson, R. (1994) Learning Together and Alone, Cooperative, Competitive, and Individualistic Learning. Needham Heights, MA: Prentice‐Hall. Larusson, J. and Alterman, R. (2009) Wikis to Support the Collaborative Part of Collaborative Learning. International Journal of Computer‐Supported Collaborative Learning, 4(4):371‐402. Lepper, M.R. and Henderlong, J. (2000) Turning" Play" into" Work" and "Work" into "Play": 25 Years of Research on Intrinsic Versus Extrinsic Motivation., chapter Intrinsic and extrinsic motivation: The search for optimal motivation and performance, pages 257‐307. Academic Press, San Diego. Levin, J. (2011) The Minecraft Teacher. online via http://minecraftteacher.net/. Mansour, S. and El‐Said, d.M. (2008) Multi‐Players Role‐ Playing Educational Serious Games: A Link between Fun and Learning. The International Journal of Learning, 15(11):229‐240. Nacke, L. (2009) Affective Ludology: Scientific Measurement of User Experience in Interactive Entertainment. PhD thesis, Blekinge Institute of Technology, Karlskrona, Sweden. Onrubia, J. and Engel, A. (2009) Strategies for Collaborative Writing and Phases of Knowledge Construction in CSCL Environments. Computers & Education, 53(4):1256‐1265. Prensky, M. (2002) The Motivation of Gameplay: The Real Twenty‐first Century Learning Revolution. On the horizon, 10(1):5‐11. Prensky, M. (2003) Digital Game‐based Learning. Computational Entertainment, 1(1):21. Reuter, C. and Wendel, V. and Göbel, S. and Steinmetz, R. (2012) Multiplayer Adventures for Collaborative Learning With Serious Games. In P. Felicia, editor, Proceedings of the 6th European Conference on Games Based Learning, pages 416‐423. Acad. Conf. Ltd. Schifter, C. (2012) Minecraft in an english class. Poster at 6th European Conference on Games Based Learning (ECGBL). Sheese, B. and Graziano, W. (2005) Deciding to Defect. The Effects of Video‐game Violence on Cooperative Behavior. Psychological Science, 15(5):354‐357. Squire, K. (2003) Video Games in Education. International journal of intelligent simulations and gaming, 2(1):49‐62. th Steinkuehler, C.A. (2004). Learning in Massively Multiplayer Online Games. In ICLS '04: Proceedings of the 6 international conference on Learning sciences, pages 521‐528. International Society of the Learning Sciences. Sweetser, P and Wyeth, P. (2005) GameFlow : A Model for Evaluating Player Enjoyment in Games. Computers in Entertainment (CIE), 3(3):1‐24. Van Eck, R. (2006) Digital game‐based learning: It's not just the digital natives who are restless. Educause Review, 41(2):16. Wendel, V. and Babarinow, M and Hörl, T and Kolmogorov, S. and Göbel, S. and Steinmetz, R. (2010) Transactions on Edutainment IV, volume 6250 of LNCS, chapter Woodment: Web‐Based Collaborative Multiplayer Serious Game, pages 68‐78. Springer. Wendel, V. and Gutjahr, M. and Göbel, S. and Steinmetz, R. (2013) Designing Collaborative Multiplayer Serious Games: Escape From Wilson Island ‐ A multiplayer 3D Serious Game for collaborative learning in teams. In Education and Information Technologies, 18(2): 287‐308. Zagal, J.P. and Rick, J. and His, I. (2006) Collaborative Games: Lessons Learned From Board Games. Simulation and Gaming, 37(1):24‐40. Zea, N.P. and Sánchez, J.L.G. and Gutiérrez, F.L. and Cabrera, M.J. and Paderewski, P. (2009) Design of Educational Multiplayer Videogames: A Vision From Collaborative Learning. Advances in Engineering Software, 40(12):1251‐1260.

578

Application of the Principles of Gamification to Facilitate Acquisition of Self‐Management Skills in Young People With Long‐ Term Medical Conditions Andrew Wilson 1 and Janet McDonagh 2 1 School of Computing, Telecommunications and Networks, Faculty of Technology, Engineering and the Environment, Birmingham City University, Millennium Point, Birmingham, UK 2 University of Birmingham & Birmingham Children's Hospital NHS Foundation Trust, Birmingham, UK andrew.wilson@bcu.ac.uk j.e.mcdonagh@bham.ac.uk Abstract: With improvements in health care children diagnosed with long term medical conditions are now more likely to live longer with the possibility that their condition will follow them into adulthood. As the young person grows older they will eventually have to transfer their care to the adult setting. Failure to plan and coordinate this has been associated with poorer health outcomes and disruption to their care. Transition planning encourages both health literacy and health promoting behaviours in an age and developmentally appropriate way. In order to gauge the attainment of these skills assessment tools have been developed. They all have items of commonality (understanding the condition, self‐ management, adherence to treatment and communication) but many are disease specific and address items that are specific to their country of origin (i.e. medical insurance). A series of transitional readiness checklists have been developed at Birmingham Children’s Hospital which are more focussed to young people in the UK. The domains they measure include knowledge, self‐advocacy, transferring to adult care, health and lifestyle, activities of daily living, school and vocation, leisure and managing emotions. Engaging and encouraging young people to work towards attaining the required skills and the assessment of their competency in them beyond the health care setting remains a challenge in every day clinical practice. The processes would therefore benefit from a more objective assessment for the doctor and structured in way to be more fun for the young person. Gamification is the term used to describe the use of game mechanics in non‐game contexts with the aim of trying to improving engagement and motivation. In this paper we will discuss how the dynamics of gamification (progress, feedback and behaviour) and game mechanics can be mapped to the self‐care and self‐ management skills associated with Birmingham Children’s Hospital transitional care checklists. This will include how badges can be used to indicate a young person’s progression towards attaining increasing levels of knowledge for example about their condition, the effects of their condition on their body as well as improving understanding of their treatments and any side effects associated with it. Other examples will include how trophies can be used to indicate successful understanding of a series of concepts at a particular developmental stage in a young person. This paper focuses on discussing the application of gamification to the Birmingham Children’s Hospital transitional care checklists. However this initial framework will give both clinicians and health care workers an insight into the use of gamification in a health care setting and provide a basis for application to other areas of knowledge and skills acquisition where engagement in the processes can be challenging. Keywords: gamification, adolescent, chronic conditions, transition, self‐care

1. Introduction It is estimated that between 10‐12% of young people in the UK may be affected by a long term medical condition (Coleman et al., 2011) which can persist with them into their adult life. Care initially starts in family centred clinics but as the young person grows older they have to transfer to the adult setting. This normally occurs between 16 and 18 years of age in the UK but depends more upon local policy rather than the young person’s confidence in being able to cope in the adult setting. Studies investigating the pre‐transfer period identified an increased risk for negative health outcomes. This included both increased incidence of graft loss (Harden et al., 2012) and chance of mortality in transplant patients (Annunziato et al., 2007), worsening disease activity in young people with long term rheumatic conditions (Hersh et al., 2009), or they are just lost in the health care system and consequently fail to attend clinics (Wacker, 2005; Yeung, 2008). Those that experience such lapses in care have also been reported to have an increased risk of subsequent readmission to hospital (Yeung, 2008; Nakhala, 2009). Even in those with stable disease, poorer vocational outcomes have been reported post transfer (Duguépéroux, 2008).

579

Andrew Wilson and Janet McDonagh The disruption to care is complicated by the fact that young people are undergoing those developmental changes that are normally associated with adolescence. It is also a time when patterns of health promoting as well as health risk behaviours are established which can be maintained throughout life (Sawyer at al., 2007). Health risk behaviours are as common if not more common in those individuals with long term conditions (Suris et al 2008) and there is the potential for greater adverse health outcomes from them (Sawyer at al., 2007; Philpott, 2011). Non adherence to treatment is common among young people with long term conditions (Hanna, 2012) and can be related to their engagement in health risk behaviours. Transition addresses the medical, psychosocial as well as educational and vocational needs of the young person. During transition the responsibilities for care moves from the parent to the young person requiring them to acquire a series of knowledge and skills that are important for them to function independently in adult care. These include understanding of health and condition specific issues, self‐management skills, effective information‐seeking skills, managing psychological and general health, effectively utilising the health care available to them, coping with social issues, maintaining education and vocation aspirations as well as living independently. These skills need to be acquired in an age and developmentally appropriate manner. They also depend on young person’s cognitive skills for example La Rosa and colleagues (2010) suggest that young children requiring organ transplants should be able to understand their condition and need for treatment, during early adolescence they should understand side effects associated with their treatment and the need for urgent care, whereas young adults should be able to schedule their own appointments, communicate with their care team and understand the need to move their care to adult centres (Table 1). Table 1: A time line of knowledge acquisition for young people with solid‐organ transplants Time Period Childhood, before transplant, early post‐transplant period. Pre‐teen period / early adolescence. Adolescence / young adulthood.

Behaviour Learning.

Activities Understanding medical condition, need for treatment, medication and frequent follow‐up.

Increasing knowledge and responsibility. Providing self‐care.

Understand indications for medications, reasons for urgent care, awareness of medical consequences of treatment. Recite medications and indications, call for appointments and refills, identify and call for urgent concerns, effectively communicate health concerns, take medications on own, anticipate and accept eventual need for transfer of care.

Adapted from La Rosa and colleagues (2010). Transitional readiness assessment tools have been developed to assist clinical care teams in evaluation a young person’s progress towards attaining the necessary skills associated with successful transfer to the adult care system. This paper will give an overview of some of the assessment tools currently available and will discuss how the application of gamification to one specific tool developed by Birmingham Children’s Hospital (UK) would benefit the process by creating a more objective framework for the clinical team as well as encapsulating the processes in a more fun way in order to incentives the young person engagement.

2. Measurements of readiness to transition Gauging a young person’s readiness to transfer their care depends upon their self‐belief, parental involvement as well as the assessment of competency by the health care team. Several assessment tools including the Paediatric Transition Readiness Score (PTRS) and Pediatric Acceptance Readiness Score (PARS) (La Rosa et al., 2010), Transition Readiness Assessment Questionnaire (TRAQ) (Sawicki et al., 2011) and The University of North Carolina (UNC) Traxnsition Scale (Ferris et al., 2012) have been developed to assist in ascertaining transitional readiness. Many of the items in these tools are common to one another for example knowledge and understanding of the condition and its treatment, self‐care and self‐management, communications with the care team and understanding appropriate support services. However they may also be disease specific and have aspects that are more relevant to their country of origin for example medical insurance and financial aspects of care. Figure 1 illustrate the “Getting Ready!” early transitional readiness checklist that has been developed for young people with rheumatic disease at Birmingham Children’s Hospital (UK).This is one of three plans which also includes “Moving along!” and “Moving up!” for mid and late adolescence. The domains assessed include

580

Andrew Wilson and Janet McDonagh knowledge, self‐advocacy, transferring to adult care as well as health and lifestyle, activities of daily living, school and vocation, leisure and managing emotions. Although not formally validated they continue to be used with all young people attending the adolescent rheumatology service from age 11 years irrespective of condition. These assessment tools are based on self‐reporting making them subjective and rely on the young person motivating themselves to maintain positive health behaviours. By implementing mechanics associated with gamification the process could be made more fun for the young person and objective for the clinician.

Figure 1. Examples of some of the questions associated with the checklists (“Getting Ready!” early adolescence) for assessing transitional readiness from the young person’s [A] and the health professionals [B] perspective. E, M and L denote early, mid and late adolescent development.

3. Gamification Gamification is a term that has been used to describe the use of game mechanics and game design techniques in non‐game contexts. The three main game dynamics associated with gamification progression, feedback and behaviour represent processes that can promote the knowledge, learning and behaviours that are important in acquisition of self‐management skills. Table 2 (a and b) shows the game dynamic and mechanics associated with gamification and how these can be related to the positive self‐care and self‐management behaviours which are required in young people for successful transfer of care. Table 2a: Mapping of gamification mechanics to self‐care behaviours Game Dynamic Progression

Game Mechanic

Mechanic definition

Achievements

Levels

Accomplishments which are unlocked after succeeding in a given task. A reward for amassing points.

Points

Numerical values given to

581

Examples relating to self‐care / management behaviours Successfully attaining self‐care and self‐ management skills. The attainment of more advanced knowledge and skills as the young person develops. Incentives to achieve skills and a measure of

Andrew Wilson and Janet McDonagh Game Dynamic

Feedback

Game Mechanic

Mechanic definition

Progression

actions. Success as measured through the completion of tasks.

Appointments Bonuses

Cascading Information Theory Combos

A predetermined time and place to participate in a game. Rewards for completing a series of challenges.

Release of information in small parts to ensure appropriate levels of understanding. Undertaking another action after successfully completing a previous one.

Examples relating to self‐care / management behaviours success. Demonstration that the skills are being attained and that satisfactory progress is being made toward transfer of care to the adult system. Encouraging the maintenance of regular contact with the clinical care team. Encouraging positive behaviour for example adherence to treatment regimes, appointments or demonstrating acquisition of skills in a timely manner. Providing information in an age and developmentally appropriate manner. Demonstrating an understanding of a series of skills or skills that are associated with one another for example knowing the types of medicine they need to take as well as their side effects.

Table 2b: Mapping of gamification mechanics to self‐care behaviours Game Dynamic Feedback

Game Mechanic Countdown

Quests

Reward schedules

Behavioural

Behavioural momentum Blissful productivity

Community collaboration Discovery

Epic meaning

Mechanic definition A time limit to accomplish a task. A series of challenges or obstacles a player must overcome. The timeframe and mechanisms through which rewards (points, prizes, level ups) are given. The tendency to keep doing activities. Being happy as a result of working hard and undertaking rewarding activities. The game community works together to solve a riddle, a problem or a challenge. The love of exploring and discovering something new and being surprised by it. The motivation to achieve something great and awe‐ inspiring.

Infinite gameplay

No explicit end.

Ownership

Used to create loyalty.

Status

The rank or level of a player.

Urgent optimism

Extreme self motivation to tackle an obstacle with the

582

Examples relating to self‐care / management behaviours A way of encouraging behaviours which are potentially lagging behind what would be expected / anticipated for that young person. A way of reducing anxiety when visiting new clinical care settings and care teams. Motivation, incentives and recognition of accomplished tasks and acquired skills.

Encouraging a state where positive health care behaviours become the norm for example adhering to treatment, maintaining a good diet and taking regular exercise.

Providing peer support, sharing information and experiences to reduce the feeling of isolation and being alone. Learning about their condition, treatment and how they can manage their care independently. A feeling of achievement that the young person has taken an active role in their own care and that they have successfully prepared themselves for adult life and the adult care system. Care for long term conditions is ongoing and will change as the person grows older. Maintaining positive health behaviours improves the chances of a better quality of life. Taking responsibility for self‐care as the emphasis shifts to the young person away from the parents. Demonstrating that the person has achieved the required skills and knowledge in relation to their peers and that it is achievable. The impetus to acquire important skills as quickly as possible even though they may be challenging

Andrew Wilson and Janet McDonagh Game Dynamic

Game Mechanic

Mechanic definition expectation of success.

Virality

“Spreading the word” in order to engage a wider audience.

Examples relating to self‐care / management behaviours for example having joint injections without general anaesthetic. The success of the system encourages more young people to participate in it and take an active role in its development and evolution.

Adapted from Gamification Wiki (source: http://gamification.org/wiki/Game_Mechanics)

3.1 Progression The Birmingham Children’s Hospital adolescence plans monitor progress and achievements from both the young person’s and doctor's perspectives. The young person rates themselves as to whether they feel they can manage the task on their own or if they need more advice and support with the doctor recording that the young person feels that they have attained the skills. By applying a point based system to each item in the domain this would give a measure of achievement and progress. For example if a young person demonstrates their confidence in being able to describe their condition at a level that the doctor would associate with early adolescence they would be rewarded with points and a badge indicating this achievement (Figure 2). Subsequently if they demonstrate their confidence in being able to describe their condition at a level that the doctor would associate with mid adolescence they would receive further points and a higher level badge. When all items, for example demonstrating the ability to successfully describe their condition, understanding the effects on their body and understands their medicines / regimes, within in a domain (knowledge) at a particular stage of adolescence have been completed this would unlock an achievement in the form of a trophy (Figure 2). Higher value trophies (for example bronze, silver and gold) are acquired as a result of demonstrating higher levels of knowledge associated with older adolescence. When all items within a domain are completed a further reward (a shield) can be earned signifying mastery of that domain at all levels.

Figure 2: Badges and trophies Badges indicate attainment of increasing levels of knowledge associated with a particular item of understanding. The trophies indicate attainment of all the required individual items of skill at a particular level (early, mid or late adolescence). There is also an overall reward (shield) for completing all items within a domain.

3.2 Feedback Clinical appointments are in important point of contact between doctor and patient in order to monitor progress and provide feedback. However failure to keep appointments is a common occurrence which results in disruption of care but also causes additional burden on clinical services. This could be enhanced further by applying gamification techniques by awarding points for attending clinical appointments which would reinforce their importance. Some of the items in the checklists (Figure 1) lend themselves to aggregation. For example the act of 1. Being able to describe both the condition and effects on the body, 2. Knows medicines / regimes and side effects of medicine and 3. Preparation for joint injections without general anaesthetic and has joint injections without

583

Andrew Wilson and Janet McDonagh general anaesthetic. If these combinations or (combos) have been completed or completed in shorter than expected time frame bonuses would be rewarded. Where there is a delay in the time that certain skills would be expected to be acquired the countdown mechanic could be used to add an incentive for the young person to engage and acquire those skills. Consideration should still be given to the fact that the young person should acquire skills in an age and developmentally appropriate manner, which reflects the cascading information theory where information is released in small parts ensuring that appropriate levels of understanding are met. Orientation to the adult system where the young person and parents visit the new environment and meet the new care team can be a cause for concern and anxiety. However by encapsulating this into a quest the uncertainty and the unknown is given an element of fun and sense of achievement by accomplishing it. Gamification uses rewards as a way of recognising achievements. There is a danger that rewards can be over used devaluing the achievements but if insufficient are used the incentive to engage with the process is lost. Reward schedule and ratios are used to manage this with randomness and surprises potentially being the most effective. Badges and trophies are the most common examples of reward however digital prizes, virtual goods and objects found or taken within the course of a game have been used and can be traded or given away giving the game an extra dimension. In terms of gamification engagement is considered the prime metric. As many are web based this is often measured by unique visitors, page views per visitor, time spent on site, total time spent per user, frequency and depth of visits, participation and conversions. Although these can be used to demonstrate engagement with the process in terms of knowledge and skill acquisition they are not as useful. Therefore other activities such as keeping blogs, diaries, and uploading videos demonstrating activities as well as participating in answering questions in on line forums can be more indicative. This user generated content (UGC) is an important part of many websites. Prompt and real‐time feedback is an important to the young person. When quizzes were incorporated into an interactive multimedia computer program that was designed to teach young people about juvenile idiopathic arthritis these were praised for helping to support consolidation of knowledge about the condition (Wilson & Young, 2009). Leader boards are a tool used to give a comparison of progress and feedback in relation to others. This can have a positive effect by motivating people to improve their performance. However it can also have negative effects where it is felt that the targets are unachievable. Marczewski (2013) suggests the use of both absolute and relative leader boards where the former would indicate progress against all others where the latter would indicate performance within a domain and / or within a developmental stage (i.e. early, mid or late adolescence).

3.3 Behaviour Education is an integral part of care for young people with long term medical conditions and forms the basis for positive health behaviours. However many young people and their family find it difficult to get the information they need and it is not always in appropriate style for them and therefore multiple formats have been requested (Shaw et al., 2004). Status is an important mark of achievement in games and provides an opportunity to identify where help and assistance can be found for new players. In terms of gamification for health care this provides and opportunity to identify mentors who can provide advice and support. Despite the importance of collaboration and social interaction for the young people anonymity is also very important. Feelings of being different and not "fitting in" can lead to social isolation and stigma. There is evidence that people with disabilities use virtual worlds and play online games as a way of combating social isolation and loneliness (Kamel Boulos et al., 2007). Anonymity in these worlds can be facilitated by the use of avatars, online characters that reflect the player’s persona whilst maintaining their privacy and hiding identity. The premise of gamification is to make activities more fun and as a result improve the chances of engagement. By using this approach in young people with long term health conditions the aim would be to foster feelings of epic meaning (the motivation to achieve something great and awe‐inspiring) and ownership of their own care (self‐care). The clinical care team would be able to influence the young people’s actions with the anticipation

584

Andrew Wilson and Janet McDonagh that the individual increasingly takes on responsibility for their care as it shifts from dependence upon the parent to themselves in states that reflect the mechanics of behavioural momentum and blissful productivity.

4. Technology The Internet has established itself as a major source for young people for discovering information about their health. This has evolved into dedicated youth‐centred online programmes to facilitate self‐management. These websites include interactive activities, self‐monitoring and reflection, youth‐to‐youth information sharing and social support, and accurate, accessible and developmentally targeted health specific information (Scal et al., 2010). Features such as discussion boards, stories of hope, and video clips help to provide support and reduce the feelings of isolation in their illness (Stinson at al., 2010). Computer games have also been used successfully to promote health education and improve both self‐management skills and self‐efficacy in young people as reviewed by Wilson & McDonagh (2012). Their ability to foster social interactions allows young people to learn about their illness with their peers and can stimulate discussions about health care with friends, family and the clinical team (Lieberman, 2001). Technology is therefore an important medium for the delivery of care to young people. The combination of online delivery and game mechanics is an important part of gamification. Services such as Badgeville (http://badgeville.com/) and gamificationU (http://gamificationu.com/) already provide resources to start creating game based systems.

5. Discussion and conclusion Maintaining the engagement of young people in their self‐care as they grow older is challenging as other factors increasingly become prominent in their lives. However the implications of not taking care of themselves are a poorer quality of life and delaying the inevitable transfer of care to the adult system. Creating a process that allows both the young person and the doctor to monitor progress and create incentives for maintenance of care in a fun way and which the young person can relate to would be advantageous. By implementing the principles of gamification this might help address these issues. However there are some factors that need to be taken into consideration. Engagement itself may be dependent upon age and stage of development that is younger people may be more likely to enjoy and participate in the challenges whereas interest may wane as the person matures. There is also the element of novelty, the challenges faced in order to maintain that game’s uniqueness and how different genders perceive and react to the game based system. Therefore there is the possibility it may only be effective within a certain time frame. As the system is a game and based upon achievements there is always the chance that people will try to cheat in order to gain the rewards which would be counter productive to the requirement that the young people are learning and achieving a set of skills and behaviour which are important to their care. As with any game development, implementation and refinement of a range of complex mechanics and balancing them to make it fair, fun and in this case affecting behavioural change is a time consuming process. Despite these reservations even if there are limitations to the effectiveness of the system as long as there are improvements in the overall acquisition of skills and in the confidence of the young person to care for themselves it would be successful and as a result has the potential to engage a wider audience. This paper has discussed the application of gamification to the Birmingham Children’s Hospital transitional care checklists. This initial framework will provide clinicians and health care workers with an insight into the use of gamification in a health care setting and provide a basis for application to other areas where engagement with knowledge and skills acquisition can be challenging.

References Annunziato, R.A., Emre, S., Shneider, B., Barton, C., Dugan, C.A. and Shemesh, E. (2007) “Adherence and medical outcomes in pediatric liver transplant recipients who transition to adult services”. Pediatric transplantation, Vol, 11 No. 6, pp 608‐614. Coleman, J., Brooks, F. and Treadgold, P. (2011) Key data on Adolescence 2011: The Latest Information and Statistics About Young People Today, Association for Young People's Health, 8th edition. Duguépéroux, I., Tamalet A., Sermet‐Gaudelus, I., Le Bourgeois, M., Gérardin, M., Desmazes‐Dufeu, N. and Hubert, D. (2008) “Clinical changes of patients with cystic fibrosis during transition from pediatric to adult care”, Journal of Adolescent Health, Vol. 43, pp 459‐465.

585

Andrew Wilson and Janet McDonagh Ferris, M.E., Harward, D.H., Bickford, K., Layton, J.B., Ferris, M.T., Hogan, S.L., Gipson, D.S. McCoy, L.P. and Hooper, S.R. (2012) “A Clinical Tool to Measure the Components of Health‐Care Transition from Pediatric Care to Adult Care: The UNC TR(x)ANSITION Scale”, Renal Failure Vol. 34, No. 6 pp 744‐753 DOI: 10.3109/0886022X.2012.678171. Hanna, K.M. (2012) “A framework for the youth with type 1 diabetes during the emerging adulthood transition”, Nursing Outlook, Vol. 60, No. 6, pp 401‐410. Harden, P.N., Walsh, G., Bandler, N., Bradley, S., Lonsdale, D., Taylor, J. and Marks, S.D. (2012) “Bridging the gap: an integrated paediatric to adult clinical service for young adults with kidney failure”, British Medical Journal, Vol. 344, e3718. Hersh, A.O., Pang, S. Curran, M.L., Milojevic, D.S., and von Scheven, E. (2009) “The challenges of transferring chronic illness patients to adult care: reflections from pediatric and adult rheumatology at a US academic center”, Pediatric Rheumatology Online Journal. Vol. 7, No. 13. doi: 10.1186/1546‐0096‐7‐13. Kamel Boulos, M.N., Hetherington, L. and Wheeler, S. (2007) “Second Life: an overview of the potential of 3‐D virtual worlds in medical and health education”. Health Information and Libraries Journal, Vol. 24, pp 233‐245. LaRosa, C., Glah, C., Baluarte, H.J. and Meyers, K.E.C. (2011) “Solid‐Organ Transplantation in Childhood: Transitioning to Adult Health Care”. Pediatrics, Vol. 127, No. 4 pp 742 ‐753, doi: 10.1542/peds.2010‐1232. Lieberman, D.A. (2001) “Management of chronic pediatric diseases with interactive health games: theory and research findings”, The Journal of Ambulatory Care Management, Vol. 24, No. 1, pp 26‐38. Marczewski A. (2013) “Gamification a Little on Leaderboards”, [online] Available at http://marczewski.me.uk/2013/01/21/gamification‐a‐little‐on‐leaderboards/ (accessed 18th March 2013). Nakhla, M., Daneman, D., To, T., Paradis, G., and Guttmann, A. (2009) “Transition to adult care for youths with diabetes mellitus: findings from a Universal Health Care System”, Pediatrics, Vol. 124, No. 6, pp 1134‐41. Philpott, J.R. (2011) “Transitional Care in Inflammatory Bowel Disease”, Gastroenterology and Hepatology, Vol. 7, No. 1, pp 26‐32. Scal, P., Garwick, A. W., & Horvath, K. J. (2010) “Making Rheumtogrow: The rationale and framework for an Internet‐based health care transition intervention”, International Journal of Child and Adolescent Health, Vol. 3, No. 4, pp 451‐461. Sawicki, G.S., Lukens‐Bull, K., Yin, X., Demars, N., Huang, I.C., Livingood, W., Reiss, J., Woodm D. (2011) “Measuring the Transition Readiness of Youth with Special Healthcare Needs: Validation of the TRAQ—Transition Readiness Assessment Questionnaire”, Journal of Pediatric Psychology, Vol. 36, No. 2, pp 160‐171. Sawyer, S.M., Drew, S., Yeo, M.S. and Britto, M.T. (2007) “Adolescents with a chronic condition: challenges living, challenges treating”, Lancet, Vol. 369, No. 9571, pp 1481‐1489. Shaw, K.L., Southwood, T.R., McDonagh, J.E. on behalf of the British Paediatric Rheumatology Group. (2004) “User perspectives of transitional care for adolescents with juvenile idiopathic arthritis”, Rheumatology (Oxford), Vol. 43, No. 6, pp 770‐778. Stinson, J., McGrath, P., Hodnett, E., Feldman, B., Ciaran, D., Tucker, L., Hetherington, R., Tse S., Spiegel, L., Campillo, S., Gill, N. and White, M. (2010) “Usability Testing of an Online Self‐management Program for Adolescents With Juvenile Idiopathic Arthritis”, Journal Of Medical Internet Research, Vol. 12, No. 3, pp e30, doi: 10.2196/jmir.1349. Suris,J.C., Michaud, P.A., Akre, C. and Sawyer, S.M. (2008). “Health risk behaviors in adolescents with chronic conditions”, Paediatrics, Vol. 122, pp e1113‐8. Wacker, A., Kaemmerer, H., Hollweck, R., Hauser, M., Deutsch, M.A., Brodherr‐Heberlein, S., Eicken A, and Hess, J. (2005) “Outcome of operated and unoperated adults with congenital cardiac disease lost to follow‐up for more than five years”, American Journal of Cardiology, Vol. 95, No. 6, pp 776‐9. Wilson, A.S. and McDonagh, J.E. (2012). “Moving on: Use of Computer Games During Transitional Care for Young People with Long Term Medical Conditions”. Proceedings of the 6th European Conference on Games Based Learning. Cork, Ireland. October 2012. ISBN 978‐1‐908272‐69‐0 / ISSN 2049‐0992 Wilson, A.S. and Young, S.P. (2009) The rise of the computer as an assistive technology for education in chronic musculoskeletal disease. (eds Demir O and Celik C). IN Multimedia in Education and Special Education. Nova Science Publishers Inc. Yeung, E., Kay, J., Roosevelt, G.E, Brandon, M. and Yetman, A.T. (2008) “Lapse of care as a predictor for morbidity in adults with congenital heart disease”. International Journal of Cardiology, Vol. 125, pp 62‐5.

586

Development of an Implementation Framework for Games‐Based Construction Learning Using Scratch in Primary Education Amanda Wilson, Thomas Hainey and Thomas Connolly University of the West of Scotland, UK amanda.wilson@uws.ac.uk thomas.hainey@uws.ac.uk thomas.connolly@uws.ac.uk Abstract: Games‐based Construction Learning (GBCL) and games‐based learning (GBL) are being promoted within the curriculum for excellence (CfE). Children in the Upper Primary (Primary 4 onwards) are now being encouraged to construct their own games. These changes have been made from the previous curriculum and are causing frustration with some teachers who believe that they have been left without adequate resources and professional development to introduce such approaches into their lessons. Surveys have shown that while teachers are making some use of GBL in their class very little is being done with GBCL. With environments such as Scratch, Kodu and Gamemaker, game construction is becoming more accessible to children at a younger age. This paper will present an implementation framework for the introduction of GBCL into PE by utilising Scratch. It will discuss the literature surrounding GBCL in PE before presenting a generalised framework for the introduction of GBCL into PE. The framework is based on empirical work carried out in 3 primary schools within Glasgow in 2011/2012 and aims to give teachers a starting point for using GBCL in the classroom. Keywords: primary education, curriculum for excellence, games‐based construction, scratch, review, pedagogy

1. Introduction Children in the Upper Primary in Scotland (Primary 4 onwards) are now being encouraged to construct their own games within the CfE. These changes have been made from the previous curriculum and are causing frustration with some teachers who believe that they have been left without adequate resources and professional development to introduce such approaches into their lessons. Barriers that have been identified when trying to implement GBL within the curriculum are a lack of knowledge in using and implementing the technology and games (Pivec and Pivec, 2008; Groff, Howells and Cranmer, 2010; Robertson, 2012). Van Eck (2006) suggests from a review of the literature three ways of introducing GBL into educational establishments either through the students creating their own games, through the students playing serious games or throughout the use of Commercial off the Shelf (COTS) games. While studies have been undertaken within primary, secondary and tertiary education, there is still insufficient empirical evidence to properly substantiate the use of computer games technology as a recognised educational approach (Meluso, Zheng, Spires and Lester, 2012; McClarty et al., 2012). With applications like Scratch (Resnick, Kafai and Maeda, 2003), game construction is becoming more accessible to children and although there are studies such as Adventure Author (Robertson and Good, 2005) and Storytelling Alice (Kelleher, Pausch and Kiesler, 2007) that enable children to create their own interactive stories that other children are then able to play, game construction has been relatively unexplored within the classroom (Baytak and Land, 2011). This paper will discuss the literature surrounding GBCL in PE in the next section before presenting a generalised framework for the introduction of GBCL into PE.

2. Related work 2.1 Games‐based construction learning Papert‘s theory of constructionism regards learning as a process in which the learner actively constructs his or her knowledge by interacting with the subject matter. The constructionist perspective puts game construction in the hands of children to encourage their knowledge through developing objects (Papert, 1991). In the game construction approach, the aim of the game construction tool is to support such an activity by providing an appropriate environment. Kafai (2006) suggested that constructionists have focused their efforts on providing

587

Amanda Wilson, Thomas Hainey and Thomas Connolly students with greater opportunities to construct their own games and to construct new relationships with knowledge in the process, rather than embedding lessons directly in games. By adapting the definition of GBL by Tang, Hannegan and Rhalibi (2009) who view GBL as “the innovative learning approach derived from the use of computer games that possess educational value or different kinds of software applications that use games for learning and education purposes such as learning support, teaching enhancement, assessment and evaluation of learners”, GBCL can be defined as “an innovative learning approach that uses appropriate tools in order to allow games to be constructed to support learning and teaching”.

2.2 Literature review A literature review was carried out to identify previous game making educational research using a number of electronic databases: ACM, Science Direct, IEEE (Institute of Electrical and Electronics Engineers) Computer Society Digital Library (CSDL), Taylor and Francis, Proquest (including Applied Social Sciences Index and Abstracts, Proquest Computing and ProQuest Dissertations and Theses A and I), EBSCO (including CINAHL PsycINFO, SocINDEX, Library, Information Science and Technology Abstracts), Cambridge Journals, ERIC, Ingenta, Infotrac and Web of Knowledge. The following search terms were used: (making OR construction OR building OR design OR creation) AND (game AND education) Kafai’s Minds in Play (1995) is viewed as a notable piece of work within games construction. The study looked at children as game designers over a six‐month period and investigated how they developed as learners during this time. Over this period for one hour a day, 16 9‐10 year old children worked together with the researcher and their class teacher to create fraction games in Logo to teach younger children. Kafai argued that by using the games design they can start to construct their own knowledge and their relationship to it. By taking Kafai’s research as a starting point the literature search started from 1995 onwards. The initial search collated 1,848 entries in total. Focusing on empirical work undertaken with games construction within PE 10 papers were returned (see Table 1). By reviewing the studies undertaken in GBCL it can be seen that using GBCL in the class is still an emerging field and is starting to grow with the advent of more child‐centred games construction tools such as Scratch and Kodu. The ten papers found cover seven studies and the games constructed within these papers fall into two categories either the children making the games are creating educational games for other children to use (Baytak, 2011; Kafai, 1995; Mendes and Romao, 2011; Vos, van der Meijden and Denessen, 2011; Koutsikos et al, 2012) or they are creating original games (Robertson, 2012; Wilson, Hainey and Connolly, 2012) with the focus being on what programming concepts they have used to create their games. Table 1: GBCL empirical literature Authors

Software Used

Study

Kafai, Franke, Ching and Shih (1998) Kafai and Ching (1996)

Logo

Robertson and Howells (2008)

Adventure Author

Baytak and Land (2011) Baytak (2009)

Scratch

Mendes and Romao (2011)

Software specifically created for the study

This study investigated game design as a learning environment with a 5th grade class creating fraction games for younger children. It showed that they were able to undertake the task with more sophisticated games being developed over time. An exploratory study into the use of game making within a class of 30 P6 (9‐10 year old) children, to develop aspects of the successful learner strand of the CfE. The study showed that game making does provide the opportunity for children to achieve successful learning, however to fully explore this lessons need to become teacher led. This study looked at the use of Scratch with a class of 5th grade children to create science games for younger pupils. It showed that children were able to create games within a short period of time and were able to make use of programming concepts. This paper discussed an authoring tool created by the research team called t‐games. It allows children to author their own games and uses a learning by teaching approach.

588

Amanda Wilson, Thomas Hainey and Thomas Connolly Authors

Software Used

Study

Vos, van der Meijden and Denessen (2011)

Software specifically created for the study

Robertson (2012)

Adventure Author

Wilson, Hainey and Connolly (2012)

Scratch

Koutsikos et al (2012)

Kodu

This study compared the difference in learning between children playing games and children making games. The researcher took both lessons within four elementary schools comprised of 235 students from 9 classes split over 5th/6th grade age 10‐12 years. 5 classes constructed the games and 4 classes played the games. It looked into the motivation of students who play vs. students who create games. Findings showed that the children who created games were more motivated in their learning. Using adventure author over a 6 week period with a class of 11‐12 year olds this study looked at the analysis of games created by children during the project and also focuses n gender issues within the class showing that girls scored higher than boys of the storytelling aspects of their games. The study looked at the use of Scratch in PE or games construction and the programming concepts that children used. It evaluated games created during an 8 week study undertaken in a school with 60 children participating. The results showed that children were constructing functional games with a variety of programming concepts being demonstrated. This pilot study was conducted in a class with 25 children using Kodu. The children were asked to design games based on an environmental theme with the objective being to introduce the principles of object‐oriented programming, and to assess the satisfaction and interest generated through this.

3. Methods 3.1 Literature search This framework has been developed to help address the research question “What framework would be suitable to implement Scratch within the Curriculum at upper school PE?” The framework presented in this chapter will act as a starting point for Head teachers in PE who may wish to introduce GBCL into the curriculum but are unsure of where to start. The research aims to bring about change within ICT lessons in upper Primary Education. This will be achieved by undertaking the research in stages with different classes in the upper primary school (classes between Primary 4 and Primary 7). The objective is to work in collaboration with the class teacher to investigate how their class works with Scratch and then develop a framework that teachers will then be able to use for themselves to introduce game making into their class without the researcher being directly involved in the class lessons in the final stage. After this study has been undertaken the results from that and the literature review will provide an implementation framework. While an initial literature search performed did not provide any implementation frameworks for GBCL by extending it to include ICT it then highlighted other frameworks and models within ICT in education that may be a suitable starting point. Table 2 provides an overview of implementation frameworks/models for ICT in education. Table 2: Implementation frameworks/models for ICT in education Author Kahn (2001)

Framework/Model A framework for web‐based training

De Freitas and Oliver (2006)

Four dimensional Framework

Tearle (2004)

Theoretical framework for identifying factors important for ICT implementation E‐capacity model

Vanderlinde and Braak (2011)

589

Purpose To help plan, design, develop and implement a learning environment Help practitioners evaluate the use of GBL within their particular learning context To identify important factors in relation to ICT implementation Designed to help with the integration of ICT process in schools

Amanda Wilson, Thomas Hainey and Thomas Connolly Kahn’s (2001) framework is designed to be used within education and training it has 8 dimensions pedagogical, technological, interface design, evaluation, management, resource support and ethical, and each dimension will vary in use according to the scope of learning taking place. The De Freitas and Oliver (2006) four dimensional framework looks at how educators can evaluate games in order to incorporate them into their own learning contexts. This framework is designed for educators to use in advance of their lesson planning. By looking at the particular context in which the learning is happening, the educator can take into account factors that may affect the lessons such as their own knowledge, technical support and access to tools. The second dimension takes the learner into account and focuses on attributes such as age and what stage of education they are at. Dimension three focuses on the game itself and the interactivity it has with the learner. Lastly the fourth dimension is pedagogical and this concerns how the game is embedded and supports the curriculum. Tearle (2004) proposes a theoretical framework that covers the influences on implementation of ICT in a school, by examining the school as a whole taking into account leadership, staff and internal processes. The implementation process then is considered with factors such as timing of the implementation, resources available, knowledge of ICT and support and training. Before looking at the individuals involved within the implementation which takes into account attitudes, belief and skills. Vanderlinde and Braak (2011) propose a conceptual model of e‐capacity that they see will help schools foster change through the optimisation of sustainable ICT. The model encompasses four layers that look at Leadership; this is seen to be an important factor in the success of any ICT implementation. Following on ICT related school conditions are considered this takes into consideration school policies, schools vision for ICT, support for ICT (including technical support) and having coordination of ICT. The next layer incorporates ICT related teacher conditions which encompasses professional development and teachers’ competence in ICT. Lastly the teachers’ actual use of ICT is seen as an extra layer and considers how teachers are making use of ICT in their classrooms. By reviewing these frameworks, key areas that should be included in an implementation framework are:

Leadership

Infrastructure

Professional development for teachers

Pedagogy.

3.2 Teacher survey A survey of primary teachers undertaken within 2 local council areas in Scotland (Razak et al, 2012) shows that while teachers are making use of GBL they are not making as much use of GBCL with only 9% of the 104 teachers stating they used both GBL and GBCL and 3% stating they used GBCL only. The survey also highlighted obstacles that teachers in Scotland face when trying to implement GBL in their class, which were lack of skills and training, lack of PCs and technology, difficulty identifying a suitable game and time and curriculum constraints. Other surveys on teachers show similar obstacles that are faced when trying to implement GBL/GBCL (see Table 3). Given the lack of literature surrounding GBCL in the classroom environment and implementation of GBCL in the classroom an initial study was undertaken to better inform a framework of implementation. Table 3: Obstacles faced by teachers Obstacles Lack of Technology Teacher Skills Suitability and Time constraints

Authors Williamson (2009) Stuart (2005), Sandford et al (2006) Wastiau, Kearney and Van den Berghe (2009)

3.3 Findings from study 1 The first study was undertaken in three different schools that all had different set ups with regards to their ICT suites and the ways in which lessons were delivered. Table 4 provides an overview of each school and their

590

Amanda Wilson, Thomas Hainey and Thomas Connolly differing ICT situations. Initial meetings with schools were set up with the schools head teacher after interest shown through the questionnaire sent to teachers. The meetings were used as a way for the researcher to inform the head teacher about the research and to evaluate what the schools were doing in terms of ICT work at present. The framework looks to address the barriers to implementation and give clear guidance on implementation lessons with Scratch in PE. From Study 1 it can be seen how these barriers can start to be addressed (see Table 5). Table 4: Comparison of schools Does the school have and ICT suite? How many PCs within the suite? Do the classrooms have PC’s? How many PCs in each class? Do teachers have access to laptops? How many laptops does the school have? Class sizes Does the school have dedicated ICT teacher Who is responsible for ICT lessons? Does the school have timetabled ICT lessons Does the school have ICT lesson plans? Are the teachers aware of CfE in relation to GBCL? Have teachers had any experience of GBCL? How does the head teacher feel about implementing GBCL? How do teachers feel about implementing GBCL?

School A Yes 20 Yes 1 Yes 14 – all teachers

School C No n/a Yes 2 Yes 7 teachers

25 Yes

School B Yes 9 Yes 2 Yes 9 – 2 in ICT room and 7 teachers 21 No

ICT teacher Yes

Class teacher No

Yes No

In progress No

Keen

Keen – some reservations in teachers confidence levels

Reservations due to lack of technology

27 No

Table 5: Addressing barriers to implementation Barrier Lack of technology

Issues noted during Study 1 3 different settings, gives good range to identify minimum requirements.

Teacher skills

Confidence in abilities Lack of knowledge of CfE with GBCL Lack of knowledge of GBCL tools Curriculum requirements – planning with staff/management Schools with no timetable were flexible, school that was timetabled changed timetable to suit.

Suitability Time Constraints

Overcoming the barrier Audit of equipment available in school Advanced planning CPD sessions Guidelines for implementing Scratch School ICT plans School ICT plans Advanced Planning

4. GBCL implementation framework Based on the teacher questionnaire, observational findings from Study 1 and the literature review, an initial model and framework for implementation of GBCL within the ICT curriculum is presented in Figure 1 and table 6. This model shows the various stages for best practise when schools would like to introduce GBCL into the curriculum and represents a high‐level summary of the framework presented in table 6, which provides a more detailed view of implementation of GBCL within the ICT curriculum. The model has seven stages school audit, GBCL tools, pedagogical develop, implement, evaluate and review. Overall, the implementation is led by head teachers and at each stage they should be showing strong leadership in order to encourage staff and progress with the implementation. The school audit phase addresses an individual school’s needs. It allows head teachers to assess what technology they have available to them and how best in can be utilised. This is extremely important – all schools have different setups when it comes to ICT suites as seen from the three schools within the initial study. School C is a school that, like a lot of

591

Amanda Wilson, Thomas Hainey and Thomas Connolly other schools, has the minimum that education services provided them with (2 PCs per class and a laptop for each staff member), however, with sufficient advance planning the school was able to ensure the class had the equipment for use on the day. Current ICT curriculum also needs to be taken account of, what are teachers doing currently in school with classes and how does this fit in with CfE and GBCL. Teachers’ skills need to be examined – do they have any experience of using GBCL tools and are they keen to use GBCL tools. On occasion external factors may also influence what is happening in school.

Figure 1: GBCL implementation model Once there is a clear picture of what is available in terms of skills and technology the next step is to find a suitable games construction tool to use within the class. This will be determined by the freedom schools have within their school network. Schools generally are limited by local authority policies and this has an impact on whether they are able to install their own software or have to go through a process within a managed network of getting software approved and installed. However, the latter option can be time consuming and costly for schools. Web‐based tools may be an option although again this relies on the infrastructure being able to support it. During Study 1, Scratch was available on one of the school’s networks and it was at that time possible to install it on the other two schools’ networks. Given the age of the classes that were undertaking the study (P4‐P7, age 8‐12 years) it was felt that Scratch was the most suitable tool to use. Step 3 considers the pedagogical aspects of implementation. Goals and objectives have to be established from the start, ideally within the school’s ICT plans with all staff having an input. Detailed lesson plans would take into account the number of available PCs/laptops the number of children in the class and what goals the teacher has for the lessons as well as curricular outcomes; e.g. are they going to make games with a particular theme (a topic for instance they are working on), will it be for another class in the school or simply to develop skills (collaborative working in primary is greatly encouraged. When a suitable tool has been selected, the next step is continuous professional development (CPD). If teachers’ skills need to be developed then CPD sessions may be required as well as detailed guidelines for use

592

Amanda Wilson, Thomas Hainey and Thomas Connolly of the tool in the class. This requires support and encouragement from head teachers in order to allow their staff the necessary time to become acquainted with the chosen game construction tool. Once teachers have addressed this, the lessons can then be implemented. Initial observations showed the confidence of teachers in their own abilities at the start of the researcher‐led lessons was low. However, after a couple of lessons and observing the confidence the children were showing through their work and collaborative working skills this encouraged the teachers and they were more participative within the lessons from then on. Initially the researcher led lessons comprised of a 5 minute talk from the researcher that then allowed the children around 40 minutes on the PC working on their game. From working alongside teachers of classes from Primary 4 through to Primary 7, teachers felt that this may not suit all classes particularly with the younger children and that possibly a more rigid approach was required. Aspects of the lessons at all ages required modifying and it was important to ensure progress was being made and that children were not spending too much time focussing on one particular aspect of the game they were creating. Observations by the researcher noted that sometimes a lot of time could be spent on the design stage of the game and while a lot of effort was going into the design no progress was being made with the actual functionality of the game. It is suggested that initially children should spend some time working on the design but they should have functionality and then when they have a functional game they could go back and modify the design of the game. Support from head teachers during the implementation is required in order for lessons to run smoothly particularly when situations arise, such as in school C when teacher laptops have to be utilised clear guidelines must be given and fellow teachers should be supportive of the teacher undertaking the lessons. It was noted in observations by the researcher that while teachers in school C sent their laptops along to the classes for use they had not been following the guidelines given to them by the head teacher – as such some laptops were inaccessible due to being locked by a teacher or they weren’t charged enough to last for the one hour lesson. After the lessons have been implemented an evaluation of the work has to be undertaken. This is important to let teachers see how the children are progressing and if they are meeting their learning intentions. Assessment can be undertaken by the children themselves either by peer assessment of the games or by self assessment. The assessments would be developed before any lessons take place and incorporate the learning intentions of the class teacher. The final stage of the model after the lessons have taken place is to review the implementation. This examines how teachers feel about the lessons and how the children did in the lessons. After Study 1 had been undertaken the teachers who had participated felt much more confident about their abilities. They felt that the children did get a lot out of the lessons though felt that lessons could be modified to suit the class that they were working with. Children also enjoyed the lessons and felt more confident in their abilities with game construction. Table 6: GBCL implementation framework 1) School audit Current ICT curriculum in school ICT infrastructure Teachers ICT skills

External influences 2) GBCL tools Tools available Staff learning 3) Pedagogy Context Detailed lesson plans

Points to consider It is important for all teachers to be aware of the CfE Experiences and Outcomes relating to ICT in particular game construction. Schools should be aware of what they have available to them and when there is It is important to discuss with teachers what skills they have and what skills they would like to develop. It is also important teachers are aware of game construction within the curriculum, This would include visits from Inspectors which can have an impact on school policy If the schools are on a managed network are they able to access any of these tools and at what cost to the school? Are they web based or do they need to be installed? Is there going to be a big learning curve for staff when introducing a new tool What tool is going to be used and how is it going to be embedded within the curriculum? Guides on using GBCL will help teachers become more comfortable

593

Amanda Wilson, Thomas Hainey and Thomas Connolly 1) School audit

4) Professional development

Points to consider when teaching they will give them a detailed introduction and show the main points often raised within lessons.

Teacher skills

Teachers should be comfortable using tools in lessons to construct games if they are not the head teacher has to put CPD in place for teachers to enhance their skills.

5) Implementation

Look to engage teachers

While only one or two teachers from the school may be participating it is useful for other teachers to support their colleagues.

Lessons to suit age and stage

Teachers will have to plan lessons according to the age and stage they are teaching.

Support of management

Head teacher should be encouraging teachers to ensure support for any teachers leading lessons this includes infrastructure problems and ensuring any problems are dealt with accordingly.

6) Evaluation

Game evaluation

Clear learning intentions – are they being met when the final game has been constructed.

Peer assessment

Peer review of games – do they hold interest of the person playing

Self assessment

Children evaluating their own games by way of statements and providing their assessment of how they did

7) Review

Teacher feedback

Reviewing lessons and making changes as necessary for teaching again. Teachers’ confidence in delivering lessons should be growing and skills continually improving.

Children’s feedback

Children reviewing lessons to help teachers make changes for future lessons.

5. 5 Discussion and future work By examining the literature it is clear that GBCL is only starting to become more widely used within PE. However, teachers feel there are many obstacles they face when trying to implement GBCL in school. The study undertaken within the three schools looked at areas such as the schools infrastructure and what software could run on the schools PCs. The lessons were then incorporated into the curriculum to help overcome what teachers felt were the barriers stopping the implementation of GBCL in school. The implementation framework presented has been developed to give PE establishments’ guidance on the implementation of GBCL within their school. It has shown the barriers that can be overcome to implement GBCL namely lack of technology, suitability and the teachers’ own skills. Further studies will involve PE establishments implementing the framework within their school.

Acknowledgements We would like to thank the staff and children of Royston Primary, Carntyne Primary and Alexandra Parade Primary in Glasgow for their participation in the study. This paper is a scientific publication as part of the Games and Learning Alliance (GaLA). GaLA is a Network of Excellence on ‗serious games‘ funded by the European Union in FP7 – IST ICT, Technology Enhanced Learning (see http://www.galanoe.eu). GaLA gathers the cutting‐edge European research and development organizations on ‘serious games‘, involving 31 partners from 14 countries. The University of the West of Scotland is one of the partners.

References Baytak, A. (2009). An investigation of the artifacts, outcomes, and processes of constructing computer games about environmental science in a fifth grade science classroom (Doctoral dissertation, The Pennsylvania State University). Baytak, A. and Land, S. (2011). An investigation of the artifacts and process of constructing computers games about environmental science in a fifth grade classroom Educational Technology Research and Development Springer Boston Vol 59 issue 7 pp 765‐782

594

Amanda Wilson, Thomas Hainey and Thomas Connolly de Freitas, S. and Oliver, M. (2006). How can exploratory learning with games and simulations within the curriculum be most effectively evaluated? Computers and Education 46 (3), pp. 249‐264. ISSN 0360‐1315. Groff, J., Howells, C. and Cranmer, S. (2010). The impact of console games in the classroom: Evidence from schools in Scotland. Futurelab, UK. Kafai, Y. B. (1995). Minds in play: Computer game design as a context for children's learning. Routledge. Kafai, Y. B., Franke, M. L., Ching, C. C. and Shih, J. C. (1998). Game design as an interactive learning environment for fostering students' and teachers' mathematical inquiry. International Journal of Computers for Mathematical Learning, 3(2), 149‐184. Kafai, Y. B., and Ching, C. C. (1996). Meaningful contexts for mathematical learning: The potential of game making activities. In Proceedings of the 1996 international conference on Learning sciences (pp. 164‐171). International Society of the Learning Sciences. Kafai, Y. B. (2006). Playing and Making Games for Learning: Instructionist and Constructionist Perspectives for Game Studies. Games and Culture 1: 36‐40. Kelleher, C., Pausch, R. and Kiesler, S. (2007). Storytelling alice motivates middle school girls to learn computer programming, ACM, San Jose, California, USA. Khan, B. H. (2001). Web‐based training. Englewood Cliffs, NJ: Educational Technology Publications. Kirriemuir, J. and McFarlane, A. (2004), 'Literature Review in Games and Learning', Nesta Futurelab series, Bristol. Koutsikos, L., Holeva, V., Zourelidis, S., Dova, M. and Patrikakis, C. (2012). Information and Communication Technology in th Greek Primary Schools: A Pilot Application In Proceedings of the 6 European Conference on Games Based Learning. Cork, Ireland, 4‐5 October 2012 McClarty, K. L., Orr, A., Frey, P.M., Dolan, R.P., Vassileva, V. and McVay, A. (2012). A Literature Review of Gaming in Education, Pearson Meluso, A., Zheng, M., Spires, H. A. and Lester, J. (2012). Enhancing 5th graders’ science content knowledge and self‐ efficacy through game‐based learning. Computers and Education, 59(2), 497. Mendes, L. and Romão, T. (2011). Children as teachers: a tangible approach. In Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology (p. 12). ACM. Papert, S. (1980). Mindstorms: children, computers, and powerful ideas, Basic Books, Inc. Papert, S. (1991). Situating Constructionism. In I. Harel & S. Papert (Eds.) Constructionism Norwood, NJ, Ablex Publishing. Pivec, P. and Pivec, M. (2008). Games in Schools, report on final literature review ISFE‐EUN Partnership. Razak, A. A., Connolly, T. M., Baxter, G. J., Hainey, T. and Wilson, A. (2012). The Use of Games‐based Learning at Primary th Education Level within the Curriculum for Excellence: A Combined Result of Two Regional Teacher Surveys, 6 European Conference on Games‐based Learning (ECGBL), 4‐5 October 2012, Cork, Ireland. Resnick, M., Kafai, Y. and Maeda, J. (2003). A Networked, Media‐Rich Programming Environment to Enhance Technological Fluency at After‐School Centers in Economically‐Disadvantage Communities., United States. Robertson, J. and Good, J. (2005). Adventure Author: An Authoring Tool for 3D Virtual Reality Story Construction. In Proceedings of the AIED‐05 Workshop on Narrative Learning Environments, 63‐69. Robertson, J. and Howells, C. (2008). Computer game design: Opportunities for successful learning, Comput.Educ., Vol. 50, No. 2, pp. 559‐578. Robertson, J. (2012). Making games in the classroom: Benefits and gender concerns. Computers & Education, 59(2), 385‐ 398. Sandford, R., Ulicsak, M., Facer, K. and Rudd, T. (2006). Teaching with Games: Using commercial off‐the‐shelf computer games in formal education, (Bristol: Futurelab). Stuart, R. (2005). Learning Through ICT, Glasgow City Council, Glasgow. Tang, S., Hanneghan, M. and El Rhalibi, A. (2009). 'Introduction to Games‐Based Learning', in Liz Boyle Mark Stansfield, Thomas Connolly (ed.), Games‐Based Learning Advancements for Multi‐Sensory Human Computer Interfaces: Techniques and Effective Practices (IGI Global), 1‐77. Tearle, P. (2004). A theoretical and instrumental framework for implementing change in ICT in education. Cambridge Journal of Education, 34(3), 331‐351. Vanderlinde, R., and van Braak, J. (2010). The e‐capacity of primary schools: Development of a conceptual model and scale construction from a school improvement perspective. Computers & Education, 55(2), 541‐553. Van Eck, R. (2006). Where do we go from here? Ten critical areas to guide future research in DGBL. Workshop for the annual meeting of the Games, Learning, and Society Conference, June 15–16, Madison, WI. Vos, N., van der Meijden, H. and Denessen, E. (2011). Effects of constructing versus playing an educational game on student motivation and deep learning strategy use. Computers & Education, 56(1), 127‐137. Wastiau, P., Kearney, C. and Van den Berghe, W. (2009). How are digital games used in Schools? Main results of the study, Synthesis Report European Schoolnet, EUN Partnership AISBL. Williamson, B. (2009). Computer games, schools, and young people: A report for educators on using games for learning. Futurelab, UK. Wilson, A. Hainey, T. and Connolly, T. (2012). Evaluation of Computer Games Developed by Primary School Children to th Gauge Understanding of Programming Concepts. In Proceedings of the 6 European Conference on Games Based Learning. Cork, Ireland, 4‐5 October 2012

595

596

PhD Research Papers

597

598

Game Literacy Revisited: Developing Critical Play in Schools Rafael Marques de Albuquerque1 and Shaaron Ainsworth2 1 PhD student funded by CNPq – Brazil 1, 2 Learning Sciences Research Institute, School of Education, University of Nottingham, Nottingham, UK Lpxrmd@nottingham.ac.uk Shaaron.Ainsworth@nottingham.ac.uk Abstract: Digital games, as other technologies, are evaluated as monsters or heroes from different researchers. Such effects are not caused by digital games per se, but by the interaction between player and digital game. The objective of media education is to improve this interaction, what is also described as developing media literacy, and in the specific case of digital games, game literacy. This paper proposes a new concept of game literacy in order to inspire an alternative game education practice within media education programmes. Using the media education and game literacy literature as a basis, we integrated the work of the critical pedagogue Paulo Freire (1970/2012) and game studies that claim both positive (e.g. system thinking, problem solving) and negative (e.g. addictive behaviour, violent affection) effects of gaming. Seven characteristics shape the proposal: (i) The work of Paulo Freire emphasises criticality, rather than previous studies that proposed an emphasis on creative, cultural or protective approaches to media. (ii) The inclusion of the potential negative effects of digital games that are pointed by researchers in classroom discussions, considering that criticality towards the negative effects may minimize them. (iii) The inclusion of positive usage of digital games claimed by researchers, considering that developing agency and criticality towards the positive potential may support learners to reach them. (iv) Regarding positive and negative effects, the proposal considers the agency of learners to understand, judge and act on their own game habits, rather than imposing gaming rules or values. (v) The curriculum is created in a dialogic process; meaning that both educationists and students collaborate on defining the relevant themes. (vi) The educational practice involves learners’ experiences with digital games, in order to enable them to transfer their criticality to their actual practice. (vii) Methodologically, the teaching method is based on coded learning objects that are decoded by learners in a process of problematization, that leads to experimentation in gaming habits and, possibly, change. Keywords: game literacy, game education, media education, digital games, critical play

1. Introduction Over the last decades digital games and education have seemed to be at war. On one side of the war, game designers want to develop incredibly engaging and profitable games. On the other side, media scholars, educationist and parents try to protect children against the potential harm of playing games such as violence, addiction, stereotypes, isolation and obesity. This war has also manifested in academia, where many publications present evidence of the harms of games, whereas others claim benefits of games such as complex problem solving, contact with highly contextualised rich simulated environments, learning through mistakes, development of design skills, and wider “life lessons” such as optimism, self‐esteem, tolerance etc. This paper discusses the potential role of schools in this war. Media education argues that schools should prepare students to develop media literacy and digital games are included in its programme. This paper proposes a concept of game literacy that aims to inspire classroom practice of game literacy teaching in schools, and that merges three perspectives: (i) some of the existing literature of game literacy, (ii) the work of the critical pedagogue Paulo Freire, and (iii) the research claims regarding the effects of gaming – including the positive and the negative. This proposal is both theoretical – by developing an understanding of what could be considered a game literate individual – and pragmatic, because the concept is methodologically oriented towards practice.

2. Literature review Four sections are presented in this review. The first and the second present, respectively, the claims regarding negative and positive effects of gaming. These ideas are presented because our concept of game literacy includes awareness of both its potential for eliciting positive and negative effects, based on the assumption that digital games are not good or bad per se, but depending of the nature of their interaction with players. Rather than reaching final conclusions based on this review, our intent is to clarify some of the topics that could be discussed in classrooms in media education practice, and also to make evident the relevance of such

599

Rafael Marques de Albuquerque and Shaaron Ainsworth intervention. The third and fourth sections present aspects of media education and game literacy literatures in order to clarify the background of our proposal.

2.1 Dangerous gaming? According to the survey in US with 1,102 teenagers carried out by Lenhart et al. (2008), 97% of teenagers (12‐ 17 years old) play computer, web, portable, or console games. The Kaiser Family Foundation study (2010) with more than 2,000 teenagers in the US found that, on average, each one spend 90 minutes playing digital games in a typical day (73 minutes on consoles and 17 minutes in computer games). As games and digital technologies are becoming more ubiquitous, these numbers have been increasing in the recent years. Consequently, it is only natural that scholars (as well as parents and teachers) are worried about the possible harm that such frequent engagement with digital games could cause. Violence within digital games is probably one of the biggest of these concerns. Researchers have reviewed a number of studies – both correlational and experimental – and concluded that the exposure to violent games is causally related to heightened levels of aggression, including aggressive behaviour, cognition and affection, as well as decreased empathy and prosocial behaviour (Anderson and Bushman, 2001; Anderson et al., 2010). However, this claimed link is questioned by others. The studies that associate games and violence, however, were also severely criticised. Based on the work of Ritter and Eslea (2005), Jones (2005) and Ferguson (2010a), some of the main critiques are: (i) the tests used in violence research usually present a situation that is suggestive for aggression for the participant. Violence studies usually (ii) ignore the perceptions and motivations of participants and (iii) consider artificial situations that are not real violence – such as rough‐and‐tumble play – as evidence of actual violence. Furthermore, some studies (iv) ignore the possible benefits of fictional violence in digital games such as psychological development, self‐esteem and wellbeing. Jones and Ferguson also discuss the exaggerated panic of the violence in games in terms of society, and not as an isolated phenomenon within individuals. Moreover, studies that tried to conduct more realistic research found no evidence of relationship between violent game exposure and aggressive affect (Valadez and Ferguson, 2012). Ferguson (2010b) also found that other important variables – for example, depressive symptoms – were not considered in previous studies and may be better predictors of violence than violent games usage. There are other concerns about the potential harm of playing digital games. These include stereotyped representation or underrepresentation regarding gender (Brenick and Henning, 2007) and ethnicity (Burgess et al. 2011) which may strengthen discriminatory conceptions of players (Kirsh, 2010). Another concern is addictive behaviour towards gaming (Grusser et al. 2007). Although Skoric et al. (2009) found no relationship between game addiction and lower academic performance of students, it stills require further discussion, mainly in an individual level. Furthermore, media usage and obesity are also associated (Strasburger, 2010), although television probably has a more significant impact than gaming on this issue. In summary, it seems that the simple and straightforward fears of early studies concerning and automatic causal link with violence may be overstated. However, we should not dismiss the potential that playing digital games can have harmful effects.

2.2 Glorious gaming? In his book, Johnson (2005) describes a shift in the paradigm about the effects of video games, from something bad towards something good. It is far from a homogeneous position amongst scholars, but several studies suggest that digital games have the potential to be beneficial and, in some cases, there is empirical evidence of these claimed benefits. According to Gee (2007), good games are intellectually challenging, they stimulate players to think about interactions between variables in complex systems replete of contextualized information to be articulated. Based on the uncertainty of the dynamics of the challenge, it was claimed that some games offer the player the chance to develop a behaviour of enlightened risk taking and entrepreneurship (Shaffer et al., 2005; Gee, 2007), that includes generation of innovative solutions (Shaffer, 2006). This is not only because such games reward these kind of behaviour, but also because players have the chance to learn how to learn from their mistakes (Gee, 2007). Players also have to learn how to manage different levels of goals, hypothetically

600

Rafael Marques de Albuquerque and Shaaron Ainsworth creating a sense of priority and management (Johnson, 2005; Gee, 2007). Gee (2007) and Shaffer (2006) claimed that some game experiences have a similar reasoning to scientific enquiry, and the different ways to manage knowledge and perform different identities, e.g. roleplaying a city manager or an airplane pilot, enable diverse learning experiences. Digital games can be an ethical experience as well. Simkins and Steinkuehler (2008) found that roleplaying games have the potential to be used as spaces of practice of ethical decisions, in which critical decision making might flourish. According to the reviews of Greitemeyer (2011) and Gentile et al. (2009), several studies – including correlational, longitudinal and experimental studies – presented evidence that games with prosocial content may affect both cognitive and affective variables, increasing prosocial outcomes and decreasing antisocial outcomes. Lenhart et al. (2008) found that the kind of games that teenagers play is strongly related to teens’ interest and engagement in civic and political activities. The context surrounding digital games may also be beneficial. Digital games offer an environment to develop effective social practices (Shaffer et al. 2005). Based on an ethnographic study, surveys and experiments, Steinkuehler and Williams (2006) suggested that online games may replace other socialization places, offering the opportunity to develop bridging social capital, and more rarely, even bonding social capital. Moreover, players have the chance to engage in literacy practice in online forums and websites related to gaming (Steinkuehler, 2007; 2010; Thorne et al., 2009), and in countries where English is not the first language, there is evidence and self‐reports of English learning by gaming (Skoric et al. 2009; Cruz et al. 2012; Vintetjärn, 2008). Players that choose to get involved in map creation, modding games, game creation, fan art or fan fiction, also develop several skills in order to produce it, such as writing, design, art creation and programming (Squire, 2011). Evidence of cognitive improvement from digital gaming is also available. Spence and Feng (2010) published a review in which empirical studies suggest that gaming may improve: (i) spatial cognition, (ii) size of the attentional visual field, (iii) visual spatial resolution, (iv) contrast sensitivity, (v) visuomotor coordination/speed, and (vi) visual memory recall. Moreover, Dye et al. (2009) stated that evidence showed that action game practice may reduce reaction times without sacrificing accuracy. Although it may sound very optimistic and the authors did comment that research points to several cognitive benefits of gaming, they also remind us that it varies immensely to each game and game genre, and that more research is needed. All these benefits are more likely to be achieved in some contexts, with some games and with some players than others. Consequently, we argue there is a role that school can play to support students to have a more positive and less harmful use of digital games. The context for improvement of the interaction between student and media is media education.

2.3 Media education The appeal of media education is based on the importance that media has on young people’s lives, and it can make schooling more relevant to pupils’ lives. Media does not only influence a citizen’s relationship with information and news, but also tastes and judgments, pleasure, and expressions of identity (Burn and Durran, 2007). The main focus of media education is to change the interaction between learners and media, and not to use media to teach, which is another interesting initiative, but not media education (Buckingham, 2003). Consequently, the use of media for teaching – as educational/serious games, for example – is not discussed in this paper. The qualities of the interaction with media that are usually desired in media education are creative, critical and cultural. The creative aspect comes from the original meaning of literacy – that is considered the ability to read and write, therefore, to produce. Buckingham (2003) emphasizes that creative engagement with media is related to an active citizenship; because such approach towards media gives learners the possibility to share ideas, and not only consume them. The critical approach to media suggests that learners should develop mindfulness towards media, rather than automaticity (Potter, 2004). It also implies that the consumer should be capable of reflecting about what is behind media – such as political and commercial interests. The cultural approach suggests that learners can understand games as cultural artefacts that are embedded in other aspects of culture (Burn and Durran, 2007),

601

Rafael Marques de Albuquerque and Shaaron Ainsworth and that popular culture such as digital games is also culturally valuable (Buckingham, 2003). The inclusion of games within media education programmes, however, requires specific discussion.

2.4 Game literacy There are a few published empirical studies exploring game literacy. For example, Squire (2008) presented ‘good’ games to students, asked them to play and debriefed the experience, discussing how games are made, how they represent real world contexts, and how they involve a mix of expertise and art expressions such as art, music and programming. The game, in this context, was carefully selected and would be an example with the function to illustrate and inspire further discussion. Following the logic that game literacy includes being creative in regard to games some perspectives proposed that game creation activities in schools could be the basis of game literacy development. The experience described in Burn and Durran (2007) and Buckingham and Burn (2007) enabled students to create a game. The assumption is that by carrying out the role of designers and developers of games in an environment supported by teachers, students would develop a critical and cultural understanding of games. Zimmerman (2007) and Owston et al (2009) also related game literacy development through game creation, but they proposed that game creation is beneficial because it develops other skills. The former claims that it may develop a sense of system, play and design, and the latter, that game creation develops the skills of writing, searching for relevant information, questioning and editing. However, we query the reliance on game creation. Game literacy is sometimes defined by what it should be – based on previous concepts of literacy, digital literacy (Zwieten, 2012) or media literacy (Burn and Durran, 2007; Buckingham and Burn, 2007), and it implies that learners should be able to create digital games in order to be game literate. We question this assumption, asking if game creation is a desired outcome to the same degree that the creation of other kinds of media such as video and websites. Consequently, we focus on developing practices that can be detached from or sequenced with game creation. We also suggest that game education should avoid relying on games that are unknown to learners, and prefer to work with the games that students play at home. This is consistent with the aim to transform schooling into something more relevant for students. Accordingly, it could be expected that the inclusion of these games in classroom would improve the chances that students transfer their new understandings to actual gaming habits in their homes. The emphasis on the actual game practice of students also inspired the perspective of Klimmt (2010), however in different ways. He had suggested that game literacy should look forward the formation of normative players by teaching strategies that would allow them to avoid the potential harm of games – such as aggressive and discriminatory behaviour and game addiction. Protective approaches to media literacy are common in media education history and are nonetheless severely criticised by Buckingham (2003), Potter (2004) and Gutiérrez and Tyner (2012) mainly for (i) victimising students, treating them as vulnerable, defenceless and uncritical individuals, for (ii) ignoring all the potential benefits of media usage, and (iii) for trying to impose external values and procedures. Accordingly, we agree with his emphasis on everyday games whilst disagreeing with the desire to inform normative play. Approaches aiming to develop a more positive usage of existing games have also been proposed. Partington (2010) experimented the creation of what he called Games and Me posters in an English class. In the posters students made explicit their game knowledge and the relevance of games in their lives. The intent was to avoid working with some kind of canon of games, focusing in the cultural background of the students and their experiences, in order to “extend students’ beyond their experiences, so that they are able to apply them independently in other contexts” (p. 85). In the context of high education Zagal (2010) developed two techniques in order to improve the understanding that students have of games. One of these approaches was the GameLog website, which mixed the ideas of electronic portfolio and blogging in order to support students in reflecting about their gaming experiences. The game literacy proposals of Partington (2010) and Zagal (2010) are the closest to my proposal, because they work with the games that learners spontaneously play in their leisure time. However, the former proposed isolated activities that are not included in a wider plan or game literacy and media education, and

602

Rafael Marques de Albuquerque and Shaaron Ainsworth the latter aimed to prepare game scholars and professionals of the game industry. Our proposal is a more extensive plan of activities, and is designed for schools. Moreover, the theoretical emphasis is different, as we aim to merge critical pedagogy and media education into a more complete proposal of game literate citizen.

3. Game literacy revisited This paper proposes a concept of game literacy that can be used to design school practice, probably within media education programmes. We build on previous concepts of game literacy and media literacy, and add two main theoretical contributions: the work of the critical pedagogue Paulo Freire and the game literature that has researched the effects and potentials of commercial games on players. Paulo Freire discussed media very briefly, and when he did so, his emphasis was on television and radio (Freire and Guimarães, 2011), not digital games. The context in which he developed his work was by teaching literacy for adults in poor regions in Brazil, and it is very different from the context we are proposing in this paper; therefore, we will draw on his work, but are aware of the need for adaptation both in his theoretical and practical contributions.

3.1 Game literate individuals To be able to develop a pragmatic concept of game literacy applicable in school contexts, we need to understand what is desired from the student, as a citizen, regarding his/her relationship with digital games. This decision is a moral one; and we want to draw on the idea of Potter (2004) when he suggests that the only value of media education is criticality. It means that rather than teaching what learners should do with their games, they shall receive opportunities to develop their criticality towards it, thus, being able to actively decide how they use media. To Freire (1970/2012) it means to respect of the potential of human beings of being better, of using their own will and intelligence to act in their life and change it accordingly. The potential harm of digital games, in this context, is prevented by criticality, not submission. It means that all the potential harm of digital games should not be “solved” by repression or by obedience to rules, and not ignored either, but problematized and contextualised into the students’ lives, empowering them with the agency to deal with potential risks. The potential benefits of digital games are also presented and experienced in classrooms as possibilities, not guidelines. As Gee (2007) claims, an active and critical approach to games may enable players to develop a very beneficial gaming habit, and although family, school, friends or media may stimulate a more critical play, it is a decision of the learner. The literate player, thus, doesn’t necessarily develop games, write fan fiction or create deep social relationships in online games, for example. Neither s/he necessarily chooses the most complex games, nor must have the most restricted gaming hours. However, the literate player is aware of the possibilities, is able to critically analyse games and the roles that games may play in one’s life, and has agency to change his/her reality according to his/her will and actual possibilities, understanding the consequences of such choices. Game literacy is one specific aspect of a critical subject in general. It is unlikely that game education alone will develop complete critical subjects. However, the formation of critical subjects is a goal that may be embedded in diverse aspects of life, including other school spaces, family, politics and possibly also game design and development (Frasca, 2001). This paper only proposes what media education could do in order to bring about greater criticality towards game practice. Hopefully this work shall contribute with other initiatives towards the formation of critical subjects.

3.2 Game literacy methodology This proposal is oriented toward practice and action in schools. To Freire (1970/2012), the dialogic nature of education means an exchangeable relationship between theory and practice and between schooling and daily life of learners. It means that the boundaries between everyday experience of learners and school practice shall be blurred (Lewin, 2004), and that formal and informal learning become related, not dichotomous (Kent and Facer, 2004). In the experiential learning model of Kolb (1984), school moment would be a moment of abstract conceptualization that stimulates the active experimentation, concrete experience and reflective observation at home. Kiili (2005) also proposed that experiential learning could be used to conceptualise game

603

Rafael Marques de Albuquerque and Shaaron Ainsworth based learning, but suggested that reflective observation and abstract conceptualization should be supported by computer based tutors in educational games. We propose that this role could be played by teachers to change the relationship that students have with commercial games. Ideally, the dynamic of experiential learning would be incorporated by students and they would be able to engage in the whole process by their selves, at home, reflecting and learning critically through gaming. The curriculum, according to Freire (1970/2012), should be a dialogic process that empowers learners to contribute with relevant aspects of their lives (Pelletier, 2009), as well as the aspects that educationists believe that are important. The dialogic definition of the curriculum aims to design educational processes that are oriented towards action and change, in which the school experience is transferred to daily life. To educate is to create consciousness of the learners’ relationship with the world and not the world alone (Freire, 1970/2012), and in this case, it means the relationship with digital games. It is the rationale for use the games that learners play, rather than emphasise the presentation of alien games or software that will probably be accessed only in the classroom environment, during the course duration. The teaching method, based on Freire (Ibid.), includes the production of coded learning objects that represents the relevant topics, called generative themes. Some examples of themes are violence in games, or games as accurate simulations of reality. The generative themes are, however, coded, meaning that they are presented as a situation‐problem, as a challenge to be understood, and the learner perceives it as an external situation. Through the discussion about the topic, the learning object is decoded and the theme is problematized. It enables the learners to see themselves as included in the theme that seemed to be external and different from their reality. It offers the chance to perceive their realities from a distance, therefore, to critically analyse it, and to relate it to their own experiences. Afterwards, the discussion leads into action and practice towards change, and in this case, towards active experimentation, concrete experience and reflective observation (Kolb, 1984), that is, basically, critical play. Critical play can allow learners to understand that their relationship with digital games is a relevant aspect of their lives that sets up positive and negative possibilities, which depend on the learners themselves. By stimulating learners to engage critically with digital games, possibly more learners will benefit from gaming, and less will find the habit to be harmful.

4. Final considerations In this paper we presented the first model of a new proposal of game literacy that is oriented towards action within media education. It approximates media education, media effects studies and the work of the critical pedagogue Paulo Freire. This proposal comes to develop – or at least to present an alternative to – the role of digital games within media education, and will be carried out in the next years and enriched with empirical studies that will collaborate towards a more mature theoretical basis and work proposal.

Acknowledgements This project is financially supported by CNPq, National Council for Scientific and Technological Development ‐ Brazil

References Anderson, C. A., Bushman, B. J. (2001). "Effects of Violent Video Games on Aggressive Behavior, Aggressive Cognition, Aggressive Affect, Physiological Arousal, and Prosocial Behavior: A Meta‐Analytic Review of the Scientific Literature." Psychological Science 12(5): 353‐359. Anderson, C. A., A. Shibuya, et al. (2010). "Violent Video Game Effects on Aggression, Empathy, and Prosocial Behavior in Eastern and Western Countries: A Meta‐Analytic Review." Psychological Bulletin 136(2): 151‐173. Brenick, A., A. Henning, et al. (2007). "Social Evaluations of Stereotypic Images in Video Games: Unfair, Legitimate, or ''Just Entertainment''?" Youth & Society 38(4): 395‐419. Buckingham, D. (1996). "Critical pedagogy and media education: a theory in search of a practice." Journal of Curriculum Studies 28(6): 627‐650. Buckingham, D. (2003). Media Education: literacy, learning and contemporary culture. Cambridge, Polity Press. Buckingham, D. and A. Burn (2007). "Game Literacy in Theory and Practice." Journal of Educational Multimedia and Hypermedia 16(3): 323‐349. Burgess, M. C. R., K. E. Dill, et al. (2011). "Playing With Prejudice: The Prevalence and Consequences of Racial Stereotypes in Video Games." Media Psychology 14(1): 289‐311.

604

Rafael Marques de Albuquerque and Shaaron Ainsworth Burn, A. and J. Durran (2007). Media Literacy in Schools: Practice, Production and Progression. London, Paul Chapman Publishing. Cruz, D. M., D. K. Ramos, et al. (2012). "Jogos eletrônicos e aprendizagem: o que as crianças e os jovens têm a dizer? Contrapontos 12(1): 87‐96. Ferguson, C. J. (2010A). "Blazing Angels or Resident Evil? Can Violent Video Games Be a Force for Good?" Review of General Psychology 14(2): 68‐81. Ferguson, C. J. (2010B). "Video games and youth violence: a prospective analysis in adolescents." Journal of youth and adolescence 40(4): 377‐391. Frasca, G. (2001). Videogames of the Opressed: Videogames as a means for critical thinking and debate. Master`, Georgia Institute of Technology. Freire, P. (1970/2012). Pedagogia do Oprimido. Rio de Janeiro, Nova Fronteira. Freire, P. and S. Guimarães (2011). Educar com a mídia: Novos diálogos sobre educação. São Paulo, Paz e Terra. Gee, J. P. (2007). What video games have to teach us about learning and literacy. New York, Palgrave MacMillan. Gentile, D. A., C. A. Anderson, et al. (2009). "The effects of prosocial video games on prosocial behaviors: international evidence from correlational, longitudinal, and experimental studies." Personality & social psychology bulletin 35(6): 752‐763. Greitemeyer, T. (2011). "Effects of Prosocial Media on Social Behavior: When and Why Does Media Exposure Affect Helping and Aggression?" Current Directions in Psychological Science 20(4): 251‐255. Gutiérrez, A. and K. Tyner (2012). "Educación para los medios, alfabetización mediática y competencia digital." Comunicar: Revista Científica de Educomunicación 38(19): 31‐39. Johnson, S. (2005). Everything Bad is Good for You: How Today's Popular Culture Is Actually Making Us Smarter. New York, Riverhead Books. Jones, G. (2003). Killing Monsters: Our Children's Need for Fantasy, Heroism and Make‐believe Violence. New York, Basic Books. Kaiser Family Foundation (2010). Generation M2: Media in the Lives of 8‐ to 18‐Years Olds, The Henry J. Kaiser Family Foundation: 85. Kent, N. and K. Facer (2004). "Different worlds? A comparison of young people's home and school ICT use." Journal of Computer Assisted Learning 20(1): 440‐455. Kiili, K. (2005). "Digital game‐based learning: Towards an experiential gaming model." Internet and Higher Education 8(1): 13‐24. Kirsh, S. J. (2010). Media and Youth: A developmental perspective. Chichester, Wiley‐Blackwell. Klimmt, C. (2009). "Key Dimensions of Contemporary Video Game Literacy: Towards A Normative Model of the Competent Digital Gamer." Eludamos. Journal for Computer Game Culture 3(1): 23‐31. Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, Prentice Hall. Lenhart, A., J. Kahne, et al. (2008). Teens, Video Games, and Civics. Washington, PEW INTERNET & AMERICAN LIFE PROJECT: 76. Lewin, C. (2004). "Access and use of technologies in the home in the UK: implications for the curriculum." Curriculum Journal 15(2): 139‐154. Owston, R., H. Wideman, et al. (2009). "Computer game development as a literacy activity." Computers & Education 53: 977‐989. Partington, A. (2010). "Game literacy, gaming cultures and media education." English Teaching: Practice and Critique 9(1): 73‐86. Pelletier, C. (2009). "Games and Learning: What's the Connection?" International Journal of Learning and Media 1(1): 83‐ 101. Potter, W. J. (2004). Theory of Media Literacy: A Cognitive Approach. Thousands Oaks (US), Sage Publications. Rittern, D. and M. Eslea (2005). "Hot Sauce, Toy Guns, and Graffiti: A Critical Account of Current Laboratory Aggression Paradigms." Aggressive Behavior 31(1): 407‐419. Shaffer, D. W. (2006). "Epistemic frames for epistemic games." Computers & Education 46(1): 223‐234. Shaffer, D. W., K. R. Squire, et al. (2005). "Video games and the Future of Learning." PHI DELTA KAPPAN 87(2): 105‐111. Simkins, D. W. and C. Steinkuehler (2008). "Critical Ethical Reasoning and Role‐Play." Games and Culture 3(3‐4): 333‐355. Spence, I. and J. Feng (2010). "Video games and spatial cognition." Review of General Psychology 14(2): 92‐104. Squire, K. (2008). Video‐game Literacy: A Literacy of Expertise. Handbook of Research on New Literacies. J. Coiro, M. Knobel, C. Lankshear and D. J. Leu. New York, Lawrence Erlbaum Associates Taylor & Francis Group: 635‐669. Squire, K. (2011). Video Games and Learning: Teaching and Participatory Culture in the Digital Age. New York, Teachers College Press. Steinkuehler, C. (2007). Massively Multiplayer Online Gaming as a Constellation of Literacy Practices. The Design and Use of Simulation Computer Games in Education. B. E. Shelton and D. A. Wiley. Rotterdam, Sense Publishers: 181‐208. Steinkuehler, C. (2010). "Video Game and Digital Literacies." Journal of Adolescent & Adult Literacy 54(1): 61‐63. Strasburger, V. C., A. B. Jordan, et al. (2010). "Health effects of media on children and adolescents." Pediatrics 125(4): 756‐ 767. Thorne, S. L., R. W. Black, et al. (2009). "Second language use, socialization, and learning in Internet interest communities and online gaming." The Modern Language Journal 93(Focus): 802‐820.

605

Rafael Marques de Albuquerque and Shaaron Ainsworth Valadez, J. J. and C. J. Ferguson (2012). "Just a game after all: Violent video game exposure and time spent playing effects on hostile feelings, depression, and visuospatial cognition." Computers in Human Behavior 28(1): 608‐616. Vintetjärn, B. (2008). Video Games – a Source of Knowledge? An investigation amongst youths about their thoughts on their video gaming habits. Master, Malmö University. Zagal, J. P. (2010). Ludoliteracy: Defining, Understanding and Supporting Games Education. ETC Press. Zimmerman, E. (2007). "Gaming literacy Game Design as a Model for Literacy in the 21st century." Harvard Interactive Media Review 1(1): 30‐35. Zwieten, M. v. (2012). Video Games Literacy: a Theoretical Framework. M. A. in New Media and Digital Culture, Utrecht University.

606

A Systematic Literature Review of Methodology Used to Measure Effectiveness in Digital Game‐Based Learning Anissa All, Elena Patricia Nuñez Castellar and Jan Van Looy iMinds, MICT, Ghent University, Ghent, Belgium Anissa.All@Ugent.be ElenaPatricia.NunezCastellar@Ugent.be J.Vanlooy@Ugent.be Abstract: In recent years, a growing number of studies is being conducted into the effectiveness of digital game‐based learning (DGBL). Despite this growing interest, however, it remains difficult to draw general conclusions due to the disparities in methods and reporting. Guidelines or a standardized procedure for conducting DGBL effectiveness research would allow to compare results across studies and provide well‐founded and more generalizable evidence for the impact of DGBL. This study presents a first step in this process by mapping current practices through a systematic literature review. The review included peer‐reviewed journal and conference publications between 2000 and 2012. Other inclusion criteria were that (1) the study’s primary aim was effectiveness measurement of cognitive learning outcomes, (2) the focus was on digital games and (3) a pre‐post design with a control group was used. Twenty‐five publications were found eligible for this study. Important differences were found in the number of control groups used and the type of intervention implemented in the control group (e.g. traditional classroom teaching, use of multimedia, computer‐based learning, paper exercises, other games, or no intervention). Regarding the implementation method of the DGBL intervention in the experimental group, two approaches can be distinguished: stand‐alone intervention or as part of a larger program. Moreover, a wide variety of effectiveness measures was used: measures for learning outcomes were complemented with time measurements and/or with self‐reported measurements for self‐efficacy and motivation. Learning effect calculation also varied, introducing pre‐test scores in the analysis, conducting a separate analysis on pre‐ and post‐test scores or conducting an analysis on difference scores. Our study thus indicates that a variety of methods is being used in DGBL effectiveness research opening a discussion regarding the potential and requirements for future procedural guidelines. Keywords: effectiveness, digital game‐based learning, cognitive learning outcomes

1. Introduction In recent years, attention for the use of digital games has grown in a wide range of sectors. Digital games that do not primarily aim at entertainment have been deployed in the field of education, health and wellbeing, government, NGOs, corporate, defence, marketing and communication (Sawyer and Smith 2008). This growing interest in digital game‐based learning (DGBL) has resulted in an increasing amount of publications on the topic (Michael and Chen 2005). One important aspect in this field of research is effectiveness measurement (Connolly et al. 2012) whereby effects of DGBL and contributing elements on learning outcomes are assessed. An important limitation in this field is the incongruity of study designs (Kharrazi et al. 2012), which makes comparison across studies problematic. A consistent approach in effectiveness measurement would create the possibility to map important aspects of effectiveness on a more general level. Furthermore, uniformity in effectiveness studies on DGBL would help us gain better insight in validity and reliability of single studies. The present study takes a first step in the development of standardized guidelines by mapping the methods currently being used in effectiveness research on DGBL.

1.1 Defining effectiveness In the literature, learning is often clarified on the basis of the generated outcomes (Gagne 1984). An effective instructional method can thus be described as a method which has a positive impact on learning achievement and therefore learning outcomes (Joy and Garcia 2000). An instructional method has been defined by Salomon (Cited by (Clark 1994) p. 23) as “Any way to shape information that activates, supplants or compensates for the cognitive processes necessary for achievement or motivation.” Effectiveness of an instructional method can thus refer to either learning outcomes and/or motivation. According to Salomon (1993) the relationship between a medium used to teach and learning is an interaction between cognitive processes and characteristics of the mediatized environment. Medium and learning content are therefore inherently connected, implying that characteristics of the medium can influence the learning outcome (Kozma 1994). A characteristic that has been detected as an important aspect in the learning potential of digital games is their intrinsically motivating character (Garris et al. 2002), meaning that the activity in itself is engaging and no

607

Anissa All, Elena Patricia Nuñez Castellar and Jan Van Looy external reward for performing the activity is expected (Jenkins 2009). Intrinsically motivating activities create an enjoyable and fun experience, increasing the likelihood of repetitive usage (Ritterfeld et al. 2009). Another aspect of effectiveness is transferability, which refers to the transfer of knowledge in a formal context to situations in real life (Kozma 1994). When the transfer between a formal context to real life situations is low, this is defined as inert knowledge (Whitehead 1959). According to several authors inert knowledge is often due to usage of traditional teaching methods, which are outdated in that respect (Renkl et al. 1996). Garris et al. (2002) state that, in the context of DGBL, this transfer can be stimulated by organizing a debriefing session after gameplay.

1.2 Effectiveness studies in DGBL Typically, an experimental design is implemented to assess learning outcomes in a DGBL context by comparing a game‐based approach with another type of instruction and/or no intervention. The types of interventions to which the game‐based approach is compared can vary, which implies that results will ultimately depend on the particular comparison that is made (Bleumer et al. 2012). According to Campbell et al. (1963) the best experimental methodology for establishing whether learning has taken place is a pre‐test post‐test approach, including both an experimental and a control group. Questionnaires are typically used to assess the motivational aspects of DGBL, gauging the motivations of participants for learning via the intervention received and their interest in participation (Hainey 2010). Questionnaires are also implemented to assess other affective outcomes, such as attitudes. Moreover, some studies use in‐game assessment – referred to as stealth assessment – which is a technique that aims at accurately and dynamically measuring the player’s progress (Shute et al. 2011). Finally, qualitative methods such as interviews and observation have also been used in the context of effectiveness studies of DGBL. Three types of effectiveness studies in DGBL can be distinguished based on learning goals embedded in digital games (Bleumer et al. 2012). Specifically, digital games can aim at either knowledge transfer (cognitive learning outcomes), skill acquisition (skill‐based learning outcomes) or attitudinal and behavioural change (affective learning outcomes). Games aimed at knowledge transfer are typically implemented in education. For example, some studies have found a positive impact of the use of digital games to teach math (Bai et al. 2012) and language (Yip and Kwan 2006). Digital games aimed at skill acquisition are typically implemented in a training and corporate context. Several studies have observed an impact of playing games to practice managerial skills (Corsi et al. 2006). Games aimed at behavioural change are typically implemented in the health sector. An example of this are the healthy eating games influencing the diet and physical activity of children (Baranowski et al. 2008). Games aimed at attitudinal or behavioural change are implemented to raise awareness on a certain topic, such as poverty (Neys et al. 2012). According to Kraiger et al. (1993) these different types of learning outcomes require different types of assessment. Including studies aimed at the three learning outcomes would result in an extra level of heterogeneity, depending on the type of outcome that is assessed. Therefore, we will focus on one type of learning outcome in this study, that is cognitive learning outcomes.

2. Method In the present study the Cochrane method was used to carry out our systematic literature review (Higgins et al. 2008). This review method has its origins in health research and aims to study the effectiveness of interventions for prevention, treatment and rehabilitation. According to Cochrane, four dimensions of study characteristics can be distinguished: 1) participants (e.g. characteristics of the sample involved), 2) intervention (contents, format, timings and treatment lengths, intervention(s) in control group(s)), 3) methods (e.g. applied research methods) and 4) outcome measures (e.g. instruments used to measure a certain outcome) and results (Higgins et al. 2008). This distinction was also made in the present study. Search engines used for our review were Web of Knowledge, EBSCO Host and the International Bibliography of the Social Sciences. The following search string was used: ((Edu* OR serious OR learn* OR digital game based learning) AND ((dig* OR video OR computer) AND game) AND (assess* OR effect* OR measur*)). This search identified 54 publications dealing with effectiveness of DGBL aimed at cognitive learning outcomes. The review included peer reviewed journal and conference publications between 2000 and 2012. Other criteria for

608

Anissa All, Elena Patricia Nuñez Castellar and Jan Van Looy inclusion were that (1) the study’s primary aim was effectiveness measurement of cognitive learning outcomes, (2) the focus was on digital games and (3) a pre‐post design with a control group was used. Eight studies had a post‐only design with a control group and 21 studies had a pre‐post design without a control group which were all excluded. Eventually, 25 studies with a pre‐post design and control group were considered eligible for analysis. A quantitative content analysis was conducted using SPSS. The codebook for this analysis was created inductively, using qualitative coding in nVivo. For this, open and axial coding (Glaser and Strauss 2009) were used for analysing procedure and methods sections of the studies based on Cochrane guidelines.

3. Results 3.1 Participants 16

14 12 10 8

6 4

2 0 not specified Children

teenagers young adults

Frequency

Figure 1: Subjects included in study (n = 25) 16 14 12 10 8 6 4 2 0

11 5

5 2

Figure 2: Inclusion criteria (n = 25 The average sample size of participants in studies reviewed was 220 (SD = 284). Although not all the studies reported the number of participants included by group (8% did not), our results showed that when reported the average number of participants was 105 (SD = 163) in the experimental and 84 (SD = 92) in the control group. Although four studies reported participants’ mean age, most studies defined subjects based on types of people, such as ‘university students’. Sixty‐five per cent of the studies included children, 24% teenagers and 12% young adults (Figure 1). Inclusion criteria for participation were thus mostly school‐related (e.g., ‘majoring in math and science). Several studies only included a certain subgroup, including participants based on ability (e.g., low achievers), socioeconomic status or a certain health condition (Figure 2).

3.2 Intervention Experimental groups (EG) were compared to a control group (CG) that either included participants that did not get an intervention (24%), got an intervention using another instructional approach (56%), or were compared

609

Anissa All, Elena Patricia Nuñez Castellar and Jan Van Looy to several control groups, combining both (16%). One study did not provide any information on interventions implemented in the CG (Table 1). Table 1: Interventions in control group (n = 25) Intervention in control group(s) Traditional classroom teaching Traditional classroom teaching, with the use of multimedia Computer‐based application, such as an educational website Other game not related to the subject of the game implemented in the EG Paper and pencil exercises No intervention Not specified

N 12 1 4 2 3 10 1

% 48 4 16 8 12 40 4

Frequency

In the larger part of the studies (64%) DGBL was implemented in a formal context (e.g., in school during school hours), 8% in an informal context (e.g., home setting) and 12% in a semi‐formal context (Figure 3) referring to an implementation in a formal institution, such as a school, but where gameplay occurred outside of school hours. Sixteen per cent did not specify the context of play and 56% did not specify the gameplay composition (Figure 4). Twenty‐four per cent let participants play individually, 4% individually in competition, 24% cooperatively and 4% in a cooperative competition, meaning groups of participants played together against other groups of participants. One study implemented all four gameplay conditions. 18 16 14 12 10 8 6 4 2 0

2 not specified

formal context

informal semi‐formal context context

Frequency

Figure 3: Context of play (n = 25) 18 16 14 12 10 8 6 4 2 0

6 1

Figure 4: Gameplay composition (n = 25) Forty per cent of the studies did not report on the presence of an intermediary, referring to a teacher or researcher present during gameplay. In 56% of the studies an intermediary was present. One study did not include an intermediary. The average implementation period was 9 days (SD = 6), with a minimum of 1 day and a maximum of 23 days. Average total interaction time with the game is 12.4 hours (SD = 14.8), with a minimum of 30 minutes and a maximum of 64 hours.

610

Anissa All, Elena Patricia Nuñez Castellar and Jan Van Looy Games were either implemented as a stand‐alone intervention (28%) or were embedded in a larger program (48%). Twenty‐four per cent did not specify implementation. Table 2 gives an overview of program specifications. Table 2: Specifications about games embedded in a larger program (n = 12) Program specifications Introduction

N % 5 20

Description An introduction concerning game content and gameplay was provided by an intermediary. This does not refer to an in‐game introduction A training session before the intervention was provided

Training of participants before intervention Extra material

Online platform

Game task formulation

Required reading

Procedural help by intermediary

The participants received help concerning the actual gameplay. This does not relate to content

Guidance by intermediary

The participants received guidance during gameplay in order to contextualize the game in the broader learning context

Supplement of course Debriefing

6 3

24 12

Gameplay occurred next to the classes A debriefing session was provided

Extra material such as articles, extra exercises, extra reading material, etc. were freely available The game was part of a larger educational online platform Certain tasks were formulated during gameplay The participants were expected to read next to gameplay

Several studies implemented the game as a supplement of a course. However, half of these provided extra time for the experimental group to interact with the game in addition to the courses, therefore spending additional time with the learning content.

3.3 Method All studies reviewed implemented an experimental design. Forty‐for per cent used a randomized controlled trial; 24% randomly assigned subjects while 20% randomly assigned classrooms to one of the conditions. Twelve per cent did not randomly assign participants to experimental and control group(s), but ‘matched’ participants in groups based on certain characteristics such as previous test scores, and 44% did not specify on group assignment of participants.

3.4 Measures Less than half (44%) implemented standardized tests, six of these only used standardized tests while 5 studies combined standardized tests with tests developed by the researchers. Twenty per cent of the studies reviewed only implemented tests developed by the researchers and 24% used school tests or exams (‘student achievement’) as an accuracy measure. Two studies used both test scores and student achievement as an accuracy measure. Twenty‐eight per cent did not report on the similarity between the pre‐ and post‐test measurements. Forty per cent employed the same test before and after the intervention, 8% changed the sequence of the questions and 8% used a similar test (e.g., other questions with the same type and difficulty levels). The latter did not report on how similarity of parallel tests were assessed. Sixteen per cent used a dissimilar pre‐ and post‐test, such as midterm exam scores and final exam scores. Two studies also implemented a mid‐test and four studies a follow‐up test. Table 3 gives an overview of measures used in the studies. Thirty‐six per cent of the studies reported on how scoring on tests occurred. Three studies (12%) included an independent coder, of which two controlled for inter‐rater reliability. One study used several, non‐ independent coders to control for inter‐rater reliability.

611

Anissa All, Elena Patricia Nuñez Castellar and Jan Van Looy Table 3: Measures used for determining effectiveness (n = 25) Objective measurements Accuracy Test scores Student achievement Time measurements Time on task Subjective measurements

N 19 16 5 2 2 N

% 76 64 24 8 8 %

Self‐measurements Self‐efficacy topic Self‐efficacy general

8 4 2

32 16 8

Perceived educational value Affective Measurements Motivation

2 N 10

8 % 40

Motivation towards educational intervention Post‐only, EG Post‐only, EG and CG Pre‐ and post, EG and CG Motivation towards learning/educational content Post‐only, EG and CG Pre‐post, EG and CG Other

7 3 2 2 3 2 1 2

28 12 8 8 12 8 4 8

Attitudes towards school Teacher expectations

1 1

4 4

The larger part of the studies (76%) did a check on pre‐existing differences between experimental and control group(s) and 36% of the studies included in this review reported on effect size. Twenty‐four per cent did not report on statistical analysis. Table 4 shows how analysis of tests occurred. Table 4: Data‐analysis (n = 18) Data analysis Absolute test scores comparison

N 7

% 28

Absolute test scores, adding pre‐ test scores to the analysis

13 52

Difference scores Item accuracy

10 40 1 4

Gain/loss scores

Percentage of improvement

Error rates

Description Absolute pre‐test and post‐test scores of EG and CG are compared separately Absolute test scores of EG and CG are compared, taking the pre‐test scores into account Use of a specific scoring system

The number of points the gained/lost between the pre‐and post‐test Percentage of improvement between pre‐and post‐test

612

Example from studies reviewed …the independent samples t‐test was applied to examine whether the differences between the mean scores of the control and experimental groups in the pre‐test and post‐test were statistically significant (Yip and Kwan 2006) …pre‐test scores on the specific subject tested were introduced as covariates in order to control for initial levels of the ability’ (Rosas et al. 2003)

Each factor was rated ‐1 if performance changed from correct to incorrect, 0 if there was no change and +1 if it changed from incorrect to correct. Then, each subject’s scores were summed to create a summary difference score…(Coles et al. 2007) …paired‐samples t tests were conducted to compare the treatment and control gain scores from pre‐test to post‐test…(Kebritchi et al. 2010) …the percentage of improvement was calculated from the primary scores by subtracting the pre‐test result from the post‐test result and then dividing the difference by the maximum result of the test (Ketamo 2003)

Anissa All, Elena Patricia Nuñez Castellar and Jan Van Looy Data analysis Out‐of‐game error rates

N 1

% 4

Description Number of mistakes made in the pre‐ and post‐test are compared

In‐game error rates

In game measurement of number of errors during gameplay are compared

Example from studies reviewed the educational effect…by comparing the number of mistakes of the students of the VR‐ENGAGE sub‐ groups with the number of mistakes of the students of the respective sub‐groups that had used the simple ITS (Virvou et al. 2005) …the position and location of the mouse onscreen were recorded every 10th second. How successful children were at solving the computer assignments immediately or after one or more repetitions can be derived from these registrations (Van Der Kooy‐ Hofland et al. 2012)

4. Discussion The results of the present study show that studies vary on different dimensions of the study design, presenting a heterogeneity of methodologies. Homogeneity is, however, an important prerequisite when conducting meta‐analyses, impeding generalizing conclusions on the effectiveness of DGBL (Higgins et al. 2008). Differences were not only found between study designs, but also in reporting. Regarding the participants dimension, several studies only used certain subgroups (e.g. certain ability, certain socioeconomic status). This does logically narrow results to this specific subgroup (Campbell et al. 1963), which is problematic, however, when generalizing claims on DGBL effectiveness are made. Therefore, reporting on inclusion criteria for recruitment and how sampling occurred, is essential. When considering the intervention dimension, studies firstly differed on the type of intervention implemented in the CG. The interpretation of the contribution of the intervention to the EG does, however, depend on the activities performed in the CG (Campbell et al. 1963). Considering that intervention in the CG can influence results and interventions implemented in CG differed across studies, comparison between results becomes problematic. Secondly, implementation of DGBL in the EG differed between studies, either implementing them as a stand‐alone intervention or in a larger program. When embedded in a program, elements of the program differed across studies as well (e.g. introduction, debriefing, extra material, required reading, etc.). The addition of other elements to the intervention can result in multiple treatment interference (Campbell et al. 1963), however, meaning the achievement gain might not be solely attributable to DGBL, but could be influenced by other activities that are part of the intervention. Results of studies implementing only DGBL and studies implementing DGBL in a larger program are thus not comparable. Thirdly, implementation of DGBL differed by the presence of an intermediary and the role of this intermediary. Most studies did not report on whether or not an intermediary was present. When an intermediary was present, they were either present to solely supervise or were present with the purpose of providing procedural help and/or guidance during gameplay. The role of the intermediary was either filled by the classes’ teachers or a researcher. Who the intermediary is (e.g. someone more familiar such as a teacher or a total stranger) and how he or she interacts with the participants is a potential confound when assessing the effect of the DGBL intervention (Leary 1995). While all studies implemented an experimental design, differences were found in the participants’ assignment to the EG and CG which was done with or without randomization or by ‘matching’ in order to attain similarity of both groups. It was, however, not clear whether this matching occurred randomly. When using matched random assignment, the participants’ scores on a measure of relevance (for example: pre‐test) are obtained in order to randomly assign participants belonging to a certain level to the conditions (Leary 1995). According to Campbell & Stanley (1963), matching is not a preferable method. In the context of educational research, randomization of the classroom as a unit is more preferable, because classrooms can then be classified for analysis on the basis of factors such as schools, teacher, subject, time of day, mean intelligence level, etc. (Campbell et al. 1963). According to Leary (1995), however, randomization of schools will jeopardize internal validity, as groups will likely differ on multiple dimensions. This is an issue that merits further discussion, considering randomized controlled trials are difficult to implement with small sample sizes, which are often a reality in DGBL effectiveness studies. How effectiveness was measured, also differed between studies, complementing learning outcome measures with affective measures and time measures. Some studies reported very specifically on how scoring on tests occurred in. According to Campell & Stanley (1963) different instruments and different scorers, can yield other

613

Anissa All, Elena Patricia Nuñez Castellar and Jan Van Looy results. Every measuring tool also entails a certain measurement error, fuelled by transient states (e.g. participants’ mood, level of fatigue), stable attributes (e.g. misunderstanding questions, individual differences in motivation), situational factors and characteristics of the measurement itself (e.g. ambiguous questions, test that induce fatigue). Incomplete information on and differences between the length of certain tests, formulating of questions or the time of measurement account for an incomparability of results across studies as well. Use of independently developed standardized tests could provide more ‘stable’ measurements and create comparability across studies. This is, however, a difficult exercise, considering the wide range of topics covered by DGBL. Although, standardized tests could be implemented to assess affective learning outcomes, such as motivation. While the larger part of the studies implemented the exact same test pre‐ and post‐intervention, others changed the sequence of the questions. Implementing the same questions in the pre‐and post‐test can however lead to a test‐retest practice effect (Campbell et al. 1963). According to Crawford et al. (1989) this is due to retention of specific test material by the participants. Other studies used similar tests, meaning these consisted of questions of the same type and difficulty level. While practice effects can still occur using a parallel version of a test on different points in time (e.g. pre‐ and post‐test), these generally tend to be smaller (Anastasi 1961). Certain studies also used dissimilar tests, when for example student achievement in school (e.g. exam scores) was used as a measure. This seems problematic, considering assumptions on the comparability of both tests cannot be made, making any significant achievement gains possibly invalid. Differences in similarity of pre‐and post‐test across studies, is another reason why it is difficult to compare results across studies. Important differences were also found when considering data analysis techniques. While indeed several analyses can be used to measure the effect of the intervention, an analysis of covariance (ANCOVA) with pre‐ test scores as a covariate, is a more preferable method (Campbell et al. 1963). In the context of randomized controlled trials, ANCOVA reduces error variance and in the context of nonrandomized designs, it adjusts mean scores of the post‐test to differences between groups on pre‐test scores (Dimitrov and Rumrill 2003). Furthermore, missing information on implementation of the intervention(s) impedes replication of certain studies, which is a basic principle of empirical research in order create the opportunity to falsify obtained results (Popper 2000). Finally, incomplete information on sampling, similarity between interventions of the EG and CG and similarity between pre‐and post‐tests, puts validity of certain results in doubt. In general, we can conclude that comparisons between studies are problematic as a result of heterogeneity in study designs, heterogeneity in reporting, incomplete information and biased results, impeding generalization. Standardized guidelines on sampling, activities in control groups, implementation of DGBL in the experimental group, measures, scoring, analysis and reporting on these elements could contribute to homogeneity in the research field and create insight in the validity of studies.

5. Limitations and further research The selection and coding of publications was conducted by one researcher, which can be considered a limitation of this study. This study also is limited to digital games aimed at cognitive learning outcomes. Further research should thus be conducted on methodologies used in digital games aimed at skill acquisition and behavioural or attitudinal change. An interesting venue for future research is exploring the possibilities for the development of a standardized procedure to measure effectiveness of DGBL. Relevant issues to investigate in this context are gathering input from experts in the methodology field in order to detect preferable methods for measuring learning effectiveness (e.g. number of control groups, activity in control group, implementation of DGBL, implementation period, etc.). Further, such a procedure should be adjusted to the requirements of the people who would benefit from this procedure and actually use the procedure. Therefore, involvement of relevant stakeholders in the process of developing the procedure is desirable.

Acknowledgements This PhD project is funded by IWT, the Flemish government agency for Innovation by Science and Technology (IWT).

614

Anissa All, Elena Patricia Nuñez Castellar and Jan Van Looy

Appendix 1: Studies included in review Anderson, J. and Barnett, M. 2010. Using Video Games to Support Pre‐Service Elementary Teachers Learning of Basic Physics Principles. Journal of Science Education and Technology, 20(4), 347‐362. Bai, H., et al. 2012. Assessing the effectiveness of a 3‐D instructional game on improving mathematics achievement and motivation of middle school students. British Journal of Educational Technology, 43(6), 993‐1003. Coles, C. D., et al. 2007. Games that "work": using computer games to teach alcohol‐affected children about fire and street safety. Res Dev Disabil, 28(5), 518‐530. Din, F. S. and calao, J. 2001. The effects of playing educational video games in kindergarten achievement. . Child Study Journal, 31(2), 95‐102. Kajamies, A., Vauras, M. and Kinnunen, R. 2010. Instructing Low‐Achievers in Mathematical Word Problem Solving. Scandinavian Journal of Educational Research, 54(4), 335‐355. Kanthan, R. and Senger, J.‐L. 2011. The Impact of Specially Designed Digital Games‐Based Learning in Undergraduate Pathology and Medical Education. The Impact of Specially Designed Digital Games‐Based Learning in Undergraduate Pathology and Medical Education, 135, 135‐142. Ke, F. 2008. Computer games application within alternative classroom goal structures: cognitive, metacognitive, and affective evaluation. Educational Technology Research and Development, 56(5‐6), 539‐556. Kebritchi, M., Hirumi, A. and Bai, H. 2010. The effects of modern mathematics computer games on mathematics achievement and class motivation. Computers & Education, 55(2), 427‐443. Ketamo, H. 2003. An Adaptive Geometry Game for Handheld Devices. Educational Technology & Society, 6(1), 83‐94. Lorant‐Royer, S., et al. 2010. Kawashima vs “Super Mario”! Should a game be serious in order to stimulate cognitive aptitudes? Revue Européenne de Psychologie Appliquée/European Review of Applied Psychology, 60(4), 221‐232. Miller, D. J. and Robertson, D. P. 2010. Using a games console in the primary classroom: Effects of ‘Brain Training’ programme on computation and self‐esteem. British Journal of Educational Technology, 41(2), 242‐255. Miller, D. J. and Robertson, D. P. 2011. Educational benefits of using game consoles in a primary classroom: A randomised controlled trial. British Journal of Educational Technology, 42(5), 850‐864. Moreno, J. 2012. Digital Competition Game to Improve Programming Skills. Educational Technology & Society, 15(3), 288‐ 297. Moshirnia, A. 2007. The Educational Potential of Modified Video Games. Issues in Informing Science and Information Technology, 4, 511‐521. Papastergiou, M. 2009. Digital Game‐Based Learning in high school Computer Science education: Impact on educational effectiveness and student motivation. Computers & Education, 52(1), 1‐12. Parchman, S. W., et al. 2000. An Evaluation of Three Computer‐Based Instructional Strategies in Basic Electricity and Electronics Training. . Military Psychology, 12(1), 73‐87. Poli, D., et al. 2012. Bringing Evolution to a Technological Generation: A Case Study with the Video Game SPORE. The American Biology Teacher, 74(2), 100‐103. Rastegarpour, H. and Marashi, P. 2012. The effect of card games and computer games on learning of chemistry concepts. Procedia ‐ Social and Behavioral Sciences, 31, 597‐601. Rosas, R., et al. 2003. Beyond Nintendo: design and assessment of educational video games for first and second grade students. Computers & Education, 40, 71‐94. St Clair‐Thompson, H., et al. 2010. Improving children's working memory and classroom performance. Educational Psychology, 30(2), 203‐219. Suh, S., Kim, S. W. and Kim, N. J. 2010. Effectiveness of MMORPG‐based instruction in elementary English education in Korea. Journal of Computer Assisted Learning, 26(5), 370‐378. Van der Kooy‐Hofland, V. A., Bus, A. G. and Roskos, K. 2012. Effects of a brief but intensive remedial computer intervention in a sub‐sample of kindergartners with early literacy delays. Read Writ, 25(7), 1479‐1497. Virvou, M., Katsionis, G. and Manos, K. 2005. Combining Software Games with Education: Evaluation of its Educational Effectiveness. Educational Technology & Society, 8(2), 54‐65. Yang, Y.‐T. C. 2012. Building virtual cities, inspiring intelligent citizens: Digital games for developing students’ problem solving and learning motivation. Computers & Education, 59(2), 365‐377. Yip, F. W. M. and Kwan, A. C. M. 2006. Online vocabulary games as a tool for teaching and learning English vocabulary. Educational Media International, 43(3), 233‐249.

References Anastasi, A., 1961. Differential psychology: Individual and group differences in behavior. Macmillan. Bai, H., et al. 2012. Assessing the effectiveness of a 3‐D instructional game on improving mathematics achievement and motivation of middle school students. British Journal of Educational Technology, 43(6), 993‐1003. Baranowski, T., et al. 2008. Playing for real: video games and stories for health‐related behavior change. American journal of preventive medicine, 34(1), 74. Bleumer, L., et al., 2012. State of play of digital games for empowerment and inclusion: a review of the literature and empirical cases. Spain. Campbell, D. T., Stanley, J. C. and Gage, N. L., 1963. Experimental and quasi‐experimental designs for research. Houghton Mifflin Boston.

615

Anissa All, Elena Patricia Nuñez Castellar and Jan Van Looy Clark, R. E. 1994. Media will never influence learning. Educational Technology Research and Development, 42(2), 21‐29. Connolly, T. M., et al. 2012. A systematic literature review of empirical evidence on computer games and serious games. Computers & Education. Corsi, T. M., et al. 2006. The real‐time global supply chain game: New educational tool for developing supply chain management professionals. Transportation Journal, 61‐73. Crawford, J., Stewart, L. and Moore, J. 1989. Demonstration of savings on the AVLT and development of a parallel form. Journal of Clinical and Experimental Neuropsychology, 11(6), 975‐981. Dimitrov, D. M. and Rumrill, J., Phillip D 2003. Pretest‐posttest designs and measurement of change. Work: A Journal of Prevention, Assessment and Rehabilitation, 20(2), 159‐165. Gagne, R. M. 1984. Learning outcomes and their effects: Useful categories of human performance. American Psychologist, 39(4), 377. Garris, R., Ahlers, R. and Driskell, J. E. 2002. Games, Motivation, and Learning: A Research and Practice Model. Simulation & Gaming, 33(4), 441‐467. Glaser, B. G. and Strauss, A. L., 2009. The discovery of grounded theory: Strategies for qualitative research. Transaction Books. Hainey, T., 2010. Using Games‐Based Learning to Teach Requirements Collection and Analysis at Tertiary Education Level. Higgins, J. P., Green, S. and Collaboration, C., 2008. Cochrane handbook for systematic reviews of interventions. Wiley Online Library. Jenkins, H., 2009. Confronting the challenges of participatory culture: Media education for the 21st century. The MIT Press. Joy, E. H. and Garcia, F. E. 2000. Measuring Learning Effectiveness: A New Look at No‐Significant‐Difference Findings. JALN, 4(1), 33‐39. Kharrazi, H., et al. 2012. A Scoping Review of Health Game Research: Past, Present, and Future. Games for Health Journal, 1(2), 153‐164. Kozma, R. B. 1994. Will Media Influence Learning? Reframing the Debate. . Educational Technology Research and Development, 42(2), 7‐19. Leary, M. R., 1995. Introduction to behavioral research methods. Brooks/Cole Pacific Grove, CA. Michael, D. R. and Chen, S. L., 2005. Serious games: Games that educate, train, and inform. Muska & Lipman/Premier‐ Trade. Neys, J., et al., Poverty is not a game: behavioral changes and long term effects after playing PING. ed. 13th annual conference on the International Speech Communication Association, 2012 Portland. Popper, K. 2000. Science: conjectures and refutations. Readings in the Philosophy of Science: From Positivism to Postmodernism, 9‐13. Renkl, A., Mandl, H. and Gruber, H. 1996. Inert knowledge: Analyses and remedies. Educational Psychologist, 31(2), 115‐ 121. Ritterfeld, U., Cody, M. and Vorderer, P., 2009. Serious games: mechanisms and effects. New York: Routledge. Salomon, G. 1979. No distribution without individuals' cognition: a dynamic interactional view. Sawyer, B. and Smith, P. 2008. Taxonomy for Serious Games. Digitalmil, Inc& Serious Games Initiative/Univ. of Central Florida, RETRO Lab. Shute, V. J., Rieber, L. and Van Eck, R. 2011. Games... and... learning. Trends and issues in instructional design and technology, 3. Whitehead, A. N. 1959. The aims of education. Daedalus, 88(1), 192‐205. Yip, F. W. M. and Kwan, A. C. M. 2006. Online vocabulary games as a tool for teaching and learning English vocabulary. Educational Media International, 43(3), 233‐249.

616

Investigating Collaborative Games to Teach Mathematics‐Based Problem Solving in the Classroom Reem Al‐Washmi, Gail Hopkins and Peter Blanchfield School of Computer Science, University of Nottingham, Nottingham, UK psxra6@nottingham.ac.uk gail.hopkins@nottingham.ac.uk peter.blanchfield@nottingham.ac.uk Abstract: Learning mathematics, and mathematical problem solving in particular, is considered a challenge for some pupils at primary schools. When solving mathematical problems several skills are needed and commonly some children face difficulties in areas like memory, language and spatial aspects. This PhD research aims to investigate whether collaborative computer games can support and motivate UK Key Stage 2 pupils in fostering better understanding of mathematical problem solving. As part of achieving this overall aim, two initial studies are reported in this paper. The research described here has taken a user‐centred design approach and has been two‐fold. Firstly, an initial study was conducted to gain an understanding of the existing teaching environment, methods used and problem solving experiences of both children and teachers in a Key Stage 2 teaching environment. The investigation involved observations of problem solving classes and questionnaires which sought to explore mathematics problem solving difficulties and the role of collaborative games in the actual learning environment. The findings from this initial study indicate that games and collaborative working in mathematics classes could be significant in supporting and enhancing Key Stage 2 pupils in their learning of mathematical problem solving. Therefore it is suggested that collaborative mathematical games could be effective instructional tools for enhancing learning and understanding of mathematics concepts, arithmetic and problem solving. The use of collaboration within such a tool may promote the sharing and discussion of problems and so may be helpful for students who have difficulty in mathematics. The second study aimed to investigate whether and how Key Stage 2 children engaged in a TM commercially available collaborative problem solving computer game, Lego Harry Potter and to examine their levels of collaboration and motivation within the game. The purpose of this study was to gather requirements for a future collaborative computer game designed to teach mathematics problem solving. A mixed‐methods approach of observation and questionnaires has been used. Key Stage 2 pupils were observed during collaborative computer game play and an analysis of their behaviour and interaction was conducted in order to identify the ways in which they collaborated and the elements of the game which promoted motivation and collaboration. Questionnaires were used to investigate elements of the game play which encouraged engagement and collaboration. The findings from the first study show that some, but not all, children found mathematical problem solving challenging but that collaboration and game playing is used and enjoyed during their classroom‐based learning of this. The second study showed that the collaborative computer game did engage the players. Elements of role play, suspense and mystery encouraged motivation and collaboration but there were issues at times with some players not knowing what to do in the game and at other times with providing enough challenge for two collaborating players. Resulting requirements gathered from this include the necessity for clear rules and goals, a playful environment with challenge, mystery and fantasy and a story line that flows through the game. Keywords: games‐based learning, collaborative learning, mathematics, problem solving

1. Introduction Mathematics in primary schools and problem solving in particular are considered challenging for some pupils (Tay Lay Heong 2005). Wu et al. (2009) confirmed that the most common difficulties that students encountered in mathematics problem solving are a lack of confidence, weakness in understanding the terms and mathematical language, confusion in determining mathematical operations and lack of conceptual and procedural skills in problem solving. There is significant research evidence that collaboration between peers during mathematics lessons can facilitate active learning among children and can be effective in improving children’s mathematical abilities (e.g, Artut and Tarim, 2007; Tarim and Akdeniz, 2008). Furthermore, many researchers have indicated that computer games can motivate children to learn mathematics and can deliver good support for learning (Quinn, 1994; Betz, 1995; Moreno, 2002; Liu and Chu, 2010; Charsky and Ressler, 2011). The overall research aim of this PhD is to investigate whether collaboration in game play can support and motivate UK Key Stage 2 (7‐11 years) pupils in fostering better understanding of mathematical problem solving. Initial research towards this aim is reported here which is two‐fold: firstly a classroom‐based study is reported which examines activities and problems that occur in mathematics classrooms and to study how

617

Reem Al‐Washmi, Gail Hopkins and Peter Blanchfield games and collaboration are used in the classroom. Secondly, observations of pupils using a collaborative computer game were conducted in order to investigate the extent to which collaboration and motivation to play were present. Ultimately, the aim of this PhD is to build and test a collaborative game which is designed to teach mathematical problem solving to UK Key Stage 2 children, in part based on the findings of the studies reported in this paper.

2. Literature review In schools, learning mathematics can be a major challenge for students because it depends on several skills such as the ability to connect information to a concept and to transform problems into mathematical sentences (Tambychik et al. 2010). Problem solving is considered as being at the heart of mathematics (Cockroft 1982). It is a complex process which can involve many sub‐problems (NCTM 2000) and where the target is obvious but not easy to reach Altun (2005). Pupils’ failures at solving problems are not often the result of a lack of mathematical knowledge but an indication of the ineffective use of their knowledge (Harskamp and Suhre 2007). Cooperative learning techniques in the classroom have been shown to encourage interaction between children (Gillies, 2006) and research also suggests that collaboration can benefit children in terms of achievement (Webb et al., 2009). In the past, teaching mathematics was often based on traditional teacher‐centred methods. However, more recently the importance of collaboration in contributing to the development of pupils’ mathematical thinking and problem solving abilities has been stressed (Johnson and Johnson 1989; Lohman and Finkelstein 2000). Serious games in the field of education are widespread and offer an inspiring area of research into instructional tools which can be used to enhance learning (Egenfeldt‐Nielsen 2005; Ke and Grabowski 2007; Coller and Scott 2009) and problem solving (Pivec and Pivec 2008). Educators believe that children learn best during play and that games give them that opportunity (Gee 2003; Morgan and Kennewell 2005). Using technological devices such as computers in the classroom has also proven to be effective in achieving motivation (Ke 2008), collaboration (Cortez et al. 2009) and learning results (Carbonaro et al. 2008; Plass et al. 2010a). Collaborative game play activities which have mathematical principles and problems embedded in them can offer special opportunities for mathematics investigations (Gee 2004; Childress and Braswell 2006; Mikropoulos 2006). Harnessing educational technologies like games in mathematics learning can make the learning process more effective (Mor et al. 2004; Simpson et al. 2006) and can present academic subjects in a more learner‐centred, accessible, enjoyable and thus, more effective manner (Malone 1980; Kafai 2001; Prensky 2003).

3. Experimental design This research was conducted over a period of two years and took place initially in the classroom where observations and questionnaires (of both teacher and pupils) were used to determine pupils’ attitudes to mathematics problem solving, collaboration and games within the classroom. Subsequently a games‐based study was conducted whereby pupils were observed playing a commercial computer game, LegoTM Harry Potter, in order to study pupils’ behaviours in terms of motivation to play the game and collaboration within the game. Again, questionnaires were used to support these observations.

3.1 The classroom study 3.1.1 Methods Observations were conducted on 31 Year 5 pupils (7‐10 years old) at a local primary school. Observations took place twice a week for a period of 4 months. Observations focussed on the types of skills used and activities performed in the mathematics classes and, in particular, looked at the difficulties experienced with mathematics and especially problem solving topics, and the use of collaboration and games to overcome these.

618

Reem Al‐Washmi, Gail Hopkins and Peter Blanchfield At the end of the observation two questionnaires were distributed; one for pupils and one for teachers. The pupil questionnaire was completed by 23 respondents. This aimed to identify students’ attitudes towards mathematics learning and aspects of collaboration. The student’s questionnaire had fifteen closed questions, each with answers represented as a "smiley face", "sad face" and "neutral face". Pupils were given help with reading and understanding the questions. Four teachers completed a questionnaire which aimed to investigate their views of the mathematics learning environment with regard to pupils’ skills, activities and difficulties during classes, especially with problem solving topics. The teacher’s questionnaire included twelve open ended questions which asked specifically about problems in mathematics classes, problem solving topics and use of mathematical software, games and collaboration. 3.1.2 Results Based on the observations in the classroom it was evident that pupils varied in their approaches to learning mathematics. Some engaged much more readily than others who were observed to become bored and to work slowly. Some children clearly lacked confidence as was evidenced by their unwillingness to answer questions in class even when they knew the right answer. However, encouragement and rewards such as stickers helped somewhat to overcome this. Some children were able to give correct answers but could not explain their reasoning. Collaboration between children varied; whilst children worked more slowly and were less motivated working alone, some were reticent in discussing answers and appeared more competitive. It was observed that different kinds of number games were used in the classroom in a collaborative manner. For example, one game involved the children standing in a circle taking turns to count forwards or backwards by 10 and they appeared to respond well to this. Other solutions included the use of pictograms with related questions which the children found enjoyable. The pupils’ questionnaire results are given in Table 1 and the qualitative data from the teachers’ questionnaires are presented in the text below. It can be seen from the results that out of 23 respondents more than half the children thought they were good at maths and more than half enjoy maths, although five children acknowledged difficulty in problem solving. The majority of pupils stated that they found mathematics exercises interesting and fun and results also indicated that they were strongly in favour of game playing in the classroom. Collaboration was viewed favourably (n=19) but less than half like to share their answers with friends. Table 1: The classroom study ‐ pupils’ questionnaire results Questions

Yes

Don’t know

Do you enjoy mathematics classes?

Do you find mathematics exercises interesting and fun? In your opinion are you good at maths? Do you find problem solving sessions difficult? Do you play any games in the classroom that involve maths? Does playing the games help you understand maths better? Do you use any computer software in maths classes? If you had this software at home would you use it? Do you like asking your teacher when you are not sure about your solution? Do you look at corrections in your maths work and try them again? Would you like a computer game that helps you to understand maths better? Do you like to discuss your answers with your friends? Do you enjoy to work together with your friends to solve maths exercises? Do you like to share your answers with your friends? Do you think working with your friends will help you to get better at problem solving?

18 12 5 15 13 12 20

0 2 6 3 2 4 2

5 9 12 5 0 7 1

619

Reem Al‐Washmi, Gail Hopkins and Peter Blanchfield Teachers' answers regarding problem solving difficulties confirmed that there are several issues which contribute to this. They reported that problem solving is a difficult subject in Key Stage 2 as there is often no set way of solving a problem. Moreover, different skills must be taught to children and they must choose the most appropriate way to solve the problem by themselves. In addition, language can also be an issue with problem solving. For example, children can find the language of problem solving exercises difficult to understand. Furthermore, children can also struggle to explain what they have done and how they found a solution. Teachers indicated that involving different kinds of games in the classroom engaged pupils in problem solving. They also mentioned that some students need physical interaction for their learning, so games would help them. 3.1.3 Discussion The observation of pupils in the classroom and the questionnaire results indicated that mathematics is a difficult subject for some pupils and this was especially so with problem solving. However, it should be noted that not all children found mathematics challenging. This leaves the teacher with the task of teaching children at different ends of a spectrum of ability, and we propose that this provides a strong argument for the role of collaboration between pupils. Even when able to give the correct answer to a problem, often children could not explain in words how they achieved that answer, but during collaborative sessions children were given the opportunity to work through this with each other and to learn to express how they achieved their answers, and many children were observed to respond well to these sessions. According to Flick and Bell (2000), science involves a lot of difficult concepts which can be hard for primary school pupils to understand. These difficulties can be related to differences in the degree of idea complexity, abstract topics and the lack of pupils’ experiences. Pupils often face difficulties in mathematics especially in mathematics problem solving (Tay Lay Heong 2005). Underwood et al. (2000) showed that the performance of children who work collaboratively will be higher than children working individually. Additionally collaboration can benefit children of different ages (Hogan and Tudge 1999), mixed ability groups (Garton and Pratt 2001) and groups with children who have different levels of confidence (Tudge et al. 1996), motivation (Gabriele and Montecinos 2001) and are of different gender (Strough et al. 2001). The observation results gave a clear view that using different kinds of games can create an environment for learning where challenge and motivation of pupils are increased, especially for those who tend to get bored with traditional methods of instruction. This is confirmed by research suggesting games offer engagement for long periods of time (Lenhart and Kahne 2008). The teachers’ questionnaire results showed that physical interaction is important for some children and games provide this physical involvement. Further to this, the results showed that there is no set method for solving a problem and that different skills are needed including the ability to understand the language of problem solving. A games‐based environment could allow pupils to try out solutions more readily and more imaginatively and to engage with the problem rather than engaging with the language. In light of this study, collaboration is considered as key for achievement in problem solving tasks due to the activities that can be involved within the collaboration. Children like to interact and communicate with each other in order to share information and ask for advice. Moreover, collaboration among children will allow them to share ideas and learn from each other. Also, we propose that combining collaboration with computer game play can provide an effective environment within which children can physically interact to collaborate, share ideas and help each other to solve problems. As such, the second study reported in the next section aimed to investigate the extent to which such a computer game could encourage collaboration and motivation.

3.2 The game‐based study 3.2.1 Methods A mixed‐methods study was conducted of children playing LegoTM Harry Potter at two different primary schools. The purpose of this was to examine collaboration during game‐play and more specifically, communication and interaction and how these contributed to motivate pupils to complete tasks and reach desired goals within the game.

620

Reem Al‐Washmi, Gail Hopkins and Peter Blanchfield Collaboration within games is considered a key component as it can boost motivation for learning while enhancing game performance (Wood et al., 1976; Walqui, 2006; Luckin, 2008). As such, observations focussed on collaboration and motivation during game play. Twenty seven groups, each consisting of two children from Key Stage 2, were observed playing the game for 25 minutes. The children’s ages ranged from 7 to 10 years old and groups consisted of male, female and mixed gender. Observations were coded according to collaboration and motivation and the codes and their frequencies are shown in Table 2. A further 30 children, aged between 8 and 10 years old, played the game in pairs for 25 minutes before completing a questionnaire. Again, groups consisted of male, female and mixed gender. The aim of the questionnaire was to investigate elements of the game play which encouraged motivation and collaboration. It also asked children about their fun whilst playing the game, whether they found it challenging or complex and whether they enjoyed collaborating during game play. Open ended questions asked them about why they gave a particular answer.

3.3 Results The results from the observations can be seen in Table 2 and Figure 1. Codes have been divided into those related to collaboration and motivation. Collaboration has been coded in terms of physical and verbal interaction. The results from the questionnaire study are shown in Table 3. Table 2 The game‐based study ‐ coded observations of game play Code

Meaning (Collaboration)

Collecting objects

Manipulating objects

Gesture

Joint manipulation

Conversation

Instruction

Sharing knowledge

Questioning

Disagreement

arg

Arguing

Sub‐category

Frequency

Code

Meaning (Motivation)

Sub‐ category

Frequency

Enthusiasm

_______

Physical movement

_______

Goal seeking

_______

Discovery

_______

Immersion

_______

Withholding information

_______

Competition

_______

Concentration

_______

Fun

_______

Aggression

_______

Challenge

Copying

Physical interaction Physical interaction Physical interaction Physical interaction Verbal interaction Verbal interaction Verbal interaction Verbal interaction Verbal interaction Verbal interaction

_______ _______

3 2

The results from the observations demonstrate that pupils definitely engaged in collaborative activity; in particular sharing knowledge, giving instructions and questioning each other, with a frequency of 32, 25 and 13 respectively. Pupils also spent a lot of time manipulating (19) and collecting (10) objects. Verbal interaction was high and there was clear evidence of collaboration in the dialogue that was recorded: Player 1: “I can’t do that” Player 2: “Because you need to listen to me” Player 1: “I did, but I still can’t do anything” Player 2: “No you did not because this is supposed to be done by helping”

621

Reem Al‐Washmi, Gail Hopkins and Peter Blanchfield

Figure 1: The game‐based study ‐ coded observations of game play In terms of their motivation to play the game, evidence of enthusiasm was high (22) and a fair amount of discovery was evident (12). This was also reflected in some of the dialogue: Player 3: “Is this Lego Harry Potter?” Researcher: “Yes” Player 3: “Ohh, I have never played it” Researcher: “Do you like to play?” Player 3: “Yes, of course” Researcher: “Why do you like Harry Potter Lego Game?” Player 3: “I like it because it’s funny” Researcher: “Why you think it’s funny?” Player 3: “Because you can smash things around you in the game and you can fly” It was noted that those children who were familiar with the Harry Potter character tended to have a better idea of how to play the game than those who weren’t and they were observed to be helping those less familiar with the game characters. It was also observed that the act of one child explaining a solution to another enabled them to vocalise their reasoning and helped their problem solving. Table 3: The game‐based study ‐ game play questionnaire results Questions

Answers & Gender Yes Don’t know No B G B G B G

Did you have fun playing the game?

Was the game complex?

Was the game challenging?

Did you feel that time passed quickly? Would you like to have more time playing the game? Did you find the game exciting? Do you prefer to play with your friends? Do you prefer to play with children of the same gender?

622

Reem Al‐Washmi, Gail Hopkins and Peter Blanchfield The questionnaire results showed clear evidence that children enjoyed playing LegoTM Harry Potter (n=29) and that they found the time passed quickly (n=26). All 30 children stated that they wanted to have more time playing the game. Twenty seven of the 30 children agreed that they preferred to play with friends. With respect to complexity and challenge, children were less certain, with only 5 agreeing that the game was challenging and only 2 finding it complex. Some did, however, state that they felt the hardest part was knowing what to do and that due to the complexity they needed instructions on how to play. Others stated that there was not enough challenge, that working together meant there was less challenge and that the game was too easy. Another problem was that children did not like the split screen, which appeared when the two players moved away from each other in the game. They stated that they preferred to have their own view of the game world and not to see both players’ views simultaneously. In terms of genre, pupils found the elements of adventure and fantasy very engaging and different characters within the game through which they could role play were important. 3.3.1 Discussion The children collaborated well with a high level of verbal interaction and were observed to be motivated to play the game. This level of engagement is something which is important to harness in an educational game that aims to teach problem solving. In terms of complexity and challenge, there was evidence that this was lacking for some children but that others just didn’t have enough instruction in how to play the game and this is something which needs careful consideration in the design of a future maths‐based educational game. Challenge and complexity need to be incorporated in such a game to engage children, but children also need to understand each stage of the game and its goals and requirements and be able to progress through the game. Therefore, the presence of clear goals and well‐structured rules for playing the game are considered as important requirements to allow progression through the game and the level of challenge needs to be such that children do not find the game too easy when collaborating with others. Also, we propose a fantasy genre for our future maths‐based educational game, which incorporates mystery and suspense and allows the players to take on the role of different characters to achieve their goals. The proposed collaborative game will be networked and provide players with their own separate views of the game world.

4. Conclusion and future work The research reported in this paper demonstrates that using collaborative games could play a significant part in teaching mathematics‐based problem solving in the classroom. Computer games can offer an environment which is motivating and allows collaborative learning. However, evidence shows that children prefer to choose computer games which are engaging more than games which are specifically educational (Facer 2003; Kerawalla and Crook 2002; Kinzie and Joseph 2008) and this engagement is key to a game’s success. The design and development of the teaching game has begun. The research findings reported in this paper form the basis for the design of the game which will be a fantasy, role‐playing game which includes the collaborative problem solving approach. This has been shown to provide a good basis for getting students to explore how to solve problems and will increase the motivation of those less willing to engage with this process. It will also allow those who understand how to solve problems to be able to express this process more clearly to others.

Acknowledgements The researchers would like to thank the staff and children of Southwold Primary School, Nottingham and The Kabin After School Club, All Hallows Primary School, Nottingham for taking part in this study. This research is supported by the Ministry of Education in Saudi Arabia.

References Altun, M. (2005) Teaching Mathematics, Erkam Press, Bursa. Artut, P. D., and Tarim, K. (2007) Effectiveness of jigsaw II on prospective elementary school teachers, Asia‐Pacific Journal of Teacher Education, Vol 35, No.2, pp 129–141. Betz, J. A. (1995) Computer games: increases learning in an interactive multidisciplinary environment, Journal of Educational Technology Systems, Vol 24, No.2, pp 195–205. Carbonaro, M., Cutumisu, M., Duff, H., Gillis, S., Onuczko, C., and Siegel, J. (2008) Interactive story authoring: a viable form of creative expression for the classroom, Computers & Education, Vol 51, No.2, pp 687–707.

623

Reem Al‐Washmi, Gail Hopkins and Peter Blanchfield Charsky, D., and Ressler,W. (2011) Games are made for fun: lessons on the effects of concept maps in the classroom of computer games, Computers & Education, Vol 56, No.3, pp 604–615. Childress, M. D. and Braswell, R. (2006) Using Massively Multiplayer Online Role‐Playing Games for Online Learning, Distance Education, Vol 27, No.2, pp 187‐196. Cockroft, W. H. (1982) Mathematics Counts: Report of the Committee of Enquiry into the Teaching of Mathematics in Schools, London: HMSO. Coller, B. D., and Scott, M. J. (2009) Effectiveness of using a video game to teach a course in mechanical engineering, Computers & Education, Vol 53, No.3, pp 900‐912. Cortez, C., Nussbaum, M., Woywood, G., and Aravena, R. (2009) learning to collaborate by collaborating: a face‐to‐face collaborative activity for measuring and learning basics about teamwork, Journal of Computer Assisted Learning, Vol 25, No. 2, pp 126–142. Egenfeldt ‐ Nielsen, S. (2005) Beyond Edutainment: Exploring the Educational Potential of Computer Games, PhD thesis, IT University of Copenhagen. Facer, K. (2003) Screenplay: Children and computing in the home. London: RoutledgeFalmer. Gabriele, A. J. and Montecinos, C. (2001) Collaborating With a Skilled Peer: The Influence of Achievement Goals and Perceptions of Partners' Competence on the Participation and Learning of Low‐Achieving Students, The Journal of Experimental Education, Vol 69, pp 152‐178. Garton, A. F. and Pratt, C. (2001) Peer assistance in children's problem solving, British Journal of Developmental Psychology, Vol 19, pp 307‐318. Gee, J. (2003) What Video Games Have to Teach Us About Learning and Literacy, New York: Longman. Gee, J. (2004) Situated Language and Learning: a critique of traditional schooling, London: Routledge. Gillies, R.M., and Boyle, M. (2006) Ten Australian elementary teachers’ discourse and reported pedagogical practices during cooperative learning, The Elementary School Journal, Vol106,No.5 , pp 429–451. Harskamp, E., and Suhre, C. (2007) Schoenfeld’s problem solving theory in a student controlled learning environment, Computers and Education,Vol 49, No. 3, pp 822–839. Johnson, D. W., and Johnson, R. (1989) Cooperation and competition: Theory and research. Edina, MN: Interaction Book Company. Kafai, Y. B. (2001) “The educational potential of electronic games: from games‐to‐teach to games‐to‐learn”, [online], Cultural Policy Center, University of Chicago, http://culturalpolicy.uchicago.edu/conf2001/papers/kafai.html. Ke, F. and Grabowski, B. (2007) Game playing for maths learning: cooperative or not?, British Journal of Educational Technology, Vol 38, No.2 , pp 249‐259. Ke, F., and Hoadley, C. (2008) Computer games application within alternative classroom goal structures: cognitive, metacognitive and affective evaluation, Educational Technology Research and Development, Vol 56, No. 5, pp 539‐ 556. Kerawalla, L., and Crook, C. (2002) Children’s computer use at home and at school: Context and continuity, British Educational Research Journal, Vol 28, No6, pp 751–771. Kinzie, M. B., and Joseph, D. R. D. (2008) Gender differences in game activity preferences of middle school children: Implications for educational game design, Educational Technology Research & Development, Vol 56, No. 5/6, pp 643– 663. Lenhart, A., Kahne, J., Middaugh, E., Alexandra, M., Evans, C. and Vitak, J. (2008) Teens Video Games and Civics, Research study conducted for the PEW Internet and American Life Project . Liu, T. Y., and Chu, Y. L. (2010) Using ubiquitous games in an English listening and speaking course: impact on learning outcomes and motivation, Computers & Education, Vol 55, No. 2, pp 630–643. Lohman, M., and Finkelstein, M. (2000) Designing groups in problem‐based learning to promote problem solving skill and self‐directedness, Instructional Science, Vol 28, No. 4, pp 291‐307. Luckin, R. (2008) The learner centric ecology of resources: a framework for using technology to scaffold learning, Computers and Education, Vol 50, No. 2, pp 449–462. Malone, T. (1980) What makes things fun to learn?, Heuristics for designing instructional computer games, Palo Alto, USA. Mikropoulos, T. (2006) Presence: a unique characteristic in educational virtual environments, Virtual Reality, Vol 10, No.3, pp 197‐206. Mor, Y., Hoyles, C., Kahn, K., Noss, R. and Simpson, G. (2004) Thinking in Progress, Micromath, Vol 20, No. 2, pp 17‐23. Moreno, R. (2002) Who learns best with multiple representations? cognitive theory implications for individual differences in multimedia learning. Paper presented at World Conference on Educational Multimedia, Hypermedia, & Telecommunications. Denver, CO. Morgan, A. and Kennewell, S. (2005) The Impact of Prior Technological Experiences on Children’s Ability to Use Play as a Medium for Developing Capability with New ICT Tools. Becta: ICT Research Bursaries. NCTM (2000) Principles and standards for school mathematics. Reston, VA: NCTM Publications. Pivec, M. , and Pivec, P.(2008) Games in Schools. Learning. ISFE‐EUN Partnership. Retrieved from http://games.eun.org. Plass, J.L., Homer, B.D., Milne, C., Jordan, T., Ng, F., and Schwartz, R. (2010a) An Efficacy study on the integration of Optimized simulations into the high school Chemistry curriculum. Paper presented at the 2010 Annual Meeting for the American educational research Association (AERA), April 2010 in Denver, CO. Prensky, M. (2003) Digital game‐based learning. ACM Computers in Entertainment, Vol1, No.1,pp 1–4.

624

Reem Al‐Washmi, Gail Hopkins and Peter Blanchfield Quinn, C.N. (1994). Designing educational computer games. In K. Beattie, C. McNaught, and S. Wills (Eds.), Interactive multimedia in university education: Designing for change, Ambserdam: Elsevier, pp 45‐57. Simpson, G. Hoyles, C., and Noss, R. (2006) Exploring the mathematics of motion through construction and collaboration, Journal of Computer Assisted Learning, Vol 22, pp 1‐23. Strough, J., Berg, C. A. and Meegan, S. P. (2001) Friendship and Gender Differences in Task and Social Interpretations of Peer Collaborative Problem Solving, Social Development, Vol 10, pp 1‐22. Tambychik, T., Meerah, T. S. M., and Aziz, Z. (2010) Mathematics Skills Difficulties: A Mixture of Intricacies, Procedia ‐ Social and Behavioral Sciences: Elsevier, Vol 7, No.1,pp 171‐180. Tarim, K., and Akdeniz, F. (2008) The effects of cooperative learning on Turkish elementary students’ mathematics achievement and attitude towardsmathematics using TAI and STAD methods. Educational Studies in Mathematics, Vol67, No.1, pp 77–91. Tay Lay Heong (2005) Problem Solving Abilities and Strategies in Solving Multistep Mathematical Problems among Form 2 Students, Kertas Projek Sarjana, Universiti Malaya. Tudge, J. R. H., Winterhoff, P. A. and Hogan, D. M. (1996) The Cognitive Consequences of Collaborative Problem Solving With and Without Feedback, Child Development, Vol 67,pp 2892‐2909. Underwood, J., Underwood, G. and Wood, D. (2000) When does gender matter?: Interactions during computer‐based problem solving, Learning and Instruction, Vol 10, pp 447‐462. Walqui, A. (2006) Scaffolding instruction for English learners: a conceptual framework, International Journal of Bilingual Education and Bilingualism, Vol 9, No. 2, pp 159–180. Webb, N., Franke, M., De, T., Chan, A., Freund, D., and Shein, P., (2009) Explain to your partner: Teachers’ instructional practices and students’ dialogue in small groups. Cambridge Journal of Education, Vol 39, pp 49–70. Wood, D., Bruner, J., and Ross, G. (1976) The role of tutoring in problem solving. Journal of Child Psychology and Psychiatry, Vol 17, No.2, pp 89–100. Wu, S. S., Chang, T. T., Majid, A., Caspers, S., Eickhoff, S. B., and Menon, V. (2009) Functional heterogeneity of inferior parietal cortex during mathematical cognition assessed with cytoarchitectonic probability maps. Cerebral Cortex, Vol19,pp 2930–2945.

625

Training Flexible and Adaptive Arithmetic Problem Solving Skills Through Exploration With Numbers: The Development of NumberNavigation Game Boglárka Brezovszky, Erno Lehtinen, Jake McMullen, Gabriela Rodriguez and Koen Veermans Centre for Learning Research, University of Turku, Finland bogbre@utu.fi ernoleh@utu.fi jamcmu@utu.fi gabrod@utu.fi koevee@utu.fi Abstract: Traditional teaching methods often fail to develop the desired type of arithmetic problem solving expertise that is, flexibility and adaptivity in problem solving. Working with numbers, and exploring different number patterns and solution methods could represent a good basis for developing a more flexible and complex understanding of numbers and operations, which could result in the development of flexible and adaptive problem solving skills. Our aim when developing NumberNavigation (NNG) educational digital game was to provide an engaging and adaptive context for exploration with numbers. In the game, players progress by strategically selecting different sequences of number‐operation combinations, using the four basic arithmetic operations within the domain of natural numbers (1‐100). Players need to collect raw materials for building settlements, by navigating their ship through different maps placed over the 100 square. The game challenges players to strategically select the most optimal path and continuously adapt their choices. In this explorative case study, the goal was to test core game features and their relationship with in‐game arithmetic flexibility. A trial game version was tested in two consecutive sessions with three elementary school children. Methods of data collection included video recorded observation, open‐ended interview questions, and the screen capture of gameplay. Aiming to trigger more variation in players’ use of number‐operation combinations, we have tested three different game modes. In exploration mode players had to choose their moves so they bypass islands placed over the maps as obstacles. In the minimum moves mode, players had to make their journey using the least amount of moves, and in the minimum energy mode players had to reduce the magnitude of numbers used for navigating their ship. Results show that players used various number‐ operation combinations and they were able to adapt their in‐game problem solving strategies according to the changing rules of the game. The position of harbours‐islands‐targets proved to be a strong predictor of the amount of variation in number‐operation combinations and of players’ explorative tendencies. Overall, results suggest that basic game features are promising in triggering exploration with numbers and engaged practice over an extended period. Results of the study will be used to inform the future development of NNG. Keywords: adaptive expertise, arithmetic problem solving flexibility, game design, strategy

1. Background Developing adaptive expertise in arithmetic problem solving is one of the main objectives of elementary school mathematics education. Due to the complexity of this skill, traditional instructional methods might not be sufficient in this process. Although, there are different views regarding the meaning of adaptive expertise, it is usually agreed that the development of fluent and flexible procedural skills together with a deep conceptual understanding of numbers and operations are both necessary requirements (Baroody 2003; Schneider, Rittle‐ Johnson & Star 2011; Verschaffel et al. 2009). Flexibility and adaptivity in arithmetic problem solving represent strong markers of arithmetic expertise. Flexibility refers to the ability to apply various strategies during problem solving while adaptivity describes the strategic choice of the most optimal type of solution for a given problem which may depend on various factors such as problem characteristic, individual differences in problem solving skills, or the norms and rules of the context where the problem is solved (Verschaffel et al. 2009). Research shows that while placing emphasis on the development and practice of problem solving strategies can enhance flexibility, and that students of various skill levels can acquire and use different problem solving strategies, in practice these skills are rarely transferred to new problems. Despite students’ familiarity with various methods they often disregard both the characteristics of the task and their individual strategy performance when selecting problem solving strategies (Torbeyns et al. 2009). Similarly, despite

626

Boglárka Brezovszky et al. understanding when and how different problem solving strategies are more adaptive than others, students often fail to apply this knowledge in practice (Blöte, Klein & Beishuizen 2000). The discrepancy between the ability to understand different problem solving strategies and the ability to apply them adaptively in new problem situations is often explained as a shortcoming of instructional methods that place too much emphasis on teaching how to solve problems, concentrating more on the surface features of a problem and less on the underlining principles. As the development of flexible and adaptive skills in arithmetic problem solving largely depends on the ability to understand numbers and operations, instead of learning to choose from a set of different methods, the focus should rather be placed on developing well‐connected knowledge of numbers and operations (Baroody 2003; Threlfall 2009). By exploring and experimenting with different number combinations, children might develop a more richly connected conceptual understanding of numbers and the relations between different numbers and operations. Thus, important features and relationships between numbers in a problem might become more visible, and so more optimal solutions could be more readily recognized (Schneider, et al. 2011). Providing children with learning environments which trigger active exploration with numbers and reflection on problem solving processes, should enhance the development of stronger connections and the understanding of underlining relationships between number and operations and as a result trigger the development of flexible and adaptive problem solving skills (Baroody 2003; Verschaffel et al. 2009). For example, in their study Canobi, Reeve & Pattison (2003) examined 5‐8 year old children’s understanding of commutativity and associativity principles in addition problems and found that irrespective of age, those children who understood that numbers can be decomposed and recombined had a higher accuracy and more flexibility in their problem solving procedures. In equation solving, Rittle‐Johnson & Star (2007) found that reflection in terms of comparing and contrasting different solution methods proves to be an efficient method for noticing important features of a problem leading to the development of more flexible problem solutions and exploration of different alternatives. Additionally, there is evidence that using different mental models such as the hundred square or the mental number line might be beneficial for triggering the exploration of alternative solutions and the understanding of underlining principles between numbers and operations (Beishuizen 1993; Blöte, Klein & Beishuizen 2000). Games by their nature are ideal for triggering exploration and discovery and can provide a motivating context to experiment with numbers (Devlin 2011). Additionally, educational computer games can provide novel pedagogical approaches to deal with educational content that has been challenging for traditional teaching methods (Siewiorek et al. 2012). However, up to date there is no educational digital game with the explicit aim to trigger flexibility and adaptivity in arithmetic problem solving. Examples of games which aim to develop mathematical skills, and which build on a strong theoretical background in their game design providing empirical evidence of their efficiency are rare (Young et al. 2012). One relatively well‐researched domain in the field of games and mathematics is represented by games that aim to develop early numerical understanding by strengthening the mental representations of numbers. The mental number line has been successfully used in developing both the spatial representation of numbers and basic arithmetic problem solving skills of young children (Kucian et al. 2011). Other game designs with similar aims focused on strengthening the connection between different representations (e.g. symbolic and non‐ symbolic) of numbers (Wilson et al. 2006; Räsänen et al. 2009). Although these games are exemplary for building upon strong behavioural and neuro‐cognitive evidence in their game design, and provide adequate empirical evidence regarding the learning gains associated with gameplay, they are also highly specialized environments aiming to help young children with more severe mathematical difficulties (developmental dyscalculia). Arithmetics constitutes only a limited part of these games, and includes only basic addition and subtraction within a limited number domain. Furthermore, what might be a highly efficient game design for treating developmental dyscalculia in young children might not work for more various samples and mathematical skills. Unfortunately, in the domain of more advanced arithmetic skills, the amount of empirically tested games is even smaller. What is more, even existing games are often criticized for their inability to integrate their game design features with their mathematical learning aims (Young et al. 2012). A meaningful educational game

627

Boglárka Brezovszky et al. design requires embedding the educational content within the core game features (Devlin 2011) in a way that it becomes an integral part of the gameplay instead of a necessary extra placed on top of a game (Habgood 2007). Until now, there has been only one example of an educational digital game with published empirical results that manages to create a fusion between educational content and game design, as well as target the development of complex arithmetic skills such as the discovery and understanding of number patterns (Habgood & Ainsworth 2011).

2. Aims Our aim when developing NumberNavigation Game was to create a motivating game environment that would provide children a context to experiment with numbers and arithmetic operations. It was our goal to design a game where strategic choice in selecting the most optimal sequence of number‐operation combinations would represent those core game features by which players make meaningful progress. In this study, we have tested a prototype of NNG in order to gather rich qualitative data to be used in our future game design processes. Our aim was to explore the type of emerging game patterns that could suggest the presence of in‐game arithmetic problem solving flexibility and to explore those game features or combinations of game features where these patterns are more prominent.

3. Design principles of NumberNavigation game In NNG the player has to navigate a ship through different maps and collect raw materials for building settlements. Navigation takes place with the help of arithmetic operations that bring the ship to different locations in a map, which is based upon the hundred square. Within each map, players have to collect four types of building materials (wood, brick, stone and iron) placed in different locations and return them to the harbour. Harbours are located on a fixed number location within a map, which changes across different maps. Figure 1 illustrates a map from NNG, here the harbour is located at number 4 and the player has to pick up wood from number 94. In this example, a player might instinctively try to navigate on a straight line, adding 90 to reach the target, but this move is blocked by an island. A blocked move is signalled by a red path and a cross (see Figure 1). Islands always represent challenges the player has to consider when selecting a move. Alternatively, the player might consider 4*3 as the first correct move, which is signalled by a green path along which the ship moves to the next location, number 12. In NNG, apart from the islands, the efficiency of the sequence of moves a player uses towards the target depends on additional challenges. The game has three modes: exploration, minimum moves and minimum energy. In the exploration mode, the only challenge is to select moves so that the ship bypasses the islands. In the minimum moves mode the goal is to reach and return the target using the least amount of moves (operations). In the minimum energy mode the aim is to reduce the magnitude of numbers the player types into the command box to navigate the ship from one number to the next. For example, in case of the map presented in Figure 1, if the map is in moves mode, then continuing the journey from 12 to 48 would be a good option, reaching the target in three moves: 4*3=12; 12*4=48; 48+46=94. However, if the map is in energy mode, then this option would cost the player 53 energy units (the total value of numbers used to navigate the ship: 3, 4 and 46). If needed, this energy could be reduced by considering alternative options and using for example the following path which would only take 37 energy units: 4*11=44; 44‐6=38; 38*2=76; 76+18=94. Naturally, many alternatives are available at once, and it is the task of the player to decide which option would be more efficient for reaching the target, considering the position of harbours, materials, and islands, as well as individual maps’ game modes. As the task is to reach then return each material, the player can always use the same numbers and inverse operations on the return, but also has the opportunity to re‐evaluate and adapt the original strategy and chose a better path on return.

4. Methods 4.1 Participants Game testing took place in two consecutive sessions. During the first session J. (age 11) played the game individually while during the second session T. (age 9) and E. (age 11) played the game collaboratively. All three participants were male elementary school students confident in using digital games (both educational and recreational) and had a self‐reported positive attitude towards mathematics.

628

Boglárka Brezovszky et al.

Figure 1: Feedback in case of right move and blocked move in NNG

4.2 Instruments and procedure Data was collected using the methods of video‐recorded observation, interviews and recording of the gameplay. Players were recorded using a digital video camera. On‐screen activity was recorded using Camtasia Studio 5.1.0 screen capture software. Video data from the two sources were synchronized and analysed simultaneously using Transana 2.42 software. Interview questions consisted of open‐ended questions regarding players’ game experience. Game sessions lasted around one and a half hours, including instructions and interview questions. The game was played in three different modes introduced in the same order. First the exploration mode, followed by playing the game according to the rules of the minimum moves mode, and lastly playing the game according to the rules of the minimum energy mode. During collaborative play, participants had an extra option to decide which game mode they would like to apply for the last map (choice mode). The tested game version included three maps with 4 different target materials in each map. The tested maps and the location of harbours and target materials within each map is presented in Figure 2. Through the article, maps will be referred to according to their names used in Figure 2 (Map A, B & C).

Figure 2: Position of harbours and target materials within the three tested maps of NNG

5. Results Table 1 presents the sequence and number of maps that players completed during the three different game modes as well as the amount of moves (arithmetic operations) made. Aside from successful moves, blocked moves were also logged and differentiated. Blocked moves include both operations that were impossible to

629

Boglárka Brezovszky et al. execute due to islands blocking the path as well as operations which did not fit within the frame of natural numbers 1‐100. During collaborative play, players chose to complete the last map under the minimum moves mode. Table 1: Sequence and number of maps in different game modes; total number of clear and blocked moves Sequence & Number of Maps Moves Blocked moves Ind. Coll. Ind. Coll. Ind. Coll. Exploration A,B,C A, B 92 57 18 9 Minimum moves A,B C, A, B 52 68 9 4 Minimum energy C C, A, B 19 67 1 8 Choice ‐ C ‐ 16 ‐ 4 Total 6 9 163 208 28 25 Game Mode

Note: Ind. = Individual gameplay; Coll. = Collaborative gameplay As Table 1 shows, compared to the total number of moves the incidence of blocked moves was relatively low. Reasons for blocked moves were more often technical than mathematical, as from certain positions paths looked possible but were blocked by land. Total time on task (playing the game) was 1 hour for the individual game, and 1 hour 20 minutes for the collaborative game. Completing all three maps in one game mode took around 25‐30 minutes irrespective of the mode. Completing one map could take 5 to 10 minutes depending on the difficulty. The time needed for retrieving a material varied from 50 seconds to 5‐8 minutes, indicating that regardless of the game mode, certain quests proved to be more challenging than others. In order to identify signs of in‐game arithmetic flexibility and possible effects of the different game modes on players’ strategies, the types of number‐operation combinations used were explored across different game modes. Figure 3 and Figure 4 show the use of numbers and number‐operation combinations in each of the three game modes for individual and collaborative gameplay. There was a clear trend in using 10 and multiples of 10 during both game sessions. This trend becomes even stronger in case of larger numbers >40 and >50 where the players almost exclusively used round numbers (numbers ending in 0). Looking at the use of numbers over the two game sessions, five numbers seem to emerge as frequently used operands: 1, 5, 9, 10, 11, and 40. From these frequently used numbers, the use of 1 and 5 can be explained by the fact that they represent easy moves from an arithmetic perspective. The high incidence of 40 is an artefact of Map B where +40 and ‐40 was very frequently used for all four target materials and during both game sessions. The use of 9, 10 and 11 can be explained by the 10x10 square characteristic of the maps where using +10 and ‐ 10 means 1 move up or down, while +/‐ 9 or 11 represent 1 diagonal move. While using multiples of 10 to move several rows up or down was very common during the game, multiples of 9 and 11 were much less frequently used. From the common numbers, 11 was used with all four operations during both game testing sessions. It was not only used for adding and subtracting when moving diagonally, but also for replacing the addition or subtraction of large numbers by multiplying or dividing them by 11 during the minimum energy mode.

Figure 3: Number‐operation combinations by different game modes during individual play

630

Boglárka Brezovszky et al. From a total of 163 arithmetic operations performed during individual play, 57 where different number‐ operation combinations, while from the 208 operations during collaborative play, 67 where different. As Figure 3 and 4 show, the use of multiplication and division was significantly lower than the use of addition and subtraction, with only 4 different multiplications and 3 divisions during individual play, and 5 multiplication and 5 divisions during collaborative play.

Figure 4: Number‐operation combinations by different game modes during collaborative play The variety of numbers used for addition and subtraction was much larger than for multiplication and division, which is understandable given the nature of these operations. This explains why only smaller numbers were used (2, 4, 5, 8, 11, 12, and 13) for the latter. Multiplications were used for reaching the target material and divisions almost exclusively for returning material to the harbour. During both sessions, minimum energy proved to be the only game mode that was able to trigger the use of all four arithmetic operations to the same extent. For certain materials, players almost exclusively used the same path in all game modes, while for other materials they chose alternative number‐operation combinations during the different game modes. The interaction of the placement of harbours, target materials and islands had an impact on the variety of paths used by participants. For example, the last material in Map B (iron) shows high variation of moves and energy across all quests and game modes during both testing sessions, suggesting the use of various alternative number‐operation combinations. To illustrate variability in this quest, Figure 5 shows a visual reconstruction of all moves players made at this map. As Figure 5 shows, reaching the target material was challenging, as islands were positioned in a way in which, apart from the usually preferred combinations of round numbers, the use of diagonal moves became desirable, which triggered addition and subtraction with multi‐digit large numbers. Video data suggests that harbour‐island‐target combinations such as this one triggered extensive planning and strategy testing during both sessions. Visible signs of strategy testing were: clicking on those numbers with the mouse which could make part of a possible route, drawing virtual lines of the possible paths on the computer screen using the mouse or fingers, using fingers as a ruler on the screen to check for available angles. Using the game collaboratively triggered significantly more planning and strategy testing, contrasting different alternatives. Combinations of harbour‐island‐targets that were too easy resulted in automatic moves, such as +/‐ 40 in Map A. On the other hand, even these combinations were useful if used in a more challenging game mode. For example, during collaborative play, the younger player (T.) became passive after the minimum energy mode was introduced in Map C, perhaps due to the relative difficulty of this mode for his age group. The older player (E.) of the pair completed Map C individually, but upon revisiting Map A under the minimum energy mode, T. became active again. T. had in previous game modes developed the habit of using 4+40 as a first move, irrespective of the location of the target materials. When it was time to make the first move at Map A, now under the minimum energy mode, T. took control of the keyboard and typed in 4x11 instead of the usual 4+40. Nonverbal signs such as smiling and nodding suggest that he made a meaningful discovery at this point of the game.

631

Boglárka Brezovszky et al.

Figure 5: Visual reconstructions of players’ moves in Map B. Left: individual play. Right: collaborative play

6. Discussion In this study, it was our aim to explore markers of in‐game problem solving flexibility in a prototype version of NumberNavigation Game and to examine the efficiency of core game features in triggering the flexible use of number‐operation combinations during gameplay. During the total playtime of 1 hour and 1 hour 20 minutes, players carried out 163 and 208 mathematical operations. Apart from these operations, players performed extensive additional mental arithmetic for testing competing alternatives when deciding to use a given number‐operation combination. During this limited playing time, three different game maps were tested, with 12 unique combinations of positioning harbours (starting number), target materials (goal number) and island (as obstacles). These 12 distinct combinations were further enriched by the 2 routes of first collecting then returning each material to the harbour, and additionally by the use of 3 different game modes in which players had to separately adapt their strategies in choosing the ideal moves. Specific combinations of these game features proved to be more efficient than others for triggering variety in player’s numbers‐operation combinations. For instance, irrespective of the game mode, the interaction of positioning harbours‐islands‐targets was a very important factor that defined the amount of variation in players’ use of numbers. For example, having several smaller islands in a central position on the map, which allow many different but equally desirable paths, proved to be an effective design feature for triggering exploration with different alternatives. Target material placement is of major importance in future game versions, since these materials can be placed so that they are progressively more challenging within and across different maps. Additionally, it is also crucial to design the game in a way that it is clear for the player which combinations are possible and which are blocked by land. Results suggest that when available moves are unclear, players can easily become frustrated and forego exploration, moving almost automatically, using one step at a time. Using different game modes and challenging players to adapt their strategies according to the changing game rules was highly effective for triggering strategy testing, comparing and contrasting different alternatives, and planning ahead sequences of steps. Keeping the exploration mode in future game versions is unnecessary, as players quickly learnt the rules of the game and they preferred having explicit rules and challenges. Having to collect target materials while using the least amount of moves (minimum moves mode) proved to be efficient in triggering mental calculations with multi‐digit large numbers and efficient in breaking the tendency of players to work with multiple of 10. The rules of the minimum energy game mode proved to be slightly more difficult to understand, especially for the young player. However, once rules were understood, the minimum energy mode managed to trigger more flexibility in player’s use of numbers and operations. It is important in future game versions that the two game modes are introduced progressively, first using the minimum moves mode, and then gradually introducing the minimum energy mode in maps with easier combinations of

632

Boglárka Brezovszky et al. harbours‐islands‐targets. Using the game collaboratively proved to be more efficient in triggering strategy testing and exploration of alternative options. Accordingly, using the game in pairs or in groups, in a collaborative or competitive manner seems as a good model to be used in future studies with NNG. Overall, the present study managed to explore in detail how in‐game markers of arithmetic problem solving flexibility and adaptivity can be better conceptualised and defined. Variety in the type of numbers used and in operations used is a good indicator of flexible game strategies, while any fixed and repeating pattern suggest a more mechanical gameplay. Strategy testing during the two different game modes, as well as contrasting different competing alternatives is also a clear indicator of more advanced strategies. However, more evidence is needed to determine if players were choosing the most effective routes, indicating more adaptive expertise with arithmetic and not simply applying a variety of non‐efficient routes (Verschaffel et al., 2009). Based on the current results, future work with NNG will focus on level design and the integration of adaptive adjustment of the challenge‐reward ratios. A very positive feature of NNG is the open game design, with minor changes the game can be engaging for older, more skilled players as well. Similarly, in future versions, the 10x10 square can be easily exchanged to alternative systems (e.g. base 9 or 12), creating instantly more variability in players’ number‐operation combinations. Adding additional rules that would trigger the more frequent and flexible use of multiplication and division would also be desirable in future game versions. As both the sample and the tested number of game maps were limited, conclusions of this study cannot go far. Overall, the game managed to trigger player’s engagement for a relatively long period of time, there were no signs of boredom during gameplay, no random clicking or guessing, and the game design was easy and straightforward. Video data showed many identifiable signs of strategic behaviour with number combinations as well as comparing and contrasting alternative options. Even in this limited amount of time, players used a large variety of numbers and many different number‐operation combinations. Additionally, players seemed to manage to adapt their strategies according to the changing rules of the game during the three different game modes. These are promising indicators that the elementary game design features of NNG are able to trigger engagement and exploration with number‐operation combinations.

References Baroody, A. J. (2003) “The development of adaptive expertise and flexibility: The integration of conceptual and procedural knowledge”. In A. J. Baroody & A. Dowker (eds.), The development of arithmetic concepts and skills (pp. 1–34). Mahwah, NJ: Lawrence Erlbaum Associates. Beishuizen, M. (1993) “Mental Strategies and Materials or Models for Addition and Subtraction up to 100 in DutchSecond Grades”, Journal for Research in Mathematics Education, Vol. 24, No.4, pp 294‐323. Blöte, A. W., Klein, A. S. & Beishuizen, M. (2000) “Mental computation and conceptual understanding”, Learning and Instruction, Vol. 10, No. 3, pp 221‐247. Canobi, K. H., Reeve, R. A. & Pattison, P. E. (2003) “Patterns of Knowledge in Children’s Addition”, Developmental Psychology, Vol 39, No.3, pp 521‐534. Devlin, K. (2011). Mathematics Education for a New Era, MA: A K Peters, Natick. Habgood, M. P. J. (2007). The effective integration of digital games and learning content, Ph.D. Thesis, University of Nottingham. Habgood, M. P. J. & Ainsworth, S. E. (2011) “Motivating children to learn effectively: Exploring the value of intrinsic integration in educational games”, Journal of the Learning Sciences, Vol 20, No. 2, pp 169‐206. Kucian, K., Grond, U., Rotzer, S., Henzi, B., Schönmann, C., Plangger, F., Gälli, M., Martin, E., von Aster, M. (2011). “Mental Number Line Training in Children with Developmental Dyscalculia”, NeuroImage, Vol. 57, No. 3, pp 782–795. Rittle‐Johnson, B. & Star, J. R. (2007) “Does Comparing Solution Methods Facilitate Conceptual and Procedural Knowledge? An Experimental Study on Learning to Solve Equations”, Journal of Educational Psychology, Vol. 99, No. 3, pp 561‐ 574. Räsänen, P., Salminen, J., Wilson, A. J., Aunio, P., Dehaene, S. (2009) “Computer‐assisted intervention for children with low numeracy skills”, Cognitive Development, Vol. 24, No. 4, pp 450–472. Schneider, M., Rittle‐Johnson, B. & Star, J. R. (2011) “Relations Among Conceptual Knowledge, Procedural Knowledge, and Procedural Flexibility in Two Samples Differing in Prior Knowledge”, Developmental Psychology, Vol. 47, No. 6, pp 1525‐1538. Siewiorek, A., Saarinen, E., Lainema, T. & Lehtinen, E. (2012) “Learning leadership skills in a simulated business environment”, Computers & Education, Vol. 58, No. 1, pp 121–135. Threlfall, J. (2009) “Strategies and flexibility in mental calculation”, ZDM Mathematics Education, Vol. 41, No. 5, pp 541‐ 555.

633

Boglárka Brezovszky et al. Torbeyns, J., De Smedt, B., Ghesquière, P. & Verschaffel, L. (2009) “Jump or compensate? Strategy flexibility in the number domain up to 100”, ZDM Mathematics Education, Vol. 41, No. 5, pp 581‐590. Verschaffel, L., Luwel, K., Torbeyns, J. & Van Dooren, W. (2009) “Conceptualizing, investigating, and enhancing adaptive expertise in elementary mathematics education”, European Journal of Psychology of Education, Vol. 24, No. 3, pp 335‐359. Wilson, A. J., Revkin, S. K., Cohen, D., Cohen, L. & Dehaene, S. (2006) “An open trial assessment of "The Number Race", an adaptive computer game for remediation of dyscalculia”, Behavioral and Brain Functions, Vol. 2, No. 20. Young, M. F., Slota, S., Cutter, A. B., Jalette, G., Mullin, G., Lai, B., Simeoni, Z., Tran, M., & Yukhymenko, M. (2012) “Our princess is in another castle: a review of trends in serious gaming for education”, Review of Educational Research, Vol. 82, No. 1, pp. 61‐89.

634

Trials to Assess Team‐Based Mixed‐reality (TBMR) Games in HE John Denholm and Sara de Freitas Serious Game Institute, Coventry, West Midlands, UK jadenholm@yahoo.com sfreitas@cad.coventry.ac.uk Abstract: This article relates to a series of trials to assess the value of games of the Team‐based Mixed‐reality (TBMR) type, a term used to denote games involving participants working in small teams to make periodic decisions in relation to a real life scenario. A game was specifically developed to represent the TBMR genre and assessed using students taking a module in the Management of Innovation. Some two hundred pre‐ and post‐tests were conducted, to capture both qualitative and quantitative data. In the former, the students’ perception of the value of the game was assessed and in the latter case, the actual learning which took place. The results were analysed both by the improvement in the scores achieved and by the percentage of students giving the correct answer. The findings showed that the perceived educational value of the game was very positive; also in terms of motivation towards the subject and in comparison to a traditional lesson. The results relating to actual educational improvement showed a slight increase in both knowledge and decision‐making ability overall, but there was a significant difference when split between Chinese students and others. The rationale for the game design and development process is also detailed, with reference to the findings of others researching game design and considerations given to scope, level of challenge, alignment to the module, the contribution of teamwork and feedback. An analysis of each of the specific test questions revealed a wide variation in results and lessons were learned as to which particular topics enhance learning the most, thus pointing to improvements in reshaping the game for future use. Keywords: game, learning, assessment, formative, teamwork, decisions

1. Introduction This study focuses on a particular genre of games, concerned with business or management training, which simulate the running of a business operation, usually played in small competitive teams where team members must make a series of decisions over a number of periods or cycles throughout the simulation. The term that has been adopted for this genre of games is Team‐based Mixed‐reality (TBMR). This article describes a series of game assessment trials and the design of a game with which to carry out the trials. It is divided into sections as follows: 1. Introduction 2. Research questions 3. Overall rationale and literature review 4. Game design and development methodology 5. Game assessment methodology 6. Analysis and results 7. Conclusions, limitations and recommendations

2. Research questions

Do participants in Team Based Mixed Reality (TBMR) games in higher education (HE) view the experience as a positive contribution to their learning?

Do TBMR games in HE actually contribute positively to the learning process, in improving knowledge and decision‐making skills?

Do cultural differences have a bearing on the perception and effectiveness of TBMR games?

Is game design critical to the educational value of TBMR games?

This research study makes use of both qualitative and quantitative methods. Baker (2000) justifies this combination of methods; he states that although there is abundant literature that compares and contrasts quantitative and qualitative methods, there is an increasing acceptance to integrate both approaches. Many authors defend the strategy of integrating qualitative with quantitative methods. According to Bericat (1998) this complementary strategy is based on the desire to obtain two different images of social reality on which the researcher is interested. Each method provides a truthful and very different picture, revealing

635

John Denholm and Sara de Freitas different aspects which allow the expansion of knowledge of social reality by implementing a methodological investigation with two parallel structures of study.

3. Overall rationale and literature review 3.1 Terminology. The designed game will be referred to as SGIBS (Serious Games Institute Business Simulation). Student assessment and game assessment are closely related, but may be taken as interchangeable in this study as student assessment is the means towards the assessment of the SGIBS game and by implication games of the TBMR genre. The research was carried out in two ways, by external or objective assessment (quantitative) i.e. conducting pre and post tests of knowledge and skills, and in part by internal or subjective assessment (qualitative) i.e. asking the participants themselves for their view on their experience, for example how it altered their motivation towards the subject. In education, there are generally two types of assessment, formative (for learning) and summative (for measuring achievement). Summative assessment is typically conducted towards the end of a course of instruction, to test a person’s understanding, retention or mastery of the subject. Formative assessment is designed to measure the learning that is taking place while the course of instruction is ongoing. Feedback is seen as an important feature of formative assessment as it allows students to reflect on and improve their level of achievement. Rapid feedback is seen as one of the features of educational games that scores over other forms of learning.

3.2 The module selected for the assessment Several possible venues, courses and cohorts were initially considered before selecting a module on which to carry out the trials. The module finally selected was entitled, “Management of Engineering and Technology Innovation” (METI). The main reasons for selecting this particular group of students were:

The method of delivery: innovative ideas being developed by students, working in small teams, allocated according to Belbin’s theories of roles in teamwork (Belbin, 1981, 1993).

The content of the module i.e. business management, is frequently a subject for TBMR games. Previous studies on games of this type used at Warwick University have been carried out (Denholm et al, 2012).

There was potentially a large available sample of participants, depending on the actual intake. In the event classes of fifteen to thirty students were used, six for game trials and five for control testing.

The design and development of a Massively Multiplayer Online Role‐playing Game (MMORPG) type game was considered but rejected on the grounds of time and cost to develop. Computer labs were not generally available, only seminar rooms provided with a single tutor‐controlled computer with projector and overhead screen. It was also considered that the game characteristics and their alignment to the pedagogical requirements were more important than the technology used. Gee (2003) has argued that the secret of a good game is not its 3‐D graphics and other bells and whistles, but its underlying architecture. It was therefore decided to use a specially developed game for the purpose of the trials, drawing on the findings of researchers on game design. It was also considered that the process itself would offer valuable insight into the issues encountered in TBMR game development.

4. Game design and development methodology With the METI module firmly in mind, the game was developed using Excel version 2010 software, enabling construction in about ten weeks. While not the preferred software choice for most advanced technology games, it provided access to built‐in features such as mathematical functions and graphical charts, ideal for displaying results. It was completed in August 2012 and tested during two separate trial runs using volunteer staff at the Serious Games Institute (SGI).

636

John Denholm and Sara de Freitas

4.1 The scenario and design features The game scenario was based on reviving an ‘ailing’ company (designing and selling coffee machines) over eight ‐ or optionally, twelve ‐ three‐month periods. Each of four teams in a class compete to manage ‘similar’ companies in the same market, to implement the right strategies and take the best tactical opportunities on offer, as presented to them. The periodic nature of the game allowed a cycle of debate, reflection, decision (i.e. trial strategy) and reaction to the results (i.e. immediate feedback). This cycle is repeated for each period, giving the possibility of correcting the effect of the previous decisions. For example, if players decide to invest in more manufacturing capacity but fail to raise the necessary funding, a large overdraft will result, something that can be corrected in the following period.

4.2 Choices and scope Although largely determined by the module Learning Outcomes, the scope was limited by what could reasonably be covered in the allotted time for game play, which, in view of time‐tabling constraints was one and a half hours. Of the eight game periods, four were designated for strategic decisions (pricing, funding and capacity expansion) and four for tactical ones (there was a choice to be made of eight out of sixteen ‘opportunities’ over the span of the game). Information was available on costs, benefits and other possible consequences of choosing these options, thus avoiding the need for participants to do calculations, but rather focus on making judgements.

4.3 Content, alignment with learning outcomes and level of difficulty Different types of knowledge and skills require different learning and teaching activities (McKeague & Di Vesta, 1996). It is essential that we understand not just how games work, but how they are aligned with taxonomies of learning (Van Eck, 2006). He also says that games that are too easily solved will not be engaging; games which are successful as teaching tools are those that create a continuous cycle of cognitive disequilibrium and resolution while also allowing the player to be successful. In order to design an educational game, we need to pay special attention to the functionality, game play and social and pedagogical issues (Konzack, 2002; Liarokapis, 2006), especially when we target learners' motivation and complex problem‐solving skills, making choices through the game play. The topic content of the SGIBS game and the nature of the decisions incorporated into it reflected the Learning Outcomes of the METI Module. Designing game‐based learning is very different from designing entertainment games as it must take into account learning objectives, instructional activities etc. Assessments need to be aligned with the learning objectives; indeed, instructional designers are usually advised to develop test items before designing instruction (Smith & Ragan, 1999). The art of game design is creating situations, challenges, rules and affordances that keep players at the leading edge of what they can do. Research suggests that when students are more motivated, assessment results are a better reflection of their ability (Schmit & Ryan, Sundre & Wise, 2003). It was in the end it was matter of the researcher’s judgement and experience as to what the content and level of difficulty should be. Simpler decisions were introduced first, followed by increasingly more complex ones. For example, in period 1, the only decision required is on product price. Thereafter decisions may involve funding, expansion, quality etc. becoming gradually more challenging, encouraging team discussion and possibly conflict.

4.4 Teamwork, goal setting and competition In order to have meaningful social interactivity among the players, a game should incorporate meaningful collaborative tasks, which allow them to interact naturally (Eseryel et al (2012). Chen et al (2006) says that, “Social interactivity during game play, such as competition and collaboration with others who are also playing the game, plays an important role contributing to learners' motivation”. Chen et al (2006) state that positive social interaction in a MMORPG, such as pro‐social behaviours, trading and collaboration can enhance players' engagement. Sweetster and Weth (2005) and Yee (2006) found that social

637

John Denholm and Sara de Freitas interactions allowed players to compete, collaborate and connect leading to game flow experiences. There needs to be a clearly defined end‐point in the game and hence the criteria for winning. Otherwise there is no straightforward way to assess the learning that has taken place. (de Freitas & Neumann, 2009). Goal‐based scenarios have long been viewed as an active primer for situated learning (Bransford et al, 2000). In a good game a player is involved in an iterative cycle of goal‐based interactive problem‐solving. Setting a goal or target as a measure of team success would seem to enhance the competitive aspect of the game. Dempsey et al (2002) found that incorporating challenges, clear goals and sufficient feedback into narratives are important for players' gaming experiences. Ideally, the narrative can bring a player into the state of flow (Csikszentmihalyi, 1991) in which a players completely focus on the task at hand and forget about self, about others, about the world around themselves. Players also lose track of time, feel happy and in control, and become creative and productive (Csikszentmihalyi, 1991). Designing appropriate assessments is central to designing games. Rupp et al (2010). Prensky (2001) developed a classification of games that identifies the content for development, the activity taking place, and the games that can support the identified activity. Some of his examples are Facts, Skills, Judgements and Behaviours. For instance with Judgements, the classification suggests that activities are: reviewing cases, asking questions, making choices, feedback coaching. And the suggested type of game for those activities are: role play, adventure games, multiplayer interaction games, strategy games. It has been suggested that games can effect a direct transfer of knowledge (Thomas & Brown, 2007) and skills such as numeracy and literacy (de Freitas, 2004). In the SGIBS game ‘Return on investment’, ‘Stakeholders’ equity’ and ‘Reputation’ were all used as final measures of team success. No random elements were introduced, ensuring a level playing field for all participants.

4.5 Feedback Feedback is the single most powerful influence in learning improvement (Black & William, 1998 & 2009). A good game reinforces a sense of control (Zimmerman & Schunk, 2001). Well‐designed games . . . employ ongoing feedback as a major mechanism of play/learning support. Feedback in formative assessment allows reflection and chance to improve. Feedback can be found at every level and unit of an efficient educational system and is an overarching concept that helps to explain and interpret the role of assessment in educational games. For assessing some key target variables of education (e.g., students; knowledge and skills) testing has been considered as the most objective and reliable way. Feedback which is used by the learner is considered the most important, distinctive attribute of formative assessment (Taras, 2005). Author Chen states that “We analyzed the similarities and differences between online diagnostic assessment and educational games. Diagnostic assessment focuses on feedback and orients students' learning process towards the next learning tasks, while educational games support learning in a more direct way by presenting learning material and development stimuli”. (Chen et al, 2006). Consideration needs to be given to the role of assessment for providing feedback for future learning (Bransford et al, 2000). (Van Eck, 2006) suggests that what is needed for the effective integration into the curriculum is practical guidance on how (when, with whom, and under what conditions) games can be integrated into the learning process to maximize their potential. Feedback has more educational value if it is immediate, rather than summative feedback at the end of a module, by which time it is too late. It can be argued that games give even more immediate feedback. The SGIBS game was not used for summative assessment i.e. gradings, but rather formative assessment, with tutor feedback after each game period and at the end. The chart below shows an example of immediate feedback during the game. Feedback in this graphical form is easily assimilated, shows clear comparisons by team and provides motivation to initiate further strategies e.g. expand capacity, promote the product and reduce defective or reject parts.

638

John Denholm and Sara de Freitas

Figure 1: Example of company results displayed e.g. for Period 3 (all teams)

5. Game assessment methodology The classifications discussed in the previous section suggests there is a distinction to be made between the acquisition of knowledge and application of skills, and it was decided in the design of the test questions, to make this distinction in both the qualitative and quantitative studies. In the latter, test questions into two parts, PART A: knowledge, and PART B: decision‐making questions, framed in the manner of, “What would you do if . . . ?” to make the participant imagine that he, or his team, are being presented with a real‐life business situation.

5.1 Quantitative study Pre‐ and post‐test experimentation is a common research method employed by educators in traditional classrooms to ascertain the effectiveness of instructional processes. The methodology may appear to be useful for assessment of game‐based learning. (Kebritchi, 2008) states that, typically, two identical tests are administered, one before and one after some experimental method of instruction (i.e. intervention). Keeping other variables constant, the difference in achievement scores i.e. D = (T2 – T1) may then be attributed to the improvement brought about by the intervention itself. Control pre‐ and post‐tests can also be used i.e. without the intervention, in which case C = (t2 – t1) enables the real improvement I = D – C to be measured. The research trials were conducted over two consecutive weeks, beginning 7th Jan 2013. In week 1 participants were asked individually to answer sixteen questions, eight questions on knowledge and eight on decision‐ making skills, by ticking one of three optional answers to each question. In week 2 students in game classes were allocated, (using a randomising register) into four teams. The teams re‐arranged themselves in groups, and the game was conducted for the eight ‘periods’ followed by a tutor ‘terminal’ feedback session, reviewing the consequences of the various decisions taken. All game students were then asked, individually, to take the test again, and also answer a short 5‐question survey, rated on a Likert scale of 1‐6. The six control classes were just given the post test, but without the 5‐question survey.

6. Analysis and results of game assessment 6.1 esults: qualitative The chart below indicates the Likert scale results for all 50 game participants responding to the five questions in the qualitative survey.

Figure 2: Likert 0‐6 point scale: Frequency of response from all game participants

639

John Denholm and Sara de Freitas When the two classes of Chinese students were averaged separately, these were seen to rate the experience less highly as the others, on all five counts.

Figure 3: Likert 0‐6 point scale responses, as percentages: Chinese classes, versus non‐Chinese The t‐test value (Chinese v non‐Chinese data) is 0.003, i.e. there is only a 0.03% chance that the difference is due to random fluctuations alone.

6.2 Analysis of individual questions In view of some scores indicating a negative improvement, a question by question analysis was done, showing for example, the results below for the Knowledge questions:

Figure 4: Percentage of students getting correct answers: Knowledge: pre v post test This indicated a high variation, most questions showing improvement, but in two cases negative improvement, question D in particular. Post test score %

Average

Knowledge Question F

Knowledge question D:

Knowledge question A

Pre test score %

Figure 5: Post versus pre scores, for the 8 knowledge question, with line of zero improvement

640

John Denholm and Sara de Freitas Figure 5 indicates that for some questions scores improved, for others they declined, the average showing a slight improvement. The exceptional cases, A, F and D are analysed in detail (below). The correlation coefficients are 0.483 for knowledge and 0.489 for decision‐taking. The following chart show the results for Knowledge questions, before and after being adjusted for the control results: The 8 knowledge questions. Inc/dec in % of correct answers (by question)

Figure6: Percentage of students giving correct answers: Knowledge (adjusted for control) A more detailed examination of each question was carried out, as indicated below. The correct option is highlighted in bold, and the pre and post test scores shown, together with the real percentage change, adjusted for the control test: The two questions showing the biggest increase in numbers answering correctly were: Knowledge question F

Answer options

For a business, the terms “gearing” or “leverage” refer to:

1 2

The company’s power over suppliers Long term debt as a proportion of funding The company’s power over dealerships

GAME GAME Pre Post test test 30 56

CTRL Pre test 38

CTRL Post test 36

Improve ‐ ment (adjusted) +28 (+93%)

Large improvement: The term ‘gearing’ was one of the key ratios displayed for each after each period, as a measure of financial risk. Knowledge question A

Answer options

GAME Pre test

GAME Post test

For a company making commercial catering equipment, which type of promotion is likely to be the most cost‐ effective:

Advertising in cafes and restaurants Advertising on television Advertising in catering and trade journals

2 3

CTRL Pre test

CTRL Post test

Improve ‐ ment (Adjusted

16 (+42%)

Large improvement: These options were the subject of alternative tactical options. Answer 3 is correct as the company was selling to the trade, not the public. The question showing the biggest decline in correct answers was Knowledge D: Knowledge question D

Answer options

What is meant by outsourcing

Buying components from another company Moving the whole operation abroad Buying components from abroad

2 3

641

GAME GAME Pre Post test test 62 34

CTRL Pre test 56

CTRL Post test 54

Improve ‐ment (Adjusted ‐26 (‐50%)

John Denholm and Sara de Freitas Negative ‘improvement’: The game terminology may have caused confusion, by offering an option to, ‘Outsource to Asia’, and led to the wrong post test answer being given. The wording of this question is not good, as none of the answers is technically ‘wrong’. This demonstrates that a single question can have a significant effect on the overall results and that a more considered choice of both questions and game ‘learning’ content can improve effectiveness.

7. Conclusions and recommendations Conclusions are formulated in relation to the four research questions:

7.1 That participants in team based mixed reality (TBMR) games in higher education view the experience as positive in the context of learning? The findings are positive, with overall ratings of 69% for both knowledge improvement and decision‐making. The next highest is motivation at 74% and comparison with a normal lesson at 78%. ‘Entertainment’ had the highest rating at just over 80%.

7.2 That TBMR games in HE actually contribute positively to the learning process, in improving knowledge and decision‐making skills? There was evidence of marginal improvement due to the game experience, in both knowledge and decision‐ making skills, as measured by the sixteen questions. The game post tests results taken altogether showed an improvement of just 2.7% in scores. However the control pre to post scores (no game played) showed an improvement of ‐ 5.2% i.e. a decline so that the net improvement due to the game is 7.9%. The results on a percentage getting the correct answer basis showed a different but more consistent picture for all students, with an improvement of 3.33% and 3.35% for knowledge and decision‐making respectively. However an analysis of the questions, while confirming that those were reasonably fair in terms of options answered in the pre test, indicated that there was a great variation in the results, if taken question by question. After adjusting for the change in control results, the average net changes by question ranged from a real percentage improvement of plus 87.6% to minus 45.5%. When adjusted for the control, this became plus 93% and minus 50%. It was clear from this analysis that in some areas at least, the game definitely contributed to the learning process.

7.3 That cultural issues may have a bearing on the above hypotheses? For all five subjective survey ratings, the Chinese scores were 20 percent below the non‐Chinese. The reasons can be the subject of much speculation e.g. language, unfamiliarity with problem based learning etc. Further research would need to be done on this. With the quantitative tests, the correlation (pre v post) for the Chinese classes was much lower (0.4) than that for the non‐Chinese classes (0.7) and it can therefore be assumed that those results were less reliable. When the Chinese classes are removed, all quantitative results show a bigger improvement. Taking the control tests into account, they show an improvement of 12.9% using the post game data. Breaking this down by knowledge versus decision‐making, the figures are 9.2% for the former and 16.7% for the latter. This would indicate that decision‐making skills showed a greater improvement than pure knowledge. The Chinese classes taken on their own showed a net increase of 2% after adjusting for the control results.

7.4 That appropriate game design is critical to the educational value of TBMR games? While the participants’ subjective feedback was positive, due account needs to be taken of lessons learned from the ‘question analysis’ which highlighted possible mismatches between test questions and game issues. A good game requires an iterative process of design, trial and error and re‐design, a bit similar to Kolb’s cyclical

642

John Denholm and Sara de Freitas theory of reflection in the learning process itself. This may be a good strategy for future improvement of the SGIBS game or games of its type.

7.5 Limitations of the research A larger data sample would have allowed for greater confidence in the results, but 204 tests were conducted and it would have been difficult to obtain a larger sample in practice. There is the limitation that the results only relate to the SGIBS game itself and those particular students involved. Postgraduate students may well have given more positive results, as was indicated by some preliminary pilot trials. Since the tests were not part of their module assessment, a minority of students’ may not always have been answering the questions to the best of their ability, though there was no clear evidence of this e.g. ticking boxes at random. An analysis of each of the questions designed to reflect the game content revealed a wide variation in results, taken by question.

7.6 Recommendations and future work The SGIBS game has the potential for further development, with account taken of which particular topics enhanced learning the most. The game could be conducted over a longer period of time or indeed over several weeks, in weekly sessions. There is the possibility that the game design could be adapted into a MMORPG game with a view to potentially much wider participation, in some form. It could also have a wider use in its present form, with modifications to suit different module or training requirements. Improvements might include providing a more friendly interface and improving data security. On the basis of those results, however, this type of game can be recommended for use in Business or Management‐style programmes in HE and possibly for industrial management training courses.

References Baker, S., 2000. Qualitative Methods Complementing Quantitative Research. Change in Learning & Practice. 13, 91‐95. Belbin, R. M. (1981) Managing Teams: Why They Succeed or Fail. London Butterworth‐Heinemann. Belbin, R. M. (1993) Team Roles at Work. Oxford Butterworth‐Heinemann Bericat, E., 1998. The Integration of Quantitative and Qualitative Methods in Social Investigations. Ariel. Barcelona. Black, P.J., & Willam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy and Practice, 5(1), 1‐73. Black, P.J., & William, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 1(1), 5‐31. Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brian, mind, experience and School. Washington D.C.: National Academy Press Chen, H., Dun, H. B. L., Phuah, P. S. K., & Lam, D. Z. Y. (2006). Enjoyment or engagement? Role of social interaction in playing massively multiplayer online role‐playing games (MMORPGS). In R. Harper, M. Rauterberg, M. Combetto th (Eds.), Entertainment computing – ICEC 2006: 5 international conference (pp. 262 – 267). Cambridge, UK. Chen, Y., Xu, H., Caramanis, C.,& Sanghavi, S. (2011). Robust matrix completion with corrupted columns. arXiv http://arxiv.org/abs/1102.2254. Csikszentmihalyi, M. (1991). Flow: the psychology of optimal experience. New York, NY: Harper Perennial. de Freitas, S. (2004). Learning through play. Internal report. London:Learning and Skills Research Centre de Freitas, S. & Neumann, T. (2009). The use of ‘exploratory learning’ for supportive immersive learning in virtual environments. Computers in Education, 52, 343‐352. de Freitas, S., & Oliver, M. (2006). How can exploratory learning with games and simulations within the curriculum be most effectively evaluated ? Computers in Education, 46, 249‐264. Denholm, J.A., Protopsaltis, A. and de Freitas, S. (2012). The Value of Team‐Based Mixed‐Reality (TBMR) Games in Higher Education. International Journal of Game‐Based Learning, 3(1), 18‐33, January‐March 2013. Dempsey, J. V., Haynes, L. L., Lucassen, B. A., & Casey, M. S. (2002). Forty simple computer games and what they could mean to educators, Simulation & Gaming, 33(2), 157‐168. Eseryel, D, Guo, Y. & Law, V. (2012). Assessment in Game Based Learning, Foundations, Innovations, and Perspectives, Springer P257. Gee, J.P. (2003). What video games can teach us about literacy and learning. New York: Palgrave‐McMillan. Kebritchi, M. (2008). Effects of a computer game on mathematics achievement and class motivation: An experimental study. PhD thesis. University of Central Florida, Orlando, Florida. Marya (Ed), CGDC conference proceedings (pp 89 – 100). Tampere, Finland: Tampere University Press.

643

John Denholm and Sara de Freitas McKeague, C.A., & Di Vesta, FJ. (1996). Strategy orientations, learner control and learning outcomes: Implications for instructional support of learning. Educational Technology Research and Development, 44(2), 29‐42. Prensky, M. (2001). Digital Game‐based Learning. New York:McGraw‐Hill. Rupp, A. A., Gushta, M., Mislevy, R. J., & Shaffer, D. W. (2010). Evidence‐centered design of epistemic games: Measurement principles for complex learning environments. Journal of Technology, Learning and Assessment, 8(4), 4‐47. Schmit, M. J., & Ryan, A. (1992). Test‐taking dispositions: A missing link? Journal of Applied Psychology, 77, 629‐637. Smith, P., & Ragan, T. (1999). Instructional design. Hoboken, NJ:Wiley. Sundre, D. L., & Wise, S. L. (2003). ‘Motivation filtering’: An exploration of the impact of low examinee motivation on the psychometric quality of tests. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago. Sweetser, P., & Wyeth, P. (2005). Gameflow: A model for evaluating player enjoyment in games. ACM Computers in entertainment, 3(3), 1–24. Taras, M. (2005). Assessment – summative and formative – some theoretical reflections.British Journal of Educational studies, 53(4), 466 – 478. Thomas, D., & Brown, J. (2007). The play of imagination: Extending the literary mind. Games and Culture, 2(2), 149‐172. Van Eck, R. (2006). Digital game‐based learning, it’s not just the digital natives who are restless. Educause Review, 41(2), 16‐30. Zimmerman, B. J., & Schunk, D. H. (2001). Self‐regulated learning and academic achievement: Theoretical perspectives, Mahwah, NJ: Lawrence Erbaum. Zonzack, L. (2002). Computer game criticism: A method for computer game analysis. In F.

644

Understanding ‘Game‐Ness’ Within the SCRABBLE® Family of English Word Games Paridhi Gupta School of Design, Hong Kong Polytechnic, Hong Kong paridhi.Gupta@connect.polyu.hk Abstract: The Scrabble® family of games primarily focus on the nurturing and development of language (E.g. English) vocabulary skills. They are usually designed for 2 or more players and by virtue are competitive and challenging. These word games are different, yet one can recognize them as the member of the same family. No single “essence” can be found among them. Juul (2003, p.35) proposed a classic model for defining the ‘game‐ness’ of a game as, “A rule‐based formal system with a variable and quantifiable outcome, where different outcomes are assigned different values, the player exerts effort in order to influence the outcome, the player feels attached to the outcome, and the consequences of the activity are optional and negotiable.” Games are transmedial, i.e., they are not limited to any single, specific medium or props and can be channelized onto a number of game medias, each having its own strengths and limitations (Juul, 2003). The objective of my paper is to identify and propose a model for the “game‐ness” quality recurrent to the selected 8 variations of Scrabble® word games. I use the word “game‐ness” to symbolize the core features that are necessary and ® sufficient for a game to be a part of the Scrabble® family of word games. My paper investigates eight Scrabble (board and or card) games from Juul’s (2003) three core aspects: the game; the game and the player; the game and the world. For the purpose of study, I have selected the following 8 variations of Scrabble® word games: Scrabble® Alphabet Scoop; Scrabble® Flash; Junior Scrabble®; Scrabble® Upwards; Scrabble® Slam; Scrabble® Dash; Scrabble® Original; and Scrabble® Trickster. This paper forms an integral part of my research study that primarily focuses on learning/teaching of English Language Subject (ELS) through board games within local Primary 4 and Primary 5 classrooms in Hong Kong. Keywords: Scrabble® word games, English vocabulary, word building, anagrams

1. History of Scrabble® During the Great Depression (a financial and industrial recession from 1929 to late 1930s), an unemployed American architect named Alfred Mosher Butts decided to invent a board game. Based on his market research, he categorized games as number games (dice and bingo); move games (chess and checkers); and word games (anagrams and crosswords). Butts wanted to create a game conjunctive of vocabulary skills and alea (chance). Initially, his game was named as LEXIKO, but later, it was renamed as CRISS‐CROSS WORDS. Butts studied the front page of The New York Times in order to analyze the cryptographic structure of the frequency of 26 letters within the English language. He discovered that vowels (a, e, i, o, u) have more frequency in comparison to the consonants. Hence nine vowels’ letter tiles were provided in the game to create frequent opportunities for making new, diverse word combinations. Furthermore, the letter S is often used to make plurals. To prevent the game from being too easy, Butts limited the quantity of ‘S’ letter tiles in the game to four. Thus, his analysis was critically reflected upon the letter‐distribution and the corresponding score‐points for each letter within the 100 letter tiles of the game. For example, the letter z is worth 10 points and shows up only once whereas the letters a and i are each valued at 1 point and show up nine times a piece. The game‐board is a symmetric grid of 15 x 15 (a total of 225 squares) with calculated opportunities for high scoring (double letter, double word, triple letter and triple word). The initial game‐prototypes were made using drafting equipment. This crude and craft‐like aesthetic appeal resisted the established game manufacturers to buy his CRISS‐CROSS WORDS game. Butt’s partner, James Brunot (and a game‐loving entrepreneur) helped him to refine the game’s rules and design and renamed it as SCRABBLE. The word scrabble means "to grope frantically," and was trademarked in 1948. The first SCRABBLE "factory" was situated in an abandoned schoolhouse in Connecticut, wherein Brunot along with his friends made 12 games per hour. Initial four years were a struggle. In 1949, they manufactured 2,400 sets and lost $450. In the early 1950s, after the president of Macy's discovered the game on his vacation, he ordered some for his store. Within a year, SCRABBLE gained immense fame and was rationed to stores across the U.S. By 1952, Butt and Brunot realized they could not supply with enough games to the overwhelming demand of the market. They sold their license to Selchow and Righter Company to market and distribute SCRABBLE in the U.S. and Canada.

645

Paridhi Gupta Gradually, Scrabble word games’ sales gained high momentum. After gaining media limelight (in Television, newspapers and magazines), Scrabble became a “must have game”. In 1972, the trademark SCRABBLE was purchased by Selchow and Righter. In 1986, COLECO Industries purchased Selchow & Righter. However, in 1989, COLECO declared bankruptcy and Hasbro, Inc., a leading U.S game company, purchased SCRABBLE. Currently, HASBRO owns the registered SCRABBLE® trademark in the United States and Canada and elsewhere, the SCRABBLE® trademark is owned by Mattel, Inc. The SCRABBLE® word‐game is sold across 121 countries in 29 different language versions. According to Mattel, about 150 million sets have been sold worldwide, and every one out of three homes in America has the SCRABBLE® game (The Guardian, 2008).

2. Scrabble® word games, play and cognitive development Scrabble® Original is a product of word knowledge, mathematics and probabilities and involves three cognitive abilities (Halpern & Wai, 2007): (a) Verbal ability as word fluency is required for rapid retrieval of appropriate words from memory. (b) Visuospatial ability is required to identify and relate the spatial layout of words and letters on the particular squares, to the probability of scoring more points. (c) Numeric ability is required to calculate the numeric properties associated with different letter and word combinations located in different places on the board. Fleishman (1972) defines ability as an individual’s general trait that is the product of learning and development. Scrabble® Original game starts from the board’s center and outwards. Players exert effort to create new, different, longer, and unusual, legal words (words listed in the Official Scrabble Player’s Dictionary) using visuospatial and numeric abilities in order to attain the intended goal ‐ high numeric scores. Scrabble® Original also nurtures a player’s diverse abilities such as concentration, alertness, speed, memorization, anagramming, words knowledge, word understanding and tile tracking. Anagramming is defined as the ability to rearrange letters to make different words in order to find the best possible play on each move (Fatsis, 2001). Word knowledge is the ability to know whether a given word is a part of the Official Scrabble Player’s Dictionary or not (Halpern & Wai, 2007). In contrast word understanding means to be able to understand the true meaning of any given word. Tile tracking is the ability of the player to keep a track of the letters that have already been played in the game so that the probability of drawing a particular letter on the future rounds can be computed (Halpern & Wai, 2007). Piaget (1951, 1962) relates play as a function of age and cognitive development. The Scrabble® games nurture the different stages of cognitive development based on the player’s age and abilities. In Figure 1 (mentioned below), I have superimposed the forms of play (Smilansky, 1968) and stages of cognitive development (Piaget; 1951,1962) with the corresponding users’ age specific Scrabble® word games. This helps to understand the relationship between the complexity of each game (as a system) in relation to the cognitive abilities and capabilities of the intended user‐groups.

Figure 1: Scrabble® word games and child development Some research studies disagree with Piaget theory regarding the stage‐like cognitive development of children (Case, 1998, 1999, 2000). However, Piaget’s theory can be used as a reference for studying how students

646

Paridhi Gupta process information through attention, memory, and using effective strategies for design of pedagogies (Case, 1987, 1997, 1998) and games.

3. What is “game‐ness”? I use the word “game‐ness” to symbolize the core essential aspects that are necessary and sufficient for a game to be defined as a game and not a toy, or a tool or a learning object. Tools can be defined as devices for clear‐cut, intended, practical use, whereas a toy is a representation of reality through its scale (miniaturization or gigantism), material, form, and or caricature, consequently pronounces itself for a reaction of fantasy (Goldstein, 1994). Zuckerman (2006) defines learning objects as physical objects, specifically designed to promote learning through hands‐on interaction. A game is a rule‐based, problem solving activity, approached with a playful attitude (Schell, 2008). Bernard Suits (1978) defines “games are goal‐directed, engaging activities for its players”. Inspired by Zimmerman and Salen’s (2003) work on game‐definitions, Juul (2003) identifies the ten commonalities among the diverse game‐definitions given by notable game researchers, game‐designers and philosophers (in Figure 2). 10 Commonalities

Game as a formal system

Player & the game

Game & rest of the world

Other

✔

RULES Fixed rules (Huizinga, 1950) Rules (Caillois, 1961; Suits, 1978; Kelley, 1988; Zimmerman & Salen, 2003) Procedure & Rules (Avedon & Sutton‐Smith, 1981) Formal system (Crawford, 1981) OUTCOME Uncertain (Caillois, 1961) Disequilibria outcome (Avedon & Sutton‐Smith, 1981) Changing Course (Kelley, 1988) Quantifiable outcome (Zimmerman & Salen, 2003) "GOALS" Bringing about a state of affairs (Suits, 1978) Opposition (Avedon & Sutton‐Smith, 1981) Conflict or a contest (Crawford, 1981) Involves decision making (Costikyan) Object to be obtained (Kelley, 1988) INTERACTION Interaction (an intricate web of cause & effect) (Crawford, 1981) GOALS, RULES, AND THE WORLD Artificial conflict (Zimmerman & Salen, 2003) "SEPARATE" Outside ordinary life/ Proper boundaries (Huizinga, 1950) Separate (Caillois, 1961) No material gain or interest (Huizinga, 1950) Unproductive (Caillois, 1961) "NOT WORK" Free / voluntary, intrinsic motivation (Caillois, 1961; Suits, 1978) Voluntary control systems (Avedon & Sutton‐Smith, 1981) Recreation (Kelley, 1988) LESS EFFICIENT MEANS Less efficient means (Suits, 1978) SOCIAL GROUPINGS Promotes social groupings (Huizinga, 1950) FICTION Representational/ sub‐set of reality/safe (Crawford, 1981) Make‐believe (Caillois, 1961) A form of art (identified as a form of culture) (Costikyan)

Figure 2: An amalgamation of various Game‐definitions (Juul, 2003) Juul (2003, p.35) defines a game as, “A rule‐based formal system with a variable and quantifiable outcome, where different outcomes are assigned different values, the player exerts effort in order to influence the

647

Paridhi Gupta outcome, the player feels attached to the outcome, and the consequences of the activity are optional and negotiable.” He (2003) suggests that a game has three levels: the game (a contained object) as a formal system, the relationship between the game and the player, and the relationship between the game and the rest of the world. The fixed rules, quantifiable outcomes and goals are the fundamental properties of the game as a formal system (Juul, 2003). Fixed rules means clear and well defined rules, which avoid ambiguity or confusion among the players. Suits (1978) observed that a game has four main elements: 1) the goal, 2) the rules, 3) the lusory means and 4) lusory attitude (game attitude). He (1978) explains that in a game, rules and ends are inseparable. Players can only play/win the game, if they obey the specified rules. These rules (constitutive rules) set certain conditions that filter out the “most efficient” possibilities and favor the less efficient means for achieving an intended goal (Suits, 1978). Rules create opportunities for the players to adopt lusory means (means only permitted by the rules) in order to influence (play and win/attain) the original goal (ibid). These lusory means are narrower in scope than they would have been in reality. The experience of “the end or winning” in a game is the based on what rules have been applied to restrict what means in order to attain a certain goal (ibid). The valorization of variable outcomes creates conflict or contest in games (ibid). Often the player is attached to the outcome as it influences the player’s reaction or behavior as being happy, delighted, dissatisfied, confident, sad, etc (Juul, 2003). The consequences of the game are negotiable and contextual; depending upon when, where, in what spirit and for what reasons is the game being played (ibid). E.g., the practical consequences for an expert player to lose a national/international game of competitive Scrabble is much more intense and serious than losing a casual, living room game of Scrabble. Games are transmedial, i.e., they are not medium‐specific or props‐ specific, but can be transported onto a number of game medias, each having its own strengths and limitations (Juul, 2003). He (2003) summarizes his game‐definition into 6 aspects (Figure 3, below):

Game as a formal system

Player and the game

Game and the rest of the world

Rules

✔

Variable, quantifiable outcome

✔

Valorization of outcomes

✔

Player effort

✔

Player attached to outcome

✔

Negotiable consequences

✔

Figure 3: Juul’s (2003) new game‐definition

4. Investigating “game‐ness” within 8 Scrabble® English word games I use Juul’s insights on game definition (Figure 3) along with the amalgamation of game‐definitions (Figure 2), as the theoretical base for investigating “game‐ness” qualities recurrent to the Scrabble® family of English word games. My objective is to identify and propose a model for the “game‐ness” recurrent to the selected 8 Scrabble® word games. As stated earlier, I use the word “game‐ness” to symbolize the core features that are necessary and sufficient for a game to be a part of the Scrabble® family of word games. For the purpose of investigation, I selected the ® ® ® following 8 Scrabble® word games: Scrabble Original, Scrabble Alphabet Scoop, Scrabble Flash, Junior ® ® ® ® ® Scrabble , Scrabble Upwards, Scrabble Slam, Scrabble Dash and Scrabble Trickster. Each game has been played 53 times (15 times by different users within the target user group ages 6‐10, as specified by the game and self‐played 38 times). The Figure 4 (mentioned below) describes the attributes observed within the diverse Scrabble games:

648

Paridhi Gupta GAME AS A SYSTEM

8 Diverse Scrabble® English Word Games

O R I G I N A L

U J S F P U C L W N O A A I O S R O P H D R S

Modifiable Rules No Rules

T R I D S C A L K S A S H M T E R

✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ DIVERSE RULES in SCRABBLE GAMES

Fixed, Constitutive Rules

RULES

✔

✔ ✔

3D letter stacking

Change words by replacing letters

✔

Change words by adding letter(s) at one/both ends

✔

✔ ✔ ✔

Make plurals

✔

✔ ✔

✔ ✔ ✔

Exchange letters with other players

✔

Exchange racks with other players

✔

✔ ✔

✔ ✔ ✔

✔

Exchange letters for new ones Make repetitive words

MEANS

Earn opponent’s score

✔

✔ ✔

Words can spell left to right (horizontally)

✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔

Words can spell right to left (backwards)

✔

Words can spell top to bottom (vertically)

✔

✔ ✔

✔

Words can spell bottom to top (vertically)

✔

Start from the center on the board and outwards

✔

✔ ✔

✔

Start from anywhere on the board

✔

Randomly choose letters ✔ ✔ Already provided with random, pre‐determined letters Players take turns to play Players play simultaneously Use of inefficient means Use of efficient means MEANS in Scrabble games (Not Medium) Play with 7 number of letters at one time Play with less than 7 number of letters at one time Play with more than 7 number of letters at one time Blank tiles Double letters (ligatures) Means for Bonus (value addition) Means for Penalty (loss) Same letters in hand throughout entire game‐play Calculated, diverse distribution of the number of letter tiles based on the frequency of letters in English language and to maintain a level of difficulty for target age groups Make 2 letter words Make 3 letter words Make 4 letter words Make 5 letter words Make 6 letter words Make 7 letter words

649

✔

Paridhi Gupta Make 7+ letter words Identical distributions of frequencies of letters Valorization of individual letters No valorization of letters Symmetric, valorized grid Empty grid No grid Common game space Individual game space Alea (Chance) Agon (competition, challenge) Specific (same) outcome Variable outcomes Quantifiable outcome Valorization of outcome

PLAY FORMS

OUTCOME

Board game

✔ ✔

Card game

Computer game

✔

✔ ✔ ✔

Technology driven tangible game

✔

8 Diverse Scrabble® English Word Games

O R I G I N A L

U J S F P U C L W N O A A I O S R O P H D R S

T R I D S C A L K S A S H M T E R

Single Player

✔

MEDIUMS

✔

✔ ✔

THE GAME AND THE PLAYER

NUMBER OF PLAYERS

GOALS, PLAYER EFFORT

INTER‐PLAYER INTERACTION PLAYER ATTITUDE PLAYER ATTACHMENT TO THE OUTCOME DURATION OF GAME‐

✔

2 Players

✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔

3‐4 Players

✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔

5‐6 Players

✔

Make unusual words

✔

✔ ✔ ✔

✔ ✔

Make long words

✔

✔ ✔ ✔

Make more words

✔

✔ ✔ ✔

✔ ✔

Finish the game first (speed / time)

✔

Make more words in limited time (speed)

Make limited and specific words

✔

✔ ✔

✔

✔ ✔

✔ ✔ ✔ ✔ ✔ ✔

Build crosswords

✔

✔ ✔

✔

Fill in the blanks (of the words) with letters

✔

Create multiple words in a move (through adjacency) Hunt for specific letters Score more points More Interaction (2+ players) Role reversal No Interaction Fun‐oriented, immersive engagement Serious, Intense engagement Yes No Time‐bound game

650

Paridhi Gupta PLAY

Short (0‐20 minutes) Medium (21‐40 minutes) Long (41‐ 60 minutes) Seemingly never ending (More than 61 minutes)

The GAME and the rest of the WORLD

8 Diverse Scrabble® English Word Games

U J S F P U C L W N O A A I O S R O P H D R S

T R I D S C A L K S A S H M T E R

Word knowledge

✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔

Word understanding

✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔

Anagramming abilities

✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔

Visuospatial abilities

✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔

Numeric skills

IMPACT OF GAMEPLAY ON THE PLAYER

CONSEQUENCES OF GAME‐PLAY

O R I G I N A L

Learn new words

✔

✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔

Memory skills

✔

Motor skills and reflex

Attention, concentration Verbal abilities Sensitivity towards spelling Serious and Intense Competitions (National & International Tournaments) Facilitates separate Social groupings Collaborative learning at homes / schools Affinity towards learning new words More affinity towards rhyming words More affinity towards alliteration (Alliteration occurs when a series of words have the same first consonant sound. E.g. claw, clad, clap, clop, clot) More affinity towards verb tenses (with alliteration) Habit of consulting dictionary for help Representational and fun‐oriented way of learning Creates opportunities to play variations of the game

✔ ✔ ✔

✔ ✔ ✔ ✔ ✔ ✔

✔

Figure 4: Game analysis across 8 diverse Scrabble word games

5. Observations and findings The Figure 4 mentioned above draws light on the following core aspects that define “game‐ness” within Scrabble family of English word games:

Defined by constitutive, fixed rules

Use of inefficient means

Variable and quantifiable outcomes

Valorization of outcomes

Players are attached to the outcomes

The consequences of game play are negotiable.

Scrabble games are primarily designed for 2‐4 players.

651

Paridhi Gupta

Through diverse lusory means and constitutive rules, scrabble games facilitate the following learning (cognitive) goals: attention, memory, concentration, verbal abilities word knowledge, word understanding, anagramming, visuospatial abilities, sensitivity to spellings, and motivates players to learn new words search. All Scrabble games share similarities in their intended learning/educational goals and encourage collaborative learning in home and schools environments. They are a fun representational tool for learning and enhancing English word building and vocabulary skills. Scrabble games are a conjunction of alea and agon.

Building 4 letter words horizontally from left to right is the common intended goal found in all Scrabble word games.

The games have pre‐determined diverse distribution of letter tiles based on the frequency of letters in the English Language and to adequately serve the intended goals in each game.

The findings mentioned above, highlight that “game‐ness” within the Scrabble family of Word games is quite generic and similar to Juul’s (2003) game definition. It is difficult to identity any one specific attribute that is common to all the Scrabble games mentioned above. In order to identify inter‐relationship among the selected 8 Scrabble word games; I used Agglomerative Hierarchical Clustering (AHC) analysis as a method to analyze the above data. The number of entities (n) is 8 (Scrabble word games) and the nominal coding of 106 variables is yes=1, no= 0. The Figures 5(a) & (b), mentioned below provide statistical analysis of the gathered data and the visual representation of inter‐ relationship between the 8 Scrabble word games through a dendogram: Variables Modifiable Rules No Rules Fixed, Constitutive Rules Earn opponent’s score 3D letter stacking Change words by replacing letters Change words by adding letter(s) at one/both ends Make plurals Exchange few letters with other players Exchange racks with other players Exchange letters for new ones Make repetitive words Words can spell left to right Words can spell right to left Words can spell top to bottom Words can spell bottom to top Start from centre of the grid and outwards Start from anywhere on the grid Randomly choose letters Provided with random, pre‐determined letters Players take turns to play Players play simultaneously Use of inefficient means Use of efficient means Play with 7 number of letters at one time Play with less than 7 number of letters at one time Play with more than 7 number of letters at one time Blank tiles Double letters (ligatures) Means for bonus points Means for Penalty (loss) Same letters in hand throughout entire game Calculated, diverse distribution of number of letter tiles Make 2 letter words Make 3 letter words

Minimum 0.000 0.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000

Maximum 1.000 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

Mean 0.125 0.000 1.000 0.125 0.250 0.500 0.625 0.750 0.125 0.125 0.750 0.250 1.000 0.125 0.500 0.125 0.500 0.125 0.875 0.125 0.625 0.375 1.000 0.000 0.750 0.500 0.250 0.625 0.125 0.750 0.625 0.250 1.000

Std. deviation (SD) 0.354 0.000 0.000 0.354 0.463 0.535 0.518 0.463 0.354 0.354 0.463 0.463 0.000 0.354 0.535 0.354 0.535 0.354 0.354 0.354 0.518 0.518 0.000 0.000 0.463 0.535 0.463 0.518 0.354 0.463 0.518 0.463 0.000

0.000 0.000

1.000 1.000

0.750 0.875

0.463 0.354

652

Paridhi Gupta Make 4 letter words Make 5 letter words Make 6 letter words Make 7 letter words Make 7+ letter words Identical distributions of frequencies of letters Valorization of individual letters No Valorization of letters Symmetric, valorized grid Empty grid No grid Common game space Individual game space Alea (Chance) Agon (competition, challenge) Specific (same) outcome Variable outcomes Quantifiable outcome Valorization of outcome Board game Card game Computer game Technology driven tangible game Single Player 2 Players 3‐4 Players 5‐6 Players Make unusual words Make long words Make more words Finish the game first (speed / time) Make more words in limited time (speed) Make limited and specific words Build crosswords Fill in the blanks (of the words) with letters Create multiple words in a move (through adjacency) Search for specific letters Score more points More Interaction (2+ players) Role reversal No Interaction Fun‐oriented, immersive engagement Serious, Intense engagement Player attached to the outcome Player not attached to the outcome Time‐bound game Short (0‐20 minutes) Medium (21‐40 minutes) Long (41‐ 60 minutes) Seemingly never ending (More than 61 minutes) Improves Word knowledge Enables Word understanding Improves anagramming abilities Improves visuospatial abilities Improves numeric skills Learn new words Improves memory skills Improves motor skills and reflex Improves attention, concentration

1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 1.000 0.000 1.000 1.000 1.000 0.000 0.000 0.000 0.000 0.000 1.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

1.000 0.875 0.625 0.625 0.500 0.375 0.250 0.625 0.375 0.125 0.625 0.750 0.250 1.000 1.000 0.000 1.000 1.000 1.000 0.625 0.250 0.500 0.125 0.375 1.000 1.000 0.125 0.750 0.625 0.750 0.375 0.375 0.750 0.500 0.250 0.500

0.000 0.354 0.518 0.518 0.535 0.518 0.463 0.518 0.518 0.354 0.518 0.463 0.463 0.000 0.000 0.000 0.000 0.000 0.000 0.518 0.463 0.535 0.354 0.518 0.000 0.000 0.354 0.463 0.518 0.463 0.518 0.518 0.463 0.535 0.463 0.535

0.000 0.000 0.000 0.000 0.000 1.000 0.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 1.000 0.000 0.000 0.000 1.000 0.000 0.000 1.000

1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 0.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

0.125 0.750 0.875 0.125 0.125 1.000 0.125 1.000 0.000 0.250 0.375 0.125 0.375 0.250 1.000 1.000 0.875 0.750 0.625 1.000 0.875 0.625 1.000

0.354 0.463 0.354 0.354 0.354 0.000 0.354 0.000 0.000 0.463 0.518 0.354 0.518 0.463 0.000 0.000 0.354 0.463 0.518 0.000 0.354 0.518 0.000

653

Paridhi Gupta Improves verbal abilities Sensitivity towards spelling Serious, Intense Tournaments Facilitates separate Social groupings Collaborative learning at homes / schools Affinity towards learning new words More affinity towards rhyming words More affinity towards alliteration More affinity towards verb tenses (with alliteration) Habit of consulting dictionary for help Representational, fun‐oriented way of learning Creates opportunities to play variations of the game

1.000 1.000 0.000 0.000 1.000 1.000 0.000 0.000 0.000 0.000 1.000 0.000

1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

1.000 1.000 0.125 0.625 1.000 1.000 0.375 0.375 0.375 0.875 1.000 0.250

0.000 0.000 0.354 0.518 0.000 0.000 0.518 0.518 0.518 0.354 0.000 0.463

Figure 5(a): Statistical analysis of Agglomerative Hierarchical Clustering

Figure 5(b): Dendrogram for selected 8 Scrabble Word games

6. Conclusion My research paper draws preliminary light on the inter‐relationships between the selected 8 Scrabble word games. It would further help academic researchers and game designers to explore possibilities for diverse constitutive rules and lusory means for creating new, interesting, and challenging conflict in Scrabble games. For instance, what happens to the Scrabble game‐play wherein certain English letters can be used in reflection (horizontally or vertically)? If “w” can be used as “m”; “b” as “d”; “q” as “p” “u” as “n”; then how would it affect the letter distribution and game play? Design a Scrabble word game that uses letters having recurrent ligatures in English language (st, Qu, Th, ch, fi, etc). This paper forms an extended part of my research study that primarily focuses on learning/teaching of English Language Subject (ELS) through board games within local Primary 4 and Primary 5 classrooms in Hong Kong.

References: Avedon, E.M. & Sutton‐Smith, B. (1981). The Study of Games. John Wiley & Sons, Inc., New York. Bateson, G. (1972). A Theory of Play and Fantasy. University of Chicago Press. Berger, A.A. (1998). Media research techniques. California: Sage Publications. Caillois, R. (1961). Man, play & games. Chicago: University of Illinois Press. Costikyan, G. (1994). I Have No Words & I Must Design. Retrieved from http://www.costik.com/nowords.html Crawford, C. (1982). The Art of Computer Game Design. Retrieved from http://www.vancouver.wsu.edu/fac/peabody/game‐book/Coverpage.html Goffman, E. (1972). Fun in Games. The Penguin Press, New York. Goldstein, J.H. (1994). Toys, Play and Child Development. New York. Cambridge University Press.

654

Paridhi Gupta Harpen, D.F. & Wai, J., (2007). The World of Competitive Scrabble: Novice and Expert Differences in Visuospatial and Verbal Abilities. Journal of Experimental Psychology: Applied, 13(2), 79 –94. Retrieved from http://www.psychologytoday.com/files/attachments/56143/the‐world‐competitive‐scrabble.pdf (Accessed on 2013‐ 07‐12). Hasbro website http://www.hasbro.com/scrabble/en_US/ (Accessed on 2012‐12‐12). Huizinga, J. (1950). Homo Ludens. The Beacon Press, Boston. Kelley, David: The Art of Reasoning. W. W. Norton & Company, New York, 1988. Juul, J. (2003): "The Game, the Player, the World: Looking for a Heart of Gameness". In Level Up: Digital Games Research Conference Proceedings, (Ed. M. Copier and J. Raessens), 30‐45. Utrecht: Utrecht University. Retrieved from http://www.jesperjuul.net/text/gameplayerworld/ (Accessed on 2012‐12‐16). McDevitt, T.M., & Ormrod, J.E. (2007). Child development and education (3rd ed.). Upper Saddle River, NJ: Pearson. Salen, Katie & Zimmerman (2003). Rules of Play ‐ Game Design Fundamentals. MIT Press, Cambridge. Schell, J. (2008). The Art of Game Design. Carnegie Mellon University: Morgan Kaufmann. "Spell bound". London: The Guardian. 2008‐06‐28. http://www.guardian.co.uk/lifeandstyle/2008/jun/28/healthandwellbeing.familyandrelationships Retrieved 2012‐12‐ 12. Suits, B. (1978). The Grasshopper. University of Toronto Press, Toronto. Suits, B. (1995)."Tricky Triad: Games, Play, and Sport". In: Morgan, William J. & Meier, Klaus V. (eds.): Philosophic Inquiry in Sport. 2nd ed. Human Kinetics, Champaign, Illinois. Suits, B. (2005) Construction of a Definition. In The Grasshopper: Games, Life and Utopia. Retrieved from http://www.child‐ encyclopedia.com/documents/Smith‐PellegriniANGxp.pdf Wrench, J.A. (2008). Quantitative Research Methods for Communication. Oxford Press. XLSTAT software. Retrieved from http://www.xlstat.com/en/ Zuckerman, O. (2006, in preparation). Historical Overview and Classification of Traditional and Digital Learning Objects. Cambridge, MA: MIT Media Laboratory. Retrieved from http://llk.media.mit.edu/courses/readings/classification‐ learning‐objects.pdf Referred Games: Scrabble® Original, Scrabble® Alphabet Scoop, Scrabble® Flash, Junior Scrabble®, Scrabble® Upwards, Scrabble® Slam, ® ® Scrabble Dash and Scrabble Trickster.

655

Interactive Story as a Motivator Element in an Educational Video Game José Rafael López‐Arcos, Francisco Luis Gutiérrez Vela, Natalia Padilla‐Zea and Patricia Paderewski University of Granada, Granada, Spain jrlarco@ugr.es fgutierr@ugr.es npadilla@ugr.es patricia@ugr.es Abstract: In our previous works, we made progresses towards defining a method to develop and evaluate educational video games (EVG) paying separate attention to Educational and Recreational Levels. In this paper, we analyse storytelling as a motivational context to enrich the effectiveness of the Educational Level of the video game. Due to the importance we give to storytelling, this paper is intended to move towards the development of a method to evaluate the story during the video game development process. For this purpose, we define a series of tasks that are based on the classification and structuration of the story of a video game. Keywords: educational video games, storytelling, game based learning

1. Introduction Currently, there is a general interest in both the integration and the study of emotions in the video games development process, and the importance and use of digital storytelling in educational processes as well as the relation between these two concepts. Driven by these trends, we performed the design of a method to analyse emotions produced by specific aspects of the game. The implementation of this method was carried out with an educational video game (EVG) designed and developed in our research group. In that experience, we analysed emotions aroused in 3‐7‐years‐old children. This experience helped us also to detect certain deficiencies in the story (Padilla‐Zea 2012) and led us to focus our next study in the field of storytelling in educational video games and the integration of the narrative aspect of a video game with its educational and recreational components. Given the need for a tool which allows us to assess whether the story of an EVG works according to the requirements, we found the formalisation of the story as a good solution. To do this, we performed an abstraction of the story that will be integrated into the game and will represent it as a hierarchical structure of scenes. Such scenes are grouped into sequences and chapters, depending on the narrative evolution represented both by the scenes themselves and by their relation with other scenes. This abstraction also allows representing the plasticity proper of an interactive story, which can dynamically change its development pursuant to the decisions or skills of a particular player, but also allows making restrictions in order to maintain consistence of the story and to keep a classical narrative structure (Vogler 1998). In order to fulfil the previous requirement, it is necessary to evaluate the story of an educational video game during the development process in order to make sure that it works as a motivator element. To do this, we will consider the model of interactive story above and our previous experience in the field of evaluation of video games. Since the narrative aspect of an educational video game is generally one of its main motivational elements, formal assessment throughout the development process is a key to obtain a final product which meets the expected requirements. For proper analysis of storytelling, it is necessary to abstract a structure that represents a story and be able to classify the types of stories that can be found in video games. The rest of the paper is structured as follows: In sections 2 and 3, we provide a theoretical basis on the classification and structuration of stories embedded in a video game. In Section 4, we describe a set of tasks that helps to evaluate a story during the game's development. These tasks are based on the concepts described in Sections 2 and 3 and our EVG representation model. Finally, in section 5, our conclusions and future work are outlined.

2. Classifying video games according to narrative content In previous work (Padilla‐Zea 2013), we were interested in video games aimed at players aged 3‐7 years old. The aim of this study was to analyse the need of these systems, the problems that existed and the direct or

656

José R afael López‐Arcos et al. indirect relationship to the existence of a narrative in the game. We performed an analysis of EVGs registered at Apple's App Store. The classification offered by this market helped us to select and analyse the most representative games from each type: pure fun, games, educational specific content and skills. This classification does not follow criteria related to the story introduced in the game and therefore it is not useful for our purpose. In order to being able to tackle the analysis of the story more correctly, we classify video games in two ways: according to the importance of its plot and according to its story linearity

2.1 Classifying video games according to the importance of the plot in the game experience The importance given to the narrative components in video games can be used to perform a classification process (Belinkie 2011) because it gives us indirect information about the size and the type of story that we face. Traditionally, video games used to consist only on skill proofs. They could test reflexes, ability to solve puzzles, or capacity to think strategically. Overall, fun caused by the game was based on the exercise of overcoming challenges or getting the highest score. However, in the last 20 years it has been seen a growth in the emphasis on the narrative section of the game. Players wanted narratives increasingly sophisticated and technology increasingly provides most appropriate means to achieve this end. Consequently, now fun in games is not based entirely on solving puzzles and challenges, but the narrative and the atmosphere give much of the experience. Thus, games have become skill tests which transmit a plot. However, while some games give more importance to the story, others give it to the challenges. The proportion in which these elements are combined in different games is what gives rise to the following classification:

No plot. Games like Tetris or Minesweeper does not establish enemies or characters at all and there is no story.

Tiny story sometimes limited to a few of lines of text. The characters can be anthropomorphized. Usually, the story is described at the beginning to make the game be in the context but it does not influence in the playability.

Little plot but well‐crafted atmosphere. An important effort is made to keep the consistency of the elements of the game (characters, scenarios, goals…) with the story that is being integrated with the game.

The story is constructed by the actions the player performs and gives meaning to them. Normally, the hero must get a main objective that requires reaching a certain number of partial goals.

A comprehensive but simple story is revealed gradually throughout the game.

Frequent scenes tell a relatively complex story with many characters.

The plot is very complex and takes up most of the playing time. In this category and the next ones it is important to remark that the most fun part generated depends directly on the story and how it evolves along time.

The game mainly intended to tell a story and even the action scenes may be optional.

The game is a playable story and the controller is only used to advance the plot.

We focus primarily on video games with educational component. We think that any game can be classified into one of these categories regardless of how much it extends its educational content. However, it is very difficult to find an educational video game that presents a complex plot and most of them are in the first, second or third category. Finally, it is important to mention that a complex plot does not have to cause great fun. In fact, in many cases a very simple story or a simple setting can cause a high degree of motivation regardless of how attractive the challenges of the game are.

657

José R afael López‐Arcos et al.

2.2 Lineal and non‐lineal stories Another way to classify stories in video games in order to permit a more accurate assessment is according to their linearity. Storytelling can be linear or non‐linear (Pearce 1994). A linear story is received by the player exactly as the author designed it. All parts of the story are supposed not to change in any case. By contrast, when a story is non‐linear it can be changed somehow in a dynamical way. All elements in a linear story are embedded in the game, because they have been inserted by the designer. Either the player or any other external element can change an embedded story. However, in a video game there is usually a story that emerges thanks to the player's actions. For example, this emergent story arises when the player goes to an area of the game environment by his own choice, when he loses or wins a challenge or when he gets an item. In addition, the computer can produce an emergent story that not depends on the writer or the actions of the player as occurring in a game with a complex virtual environment. Most of the events of the game cannot be classified as emergent or embedded storytelling but at an intermediate point between them. For example, the player may direct the main character’s actions or there may be other characters controlled by the computer, but these events occur in a context designed by the creators of the game. Therefore, each element of the story may be more or less emergent and also embedded simultaneously (Glassner 2004).

2.3 Benefits of Including story in an educational video game After analysing and classifying the different ways to integrate the story in a video game and according to the conclusions of our previous work (Padilla‐Zea 2013), we can give a list of benefits that can be obtained using the story, which let us to see what aspects should be analysed in a video game to see the efficiency of the story in it. Keep the player’s attention based on his curiosity.

Contextualize a set of ludic and educative activities inside a story.

Facilitate the coherence of the different elements of the game (character, scenarios, age...).

Let the synergy between players and characters or video game situations along all the game.

Facilitate the interaction with the game generating self‐determination process in the players (story modified with emergent elements), which in the most cases generates a substantial increase of motivation.

Generate “identity” around the game and its elements.

These benefits will manifest differently depending on the type of integration of the story in the video game. For example, in an educational video game for kids is common to use characters that teach the player how to perform the activities. Funny characters cause enjoyment to children, especially if they are quite young. In that case, entering a short story at the beginning of the game to introduce these characters facilitates the integration of the activities with the game and increases the degree of fun that the players experience.

3. Structuring the story Due to the fact that the structure of a story can be complex, it is necessary to analyse each part separately. For this purpose, the story should be pre‐structured. Indeed, in the case of educational video games, the educational content is integrated with the ludic ones and with the story itself making the analysis more complicated. A possible structure that can be observed in a story is its division into different elements. To do this, we can draw on the language of cinema (Fernández 1979), which divides stories in different units of narration. In addition, we can separate the story into an optional part and a required part. Another way to structure the story is divide it into different stages.

658

José R afael López‐Arcos et al.

3.1 Narrative units In a literary work there are sentences, paragraphs, scenes, chapters and parts. A movie has its own units of storytelling. The shot is the first one. Each shot of a film corresponds to a sentence. A set of phrases forms a scene if they tell something continuous, for example, a conversation between two characters. Several scenes that represent an unit, form a sequence. The scenes are different but all together or properly ordered make sense, a unity. For example, on a long chase of an action movie, many scenes happen (stealing a car, jumping over a bridge, going through a window...) which together form a sequence. There are also alternative actions. In the example above, the chase scene can be told mixing shots from persecutors and persecuted. Do not confuse the alternative images with parallel images. In parallel images there is no relationship between them at a glance, but comparing both images creates an idea. We have said that a shot is equivalent to a sentence, but sometimes there is a single shot in which the characters move for some time and the camera follows them forth. It is equivalent to a paragraph in written language. This paragraph shot is called sequence shot in the language of film. In short: A film (in the case of video games, a story) consists of shots, scenes and sequences. Based on this structure, we have determined that the narrative structure of a video game includes: stories, in turn composed by chapters, sequences and scenes. A game may consist of different stories. At a level of styling in the transmission of story, we can also analyse the shots that form a scene from a video game. This study is beyond the scope of this paper but will be addressed in the future in our research.

3.2 Optional or required story A video game may contain parts of story which is not mandatory to play. There is always a main plot, but can arise scenes or even entire stories that the player directly or indirectly decides whether to play or not. We call stories, chapters, sequences and scenes of this main plot required. All others will be optional or non‐ required. Using optional parts in story allows that not all players have to receive the story the same way. Thanks to this, players can carry out educational activities and it is possible to display educational content personalized to each player. In learning, personalization is an essential requirement as each student requires different speed and support to the learning process.

3.3 The hero’s journey According to (Vogler 1998) in all narrative compositions a number of common elements somehow appear. The author calls this narrative structure the hero’s journey and describes it by a set of stages that abstract the various milestones typical of a storyline: (1) The Ordinary World; (2) The Call to Adventure; (3) Refusing the Call; (4) The Mentor; (5) The First Threshold; (6) The Journey/Tests, allies, enemies; (7) The Final Dungeon; (8) The Great Ordeal / The Odyssey; (9) The Prize; (10) The Road Home; (11) The Return. This theory is based on a thorough analysis of all kinds of stories. Currently, it is often used as a guide to develop screenplays. This structure provides valuable information about what is classically considered as a well‐constructed story. Therefore, it gives us very useful information when analysing the plot of a video game. It’s important to realise the hero’s journey could not be followed literally. The hero’s journey is a set of guidelines. Although many good stories follow the hero’s journey pretty closely, many others veer wildly off course and still produce excellent results. Furthermore, as said in (Lebowitz 2011), the hero’s journey isn’t the only structure, the author must feel free to find another more appropriate structure or even create one of his own.

3.4 Archetypes As was the case with the stages of the Hero's Journey, in (Vogler 1998) the author proposes a set of models that can help to create consistent characters. These models, which are called archetypes, are not supposed to

659

José R afael López‐Arcos et al.

Authors of (Lebowitz 2011) evolve the previously explained theory and propose a set of more specific archetypes to the world of video games. Here we describe the archetypes that we consider most interesting in both theories:

The Hero may appear in two forms:

The Young Hero is eager to prove him or herself and go on an adventure. He is enthusiastic, optimistic, cheerful and philanthropist.

The Reluctant Hero is forced to act by circumstances beyond his control. Although initially he is not a nice guy and he avoids helping others if it is a burden, often he changes his attitude throughout story.

The Best Friend balances the hero's actions.

The Special Person can be on the side of the hero or be against him. He is a lonely person and he has some special power that others try to steal.

The Herald often delivers the Call to Adventure and he announce the coming of significant change or the need of changes in the hero's life.

The Mentor usually provides a gift or tool to the hero that will be helpful to complete his adventure.

The Veteran experienced the same kind of tasks that the hero must complete. He sometimes tends to act as a mentor.

The Gambler: Selfish and risky, luck is always on his side. He shows ambiguity in demonstrating if he is on the side of the hero or the villain and he easily changes sides to ensure his success.

The Seducer: Greedy and opportunistic, he will use his charms to get what she wants.

The Threshold Guardians are characters who are a threat to the hero and an obstacle to progress in the story. They are usually displayed as the henchmen of the villain.

The Hardened Criminal can get any item or any work but always through questionably legal methods.

The Cold, Calculating Villain (The Shadow) is smart and ruthless. His plans are large and complex and he has never hesitated to make any sacrifice or to eliminate anyone. The function of the shadow is to challenge the hero and give him a worthy opponent.

The Tricksters affect the lives of others characters but are unchanged themselves. They draw attention to the imbalance or absurdity of a stagnant situation and often provokes laughter.

In conclusion, archetypes allow defining characters in a coherent way and they help the players to identify with the characters.

3.5 The interest curve and the flow engagement There are a lot of moments in a story. All of these moments contribute to generating an emotional response. As seen in (Lebowitz 2011), it is called “pace” to the continuous contrasts between these moments. In (Schell 2008), he talks about the “Interest Curve”. This concept is used to describe the design of the game wherein the player gets changes in his emotions moment by moment. An important aspect of storytelling is that every scene, every act and every sequence must have an attack, a complication, a crisis, a climax and a resolution. It is related to the concept of flow. Csikszentmihalyi (1997) says that when someone is doing the same thing at the same level for long time, he becomes bored or frustrated, and the need to try our best arises.

660

José R afael López‐Arcos et al.

Figure 1: Story structure as a five‐act emotional journey (Lebowitz 2011)

4. Evaluating a story along the develop of an educational video game Previously in this paper, we have explained various concepts that help us to better understand how to tell a story. In sections 4.1, 4.2, 4.3 and 4.4, we describe a set of task that will help the author to apply this knowledge in order to analyse and evaluate a story during the design and development of a video game.

4.1 Choosing the correct kind of video game story In section 2.1, 9 levels to describe the different ways to integrate a story in a video game have been detailed. It is important to note that this classification does not consider the complexity of the plot. A video game can tell a simple story, but be classified in the highest level. For example, a video game that tells a fairytale. We propose the use of this classification as an aid to decision‐making in the game design. Because our focus is Educational Video Games, we must face the question: What kind of game is more appropriate for the Educational Content I want to convey? The educational content that requires solving exercises and tasks require a game that combines challenges and plot. However, there are many concepts that are transmitted better if it is given more importance to the story. It is very important to note how old the players are and to know what the best way to convey the content is.

4.2 Building the EVG structure EVG have three main components: the ludic component, the educational component and the narrative component. To formalize this structure and facilitate the design of these three components together, we worked on the definition of a design model (Padilla‐Zea 2013). In Figure 2, the conceptual model of the narrative component of a video game is shown. As can be seen, the design is based on the narrative units described in section 3.1. Thanks to this, the representation of the story can be performed intuitively. A representative example of a story can be seen in Figure 3. In (Padilla‐Zea 2013) we describe how to use the items shown in this diagram. It represents all narrative units of story, their relationships and their binding order. The order in which the scenes happen in the story is marked by the arrows between them. The diagram scrolls through the scenes in several different ways. These different paths correspond to different ways in which the player gets the story, because the order of the scenes is different. We have also defined conceptual models for educational and ludic content (Figure 4). The different educational objectives of the game are organized in a hierarchical structure as well as recreational activities. With this design of the video game in three structures, it is easier to relate each specific educative task with one or several scenes of the story and with one or several ludic activities. Table 1 shows an example of how this model allows organizing in an affordable way the large number of elements found in the three facets of an educational video game (Padilla‐Zea 2013). The sample is a small portion of the storyboard that was included in the Game Design Document of our game "Ato's Adventure," which was aimed at children aged from 3 to 5 years. Each scene of the game has educational content, ludic content and story content. This structure greatly facilitates us the process of analysing the integration of the rest of the game story.

661

José R afael López‐Arcos et al.

Figure 2: Storytelling conceptual model (Padilla‐Zea 2013)

Figure 3: Example of the structure of a chapter of the story from “Ato’s Adventure”

4.3 Analysing the complexity of the story Including emergent elements in the story increases its complexity. The complexity of the story can be detrimental to the transmission of knowledge if it causes a distraction. However, it is possible that it will be beneficial if the game adapts itself to the player's evolution with changing the plot according to the educational content that is most appropriate to reinforce at the time. Again, we have to decide how to design the story considering the educational content. Thanks to the model described in section 4.2, we can both represent and quantify this complexity. Two factors help us in this task:

662

José R afael López‐Arcos et al.

The linearity of the story can be seen in the number of threads or possible paths that can be followed in the representation of a story. A lineal story will have a single possible path and will not allow the game to suit the player's needs.

The amount of optional scenes. Many optional scenes help eliminate educational content dynamically.

Figure 4: Educational (a) and Ludic (b) content conceptual models (Padilla‐Zea 2011) Table 1: Portion of the Storyboard from the EVG “Ato’s Adventure” (Padilla‐Zea 2013) Chapter 1: Ato’s World Sequence 1.1 Scene

Educational Content

Ludic Content

Story Content

SC1.1.1

Introduction of Ato and his friends

Illustration

Helping Ato to reach de sandbox in which he can play

SC1.1.1

Ato meets Atalita in the sand box

SC1.1.1

Helping Ato’s friends to reach the other side of the swing

Ato and friends spend a while playing at the fun park

4.4 Analysing if the story is well built As described in section 3.3, The Hero’s Journey helps us to know if a story is considered to be well built and how well are the characters designed because it is a classical and widely accepted structure. Specifically in the

663

José R afael López‐Arcos et al. case of Educational Video Games, when the author writes the story he can observe the different stages of The Hero’s Journey and wonder: Is this stage useful to my plot? What contribution would include this stage in my story? Is this stage an obstacle to learning the educational content that I want to include? Once the model of a story has been built by the designer (as seen in Figure 3), each scene can be labelled with its corresponding stage number. The scenes of each stage can be counted and the percentage with which each stage is represented in the story can be calculated. By this analysis, the story in the video game can be improved throughout the development process. A similar process can be followed to analyse the characters in the game. The author can reflect about each archetype (Has this archetype been represented in this story? Is it essential or appropriate for my story?) or about each character: (Does the character have a justified role in the story? Would it be better to unify this character with another who performs the same function and simplifying the story? Could the character be divided in two or more different characters?). In the case of Educational Video Games, the story and the characters should be designed with the purpose of keeping motivation along the video game. Furthermore, they should help to convey the desired educational content.

5. Discussion We have described a set of tasks that can help a developer to evaluate his game taking into account the narrative. These tasks are based on both the classification of video games depending on their story, and in different ways to structure the narrative. In order to improve and develop the design of these tasks, it is necessary to delve into how to apply in practice the theoretical concepts on which we have based. These concepts won't be equally relevant in all cases. It is therefore interesting to explore how different modifiers influence for studying the story of the video game. For example: the age of the players or the educational content and recreational tasks that are going to be included in the game Furthermore, the theoretical basis that has been studied is based on the narrative of the film and literature: narrative units, the hero's journey, archetypes and the curve of interest. It is interesting to extend these theories to make them more specific to the world of videogames and, thus, more useful to the developer for evaluation of their product.

6. Conclusions and future work Storytelling is becoming increasingly important in video games. Due to this fact, we have realized that we need a method to quantify how good our story is and how well it adapts to the needs of our educational video game. The story analysis includes its classification and structuration. For this purpose, we have done a set of tasks that will help the author to evaluate the plot of his game while he is developing his educational video game. Our immediately future work is the improvement of the method and the realisation of a style guide to the development of the story well built that works as a motivational element in the educational video game. In addition, we are intended to extend the evaluation method focusing in the player through experiences with the users. We can base on our experience in previous works with the evaluation of emotions using tests (Padilla‐Zea 2012).

7. Acknowledgements This work is financed by the Ministry of Science & Innovation, Spain, as part of VIDECO Project (TIN2011‐ 26928), Iberoamerican Network support the teaching and learning of professional competences through collaborative and ubiquitous environments (CYTED ‐ 513RT0481) and Excellence Project P11‐TIC‐7486 financed by Junta de Andalucía.

References Belinkie, M. “The Video Game Plot Scale”, [online]. August 30th, 2011. Available on the Web: http://www.overthinkingit.com/2011/08/30/video‐game‐plot‐scale/ Csikszentmihalyi, M. (1997). Finding flow: The psychology of engagement with everyday life. Basic Books.

664

José R afael López‐Arcos et al. Fernández, C. (1979). Iniciación al lenguaje del cine. Dirección general de cinematografía. Ministerio de Cultura. Glassner, A. S. (2004). Interactive storytelling: Techniques for 21st century fiction. AK Peters. Lebowitz, J. & Klug, C. (2011). Interactive storytelling for video games: A player‐centered approach to creating memorable characters and stories. Focal Press. Padilla‐Zea, N. (2011). Metodología para el diseño de videojuegos educativos sobre una arquitectura para el análisis del aprendizaje colaborativo. PhD dissertation. University of Granada, Spain. Padilla‐Zea, N., González Sánchez, J. L., Gutiérrez Vela, F. L, Abad‐Arranz, A., López‐Arcos, J.R. (2012). Evaluación de Emociones en Videojuegos Educativos. El caso particular de los Niños. En Actas del XIII Congreso Internacional de Interacción (Elche, España, 3‐5 octubre 2012). Padilla‐Zea, N., Gutiérrez Vela, F. L, López‐Arcos, J.R., Abad‐Arranz, A., Paderewski, P. (2013). Modelling Storytelling to be used in Educational Video Games. J. Computers in Human Behavior. (In press). DOI=10.1016/j.chb.2013.04.020. Pearce, C. (1994). The Ins & Outs of Non‐Linear Storytelling. SIGGRAPH Comput. Graph.,Vol. 28, pp.100–101 Schell, J. (2008). The Art of Game Design: A book of lenses. Morgan Kaufmann Pub. Vogler, C. (1998). The Writer's journey. Michael Wiese Productions.

665

A Domain Ontology of Game Theory Applied to Game Based Learning Yemna Mejbri, Maha Khemaja and Rafik Braham PRINCE Research Group, ISITC, University of Sousse, Hammam Sousse, Tunisia mejbriyemna@yahoo.fr maha_khemaja@yahoo.fr rafik.braham@ensi.rnu.tn Abstract: Game theory is the formal study of interactions. All the fields involving strategic interactions may benefit from it. In this context, game theory provides a frame for Game Based Learning (GBL). Its concepts (e.g. algorithms, statements, etc.) provide for Game Based Learning designers a mathematic language to formulate, structure, analyze and understand strategic scenarios. This mathematical foundation of game theory makes it unambiguous but only humans can interpret its content. Therefore, game theory is difficult to be processed automatically when users don’t have expert skills. Several knowledge representation techniques exist in literature like taxonomies, thesauruses and ontologies. To establish a choice, we looked for a formal as well as a semantic formalism. So, we propose in this paper a domain ontology of game theory which will be applied to Game Based Learning with the purpose to help designers when they use game theory to validate Games Based Learning strategic scenarios and interactions. Keywords: game theory, game based learning scenarios, domain ontology, game theory ontology, WSML

1. Introduction Game theory is a mathematical field (Zamir 2003) that aims to develop and study the mathematical rules and principles which may be involved during the analysis of different types of players’ behavior and possible outcomes of a game. Game theory is widely applied in many disciplines. It studies (Bonzon 2007) situations in which the payoff of each player depends not only on his decisions, but also on other players’ decisions. The optimal choice for one player therefore generally depends on other players’ choices. As each player is not completely mastering his payoff, all players are in a strategic interaction situation. The same goes for Game Based Learning which presents situations where players interact to make choices. For instance, GBL is an educational method employed in a learning process. A GBL (Mourlas 2012) is a branch of serious games that deals with applications that have defined learning outcomes. It refers (Bottino and Ott 2013) to learning actions carried out in formal and/or informal educational settings by adopting games. It encompasses the use of both games designed expressly for fulfilling learning objectives (GBL) and "mainstream games". GBL brings together two concepts: serious learning scenario and interactive entertainment. As such, GBL generates strategies and actions that can be beneficial not only for student success or failure but also for student teamwork and collaboration preferences, competition, learning styles, and a variety of other learning issues. In order to determine strategies that should be adopted by learners to enhance their learning experience and to obtain higher scores, GBL needs to be subject of study using a game theory based approach. Indeed, to succeed to learn something while playing, GBL must typically involve principal game theory concepts and equilibrium. Game theory algorithms can lead the strategic issue to the desired solution by, firstly test equilibrium in order to identify the absorbing states of the game and the forces that maintain them. These absorbing states are the result of decisions’ combination, so that no player would have an advantage to deviate. Secondly, algorithms let evaluate, in an iterative way, scenarios and behaviors. Actually, game theory concepts are expressed in a natural pseudo language based on mathematics and could not be used in an automatic processing. And, up to now, there has been no approach that attempted to validate GBL by game theory. Therefore, we propose a Semantic Web services based solution for that aim. More specifically our proposal, in this paper, will focus on construction of a consistent and comprehensive domain ontology of game theory (GTO) and on the approach where this domain ontology will be used.

666

Yemna Mejbri, Maha Khemaja and Rafik Braham This paper is organized as follows: section II briefly introduces game theory as a subject matter of our study. Section III describes the impact of game theory on interaction analysis in GBL domain. Section IV shows the related works. Section V outlines our approach. Finally, section VI concludes our work and presents some future works.

2. Partial taxonomy of game theory Game theory is the formal study of games (Guerrien 1995). A game is a formal description of a strategic situation where players are driven to make choices among a number of possible actions. Choices of each player are limited within an already defined framework called game rules. Those choices lead to a result called outcomes with corresponding payoffs. A payoff is a number (also called utility) that measures how much the player likes the outcome. There are two different but equivalent ways to represent a game. The strategic form (Turocy and Stengel 2001) (or normal form) is the basic type of game studied in non cooperative game theory. A game in strategic form lists each player’s strategies, and the outcomes that result from each possible combination of choices. The extensive form, also called the game tree, is more detailed than the strategic form. It is a complete description of how the game is played over time. This includes the order in which players take actions, the information that players have at the time they must take those actions, and the time at which any uncertainty in the situation is resolved. A game in extensive form may be analyzed directly, or can be converted into an equivalent strategic form. A game can combine several of the following characteristics (Bonzon 2007): it can be static or dynamic, cooperative or non cooperative. A static game is a game where players choose firstly their actions simultaneously and receive later their outcomes. A dynamic game is a game that takes place in several steps. A cooperative game involves players to work together and form coalitions in order to obtain payoffs. However, in a non cooperative game, each player looks for his/her personal interest. Non cooperative games distinguish two principal categories: zero sum games and non zero sum games. Once the form for reporting payoffs of all possible outcomes of the problem is chosen, we must determine criteria (algorithms) which are used to select the best state in which no player wishes to change his/her behavior given the behavior of other players. This statement reflects an equilibrium in the game. Once this equilibrium is reached, there is no reason to change strategy. These criteria include dominant strategies, Nash equilibrium, Pareto optimality, etc. In figure 1, we represent game theory partial taxonomy (Guerrien 1995).

Figure 1: Partial taxonomy of game theory

667

Yemna Mejbri, Maha Khemaja and Rafik Braham As game theory constitutes a methodology used to build models of real world interactions (social, professional, political, etc.), we will use it for GBL interaction scenarios analysis. So, in the following section we will outline the GBL interaction scenarios.

3. Game based learning interaction scenarios GBL is a description of a situation (Logofatu, Dumitrache and Gheorge 2010) in which learners are encouraged to combine knowledge from different areas, to choose from a number of given solutions or to make a decision at a certain point in order to achieve the defined objectives. In a GBL scenario, interactions could partly defined by game and/or learning rules. For instance, learners are encouraged to communicate with other group members, discuss, collaborate, get in competition and negotiate subsequent steps, thus improving, among other things, their social, interpersonal, communicative and cognitive skills. To make GBL truly engaging, developers and/or designer of GBL scenarios need to plan actions, choose actions and experience the consequences of those actions along the way, check their initial plan, change strategies if needed, evaluate actions, choose strategies towards a goal, etc. We can model a GBL scenario as illustrated in figure 2.

Figure 2: General GBL scenario

4. Impact of game theory on interaction analysis in the GBL domain GBL scenarios analysis by game theory can predict equilibrium reached if players are rational. Equilibrium is a situation or a condition in which there isn’t any player who wants to change his behavior when he knows other players’ behavior. More precisely, when equilibrium is reached, there is no reason for the player to change his/her strategy. A central assumption in game theory is that players are rational (Turocy and Stengel 2001). So, the first goal of GBL validation by game theory is to predict how the scenario will be played rationally. Otherwise, game theory hints players how best to play against rational opponents. This could be the second goal of using game theory. The learning or gaming process has undeniably repetitive aspects. Interactions recur in similar conditions. In this context, it seems logical to include repetition in GBL, with the prospect of revealing important phenomena presented in real life. Hence, as a third goal, the idea to consider repeated GBL scenario will allows returning to situations where rational individual choices lead to sub‐optimal solutions. Finally, game theory reveals that cooperative behavior can be justified from the point of view of individual rationality. As a conclusion, in a GBL scenario several rational players interact in order to best satisfy their preferences. Our objective throughout this work is to validate concise and effective interactions between rational learners. For this aim, we chose game theory. A major shortcoming of this theory is its pseudo natural language. For this reason, we present, in the next section, firstly, different works which have tried to resolve this shortcoming and secondly, those who used game theory for GBL scenarios interactions analysis.

5. Related works The main goal of game theory is to establish and study the principles and mathematical rules that may occur in the analysis of different types of players’ behavior and the possible outcomes of a game. In the context of our work, we are interested in two axes namely the formalization of game theory on the first hand and the interactions’ analysis of games and/or learning process by game theory on the second hand.

668

Yemna Mejbri, Maha Khemaja and Rafik Braham There are several ways to formalize the game theory (Bonzon 2007). When the rules stipulate that players are involved sequentially in a specific order, and that the number of actions, including their choices is limited, the most appropriate representation is a tree with branches and leafs. The normal form can significantly reduce the size and time of graphical representation of a game by means of a table (matrix) of payoffs. Normal and extensive forms are always in pseudo natural language. Diverse attempts aim at modeling ontologies for games. Stephen Tang and Martin Hanneghan (2011), for example, introduce a new game content model based on ontology that can help game designers for game designing documents’ specification. Their game ontology is used in a model‐driven serious games development framework to provide novices and non‐technical domain experts, who have to create gaming content, a template and a formalized language to describe their game idea. In Game Ontology Project (GOP) (Zagal, Mateas and Vara 2005) an ontological framework is created in order to describe, analyze and study games by defining a hierarchy of concepts extracted from the analysis of several specific domains. It is a language for game design and also a framework for game designers to improve their understanding on what is designed in computer games. It is therefore useful for only studying and analyzing computer games, but has little use in producing a complete specification of computer game for the purpose of development. Therefore, Stephen Tang and Martin Hanneghan work and GOP project focus on game modeling and not on game theory modeling. They don’t cover all the aspects of game theory especially algorithms and solutions’ concepts. Other various works have exploited the game theory as a tool for analysis approach. For instance, Lori Shyba work (2006) concerns the video game domain and Louis R. Iziquierdo work (Izquierdo, Izquierdo and Vega‐ Redondo 2008) concerns the process scenario analysis. Author in (Shyba 2006) has tried to use examples of serious video games to show how they might be analyzed in the context of game‐theoretic strategies such as the mathematical Nash Equilibrium and as interactions within philosophical considerations of Virtual Ethics. Lori have used a series of games set in Darfur, Africa that might, if produced, enable a more solid understanding of what the future will hold in Africa and how priorities might have to be established or changed. Louis R. Iziquierdo et al. have introduced in their work a new branch of game theory called Learning Game Theory (LGT). The process of learning in LGT can take many different forms, depending on the available information, the available feedback, and the way these are used to modify behavior. They represent different models of learning. In most models of LGT, players use the history of the game to decide on actions to take. In the simplest form of learning (e.g., reinforcement or imitation) this link between acquired information and action is direct (e.g., in a stimulus– response fashion). In more sophisticated learning, players use game history to form expectations or beliefs about the other players’ behavior, and then react optimally to these inferred expectations. Louis et al. have presented a list of learning models studied in LGT such as Reinforcement Learning, Learning by Imitation, Static Perceptions and Myopic Response, Fictitious Play, Rational Learning, etc. Game theory in the work of Raymond and Jelena (Chiong and Jovanovic 2012) exceeds the role of analysis to have the role of solving and understanding collaborations. In fact, Raymond Chiong and Jelena Jovanovic aimed to better understand and contribute to the problems’ resolution of effective online group work. They followed an approach based on Evolutionary Game Theory (EGT) and presented a study they have conducted in order to investigate whether, and to what extent, EGT can be applied to explain students’ participation in collaborative study groups. All works previously described haven’t focused on a semantic and formal representation of game theory for the purpose of GBL scenarios analysis. So, in this paper, we seek to share a common reference to describe the content of game theory in a clear, formal, precise and semantic manner in order to automatically analyze GBL scenarios interactions.

6. Our proposed approach In this section, we will present firstly our game theory ontology, secondly, we will illustrate the general approach in which our domain ontology will be used for GBL scenarios analysis.

6.1 The game theory ontology (GTO) Diverse representation techniques exist in literature. Among those techniques, we quote controlled vocabulary, taxonomy, thesaurus and ontology. In this paper, we adopted ontology formalism. According to

669

Yemna Mejbri, Maha Khemaja and Rafik Braham Studer (Studer, Benjamins and Fensel 1998) “an ontology is a formal and explicit specification of a shared conceptualization”. Formal means machine understandable, explicit means that concept types and their usage constraints are explicitly defined. Conceptualization means an abstract model of the world that we wish to represent for some purpose. Shared means consensual knowledge accepted by a community of users. There are different methods and methodologies to develop domain ontology. All those methodologies present some commonalities and equivalences and we assume that there isn’t a best one. In this paper we have adopted the Noy and McGuinness’s guide to build our game theory ontology for various reasons. First, the Noy and McGuinness’s method is simple to apply and straight forward. Second, this method presents an iterative approach to ontology development. Finally, some ideas in the Noy and McGuinness’s guide come from the literature on object oriented (OO) software design. This encourages us to adopt this guide especially because we are convinced that OO had proven a good abstraction of the real world, therefore, it is suitable for domain ontologies. Therefore, we have carried the following seven steps (Noy and McGuinness 2001): 1. Determine the domain, scope and purpose of the ontology, 2. Consider reusing existing ontologies, 3. Enumerate important terms in the ontology, 4. Define the classes and the class hierarchy, 5. Define the properties of the classes – slots, 6. Define the facets of the slots, and finally 7. Create the instances. The game theory ontology is based on a knowledge model that includes complex taxonomic relationships (hierarchic relations, aggregation, and composition) as well as formal descriptions of axioms (cardinalities, integrity rules). In the first step of the adopted methodology, a set of the following competency questions must be answered.

What are the various types of a game?

What are the characteristics of a game?

What are the various exits of a game?

What are the forms of representations for a game?

What is the exit valid for each form of representation?

What are the characteristics specific to each type of game?

What are the rules of a game?

What is the best description for each type of game?

Does the game present clear goals and objectives that the player will have to accomplish in order to complete the game?

Are the game’s rules clear and consistent throughout the whole game?

How the scenario will be played rationally?

How best play against rational opponents?

When consider repeated scenario?

When player follow cooperative strategy?

In the second step, there wasn’t a game theory ontology in literature to reuse. In the third and the fourth step, we imported terms, classes and hierarchies from game theory taxonomy described in section 2. In order to achieve fifth and sixth step, we have defined constraints like:

The number of the players in a GBL must be higher than 1.

For each GBL in extensive form, there is always an equivalent game in strategic form.

A GBL in strategic form can have several representations in extensive form.

670

Yemna Mejbri, Maha Khemaja and Rafik Braham

Any equilibrium in dominant strategies is also equilibrium of Nash.

Each equilibrium of Nash is not necessarily equilibrium in dominant strategies.

A player can take part only in one and only one coalition in a cooperative GBL.

The actions of the players are sequential in a GBL in extensive form.

The actions of the players are simultaneous in a GBL in normal form.

The last step is achieved by implementing and querying our ontology.

6.2 A semantic web services based approach using game theory ontology The Game theory ontology (GTO) which we propose, will be used later for situation analysis within GBL scenarios in order to allow learning/gaming objectives achievement. In fact, the role of GTO in the process of designing GBL is to help the designer to automatically analyze his/her strategic situation by game theory. The GBL scenario designer starts by defining his game (GBL scenario) characteristics (rules, payoffs, strategies, etc.). This step will results in a GBL scenarios ontology which is annotated by our GTO. In the next step, the GBL scenarios designer wants to predict possible equilibrium of the game. He/she should, in this step, query the GTO for identifying the kind of algorithms to execute for equilibrium prediction. In the third step, the Web Service ontology (describing web services capabilities) will be queried for invoking the relevant algorithm for equilibrium prediction. In the final step, the GBL scenario designer will decide whether to keep the already defined characteristics or to introduce some changes.

Figure 3: Analyze GBL scenario interaction through domain ontology of game theory

7. Experimentation and validation In the context of the semantic Web, there are several languages (Lacot 2005) based on the simple XML (Extended Markup Language) like OWL (Web Ontology Language) and WSML. WSML (Web Service Modeling Language) (Bruijin et al. 2007) provides a formal syntax and semantics for WSMO (Web Service Modeling Ontology). WSML (Bruijin et al. 2007) is based on different logical formalisms, namely, description logics, first‐order logic and logic programming, which are useful for modeling Semantic Web Services. GBL scenarios designers need a modeling tool able to explicitly and formally describe semantics of GBL scenarios elements. For this purpose, we used WSMO studio (Dimitrov et al. 2006) that provides a conceptual framework and a formal language for semantically describing all required ontologies. WSMO studio allows sketching all the concepts and semantic links of game theory and describes Semantic Web Services. WSMO studio uses WSML language. The figure 4 shows an excerpt of the GTO.

671

Yemna Mejbri, Maha Khemaja and Rafik Braham

Figure 4: An excerpt of game theory domain ontology WSMO contains a reasoner WSML2 for checking consistency of the ontology. Indeed, this engine allows checking ontology’s coherence. An ontology is consistent if its components do not conflict with each other. Figure 6 illustrates a verification test of consistency for our ontology.

Figure 5: Verification of consistency For Querying ontology, WSMO contains an integrated Stratified IRIS reasoner to query the ontology described in WSML. These queries are formulated in logical expressions. Figure 7 shows a case of queries applied to our ontology and the results provided by the reasoner. This query returns the different type of a game.

672

Yemna Mejbri, Maha Khemaja and Rafik Braham

Figure 6: An example of a query: “games that could be solved by Nash equilibrium”

8. Conclusion In this paper, we had studied the issues of game theory in the GBL domain predominately the non use of game theory in the process of design GBL and the lack of game theory formalism. Our work leads us to conclude that the ontological formalization of game theory enhances the entertainment and the learning payoff. Finally, we believe that this work is only a starting point and that the prospects for research in the field of GBL are many and varied. Up to now we have only validated the consistency of our ontology. The remaining steps of GBL scenarios interactions analysis by game theory will be considered in our future work.

References Bonzon, E. (2007) Modélisation des interactions entre agents rationnels : les jeux booléens, l’Université Paul Sabatier, Toulouze III. Bottino, R.M and Ott, M. (2013) online [available] : http://www.tel‐thesaurus.net/wiki/index.php/Game‐ based_learning, www page, ITD‐CNR, Genoa Bruijin, J. et al. (2005) Web Service Modeling Language (WSML), WSML working group, http://www.w3.org/Submission/WSML/ Chiong, R. and Jovanovic, J. (2012) Collaborative Learning in Online Study Groups: An Evolutionary Game Theory Perspective, Journal of Information Technology Education: Research Dimitrov, M. et al. (2006) WSMO Studio Users Guide v.1.8 [online], Available: http://www.wsmostudio.org ème Guerrien, B. (1995) la Théorie des Jeux, 2 édition ISBN 2‐7178‐2737‐4, Ed.ECONOMICA, Paris Izquierdo, L.R. and Izquierdo, S.S. and Vega‐redondo, F. (2008) Learning and evolutionary game theory, Spain. Lacot, X. (2005) Introduction à OWL, un langage XML d’ontologies Web. Logofatu, M. and Dumitrache, A. and Gheorghe, M. (2010) Game based learning in education, 4th International Conference "Education Facing Contemporary World Issues", Piteşti, Romania. Mourlas, C. (2012) Learning in a gaming environment: from e‐learning to serious games, Dept. of Communication and Media Studies, Athens Noy, N.F and McGuinnesss, D.L. (2001) Ontology development 101: a guide to creating your first ontology, Stanford Knowledge Systems Laboratory Technical Report and Stanford Medical Informatics Technical Report, California. Shyba, L. (2006) Rational Game Theory and Serious Video Games. Proceedings of FuturePlay 2006. Studer, R., Benjamins, V.R. and Fensel, D. (1998) Knowledge engineering: Principles and methods, online [available]: http://www.jfsowa.com/ontology/guided.htm Tang, S. and Hanneghan, M. (2011) Game content model: An ontology for documenting serious game design , School of Computing and Mathematical Sciences, Liverpool John Moores University Byrom Street, Liverpool, L33AF, UNITED KINGDOM. Turocy, T.L. and Von Stengel, B. (2001) game theory, CDAM Research Report LSE‐CDAM. Zagal, J.P and Mateas, M. and Vara, C.F. (2005) Towards an ontological language for game analysis, Georgia Institute of Technology, USA ZAMIR, S. (2003) Cours de théorie des jeux , CNRS, EUREQua, Paris.

673

Puzzle‐Based Games as a Metaphor for Designing in Situ Learning Activities Javier Melero1, Patricia Santos2, Davinia Hernández‐Leo1 and Josep Blat1 1 Universitat Pompeu Fabra, Barcelona, Spain 2 University of the West of England, Bristol, UK javier.melero@upf.edu Patricia.Santosrodriguez@uwe.ac.uk davinia.hernandez@upf.edu josep.blat@upf.edu Abstract: In situ learning activities are recently becoming of higher interest in education because they facilitate contextualized learning experiences. A particular case is the creation of learning routes containing geo‐located questions. In this line, teachers are increasingly more interested in getting involved in the design of such learning experiences according to their specific needs. QuesTInSitu is a mobile application that supports the creation of questions that have to be answered in the real location. Previous research experiments reported educational benefits of carrying out this type of activities and highlighted the requirement of incorporating mechanisms that promote problem solving and students’ motivation. Games and educational puzzles are considered interesting and feasible approaches to address these goals. On the one hand, puzzle‐based games are especially suitable to feasibly involve teachers as designers of the games. On the other hand, this type of games can promote engagement in the subject topics, while fostering students’ problem solving, analytical and memory skills. Therefore, this paper proposes a puzzle‐based game metaphor as an innovative way to design gamified in situ learning activities. A game design task, completed by secondary education teachers, has been carried out to evaluate the proposed puzzle‐based game metaphor. In order to computationally represent and enact the designed gamified in situ learning activity following the proposed metaphor, a new version of the QuesTInSitu has been developed. Evaluation results show that teachers are able of using the metaphor to design potentially fruitful gamified in situ learning activities according to their educational objectives. Keywords: game‐based learning, puzzles, m‐learning, in situ learning activities, game design task, gamification

1. Introduction The use of m‐learning is growing in education since it brings the possibility of creating situated learning activities that take place in physical spaces (Jeng et al. 2010). Benefits derived from this type of activities are related to developing exploration skills and cooperation (Hwang et al. 2008). In this line, QuesTInSitu is a m‐ learning app to support assessment in situ (Santos et al. 2011). The system enables the creation of routes containing geo‐located questions that have to be answered with a smartphone in the associated real area. The dynamics of the in situ learning activity is similar to a traditional test: students have one attempt to answer each geo‐located question. Results of previous experiments showed that, using QuesTInSitu, students put explorative and spatial skills into practice and fosters their motivation, and personal observation. However, students were mainly focused on achieving higher scores than reflecting about the proposed questions (Santos et al. 2011). Puzzle‐based games can engage students in the subject topics, while at the same time foster students’ problem solving, analytical and memory skills (Huang 2007; Bottino 2008). Besides, the nature of puzzle‐based games seems relevant to consider as potential educational strategy to feasibly involve teachers as game designers (Huang 2007; Crawford 1982). In this line, with the aim of allowing teachers designing this type of learning activities, we propose the use of a puzzle‐based game metaphor as an innovative way to design gamified in situ learning activities. As a result, a new version of QuesTInSitu has been developed. This version considers traditional puzzles and gamification strategies to improve learning in situ. In order to evaluate the potential educational benefits, as well as the most relevant issues of using the proposed metaphor, an evaluation with teachers of secondary education has been carried out. The paper is structured as follows. Section 2 describes the main elements to consider when designing in situ learning activities according to the puzzle‐based game metaphor. Section 3 presents the game design process with secondary education teachers to create a gamified in situ learning activity. The results obtained from the game design task are reported in Section 4. Finally, Section 5 concludes with the main highlights obtained from the described results and future research lines.

674

Javier Melero et al.

2. A puzzle‐based metaphor as an educational strategy to designing in situ learning activities Educational puzzles (also known as educational jigsaw puzzles) are characterized by promoting problem solving. The objective of any jigsaw puzzle is the arrangement of a set of given pieces into a single, well‐fitting structure, with no gaps left between adjacent pieces (Williams 1997). Players can try to achieve the solution as many times as they want, and the shapes of each piece are a hint themselves concerning where the piece has to be placed. On the other hand, considering also several research studies that have identified different factors when designing educational games (Fisch 2005; Jones 1998; Kirriemuir & McFarlane 2004; Malone 1981; Sandford & Williamson 2005; Squire & Jenkins 2003), and a broadly recognized approach intended for helping teachers to evaluate the potential of using games‐ and simulation‐based learning (de Freitas & Oliver 2006); a conceptual model (Melero et al. submitted) has been proposed for the design of technology‐supported puzzle games including virtual and physical objects. Overall, this conceptual model defines the learning flow of the whole game, the context where the game takes places as well as the puzzles associated to each activity (see Figure 1). In concrete, the learning flow consists of a story structured by levels. Each level of the game presents either a single activity or a group of activities to be performed indoors or outdoors. An activity has associated a puzzle, and players perform specific activities depending on their role. Each puzzle is represented by relating pieces among them or by relating pieces with specific positions (i.e. slots) of a board. Both pieces and slots can represent virtual objects or as computer‐recognised physical objects. Finally, different punctuation, feedbacks and hints can be associated to both activities and puzzles in order to scaffold the learning process and guide students to the correct expected solutions.

Figure 1: Overview of the conceptual model Considering this conceptual model, our claim is that this can be applied for creating gamified in situ activities to enhance the students’ learning experience. In particular, the aim of applying this strategy is to engage students to reflect on the correct solution. Similarly to jigsaw puzzles, players could try to solve the different questions as many times as needed until reaching a correct solution. Following the puzzle‐based approach we escape from giving the immediate feedback when solutions are incorrect. Instead we believe that we promote a more reflective methodology in which the answer is not provided to the students right away. In that sense, students have the possibility of finding the correct solutions either by reflecting on their wrong choices or taking benefit of the resources provided not only by the game design itself but also by the information that can be found in situ in the environment (e.g. people, buildings, etc.). Besides, the different elements of the puzzle‐based game metaphor are described as follows:

The “board”. That means, the map containing the geo‐located questions of the puzzle‐based game.

The “slots” included in the map will represent each of the questions designed for the in situ learning activity.

The “puzzle pieces” that are the different options for each question of the puzzle game. Just one puzzle piece can fit in a concrete slot, meaning that there is only a correct option for each of the designed geo‐ located questions.

675

Javier Melero et al.

The “puzzle”, formed by a group of questions (i.e. slots).

The “level”. Different levels can be designed for each in situ learning activity. A level contains just one puzzle. It can be defined as many levels as the designer wants. For instance, the designer can differentiate different areas of a map, and associate a level to each place.

The “points”, defined to express the students’ performance: a) correct answers add points to the overall player’s punctuation, b) incorrect answers subtract points the overall player’s punctuation, c) consulting hints subtract points the overall player’s punctuation.

The “bonus”. When all the questions for a given level have been correctly answer, students will obtain a bonus that adds extra points to the overall punctuation. The extra bonus is a reward to engage and encourage students to correctly complete the different puzzles.

The “feedback”. This means, textual information associated to specific range of points in order to show the students how is their performance.

The “hints”. Hints are provided to scaffold the learning process in order to avoid frustrations and advance forward the gamified activity. Besides, somehow similarly to traditional puzzles, in which the position of the last piece is trivial; when all questions, except one, have been correctly answered, the student will obtain a “free hint”. This means no points are subtracted when consulting the hint of the last question to be solved of a given puzzle.

In order to evaluate whether the different elements are meaningful and understandable by teachers, a game design task for a real in situ learning experience has been carried out with secondary education teachers. The evaluation will also give insights whether the teachers are able to use the proposed metaphor to design this type of learning scenarios.

3. Game design process A group of 7 secondary education teachers, from different subject matters were interested in designing an in situ learning activity to help their students to understand better the city where they are studying by solving in situ different geo‐located questions. As students advance forward the game, new levels of the game are discovered, meaning that new areas of the city can be explored. To this end, we suggested the use of a puzzle metaphor for the design of the gamified in situ m‐learning activity. Our main objective is to understand the benefits of using a puzzle game metaphor for the design of in situ activities, as well as the limitations and affordances of the conceptual model for creating technology‐supported puzzle games. The following methodology has been designed to analyse the teachers’ comprehension about the use of the puzzle game metaphor:

An introduction is given about puzzle games and the key elements of the conceptual model that has to be understood for designing the activity. Some examples using the elements of the conceptual model are shown as well.

Use of templates (see Figure 2). The teachers use several templates following the elements of the conceptual model to design the gamified in situ learning activity. These templates consist of defining: the general aspects of the game, the properties of each level of the puzzle game, and the slots and their related pieces for each puzzle.

Learning design task. First, the different templates are presented, as well as an explanation about how to use them. Then, teachers discuss by themselves the information of each template and expose their concerns. A meeting with the researchers is carried out to discuss and share doubts, worries, interpretations and suggestions of the teachers. Then, the teachers fill out the different templates according to their needs. An example of a complete level designed by the teachers is available as an on‐ line appendix to this article (http://www.dtic.upf.edu/~jmelero/ecgbl‐2013/template_lv1.pdf).

Finally, the information provided by the teachers in the different templates is used to implement the puzzle‐based game for the activity. In this sense, we developed a new version of the QuesTInSitu application. Unlike the former version of QuesTInSitu, where student only have an attempt to solve each geo‐located question and no gamification strategies were included; the new version considers the puzzle‐ based metaphor described in the previous section. This means the application is compliant to the

676

Javier Melero et al. conceptual model and, therefore, its input is the corresponding XML binding, including the different elements of the puzzle‐based game (see Figure 3).

Figure 2: Sample of Template game

Figure 3: Some screenshots of the “QuesTInSitu: The Game” and chunks of its computational representation (compliant with the conceptual model)

677

Javier Melero et al. “QuesTInSitu: The Game” works as follows. First, the player must enter a group’s name. Then, a gamified route containing geo‐located questions should be chosen. Once this is done, the map is loaded as well as the questions of the first level of the game. Also, the player can view at any time the textual information associated with the level. In addition, in order to display the content of each of the questions, the player has to go to the indicated place. The content then will be displayed automatically. In order to access again to the questions, that has been unanswered or incorrectly answered, the player has to click manually on the icon. On the other hand, the player can always see what your overall punctuation in the game. The punctuation will be updated every time a player fails o answer correctly the questions, access to the hints, or get a bonus for having successfully solved all the questions of a level. To advance forward the different levels of the game, this can be done automatically (if all questions of a level have been answered correctly) or manually (if there are bad questions answered or unanswered). The game ends once the last level of the game has been completed. Once the player reaches this point, will see the ranking scores obtained by the other groups of students.

4. Evaluation An evaluation was carried out in order to analyse a) the perceived usefulness and educational benefits of the puzzle‐based game metaphor by the teachers, and both b) the understanding and c) importance of the different elements to consider for the game design task of the in situ learning activity.

4.1 Data gathering techniques Since different aspects, such as participants’ satisfaction or the game design task complexity, have to be analysed; a mixed evaluation method has been followed (Cairns & Cox 2008) combining and triangulating (Guba 1981) the qualitative and quantitative data. Table 1 lists the different data sources: a) Questionnaires to evaluate quantitative and qualitative data about the different elements of the metaphor, and b) observations from the research team during the different meetings. Quantitative data will provide insights into teachers’ opinions, while qualitative data will support or reject those teachers’ perspectives (Guba 1981). Table 1: Data gathering techniques Data source Questionnaires

Type of data Quantitative ratings and qualitative opinions

Observation

Observations during the different meeting s by 2 different researchers

Label [Teacher‐X] Where X is the number of the teacher, from 1 to 7. [Observation‐Y] Where Y is the number of the observer, from 1 to 2.

4.2 Results about the educational benefits and dynamics of the puzzle‐based game metaphor In general, when asking the teachers about the educational benefits of the puzzle‐based game metaphor, they found it stimulating, encouraging, a good approach to motivate students, useful to structure knowledge in parts and to promote the empowerment of teamwork and collaboration. These were the gathered comments: "It's stimulating. I think the activity moves away from heavy exercises in the classroom and situates students in a more playful context" [Teacher‐1], "Motivation and empowerment efforts from teamwork and the achievements of goals" [Teacher‐2], "The idea is great, it is very encouraging, it is like an updated “treasure hunt” approach " [Teacher‐4], "I believe it has many benefits as it raises different educational and learning situations, like having to do research and meeting the environment, the heritage, etc. Besides, it is a good teaching tool that empowers collaboration and teamwork" [Teacher‐5], "The metaphor allows understanding that knowledge is formed by parts that are related afterwards. The game encourages, challenges the student with positive and negative reinforcement, and the student interacts actively in the learning process" [Teacher‐ 7]. In a 5‐rating scale, different answers resulted when asking about having the possibility of solving a question as many times as necessary until reaching the correct solution [Teacher‐All]: two teachers completely disagree, while two other teachers agree o completely agree. The rest were neutral on this assumption. For instance, a teacher pointed out that: “I find the bonus, hints and punctuation more motivating and interesting than trying and trying to reach the correct answer or having a free hint. It is not bad, but I find these elements dispensable" [Teacher‐4].

678

Javier Melero et al. Most of the teachers argued that the metaphor did not limit the activity design task; applying the metaphor only implied to change the point of view; then the approach is very attractive [Observer‐1]. Further comments were: "No, I was not limited at all" [Teacher‐1], "I have not experienced any limitation" [Teacher‐2], "Yes, but obviously any approach limits the design task" [Teacher‐3], "No, it is ideal for working in situ, make different questions, etc." [Teacher‐4], "Not limitations, but rethinking and changing the dynamics of the game and its development" [Teacher‐5], "No, but I have to formulate the questions differently" [Teacher‐6], "The approach does not limit but it extends the possibilities" [Teacher‐7]. Furthermore, in order to analyse the correct use of the proposed metaphor, a key point is to evaluate whether the teachers properly understand the meaning of the puzzle‐based game metaphor’s elements.

4.3

Results about understanding the elements of the puzzle‐based game metaphor during the game design task

Overall, the metaphor used to identify each element of the puzzle‐based game approach was found appropriate; just the metaphor of the “level” was the most troubled element. More specifically, all the teachers found very or quite appropriate the metaphor of a question meaning as a slot. 4 out of the 7 teachers found quite appropriate the metaphor of the question's options meaning as puzzle pieces. 6 out of the 7 teachers found very or quite appropriate the metaphor of a map meaning as a puzzle board. Different opinions resulted when asking about whether the metaphor of the slots belonging to a same geographical area meaning as a level of the game is appropriate. In fact, 6 out of the 7 teachers quite or totally agreed that they had difficulties understanding what a level means: "The greatest difficulty was the confusion between area and level. We understood area, while they [the researchers] understood level" [Teacher‐4], "It is difficult to imagine the design by levels" [Teacher‐3], "The only thing that I found difficult to understand was the definition of the level concept" [Teacher‐7]. Also, in the different meetings some teachers asked about what a level means [Observer‐1], and some asked whether the levels are associated to grades of difficulty [Observer‐2]. Furthermore, regarding the meaning, half of the teachers agreed that the elements, “slot” and “pieces”, were unclear or difficult to understand. Besides, this difficulty was just at the beginning [Observer‐1‐2]; once the teachers got familiar with the puzzle‐based game metaphor, they did not have problems to correctly perform the game design task. Results show that 4 out of the 7 teachers quite or totally agreed that they had difficulties understanding what a slot means. However, two of the teachers totally disagreed, considering the meaning of the slots easy to understand. Half of the teachers quite or totally agreed that they had difficulties understanding what a puzzle piece means, despite the rest totally disagreed. All the teachers quite disagreed or totally disagreed that they had difficulties understanding what a bonus means. However, a teacher understood that a bonus is only obtained if all the questions are correctly answered at the first attempt [Observer‐2]. 5 of the teachers totally disagreed that they had difficulties understanding what a hint means. Different rating resulted when asking about the difficulties the teachers had for understanding what a feedback (associated to either reaching a level or completing the game) means. Some gathered comments were: "At first understanding the dynamics wasn't easy. Also, because I joined the group later" [Teacher‐3], "At first it was a bit difficult to understand the metaphor, but once reached the process of the game, it was much simpler to understand the above items. I mean, the difficulty was at first, when the metaphor was described, understand everything and having to apply it later. Once the elements were understood, filling the templates was easy" [Teacher‐5]. Also, during the discussion of a meeting, some teachers argued that at first the metaphor is quite abstract, and it is needed to recall and interpret the meanings of each element [Observer‐1]; but once the elements were understood, it was easy [Observer‐2].

4.4 Results about the importance of the elements of the puzzle‐based game metaphor Apart from correctly understanding the different elements involved in the proposed metaphor, it is important to analyse which of the elements are considered indispensable in designing gamified in situ learning activities, and which of them depends more on the educational situation. In general, when asking teachers about the importance of the different elements of the puzzle‐based game metaphor, the most discussed elements were the hints and the punctuation’s mechanisms. Overall, the hints were considered a good mechanism to allow students advance in the game, but not all activities should have associated hints, they should be considered only in those cases that having a hint could

679

Javier Melero et al. be meaningful or relevant to the activity. 5 out of the 7 teachers agreed or totally agreed that the hints can guide the students in case they are lost or stuck. Also, 5 out of the 7 teachers found important or quite important subtracting points when hints are consulted. However, half of the teachers thought that providing hints to each question was not very important or not important at all. In this line, some teachers highlighted that not all the questions had hints because it was not make sense and it would be very heavy work [Observer‐ 1]. Besides, the design of hints were difficult [Observer‐1‐2], and not all the hints are designed to promote reflexion because the activity would be too complicated and students could become exhausted if they have to reflect in each of the questions [Observer‐2]. Regarding the free hints, different opinions were gathered. Most of the teachers did not recall on the use having free hints nor found them useful. The resulted comments were: "I think I do not know what a free hint means. I think that no points are subtracted" [Teacher‐1], "I understand that [free hints] can be a motivation, but perhaps the groups that more achieving them, less they do needed" [Teacher‐3], "I find bonus, hints and punctuation more motivating and interesting than trying and trying to reach the correct answer or having a free hint. It is not bad, but I find these elements dispensable" [Teacher‐4], "I do not recall that there were free hints" [Teacher‐7]. Also, 5 out of the 7 teachers found slightly significant or not significant at all the free hints. In this regards a teacher pointed out that, "The free hint does not seem anything special to me" [Teacher‐1]. Teachers found the different punctuation’s mechanisms a good approach to allow students self‐reflecting on their performance. They also highlighted the possibility of defining adapted punctuations. All the teachers agreed or totally agreed that the punctuation and feedback give students information about whether their decisions are correct or not. As a consequence, 6 out of the 7 teachers agreed or totally agreed that the punctuation mechanism allows the students to reflect more on the different questions. Also, 5 out of the 7 teachers found very important the adapted punctuation depending on the number of wrong attempts. Regarding the bonus, all the teachers agreed or totally agreed that this element can keep students motivated throughout the activity. But, half of the teachers thought that providing a bonus each time a level is accomplished was slightly important or not important at all. The other half totally disagreed.

4.5 Results about the relevant data to consider for gathering in gamified learning in situ activities Finally, since the design of gamified in situ learning activities are still in an early stage in education, it is important to know what type of data would be the most relevant to the teachers as a worthy outcome when students are carrying out this type of learning experiences. Overall, the teachers found important to gather the data about the satisfaction of the students while performing the activity, the time and punctuation obtained in the different stages of the game, and the awareness about the performance of other groups of students while doing the activity. Also, the teachers highlighted the importance of having a communication module to keep in contact with their students. The gathered comments were: "The route followed by students. The punctuation. The time required. Degree of fun" [Teacher‐1], "The time devoted reaching a question and answering it" [Teacher‐2], "Time elapsed from the beginning. Current punctuation of the other groups" [Teacher‐3], "Time performing the activity. Allow students to consult where the teachers are at any time" [Teacher‐4], "I believe that time is a good indicator, as well as the punctuation of other participants" [Teacher‐5], "Initial presentation of the objectives of the activity. A final evaluation of the game (similar to a 'telephone satisfaction survey'). The overall punctuation of the other groups. The situation of the other groups in the map. Constant contact with the teacher. A forum where the students can describe their experience" [Teacher‐7].

5. Conclusions and next steps This paper has presented an innovative approach to design in situ learning activities with potential of motivating students and supporting them in reflective learning processes. To this aim, a metaphor based on puzzles and game elements have been described and evaluated with secondary education teachers, who have been involved in a design process applying the metaphor. For this particular case, results encourage us to continue working in the proposed direction. The teachers perceived as useful the proposed approach to design gamified in situ learning activities, highlighting its possibilities and potential educational benefits. The proposed puzzle‐based game metaphor has been found stimulating and a good approach to motivate students. Besides, punctuation and feedbacks were considered good strategies to encourage students during the game. In this line, further work includes evaluating whether similar findings are obtained in the design of

680

Javier Melero et al. other learning scenarios that involve in situ activities or other gamified activities that are not necessarily in situ. Regarding the understanding of the elements considered in the game design task, results show that only the “level” element was the most troubled (using geographical‐related vocabulary as examples improved understanding, i.e. “levels” representing “zones”). Teachers needed a time to internalize the different elements; some problems understanding how to apply some of the elements where detected at the beginning of the game design process. In this line, teachers understood easier the elements related to games (e.g.: hints or bonus) than those related to puzzles (e.g.: pieces, slots). Thus, it is crucial to be careful when presenting the puzzle‐based metaphor in order to foster the teachers’ understanding. It is important a good initial explanation about the different elements and providing significant examples showing the applicability of them, as well as, supporting the teachers at the beginning of the design task. From all the elements of the conceptual model, an especial attention deserves the use of hints. In this regards, teachers seem not to appreciate the usefulness or the added value of the free hints and the bonus. All these features will be important to analyse with the students as well. Students are the final users of the designed activities, and they will provide us with useful information about how they consider the inclusion of the free hints or bonus in gamified in situ learning activities. Future research lines also includes to understand the reasons about why the dynamics of answering the different questions does not make sense to some of the teachers, as well as a more clear definition of what elements will be important to consider for future experiments. Also, another research line derived from this study is the inclusion of communication modules and how to visualize and present the data of the elements that are of interest by the teachers. Different experiments have been carried out in March and May 2013 with groups around 30 students (boys and girls, average of 16 years old) using the resulted designed activity. The teachers’ goal is to have the opportunity of involving all the schools and educational centres of l’Hospitalet city (Spain) in a collaborative game‐based m‐learning in situ activity in the future.

Acknowledgements This research has been partially funded by the Spanish Ministry of Economy and Competitiveness in the EEE Project (TIN2011‐28308‐C03‐03). The authors would also like to thank the teachers of the Dolmen Educational Centre in L’Hospitalet.

References Bottino, R.M., Ott, M. and Tavella, M. (2008) “The Impact of Mind Game Playing on Children’s Reasoning Abilities: Reflections from an Experience”, In Proceedings of the 2nd European Conference on Game‐Based Learning, Barcelona, Spain, October. Cairns, P. and and Cox, A.L. (2008) Research methods for human‐computer interaction, Cambridge University Press New York, NY, USA. Crawford, C. (1982) The art of computer game design, Osborne/McGraw‐Hill, Berkeley, CA. de Freitas, S. and Oliver, M. (2006) “How can exploratory learning with games and simulations within the curriculum be most effectively evaluated?”, Computers & Education, Vol. 46, No. 3, pp. 249‐264. Fisch, M.S. (2005) “Making educational computer games educational”, In Proceedings of the conference on Interaction, design and children, Boulder, CO, June. Guba, E. G. (1981) “Criteria for assessing the trustworthiness of naturalistic inquiries”, Educational Communication and Technology, Vol. 29, No. 2, pp. 75‐91. Huang, O.W.S., Cheng, H.N.H. and Chan, T.W. (2007) “Number Jigsaw Puzzle: A Mathematical Puzzle Game for Facilitating Players’ Problem Solving Strategies”, In Proceedings of the First IEEE International Workshop on Digital Game and Intelligent Toy Enhanced Learning, Jhongu, Taiwan, March. Hwang, G., Tsai, C. and Yang, S. J. H. (2008) “Criteria, strategies and research issues of context‐aware ubiquitous learning”, Educational Technology & Society, Vol. 11, No. 2, pp. 81‐91. Jeng, Y., Wu, T. Huang, Y., Tan, Q. and Yang, S. J. H. (2010) “The add‐on impact of mobile applications in learning strategies: A review study”, Educational Technology & Society, Vol. 13, No. 3, pp. 3‐11. Jones, M.G. (1998) “Creating Engagement in Computer‐Based Learning Environments”, [online], http://it.coe.uga.edu/itforum/paper30/paper30.html Kirriemuir, J. and McFarlane, A. (2004) “Literature review in games and learning: A report for NESTA Futurelab”, [online], http://www.nestafuturelab.org/research/reviews/08_ 01.htm

681

Javier Melero et al. Malone, T. (1981) “Toward a theory of intrinsically motivating instruction”, Cognitive Science, Vol. 4, pp. 333‐369. Melero, J., Hernández‐Leo, D. and Blat, J. (submitted) “A Model for the Design of Puzzle‐based Games including Virtual and Physical Objects”. Sandford, R. and Williamson, B. (2005) “Games and Leaning, A handbook from Futurelab”, [online], http://archive.futurelab.org.uk/resources/publications‐reports‐articles/handbooks/Handbook133 Santos, P., Pérez‐Sanagustín, M., Hernández‐Leo, D. and Blat, J. (2011) “QuesTInSitu: From tests to routes for assessment in situ activities”, Computers & Education, Vol. 57, No. 4, pp. 2517‐2534. Squire, K. and Jenkins, H. (2003) “Harnessing the power of games in education”, Insight, Vol. 3, No. 1, pp. 5‐33. Williams, A. D. (1997) “Jigsaw Puzzle – A Brief History from the 1760s to Modern Day Puzzle Makers”, [online], http://mgcpuzzles.com/mgcpuzzles/puzzle_history/index.htm.

682

Supporting and Facilitating Collaborative Learning in Serious Games Kimmo Oksanen and Raija Hämäläinen Finnish Institute for Educational Research, University of Jyväskylä, Finland Kimmo.t.oksanen@jyu.fi raija.h.hamalainen@jyu.fi Abstract: Serious games have proven to have the potential to support collaborative learning. However, the interrelationship between decisions related to the design and use of collaborative serious games and the learners’ experiences and collaborative knowledge construction has rarely been studied. An insight into learners’ experiences is essential because emotional experiences ultimately work as a motivator for cognitive decisions during a game. Similarly, the instructional design perspective is important, because new types of learning environments pose new challenges for teachers in their pursuit to support learning. This study attempts to narrow the knowledge gap in the understanding of the influence of game design and instructional design on game experiences and collaborative learning in serious games. The aim of the study is twofold. The first aim is to design a game environment for practicing interprofessional knowledge construction. The second aim is to empirically investigate learners’ subjective experiences generated by the games and to determine teachers’ roles from the perspective of collaborative knowledge construction processes in game settings. The findings showed that in general, game experiences evoked during gameplay were positive and players felt an average degree of engagement. Moreover, playing the game generated a high degree of social presence amongst players, and the game’s sociability stimulated the creation of a supportive and trustful atmosphere for social interaction and collaboration. Furthermore, a sense of social presence and the sociability of the game were found to be strongly associated with the core game experiences. Thus, apparently, game engagement in these settings may be augmented through the social dimension of gaming. However, the results also revealed that high‐level collaborative learning during game play does not necessarily emerge without teachers’ real‐time orchestration. In conclusion, the results of the study indicate that collaborative serious games represent a specific type of sociable computer‐supported collaborative learning (CSCL) environments, which can act as engaging and pleasant spaces for social interaction and collaborative learning. Keywords: computer‐supported collaborative learning, collaborative serious game, game design

1. Introduction In this study, we focus on two key challenges that hinder the emergence of social interaction and collaborative activities. First, to facilitate high‐level collaborative activities, it is necessary to design environments that guide and promote collaborative activities, while supporting the learners’ socio‐emotional processes (Kreijns et al. 2007). Second, besides the game’s internal guidance, new types of learning environments challenge teachers to find new ways to support collaborative learning. Teachers’ real‐time orchestration is presented as a means to meet this challenge (Hämäläinen and Oksanen 2012).

2. Theoretical background Serious games have proven to be a potential means for promoting learning in complex situations, such as problem solving related to historical disease epidemics (Kennedy‐Clark and Thompson 2011) and water management from different perspectives (Hummel et al. 2011). Despite optimistic notions related to collaborative serious games, such games should not be seen as a “silver bullet” that necessarily leads to the emergence of social interaction and collaborative activities (Hämäläinen 2011), which are the main prerequisites for collaborative learning. To harness the full potential of collaborative serious games, both educational and gameplay perspectives need to consider when designing such games. Currently, the integration of education theory and game design perspectives in the design of collaborative serious games is rare (Echeverria et al. 2011). Such findings raise an important question: Why is it difficult to achieve successful engaging collaboration in computer‐supported collaborative learning (CSCL) environments, such as collaborative serious games? Establishing initial conditions, guiding the collaboration itself, and maintaining the collaboration are considered crucial elements in the design of socio‐technical environments that promote collaboration in group activities (Collazos et al. 2007). Furthermore, Nussbaum et al. (2011) outlined favorable conditions for social interactions and collaborative activities in CSCL environments. These conditions include a common goal, positive interdependence, coordination and communication, individual accountability, awareness, and joint rewards. Kreijns et al. (2007) proposed three essential factors in the design of sociable CSCL environments, namely sociability, social presence, and pedagogical techniques. By considering these factors, the probability

683

Kimmo Oksanen and Raija Hämäläinen of the emergence of fruitful social interaction can be improved. Through effective interaction, members of the group can build a solid social space for collaborative knowledge construction, which further reinforces interaction (Figure 1).

Figure 1: Factors influencing the emergence of social interaction in CSCL environments (Kreijns et al. 2007) Sociability refers to the specific characteristics of the CSCL environment, which aim to facilitate the emergence of a sound social space to engender a safe and supportive environment for social interaction and collaborative activities (Kreijns et al. 2007). In practice, this means that the environment, for example, enables the members of the group to easily contact their teammates and make them feel comfortable. Social presence refers to the subjective experiences of learners. According to Kreijns et al. (2007), social presence represents the feeling that the other person is a socially present, or “real,” physical person through either immediate or delayed communication. Social presence can be further divided into positive and negative psychological and behavioral involvement (Poels, De Kort and IJsselsteijn 2008). Good sociability in an environment together with a positive feeling of social presence provides a safe and supportive space for social interaction and collaboration. This further encourages learners to share and critique tentative ideas without interpreting criticism as a personal insult but rather as something valuable (Rourke 2000). Finally, pedagogical techniques define learning activities and guide learners’ actions during the learning process. In practice, this implies, for example, the pedagogical scripting of game tasks and participation of the teacher in the game play as an external orchestrator. The objective of pedagogical scripting (Kobbe et al. 2007) in serious game design is to define the tasks and mechanics of a game that requires the participation of several players and necessitates collective knowledge construction (Bluemink et al. 2010). Besides the pure pedagogical perspective regarding game design, it is essential to consider the gameplay perspective. According to Echeverria et al. (2011), in the context of serious game design, the gameplay perspective is subject to the educational dimension. This implies that the aim of game design of collaborative serious games is to find ways to take advantage of game design elements to support and promote collaborative activities. Besides the internal guidance of the game, teachers’ real‐time orchestration has recently received increased attention as a means of facilitating joint knowledge construction processes in new types of CSCL environments, such as serious games. The aim of real‐time orchestration is to guide and support learners’ knowledge construction processes on the fly. In practice, this could mean asking specific questions, requesting reasoning, or highlighting real‐life scenarios (Hämäläinen and Oksanen 2012).

3. Aims of the study The aim of the study is twofold. The first aim is to design a pedagogically scripted game environment to practice interprofessional knowledge construction. The second aim is to empirically investigate learners’ subjective experiences generated by collaborative serious game and determine teachers’ roles from the perspective of collaborative knowledge construction processes in game settings. In particular, the research questions of the study are as follows:

What kind of experiences, including core game experiences and sense of social presence, are generated by the players by playing a collaborative serious game?

684

Kimmo Oksanen and Raija Hämäläinen

What is the relationship between the social dimension of gaming, including the sense of social presence and sociability of the game, and core game experiences?

What are the main differences in the knowledge construction processes between games conducted with and without real‐time orchestration?

4. Collaborative serious 3D game The game used in this study is a collaborative serious game called Game Bridge (for a detailed description of the game, see Oksanen and Hämäläinen 2013). Game Bridge is based on the theoretical knowledge of collaborative learning and game design, and it has been developed in collaboration with professionals from different areas (including researchers, teachers, work‐life instructors, game designers, and programmers). The primary goal of the game is to enhance interprofessional knowledge construction in the area of human sustainability (Interprofessional Education Collaborative Expert Panel 2011). Game Bridge requires the participation of four to five players who are linked to each other through the game server, which runs the virtual world in which all gameplay occurs. The game offers each player a first‐person view into the limited and shared 3D game world. The narrative story of the game is that players work as volunteers at a charity concert for human sustainability, and they are expected to ensure that customers are satisfied and all the necessary preparations are made for the band. The game comprises three pedagogically scripted multiplayer puzzle‐type tasks that need to be solved in a particular order. The intended duration of the game is between one and two hours. In the subsequent section, we will briefly describe the aims of the puzzles in general and those from the collaboration perspective in particular. The first puzzle (Gate) acts as an introduction to the gameplay. The general aims of the level are to practice moving and performing actions in the game to facilitate group formation and open a discussion between the players. In terms of collaboration, the goal of the level is to promote coordination, including personal responsibility, dependency among players, and control of an aggregate of individuals (Barron 2000). The second puzzle (Restaurant) is more complex than the first. The general aim of this level is to encourage players to share information with each other and to keep others informed of their positions and activities to organize and synchronize their individual actions as a whole (Figure 2). In terms of collaboration, the goal is to create dependencies among the group members by distributing knowledge and resources to each of the players (Price et al. 2003). The third puzzle (Stage) is structurally very simple; however, it poses challenges in terms of the interactions among the players. The general aim of the puzzle is to encourage each of the players to share all the information that they have with other players, actively participate in shared problem solving, and build a shared understanding on the basis of partially contradictory information. In terms of collaboration, the goal is to drive the players into a situation where they are faced with cognitive conflict; players simultaneously receive inconsistent information which without proper coordination creates an unsolvable problem (Chan and Chan 2001). In Figure 2, we present an example to illustrate the simplified and collaboration‐focused structure of the second pedagogically scripted multiplayer task included in the game. The figure shows how the players’ actions are associated with the state of the key object (customer) of the task. Moreover, it demonstrates how players’ individual tasks are interconnected to necessitate interaction and collaboration among the group members.

5. Study outline, participants, data collection, and methods Three empirical component studies were conducted to respond to the set of research questions. Empirical studies were conducted in an authentic classroom setting from fall 2010 to spring 2012. Data were recorded from 62 students and 24 teachers (18 groups of four to five persons) in vocational (N = 69) and higher education (N = 17) (total N = 86) schools. Of these participants, 65% were males (N = 56) and 35% were females (N = 30). The participants in the empirical studies were randomly selected from all the schools involved in the study. The experiment included a one‐ to two‐hour gaming period in Game Bridge at the college of Jyväskylä, Finland, and at the University of Jyväskylä, Finland. The average duration of the game was 104 minutes.

685

Kimmo Oksanen and Raija Hämäläinen

Figure 2: Event chart of restaurant‐puzzle The number of participants varied among the component studies (Table 1). All the participants took part in component studies I and II (N = 86). Component study III comprised 20 participants (4 groups), in which two groups played with (setting A) and two groups played without (setting B) real‐time teacher orchestration. Data gathered during the experiment comprised survey data (component studies I and II) and discussion data (component study III). Table 1: Outline of the study Component study I

N 86

Data Survey data (Game Experience Questionnaire)

III

Survey data (Game Experience Questionnaire and Sociability Scale) Discussion data

Methods Descriptive statistics, one‐sample t‐test, and analysis of variance (ANOVA) Bivariate correlation Content analysis

Survey data were gathered by using the Game Experience Questionnaire (GEQ) (Poels, De Kort and IJsselsteijn 2008) and the Sociability Scale (Kreijns et al. 2007). The GEQ intended to cover a range of digital game experiences (Poels et al. 2008) and included its own modules for core game experience and social presence. The core module covered the following dimensions of the game experience: (1) flow, (2) immersion, (3) competence, (4) challenge, (5) positive affect, (6) negative affect, and (7) tension. The social presence module comprised (1) empathy, (2) negative feelings, and (3) behavioral involvement. Overall, the questionnaire included 59 statements. Each statement was evaluated on a five‐point scale, ranging from one (not at all) to five (extremely). The Sociability Scale is intended to measure the perceived sociability of a CSCL environment (Kreijns et al. 2007) and comprises 10 statements. The original questionnaire was modified on one item because the original statement, “This CSCL environment enables me to make close friendships with my teammates,” was considered irrelevant because the gaming sessions were short. The original statement was replaced by another, “This CSCL environment helped me to understand the perspective of other group members,” which was considered more relevant in this context. Each statement was evaluated on a five‐point scale, ranging from one (not applicable at all) to five (totally applicable). To answer the first research question (What kind of subjective experiences (core game experience and sense of social presence) are generated by the players by playing a collaborative serious game?), we used descriptive statistics, one sample t‐test, and analysis of variance (ANOVA) for analysis purposes. With respect to the second research question (What is the relationship between the social dimension of gaming, including sense of

686

Kimmo Oksanen and Raija Hämäläinen social presence and sociability of the game, and core game experiences?), bivariate correlation was used as the method of analysis. Statistical analysis was conducted by using SPSS version 19. To answer to the third research question (What are the main differences in the knowledge construction processes between the games conducted with and without real‐time orchestration?), we employed qualitative and quantitative content analysis (Berelson 1952). The following four phases were used for analyzing the discussion data: (1) revising the data, (2) comparing the time used, (3) analyzing collaboration with content analysis, and (4) comparing the differences among the learning settings (for a more precise description, see Hämäläinen and Oksanen 2012; Oksanen and Hämäläinen 2013).

6. Results This section reveals new understanding of the game experiences, relationships among the social dimensions of gaming, and teachers’ roles in game‐based learning. The results of component studies II and III have already been published elsewhere (Hämäläinen and Oksanen 2012; Oksanen and Hämäläinen 2013). The article concerning the results of component study I has been accepted for publication (Oksanen, in press). Subjective experiences generated by collaborative serious games Players rated the dimensions of the core game experience as follows: flow (M = 2.95, SD = 0.93), competence (M = 3.12, SD = 0.67), immersion (M = 2.79, SD = 0.79), challenge (M = 2.35, SD = 0.60), positive affect (M = 3.44, SD = 0.77), negative affect (M = 1.98, SD = 0.74), and tension (M = 2.00, SD = 0.71). Thus, in general, players felt that game experience was fairly positive and besides they felt feelings of competence. This is supported by the fact that positive affect and competence received values that were higher than the scale midpoint. Positive affect was further found to be significantly higher than the scale midpoint (t(85) = 5.28, p < .001). From the perspective of engagement, players reported an average level of engagement, especially in the form of flow. Since flow can be interpreted as involvement in an activity and is thus related to the characteristics of the game tasks (Weibel and Wissmath 2011), these results suggest that the collaborative game tasks were meaningful and challenging enough to capture the players’ attention. From the social presence perspective, players felt strong positive psychological involvement (M = 3.35) and behavioral involvement (M = 3.46) with each other. This finding indicates that solving the puzzles required interprofessional expertise and shared knowledge construction. Further, deep positive psychological involvement (i.e., empathy) is a strong indicator of the positive interdependence that existed among the teammates. Negative feelings were weaker (M = 1.94, SD = 0.49) and the deviation from the scale midpoint was statistically significant (t(85) = ‐19.99, p < .001). The results also showed that as compared to the students, the teachers (usually with lower gaming activities) felt that the gameplay of the game was more challenging (F (1, 84) = 11.03, p = .001). However, this does not necessarily mean that the gameplay was not a positive and engaging experience. The participants’ background did not significantly influence the ratings of the sense of social presence. Relationship between the social dimension of gaming and core game experiences Players rated the sociability of the game to be fairly high (3.59) and this value was significantly higher than the scale midpoint (t(85) = 6.60, p < .001). Besides the description of the general game experiences, the relationship between the feelings of social presence and sociability in the game environment are crucial from the perspective of collaborative learning. Our previous results (Oksanen and Hämäläinen 2013) have indicated that the collaborative serious game facilitated and supported players’ socio‐emotional processes. This was further proposed to be associated with the emergence of productive social interactions and collaborative activities (Rourke 2000; Guzzo and Dickson 1996). Furthermore, the current results showed that the sociability of the game and sense of social presence are strongly associated with the core game experiences. More precisely, the sociability of the game correlated significantly with every dimension of the core game experience except tension. The strongest correlation was between sociability and positive affect (r = .638, p < .001). From the perspective of social presence, the highest correlation was found between empathy and positive affect (r = .705, p < .001). Additionally, empathy and behavioral involvement correlated strongly with flow, positive affect, and immersion. Conversely, negative

687

Kimmo Oksanen and Raija Hämäläinen psychological involvement seemed to reinforce the negative aspects of the game experience. This finding indicates that behavioral involvement and empathy among the group members strengthen the positive dimensions of the gaming experiences (such as positive affect and flow). Moreover, apparently, there is also a strong association between the sociability of the game and positively toned core game experiences. Role of real‐time orchestration for the collaborative knowledge construction process in game settings New types of learning environments, such as serious games, challenge teachers to find new ways to support the learning processes of students. Our results indicate that teachers’ participation in the gameplay as external orchestrators influences the knowledge construction processes of learners (Hämäläinen and Oksanen 2012). This is supported by the fact that game groups that incorporated teachers’ real‐time orchestration invested more effort in providing knowledge (A = 41.7% of all utterances, B = 24% of all utterances), especially when explaining their own situations (A = 18%, B = 5.9%). This result indicates that teachers’ real‐time orchestration encouraged learners to explain one’s own activities more actively to develop mutual knowledge and solve scripted puzzles. Another major difference between these two settings is related to the focus on problem solving. More precisely, real‐time orchestration focused learners’ discussions on task solving, thereby significantly reducing the amount of time spent on off‐task talk (A = 1.5%, B = 15.2%). This finding indicates that groups playing without real‐time orchestration become more easily disconcerted, drifting to discussing other issues, when the same problem occurred. Thus, groups playing with real‐time orchestration succeeded in productive knowledge construction, that is, building on others’ ideas and thoughts.

7. Conclusions Collaborative serious games are suggested to have the potential to support and facilitate collaborative learning by functioning as a particular type of sociable CSCL environment (Oksanen and Hämäläinen 2013). However, a game as a technological solution is not an overarching solution to the multiple challenges faced when striving to achieve high‐level collaborative learning (Hämäläinen 2011). Promising attempts have been made to bridge the gap between instructional design and game design in the design of serious games (Hummel et al. 2011). However, the interrelationships among decisions related to the design and use of collaborative serious games, learners’ experiences, and collaborative knowledge construction has rarely been studied. An insight into learners’ experiences is essential because emotional experiences ultimately guide players’ actions and decisions during the game. Similarly, the instructional design perspective is important because collaborative serious games pose new challenges for teachers in their pursuit to support learning. In this study, which combined three empirical component studies, we focused on different aspects of facilitating and supporting collaborative learning in serious games. In particular, we aimed to empirically investigate learners’ subjective experiences generated by collaborative serious games and determine teachers’ roles from the perspective of collaborative knowledge construction processes in game settings. The limitations with regard to the findings of the study are as follows. First, our empirical study could not confirm any linkage between the results of the component studies related to the players’ subjective experiences and the amount and quality of social interaction that emerged during the game; therefore, conclusions are based on assumptions. Second, our study did not measure the actual learning outcomes of the game; instead, it focused on evaluating the processes of collaborative learning during the gameplay session. Third, because only one game environment was used in the empirical study and the number of participants was limited, the results are not widely generalizable. Strength of the current study is that the game used in this study was specifically designed to promote and support social interaction and collaborative activities. Thus, it was more suitable for use in this framework than, for example, in commercial multiplayer online games. The results of the empirical studies indicated that, from the game experience perspective, playing the game generated fairly positive experiences and an average level of engagement. Engagement is reflected in players’ subjective feelings of flow, which indicated that the designated game tasks were meaningful and challenging enough to capture the players’ attention (Weibel and Wissmath 2011). From the perspective of social presence generated during the game, the results revealed that the members of the group exhibited strongly positive psychological and behavioral involvement with each other. This finding indicates that solving the

688

Kimmo Oksanen and Raija Hämäläinen designated game tasks required interprofessional expertise and shared knowledge construction, which is in line with our original intention. The results of this study also revealed that besides a fairly positive general game experience, the sociability of the game facilitated and supported players’ socio‐emotional processes. Thus, our results are consistent with the theory that the players’ socio‐emotional processes may be determined by the sociability of the environment (Kreijns et al. 2007). This, together with a high degree of social presence, has been suggested to have a significant effect on collaborative activities among the group members (Rourke 2000). Further, because a sense of social presence and the sociability of the game are strongly associated with the core game experiences, especially in collaborative game settings, game engagement in these settings may be augmented through the social dimension of gaming. Finally, the results of the study revealed that teachers’ participation in the game as an external orchestrator influences the knowledge construction processes. Real‐time orchestration enabled learners to focus on task solving, develop a joint understanding of the interprofessional nature of tasks, and explain their own situation and activities as a part of interprofessional task solving. In conclusion, teachers’ real‐time orchestration seems to be a plausible way of improving the joint knowledge construction processes and enhancing the productivity of collaboration in a game context, while simultaneously improving the internal guidance of the game. In sum, the key challenge in a serious design game is to design environments that promote collaborative activities, while supporting learners’ socio‐emotional processes. This study indicated that scripted games may be one way to support these social dimensions of gaming. However, it also illustrated that serious game environments challenge teachers to find new ways to support learning. Therefore, in the future, systematic empirical studies need to be conducted not only on technological development but also on learners’ and teachers’ experiences related to high‐level learning in game settings.

Acknowledgements This research was supported by the University of Jyväskylä, Faculty of Education. The development of the game was supported by EU Structural Funds and nationally by the Centre for Economic Development, Transport, and the Environment of Central Finland.

References Barron, B. (2000) “Achieving coordination in collaborative problem solving groups”, The Journal of the Learning Sciences, Vol. 8, No. 4, pp. 403‐436. Berelson, B. (1952) Content analysis in communication research. New York: Free Press. Bluemink, J., Hämäläinen, R., Manninen, T. and Järvelä, S. (2010) ”Group‐level analysis on multiplayer‐game collaboration: how do the individuals shape group interaction?” Journal of Interactive Learning Environments, Vol. 18, No. 4, pp. 365‐383. Chan, E.H.W. and Chan, A.P.C. (2001) “Conflict management pertaining to design information, in international construction projects”, Journal of Architectural Management, Vol. 16, pp. 32‐57. Collazos, C., Guerrero, L., Pino, J., Ochoa, S. and Stahl, G. (2007) “Designing collaborative learning environments using digital games” Journal of Universal Computer Science, Vol. 13, No. 7, pp. 1022‐1032. Echeverria, A., García‐Campo, C., Nussbaum, M., Gil, F., Villalta, M., Améstice, M. and Echeverria, S. (2011) “A framework for the design and integration of collaborative classroom game”, Computers & Education, Vol. 57, No. 1, pp. 1127‐ 1136. Guzzo, R.A. and Dickson, M.W. (1996) “Teams in organizations: recent research on performance and effectiveness”, Annual Review of Psychology, Vol. 47, pp. 307‐338. Hummel, H., van Houcke, J., Nadolski, R., van der Hiele, T., Kurvers, H. and Löhr, A. (2011) “Scripted collaboration in serious gaming for complex learning: effects of multiple perspectives when acquiring water management skills”, British Journal of Educational Technology, Vol. 42, No. 6, pp. 1029‐1041. Hämäläinen, R., Oksanen, K. and Häkkinen, P. (2008) “Designing and analyzing collaboration in a scripted game for vocational education”, Computers in Human Behavior, Vol. 24, No. 6, pp 2496‐2506. Hämäläinen, R. (2011) “Using a game environment to foster collaborative learning: a design‐based study”, Technology, Pedagogy and Education, Vol. 20, No. 1, pp 61‐78. Hämäläinen, R. and Oksanen, K. (2012) “Challenge of supporting vocational learning: empowering collaboration in a scripted 3D game—how does teachers’ real‐time orchestration make a difference?”, Computers & Education, Vol. 59, No. 2, pp 281‐293. Interprofessional Education Collaborative Expert Panel. (2011) Core competencies for interprofessional collaborative practice: report of an expert panel. [e‐book] Washington, D.C.: Interprofessional Education Collaborative. Available through:<https://www.aamc.org/download/186750/data/core_competencies.pdf> [Accessed 10 June 2013]

689

Kimmo Oksanen and Raija Hämäläinen Kennedy‐Clark, S. and Thompson, K. (2011) “What do students learn when collaboratively using a computer game in the study of historical disease epidemics, and why?” Games and Culture, Vol. 6, No. 6, pp. 513‐537. Kobbe, L., Weinberger, A., Dillenbourg, P., Harrer, A., Hämäläinen, R. and Häkkinen, P. (2007) “Specifying computer‐ supported collaboration scripts”, International Journal of Computer‐Supported Collaborative Learning, Vol. 2, No. 2/3, pp. 211‐224. Kreijns, K., Kirschner, P., Jochems, W. and van Buuren, H. (2007) “Measuring perceived sociability of computer‐supported collaborative learning environments”, Computers & Education, Vol. 49, No. 2, pp. 176‐192. Nussbaum, M., Szewkis, E., Rosen, T., Abalos, J., Denardin, F., Caballero, D., Tagle, A. and Alcoholado, C. (2011) “Collaboration within large groups in the classroom”, International Journal of Computer‐Supported Collaborative Learning, Vol. 6, No. 4, pp. 5611‐575. Oksanen, K. (in press) “Seriously fun – assessing subjective game experiences in a 3D collaborative serious game”, Simulation & Gaming. (Accepted for publication June 2013) Oksanen, K. and Hämäläinen, R. (2013) “Perceived sociability and social presence in a collaborative serious game”, International Journal of Game‐Based Learning, Vol. 3, No. 1, pp 34‐50. Poels, K., de Kort, Y.A.W. and IJsselsteijn, W.A. (2008) The game experience questionnaire: development of a self‐report measure to assess player experiences of digital game (FUGA Deliverable D3.3.Technical Report). Eindhoven: TU Eindhoven. Price, S., Rogers, Y., Scaife, M., Stanton, D. and Neale, H. (2003) “Using ‘tangibles’ to promote novel forms of playful learning”, Interacting with Computers, Vol. 15, No. 2, pp. 169‐185. th Rourke, L. (2000) “Operationalizing Social Interaction in Computer Conferencing” In Proceedings of the 16 Annual Conference of the Canadian Association for Distance Education. Quebec City. Weibel, D. and Wissmath, B. (2011) “Immersion in computer games: the role of spatial presence and flow”, International Journal of Computer Games Technology, Article ID 282345. doi: 10.1155/2011/282345

690

Playing for the Future ‐ Examining Gameplay, Narrative and fun in Games‐Based Training Mark O’Rourke Victoria University, Melbourne, Australia mark.orourke@vu.edu.au Abstract: This research demonstrates how computer games can provide effective skill acquisition and knowledge transfer in the vocational education and training (VET) context and, in particular, increase learner engagement in theoretical subjects. The study tested the rationale behind making a pedagogical shift from content delivery to designing experience. It further investigated whether games‐based learning adds meaning and relevance to VET outcomes through considering the impact of game components of narrative, fun and gameplay in a games‐based learning activity system by utilising a Design Based Research methodology, within an Activity Theoretical framework. By identifying and measuring the game components that impact most on educational outcomes, the research informs the development of alternative delivery strategies and training tools, and advances the knowledge base of utilising games‐based learning. This study importantly addresses a distinct gap in the VET market for effective and engaging immersive training that taps into learners’ individual needs. It finds that games‐based learning, which is agent‐driven, experiential and process‐based, is particularly suited to VET learners as critical information is delivered in real time through in‐game actions and interactions, rather than through alienating paragraphs of written text. Data was collected from the design, development and trialling of three 3D first‐ person shooter learning games, which were offered as alternatives to existing VET curriculum and delivery. Game trials were undertaken and learners reported a preference for games‐based learning over traditional delivery methods. They also expressed greater understanding of both the learning content and the connection to vocational outcomes. The learning context was transformed from one of conventional pedagogy to a "context with consequentiality" (Barab et al. 2010) where game parameters provided the impetus to increase student agency (Calleja 2009) in achieving learning outcomes. Data included communication documentation, observations, pre‐ and post‐tests, surveys, interviews, and in‐ game data collection measuring students’ game playing performances. This research offers a novel approach by aligning data collection from the use and development of game products and research outcomes in order to meet current industry training needs. Customisation of the game environment allowed learners to take on workplace identities. By engaging with virtual work‐based situations, learning was contextualized and expertise developed through cycles of learning and practice (Yelland 2007). Games‐based learning was found to cater for the learning styles of VET students who tend to be: more visual than verbal (they like to watch and see rather than read and listen); hands‐on learners (prefer to learn by doing and practicing); characterised by socially contextualised learning preferences (learning in groups with others); and are not self‐ directed learners, but like to have instructor guidance and a clear understanding of requirements, which was addressed through the scaffolded nature of learning through gameplay (Smith & Dalton 2005). In addition, it was noted that the game also fostered teacher development in utilising interactive delivery tools. Adoption of innovative learning technologies is challenging in VET, however the design and implementation of the games‐based tools effectively facilitated teacher usage by focusing on the work‐based game scenarios rather than the technological functionality. Keywords: immersive; engagement; vocational; agency; games‐based

1. Introduction The purpose of this research was to analyse the components and interactions of games‐based learning contexts by describing ways in which game parameters impact on specific vocational education and training (VET) learning outcomes. This research presents a novel approach by aligning data collection and research from the development and use of game products to meet current industry training needs. Through the design, creation and evaluation of computer‐based learning games as alternatives to existing VET curriculum and delivery, the research informs the development of alternative games‐based delivery strategies. There is substantial discussion in the literature about how learning technologies are best implemented in teaching contexts (Carliner & Shank 2008; Laurillard 2009; Moyle 2010). Early use of computer based education had limited success because the implementation involved students learning from technologies as disseminators of knowledge rather than "cognitive tools" (Kim & Reeves 2007). Yelland (2007, p. 2) proposes that rather than "mapping the use of new technologies onto old curricula" we should be rethinking curriculum and pedagogy to leverage the impact that new technologies can have on "learning and meaning making." This impact is expressed in the ability of new technologies to engage, motivate and be mobile. New approaches for delivering VET curriculum are critical if learners are to be equipped to meet the needs of st contemporary society and develop capabilities to manage the complexities of living and working in the 21

691

Mark O’Rourke century. The VET sector has traditionally been competency based, however being truly competent requires demonstration of skills in a workplace context that involves social situations and skills beyond basic training competencies. This research study aligns VET performance criteria (learning outcomes) with gameplay scenarios and engages with developers, teachers and learners in the development and trial of three immersive first‐person shooter style educational games. The benefit of using games technology is not restricted to simply modeling a simulated environment, but includes real world scenarios and decision making processes as an integral part of the game. The results from this approach of aligning performance criteria with gameplay scenarios indicated particular suitability to VET delivery and suggested a new digital learning ecology for the VET sector. The games‐based contexts offered an agent‐driven, experiential, process‐based learning method, a style of delivery that is particularly suited to VET learners who are:

more visual than verbal, in that they like to watch and see rather than read and listen;

hands‐on learners who prefer to learn by doing and by practicing;

characterised by socially contextualised learning where they like to learn in groups with other learners;

not self‐directed learners, but like to have instructor guidance and a clear understanding of what is required of them.

(Smith & Dalton 2005) The research examines a pedagogical shift from content delivery to designing learning experiences and investigates whether games‐based learning adds meaning and relevance to VET outcomes. In particular whether games‐based training is more relevant than conventional training for achieving VET outcomes in the st 21 century, in its capacity to offer simulated real world scenarios and enhance learner agency. The research also explores the game parameters of narrative, fun and gameplay, and their impact on the educative process through the games‐based learning activity. These considerations are examined by adopting a Design Based Research approach, within an Activity Theoretical framework.

2. Background In the context of games‐based learning new knowledge is created in the process of exploring the game world and taking on the challenges presented to the learner. While content integration provides an engaged experiential learning experience (Egenfeldt‐Nielsen 2007), the rapid evolution of technologies and software offers challenges for educators to reconceptualise curricula and pedagogy and develop digital literacies that will provide optimal conditions for teaching, learning and creative inquiry. Yelland and Tsembas (2008, p. 107) propose that "pedagogies need to be reconceptualised to suit the new learning environments" and that we should focus on the nature of the content that learners are encountering rather than simply adopting a process of mapping new technologies onto outdated pedagogical models. This is particularly pertinent for VET in that the opportunity for learners to get on the job training with the appropriate amount of supervision to ensure a safe and meaningful experience is often limited. The introduction of games‐based virtual learning can address this gap. When learners are required to remember static knowledge that does not support meaningful understanding and where there is no accountability and authority attached to the knowledge acquisition and use, the learning process is undermined (Gresalfi et al. 2008). In contrast, games are action and goal oriented that rewards player’s agency and problem solving skills. Games provide opportunities for players to choose when and where they use different content. Games‐based learning systems are programmed to constrain user’s actions and movements in the game world, which subsequently directs attention to important tasks and learning goals through these predefined rules. In this way they can facilitate learning by scaffolding learners through unique representational environments and allocating extraneous cognitive load to technological tools (Sharma & Hannafin 2007). Computer games have a unique capacity to enable game players to generate a narrative by being able to designate author, performer and audience roles to the user simultaneously. Game players have the opportunity to perform actions, experience consequences, and reflect on the decisions they make. The capacity for computer games to be powerful pedagogical tools can leverage off this narrative generation by

692

Mark O’Rourke scaffolding students in engaging with and critically analysing pedagogical content embedded in the narrative structure. This person–story coupling has been called narrative transactivity (Barab et al. 2010) and performs a metacognitive function in the analysis of action and understanding in gameplay. Making learning fun is a powerful incentive to engage students in the educational process. Whether or not a computer game offers enjoyment to a player impacts on the level of involvement and engagement in the game. This can subsequently be a determining factor for learning when games are used for education. Players experience the results of decisions they make and are able to influence the game world with a responsive agency that delivers determination and empowerment to the player (Klimmt, Hartmann & Frey 2007). This challenging active experience had from mastering hard and complex gameplay has been coined "hard fun" (Papert 1998). Klimmt (2003) proposed a conceptual model of game enjoyment that was dependent on factors at three levels. The first level describes immediate feedback loops from the game system that supplies user agency and interactivity in the gameplay. The second level consists of a series of episodes triggered by the player's motivations and are expressed as cyclical feelings of suspense and relief. At the third level the player's engages with the narrative and experiences the alternative reality of the game environment. The theory describes how the feelings of suspense, anxiety and physical arousal that occur during gameplay are interpreted positively because players anticipate a resolution such as winning the game or completing a task. When the task is completed the arousal is turned into euphoria, and the player thereby experiences the cycles of suspense and relief as pleasurable (Ermi & Mayra 2005). Gameplay can be defined as the mechanics of the game, or the game components that produce decision and action by the player. These components include aspects of the narrative as it guides the player in the game world and enjoyment as it provides the motivation to develop the agency to undertake tasks and achieve goals. The research examined the interplay of the game system components and the effect they had on learning outcomes. Integral to this are the game mechanics or physical interaction with the game world through the game controller interface (Dickey 2011). Cognitive research suggests that perception and action are deeply interconnected (Gee 2007) and that in the game world players feel as though their bodies and minds have extended into the virtual environment (Clark 2003). Gameplay can be examined by focusing on the player as "the essence of a game is rooted in its interactive nature, and there is no game without a player" (Ermi & Mayra 2005). Gameplay is experiential with players being situated in a space where they have a defined role and their actions affect a specific context. Unlike many pedagogical situations where the trainer is responsible for outlining a context and delivering content that may be relevant at some future time, games supply an actionable context that resonates with and is responsive to the players requirements and goals (Barab, Gresalfi & Ingram‐Goble 2010). This context with consequentiality is an experiential consequentiality in pedagogy that is quite different to the arbitrary consequentiality of traditional assessment practices of submitting assignments in exchange for grades. Games also supply consequential feedback, which empowers players by allowing them to experience the impact of their in‐game decisions, learning through both their successful actions in the game and from making mistakes and failing tasks.

3. Research framework The research methodology used in this research implements a Design Based Research approach, within an Activity Theory framework that facilitates the analysis of interactions between components and their impact on learning outcomes in the games‐based learning environments. The computer game components of the activity system examined include gameplay, narrative, and fun. The design includes a focus on how the curriculum can be integrated most effectively so that learning becomes implicit whilst the user plays the game, rather than explicitly emphasising the educational content through the use of text based material presented outside the gameplay scenario. The Activity Theory (Engestrom 1987) framework facilitates this analysis because many of the assumptions of Activity Theory are represented in the games‐based activity system, in particular Activity Theory illustrates how the effectiveness of any learning process is dependent upon the interactions and consequent transformations of personal, social, cultural and technical elements within its system.

693

Mark O’Rourke Through the collaboration of researchers and practitioners in real‐world settings (Wang & Hannafin 2005) Design Based Research was used to examine the educational activity and gain insight into "how, when, and why innovations work in practice" (Dede et al. 2004). When applied to the application of technology in education it actively involves: students acquiring skills or knowledge; teachers or facilitators; learning support tools; and technological resources. Consequently, the use of games technology for training impacts significantly on the social organisation of the learning environment, affecting both student learning and teaching practice. The games‐based activity system in this research includes both object‐oriented production and person‐ oriented communication, and cognition is distributed across all components of the system. The Activity Theoretical framework enables a focus on the interplay of game components and cultural factors in the games‐based learning activity in order to examine cognitive transformation. This included the intentions and interactions of: the VET teachers using the games‐based tools; the designers developing the games; the students who were learning from the games; and the researcher as a critical participant. The three games developed for the research targeted different student cohorts, had different members in the development teams, and addressed different VET curriculum, however all included aspects of workplace safety in the learning outcomes. Play It Safe addressed 3 Units of Competency at Certificate and Diploma level from MEM05: Metal and Engineering Training Package. Lab Safe addressed one Unit of Competency at Certificate and Diploma level from PML04 ‐ Laboratory Operations Training Package. The White Card Game addressed one Unit of Competency at Certificate level from CPC08 ‐ Construction, Plumbing and Services Training Package. Data collection techniques included in‐game data collection of students’ gameplay activity during trials, hard copy surveys of 12 questions administered to students immediately after the game trials, interviews with students after the trials, observations of students and teachers in the classroom while students were engaged in playing the game and communication documentation which included emails and notes from discussions and meetings with VET teachers and game developers during planning, production and trialing of the games. In addition user data was collected from the VET games through in‐game data generation mechanisms that measured the student’s game playing performance. The dynamic nature of the games technology allows for data collection mechanisms to be programmed into the game design thereby generating useful data about the decisions users are making in‐game and the choices they make in relation to educational content.

4. Findings and discussion The components of the games‐based learning activity systems were analysed, in particular examining the contradictions and mediators and how these have impacted on progressing the activity outcome. Observations of students playing the VET games clearly indicated that they were having fun. They were engaged with the learning content and interacting with their peers and teacher, and the majority found this an entertaining experience. The survey responses indicated that 74% of students enjoyed playing the games, 70% had fun and 78% found the games engaging. Coupled with 71% stating that they learnt about the topic playing the game it is apparent that having fun and being engaged is linked to successful learning outcomes. This is supported by students’ comments: PP: "Better than doing the written work, and sittin down. You learn more if you are having a good time." SG: "It was better than doing the text, it was more interactive, better than just sitting there and looking at a bit of paper, more enjoyable" LP: "humour good though, you joke about things, not obviously though, keep it entertaining in that regard, the apprentices learn about it more, sometimes coming across as deadly serious doesn't drum it in as much" DD: "A lot better, good concept, more visual and fun, more interaction, you feel better when you do something." AC: "sticks with you, you might joke about it but you are always thinking about it" The positive relationship between fun and cognitive processing would suggest that learning is more successful when learners are being entertained (Singhal & Rogers 2002). In the development of a scale that effectively

694

Mark O’Rourke measures the enjoyment offered by educational games Fu and colleagues (2009) found that game enjoyment is a key factor in determining player involvement and whether players continue to learn through the game. In addition, the capacity for games to engage through flow (Csikszentmihalyi 1990; Sherry 2004) and cognitive challenge while presenting less interesting subject material can enhance engagement with the content, and provide an opportunity to convey critical knowledge. This was confirmed by teachers who commented that it was not unusual for students to fail to return to class after a break when workplace safety issues were presented in a conventional powerpoint presentation style. This was in contrast to the White Card Game trials where students stayed after class playing the game multiple times in an atmosphere of engaged competitive enjoyment trying to improve their scores and being emotionally involved in the learning content. The game narrative enhances transformative learning by framing game expectations within a vocational context and directly influencing the meanings derived from players’ experiences (Mezirow 2000). All the VET games present critical events to the user and the subsequent revision of meaning occurs through the game story. This was supported by 83% of students agreeing or strongly agreeing that there was a logical sequence of events in the game. Students’ comments also represented the relevance of the narrative to their learning: JB: "You couldn't just do random things, you had to know where things go, lift things properly, safely, know what you were doing." TO: "Made you think, rather than rushing it, what I should really do here, thinking how to do hazards, to prevent them." KO: "Found I was always going back to supervisor to check what I had to do." BB: "A little bit, like forgot to put the guard down then realised. Learnt stuff you wouldn’t normally pick up on. The game world enables players to be protagonists to make choices that advance the unfolding story line. This was supported by 84% of students agreeing or strongly agreeing that they became more involved in the game as the game progressed. The player as "Person With Intentionality" (Barab, Gresalfi & Ingram‐Goble 2010) makes choices in the game context that reveals consequences for players’ decisions. Game‐world dilemmas aligned with the learning content are resolved by player action. Student’s comments indicated that learning was occurring through the resolution of issues presented in the narrative: LK: "Make sure you don’t do things too quickly, skip process you end up getting hurt." MN: "You got consequences if you stuff up." TM: "Useful to do everything in sequence, like switching off gas and electricity in the fire emergency before leaving. These I cannot find in real life. When I have to do experiments, disposing of waste, when I finish experiments I have to dispose of waste properly, not just get the result of the experiment." LL: "You are forced to, play it through the game, regardless you pick up the information the game is trying to teach and if you want to finish the game…" The narrative in the games is particularly relevant to a VET context where players are training to achieve vocational outcomes. Game players draw on culturally available narrative components to contextualise their actions (Louchart & Aylett 2004). In the games trials VET learners were able to draw on or refer to the vocational context, and in doing so were able to create and explore new possible scenarios relevant to their future vocation. Play It Safe and the White Card Game implemented Narrative Games‐based Learning Objects (NGLOBs) (Göbel et al. 2009) in the form of cut scenes that present the consequences of failing a scenario. These are represented as the graphic death sequences of failing to perform operations safely. This implementation is representative of a "pull" narrative instead of the more traditional "push" mode of communicating story (Calleja 2009), whereby the story emerges from the players’ interaction with the environment and that interaction generates, not excludes the narrative. The capacity for computer games to be powerful pedagogical tools leverages off this narrative generation by scaffolding students in engaging with and analysing pedagogical content embedded in the narrative structure. This narrative transactivity (Barab et al. 2010) performs a metacognitive function in the analysis of action and understanding in gameplay.

695

Mark O’Rourke Gameplay is the motivating force that ensures players move towards achieving the learning outcome. Observations of students interacting with the VET games indicated high levels of engagement, social interaction and motivation to achieve game goals. Survey responses supported this with 78% responding that they found the game engaging. The actionable context, which is responsive to the players’ requirements and goals (Barab, Gresalfi & Ingram‐Goble 2010), provides a level of immediacy and supplies responsive consequential feedback. This feedback empowers players by allowing them to experience the impact of their in‐game decisions, learning through both their successful actions in the game and from making mistakes and failing tasks. The gameplay is experiential with players having a defined role and being situated in the vocational space where they and their actions affect a specific context. LM: "Thing with a game is that like you want to win as well, you want to beat it, and obviously if you want to beat it you got to do it properly." DR: "The gaming part is that you gotta learn it, you cant bluff your way through it, you gotta know it, and that's really good." Through performing actions, experiencing consequences, and reflecting on the decisions they make, users develop a goal directed sense of agency as they engage with learning content in games‐based delivery. The game structure offered affordances for the player to develop skills while interacting with the learning content in the vocational setting. Player agency is demonstrated by the students’ comments below, and also through classroom observations that revealed active engagement with the game, motivation to succeed in game tasks, and immersion in the game world. HG: "Kept you more on track, more focused, instead of sitting down and writing or anything like that." DR: "The gaming part is that you gotta learn it, you cant bluff your way through it, you gotta know it, and that's really good." TO: "Made you think, rather than rushing it, what I should really do here, thinking how to do hazards, to prevent them." The VET game context intrinsically scaffolded learning by engaging students in an active goal directed experience, integrating learning and assessment with gameplay and providing responsive real time feedback. In addition the games enabled extrinsic learning by fostering peer to peer interaction in the classroom; and empowering teachers through the capacity to facilitate and advise on correct gameplay choices by drawing on their expert knowledge.

5. Conclusion This research engaged with developers, teachers and learners and gathered rich data from their interactions with each other and the game context, and suggests new learning ecologies for the VET sector. The results show that games‐based training offers enhanced learning and teaching outcomes by aligning gameplay and performance criteria with VET learning styles. The research outcomes include strategies for best situating subject matter in educational games‐based contexts that enable learners to meaningfully apply disciplinary content, and in particular highlights the importance of considering the role of game design elements as narrative, fun, and gameplay. The research provides evidence that learning is enhanced in vocationally represented virtual environments that supply action and goal oriented consequential experiences. The research outcomes make a significant contribution to sustainable training practices by providing the framework to develop and evaluate new games‐based learning contexts for ongoing education and training solutions. Operational changes and technical developments in industry can have a negative impact on productivity if timely upskilling and training of staff is not addressed. This research suggests an exciting future for targeted games‐based training tools in industry, with the potential for increased efficiency, a decrease in avoidable workplace injuries and effective engagement with VET. The research also aligns with current policy directions that are emphasising a shift to work situated vocational education delivery (Skills Australia, 2010).

Acknowledgements I would like to acknowledge Professor Nicola Yelland for her insightful advice and support in the journey of this research.

696

Mark O’Rourke

References Barab, S.A., Dodge, T., Ingram‐Goble, A., Peppler, K., Pettyjohn, P., Volk, C. and Solomoua, M. (2010), "Pedagogical dramas and transformational play: Narratively rich games for learning", Mind, Culture, and Activity, Vol. 17, No. 3, pp. 235‐64. Barab, S.A., Gresalfi, M. and Ingram‐Goble, A. (2010), "Transformational Play : Using Games to Position Person, Content, and Context", Educational Researcher, Vol. 39, No. 525. Calleja, G. (2009), "Experiential Narrative in Game Environments", Digital Games Research Association (DiGRA ) proceedings. Carliner, S. and Shank, P. (2008), The e‐Learning Handbook : Past Promises, Present Challenges, Pfeiffer, San Francisco. Clark, A. (2003), Natural‐born cyborgs: Minds, technologies, and the future of human intelligence, Oxford University Press, USA. Csikszentmihalyi, M. (1990), Flow: The Psychology of Optimal Experience, Harper Perennial, New York. Dede, C., Nelson, B., Ketelhut, D.J., Clarke, J. and Bowman, C. (2004), "Design‐based Research Strategies for Studying Situated Learning in a Multi‐user Virtual Environment", Proceedings of the sixth international conference on the learning sciences, pp. 158‐65. Dickey, M.D. (2011), 'Murder on Grimm Isle: The Design of a Game‐based Learning Environment', in S de Freitas, H Jenkins & P Maharg (eds), Digital Games and Learning, Continuum international Publishing Group, London, p. 142. Egenfeldt‐Nielsen, S. (2007), "Third generation educational use of computer games", Journal of Educational Multimedia and Hypermedia, Vol. 16, No. 3, pp. 263‐81. Engestrom, Y. (1987), Learning by expanding. An activity‐theoretical approach to developmental research, Orienta‐ Konsultit Oy, Helsinki. Ermi, L. and Mayra, F. (2005), "Fundamental components of the gameplay experience: Analysing immersion", in Proceedings of DiGRA 2005 Conference: Changing Views – Worlds in Play : International perspectives on digital games research, p. 37. Fu, F.L., Su, R.C. and Yu, S.C. (2009), "EGameFlow: A scale to measure learners' enjoyment of e‐learning games", Computers & Education, Vol. 52, No. 1, pp. 101‐12. Gee, J. (2007), Good Video Games + Good Learning, Peter Lang, New York. Göbel, S., de Carvalho Rodrigues, A., Mehm, F. and Steinmetz, R. (2009), "Narrative Game‐based Learning Objects for Story‐based Digital Educational Games", in Proceedings of the 3rd European Conference on Games Based Learning, Vol. 14, p. 16. Gresalfi, M., Martin, T., Hand, V. and Greeno, J.G. (2008), "Constructing competence: An analysis of students’ participation in the activity system of mathematics classrooms.", Educational Studies in Mathematics, Vol. 70, pp. 49‐70. Kim, B. and Reeves, T.C. (2007), "Reframing research on learning with technology: in search of the meaning of cognitive tools", Instructional Science, Vol. 35, pp. 207–56. Klimmt, C. (2003), "Dimensions and determinants of the enjoyment of playing digital games: A three‐level model", in M Copier & J Raessens (eds), Level up: Digital games research conference, Utrecht, The Netherlands, pp. 246‐57. Klimmt, C., Hartmann, T. and Frey, A. (2007), "Effectance and control as determinants of video game enjoyment", CyberPsychology & Behavior, Vol. 10, No. 6, pp. 845‐8. Laurillard, D. (2009), "The pedagogical challenges to collaborative technologies", International Journal of Computer‐ Supported Collaborative Learning, Vol. 4, No. 1, pp. 5‐20. Louchart, L. and Aylett, R. (2004), "Narrative theory and Emergent interactive Narrative", International journal of Continuing Engineering education and Lifelong Learning, Vol. 14, No. 6, pp. 506‐18. Mezirow, J. (2000), Learning as transformation : critical perspectives on a theory in progress, Jossey‐Bass San Francisco. Moyle, K. (2010), Australian Education Review, Building innovation: learning with technologies, ACER, Camberwell, Australia Papert, S. (1998), "Does Easy Do It? Children, Games, and Learning", Game Developer, p. 88. Sharma, P. and Hannafin, M.J. (2007), "Scaffolding in Technology‐Enhanced Learning Environments", Interactive Learning Environments, Vol. 15, No. 1, pp. 27‐46. Sherry, J.L. (2004), "Flow and media enjoyment", Communication Theory, Vol. 14, No. 4, pp. 328‐47. Singhal, A. and Rogers, E.M. (2002), "A theoretical agenda for entertainment—education", Communication Theory, Vol. 12, No. 2, pp. 117‐35. Skills Australia. (2010). Australian Workforce Futures – A National Workforce Development Strategy. Retrieved from http://www.skillsaustralia.gov.au/PDFs_RTFs/WWF_strategy.pdf. Smith, P. and Dalton, J. 2005, Getting to grips with learning styles., National Centre for Vocational Education Research, <http://www.ncver.edu.au/research/proj/nd3103b.pdf> Wang, F. and Hannafin, M.J. (2005), "Design‐based research and technology‐enhanced learning environments", Educational Technology Research and Development, Vol. 53, No. 4, pp. 5‐23. Yelland, N. (2007), Shift to the Future: Rethinking Learning with New Technologies in Education, Routledge, New York. Yelland, N. and Tsembas, S. (2008), 'Elearning: Issues of pedagogy and practice in the information age', in P Kell, W Vialle, D Konza & G Vogl (eds), Learning and The Learner: Exploring Learning for New Times., Faculty of Education, University of Wollongong. , Newcastle, Australia, pp. 95 ‐111.

697

Towards Game Based Learning Design Process Based on Semantic Service Oriented Architecture (SSOA) Kaouther Raies, Maha Khemaja and Rafik Braham Prince Research Group, Isitc, University Of Sousse, Hammam Sousse, Tunisia kaoutherraies@yahoo.fr maha_khemaja@yahoo.fr rafik.braham@ensi.rnu.tn Abstract: Game Based Learning (GBL) provides increasingly motivating learning environments. Design creation and adaptation of this kind of environments still face many pending problems. In this paper, we propose a novel approach based on Semantic Service Oriented Architecture (SSOA) that aims (i) overcoming GBL design process challenges caused by interdisciplinary actors participating in that process and heterogeneity of exchanged data produced by several and different used applications (ii) integrating an intelligent support in that process in order to help novice actors while carrying their tasks and (iii) producing efficient Game Based Learning Systems where educational gain an fun qualities are certified. Keywords: GBL, GBL design process, SOA, SSOA, BPMO, domain ontology

1. Introduction Games or videogames constitute an increasingly important resource for entertainment and education. They provide exciting and engaging experiences that capture learners’ attention. Therefore, they can be used to get more motivating and engaging educational activities, and could be involved to create an innovative learning technology generation. In this paper, we will focus on the design process of Computer Games Based Learning Systems (GBLS) and the architecture of their underlying execution environment. (Corti 2006) gives an important overview of GBL, and summarizes it as a branch of serious games that deals with applications that have defined learning outcomes. It considers GBL and serious games to be almost close to each other. GBLS seem to have much potential as learning tools, but their use in real scenarios is limited by several factors that span all over the product life cycle affecting the design, implementation and deployment phases. The rest of this paper is structured as follows. In section 2, we present some challenges that face GBLS design phase. In section 3 we present related works and we discuss similarities and divergences with ours. In section 4, we describe our approach, where we outline SOA principles and advantages to fulfill those points. Thereafter, we focus on semantics advents and show how we related them to our future system. Finally, in section 5 we conclude and outline our future works.

2. Challenges of GBLS design process In general, GBLS design involves many actors from different fields and different skills. Game designers like (sound, graphical and instructional designers...) may belong to different companies and work together to keep budgets, schedules, features and bug lists up‐to‐ date. They collaborate across disciplines to bring games to life. Developing GBLS is a time consuming and costly process. It poses not only a budgetary challenge, but also it faces challenges of integrating enough educational value without sacrificing the fun characteristics. Actors participating in this process suffer from many problems like:

Interoperability and communication problems due to interdisciplinary actors participating in that design process, use different tools, manipulate heterogeneous data with non‐standard formats.

698

Kaouther Raies, Maha Khemaja and Rafik Braham

Flexibility problems of pedagogical strategy.

Complexity of each step in a GBL design process which requires relevant expertise and know how even when an appropriate environment is provided.

Figure 1 illustrates major difficulties faced by actors participating in that process.

Figure 1: Technical and pragmatic problems of GBLS design To overcome these challenges, we shall have: (1) an appropriate environment allowing the participating actor to curry out his/her tasks efficiently either alone or collaboratively. (2)The same environment should allow the process monitoring. (3) The overall system should be enough flexible and able to cope with business domain changes or IT changes. (4) The solution should allow automatic services and processes re‐use, which is meant by intelligent composition over services and processes. So, to address point (1), (2) and (3), we are convinced that Service Oriented Architecture (SOA) constitutes the best approach. However, to address point (4) more semantics should be considered. So more specifically, we attempt in this paper to define an approach based on both Semantic Services Oriented Architectures (SSOA) and some ontologies as (domain ontology of Game Based Learning design) which aim to provide relevant environment to instructional and game designers allowing them to design more easily Game‐ Based Learning materials and processes.

3. Related works As stated previously, GBLS are considered as serious games. Design process of those systems are considered as works related to ours as they deal with a part of the overall Game Based Learning design process Most research works related to serious games and GBLS design has defined the most important steps of that process. For instance, authors in (Marne et al 2012) had defined design patterns inspired from existing solutions to facilitate communication between actors participating in that process. Research work in (Sanchez and Ney

699

Kaouther Raies, Maha Khemaja and Rafik Braham 2011) had focused on integrating funny qualities and pedagogical aspects in the same application and described fundamental steps to follow for producing an efficient GBLS. Authors in (Marfisi‐Schottman et al 2010) had provided a conceptual model describing major steps and tools to use by actors to create a GBLS where production team must collect software components that could be used. Those components are described with an extension of LOM (Learning Object Metadata 2002). For this end a system was defined to search these descriptions. Same authors present in (Marfisi‐Schottman et al 2010) a multi‐actors Web oriented environment for designing GBLS. This environment aims to help actors participating in this process by proposing ergonomic and intuitive interfaces based on drag & drop mechanism. The aforementioned related works mostly focused on the combination of both fun and educational aspects. They present GBLS design as a process by introducing steps to follow, using their own semantic for sub‐process orchestration without much attention to problems inherent to the diversity of tools used by actors and thus the problem of interoperability is still open. Moreover, actors continue to exchange data and artifacts in non‐ standard formats. Other works related to GBLS design as the e‐adventure framework (Moreno‐Ger, Martínez‐Ortiz and Sierra 2008) aimed to facilitate the integration of games with IMS LD (Instructional Management System Learning Design). E‐adventure is an authoring environment for adventure games creation. The author can design virtual characters, conversations, player’s avatar (as well as its movement trajectory) and scenes; but these tools are specific to adventure games that are usually built around a fantastic world. Although, other works propose authoring tools that can facilitate the development process of serious games. We can distinguish between two big families. The first one presents tools that aim to develop different kinds of games in (MARNE et al 2010), (e.g) Construct, Multimedia Fusion Creator and Game‐Editor. They offer a programming environment based on specific event programming paradigm and object libraries that may facilitate the development process. The Second one includes authoring tools for specific video games like GBL; they are founded on Domain Specific Language. Example of those tools are IBIS (Intuitive Builder for Intelligent Systems), (Blanchard and Frasson 2006) or SeGAE (Serious Game Authoring Environment) (Yessad, Labat and Kermorvant 2010). In the table 1 we summarize the main characteristics of works cited above, taking into account GBL designer needs. Table 1: Main characteristics of authoring tools used in GBL design process Criteria Tools Construct Multimedia Fusion Creator Game editor IBIS SeGAE E‐adventure

Standard data format ‐ ‐ ‐ + + +

Interoperability Gameplay IMSLD

‐ ‐

+ +

‐ ‐ ‐ ‐ ‐ ‐ ‐ + Criteria supported + Criteria not supported

Collaborative work

Assistance

‐ ‐

‐ ‐ ‐ +

‐ ‐ ‐ ‐

To conclude, we didn’t find any research that proposes responses to the aforementioned limits. Moreover, the proposed solutions have not integrated assistance in the process of designing a GBLS. For these reasons we propose a solution based on Semantic Service Oriented Architecture that aims to present a relevant environment integrating assistance for actors who don’t have the relevant skills to achieve their tasks.

700

Kaouther Raies, Maha Khemaja and Rafik Braham

4. Proposed approach We already mentioned that GBL design is based on two fundamental processes, the first one is related to game design process, and the second one is related to learning design process. Every process has its own specificity in terms of modeling, execution languages, tools and participating actors. That is why we propose to integrate them in a global and coherent system. Moreover, we are convinced that assisting or supporting designer could be done with additional process called assistance process. This latter should be combined and integrated to both processes intelligently and automatically at run time.

4.1 Fundamental stages of a GBLS design process GBLS design process is a long process involving several actors and using methods inspired from both video games and educational systems development. Figure 2 illustrates fundamental steps used by actors to produce a GBLS.

Figure 2: Fundamental step of design GBLS

Gameplay definition Domain specific knowldge extarction

learning objectives and learning scenarios definition

NOT OK

Artistic Team

We should stress that we have adopted this process from that proposed in (Marfisi‐Schottman and George 2010) and that all those steps represent on their own complex business processes, using different artifacts and producing different other outcomes with different formats. Moreover, actors participating in this process should select relevant software components that may allow them to do their tasks. To model processes related to gaming scenarios and their supporting scenarios, GBLS development companies may adopt BPMN (Business Process Modeling Notation) for two reasons: firstly, regardless of the business domain, almost business companies use it for designing their business processes. Secondly it is more straightforward for SOA paradigm. The BPMN model of the design process is presented in figure 3.

scenario edition

characters, graphs and environment descriptions

Create animated sequence

music and sound composition

Figure 3: BPMN model of design process However, for the Learning design process which belongs more specifically to educational professionals, we can note that during the two last decades, many e‐learning specifications were defined such as Dublin Core MetaData (Dublin Core Metadata 2012), LOM (Learning Object Metadata 2002), SCORM(Changtao and Wolfgang 2001) and IMS Learning Design (IMS LD) (Britain 2004). For instance, IMS LD constitutes the relevant solution to make digital information encoding learning designs consistent and thus both transportable and re‐ usable in different software packages such as GBL development domain.

701

Kaouther Raies, Maha Khemaja and Rafik Braham Unifying and bridging the gap between BPMN and IMS LD towards a unified process and a coherent system will be described in the following sections.

4.2 A service oriented architecture for an appropriate environment With an increasing number of new components used to make Game design more efficient, a relevant solution that facilitates component use and reuse is beneficial. This can be achieved by introducing a solution based on SOA paradigm. SOA, (Bell 2008) has been famous for several years as its loosely coupled nature allows integration of legacy and existing systems in a granular way that can easily accommodate changing needs. SOA describes an information infrastructure which allows different applications to exchange data with one another as they participate in collaborative working processes. Therefore, we are convinced that SOA will have a significant impact when applied to GBL design. Indeed, it could grant a greater flexibility to changes without investing considerable time and effort in modifying underlying IT infrastructures and software. Moreover, SOA enables enterprises participating in that process to not only transform internal systems to be more service oriented, but also permits best collaboration amongst them. It also grants more agile business processes because it reduces the gap between business process model and implementation. This enables changes to business processes already implemented as orchestrations of services to be easily captured and implemented. A general high‐level overview of architecture for GBL design platform is depicted in Figure 4.

Figure 4: GBL design platform architecture The architecture of the run time platform follows a multi‐layer approach, consisting of four main layers described as follows.

BPEL4WS: Business Process Execution Language for Web Services is an executable language for specifying interactions with Web Services for the composition and implementation of above mentioned process.

Services: combine and link existing services, either atomic or composite services, creating new services.

Components: existing software components that may be used by Game designers and which expose their throw interfaces as Web Services.

702

Kaouther Raies, Maha Khemaja and Rafik Braham

Legacy applications or resources: Existing applications that can be reused by Game Designers.

Even if SOA can satisfy some GBLS design process requirements, many other aspects cannot be solved as: Diversity of SOA paradigms and languages used to model business process in one hand and its difference with Learning design model and assistance process in other hand. For instance, each framework proposes a specific vocabulary and has its own assembly semantics. These challenges make it difficult for none initiated to work with models whose languages are different as BPMN and IMSLD. Besides, although the re‐use of legacy functions via Web services, existing process modeling notations are not flexible enough: only services or sub‐processes with known syntactic interfaces can be composed, while the semantics of the tasks in a process model and available functions is disregarded (Jędrzejczak and Filipowska 2008). That’s why, adding a semantic layer becomes a necessity. In the following section, we present interesting aspects of that additional layer and their benefits to overcoming some technical and pragmatic requirements of GBL designing process.

4.3 Semantic business process of a GBL designing process Semantic Service Oriented Architecture grants a greater flexibility for the overall process of designing a GBLS through integration of heterogeneous systems. It also allows a best coordination activity from different functions around a common goal. This solution provides a solution of interoperability and standardization problems. Furthermore, it grants an increased collaboration within and across enterprises (actors belonging to different enterprises). Moreover it overcomes several problems at run‐time as well as at design‐time due to the use of different business process modeling languages by the collaborating companies as for example BPMN and IMSLD. More specifically we have adopted a subset of SUPER’s project (Integrated Project SUPER 2010) ontology stack. We have used namely, the BPMO (Business Process Modeling Ontology) which provides a semantic layer that bridges the gap between different formalisms and guaranties automatic discovery, selection, execution, and composition of business processes or Web services. We have also adopted additional organizational ontologies. Those ontologies constitute firstly a part of the semantic process representation for the needs of the Semantic Business Process Management (SBPM) and aim to provide all needed vocabulary and constraints for describing the environment in which processes are carried out from the organizations’ perspective. Secondly they allow describing organizations and resources, define common types of divisions, roles and tasks, and define common types of business resources, as business function ontology, business strategy ontology, business domain ontology and business role ontology. We should stress however, that our organizational ontologies are much more specific and are GBL design domain dependent. These ontologies and the relationships between them are depicted in figure 5.

Figure 5: Ontologies and relations between them

703

Kaouther Raies, Maha Khemaja and Rafik Braham For instance, in Figure 6 we illustrate a Semantic Business Process related to designing the pedagogical aspects of a GBL. In Figure 7, we present the domain ontology related to GBLS design. The communicating processes are most probably using their own domain ontology to define concepts and relations representing the exchanged messages. They thus cover the relative taxonomy of that domain as pedagogical aspect, fun aspects and game play.

Figure 6 BPMO of pedagogical aspect of GBL

Figure 7: Domain ontology of GBL design

5. Conclusion and future work The principal aim of the work presented in this paper is to define GBLS design processes. The idea is to lighten and reduce actors’ workload, giving them the appropriate environment to produce more attractive and efficient GBLS. This responds to GBLS users’ needs and solves many problems like flexibility, modularity, and reusability.

704

Kaouther Raies, Maha Khemaja and Rafik Braham While, there are several attempts that aim to respond to technical and pragmatic actors’ requirements, wide variety of problems are undoubtedly not solved. For these reasons, the idea of adopting SSOA to design GBLS is proposed, and a general high‐level overview of the system’s architecture is described. Our future work will consist on, adapting GBL design processes; accordingly to profiles of people with special needs, therefore there are several considerations that must be taken into account when designing accessibility for a video game and especially for GBLS, some additional peculiarities must be considered. We will equally focus on development of the assistance process, which is based on actor’s context, skills and roles analysis.

References Barry, D. (2003) Web Services and Service‐Oriented Architecture: The Savvy Manager’s Guide, Morgan Kaufmann Pub, 2003. Bell, M. (2008) Introduction to Service‐Oriented Modeling: Service Analysis, Design, and Architecture. Wiley & Sons. pp. 3. Blanchard, E. and Frasson,C. (2006) ‘ Faciliter la création d'environnements virtuels d'apprentissage s'inspirant des jeux vidéo ‘, TICE'2006, Toulouse, France. Britain,S (2004)’ A Review of Learning Design: Concept, Specifications and Tools’, A report for the JISC E‐learning Pedagogy Programme. Changtao, Q. Wolfgang, N (2001)’ Towards Interoperability and Reusability of Learning Resource: a SCORM‐conformant Courseware for Computer Science Education’, Institute of Computer Engineering University of Hannover Corti, K. (2006)’Games based Learning; A serious business application’, PIXELearning Limited. Integrated Project SUPER. (2010) Semantics Utilized for Process management within and between Enterprises (Project IST 026850 SUPER), [on line] http://www.ip‐super.org/content/view/129/136/ Dublin Core Metadata (2012) [Online] http://dublincore.org/documents/dces/ Jędrzejczak, A. and Filipowska,A. (2008) ‘sBPMN and sEPC to BPMO Translation’, Semantics Utilized for Process management within and between Enterprises (Project IST 026850 SUPER), Delivrable 4.5, 2008. Learning Object Metadata (2002), ‘Draft Standard for Learning Object Metadata’, Learning Technology Standards Committee of the IEEE, New York, NY 10016‐5997, USA. Marfisi‐Schottman,I. Sghaiers, A. and George,S .(2009) ‘Towards industrialized conception and production of serious games’,ICTE Internaltional Conference on Technology and Education, Paris, France, PP. 1016 ‐ 1020 . Marfisi‐Schottman,I. Sghaiers,A. and Tarpin‐Bernard, F .(2010) ‘Tools and Method for Effeciently Designing Serious Games’, 4th Europeen Conference on Games Based Learning ECGBL2010, Copenhagen, Denmark, pp. 226‐234. Marfisi‐Schottman,I. and George,S. (2010) ‘Environnement Web multi acteur pour la conception de serious Games ‘ Actes de la Journée scientifique "Conception des EIAH à l'ère du Web 2.0 et à l'aube du 3.0", Amiens. Marne, B. John, W. Kim‐Bang, B.and Labat, J. (2012) ‘The Six Facets of Serious Game Design: A Methodology Enhanced by Our Design Pattern Library’, Proceedings of 7th European Conference on Technology Enhanced Learning, Saarbrücken, Germany, pp. 208‐221, (Springer‐Verlag) (2012) Marne, B (2010) « Évaluation des outils auteurs de jeux sérieux, sans programmation » [online] http://wwwia.lip6.fr/~marneb/Rapport_M2_IFL_B‐Marne_2010.pdf Moreno‐Ger,P.Martínez‐Ortiz,I and Sierra,J (2008) , ‘A Content‐Centric Development Process Model’, Journal of comupter volume 41 Issue 3, IEEE Computer Society Press Los Alamitos, CA, USA, March 2008. Object Metadata (2002) Standard for Learning Object Metadata, IEEE 1484.12.1. Sanchez, E.Ney, M and Labat,j.(2011) ‘ Articuler motivation et apprentissage grâce aux facettes du jeu sérieux’, Revue internationale des technologies en pédagogie universitaire / International Journal of Technologies in Higher Education, vol. 8, n° 1‐2, 2011, p. 48‐57. Yessad,A . Labat,J.and Kermorvant, F. (2010) ‘SeGAE: A Serious Game Authoring Environment’, IEEE International Conference on Advance Learning Technologies, Sousse, Tunisia, pp. 538‐540, (ISBN: 978‐1‐4244‐7144‐7) (2010).

705

Using Games for Learning, From the Students’ Perspectives Aishah Abdul Razak and Thomas Connolly School of Computing, University of the West of Scotland, Paisley, UK aisya.razak@uws.ac.uk thomas.connolly@uws.ac.uk Abstract: The hours of exposure to the digital world is claimed to be profoundly affecting and changing this generation’s learning styles and abilities. To make learning more appealing to the students, teachers have started to use digital games within classroom learning. This paper presents the findings from structured interviews conducted with two classes of primary school students from two different schools in Scotland. The interviews were intended to obtain the students’ background on game playing and their views on learning particularly in using games for learning. The participants were exposed to digital games‐based learning (DGBL) and non‐DGBL learning approaches at school prior to the interviews. The findings revealed that most students spent on average less than 1 hour per day playing digital games and the most popular game platform was the console platform. The overall mean for the attitudes towards learning by playing games, learning by making game, towards DGBL and non‐DGBL approaches and towards school is above 2.00 (Mean: 2.50, SD: 0.10), indicating positive attitudes in all observed areas. No gender difference was observed in all of the attitudes. However, some significant differences were found across some items for different age groups. The findings from this study will contribute to the generation of empirical evidence in the field of digital games‐based learning particularly with regards to its application in primary school education. Keywords: student view, digital games‐based learning, Curriculum for Excellence CfE, empirical evidence, primary school

1. Introduction The concept of digital natives was introduced by Prensky (2001) to define the younger generation of children who have always been surrounded by, and interacting with, digital technologies such as mobile phones, MP3 players, computers, the Internet and video gaming, all of which became pervasive at the end of the twentieth century. It has been claimed that hours of exposure to the digital world has a great influence in the lives of younger generations, affecting and changing this generation’s learning styles and abilities. Among the cognitive styles of this generation that have changed from the previous generation according to Prensky are: the digital natives can process information very fast, and can parallel process and multi‐task, prefer random access rather than step‐by‐step linear access, have sharp visual sensitivity compared to text, enjoy networking, active roles, fantasy and using technologies. Some researchers argue that these changes have profound implications for education (Underwood, 2007; Prensky, 2001; Gibbons, 2007), and in some cases digital natives have mistakenly been labelled as having some form of learning disability (Simpson, 2005). There is a growing body of research suggesting that to teach or even engage the youth of today, the teaching methodology and content needs to be changed to be more game‐like, as opposed to traditional classroom instruction (Gee, 2005; Kirriemuir and Mcfarlane, 2004; Van Eck, 2006; Visser, 2001). In Scotland, the Scottish Government has made radical changes in its education policy by introducing a new curriculum known as Curriculum for Excellence (CfE). Unlike the previous curriculum, CfE documents consist of guidelines known as the Experience and Outcomes, presented in such a loose way that the teacher can decide how to interpret it, to be creative and take forward children’s interest. Consequently, moving away from didactic teaching to more active learning and encourage child‐centred learning where children have more influence over the choice of learning, ownership over the direction of the study and personalising the learning to their own interest. One of the teaching and learning approaches that are promoted under the CfE is the use of digital games‐based learning (DGBL) to promote challenge and enjoyment in children's learning. The next section will present some findings from previous work on DGBL within the CfE and some review of literature on the students’ view on the use of games or technologies in the UK. This is followed by detail description of the study presented in this paper, and finally the result and discussion.

2. Previous research A series of surveys have been conducted across a number of regions in Scotland to gauge the views and implementation of DGBL in primary schools from the teachers’ perspective. Although the overall findings indicated that the use of this approach is still limited, it was discovered that the main motivation for the teachers to use this approach was that students enjoy learning using computer games (Razak, Connolly and

706

Aishah Abdul Razak and Thomas Connolly Hainey, 2012; Razak, Connolly, Baxter, Hainey and Wilson, 2012). Based on this finding, it is important to gain a better understanding of the student’s usage of technologies in everyday life and their own view and expectation on the uses of games for learning. This information has great implications on policies and practices for the development of technology rich environments that are equitable, engaging and support quality outcomes (Gosper, Malfroy, Mckenzie and Rankine, 2011). Gibbons (2007) emphasized that understanding students and their academic and social practices is necessary if students are to be placed at the centre of decisions about services and facilities. Reviews of the literature on the students’ view on the use of games or technologies in the UK are limited and mostly involved teenagers or adults. One example of such studies is by (Helsper and Eynon, 2009) that provided evidence on how the British population, aged 14 and above, access and use the Internet and other new technologies from a nationally representative face‐to face survey (the Oxford Internet Surveys). Another study was done by Ulicsak, Wright and Cranmer (2009), which reports literature reviews on access to technology, availability of suitable games and how game playing is supported in family settings. BBC (2005) reported on a survey conducted by TNS in summer 2005 on a sample of UK residents aged between 6 and 65 years old, which aimed to find out the size of the gaming market in the UK and to profile “gamers”. An older study was performed by McFarlane, Sparrowhawk and Heald (2002) that collected and analysed over 700 completed questionnaires from various primary and secondary schools in England involving KS2, 3 and 4 students on game playing habits, use of the Internet and computer games at school.

3. This study This paper presents the findings from student interviews conducted in two Scottish primary schools in 2012. The interviews were intended to get the students’ background on game playing and their views on learning particularly in using games for learning. Structured interviews were chosen as a suitable method for data collection, where a list of predetermined questions, mostly closed ended, with some simple open ended questions was used for the interviews. Using structured interviews, the researcher is able to explain to the student if he or she does not understand the question or feeling confused. This is very important considering the student is primary school age. As time with the respondent is also limited because the interviews were conducted during school hours, this method allowed the data to be collected quickly and easily and at the same time can be used as a form of formative assessment to select students for a more detailed interview later.

3.1 Participants The selection of participants for these interviews was based on homogeneous sampling, where all the selected participants had been exposed to DGBL and non‐DGBL learning approaches for the whole school term prior to the interviews. This was to ensure that the participants clearly understood and had recently experienced learning using games and without games to be able to answer the questions. Table 1 gives the details of the student respondents for these interviews. Table 1: The detail of student respondents of the structured interviews

School A (Primary 5)

School B (Primary 4)

Total

Age 8‐9 7‐8 7‐9 Total in class 32 28 60 Actual participation 25(13M, 12F) 28(13M, 15F) 53 (26M, 27F)

3.2 Materials The interview questions were divided into 5 parts:

Part 1: General attitude towards the use of games

Part 2: Attitude towards learning by playing game

Part 3: Attitude towards learning by making game

Part 4: Attitude towards learning using DGBL and non‐DGBL approaches

Part 5: Attitude towards school

707

Aishah Abdul Razak and Thomas Connolly Part 1 consists of open‐ended questions with sample answers to help the students answer the questions. Part 2‐Part 5, students were asked to state their agreement on attitude towards playing and making games, DGBL and non‐DGBL approaches and school using a 3‐point Likert scale, ‘disagree’ (1), ‘don’t know’ (2) and ‘agree’ (3). Initially, a 5‐point scale was used but the students were found to be confused and found it difficult to differentiate between ‘strongly’ and ‘somewhat’, which might have been due to the participants’ age. Thus, it was decided to use a 3‐point scale instead. The items for Parts 2 and 3 were devised based on the Intrinsic Motivation Inventory (IMI) by Ryan & Deci, (2000) which is intended to assess the participants’ experience related to a target activity. Five subscales were chosen which were interest/enjoyment, perceived competence, effort, choice and value/usefulness while performing a given activity. In addition, some other items which were related to the study were also included such as the participants’ knowledge on game playing and game making activities and its influence. Part 4 tries to capture the students’ view on learning using DGBL and non‐DGBL while Part 5 investigates the children’s’ attitudes toward school, modified to use a 3‐point scale from a previously published instrument (Miller and Robertson, 2011).

3.3 Procedures The interviews were conducted at the end of the last school term of session 2011/2012. Prior to the interviews, the students from School A had learned problem solving using both DGBL and non‐DGBL approaches while students from School B had learned times tables using both DGBL and non‐DGBL approaches. The researcher was introduced to the students from the beginning of the school term thus they had met at least on 6 sessions prior to the interviews. During the sessions, the researcher played the role of an observer of the class but at some points had chances to ask the students some informal questions. It was important to establish some rapport with the students prior to the interviews. Initially, the students were interviewed individually. However, after some attempts it seemed that some students were still shy and less confident and it was difficult to keep them motivated until the end of the interview, which took about 15 minutes. Thus, it was decided to conduct the interviews in groups of 3‐4 students. The interviews lasted for about 20‐30 minutes for each group and the students’ responses seemed to improve as they felt more confident and eager to comment on each other’s answers. The drawback of this group interview was that sometimes the discussion got off topic and the researcher had to put them back on track. The data were coded during the interviews and analysed using SPSS.

4. Findings 4.1 General attitude towards the use of games Table 2 shows the response from the students on how they would normally spend their free time at home. The findings show that 50.9% of the students would spend their free time playing outdoors such as playing football, cycling, going to the playground when the weather permit. The least popular activities for the children to spend their free time were sleeping, watching television and chatting with friends on the phone (3.8% each). Table 2: Spending free time

Frequency

Percent

Outdoor game

50.9

Reading

15.1

Homework/study

11.3

Play computer/console game

11.3

Sleeping

3.8

Watching TV

3.8

Chatting

3.8

Total

100.0

Table 3 shows the average time the students spent on playing games daily. The majority of the students spent less than 1 hour on playing games daily (52.8%), followed by 26.4% who spent more than 3 hours and 20.8%

708

Aishah Abdul Razak and Thomas Connolly who spent between 1‐3 hours. This finding is slightly higher than the study conducted by McFarlane, Sparrowhawk and Heald (2002) which found that 46.9% primary school students played up to 1 hour at a time. Table 3: Average time spent on playing games daily

Frequency

Percent

less than 1 hour

52.8

more than 3 hours

26.4

between 1‐3 hours

20.8

Total

100.0

The results in Table 4 shows that the majority of the students were more likely to play games on a games console (60.4%) and the least group played games online (5.7%). This finding is similar to the findings of McFarlane et al. (2002) and Ulicsak et al (2009). The study in the BBC report (Bbc, 2005) showed a slightly different result where PC games were the most popular, followed closely by consoles. Table 4: Game platform

Frequency

Percent

console games

60.4

mobile phone games

22.6

PC games

11.3

online games

5.7

Total

100.0

Table 5 shows the reasons for playing computer games. The highest reason was for social activity where the students enjoyed playing games with their friends and family. This finding is in agreement with the one in the BBC report which found that players aged 6‐10 are the most sociable players of all the age groups interviewed, with 54% agreeing that they preferred playing games with friends as a social activity rather than on their own. The least reason for playing computer games were for fantasy and other reasons such as addicted to game playing. A Mann and Whitney U test showed no significant difference in gender and age on how the students spend their free time, average time spent on playing games, game platform and reason for playing game. Table 5: Reason for playing game

Frequency

Percent

social activity

32.1

learning

24.5

competition

20.8

relaxation

18.9

fantasy

1.9

other

1.9

Total

100.0

4.2 Attitudes towards learning by playing games Table 6 shows the students’ attitudes towards learning by playing games. It is surprising to see that all 53 students agreed that playing games is fun and enjoyable (Mean: 3.00, SD: 0.00). Overall the results show that the majority of the students have positive attitudes towards playing games and its potential for learning. The students put effort into mastering and gaining scores in the games and felt competence in playing the games. They believed that they can learn from games and were aware of the websites for educational games. They thought that they could remember things better when playing a game compared to reading it and most of the time they played a game because they wanted to. The lowest scored item ’I don’t like a Game that requires me to answer question’ (Mean: 2.25, SD: 0.92), which is mostly seen in the drill and practice type educational game. Similarly the BBC report found that 76% of the respondents within 6‐10 years of age agreed that games could be used for education as well as entertainment. A Mann and Whitney U test shows no significant difference in gender and age for all items in Table 6 except for ‘Fighting games do not makes me want to fight.’

709

Aishah Abdul Razak and Thomas Connolly where Primary 5 scored significantly higher than Primary 4 ( p<0.001, Z=3.629). This might suggest that older children were less influenced by the violence in the game compared to younger children. Table 6: Attitude towards learning by playing game

Sub Scale

Mean

Std. Error

Playing computer game is fun and I enjoyed doing it very much.

interest

3.00

0.00

When I was playing a game, I tried my best to do well in the game.

effort

2.83

0.06

0.43

I think I am pretty good at playing a game.

competent

2.83

0.06

0.47

I enjoyed playing game with a friend compared to playing alone.

collaboration

2.66

0.10

0.76

I think playing a game could help me to learn.

usefulness

2.62

0.09

0.63

I know lots of website to play educational games.

knowledge

2.57

0.11

0.82

I remember things better when I play with it compare to I read it.

other

2.40

0.11

0.82

I don’t think fighting game will make me want to fight.

other

2.40

0.12

0.91

I always wanted to play game.

choice

2.36

0.12

0.88

I don’t like a game that requires me to answer questions.

other

2.25

0.13

0.92

4.3 Attitude towards learning by making game Table 7 shows the students’ attitudes on the game making activity. The highest scored item was ‘I can create better game if I do it with a friend.’ (Mean: 2.96, SD: 0.27). The lowest scored item was ‘I know many available games making tool.’ (Mean: 1.58, SD: 0.84), followed by ‘I think I am pretty good at making a game.’ (Mean: 1.92, SD: 0.83). For other items, the majority of the students agreed that making games is fun and enjoyable, they wanted to create their own game and put anything that they like in it such as making their own characters, they will share their game with others and thought that they can learn through the game making activity. A Mann and Whitney U test shows no significant difference in gender and age for all items in Table 7. Table 7: Attitude towards learning by making game

Sub Scale Mean

Std. Error

I can create a better game if I do it with a friend.

collaboration

2.96

0.04

0.27

Making my own game is fun and I enjoyed doing it very much.

interest

2.85

0.07

0.50

I want to put anything I like in my game.

other

2.83

0.07

0.55

I always wanted to create my own game.

choice

2.75

0.09

0.65

When I was making a game, I tried my best to create a good game.

effort

2.70

0.08

0.57

I like to share the game that I made with others.

other

2.68

0.10

0.70

I like to make my own character when I play/make a game.

other

2.55

0.11

0.82

I think making my own game could help me to learn.

value

2.49

0.10

0.72

I think I am pretty good at making a game.

competence

1.92

0.11

0.83

I know many available games making tool.

knowledge

1.58

0.12

0.84

4.4 Attitude towards learning using DGBL and non‐DGBL approach Table 8 shows the students’ attitudes towards learning using DGBL (involving game) and non‐DGBL (direct teaching by the teacher). The highest scored item was ‘Learning is really fun when I got to use the computer.’ (Mean: 2.75, SD: 0.65) while the lowest scored item was ‘I think I learn better when playing the computer game compare to when my teacher teaches me.’ (Mean: 2.30, SD: 0.85). Overall the findings have shown that the preference for DGBL approach is slightly higher than the non‐DGBL approach. However, more students agreed that they can easily practise what they learn from their teacher at home (Mean: 2.70, SD: 0.57) because it usually involved writing or reading exercises compared to playing/ making games (Mean: 2.62, SD:

710

Aishah Abdul Razak and Thomas Connolly 0.69), which required certain equipment such as the game itself. A Mann and Whitney U test shows no significant difference in gender and age for all items in Table 8 Table 8: Attitude towards learning using DGBL and non‐DGBL approaches

Mean

Std. Error

Learning is really fun when I got to use the computer.

2.75

0.09

0.65

I can practise what I learnt from my teacher at home.

2.70

0.08

0.57

I always enjoyed learning using computer game.

2.68

0.10

0.70

I can continue to play the game I used /created at school when I go home.

2.62

0.09

0.69

I always enjoyed learning with my teacher.

2.38

0.10

0.74

I think I learn better when playing the computer game compare to when my teacher teaches me.

2.30

0.12

0.85

4.5 Attitude towards school Table 9 shows the result on the students’ attitudes towards school. The top two scores were on ‘Learning at school is necessary for future life.’ (Mean: 2.92, SD: 0.33) followed by ‘What children learn at school is important.’ (Mean: 2.60, SD: 0.77). This result shows that majority of the students acknowledged the importance of learning at school for their future life. The lowest score was on ‘The best days of the week are those at the weekdays’ (Mean: 1.64, SD: 0.92), indicating that the students enjoyed the days not spent at school better. A Mann and Whitney U test shows no significant difference in gender and age for all items in Table 9 except for ‘School usually makes children excited.’ where Primary 4 scored significantly higher than Primary 5 ( p<0.001, Z=4.061). This might suggest that older children were less excited at school compared to younger children. Table 9: Attitude towards school

Mean

Std. Error

Learning in school is necessary for future life.

2.92

0.05

0.33

What children learn at school is worthless (**important).

2.60

0.11

0.77

School usually makes children bored (**excited).

2.26

0.12

0.88

Going to school every morning makes me happy

2.17

0.11

0.78

The best days of the week are those at the weekends (**weekdays).

1.64

0.13

0.92

**Reversed

5. Discussion Overall, the students’ general background towards the use of digital games is considered moderate. Many students would spend their free time playing outdoors or reading before choosing digital games or television. Most students also spent on average less than 1 hour per day playing digital games and the main reason for that was for having social activity with family and friends. The most popular game platform was the games console. The reason for this may be because games consoles like the Xbox and Nintendo Wii include more movement and interaction compared to PC games where the students would only be sitting in front of the computer during the game play. Console games also support multiplayer mode, which is more suitable for social activity with family and friends. Studies on the use of console games at school have also shown that using console games for learning within classroom settings has received a good response from the students and are able to motivate and engage them to learn (Groff, Howells and Cranmer, 2010; Razak and Connolly, 2013).The online game was least popular for this age group of student perhaps because of limited access to the Internet at home. Table 10 shows the overall mean for the attitudes towards learning by playing games, learning by making games, towards DGBL and non‐DGBL approaches and towards school. Overall the average score is above 2.00

711

Aishah Abdul Razak and Thomas Connolly (Mean: 2.50, SD: 0.10) which indicates that, in general, the majority of the students have positive attitudes in all observed areas. The highest score was in the attitudes towards learning by playing games and the lowest was in the attitudes towards school. Table 10 Descriptive statistics scores on overall mean

Mean

Std. Error

Attitude towards learning by playing games

2.59

0.09

0.66

Attitude towards learning by making games

2.53

0.09

0.65

Attitude towards learning using DGBL and non‐DGBL approaches

2.57

0.10

0.70

Attitude towards school

2.32

0.10

0.74

Total Average

2.50

0.10

0.69

Another important finding was that there was no gender difference in all the attitudes. However, some significant differences were found across some items for different age of student such as Primary 4 children thought that school was more exciting compared to Primary 5 children and Primary 5 children thought that violence in the game was not affecting their behaviour as much as Primary 4 children. Finally, this study also found that the students were more exposed to game playing compares to game making. Although the students agreed that both approaches were fun and enjoyable, they do not see themselves as competent in making game. They were not familiar with many game making tools as much as they were familiar with the educational games website/resources. One limitation that should be noted in this study was that only the Primary 4 student were exposed to both game playing and game making approaches during the school term prior to the interviews, while the Primary 5 students have used only game playing approach during that term. However, the Primary 5 students informed that they have experience in game making before and have responded to the interview questions based on that experience.

6. Conclusions This paper presented some findings on the current game playing habits of students from two primary schools in Scotland. The aim is to understand the children’s use of technology particularly computer games and their views on learning with games in order to reflect on the applicability of a DGBL approach which is currently being promoted under the CfE. Overall the findings revealed that the children use of game at home is moderate with an average of less than 1 hour daily for playing digital games and most popular game platform was the console platform. Familiarity with console games explains why console games was very popular and received good response from the students when introduced at school under the DGBL approach. The overall mean for the attitudes towards learning by playing game, learning by making games, towards DGBL and non‐ DGBL approaches and towards school is above 2.00 (Mean: 2.50, SD: 0.10), indicating positive attitudes in all observed areas. No gender differences were observed in any of the attitudes. However, some significant differences were found across some items for different age groups. The findings give some important insight on the students’ trend on using games in their social life and their view on games for learning which is necessary for the development and implementation of DGBL approach within the school setting. In addition, the findings will also cont ribute to the generation of empirical evidence in the field of DGBL research.

Acknowledgements This paper is a scientific publication as part of the Games and Learning Alliance (GaLA). GaLA is a Network of Excellence on ‘serious games’ funded by the European Union in FP7 – IST ICT, Technology Enhanced Learning (see http://www.galanoe.eu). GaLA gathers the cutting‐edge European research and development organizations on ‘serious games’, involving 31 partners from 14 countries. The University of the West of Scotland is one of the partners.

References Bbc 2005. Gamers in the UK: Digital play, digital lifestyles. Gee, J. P. 2005. What Video games Have to teach Us about Learning and Literacy. ACM Computers in Entertainment, 1. Gibbons, S. 2007. Redefining the role of information professionals in higher education to engage the net generation. EDUCAUSE. Australia.

712

Aishah Abdul Razak and Thomas Connolly Gosper, M., Malfroy, J., Mckenzie, J. & Rankine, J. Year. Students‘ engagement with technologies: Implications for university practice. In: G.WILLIAMS, STATHAM, P., BROWN, N. & CLELAND, B., eds. ASCILITE 2011 : Changing demands, changing directions, 2011 Hobart, Australia. ASCILITE, 504‐508. Groff, J., Howells, C. & Cranmer, S. 2010. The impact of console games in the classroom:Evidence from schools in Scotland. Learning and Teaching Scotland Helsper, E. & Eynon, R. 2009. Digital natives: where is the evidence? British educational research journal, 1‐18. Kirriemuir, J. & Mcfarlane, A. 2004. Literature Review in Games and Learning. Nesta Futurelab series. Bristol. Mcfarlane, A., Sparrowhawk, A. & Heald, Y. 2002. Report on the educational use of games. Teachers Evaluating Educational Multimedia. Cambridge. Miller, D. J. & Robertson, D. P. 2011. Educational benefits of using game consoles in a primary classroom: A randomised controlled trial. British Journal of Educational Technology, 42, 850‐864. Prensky, M. 2001. The Games Generations: How Learners Have Changed Digital Game‐Based Learning. McGraw‐Hill. Razak, A. & Connolly, T. 2013. Using games as a context for interdisciplinary learning: A case study at a Scottish primary school. IEEE Educon 2013. Berlin: IEEE. Razak, A., Connolly, T. & Hainey, T. 2012. Teachers’ Views on the Approach of Digital Games‐Based Learning within the Curriculum for Excellence. International Journal of Game‐Based Learning (IJGBL), 2. Razak, A. A., Connolly, T. M., Baxter, G. J., Hainey, T. & Wilson, A. 2012. The Use of Games‐based Learning at Primary Education Level within the Curriculum for Excellence: A Combined Result of Two Regional Teacher Surveys. 6 th European Conference on Games‐based Learning (ECGBL). Cork, Ireland. Ryan, R. M. & Deci, E. L. 2000. Self‐determination theory and the facilitation of intrinsic motivation, social development, and well being. American Psychologist, 55, 68‐78. Simpson, E. 2005. Evolution in the Classroom: What Teachers Need to Know about the Video Game Generation. TechTrends: Linking Research & Practice to Improve Learning, 49, 17‐22. Ulicsak, M., Wright, M. & Cranmer, S. 2009. Gaming in families. Future Lab. Unde rwood, J. 2007. Rethinking the Digital Divide: impacts on student tutor relationships. . European Journal of Education, 42, 213‐222. Van E ck, R. 2006. Digital Game‐Based Learning: It's Not Just the Digital Natives Who Are Restless. EDUCAUSE Review, 41, 16‐18. Visser, J. 2001. Integrity, Completeness And Comprehensiveness Of The Learning Environment: Meeting The Basic Learning Needs Of All Throughout Liferethinking In: ASPIN, D. N. (ed.) International handbook of lifelong learning, Volume 2. Kluwer Publishing.

713

Incidental Learning in a World of Warcraft Guild, a Case Study Gabriela Rodríguez University of Turku, Turku, Finland gabrod@utu.fi Abstract: Digital Games inherently include scaffolding, problem‐based learning, and communities of practice, and they are of relevance because they are interactive, have situated meanings, and offer simulations in which to try new ideas in a risk‐ free environment. Many endeavors have been made to describe and discover different ways that Digital Games and education work or could someday work together. The three approaches usually explored are within the context of Serious Games, Game‐based Learning and Incidental Learning. The following investigation undertakes the Incidental Learning approach to find out whether the cognitive learning outcomes mentioned in the literature do indeed result from playing in a World of Warcraft type of player community called a guild, and if so, to describe the ways these outcomes are yielded. It will also address the question of how players perceive themselves as learners within the game. A guild was chosen as a case study and observed for a three month period. Interactions were recorded in the format of chat logs and transcriptions of audio recordings, summing up to 170 hours of material. Fifty‐one members of the guild (M=44, F=7) then answered an online questionnaire. Literature on the topic was reviewed and a list was compiled in regards to what are some of the cognitive outcomes that may result from playing Digital Games. These previous claims on learning outcomes were classified into four main categories: Resource Management, Interpersonal Skills, Information Handling and Technology, and Systemic Thinking. These learning outcomes were seen to occur throughout the observation, though the degree in which they were exemplified varied depending on whether players had a position of leadership or not, and other factors such as occupation or whether the player engaged in raids or not. Results indicate that players with positions of leadership are more prone to believe they are learning than regular players are, as are older players when compared to younger players. Keywords: digital games, incidental learning, guild, World of Warcraft, cognitive learning outcomes

1. Introduction The three approaches to describe and discover the different ways Digital Games and education work together are that of Serious Games, Game‐based Learning and Incidental Learning. “Serious Games” are games designed specifically with educational objectives in mind. In “Game‐based learning” it is argued that certain pre‐existing commercial off‐the‐shelf games can be used by educators within a proper framework. The third type of approach to Education and Digital Games is what MacCallum‐Stewart (2011) calls “Stealth Learning”, which she defines as the way players unintentionally absorb information within games in ways unintended by designers. The word “stealth”, however, may have a negative connotation and hints that the designers might have had intentions they surreptitiously planted. “Incidental Learning” is here considered to be a neutral form of expressing the same idea – it is a type of Informal learning that is unintentional and spontaneous and can happen in many different environments or situations. Marsick and Watkins (2001) believe Incidental Learning happens unintentionally and unconsciously. They add that Incidental learning is unstructured, and stress that while related to other concepts such as nonformal and “en passant” learning, it differs from them. Garrick (1998) himself chooses the expression “Learning Informally” to refer to Incidental Learning, as in his view “Learning Informally” conveys the important element of spontaneity. This investigation considers that “Incidental Learning”, “Learning Informally” and “Stealth Learning” (MacCallum‐Stewart, 2011) in practice all refer to the same posit. The biggest argument against Incidental Learning in Digital Games is that it is very difficult to prove it is happening. Players themselves don't always recognize they are learning, and MacCallum‐ Stewart claims they might even “actively resist/refute it if this is pointed out to them.” (2011, p. 113) This idea is also supported by Marsick and Watkins (2001), who point out that “When people learn incidentally, their learning might be taken for granted, tacit, or unconscious.” (p. 26). Several claims as to what some of the cognitive outcomes are that may result from playing Digital Games have been made or quoted in the literature, usually followed by a disclaimer explaining that these are applicable to different degrees and dependent on situations, yet without going into detail as to what these situations or degrees are. These claims are presented in Table 1: Cognitive Learning Outcomes as suggested in previous research. The claims can be classified into four main categories: Resource Management, Interpersonal Skills, Information Handling, and Systemic Thinking. Outcomes belonging to the Resource Management category include the coordination and sharing of resources. A lot of research has strongly postulated on cognitive outcomes

714

Gabriela Rodríguez belonging to the Interpersonal Skills category, such as leadership, teamwork, and communication. The outcomes pertaining to the Information Handling category refer to the usage of different sources for decision‐ making. The last category, Systemic Thinking, concerns problem solving and analytical thinking. While this categorization might appear simplistic, it serves as a basic starting framework which facilitates the structure of the analysis, and helps scrutinize whether these or other learning outcomes were displayed in the case study. To some extent, these four main categories correspond with the main educational competencies mentioned in the SCANS Report, as cited by Collins and Halverson (2009). The purpose of this investigation is to find out whether a WoW guild provides opportunities to acquire these learning outcomes and to describe the contexts in which this happens, as well as to explore players’ perceptions of themselves as learners.

2. Methodology This is a mixed‐methods study which aims to find out: Does playing in a World of Warcraft guild offer learning opportunities? How do players perceive their own learning in the game? The first question aims to corroborate, refute and/or complement previous research in a descriptive and exemplified way. The second question addresses players' views and attitudes towards both the game and their own learning inside the game. The hypothesis is that most players are unaware of any learning taking place. If pressed to reflect upon it, most players will probably only be able to identify particular skills which are most closely related to traditional schooling, such as spelling or literacy skills. Guilds are a type of player interaction this study focuses on. Players are able to form and join guilds, stable online communities of varying sizes with shared goals. Some guilds are mostly social, and in them members provide each other company when playing. There are also raiding guilds, like the one chosen for this study, which regularly try to overcome the game's biggest challenges together. ”ABC” guild was chosen out of convenience, as the author knew a high‐level player in it willing to commit to collect data. The guild was quite international, with members living in 16 different countries. ABC plays in a European, English‐speaking player‐ versus‐player server. It had two official “25‐man” raids per week, which means that twice a week, twenty‐five players would spend approximately 4 hours trying to advance in the game together. At the time data was collected, ABC was made up of around 120 people, half of which played regularly. Members had clearly defined statuses, either social players, raiders (candidates or official ones) and guild officers, who usually are raiders as well. At the time of the observation ABC's guild leader no longer played WoW and the actual running of the guild fell to guild officers. Observations were carried out from December 2010 to February 2011, and are divided in two parts. The first part was carried out during the first two weeks of the ”Cataclysm” game expansion, and in it, all ABC players were observed as they leveled and discovered the new content. A total of 114 hours were recorded for this part. For the second part, only official ABC raids were recorded within a period of two months. Table 1: Cognitive learning outcomes as suggested in previous research Category Resource Management

Claim Resource coordination Resource sharing skills Financial and economic concerns Communicating with different kinds of people

Interpersonal Skills Negotiation and Bargaining ‐ elaborated by Squire (2011) as procedures, goals, efficient play, and room for experimentation and mistakes Forming alliances Leadership ‐ evaluating members ‐ recruiting members ‐ intervening in disputes

715

Author Collins and Halverson (2009) Kirriemuir (2003) MacCallum Stewart (2011) Collins and Halverson (2009) Kirriemuir and McFarlane (2004) Kirriemuir (2003) Kaplancali (2008) Prax (2010) Collins and Halverson (2009) Kirriemuir and Mcfarlane (2004) Squire (2011) Kaplancali (2008) Collins and Halverson (2009) Seely Brown in MacCallum Stewart (2008) Reeves et al (2007) Kaplancali (2008) Prax (2010)

Gabriela Rodríguez Category Resource Management

Claim Resource coordination Resource sharing skills Financial and economic concerns

Group decision making Teamwork and collaboration

Information Handling

Application of numbers Handling multiple real‐time sources of information upon which to make decisions Systemic Thinking Complex strategic thinking

Planning Systemic thinking Problem‐solving

Author Collins and Halverson (2009) Kirriemuir (2003) MacCallum Stewart (2011) Beck, J. & Wade, M. (2006) Kirriemuir and McFarlane (2004) Kirriemuir (2003) MacCallum Stewart (2011) McGonigal (2011) Kirriemuir and McFarlane (2004) Kirriemuir and McFarlane (2004) Reeves et al (2007) Collins and Halverson (2009) Kirriemuir and McFarlane (2004) Kirriemuir (2003) Kirriemuir and McFarlane (2004) Squire (2011) Gee, in Squire (2011) Collins and Halverson (2009)

Only players who took part in these raids were observed. In total there were fourteen 25‐man raids and two 10‐man raids, happening every Wednesday and Sunday. This part of the observation summed up to 56 hours. Permission to record chat logs and Teamspeak was acquired from the guild beforehand. Written interactions, namely guild chat and raid chat, were recorded using a program called WoW Scribe 3.3.1. Private conversations between guild members were not recorded. WoWScribe only logs what is typed in the game. However, during raids most guilds rely heavily on voice‐software that lets them communicate with each other using a microphone in something much like a conference call, thus leaving their hands free to play. One of the most popular of voice‐software is TeamSpeak, which is used by ABC guild to communicate. TeamSpeak3 permits recording voices, so this was used to record guild voice interactions during raids. Unfortunately TeamSpeak did not record silences. Data gathered was coded according to predefined core categories derived from the literature: Resource Management, Interpersonal Skills, Information Handling and Technology, and Systemic thinking. After the observation, all guild members were asked to answer a questionnaire. Out of N=51 guild members who did, N=44 were male and N=7 female. Several factors were considered when analyzing the questionnaire, mainly player age, player occupation, player rank and raid participation. The questionnaire consisted of 79 points, out of which four were mandatory open‐ended questions and the rest were close‐ended questions. Fifty‐five of those close‐ended questions consisted of Likert scales.

3. Results 3.1 Resource management Resources were classified as financial (guild gold, DKP points, Auction House), inventory (gear and materials), human (abilities/specializations and roles), or production (professions, food and flasks). In the observation, it became clear that while all players learn to locate and manage certain resources, and that in fact, these skills are essential to be considered a proficient player, the degree to which these skills are acquired varies greatly. Players in a leadership position had more resource‐related tasks than regular players, and said tasks were considerably more complex. 3.1.1 Financial Players are personally awarded as rewards for completing quests, and with the Cataclysm expansion, an extra 10% of that amount was automatically awarded to the guild as a group. ABC guild officers were faced with the new task of deciding how to use this gold, such as what the daily amount of guild gold available should be and who should have access to it. Guild officers were also in control of the Guild Bank, a game interface tool used to manage gold and other inventory resources. Besides monetary currency, guild members used the DKP, Dragon Kill Points, system. ABC members would earn DKP points for participating in official raids and defeating raid bosses, or powerful enemies controlled by the game. Defeating these raid bosses resulted in valuable and rare items, which were assigned a certain DKP price by a guild officer. If a raid member wanted this item, he or

716

Gabriela Rodríguez she will need to have earned enough DKP points to exchange for it, or agree to have negative DKP points which will have to be paid for with future raid participation. In ABC's case an officer kept track of points earned and spent, a time‐consuming and meticulous task. The Auction House (AH) is another important example of resource management inside the game. As has been mentioned, after completing challenges players earn rewards such as gold and items. Items are not always needed by a specific character, and the AH is used to sell these items to other players with the highest bid. Cataclysm upset the delicate balance of the economy, as the new expansion offered improved products and materials. Since not everyone had access to the game at the same time, the best products and materials were very expensive, while old ones became devalued. 3.1.2 Inventory Some of the items players earn by completing challenges are gear, or weapons and armor which make players more powerful. Selecting the right gear is not always a clear‐cut choice, and having the right gear is a way to demonstrate competency and expertise within the game. Apart from personal management of gear, this also happened at a guild level. ABC earns gold and items as a group through raiding. All items follow certain binding rules ‐ some are BOP, or bound when picked up by a player, while others are BOE, or bound 'on equip', when the player uses it. This means that BOP items must forcibly be given to a raider during the raid. BOE items can be stored or given to some other player at a later time, even one who wasn't in the raid. Item distribution is a complex and time‐consuming process in which DKP masters ‐ in ABC's case, a guild officer ‐ are responsible for the coordination and distribution of resources depending on many different factors (Figure 1). The right management of inventory resources is also manifested by individual players, as they also give a lot of thought to the different aspects involved in choosing the right gear.

Figure 1: Item distribution process followed by ABC Guild

717

Gabriela Rodríguez 3.1.3 Human In a guild, especially a raiding guild like ABC, the most important resources are the players themselves, and it's up to guild officers to administer them. It is important for guild officers to know what skills and specialization their guild members have, as certain challenges within the game require a specific set‐up or configuration of players. Knowing in advance what they have at their disposal and what they are lacking is essential in order to prepare raids and start recruiting new guild members. 3.1.4 Production During the observation it was seen that raiders in particular had a responsibility to prepare themselves for raids with all possible resources. Raiders were expected to be both properly geared and buffed. A buff refers to bonus points that make players temporarily more powerful. These points can be gained from player spells, food, and flasks. In ABC guild, one of the most difficult things for players to learn was to prepare with enough food and flasks for raids. This was a problem over and over again throughout the observation period. Even though food and flasks were sometimes freely shared, players still forgot to use them. In more than one occasion, an officer reprimanded guild members, complaining about having to constantly remind people to flask up and eat. There were frequent discussions about what could be a punishment for those who consistently needed to be reminded to properly prepare and use their resources. In the questionnaire, members were asked whether they felt they were well prepared for guilds, and there was a significant difference between the answers of officers and non‐officers. This indicates that there is a wildly different perception on what it means to be prepared, which is perhaps the reason this issue kept coming up during the observation. So while there are a great number of cases in which members demonstrated awareness in regards to resources and logistics, such as learning about spell rotations, DKP points, professions and gear, it also became clear that members did not always agree on the importance of some resources. Despite knowing where and how to acquire them, sometimes with very little personal effort, some members neglected to use all the resources at their disposal.

3.2 Interpersonal skills Also corroborating the cognitive learning outcomes mentioned in the literature are many in the Interpersonal category, as results were mostly positive in this category. Members were seen to improve their negotiation abilities throughout the observation, and they displayed a strong sense of loyalty towards the guild as well as team‐work skills. Social gameplay and support was evident amongst ABC members, although views on social aspects depended on factors such as whether players raided or not. The guild was an important source of information and help, but also of emotional support. Generally, relationships between guild members were very good. A yearly ABC guild party had been organized for the last four years, each time hosted by a different person in a different country. Members often met each other in person if they had the chance, and they kept in touch through social media. Inside the game, guild members joked and talked with each other on a variety of topics. In the questionnaire members, both raiders (N=33) and non‐raiders (N=18), were asked to give their levels of agreement to different statements which all together measure the importance of social aspects. A one‐way analysis of variance ANOVA was conducted to compare raiders' and non‐raiders' attitudes towards different social aspects of the game. As can be seen in Table 5, for most of the items, there were no statistically significant differences between the groups of raiders and non‐raiders, except for the last two statements. The mean of agreement amongst raiders towards the statement “Sometimes I’m too busy or tired for WoW, but I play anyway because the guild needs me” was 3.94, whereas regular players’ was 3.00 at a .002 significance level. The high time and performance demands of regular raiding had been mentioned in an open‐ended question of the questionnaire as drawbacks of belonging to a guild. It can be interpreted that raiders are more likely to see the guild as something that requires commitment and ties them down to a certain schedule. All this, in turn, helps explain the mean difference in attitudes of raiders and non‐raiders to the statement “I feel I have made real friendships in the guild”. The mean of agreement of raiders was 3.64, whereas of non‐ raiders 4.28, a statistically significant difference at the .015 significance level. It had been presumed that as raiders engage in teamwork more than regular players, they would have a more positive attitude towards the social aspects of the game, yet according to the questionnaire it is the other way around. It could be that non‐

718

Gabriela Rodríguez raiders get the social support of a group without having the responsibilities, pressures, or conflicts of a raider, which might lead to a more positive perception of the guild. Perhaps the difference is also in players' main focus – raiders may be more goal‐oriented while non‐raiders might be more interested in socialization. This suggests that non‐raiders focus on “positive” social aspects of being in a guild, like companionship, whereas raiders may focus on what may be felt as “negative” aspects of being in a guild, such as commitment. Overall, the observation and the statistics from the questionnaire support the idea that Interpersonal learning outcomes are taking place. However it became apparent that regular players and raiders have vastly differing views on the feelings of community, teamwork and other social aspects. Many of the outcomes in this Interpersonal category were seen amongst all members, such as bargaining and forming alliances, while other outcomes, such as recruiting and negotiating, were seen only amongst officers. Despite all this, conflict was not always properly addressed nor resolved in ABC guild, as seen in the case of one female player who was harassed, in which officers did not intervene.

3.3 Information handling ABC members were observed discussing the importance of finding the most useful data when faced with extensive amounts of information available yet limited time. Both their frequent discussions on this topic, as well as the situation‐dependent use they made of several different sources of information, was seen as fully supporting the claims made in the literature. Overall, the sources of information used by ABC members were varied and numerous. What ABC members were most concerned about was the reliability and usefulness of some of the information available, as well as how to best acquire information when sources are numerous yet time is not. In some occasions, despite receiving information through game add‐on software pointing out that something was going wrong, ABC members sometimes disregarded this information. This could be because of previous experience with bugs or add‐ons' unreliability. So part of knowing how to use technology was knowing when not to rely on it 100%. Learning different ways to handle data was seen to be a learning outcome for ABC guild members, although members disagreed over the best methods for acquiring and using this data. In the observation, members acquired these data‐handling skills by doing research on a variety of technological sources, getting information from peers, and through experience.

3.4 Systemic thinking Systems refers to a person's understanding of complex interrelationships, which Collins and Halverson (2009) claim involves the ability to develop strategies to solve complex problems, as well as outwitting opponents, calculating which approach is most likely to work, coordinating large‐scale movements, and making decisions about political values. Kirriemuir and McFarlane (2004) claim that when playing Digital Games, players develop strategic thinking and planning skills. All the learning outcomes mentioned in the literature review, such as planning, problem‐solving, and strategic/systemic thinking, were seen. It is important to note that these outcomes were mostly observed in a 25‐man raiding environment, which is not to say that they cannot occur outside a raid of this size, only that they were not observed to such a high degree. However, it was also observed that the raid sometimes failed to capitalize on many of the learning opportunities afforded by their shared experience, resulting in future difficulties overcoming the same obstacle. Members seemed to recognize the need to identify, discuss and correct mistakes in strategy, yet lost opportunities to do so, both during the raid as well as afterwards in raid reports written by officers. Respondents' answers in the questionnaire point to reluctance in bringing up the mistakes of others', which might also slow progress. Based on the questionnaire, there are also considerable differences in how officers and non‐officers perceive both overall performance as well as the responsibilities of leaders, and it is this ambiguity of roles and disparity in expectations which can also slow down learning and the advancement of the guild.

3.5 Perception of learning The second research question relates to how players perceive their own learning within the game. First, respondents were asked whether they felt they had learned more or become better players by being in a guild. The most relevant means difference was surprisingly not dependent on whether players raided or not or were officers or not, but on age. While both the group of players 30 years old or younger and the group of players 31 years old and older show strong agreement, the trend seems to be stronger with players 31 years and older (mean of agreement 4.64 compared to 3.92). This statistically significant difference at a .021 significance level

719

Gabriela Rodríguez indicates that older and presumably more mature players are likelier to perceive the benefits of belonging to a guild. It was found that ABC members had very different views on their own learning within the game, depending on different factors. Remarkably, players who did not feel guilty about how much time they spend playing WoW were more likely to agree or strongly agree with the statement “I learn a lot by playing WoW”. The relationship between these factors is not clear. It could be that players who have feelings of guilt are less susceptible to the learning opportunities offered by the game, or that they let their guilty feelings overshadow these learning outcomes. Another possibility is that when players either fail to acquire learning outcomes or remain unaware of any learning taking place, they tend to feel guiltier about their playing habits, as they see no distinct value in their activities. Either way, this relationship between guilt and value gained is worth exploring. Another important discovery was that older players were more likely to agree that they have become better players because of the guild, indicating that age might play a role in learning within the guild. When players were asked “What do you learn when you play WoW?”, their answers showed some variation depending on factors such as whether they were officers or not and also their age.

4. Conclusions Part of the purpose of this investigation was to describe concrete examples of opportunities offered within a WoW guild which may result in the learning outcomes mentioned in the literature. Based on the results, some suggestions for further research would be to find out whether players who repeatedly failed to prepare themselves for raids were either unwilling to do so or unable to learn and remember the impact their personal use of resources would have upon the group. In either case, these players' attitudes towards group efforts and or learning should also be studied more profoundly. It would also be interesting to discuss with members their own information‐seeking strategies, or what kind of criteria they follow for choosing specific kinds of sources for certain situations. In regards to the results found in the Interpersonal Skills category, it would be worthwhile to investigate whether harassment is a problem faced by many females within the game, or whether unlike ABC, other guilds have strategies for communication/conflict‐resolution skills. It would also be valuable to analyse a guild's progress throughout different raids, to study whether knowledge or learning‐ abilities acquired in one raid are transferred to later ones. In regards to the second research question, the relationship between guilt and perception of learning is worth exploring. As this is a case study chosen out of convenience, ABC guild is not meant to be representative of all guilds. Furthermore, as mentioned before, the fact that silences were not recorded by Teamspeak software is a limitation of the study. Another limitation is that the only chatlogs available were guild chats, and so it is likely that behind‐the scenes communication might have contributed important insights to the observation. It is also possible that given the large amount of data collected, certain aspects could have been overlooked for not fitting into predefined codes. While every attempt has been made to go over all data carefully, it is possible that the results presented are skewed toward the learning outcomes mentioned in the literature review, having a blind‐spot towards other interesting developments. WoW is a rapidly evolving game, with constant updates and adaptations made. On November 2011, Blizzard introduced the Raid Finder option, which makes it easy for players to find others to raid with without needing to belong to a guild. On September 2012, a new expansion was released, with further changes and adaptations to the game. Despite the limitations of this investigation, hopefully it offers new insight into the varying types of player experiences within a same game, and even within a same guild. While most of the learning outcomes described in the literature were observed to be acquired incidentally to some degree or another, they are not all automatically and uniformly gained simply by playing the game. WoW is a game designed to support multiple kinds of players, and some of the learning outcomes, such as those in the Systems category, might require time, dedication, and effort that not all players are willing to make.

References Beck, J. & Wade, M. (2006) The kids are alright: How the gamer generation is changing the workplace. Harvard Business School Press, Boston. Collins, A. & Halverson, R. (2009) Rethinking Education in the age of technology: The digital Revolution and Schooling in America, Teachers College Press, New York. Garrick, J. (1998) Unmasking Workplace Learning: The Subtle Power of the Informal, Routledge, New York.

720

Gabriela Rodríguez Kaplancali, U.T. (2008) Leadership in multiuser online environments: the effects of user‐generated content on leadership styles across crafting and creation environments in a multiplayer online game, PhD thesis, Claremont Graduate University, California. Kirriemuir, J. and McFarlane, A. (2004) “Literature Review in Games and Learning”, Futurelab Series [online], Futurelab Series. Available from: http://hal.archives‐ouvertes.fr/hal‐00190453/ [2.5.2013] Kirriemuir, J. (2003) “The relevance of video games and gaming consoles to the higher and further education learning experience”, JISC TechWatch Report [online], v3(12). Available from: www.jisc.ac.uk/uploaded_documents/tsw_02‐01.rtf. [2.5.2013] MacCallum‐Stewart, E. (2011) Stealth Learning in Online Games. In: deFreitas, S. & Maharg, P. (Eds.), Digital Games and Learning, Continuum, New York. 2011, pp. 107‐128. MacCallum‐Stewart, E. (2008) Never Such Innocence Again: War and Histories in World of Warcraft, In: Corneliussen, H. and Walker Rettberg, J. (Eds.), Digital Culture, Play, and Identity: A Critical Anthology of World of Warcraft Research, MIT Press, Cambridge. 2008, pp. 39‐ 62. Marsick, V. J. and Watkins, K. E. (2001). Informal and incidental learning. In: S. B.Merriam The new update on adult learning theory: NewDirections for Adult and Continuing Education, No. 89. Jossey‐Bass, San Francisco. 2001, pp. 25‐34. McGonigal, J. (2011) Reality is Broken: Why Games Make us Better and How they can Change the World. Jonathan Cape, London. Prax, P. (2010) “Leadership Style in World of Warcraft Raid Guilds” [online], Paper read at DiGRA Nordic 2010. Stockholm, Sweden, August. Available from: http://www.digra.org:8080/Plone/dl/display _html?chid=http://www.digra.org:8080/Plone/dl/db/10343.52340.pdf [2.5.2012] Reeves, B. et al (2007) “Leadership in Games and at Work: Implications for the Enterprise of Massively Multiplayer Online Role‐playing Games” Seriosity Inc. [online] Available from: http://www.seriosity.com/downloads/Leadership_In_Games_Seriosity_and_IBM.pdf [2.5.2013] Squire, K. (2011) Video Games and Learning: Teaching and Participatory Culture in the Digital Age, Teachers College Press, New York.

721

In Search for the Right Measure: Assessing Types of Developed Knowledge While Using a Gamified Web Toolkit Martin Ruskov1, Paul Ekblom2 and Angela Sasse1 Information Security Research Group, University College London, London, UK 2 Design against Crime, Central Saint Martins College of Arts and Design, London, UK m.ruskov@ucl.ac.uk p.ekblom@csm.arts.ac.uk a.sasse@cs.ucl.ac.uk 1

Abstract: Game‐based learning has been used to teach topics in diverse domains, but it is still hard to determine when such approaches are an efficient learning technique. In this paper we focus on one open challenge – the limited understanding in the community of the types of knowledge these games help to develop. Using a taxonomy that distinguishes between declarative, procedural and conditional knowledge, we evaluate a game‐based toolkit to analyse and solve an information security problem within a holistic crime prevention framework. Twenty‐eight participants used the toolkit. We designed a portfolio of learning assessment measures to capture learning of different types of knowledge. The measures included two theoretical open‐answer questions to explore participants' understanding, three problem‐ specific open‐answer questions to test their ability to apply the framework, and 9 multiple‐choice questions to test their ability to transfer what was learned to other contexts. The assessment measures were administered before and after use of the tookit. The application questions were analysed by classifying suggested ideas. The theoretical questions were qualitatively analysed using a set of analytical techniques. The transferability questions were statistically analysed using t‐ tests. Our results show that participants' answers to the application questions improved in quality after the use of the toolkit. In their answers to the theoretical questions most participants could explain the key features of the toolkit. Statistical analysis of the multiple‐choice questions testing transferability however failed to demonstrate significant improvement. Whilist our participants understood the CCO framework and learned how to use the toolkit, participants didn't demonstrate transfer of knowledge to other situations in information security. We discuss our results, limitations of the study design and possible lessons to be learned from these. Keywords: learning assessment, open questioning, information security, types of knowledge, SOLO taxonomy

1. Introduction A number of serious games have been developed and used to support learning in a wide range of domains. However, there is still an ongoing debate to identify when game‐based approaches are more efficient than traditional learning techniques. This discussion has been approached by both addressing the pedagogic principles that could be embedded in games to support learning, see e.g. (Kebritchi & Hirumi 2008; Hmelo‐ Silver et al. 2007), and by looking into the kinds of knowledge that are being developed and into the possible ways to assess it, e.g. (Gijbels et al. 2005; Anderson & Lawton 1992). The focus of this paper is the limited understanding in the community of the types of knowledge that game‐based learning helps to develop. Despite the presence of differing taxonomies there is a wide agreement that knowledge is not a simple construct and different types of knowledge exist. Here we review three different approaches to classifying knowledge and choose to use one that is simple, yet representative. Probably the most widely used knowledge classification is Bloom's taxonomy (Bloom et al. 1956), used e.g. in (Anderson & Lawton 1992). Out of the three domains of this taxonomy, the most relevant to the type of learning considered here is the one on cognitive learning. It features six incremental levels. They start with knowledge – the ability to recall data or information. The second level is comprehension ‐ understand the meaning of that data and the ability to state a problem in one's own words. Bloom's third level in the cognitive domain is application – the ability to use a concept in a new situation (also referred as transferability later in this text) or unprompted use of an abstraction (the meaning used later in this text). The fourth level is analysis, or the ability to separate material or concepts into component parts so that the structure of how they are organised may be understood. The final two levels are synthesis – building a structure or pattern from diverse elements and putting parts together to form a whole, with emphasis on creating a new meaning or structure; and evaluation – making judgements about the value of ideas or materials.

722

Martin Ruskov, Paul Ekblom, and Angela Sasse Another taxonomy that was developed by Biggs (1994) and that acknowledges that cognitive learning can hardly be ordered along a single hierarchy, considers seven types of knowledge. This classification considers the less tangible tacit and intuitive types of knowledge. According to Biggs tacit knowledge “is manifested by doing and is not verbally accessible”. Intuitive knowledge is felt and might develop before being expressed symbolically. Similar to Bloom, Biggs argues that these two types of knowledge develop further in a hierarchy into declarative, theoretical and meta‐theoretical. Here declarative is the widely understood formulation of facts, theoretical represents an abstraction from declarative, whereas metatheoretical is the level when scientific work around abstractions may lead to paradigm shifts, i.e. possibly introducing some sort of revision of previous knowledge. Finally Biggs considers the procedural and conditional types. Procedural is the knowledge of how things need to be done, formulating necessary event sequences or order of actions. Conditional knowledge involves making decisions, based on the circumstances. In the author's words “conditional knowledge provides the metacognitive support to procedural knowledge”. Although the taxonomy that Biggs suggests, provides extensive coverage of a wider idea of knowledge, it could be very difficult to work with. Intuitive and tacit by definition are very difficult to assess externally, and metatheoretical might be contradictory. That's why, for practical purposes, another simpler taxonomy is of interest. It was suggested by Sugrue (1995) and considers only three types of knowledge: declarative, procedural, conditional. Whereas the later two types could be considered to overlap with the ones suggested by Biggs, declarative knowledge could be seen as the combination of the explicit types within the hierarchical part of his taxonomy. The distinction that Sugrue's taxonomy captures, has been acknowledged as useful also in other studies, e.g. (Gijbels et al. 2005). Different assessment techniques have been recommended to be used to capture different types of knowledge. Typically, closed questioning (i.e. possible answers are pre‐suggested by the question) is widely used for declarative knowledge and the lower levels of Bloom's taxonomy, whereas open questioning is used where there is more ambiguity or there is interest in the reasoning process behind the answer (Atherton 2011). A typical closed assessment technique are multiple‐choice questions (MCQ), and possible open assessment techniques are essay questions or concept maps, as used by Hay and Kinchin (2008). As written assessment is common in educational systems, and concept mapping might require some form of training, the former was preferred in this study. Naturally, there is a trade‐off between the potential of a measure to capture depth of learning, and the difficulty to actually assess the provided answer. Whereas when using multiple‐choice questions, correct answers only need to be checked and counted and after that could be statistically analysed with standartized procedures, open questioning requires careful consideration of each answer and analysis of its content. Such analysis is typically specific to the topic and sometimes does not result in an unique interpretation. Nevertheless, different general‐purpose techniques to analyse text exist. One common approach, used both in education and research is content analysis (Weber 1990). Content analysis can consider words, sentences or paragraphs as its unit of analysis and is meant to be independent of any implied context, thus trying to be as objective as possible. While widely used across a variety of domains, content analysis could not capture aspects of assessment, when questions are put in the context of a specific learning activity and, due to time pressure, relatively short responses are expected. Another technique, that allows for better consideration of the learning context, is thematic analysis (Aronson 1994). Thematic analysis allows to interpret a single construct in written responses as belonging to several different themes that it might relate to, thus making it possible to interpret references to content that has been learned. Finally, as represented by more complex forms of knowledge (e.g. synthesis in Bloom's taxonomy and both procedural and conditional in the other reviewed taxonomies), learning might not be represented by accumulating new factual knowledge, but rather better understanding the interdependencies between knowledge constructs that were already accessible to the learner. To address these, Biggs and Collis (1982) have developed a taxonomy meant to assess written answers and essays in depth – the structure of observed learning outcomes (SOLO in short). It represents a hierarchy used to classify written text according to its complexity and features levels from prestructural and unistructural through multistructural to relational and, finally, extended abstract. The first part of that hierarchy explores the ability of respondents to explain one or several themes within the domain, whereas the later levels correspond to their ability to explain their interrelatedness or draw more general conclusions, based on the considered themes.

723

Martin Ruskov, Paul Ekblom, and Angela Sasse The learning domain of the study was information security and its content adapted the Conjunction of Criminal Opportunity (CCO) framework from crime prevention and community safety (Ekblom 2010) to the purposes of information security.

2. Method We report the evaluation of an early prototype of a game‐based toolkit. The toolkit employs gamification techniques to analyse and solve an information security problem with the CCO framework. Twenty‐eight participants took part in a study with an experimental aspect. Their age ranged from 20 to 65 with an average of 26.5 and median 23.5. There were 19 male and 9 female participants. Participants were randomly assigned in two groups of 14 participants each. The experimental part of the study was used to examine the effects of variation on learning with the toolkit. As part of the process guided by the toolkit and described below the experimental group was asked to assess a predetermined set of ideas, presented as if other study participants had written them, whereas the control group merely assessed their own ideas. This experimental component of the study was considered to be independent of the results reported in this paper. CCO combines situational and offender‐oriented approaches to crime prevention (Ekblom 2010). It comes at the price of greater, but necessary, complexity relative to other widely used frameworks. Paradoxically this equips practitioners to better handle the complex reality of crime. The framework suggests 11 circumstances the conjunction of which leads to the criminal event and has been visualized on a diagram (refer to the background image of Figure 1a). The diagram features these circumstances as rays coming together to form the final conjunction. Considering each of the eleven generic causes in the CCO diagram, naturally leads to ideas for their intervention counterparts. These intervention ideas could block, weaken or divert the causes, such that the criminal event is less likely to be attempted, or to succeed. The variety of possible intervention ideas and the exact details of their implementation lead to a classification of how specific, or general these ideas are. The CCO framework distinguishes between principles and methods. Methods represent the context‐dependent practicalities of an intervention. These are often difficult to transfer to other situations – the success of interventions is very context‐dependent. Principles, on the other hand, are the more general description of what is being done that is formulated in a way that could be re‐ applied, customised to context in other situations. Six intended learning outcomes were sought when learning the CCO framework with the toolkit. These were listed and classified according to Sugrue's taxonomy as illustrated in Table 1. Table 1: The intended learning outcomes (ILO) that were considered to be important to achieve with the game‐based toolkit and the corresponding type of knowledge according to Sugrue's taxonomy Code

Intended Learning Outcome (ILO)

Type of Knowledge

ILO1

Understand what exactly CCO is, what it’s for, and the wider process in which it can be used.

Declarative

ILO2

Use CCO to interpret causes of criminal events within the worked examples.

Procedural

ILO3

Use CCO to identify preventive intervention principles that they could bring to bear against these causes.

Conditional

ILO4

Generate greater numbers of plausible intervention ideas – i.e. the first stages of innovation.

Conditional

ILO5

Grasp of the key threshold concepts, e.g. ecological level.

Declarative

ILO6

Use CCO terminology correctly.

Declarative

The toolkit guided participants through a process consisting of four consecutive parts: introduction to the scenario, idea generation, idea assessment and score review. The first screen of the toolkit introduced participants to an insider attack scenario. This included a textual description of the problem and a list of incidents that exemplify it. Once participants were introduced to the scenario, they were taken to the subsequent idea generation part. It featured an interactive version of the CCO diagram. When a participant clicked on one of the 11 contributing

724

Martin Ruskov, Paul Ekblom, and Angela Sasse rays of CCO, further information appeared and the participant was prompted to identify causes in a dedicated dialogue box (see dialogue box of Figure 1a). Once done with causes, participants were taken to a similar screen where they worked on interventions. Here they suggested their own methods and matched them to a customizable list of principles.

Figure 1: Screenshots illustrating key steps in the process, embedded in the CCO toolkit. From top to the right, then moving to lower row to the right: a) the screen prompting participants to generate ideas for causes, featuring the CCO diagram in the background and the interface to enter ideas; b) askinig participants to consider if their ideas relate to other generic causes; c) participants assessing and commenting on ideas of others; d) the final score screen with comments fed back to the idea originators. These are provided for illustration only and the textual page content is not relevant to this publication. After these phases of generation participants were presented with an opportunity to identify possible matches between any of their suggested interventions and the 11 generic causes (see Figure 1b). This way they were given an opportunity to further explore the influence a suggested intervention could have on wider causes and subsequently how it is interconnected with other interventions. The assessment part of the process prompted participants to evaluate ideas of interventions. As already explained in the beginning of this section, the two groups got to review different sets of ideas in their assessments. Participants were asked to grade each idea along a 5‐point Likert‐scale and were provided with an empty text field if they wanted to provide further comments to clarify their assessment. After that the toolkit engaged participants in a role‐based assessment of the proposed interventions. In this assessment (Figure 1c) participants reviewed a predetermined set of ideas (as already described), both by

725

Martin Ruskov, Paul Ekblom, and Angela Sasse rating them along a 5‐point Likert scale and commenting whenever they found appropriate. Finally, the toolkit used a simple pattern matching mechanism to cluster intervention ideas. This entire process made it possible for the toolkit to provide feedback on ideas, suggested by participants, and to ultimately assign a score to participants in the study. This happened within the final score and ranking screen (Figure 1d) that showed to participants their performance. This included a table with intervention ideas, suggested by the participant, and overall statistics and ranking of their performance. The table featured a breakdown of the three scores these ideas cumulatively received from other participants and feedback other participants provided via comments they gave while assessing previous similar ideas. The game‐based toolkit is described in further detail in other publications. The development and design decisions are reported in (Ruskov, Celdran, et al. 2013), and the learning and usability analysis is reported in (Ruskov, Ekblom, et al. 2013). Table 2: Questions used in the assessment of this study and corresponding intended learning outcomes Code

Question

Addressed ILO

TQ1

How would you describe the CCO framework?

ILO1

TQ2

What is the CCO framework used for?

ILO1

PQ1

What are the key causes of the insider attacks in the above scenario?

ILO2

PQ2

What are possible interventions methods that would reduce or prevent further attacks of this sort?

ILO4

PQ3

For each of the methods, please suggest one reusable principle that generalizes the approach that has been used.

ILO3

MCQ1

Which of the following (if any) are causes working on an employee at a bank to help the commitment of financial fraud?

ILO5

MCQ2 Which of the following actors (if any) have interest in secretly planting a trojan onto a home computer?

ILO5

MCQ3

ILO3

MCQ4

Which of the kinds of methods below apply to a “use secure password on your private computer” publicity campaign within a company?

Which of the following (if any) could be parts of the enclosure around a file that is potential ILO1, ILO5 target?

MCQ5 At an open access internet café which of the following (if any) are potential non‐professional crime preventers?

ILO5

MCQ6

ILO5

Which of the following (if any) are well‐formulated intervention principles?

MCQ7 Which of the following (if any) are intervention methods rather than intervention principles?

ILO5

MCQ8 Which of the following (if any) are resources for a potential offender to commit an insurance fraud?

ILO5

MCQ9

ILO1

An IT company has several cases of intellectual property leaks to competitors. For which of the following (if any) could they use the CCO framework?

We designed a portfolio of learning assessment measures in order to capture progress corresponding to each of the intended learning outcomes, and resp. to Sugrue's knowledge taxonomy. The measures included two theoretical open‐answer questions to explore participants' understanding (see TQ1 and TQ2 in Table 2), three problem‐specific open‐answer questions to test their ability to apply the framework (PQ1, PQ2, PQ3, see Table 2), and nine multiple‐choice questions to test their general understanding and their ability to transfer what was learned to other contexts (MCQ1‐9). Each of the multiple choice questions included four possible answer options and participants were allowed to select any number of correct answers, or none. Participants were also given the opportunity to provide further comments or clarifications to each possible answer. The study procedure engaged each participant for 90 minutes. The assessment measures were administered before and after the task of using the game‐based tookit. Participants were allowed up to 20 minutes (whereas

726

Martin Ruskov, Paul Ekblom, and Angela Sasse it typically took them around 15 minutes) to do each of the measurements and were asked to proceed further with the next step if they finished earlier, thus receiving more time to work with the toolkit. The application questions were analysed by means of thematic analysis and by coding the suggested ideas into predetermined broad categories. Because PQ2 and PQ3 addressed two aspects of the same issue, respectively the methods and the principles of a small set of interventions, they were analysed together. The theoretical questions were analysed both with thematic analysis and with the SOLO taxonomy, the former used to explore the particular themes, and the latter – their interrelatedness. The transferability questions were statistically analysed using unpaired samples t‐test with the assumption of equal variances.

3. Results Our results indicate that participants were able to understand, engage with and use the toolkit. This is reported in another currently pending publication (Ruskov, Ekblom, et al. 2013). As Figure 2 demonstrates they showed some improvement in the way they identified problem causes and solutions (answers to the problem questions).

Figure 2: Distribution of answers to application questions: left ‐ PQ1 and right ‐ PQ2 and PQ3 (same distribution). The graphs compare answers before and after engaging with the game‐based toolkit For the identification of causes (PQ1) this meant that there was a stronger shift from causes implied by the scenario or variations of them to more indirect and complex causes that were original suggestions of participants. The average proportion of own ideas almost doubled from 19% to 36%. The proportion of interventions (methods – PQ2 and principles – PQ3) that have to do with staff development increased from 22% to 28%. Such method ideas had to do with solutions such as training or showing better leadership, which are essential to security issues related to disgruntlement (Kirlappos et al. 2013). Most participants showed some form of learning in their answers to the theoretical questions (TQ1 and TQ2) when comparing answers before and after using the game‐based toolkit. These included rephrasing or relating to previous knowledge, explaining new themes within the subject, integrating different themes learned with the toolkit, or adopting the professional language of the toolkit. However, as can be seen in Table 3 these pieces of evidence were diverse. Commonly, study participants expanded their answers after using the game‐based toolkit, thus showing what new understanding they had developed. Typically in such situations they provided further necessary details in their responses after using the toolkit. In Table 3 E11/TQ1 is an example of a participant not giving anything specific in their answer before using the toolkit, but adding relevant concepts after that. This is an example of someone reaching the unistructural level in the SOLO taxonomy in their answer after using the toolkit. On the other hand C11/TQ1 exeplifies a second aspect (interventions) being added to the one already discussed (causes), thus exemplifying a newly developed multistructural answer. In other cases the comparison of the two answers showed a change in the way that a given participant considered the topic and demonstrated their ability to better integrate what they had learned. Commonly they first wrote of the process that the toolkit took them through. After using it, they also wrote of its goal or implications, alongside the process (see C01/TQ2 in Table 3 for an example). This is typical for the relational

727

Martin Ruskov, Paul Ekblom, and Angela Sasse level of the SOLO taxonomy. Among our participants there were no examples of participants moving to an extended abstract level in the SOLO taxonomy. In two cases participants used their own terminology in their answers. For example. in C03/TQ1 the participant talks of “stakeholders” and “curbing the occurrence” of crime, but neither of these phrases was used in the toolkit or by the facilitator. This shows that they went through a process of relating what they experienced in the study to what they had previously known, describing the new knowledge in their own vocabulary. Eight study participants demonstrated no form of learning in either of their answers to the theoretical questions. Instead they provided the same or even less information in their answers after using the toolkit. In two other cases, although participants showed that their understanding had developed in the answer to one of the questions, they only superficially answered the other. In these responses participants used the professional language of the toolkit, but didn't provide a response with substantial information in their answers (see C13/TQ2). These cases were considered as cases of mimicry, rather than learning. Table 3: A selection from the assessment results, illustrating different types of evidence The third and fourth colums contain unedited participant responses. None of the answers exemplified the extended abstract level of the SOLO taxonomy participant /question

type of evidence

before

after

E11/TQ1

SOLO unistructural

It tries to take a micro‐approach in terms of identifying small problems in society that lead to a crime being committed (in terms of Information Security)

tries to reduce the risk and occurrence and severity of attacks by interrupting in the causes

C11/TQ1

SOLO CCO framework is used to reduce crime it is a framework to identify the cause of multistructural related to leakage or attack of crime related to information system followed information by investigating their by intervention to eliminate the crime. causes.

C01/TQ2 SOLO relational

CCO is used to identify current or potential breaches and to work through all the chains of effect, thus creating watertight solutions.

CCO is used to examine the many potential causes of incidents, and to explore what the implications of potential solutions would be, from all angles. Sometimes the implications are massive.

C03/TQ1

own vocabulary

a comprehensive method of curbing the occurrence of a particular crime with minimal effect to stakeholders.

a comprehensive method of identifying causes, possible solutions and assessing their impact to a particular criminal activity with little impact to stakeholders.

C13/TQ2

mimicry

it help people to reduce the chance of protect vulnerable people from cyber crimes. being cheated during online security. give people an insight about increasing cyber crimes.

Statistical analysis running an unpaired t‐test of the transferability multiple‐choice questions (MCQ1‐9) showed no significant improvement after using the toolkit (test result: mbefore = 23.3, mafter = 23.9, df=54, t=1.674, p=0.226). However, on average the number of participant's correct responses improved by slightly more than a half, i.e. every other participant indicated one more correct option after using the prototype. Nine participants provided 21 comments for clarification of their responses to the multiple‐choice test before engaging with the toolkit, with one responsible for eight of these comments. Only one of the participants provided clarifications to his answers in the final test, repeating one of his previous comments and providing two new ones.

4. Conclusion The learning assessment showed mixed results. All measures showed some improvement, but this learning could not be quantified into statistical significance. Whilst there is evidence that our participants understood the framework and larned how to use the toolkit, there is insufficient evidence to conclude that participants were able to transfer their knowledge to other problems in information security. The answers to the theoretical questions showed indications to the various forms of learning and corresponding varying evidence.

728

Martin Ruskov, Paul Ekblom, and Angela Sasse While there were indications for improvement by the majority of participants, it was difficult to generalize these into distinctive common patterns for the whole group. We hypothesize that these inconclusive results are due to two reasons: shortness of the learning experience and imperfections of assessment. We develop an argument of the limited opportunity for engagement in the learning process that lab‐based learning experiments allow for. This paper presents results of a formative study of a prototype in a lab setting. While it is useful to evaluate serious games in a lab setting in order to improve their usability, progress in learning might be more difficult to capture in a typical one‐hour lab experiment session. Whereas lab‐based studies are still necessary as formative assessment during the development phase of game‐based learning tools, we suggest that class studies or longitudinal web‐based studies are more appropriate to assess learning happening with their help. The different assessment techniques aiming at different types of knowledge allowed us to draw a comparison between the forms of knowledge that participants developed with our game‐based toolkit. It seems that participants were better able to apply their knowledge in context, than to formulate, explain or generalize it. Two reasons for this come to mind. One could be that they actually needed more time and broader perspectives to get a deeper understanding. Another possible explanation is that our toolkit is more suitable for developing procedural and conditional knowledge, rather than declarative, similar to problem‐based learning techniques, assessed by Gijbels and colleagues (2005). The large number of cases when participants provided shorter answers after using the game‐based toolkit, led us to consider several possible reasons for that behaviour. One obvious reason could be that they found that the essence of what has learned could be described with fewer words. However, another reasonable assumption is that they experienced assessment fatigue and were less motivated to put effort into their second answer. A third potential reason that we identified is that they might have considered it unnecessary to repeat something that they had written before using the game‐based toolkit not that long ago. This problem could also be overcome by engaging with studies that would take participants through longer learning periods. Despite the fact that such studies require more effort to yield results, they might lead to more conclusive findings.

References Anderson, P.H. & Lawton, L., 1992. A Survey of Methods Used for Evaluating Student Performance on Business Simulations. Simulation Gaming, 23(4), pp.490–498. Aronson, J., 1994. A Pragmatic View of Thematic Analysis. The Qualitative Report, 2(1). Available at: http://www.nova.edu/ssss/QR/BackIssues/QR2‐1/aronson.html/. Atherton, J.S., 2011. Learning and Teaching: Forms of Assessment. Available at: http://www.learningandteaching.info/teaching/assess_form.htm [Accessed April 30, 2013]. Biggs, J.B., 1994. Modes of learning, forms of knowing, and ways of schooling. In M. S. Andreas Demetriou, ed. Neo‐ Piagetian theories of cognitive development: implications and applications for education. Routledge. Biggs, J.B. & Collis, K.F., 1982. Evaluating the Quality of Learning: The Solo Taxonomy : Structure of the Observed Learning Outcome (Educational Psychology Series), Academic Pr. Available at: http://mie.sagepub.com/cgi/pdf_extract/1/4/20. Bloom, B.S. et al., 1956. Taxonomy of Educational Objectives: The Classification of Educational Goals, Handbook I: The Cognitive Domain B. S. Bloom, ed., Susan Fauer Company, Inc. Ekblom, P., 2010. The conjunction of criminal opportunity theory. In B. S. Fisher & S. P. Lab, eds. Encyclopedia of Victimology and Crime Prevention. Sage. Gijbels, D. et al., 2005. Effects of Problem‐Based Learning: A Meta‐Analysis From the Angle of Assessment. Review of Educational Research, 75(1), pp.27–61. Hay, D. & Kinchin, I., 2008. Using concept mapping to measure learning quality. Education + Training, pp.167–182. Hmelo‐Silver, C.E., Duncan, R.G. & Chinn, C.A., 2007. Scaffolding and Achievement in Problem‐Based and Inquiry Learning: A Response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), pp.99–107. Kebritchi, M. & Hirumi, A., 2008. Examining the pedagogical foundations of modern educational computer games. Computers & Education, 51(4), pp.1729–1743. Kirlappos, I., Beautement, A. & Sasse, M.A., 2013. ‘Comply or Die’ is dead: Long live security‐aware principal agents. In Workshop on Usable Security. Ruskov, M., Celdran, J.M., et al., 2013. Unlocking the Next Level of Crime Prevention: Development of a Game Prototype to Teach the Conjunction of Criminal Opportunity. Information Technologies and Control, 8(1). Ruskov, M., Ekblom, P. & Sasse, M.A., 2013. Getting Users Smart Quick: Results from 90 Minutes of Using a Persuasive Toolkit for Information Security Learning by Non‐Professionals. In SECRYPT 2013. Sugrue, B., 1995. A Theory‐Based Framework for Assessing Domain‐Specific Problem‐Solving Ability. Educational Measurement: Issues and Practice, 14(3), pp.29–35. Webe r, R.P., 1990. Basic Content Analysis (Quantitative Applications in the Social Sciences) 2 Sub., Sage Publications, Inc. Available at: http://www.amazon.com/exe℅bidos/redirect?tag=citeulike07‐20&path=ASIN/0803938632

729

The Influence of Digital Games on Learning Reading: A Closer Look Mas Idayu Md Sabri1, Peter Blanchfield2 and Gail Hopkins2 1 University of Malaya, Kuala Lumpur, Malaysia 2 University of Nottingham, Nottingham, United Kingdom masidayu@um.edu.my pszpxb@exmail.nottingham.ac.uk Gail.Hopkins@nottingham.ac.uk Abstract: It has been identified that not all students learn effectively during class lessons. Different learning styles call for different approaches but limitations in a classroom setting restricts efficient in‐class teaching and learning. It has also been observed that individual attention given to struggling students does influence and increase learning activities and interest in a subject, hence the existence of teaching assistants (TAs). The question addressed in this paper is ‘would a computer‐ based intervention give the same effect as an individual tuition but without the need for human resource?’ An example of such software is Nessy™, a learning system that has won awards for its digital teaching approach. However, some groups benefit more from kinaesthetic teaching so this aspect should be taken into consideration when choosing a teaching strategy. This paper will investigate the effects of using Nessy™ on its own, Nessy™ with individual support, and individual tuition without the use of Nessy™. It also discusses the design and development of a digital kinaesthetic game to support struggling readers and study the effects of its use with and without individual support. So far, the study carried out has used Nessy™ with individual support as the intervention for 30 minutes a week over a 10 week period. For the initial study the researcher guided the participants when using the digital game. The data obtained from the pre and post tests showed greater improvement in reading for every participant when compared to students of the same class who were not given the intervention. The second study was a cross‐intervention between Nessy™ without individual support and the use of individual tuition without Nessy™. Two groups of participants went through each intervention for 6 weeks each for 30 minutes per week. The pre and post tests showed that both groups, after going through the personal tutoring, had a higher improvement in reading than when they had gone through the digital game intervention alone. To conclude, the computer‐based intervention under proper supervision does help struggling readers and has been shown to give better results compared to classroom lessons, individual tuition and unguided computer‐based intervention. The results of this study will become the design basis of the kinaesthetic game to be developed for the next study. If computer‐based intervention is proven to be beneficial, it is expected that a digital kinaesthetic game would improve learning even more in kinaesthetic learners. Keywords: digital games, game‐based learning, kinaesthetic learning, literacy, reading

1. Introduction According to the UK’s primary school league table released in 2011 by the Department for Education, 1310 primary schools in the country fell below the expected standards (Vasagar 2012). That is one‐in‐ten schools failing the level set by the department (Paton 2011). The national target literacy is Level 4 by which, at the end of primary school, students are expected to obtain certain skills. In reading they are expected to respond to a range of texts, show understanding of significant ideas, themes, events, and characters, begin to use inference and deduction, refer to text when explaining their views, and use ideas and information (source from Department of Education, UK). A report published by the National Literacy Trust states that 22.2% of young people aged eight to sixteen enjoyed reading very much but 10.2% said they do not enjoy reading at all. And among those who do read, technology based materials are the most frequently read, such as websites, blogs, and networking websites, which are read every week (Jama and Dugdale 2012). Studies have shown that digital games prove to be a method of learning which is engaging as well as beneficial. It is said to increase motor skills as well as cognitive skills (Prensky 2003). Games have been the basis of much learning software that is available in the market for instance Nessy™ Learning System, WordShark™, and the Carmen Santiago Series. Even so, such games are not without flaw and many problems have also been discovered with game‐based learning. For instance, games that are designed to be playable but are too easy and not challenging will lose the interest of the players. On the other hand, a game that is too complicated will result in users quitting the game out of frustration (Gee 2003). So developers struggle to design games that are challenging yet able to hold the users’ interest and encourage long and repetitive play.

730

Mas Idayu Md Sabri, Peter Blanchfield and Gail Hopkins

2. Background The aim of this research was to investigate the effectiveness of a current learning intervention. The outcome from this research will drive the design for a more effective game‐based learning environment which would incorporate kinaesthetic learning. This kinaesthetic digital learning game is to supplement student learning at home or in schools. Kinaesthetic games are believed to be able to help improve literacy in primary school students due to the immersive and multi‐sensory teaching approach when presented with related educational contents. Multisensory teaching and learning has been around since the 1960s as introduced by John Locke. He introduced the movement that ‘one learns from experience’ which is the root of today’s ‘learning by doing’ principle (Zuckerman 2006). Locke believed that children should be allowed to satisfy their curiosity by learning through the means of games and toys. Playing with toys will initiate interaction with the given object and this experience will be interpreted by the mind as an idea which is learnt and retained (Locke 1968). In this technology driven era most children and especially boys are more inclined to play digital games than read a book and this has caused less interest in reading (Jama and Dugdale 2012). This has driven game designers to focus on developing educational games due to the need for a learning supplement that will attract gamers as well as educators. Currently, interventions used in schools to overcome illiteracy are the use of digital learning methods (learning software, online tutorials), individual tutorials given by an adult (teacher, parent, teaching assistant), and extra guidance in the form of additional lessons or activities. Schools have also acknowledged the benefits of multisensory teaching approach and more activities are focused on including this method of learning. According to Kim (2012), every person has a different learning style where one mode would be the dominant one for the learner. Even so, most would learn best when a combination of all three modes (Audio‐Visual‐ Kinaestethic) are presented during the teaching process. While there are numerous digital learning games available for use in schools, teachers are more comfortable including physical activities when it comes to kinaesthetic learning due to the limited resources for digital kinaesthetic games available that could be used to blend in with current teaching content. For example, to learn phonics, students would shape out the forms of letters using modelling clay, and then associate the shape with the name and the phonics of the letter. Or teachers would encourage more active participation from students by teaching them certain actions to relate to the phonics of the letter. For instance teachers in one local school teach phonics actions like weaving hands in an S shape, like a snake, and saying ‘sssss’ to teach the sound of the letter S. What is observed is that teacher participation in this teaching method is essential and it might not be possible to give adequate individual attention to all students in a class during lessons. A more independent digital solution would be highly desirable to overcome the shortage of teaching assistants but to what extent is the teacher’s guidance indispensable in the learning process? This study compares the effectiveness and benefits of the different interventions used. The 3 interventions in this study are the use of a digital learning game with supervision, the use of a digital learning game without supervision, and personal tutoring without any digital support.

2.1 Nessy™ as the chosen digital intervention While the full study is focused on kinaesthetic learning, at the time of the experiment there are no suitable digital kinaesthetic game for reading available in the market, apart from one in Mandarin (www.little‐ prince.com.hk) which utilises the benefits of Kinect™ (Hsu 2011). Even though not entirely promoting kinaesthetic learning, Nessy™ was chosen because it is available commercially and is the winner of the Education Resources Award 2010 in the special educational needs ICT category. This software was developed in cooperation with Bristol Dyslexia Centre to cater for dyslexics but can also be used by all children. Nessy™ is a learning system that aims to help students with dyslexia learn to read and spell as well as promote independent learning. The software utilizes the visual‐audio‐kinaesthetic‐tactile (VAKT) model where colourful graphics, sound effects and music, and both keyboard and mouse interaction are available to present the students with the different learning mediums. Based on the positive review displayed on its website (www.nessy.com), the Nessy™ Learning Program has helped many children, whether dyslexic or not, to improve their literacy.

731

Mas Idayu Md Sabri, Peter Blanchfield and Gail Hopkins There are other examples of educational reading software commercially available such as Jumpstart Reading Blaster, and Reader Rabbit. Both of these offer a fun gaming or game‐like learning environment to learn reading. Even so, Nessy™ was the obvious choice because its learning structure follows the UK National Curriculum closely. Since the experiments were carried out in UK schools, Nessy™ would complement the classroom teaching.

Figure 1: Jumpstart Reading Blaster

Figure 2: Reader Rabbit

3. Methods The experiments were conducted on different groups of students but the participants were from the same school and of the same age (6 to 7 years). The Year Two consists of students of mixed gender and ethnic background. This class is facilitated by one class teacher, one teaching assistant, and the occasional adult/parent. Participants were selected based on their initial assessment of the National Curriculum Level (NCL) at the start of the school year. Most of these students were below the average class level. For each experiment, the participants went through the selected intervention for 30 minutes a week over a certain period. They also participated in their daily classroom lessons along with other students, so the interventions acted only as a supplement.

3.1 Pre and post test method 2 types of tests were used to determine participants’ progress throughout the experiment. These tests were given at different stages of the experiment (initial, midway, and end). The National Curriculum Levelling tests were administered by the class teacher and the Nessy™ Reading Challenges were overseen by the researcher. 3.1.1 National curriculum level and average point score (APS) At Key Stages 1, 2, and 3, the National Curriculum ranges from level 1 to 8, where level 1 describes the achievement expected of a 5 year old and level 8 of a 14 year old. These are used to measure a child's progress compared to pupils of the same age across the country. Any levels below level 1 are considered a P level. Each level is split into sub‐levels ‘a’, ‘b’ and ‘c’. So an average student in Year 2 would be between level 2a and 2c. Within each sub‐level, it is further divided into a ‘+’ and ‘non‐plus’ where an a+ indicates a point score higher than the non‐plus. Table 1 below describes the meaning of each sub‐level. Table 1: Sub‐levels in the national curriculum level Sub‐level Description A the child has reached the top of the level and is working towards the next level B working well within the level C the child has started to work at the level

An average point score is the scoring method used to categorize the students into their NCL. A 3.1.2 Nessy™ reading challenge (NRC) test This challenge consisted of a simple listen‐and‐click test where a list of 3 words was displayed on the screen and the student had to select the correct word that was played. There were 10 groups of 10 words. The test was terminated when the student obtained a total of 5 incorrect responses. The reading level was determined by the stage the test was terminated. There are a total of 10 levels. Figure 1 shows the screen for the NRC.

732

Mas Idayu Md Sabri, Peter Blanchfield and Gail Hopkins

Figure 3: Nessy™ reading challenge screen

3.2 Experiments 3.2.1 Nessy™ with supervision This research had two approaches which were an ethnography study and an experimental study. This study was carried out for approximately 6 months. Figure 2 depicts the research model for this study. For the first 6 weeks no intervention was applied and the researcher came into the class as an observer. The researcher also participated as an assissting adult during the reading exercises. This gave the participants an opportunity to be familiar and comfortable with the researcher to lessen the Hawthorne effect (Monahan and Fisher 2010). Five key areas were observed during the observation phase. They were i)teaching and learning techniques, ii) physical setting, iii) reading skill, iv) student behaviour, and v) interaction with Nessy™.

Figure 4: Nessy™ with supervision research model In this initial experiment, Nessy™ along with adult supervision was implemented as the intervention. Participants were given guidance where the researcher would teach the participant the list of words used in the Nessy™ games. Participants were also given help on instructions and navigation as well as the selection of content and games. The researcher would also guide the participants through difficult levels. Each participant accumulated a total of 5 hours of intervention by the end of the experiment. Participants were given a test after the observation phase to identify initial reading level. Another test was administered after 5 weeks of intervention to determine their mid‐progress and another test after 10 weeks. A final test was given 3 weeks after the last intervention. This final test was to determine any carry‐over effect from the intervention that might continue to affect their learning progress.

733

Mas Idayu Md Sabri, Peter Blanchfield and Gail Hopkins 3.2.2 Nessy™ without supervision and personal tuition (cross‐experiment) The second experiment was conducted by a different group of participants but also from the same school and level. Selection of participants were made to closely match the previous group where the participants were also from the lower end of the reading spectrum in the class for that year. However, for this experiment the class teacher selected only 5 students to participate as opposed to the previous 8. 2 types of intervention were implemented in this second study. The first one was the use of Nessy™ without any supervision. For this second study, the use of Nessy™ did not include any direct supervision from the researcher. The researcher was only there to help the participants navigate through the system. Participants were able to choose whichever game they liked. Even so, the researcher would help the participants select the appropriate level of content. The second intervention was personal tutoring without any computer software support. Participants were tutored individually by the researcher. The sets of words taught were the same as the words used in Nessy™. The print‐outs provided by Nessy™ were used as the content and teaching guideline. Examples of teaching activities are:

Going through a list of words for a specific level. Teaching the participants how to sound the words and how each word in the list should sound similar. Example (cvc – cat, bat, hat, sat).

Circling the correct word from a choice of three that describes the picture shown.

Finding a group of words that rhyme.

Finding words in a scrambled puzzle.

Randomly picking words from a bag of cards and reading out the words. If the participant gets it right, he or she gets to keep the card.

Finding a word from a collection of cards and matching it with the correct picture on the print‐out.

For this second study, a cross‐experimental approach has been implemented. Participants were divided into two groups (group A and group B). Figure 3 depicts the research model used for this study where each group were given 6 weeks to go through each intervention alternately. The reason for a cross intervention is that the researcher wanted the participants to be given the opportunity to experience both interventions. The researcher also wanted to observe and answer many questions pertaining to learning behaviour.

How would switching between the two interventions affect their learning curve?

Would their learning increase at a steady rate or drop?

Would a certain type of intervention facilitate better learning when compared?

Unlike the initial study, the second study had no formal observation phase at the start and the end of the experiment. Even so, the researcher still took notes of the participants’ behaviours throughout the experiment. Only 3 tests were administered which were at the start, before the cross‐over, and at the end.

Figure 5: Nessy™ without supervision and personal tutoring (cross‐experiment) research model

734

Mas Idayu Md Sabri, Peter Blanchfield and Gail Hopkins

4. Results 4.1 Nessy™ with supervision 4.1.1 Observation findings Among the relevant and important findings during the observation were that the learning approach in reading adopted in this class were to recite and repeat phonics and new words everyday as a memorization method to ensure familiarity with the newly acquired knowledge. Students went through a group lesson as well as being given individual time with the teacher or an adult in the classroom. Most of the time the students were given the freedom to decide the direction and content of their activities (book selection, general topic essays). The teacher provided many other learning aids such as blocks, puzzles, moulding clay, etc, which were sometimes used in lessons to give a variety in teaching methods as well as a more hands‐on (kinaesthetic) learning alternatives. As for the physical setting, the materials put up all over the classroom were intended to inspire and guide the students in their lessons. The classroom setting was very lively and creative. Students’ works were displayed for others to see and this served as recognition to the students. The range of reading skills in the class is quite wide (from readers at level 3C down to those at level W/P8). Each reading groups (level) was given different set of books to read from, and different set of tests. Students who were struggling readers would sound out the words using ‘fred talk’ to help them read the words. This is a successful method for most. Some are able to sound out the words but read out an entirely different word. For example ‘b‐o‐r‐e‐d’ becomes ‘bread’. Students in this class were always seeking the teacher’s or an adult’s approval. They would often show their work to these people in exchange for praise and constructive feedback. They prefer to decide what to read and not follow the guideline on the books that are assigned to them. They liked to explore other options and challenge themselves. Sometimes the participants refused to play the Nessy™ game chosen for them because they wanted something more challenging or different. Nessy™ promotes independent play but the flow of activities is quite confusing so adult guidance is preferably needed to help the users. Navigation is not structured. Users were able to browse whichever lesson/game they wanted without having adequate proficiency at previous levels. This is not a digital teaching software. There are only minimal teaching modules/elements. All games are for practice. Some games do not promote reading but focus more on memorizing. ‘Jigsore’ (figure 4) does not require the users to read the words, they can just remember the placement of letters in order to play the game. Some participants showed a decline in interest at playing Nessy™ as time passes. This is evident in the group of lower than average students. Students who demonstrated increased reading proficiency remained very interested in continuing to use the game.

Figure 6: Jigsore game in Nessy™

735

Mas Idayu Md Sabri, Peter Blanchfield and Gail Hopkins 4.1.2 National curriculum level and Nessy™ reading challenge score NCL assessment administered by the class teacher was obtained to be used as a reference to validate the progress of the participants in this study. The assessment was made at the start and at the end of the school year. In the study, these results were used to compare the progress between the experimental group (who used the intervention) and the control group (who were not introduced to the intervention). Based on figure 5, the experimental group who were mostly comprised of the bottom achievers in the class, has shown progress in their reading ability. The control group has also improved their reading. The regression line for both groups indicates the rate of change in their progress and it is noticeable that the experiment group has improved more than the control group. Also, although the control group has students starting out at a higher skill level, the figure shows that the participants in the experiment group have progressed to a higher point in their individual reading skill.

Figure 7: NCL assessment for initial experiment The NRC scores were calculated by the Nessy™ system. These scores show the progress the students made in their reading skill as well as their skill at using the computerized teaching software. As shown in figure 6, the participants’ improvement from the start to the end of the experiment were quite significant. The highest possible level for the NRC is level 10 which 2 participants managed to reach by the end of the duration. NRC 4 indicates the score the participants obtained after the absence of the intervention. Here it can be said that only a few participants did not show any progress (not including those at level 10) whereas the rest still showed improvement.

Figure 8: NRC score for initial experiment

736

Mas Idayu Md Sabri, Peter Blanchfield and Gail Hopkins

4.2 Nessy™ without supervision and personal tuition 4.2.1 Observation findings Even though no formal observation phase was allocated in this second study, the researcher still took note of some relevant findings throughout the study. Participants who were using Nessy™ were given the freedom to choose whichever game they wanted to play provided that it was within the appropriate level. Some participants were quick to abandon the more difficult games and opted to play the less challenging ones which were below their reading level. Sometimes, when left unsupervised, the participants rushed through a game without much effort made to read or think of the right answer. This was more apparent when the researcher selected the games that required more in depth cognitive skills from the participants. Participants from the personal tutoring group would always ask to use the computer. But as the lesson progressed and the researcher made the teaching approach varied they started to enjoy and concentrate on the lesson at hand. The personal tutoring group preferred lessons that made use of cards (active learning – pick cards, arrange cards, collect cards). The lessons that had them circle or write down the answers on paper they seemed to enjoy less. 4.2.2 National curriculum level and Nessy™ reading challenge score In this second study, NCL assessment was only able to determine the overall progress of the participants without being able to differentiate the progress made between different interventions (Nessy™ without supervision and personal tutoring). Based on figure 7, the rate of improvement depicted by the regression lines for both groups is similar. Both groups had progress at the same rate. No noticeable difference can be said about the progress of both groups. Even so, it is believed that it would have been possible for the experiment group to achieve lower progress if they were not given any form of intervention at all. Given that the participants in the experiment group we all from the lower achievers in the class, their progress is comparatively at par with those from the higher achievers.

Figure 9: NCL assessment for second experiment The NRC scores were able to track the different progress made by different types of intervention implemented by the different groups as shown in figure 8. Participants in group B (P1, P2) were given personal tutoring at the start of the experiment and those in group A (P3, P4, P5) were given Nessy™ to use on their own. Both participants from group B (personal tutoring) have increased in their NRC score even though they did not use Nessy™ for that phase. On the other hand, 2 out of 3 in group A who had used Nessy™ on a weekly basis did not show any progress in their NRC score.

737

Mas Idayu Md Sabri, Peter Blanchfield and Gail Hopkins

Figure 10: NRC score for second experiment

5. Summary of findings Overall, all participants whether from the initial study or the second study had shown progress in their reading skill. According to their NCL assessment, participants who have used Nessy™ along with supervision had the most noticeable improvement when compared to the control group (without any intervention), those who used Nessy™ without supervision, and those on personal tutoring. However, the significance level of this data is insufficient for making definite conclusions. The standard errors for the regression lines for the initial experiment (figure 5) are 1.4 for the control group and 0.7 for the experimental group. The standard errors for the second study (figure 7) are 1.5 for the control group and 0.5 for the experimental group. Since the number of participants in this study is quite small, these numbers do not give a conclusive outcome but the findings are still quite relevant to the study. The NRC scores obtained by the participants further validates the claim that using Nessy™ with supervision is the more effective intervention. On average, the participants who used Nessy™ with supervision obtained an increase of 1.4 level. On the other hand, participants who did not have supervision while using Nessy™ only obtained a 0.4 increase in level and participants who went through personal tutoring obtained a 0.6 increase. What can be said here is that using Nessy™ is more beneficial when used with supervision. Using Nessy™ alone is even less beneficial than getting personal tutoring without any digital support.

6. Discussion and conclusion In this experiment, digital intervention with supervision was observably better than any of the other approaches which were traditional classroom lesson alone, digital intervention without supervision, and personal tutoring. Students were more focused and enthusiastic during the learning process when supervision was available. This was probably due to the constant recognition and positive reinforcement given by the adult. Navigation and game exploration problems were easily resolved with the adult’s help whereas those without any supervision spent more time figuring out the flow of the game via trial and error. This finding poses a slight challenge to designing and developing a truly supervision‐independent digital kinaesthetic learning game that would be efficient in helping struggling readers. Even so, the advantage of digital learning games is that teachers are still able to monitor the progress of students without being physically present when the learning process takes place. A question that may be suggested is ‘What game design would facilitate independent literacy learning effectively?’ Even though a teacher’s guidance is much preferred, there will be situations where it would not be possible to get personal supervision for each student. A digital learning game that is self‐sufficient would be very beneficial in this situation. Also, based on current teaching methods, assimilating kinaesthetic in the teaching approach would result in a more efficient learning process for the students. Adapting classroom kinaesthetic lessons into a digital game would hopefully help teachers allocate their attention more equally among students. The continuation from this research would be to design a digital kinaesthetic learning game to teach reading that fulfils the requirements based on the findings in this paper. The game should also focus on independent learning as this is the main issue in current digital learning systems.

738

Mas Idayu Md Sabri, Peter Blanchfield and Gail Hopkins

Acknowledgements The first author is a PhD student who is sponsored by the Ministry of Higher Education of Malaysia. This study was carried out at Southwold Primary School, Nottingham. The author would like to thank Mrs Sally Holmes, class teacher of Year Two, for her help and cooperation during the research.

References Gee, J. P. (2003). "What video games have to teach us about learning and literacy." ACM Computers in Entertainment 1(1). Gee, J. P. (2005). "Learning by design: good games as learning machines." E‐Learning 2(1). Hsu, H.‐M. J. (2011). "The Potential of Kinect in Education." International Journal of Information and Education Technologies 1(5): 365‐370. Jama, D. and G. Dugdale (2012). Literacy: State of the nation ‐ A picture of literacy in UK today, National Literacy Trust. Kim, M., J. Hwang, et al. (2012). Evaluating Multisensory Learning System for Teaching English Intonation. Human Centric Technology and Service in Smart Space. Springer Netherlands. 182: 43‐50. Locke, J. (1968). International Encyclopedia of the Social Sciences. Monahan, T., & Fisher, J. A. (2010). Benefits of ‘observer effects’: lessons from the field. Qualitative Research, 10(3), 357‐ 376. Paton, G. (2011). "Primary school league tables: one‐in‐10 schools `failing'." Retrieved 25 September 2012, from http://www.telegraph.co.uk/education/primaryeducation/8956946/Primary‐school‐league‐tables‐one‐in‐10‐schools‐ failing.html. Prensky, M. (2003). Digital game‐based learning. Computers in Entertainment (CIE), 1(1), 21‐21. Vasagar, J. (2012). "Raise literacy target in primary schools, says Ofsted chief." Retrieved 25 September 2012, from http://www.guardian.co.uk/education/2012/mar/15/raise‐literacy‐target‐schools‐ofsted‐chief. Zuckerman, O. (2006) Historical Overview and Classification of Traditional and Digital Learning Objects.

739

The Mediatization of Digital Games for Learning – a Dual Rub‐Off Effect Helga Sigurdardottir and Robin Munkvold Department of Media Technology, Nord‐Trøndelag University College, Steinkjer, Norway Department of Interdisciplinary Studies of Culture, Norwegian University of Science and Technology, Trondheim, Norway helga.sigurdardottir@hint.no robin.munkvold@hint.no Abstract: This paper analyses the contrasting ideas expressed in the Norwegian popular media discourse in 2012, regarding digital games and how the indirect mediatization of digital games can play a vital role in the success of the relatively new field of digital game based learning. The paper reveals how the connection to educational context seems to be a defining factor regarding whether or not, or to which degree, the coverage in each case is positive or critical towards digital games. Three categories of positive arguments for digital games in a learning context, as expressed through the media discourse over the past 3 years are brought to light, as well as two categories of critical arguments. Analysis of the arguments discloses a certain concordance between the seemingly contradicting arguments, that make the purpose of playing, as either educational or entertainment, a key factor. As a fairly new approach to education digital game‐based learning has yet to fully earn the publics’ trust and this paper analyses how the Norwegian printed media conveys the image of this fresh new field to its readers. The paper reveals a “dual rub‐off effect” between the indirect mediatization of digital games for entertainment on one hand and learning on the other. Digital games for learning are generally portrayed in positive ways, based on the learning potentials they have to offer. However, the critical views towards digital games for entertainment seems to “rub off” onto games for learning, as can be seen by mentions of the possible damaging effects, even in articles that are otherwise very positive towards games for learning. At the same time games for entertainment are generally met with scepticism, but the positive view towards digital games in a learning context also seems to influence peoples’ notions of the learning potentials of digital entertainment games. Keywords: digital game‐based learning, mediatization, games as a learning tool, contrasting views on digital games

1. Introduction Although experiments with digital games for learning are nearly as old as digital games themselves, the constructive use of digital games as a learning tool within formal education is fairly new. The growing field of digital game‐based learning has been emerging in the past two decades or so. However, as a learning technology, digital games have to deal with the widespread skepticism aimed at digital games for entertainment. Most of us are aware of the threats that digital games allegedly pose to our society. We have heard discussions on the radio, seen it on TV, read about it on the internet, in our newspapers, etc. Many have also participated in discussions in the lunch break or at social gatherings. Some have heard first hand narrations of how badly excessive digital game play has effected someone we know, or someone close to someone we know – and some of us have even encountered challenges ourselves, that we trace to our own excessive use of digital games. And while many of us may admit that digital games for learning may have some potential, we remain skeptical to our children and young adults being exposed to something so harmful in school or other educational institutions. It is not the purpose of this paper to disprove the notion that digital games can be harmful. It is, however, important to put some of the key elements of the discourse into perspective. The paper is based on a media analysis on the printed Norwegian media discourse on digital games in a learning context, over the last 3 years. We have seen that there is a difference in how digital games are discussed depending on whether or not they are tied to an educational context. Thus we seek answers to the questions of how the connection to an educational context affects the media discourse on digital games and what may cause this difference. To do so we will shed a light on some concepts regarding the message from the media, building on the concept of mediatization, and furthermore paying attention to the meaning of contrasting views for a growing field as well as the concept of media panic. We reveal in which ways digital games are presented in a learning context, to the readers of Norwegian newspapers. We draw forward the positive arguments as well as the critical ones and sort them into categories, according to their nature, to reveal the central elements of the discourse. In this

740

Helga Sigurdardottir and Robin Munkvold paper we do not distinguish between games that are designed for education and entertainment games that are sometimes also used for educational purposes.

2. Defining concepts The word mediatization refers to how media is not only an influencing agent to society, but is in itself an integrated part of society. Social sciences have often treated media as something separate from society, focusing on how media affects society and culture. However, through the concept mediatization, we examine how media is “omnipresent” in society and how it has contributed to the changed character, function and structure of social and cultural institutions and processes (Hjarvard, 2008; p. 105 – 106). Mediatization can be divided into direct and indirect mediatization. Direct mediatization refers to the process when something that we know as a non‐mediated activity transforms into a mediated form. Indirect mediatization refers to how something in society gets shaped by the way media presents it. As an example of direct mediatization, Stig Hjarvard (2008) takes an example of the game of chess. Formerly known as a board game, demanding physical presence, chess is now increasingly known and played digitally. As an example of indirect mediatization Hjarvard names the knowledge of the Danish nation about USA. Most Danes get the majority of their knowledge about USA from media. Indirect mediatization is therefore merely a different form of mediatization, and no less influential than direct mediatization (Hjarvard, 2008, p. 114 – 115). As the analysis on the recent media discourse will reveal, digital games for learning can be seen as a subject of both direct and indirect mediatization. Tensions in society are often reflected through disagreements and controversies in science and technology. The media discourse thus reflects the scientific disputes tied to game studies and digital game based learning, as the discourse within professional environments and the media discourse are interconnected. Scientific disputes represent a way of sustaining certain social norms and values, as well as maintaining certain political boundaries. At the same time, those contrasting views often serve as a vehicle of change, particularly in emerging and fast developing fields or disciplines (Hess, 1997, Nelkin, 1992, Martin, 2008, Sismondo, 2010). Since digital game‐based learning is a fairly new and highly disputed field, looking into the current controversies of the discourse seems relevant. When writing about the recent media discourse on digital games, mentioning the so‐called media panic seems inevitable. Whenever a new type of media appears and people become afraid of it, it has been referred to as media panic. Since children and young people often adapt quicker to new media than older generations, a media panic is often characterized by fear for young peoples’ wellbeing, and attempts to influence their behavior so that the new medium won’t “pollute” their minds. Not uncommonly, the “old” media (that is, preexisting forms of media the new media may be considered to compete with in one way or the other) fuel the fear with frightening tales from reality (Marwick, 2008; Spilde, 2010; Børsum, 2012). As mentioned above most of us can relate to digital games being mediatized, as the public opinion of digital games is largely based on the views the media conveys. In the following text we will seek to find out how digital games as a learning tool have been mediatized and what sort of views the media is portraying on digital games in an educational context.

3. Collection of data As a source for our research, we chose 5 newspapers from among the most widespread newspapers in Norway. 4 are among the 10 largest nationwide newspapers, that is Aftenposten, Klassekampen, Verdens Gang (VG) and Dagens Næringsliv. Those were chosen out of the 10 largest ones as they represent a variety of views and have, to our knowledge, no common ownership interests between them. I also added the largest local newspaper in the region where we live; Trønder‐Avisa. We chose to search for articles on digital games in an educational context from the past 3 years (from April 2010 to April 2013). We tried out 4 compositions of key words for search. In all cases we used the word digital games (Norwegian “dataspill”), in a combination with the following 4 words:

learning (Norwegian “læring”)

school (Norwegian ”skole”)

741

Helga Sigurdardottir and Robin Munkvold

teaching (Norwegian ”undervisning”)

education (Norwegian ”utdanning”)

As expected several articles came up as a result for more than one of the abovementioned keywords. When those had been sorted out we had a total of 114 newspaper articles. Those were roughly sorted into:

Category A: Articles where digital games in an educational context are the main theme

Category B: Articles where digital games in an educational context are mentioned, but are not the main focus of the article

Category C: Articles where both digital games and educational context are mentioned, but not in any correlation with each other

Category D: Articles that mentioned either digital games or a learning context, but not both

In the first round of analysis the focus has been on categories A and B, as the focus of my research is to analyze the media discourse on digital games in an educational context. The articles in the C category will, at this point, only be briefly used for comparison, whereas D will be viewed as a search engine mistake, not to be analyzed further. Out of the 114 articles, 11 turned out to have digital game based learning in one form or the other as a main subject and thus belong to category A. 29 turned out to have digital game based learning as a secondary subject; that is, category B.

3.1 Counting positive, neutral and negative mentions The first round of analysis revealed that in each and every one of the articles in category A, with digital game based learning as a main theme, the educational properties of digital games are praised, to various degrees. The games are both educational and commercial and in all the cases the discourse about digital games for learning is distinctively positive. One might say that out of the 11 articles, 7 portray digital games for learning as something entirely positive, without any critical views. Two of those articles are reviews of edutainment games for children (a total of about 20 games), and even in the cases where certain educational games get a poor rating, for inadequate educational properties or low entertainment value for instance, the discourse on digital games for learning and the potentials it proposes is still highly positive. In 3 of the articles, digital games are praised as a learning instrument, while some skeptical or critical views are mentioned. In 1 article the main emphasis is on the positive effects of digital games for learning, whereas an undertone of skepticism is also clearly present. In the articles in category B, where games are not the main theme but linked to school / learning in some way in the text, the discourse has been sorted into three categories; positive, neutral and negative. Although the articles in this category do not focus on digital game based learning as a main subject, they reveal a greater variety of views. 14 of the articles count as mostly positive narratives about digital games in a learning context, 4 of them tell about digital games and learning in a fairly neutral way and in 11 cases digital games are largely depicted as a negative influence on learning. It is interesting to note that the type of connection to learning is important. Roughly speaking, when the topic is games for learning, the mention is of a positive nature and when the topic is the effect that commercial games have on students’ performances in school, the mention is usually negative. In the C category, where both digital games and education happened to be mentioned, but without any visible common context, the discourse on games seems to be predominantly negative. In only 12 of those articles games are alluded to in a positive way, in 25 articles there is a neutral depiction of digital games and in 30 articles games are mentioned or discussed in a negative context. It is important to note that in this category the digital games are not specifically educational games or commercial games in an educational context, but digital games in general, unspecified. It is also important to keep in mind that this was the category where the focus was on something unrelated to digital game‐based learning, and most often not even related to either education or digital games.

742

Helga Sigurdardottir and Robin Munkvold

Figure 1: Percentage of positive, neutral and negative mentions in articles from groups A, B and C On the following pages we will further explore the types of both positive and negative mentions. We will explore and describe the trends that occur and reveal what we refer to as a rub‐off effect between the indirect mediatization of digital games for entertainment and digital games for learning.

4. Positive views on digital games for learning The articles reveal that the context in which digital games are discussed seems to matter significantly. Digital games are presented as a positive phenomenon when applied for the purpose of learning, but a largely negative phenomenon in other contexts. But what is it that makes digital games such a positive thing in relation to digital games? To find out about the positive characteristics of digital games, we looked at the arguments made for and around digital games in a learning context. Starting with the 11 articles in category A, there were 69 different mentions of digital games for learning, and in the 29 articles of category B there was a total of 60 mentions. Analyzing the arguments where digital games for learning were favored revealed the following 3 categories of arguments for digital games for learning: Table 1: The types of arguments for digital games for learning and their frequency in the categories of articles Types of arguments The digital society Concrete learning / training Computer games as an vehicle for educational reform

A 15 46 8

B 21 29 10

Total 36 75 18

129

In the following subchapters we will explain the kinds of arguments that the categories above consist of. Our main focus here is on the A‐articles, which provide the richest data on the digital games for learning discourse in particular and we will present examples of how those views are presented.

4.1 The digital society Statements in the first category, The digital society, all refer to the notion that modern society is largely influenced by digitalization, in one way or the other. There are mostly two sorts of arguments within this category. Firstly, there are accounts of how our contemporary society both offers and demands that we explore and apply the digital technology available for a variety of purposes, including education – and how important this is for us. As an example of a statement of this kind we can look at the words of a 19 year old game enthusiast: “When learning extends beyond the four walls of the classroom, one understands what an important tool interactive media ‐ like digital games – are for society” (February, 2013). Secondly, there are accounts of how modern children and young people are already indigenous in the digital world, how they are used playing digital games, and adapted to the modes of learning that digital games provide. Within this angle there is also some concern regarding a generational gap or worries that parents and teachers are not aware of how digital games function nor of what the younger generations are actually doing

743

Helga Sigurdardottir and Robin Munkvold within the digital games. To give an example on how this view is expressed, we quote an article written by two representatives of The Norwegian Media Authority: “Children and young people often indicate that video games, web, mobile and other digital media are an important part of their everyday lives. Children's relaxed attitude to digital media often clashes with the adults and teachers considerably more restrained attitude to the same media. A dialogue between those two groups will increase media competence on both sides and probably give rise to new ideas and teaching methods” (September, 2011).

4.2 Concrete learning / training A majority of the arguments for digital games for learning, a total of 58% out of all the arguments in both the A and B articles, evolved around the concrete learning and training properties of digital games. A small number of the arguments in this category, around 7%, could be considered a subcategory of their own, as they focus on the skills a player has to have to be able to play the game. In each of those accounts we are reminded that becoming good at a particular game requires certain abilities, either individual abilities that the person seems to be born with or to have acquired through life itself, or abilities and skills that can be learned and trained through playing a digital game. A quote from a full time professional gamer makes a good example: “It is exceptionally mentally demanding and requires good planning, coordination, endurance, a clear mind and full focus to be on top” (April, 2013). All the other quotes in this category unanimously tell us that a variety of skills and knowledge can be taught and trained through digital games. Summarizing the listing from only 4 of the A‐articles to give a clue, games are claimed to be of aid in the following subjects; English, Norwegian, social subjects, numbers, letters, first aid, about the universe and planets, black holes and satellites, environmental protection, recycling, rinsing dishes, how to use a PC, better reasoning and decision making skills, increased leadership skills, social skills, better surgery skills, increased capacity to understand the relationship between objects, or as one game developer puts it: “Anything from Egyptian Hieroglyphics to Morse and real history” (April, 2013).

4.3 Computer games as an vehicle for educational reform Quite many of the arguments for digital games inform us that the ways games motivate, engage and, of course, teach students to solve the tasks the game requires, exceed the common contemporary teaching practices of the mainstream school system. There are three approaches to the argument. In one, the focus is on the educators and how they could learn from games. In another one the focus is on the school system as a whole, and how it would benefit from incorporating digital games into the curriculum. Thirdly there are general statements about how much features of digital game‐based learning can teach all of us, both specific features like individualized speed and fitting levels of difficulty, as well as more objective aspects, like joy and engagement. A descriptive statement of this kind comes from a teacher who became an educational game developer to further assist his students in tackling math: “As a learning medium games are superior; they give you continuous feedback, the learning is individualized and it takes place at your own speed” (August, 2012).

5. Critical views on digital games for learning Although critical views are relatively few in the A‐category articles and fewer than the positive views in the B‐ articles, the criticism may, no less than the praise, give an idea as to what is expected of digital games in a learning context. A basic counting of arguments in articles of category A revealed 4 arguments directly presenting a critical view regarding digital games in relation to education. A counting of arguments in category B reveals a total of 48 arguments presenting critical views on the matter. The critical arguments can be divided into 2 main categories:

744

Helga Sigurdardottir and Robin Munkvold Table 2: The types of arguments against digital games in educational context and their frequency in the categories of articles Types of arguments Negative cognitive effects A threat to temporal management

A 2 2

B 22 26

Total 24 28

In some cases the line between arguments regarding temporal management and arguments regarding negative cognitive effects was somewhat fine and arguments of those two sorts often go together. To explain briefly what the above listed categories consist of, let us first look at the second one first, a threat to temporal management. The connection to education in this kind of arguments revolves around digital games that are played outside of school settings as a direct rival of school work. The games also get blamed for getting in the way of other socially acceptable activities, such as participating in sports or social life or spending quality time with friends and family. A quote from a representative for a gaming addiction prevention group makes a very descriptive example: “Some get so deeply involved in the game that they lose interest in other things. They do not go to school, lose interest in doing things with friends or family, they live only in the game” (April, 2013). There is a wide range in the arguments that regard the negative cognitive effects. Although none of the statements contradict the positive claims of direct beneficial effects of digital games for school based skill and knowledge training and learning as listed above, there is a range of statements referring to negative mental, emotional and social cognitive effects, such as increased violent behavior, depression, social isolation and so on. A Norwegian game educator that the newspaper simply refers to as a “game expert” puts it this way: “It is a known effect that violence begets violence” (March, 2013). As with the temporal management arguments the connection to an educational context here is first and foremost that the negative effects of excessive game playing may in one way or the other have a negative children’s’ and young peoples’ performance in school.

6. Mediatized controversies and skepticism From the point of view of media being an integrated part of society we can see how important the contribution of the printed media that is the forcus of this analysis, is to the growth of the digital games for learning, as a medium. Many of the digital games used for learning are themselves a clear example of direct mediatization, turning games and activities we know from “real life” into a digital medium. However, the effect of the indirect mediatization is of a bigger importance in this paper. As the generations of parents and teachers – decision makers – rarely share the inside knowledge of digital games with younger generations, most of their knowledge on digital games stems from media. It is not hard to relate to digital games as a topic of a media “panic” or “media skepticism” as it might also be labeled. The reactions of both media and public to the relatively new medium of digital games bear the signs of a classical media panic, all the way from the “old” and more established media revealing skepticism towards the phenomenon to profound worries about the wellbeing of younger generations. Some of the quotes in the articles we have analyzed in this research are more or less unanimous with quotes from previous media panics, about TV and comic books for instance (Spilde, 2010). The picture that the media currently draws of digital games is, however, a contrasting one. At the same times as we read in the newspapers that games pose serious threats players, we also get to read that our modern society demands games as an educational tool to train a variety of skills and eventually reform the whole education system. The contradictions in the media text reflect the contradictions of experts within the young field of digital game‐based learning. Some scholars praise digital games as the optimal way of teaching and learning nearly any subject while others express great skepticism, usually based on concern for possible negative effects on children and young people – spanning a variety of views between those extremes (See for example Resnick, 2004; Prensky, 2006; Burgan, 2006; Van Eck, 2006; Williamson, 2009). In some cases the contradictions seem very direct, when media tells us that digital games can help to train social skills at the same time as it poses a threat of negative social cognitive effects. However, with a closer look, such different views may, however, be seen as complimentary to each other. In both views the potentials

745

Helga Sigurdardottir and Robin Munkvold of digital games as a powerful learning tool, seem to be acknowledged. Digital games are seen as such a powerful learning agent that they can either make children and young people isolate themselves and refrain from social interaction – or they can be a tool of social empowerment, teaching and training communication skills and even facilitating friendships and contacts that beyond other ways of social interaction. The critical views of digital games in relation to education that have been transmitted to the readers of Norwegian newspapers over the last 3 years by means of indirect mediatization are both of a lesser variety than the positive views, and in fact not so contradictory in their nature. However, readers may still be thoroughly confused. In spite of the wide range in well articulated praise of digital games for learning, the message from the indirect mediatization discourse tells us that digital game play is still disadvantageous for children and young people, unless it is directly applied as a learning tool, within the safety of school settings. Analyzing further, we might say there is a certain dual rub‐off effect between the indirect mediatization of digital games for entertainment and digital games for learning. While digital games for learning seem to be predominantly met with positive views, based on the various, modern learning potentials they have to offer, the skepticism regarding digital games for entertainment also seems to “rub off” onto games for learning. The mentions of the possible damaging effects, even in articles that are otherwise very positive towards games for learning, is evidence of this. Likewise, while games for entertainment are suspected of being of harm to those who play them, the positive faith in digital games in a learning context also seems to affect peoples’ notions of the learning potentials of digital entertainment games.

7. Conclusion Although the first attempts at designing games using computer technology were made more than 60 years ago, we can safely say that digital games in a learning are a fairly young phenomenon. As with any other new medium digital games have been met with a great deal of skepticism, although decision makers seem to be increasingly discovering the positive potentials for games for learning. In this paper we have given an insight into the recent media debate around digital games, focusing on the different perspectives expressed in relation to digital games in an educational context. We have sought to find out how an educational context affects the media discourse on digital games and what may be the cause of this contrast. We have shown that the extent of positive discourse seems to be affected by a connection to an educational context. Digital games in an educational context have been mediatized in a positive way whereas the mediatization of digital games for entertainment draws a gloomier picture. To summarize quickly it can be said that digital games are praised as a learning tool, but met with suspicion when presented as a pass time activity. As a learning tool games are acclaimed for their potential to teach and train skills and knowledge, as a tool that younger generation already are both well acquainted with and have a positive relation to and as a logical answer to modern day demands for up‐to‐date learning methods. The criticism to some degree agrees that digital games have great potentials as a learning tool, yet paying greater attention to how players can learn “wrong” things from the games and how excessive playing may get between a student and her school work. This is what we refer to as a dual rub‐off effect in the indirect mediatization between of entertainment games and educational games. The indirect mediatization of a young field of studies like digital game based learning plays a vital role for its’ social acceptance. The media can both fuel fears about disadvantageous effects of digital game play and open peoples’ minds towards the possibilities that learning through digital games may offer.

References Børsum, T. S. (2012). Når pikslene bestemmer. Kan et dataspill styre en familie? Et studie av konfliktene et problematisk dataspillbruk kan skape we familiegrupper (Master’s thesis). Retrieved from: https://www.duo.uio.no/bitstream/handle/10852/34066/Bxrsum_Master.pdf?sequence=2 Burgan, M. (2006). “In Defense of Lecturing” Change, Nov./Dec. 2006, 38(6), 30–34 Hess, D. J. (1997). Science Studies – An Advanced Introduction. New York University Press, New York Hjarvard, S. (2008). The Mediatization of Society. A Theory of the Media as Agents of Social and Cultural Change, Nordicom Review, 29(2): 105‐134. Martin, B. (2008). “The globalisation of scientific controversy”. Globalization, 7 (1).

746

Helga Sigurdardottir and Robin Munkvold Marwick, A. E. (2008). To catch a predator? The MySpace moral panic. First Monday, 13(6). Nelkin, D. (1992). “Science, technology and political conflict: Analyzing the issues”. In D. Nelkin (ed.), Controversy: Politics of technical decisions (3rd ed.). Sage, Newbury Park Prensky, M. (2006). Don’t Bother Me Mom–I’m Learning! Paragon House Publishers, St. Paul, MN Resnick, M. (2004). Edutainment? No Thanks. I Prefer Playful Learning. Retrieved from: http://www.roboludens.net/Edut_Articoli/Playful_Learning.pdf Sismondo, S. (2010). An introduction to science and technology studies (2nd ed.). Wiley‐Blackwell, Chichester, West Sussex, U.K.; Malden, MA Spilde, I. (2010). Farlig ny verden. Retrieved from: http://www.forskning.no/artikler/2010/august/ 258116 Van Eck, R. (2006). “Digital Game‐Based Learning: It's Not Just the Digital Natives Who Are Restless…” EDUCAUSE Review. 41(2) Williamson, B. (2009). Computer games, schools, and young people. A report for educators on using games for learning. Futurelab. Bristol. Retrieved from: http://archive.futurelab.org.uk/ resources/documents/project_reports/becta/Games_and_Learning_educators_report.pdf

747

Efficacy of Reward Allotment on Children’s Motivation and Learning Zhenhua Xu1, Earl Woodruff1 and Bodong Chen2 1 The Department of Applied Psychology and Human Development, OISE, University of Toronto, Canada 2 The Department of Curriculum, Teaching and Learning, OISE, University of Toronto, Canada zhenh.xu@mail.utoronto.ca earl.woodruff@utoronto.ca bodong.chen@utoronto.ca Abstract: With the recent commercial success and increased use of digital games, the study of motivation has gained new impetus in the field of gaming. Indeed, studies of video games have provided a different route to understanding motivational processes. Recent books by game theorists and researchers have pointed out that the world of gaming has produced a large amount of data illuminating how games enhance self‐directed learning, deepen engagement, and foster important 21st century learning skills (Koster 2005; Annetta 2008; Chatfield 2010; Rupp, et al 2010; Shih et al 2010; Thomas & Ge 2011; Shute & Ke 2012). The present study has a narrow focus on game reward structures and motivation in game‐ play. It explores how a reinforcement schedule sustains children’s motivation in a game context. Specifically, it assesses the effects of reward allotment for an interactive game through the examination of students’ variations in response to different reinforcement schedules. Fifty‐four Chinese children from preschool to grade three were recruited to play a number‐matching game on Sifteo cubes. Two types of reward allotment—25% only, and an escalating 25‐75% reward reinforcement—were examined in this number‐matching game. Overall, this experimental study revealed that both the 25%‐chance‐of‐winning reinforcement schedule and the escalating 25‐75% reinforcement schedule both effectively sustained children’s motivation in the game‐play on Sifteo cubes. Most importantly, however, children showed a higher level of engagement when the reward frequency changed from 25% to 75%. Given that motivation plays a central role in determining how we select and persist in processing information, the present study speculates on how the use of extrinsic st motivation engages students in 21 century knowledge building. Bereiter and Scardamalia (1993) indicate that knowledge building involves the mastery of expert problem solving skills, and that begins with participation in the collaborative process of sharing and distributing expertise. To us, pursuing new ideas in order to push boundaries or to increase one’s expertise may itself be a motivational process. Thus, we need to look for the critical extrinsic factors built into the st knowledge‐building environment and to explore how to appropriately use extrinsic motivation in the development of 21 century skills. st Keywords: motivation, reinforcement schedules, game engagement, Sifteo interactive game, feedback, 21 century skills

1. Introduction In the last couple of decades digital games have rapidly gained popularity amongst children and young adults (Prensky 2002; Koster 2005; Rigby & Przybylski 2009; Chatfield 2010). In the United States, 97% of 12‐to‐17 year‐olds play digital games. By the time they turn 21, these individuals will have spent 9,000 hours playing digital games, compared to 3,000 hours reading books (Peneberg 2010). Globally, 350 million people spend a combined 3 billion hours per week playing video games (Gladwell 2008). This increased participation in digital games has prompted researchers to seek links between instructional strategies, motivational processes, behavior and learning outcomes. Specifically, over the last two decades we have seen a large number of studies focusing on the psycho‐structural aspects of video games in an attempt to examine the motivational appeal with respect to learning (Ryan, Rigby & Prezybylski 2006). Recent books by game theorists, researchers and designers have pointed out that the world of gaming has produced large amounts of data illuminating how st games are deeply engaging and how games foster skills that are important for survival in the 21 century (Koster 2005; Annetta 2008; Chatfield 2010; Rupp, et al 2010; Shih et al. 2010; Thomas & Ge 2011; Shute & Ke 2012). In a recent TED Talk, Tim Chatfield described how the gaming world discovered that changing the reward frequency from 25% to 75% after 15 or 16 trials increased sustained play time. In prior studies we have seen motivation theories (e.g., Bandura, 1977) that are concerned with outcomes and the process that direct behavior toward desired outcomes. We have not seen any further investigations, however, that examine the reward structure and how changing the structure may affect motivational process and outcomes. The reasons for this may be threefold. First, a stereotypical picture of the detrimental effects of extrinsic motivation has long been accepted, based on the vast majority of past research on motivation. Second, there has been a great deal of ambiguity and uncertainty in defining the nature/strength of intrinsic motivation together with questionable measurements (i.e., self‐reporting satisfaction and enjoyment levels) employed for evaluating

748

Zhenhua Xu, Earl Woodruff and Bodong Chen intrinsic motivation, which may act as a hurdle to the evaluation of extrinsic motivation. Third, there is a distinct lack of consensus on the role of the motivational process in a game context. It appears that the meaning of the motivational process is muddled when game play triggers repeated cycles of user judgment (e.g. fun), behavior and feedback (Garris, Ahlers & Driskell 2002). According to a number of theorists (e.g Arnold 1976; Crino & White 1982; Huang 2011), both “fun” and “feedback” have extrinsic motivational functions to intrinsic motivation, so it is possible that these extrinsic functions could be treated as an intrinsic reinforcement.

1.1 Extrinsic motivation In general, extrinsic motivation refers to the engagement or the performance of an activity being directed by obtaining an external reward or to avoid punishment (Cameron & Pierce 1994; Schunk, Pinterish & Meece 2008). If an activity contains a specific goal and this goal provides satisfaction, rather than the actual activity itself, then we could call the behavior in this activity an extrinsically motivated behavior. Factors such as financial incentives, work conditions, feedback, rewards, social recognition, and competition can affect extrinsically motivated behaviors (Lepper, Henderlong & Gingras 1999; Deci & Ryan 1985, 2012). These factors are often treated as reinforcers; moreover, according to Morse and Kelleher (1970), the factors can be defined as reinforcers only when there is a particular response following the event or “there is a subsequent increase in the occurrence of similar responses” (p. 139). Overall, an environmental event such as reward or feedback is a stimulus only when a change in behavior appears.

1.2 Engagement Engagement is a fully absorbing and voluntary state of being (Wang, 2008); it is one of the aspects that explain the motivational appeal of video games. Theorists have identified subjective experience as the core element to the process models of enjoyment and engagement (Vorderer et al. 2004, O’Brien & Tom, 2008, cited in Bolye, et al. 2012). Malone (1981) hypothesized that the appeal of games was largely a function of their ability to evoke challenge, fantasy, and curiosity in players (cited in Rigby & Przybylski, 2009). People play games because they think the process of game playing is engaging (Przybylski, et al. 2009).

1.3 Fun Games are fun (Garris 2002; Przybilski, Rigby & Ryan 2009), and fun motivates voluntary behavior. Fun enables people to take initiative and put forth an effort on whatever they are undertaking without having any resentment. According to Przybilski, Rigby and Ryan (2010), the idea of “having fun” during the game play itself satisfies people’s needs for competence, mastery and autonomy (p.216). Researchers have tried to clarify why people play video games. LeBlanc (2004) used an MDA model for game design analysis and identified eight types of fun in playing video games that includes sensation, fantasy, and narrative (cited in Wang & Sun 2011). In addition, Lazzaro clarified that video games can provide different types of fun‐‐ “hard fun”, “easy fun”, “people fun” and “serious fun” (Wang & Sun 2011; Werbach 2012). In addition, video game reward systems (i.e. points, visual items, choices, feedbacks and animations) can also provide players with fun experiences (Wang & Sun 2011). Indeed, many social games such as World of Warcraft have many multi‐level goals for players to complete in order to master specific skills.

1.4 Research hypotheses The following hypotheses were addressed in this study: 1. The variable reinforcement schedule would lead to a higher perception of fun. 2. The reward scheme would keep children engaged in activities all the time. 3. A change in reward allotment would influence children’s engagement.

2. Method 2.1 Sample A total of 54 Chinese children (23 boys and 31 girls, Mage = 6.9 years, age range: 5‐10 years) participated in this study (see Table 1). They were divided into a control group (10 boys and 16 girls) and an experimental group

749

Zhenhua Xu, Earl Woodruff and Bodong Chen (13 boys and 15 girls). Each child was randomly assigned to one of these two groups. Children in the control group played the number‐matching game with a 25% reinforcement schedule; the children participating in the experimental group played with an escalating 25‐75% reinforcement schedule.

2.2 Sifteo cubes and Number‐matching game 2.2.1 Sifteo cubes Sifteo cubes are motion‐aware blocks (36mm × 36mm × 10mm) with touch‐sensitive screens (see https://www.sifteo.com/) (see Figure 1). Each Sifteo cube can sense its own motion when it is being lifted, tilted shaken, pressed or placed next to another cube. The cubes can also “sense” and “communicate” wirelessly with one another or with a nearby computer (Merrill & Maes 2007; Hunter, Kalanithi & Merrill 2010; Roark 2012). Sifteo cubes are aimed mainly at children (Barbara 2011). Given that Sifteo cubes are tactile in nature, they can create a gaming experience that enhances children’s engagement with digital content by merging physical and digital representations in an interactive environment.

Figure 1: Sifteo cubes‐motion aware blocks (36mm × 36mm × 10mm) 2.2.2 Number‐matching game The number‐matching game (See Figure 2) is a simple video game using Sifteo cubes that have been programmed based on B. F. Skinner’s reinforcement theory. A reinforcement schedule is the precise rule used to present reinforcing factors following a specified type of operant behavior; the rule can be defined by the time or the number of responses required presenting a reinforcer (Catania 1970). Two reinforcement schedules were embedded in the number‐matching game: a 25% chance of getting reinforced from having a winning condition and an escalating 25‐75% chance of having a winning condition. Both reinforcement schedules are fixed ratio schedules. A fixed ratio schedule requires a certain number of operant responses to produce the next reinforcer. For example: in the number‐matching game, the 25% reinforcement schedule means that children can get reinforced after every four responses. The escalating 25‐75% reinforcement schedule followed the idea that the required number of responses may be fixed from one reinforcer to the next. Each reinforcement schedule was tested separately by randomly chosen participants. A total of six Sifteo cubes were used in the number‐matching game, among which there was a master cube to direct a player’s decisions to start or stop the game. A player could also use the master cube to choose a reinforcement schedule. The rest of the five cubes were assigned five Arabic numbers (1, 2, 3, 4, and 5), and each cube could only display one number at a time. In the number‐matching game, each cube would display a randomly chosen number, and two of the five cubes were programmed to display the same number. Once a trial game was finished, the numbers would automatically be shuffled and randomly re‐assigned to the five cubes. 2.2.3 The number‐matching game‐play procedure The first step playing the game is to place the touch‐screen of each cube (except for the master cube) on the table surface. Then the player shakes the master cube once in order to choose a reinforcement schedule. Once the reinforcement schedule is chosen, the master cube is placed on the table and with its touch screen facing up.

750

Zhenhua Xu, Earl Woodruff and Bodong Chen

Figure 2: The Number‐matching game is tested on the Sifteo simulator The second step is to press the touch screen of the master cube to start the game. When the master cube is pressed, a short piece of music starts to play indicating a match. The game player can randomly pick one of five cubes, flip it over and place it on the table with its screen facing up. The player will keep choosing the cubes and flipping them over until he or she finds the number that matches the very first cube that was flipped over. Once a pair of matching numbers has been found, the master cube emits a “beep” – an audio signal to indicate the game player wins the match. At the same time, a small picture of a “smiling face” appears in the lower right‐hand corner of the master cube to indicate the win. Soon after a pair of matching numbers has been found, one round ends and the player can start another round by flipping the five cubes over and keeping the screens facing down on the table. In each round the gameplay sequence remains the same as stated above. When the player finishes playing the game, the master cube records the number of trials being played and demonstrates it at the center of its screen (see figure 3). The number of trials is unlimited, and it is determined by a player’s interest or motivation in the game‐play.

2.3 Design This study consists of two testing sections. The purpose of the first section was to observe each participant’s game‐play state, and it was 10 to 20 minutes in length (Mtime = 10.52, SD = 7.18). During this section, the researcher first did a brief demonstration to teach each participant how to carry out the number‐matching game, and then asked the participant to start his or her game‐play. The second section involved a paper‐and‐ pencil demographics questionnaire and a post‐test questionnaire evaluating the participants’ responses to the two reinforcement schedules. The paper‐and‐pencil demographics questionnaire requested information from the participants about their gender, date of birth, whether they played video games, what kind of video games they played and how often they played. Considering the participants were very young and low levels of literacy, the researcher verbally administered the questionnaires and recorded the participants’ responses to each question. The post‐test questionnaire was a 5‐item instrument; it was administrated on an individual basis to all of the participants in a quiet empty classroom immediately after his or her gameplay. Each item was presented on a five‐point Likert scale (“strongly disagree”, “disagree”, “neutral”, “agree”, and “strongly agree”) and asked the participant to rate his or her level of enjoyment and task persistence. In order to make the rating scale compatible with the participants’ age and their level of understanding of the task they were undertaking, the researcher employed five different cartoon facial expressions to represent the scale levels. The researcher also verbally administered the questionnaires and recorded participants’ responses to each of the questions. Finally, there were two open‐ended, follow‐up questions to request information related to the participants’ perceptions of the number‐matching game using Sifteo cubes and his or her game‐play experience. The questions were: “What did you like the most when you played?” and “Why did you think it was fun?” In addition, apart from a “beep” audio signal indicating a win in that particular round, no external rewards such as verbal praise or prizes were offered to reinforce children’s participation in the game. A participant’s engagement in play was measured based on three aspects: the number of trials a participant played, his or her judgments about whether the game was fun, and whether he or she wanted to continue to play.

751

Zhenhua Xu, Earl Woodruff and Bodong Chen

2.4 Measures Three statistical analyses were conducted in this study to test the three hypotheses.

An independent t‐samples test was to examine whether the average number of trials children played differed between the control group (25%) and the experimental group (25‐75%).

Two ANOVA tests were to test whether children’s weekly video game‐play routine and their developmental stages related to their motivation revealed in the number‐matching game‐play.

An ANCOVA test was conducted to evaluate: a) whether there is a relationship between reinforcement schedules and children’s motivation to play the game and, b) the extent to which children’s motivation related to the reinforcement schedules.

In this study, the evaluation of a child’s engagement in the number‐matching game was based on the number of game trials and the amount of time spent playing the game. The assessment of a child’s motivation level was based on his or her level of enjoyment, willingness and task‐persistence. Thus, the variable (MOTIVATION) is a composite variable of enjoyment, willingness and task‐persistence. However, given that there were approximately 10 missing cells presented in the category of task‐persistence in the SPSS data file, the composite variable (MOTIVATION) only counts 44 cells, thus there were 44 participants involved when analyzing the relationship between motivation and reinforcement schedules. In addition, the variable (i.e., the number of games trials) was the overall number of trials played in the game, rather than by the number of cubes being flipped in each trial. The definition of the variable (i.e., the time spent playing the game) was also based on the overall time a child spent playing the entire game, rather than the time spent to find the matching number in each trial. Table 1: Descriptive statistics for categorical variables (gender, developmental stages, and reward conditions) Developmental Stages Gender Early Childhood (24‐ Mid‐childhood (N = 54) 72 months) (73‐120 months) Boy 9 14 Girl 4 27 Total 13 41

Total 23 31 54

Reward Conditions Control group Experimental (25%) group (25‐ 75%) 10 13 16 15 26 28

Total 23 31 54

3. Results 3.1 Two‐tailed t‐test The two‐tailed t‐test was performed indicating that, on average, children from the experimental group (M = 53.18, SD = 26.64) played more trials than those from the control group (M = 33.88, SD = 42.82) (see Figure 3). However, the difference was not statistically significant t(52) = ‐1.97, p = .051. The 95% confidence interval for the difference in means was large, ranging from ‐38.62 to .03.

Figure 3: The mean value of the number of trials for the control group (a 25% reinforcement condition) and the experimental group (a 25‐75% reinforcement condition)

752

Zhenhua Xu, Earl Woodruff and Bodong Chen

3.2 One‐way ANOVA on motivation The ANOVA test suggested there was no relationship between the motivation revealed in the number‐ matching game‐play and children’s weekly video game‐play routine, F(3, 34) = .53, p = .71. The effect size, partial η2= .06, was statistically small (see Table 2). In terms of the relationship between the children’s developmental stages and their motivation revealed in the number‐matching gameplay (see Table 3), the results from the one‐way ANOVA showed a statistically significant relationship between children’s developmental stages and their motivation in the game‐play F(1, 42) = 8.05, p = .01. The effect size, partial η2 is .16, indicating the children’s developmental stages accounted for 16% of the variance of the dependent variable (MOTIVATION). Table 2: Descriptive statistics for motivation and children’s gaming experience

None Frequency of video game‐play weekly

Rarely Occasionally Usually (once per week) Frequently (2 and more than twice per week)

St.d Error

8 6

2.38 2.50

.44 .89

.28 .32

6 3

2.75 3.00

.69 1.00

.32 .46

2.75

.86

.20

Table 3: Descriptive statistics for motivation and children’s developmental stages

Early childhood (24‐72) months

N 12

M 2.17

SD .54

St.d Error .20

Mid‐childhood (73‐120) months

2.84

.76

.13

Developmental stages

3.3 ANCOVA on motivation The ANCOVA test indicated the strength between reward schedules and motivation was significantly strong, F(1, 41) = 13.84, MSE = 5.71, p = .00, partial η2 = .25. As assessed by the eta square index (η2), the reward schedules accounted for 25% of the variant in children’s motivation in the number‐matching game‐play, holding constant with their age (see Figure 4).

Figure 4: Scatterplot of motivation in the Number‐matching game. Graph showing the differences in motivation between the 25% reward condition and the 25‐75% reward condition

4. Discussion Overall, this study demonstrated that both the 25% only and the 25‐75% chance‐of‐winning reinforcement schedules were effective in motivating children to play the number‐matching game on Sifteos. However, the children further demonstrated a higher level of engagement in the game‐play when the reward frequency changed from 25% to 75%. This finding also provided evidence that when a similar reinforcement is scheduled

753

Zhenhua Xu, Earl Woodruff and Bodong Chen for each of the concurrent responses, the response receiving the higher frequency of reinforcement will increase in rate. Furthermore, the response requiring the least effort will increase in rate, and the response providing the most immediate reinforcement will increase in rate. In the experimental group, children were reinforced by the 25% chance of winning, and after 16 trials, the number‐matching game was automatically changed to the 75% reinforcement schedule. Thus, children had a much higher chance of getting reinforced. Furthermore, since the frequency of getting reinforced after one response is much higher in the 25‐75% reinforcement schedule than in the 25% only schedule, the chances of children getting reinforced could also become much higher. Thus, the efforts to win became less demanding to the children. The overall findings of this study replicated Garris, Ahlers and Driskell’s (2002) statement that a variable payoff schedule in a game can result in greater persistence on the task, and can also improve performance. This study also suggests children’s developmental stages had an effect on their motivation in playing the number‐matching gameplay. Children in their mid‐childhood (age 6 to 10) were more likely to present higher levels of motivation in the game‐play. The results from the post‐test questions also support this finding. A child’s perceptions and attitudes toward a video game have an important influence on his or her willingness and engagement in playing the game. For example, some participants treated the number‐matching game as an opportunity to compete or as a game that was good for their memory and intelligence development‐‐so they wanted to win as many trials as possible. As mentioned in Bolye, Main and Katz’s (2012) article, a player’s motives for playing games provide an alternative perspective on understanding his or her engagement. Thus, we can conclude that a child’s idea of competition or an opportunity for developing memory and intelligence can be treated as motives explaining his or her engagement in the number‐matching game. However, we can also assume that the two reinforcement schedules may not necessarily have been the only motivators that influenced the children’s engagement. On the other hand, children’s cognitive skills may have some potential influence on their engagement in game‐ play. Children who have a higher level of cognitive skill may have a better chance of figuring out how to win the game. Thus, they are more likely to get immediate reinforcement, and their responses are also likely to increase in rate. According to Piaget’s stages of cognitive development, children start to show many forms of logical thought when they are 6 or 7 years old, while children who are in the middle childhood stage (age 6 to 10) are more likely to present a higher level of logical thinking skills (Mcdevitt & Ormrod 2011). This could explain why children’s developmental stages have a strong relationship with motivation.

5. Conclusion and future directions The overall findings in this study indicate that a reinforcement schedule results in greater persistence in the player’s performance. In the present study we hypothesized that if the findings could be replicated in an extremely simple game that does not have the manifold array of additional motivators found in commercially successful video games, we could have a powerfully motivating element that could be used in educational games. In the study we wanted to see if the students would play longer and if they perceived their play as fun. The results confirmed these conjectures. This study reveals how one finding from the gaming world can be used to improve the efficacy of educational games. Gaming is now helping to advance our understanding of educational games well beyond the work of Thomas Malone (1980). Given the powerful motivating effects that commercial gaming has demonstrated, we believe more study is required on how extrinsic motivators can be used to increase student’s intrinsic perceptions of fun and the desire to learn.

References Annetta, L. A. (2008) Video Games in Education: Why They Should Be Used and How They Are Being Used. Theory Into Practice, Vol. 47, pp 229‐239. Arnold, H. J. (1976) Effects of Performance Feedback and Extrinsic Reward Upon High Intrinsic Motivation, Organizational Behavior and Human Performance, Vol. 17, pp 275‐288. Barbara, O. (2011) Sifteo Cubes Show Promise for New Game [online] http://search.proquest.com.myaccess.library.utoronto.ca/docview/886037328/139A2684C8F4FD6B79B/1?accountid =14771

754

Zhenhua Xu, Earl Woodruff and Bodong Chen Bereiter, C. and Scardamalia, M. (1993) Surpassing Ourselves: An in Inquiry into the Nature and Implications of Expertise, Open Court, Chicago. Boyle, E. A., Connolly, T. M., Hainey, T. and Boyle, J. M. (2012) Engagement in Digital Entertainment Games: A Systematic Review, Computers in Human Behavior, Vol. 28, pp 771‐780. Cameron, J. and Pierce, W. D. (1994) Reinforcement, Reward, and Intrinsic Motivation: A Meta‐analysis. Review of Educational Research, Vol. 64, No. 3, pp 363‐423. Catania, A. C. (ed.) 1970 Reinforcement Schedules and Psychophysical Judgments: A Study of Some Temporal Properties of Behavior, the Theory of Reinforcement Schedule, Meredith, New York. Chatfield, T. (2010) Fun, Inc.: Why Games are the 21st Century’s Most Serious Business, January, The Guardian. Crino, M. D. and White, M. C. (1982) Feedback Effects in Intrinsic/Extrinsic Reward Paradigms, Journal of Management, Vol. 8, No. 2, pp 95‐108. Deci, E. L. and Ryan, R. M. (ed.) 2012 Motivation, Personality, and Development Within Embedded Social Contexts: An Overview of Self‐determination Theory, The Oxford Handbook of Human Motivation Oxford, New York. Deci, E. L. and Ryan, R. M. (1985) Intrinsic Motivation and Self‐Determination in Human Behavior, Plenum, New York. Garris, R., Ahlers, R. and Driskell, J. E. (2002) Games, Motivation, and Learning: A Research and Practice Model. Simulation and Gaming, Vol, 33, No. 4, pp 441‐467. Gladwell, M. (2008) Outliers: The Story of Success Little Brown, New York. Koster, R. (2005) A theory of fun for game design, Paraglyph Press, Phoenix. Hunter, Kalanithi, J. and Merrill, D. (2010) Make a Riddle and TeleStory: Designing Children's Applications for the Siftables Platform, Paper talked at the 9th International Conference on Interaction Design and Children (IDC 2010). Barcelona, Spain. Huang, W. H. (2011) Evaluating Learners’ Motivational and Cognitive Processing in an Online Game‐based Learning Environment, Computers in Human Behavior, Vol. 27, pp 694‐704. Merrill, D. and Maes, P. (2007) Augmenting Looking, Pointing and Reaching Gestures to Enhance the Searching and Browsing of Physical Objects, Paper talked the 5th International Conference on Pervasive Computing (Pervasive'07). Toronto, Canada. Mcdevitt, T, M. and Ormrod, J. E. (2011) Child Development and Education, Pearson, New Jersey. Morse, W. H. and Kelleher, R. T. (ed.) 1970 Schedules as Fundamental Determinants of Behavior, The Theory of Reinforcement Schedule, Meredith, New York. Penenberg, A. (2010) How Video Games are Infiltrating—and Improving—Every Part of Our Lives [online], http://www.fastcompany.com/1702209/how‐video‐games‐are‐infiltrating‐and‐improving‐every‐part‐our‐lives Prensky, M. (2002) The Motivation of Gameplay: The Real Twenty‐first Century Learning Revolution. On the Horizon, Vol. 10, No. 1, pp 5‐11. Przybylski, A. K., Rigby, C. S. and Ryan, R. M. (2010) A Motivational Model of Video Game Engagement, Review of General Psychology, Vol. 14, No. 2, pp 154‐166. Przybylski, A. K., Weinstein, N., Ryan, R. M. and Rigby, C. T. (2009) Having to Versus Wanting to Play: Background and Consequences of Harmonious Versus Obsessive Engagement in Video Games, Cyber Psychology & Behavior, Vol. 12, No. 5, pp 485‐492. Rigby, C. S. and Przybylski, A. K. (2009) Virtual Worlds and the Learner Hero: How Today’s Video Games Can Inform Tomorrows’ Digital Learning Environments. Theory and Research in Education, Vol. 7, No 2, pp 214‐233. Roark, R. (2012). Sifteo Cubes Create a New Way to Play [online]. http://siliconvalleymamas.com/2012/08/sifteo‐cubes‐ create‐a‐new‐way‐to‐play/ Rupp, A. A., Gushta, M., Mislevy, R. J. and Shaffer, D. W. (2010) Evidence‐centered Design of Epistemic Games: Measurement Principles for Complex Learning Environments, the Journal of Technology, Learning, and Assessment, Vol. 8, No. 4, pp 1‐47. Ryan, R. M., Rigby, C. S. and Przybylski. A. (2006) The Motivational Pull of Video Games: A Self‐Determination Theory Approach. Motiv Emot, Vol. 30, pp 347‐363. Schunk, D. H., Pinterich, P. R. and Meece, J. L. (2008). Motivation in Education: Theory, Research and Application, Pearson, New Jersey. Shih, J. L., Shih, B. J., Shih, C. C., Su, H. Y. and Chuang, C, W. (2010) The Influence of Collaboration Styles to Children’s Cognitive Performance in Digital Problem‐solving Game “William Adventure”: A Comparative Case Study, Computer & Education, Vol. 55, pp 982‐993. Shute, V. J. and Ke, F. F. (2012) Games, Learning and Assessment, Assessment in Game‐based Learning: Foundations, Innovations, and Perspectives [online]. http://link.springer.com/chapter/10.1007/978‐1‐4614‐3546‐4_1 Thomas, M. K. and Ge, X. (2011) Foster 21st Century Skill Development by Engaging Students in Authentic Game Design Project in a High School Computer Programming Class, J. Educational Computer Research, Vol. 44, No. 4, pp 391‐408. Wang, H. and Sun, C. T. (2011) Game Reward Systems: Gaming Experiences and Social Meanings. Poster session presented at the DiGRA 2011 Conference: Think Design Play, Netherlands. Werbach, K, (2012) Gamfication [online]. https://class.coursera.org/gamification‐2012‐001/lecture/37

755

Applying Ideas From Intelligent Tutoring Systems for Teaching Programming in Game Based Learning Matej Zapušek and Jože Rugelj University of Ljubljana, Faculty of Education, Ljubljana, Slovenia matej.zapusek@pef.uni‐lj.si joze.rugelj@pef.uni‐lj.si Abstract: Students in introductory programming courses often have problems understanding cognitively complex concepts. In our research work we are trying to find innovative methods for teaching programming to support deeper learning and to foster better learning outcomes. Game based learning is recognized as motivating for students; facilitates active learning and presentation of learning in a context that is fun and engaging. There has been a lot of work done in the field of intelligent tutoring systems, proving that learning with software that can adapt to students' specific needs and performance have a positive effect on learning outcomes. Games that are developed for education often follow computer assisted instruction concept, which is rigid, predefined and doesn’t have the ability to adapt to each student. The goal of this paper is to introduce some ideas from intelligent tutoring systems and to suggest how to use them in educational game for teaching introductory programming. Paper begins with the explanation why programming domain is so hard to teach/learn and follows by arguing why intelligent tutoring systems are so successful compared to other teaching methods. We explain differences between CAI and ITS and describe typical components of ITS. Paper concludes with the propositions for our educational game for teaching programming, based on design principles of ITS. Keywords: game base learning, introductory programming, intelligent tutoring systems

1. Introduction In this paper we address the following question: "How can we adopt the ideas from intelligent tutoring systems in game based learning to support learning in introductory programming domain?" Learning how to program is hard for students because it is complex, cognitively demanding and requires well developed algorithmic thinking and problem solving skills. Difficulty reflects in high dropout rates in introductory programming courses. To cope with the challenges of teaching how to program, it is important to search for innovative methods that ease and facilitate learning process. Our initial idea was to present programming knowledge in a form of computer game in which the in‐game activities resembles the logic of programming concepts. For example, the problem of writing a sentence “I will not talk in class.” for hundred times, transforms in activity that promotes usage of loop to automate the process. Student is solving real‐world problems by intuitively interacting with teaching material, using logic and mechanics that we find in programming. In this way student is focused on meaning of a programming construct (semantic) rather on how to write the code (syntax).

2. Challenges with teaching how to program As mentioned in the introduction, learning how to program is a challenging task. Several authors tried to identify the source of difficulty for programming domain. We are going to summarize the two that are in our opinion the most relevant. Du Boulay (Boulay, 1989) identifies five overlapping domains that must be mastered by novice programmer and that can be also viewed as a possible source of problems: (1) general orientation, (2) the notional machine, (3) notation, (4) structures, and (5) pragmatics. Gomes and Mendes (Gomes & Mendes, 2007) specify the reasons why programming is so difficult to teach/learn: (1) it demands a high abstraction level, (2) it needs a good level of both knowledge and practical problem solving techniques, (3) requires a very practical and intensive study, which is quite different from what is required in many other courses (more based in theoretical knowledge, implying extensive reading and some memorization), (4) usually teaching cannot be individualized, due to common classes size, (5) it is mostly dynamic, but usually thought using static materials, (6) teachers’ methodologies many times don’t take into consideration the student’s learning styles. Different students have different learning styles and can have several preferences in the way they learn, (7) Programming languages have a very complex syntax with characteristics defined for professional use and not with pedagogical motivations.

756

Matej Zapušek and Jože Rugelj

3. Intelligent tutoring systems 3.1 Effectiveness of human tutoring, ITSs, and teaching in classroom Bloom’s 2‐points sigma problem (Bloom, 1984) shows that the most efficient tutoring is human one‐on‐one. This is because human tutor can adapt to the student's personal needs, can provide frequent feedback on misconceptions and rectify them efficiently. She can also provide optimal scaffolding by selecting an optimal problem in a term of difficulty. Scaffolding originates from Vygotsky's idea of zone of proximal development. He defined it as the distance between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance, or in collaboration with more capable peers (Vygotsky, 1978). The term scaffolding is based on the zone of proximal development and was introduced by Wood. He defines it as: "those elements of the task that are initially beyond the learner’s capacity, thus permitting him to concentrate upon and complete only those elements that are within his range of competence" (Wood, 1976). Intelligent tutoring systems (ITS) are computer tools designed for personalized teaching that facilitate autonomous and intelligent adaptation to students needs. ITSs imitate the human tutors by adapting to student, providing problem selection based on a student performance, assist with hints at hard parts of the problem or when hinting is needed and notifying if the steps taken by the student are incorrect. Learning outcomes of using ITSs are one standard deviation lower compared to one‐to‐one tutoring but are for the same amount higher compared to teaching in classroom. In the ideal world every student would have a personal human tutor but obviously that is not possible. ITS provides the best alternative to one‐to‐one human tutoring because of its efficiency and ability to be easily distributed and therefore widely used. Rationale for using the ideas from intelligent tutoring systems in game based learning is therefore evident.

3.2 CAI vs. ITS Games that are developed for education usually follow the computer assisted instruction (CAI) approach. Problems and solutions are predefined, scenarios are predictable, there is no mechanism that would address student inner process that led her to solution and are not capable to adapt to the students specific needs. In most cases they follow the following model: game presents a problem and then obtains the answer from a student. Her answer is then compared with predefined computer solution. The incorrect solutions are followed by consequences and the correct ones by choosing the next problem in a set (Shute & Potska, 1996). Intelligent tutoring systems differ from computer assisted instruction in four main characteristics: 3.2.1 Knowledge and process representation In ITSs designed for programming knowledge is formalized using one of the following methods: (a) In ACT‐R cognitive theory declarative knowledge about domain in transformed into a set of production rules (if ‐ then), which represents procedural knowledge. For example, declarative knowledge “function firstElement takes an array as an input and returns the first element” is transformed into production rule “If the goal is to write an expression that returns the first element in array, then write operator firstElement and make a sub‐goal of providing array as an argument. (b) Constrained based tutors model its knowledge by analysing similarities and differences between distinct solution to the same problem and generating semantic constraints. (c) canonicalized model where programming solutions are presented in a form of abstract syntax trees. ASTs represent the underlying structure of a program by branching complex statements out into smaller sub‐ statements (Rivers & Koedinger, 2012). 3.2.2 Intelligent selection of optimal problems The high‐level operation of a tutor can be split into the outer loop that handles problem selection, and the inner loop that helps the student with the steps required to perform a single task. Selection of the next problem in the outer loop is based on a student model. The goal is to choose the optimal problem that facilitates scaffolding. 3.2.3 Hints and consequences ITSs are providing relevant hints at hard parts of the problem or when student makes a wrong step in a problem solving process.

757

Matej Zapušek and Jože Rugelj 3.2.4 Learning from past experiences Activities within ITSs follow this model: at first ITS chooses a problem, then it generates step‐by‐step solution, using the formalized knowledge about domain. Computer's solution is then compared to the student’s, one step at the time. Based on her problem solving performance, the student’s model is adjusted, taking into account her knowledge, learning goals, skills, problems, mood, experiences, features… Based on adjusted model it generates and executes further actions.

3.3 Components of ITS Components of intelligent tutoring systems are domain model, which formalizes the expert knowledge about domain and consists of concepts, rules, definitions, processes, and skills, teacher model, which represents pedagogical aspect of the tutor combining curriculum, teaching strategies, and error handling, communication model, consisting of communication strategies, visualizations and natural language, and student model, which represents each student’s knowledge, skills, problems, experiences, and characteristics.

3.4 Principles of ITS design Designing an ITS, especially in programming domain, is a challenging task. In order to make an efficient ITS, we have to follow certain rules that are summarized as eight design principles of ITS design: (1) present domain knowledge in a form of production rules, (2) notify students about the structure of learning goals, (3) activities within ITS should be in a context of problem solving, (4) emphasize problem based learning in abstract form, (5) minimize the working memory load, (6) immediate feedback when error is made, (7) adjusting the complexity of problems based on students performance, and (8) easing the process of acquiring the skill.

4. Educational game as ITS In our proposition for educational we want to adopt the features of the ITSs that are proven to facilitate deeper learning and better learning outcomes. The main idea behind it is to design a game that can adopt to students prior knowledge about the subject, that can follow her progress throughout the learning process, monitor the misconceptions, provide her with the optimal problems that focuses on concepts that she doesn’t understand and giving hints when she is stuck at problem for too long. Our proposition for game based model follows the ITS structure. In order to design an ITS we have to define domain model, teacher model, student model, and communication model and present it in a form of an educational game.

4.1 Domain model The content of the domain model is introductory programming. Our initial attempt was to design a uniform curriculum by comparing curriculums from various countries and different propositions for national curriculums, such as ACM‐K12 and Computing at School Working Group (endorsed by BCS, Microsoft, Google and Intellect). We decided to design it to be language independent, so it could be easily implemented in aforementioned semantic method. We categorized domain knowledge into learning units and further into detailed learning goals. The content of our application is: variables (65 learning goals), commands (11), conditionals (15), loops (51), arrays (21), strings (7), expressions/statements (10) and subprograms (26). In order to formalize the domain of introductory programming we suggest using knowledge space theory which is a set‐theoretical framework, which proposes mathematical formalisms to operationalize knowledge structures in a particular domain. The most basic assumption of knowledge space theory is that every knowledge domain can be represented in terms of a set of domain problems or items. Moreover, knowledge space theory assumes dependencies between these items in that knowledge of a given item or a subset of items may be a prerequisite for knowledge of another, more difficult or complex item. These prerequisite relations are realized by surmise relations, which create a quasi‐order between different items. One advantage of these surmise relations is that they reduce the quantity of all possible solution patterns to a more manageable amount of knowledge states. Each of these knowledge states represents the subset of items an individual is capable of solving. The collection of all knowledge states captures the organization of the domain and is referred to as knowledge structure. Knowledge space for introductory programming domain will be made by experts that teach university level courses. We are going to collect data from students and make adjustments to initial knowledge space.

758

Matej Zapušek and Jože Rugelj Once we have formalized knowledge domain we have to effectively identify which knowledge state resembles student’s current knowledge about the subject. This is relatively simple because it can be done by performing initial activities that monitor which concepts student understands. This is used to map student to according knowledge state.

4.2 Student model Student model has to include all the relevant information about the student. First there is a list of concepts that she does or doesn’t understand so we can map to corresponding knowledge space. We also want to track information about student performance: time spent on solving a problem, correct/incorrect attempts, coding style, number of successfully learned learning goals, misconceptions, points in a problem where she made a mistake, when she make a break for thinking. We are going to gather this information by tracking every input action from student. This will enable us to completely reconstruct the problem solving activity with each individual problem. Student model will provide gathered information to the pedagogical module.

4.3 Pedagogical model Pedagogical model is a strategy for learning; it defines when to repeat the problem, when to proceed to the next problem, or which problem would be the most suitable to present in next step. In order to provide an optimal selection for scaffolding, problems need to be evaluated by experts to give approximate assessment of a problem difficulty. It would be convenient if we could automate the process with some sort of difficulty predicting algorithm but this seems to be too optimistic for programming domain. The selection of next problem will indirectly implement the game scenario and it is analogue to outer loop of ITS activity. Choosing the right difficulty is only one challenge but we also have to choose which concepts we want to teach in a next step. This task is simpler and can be done by identifying those concepts she didn’t understand and that omits her to be placed in a higher knowledge state. In the next step pedagogical module will select appropriate problem in the database, which will contain misunderstood concepts and provide it to student. This will be performed repeatedly until student masters all concepts of the domain. In our further work we will try to find an "intelligent" solution to implement such scenario so the game scenario will still make sense.

4.4 Communication model Communication model will be implemented as a game. Activities within the game will follow the aforementioned semantic method. Students will interact with the learning material with simple and intuitive pseudo‐language because we don’t want students to be focused on syntax of a specific programming language which is often the case in introductory courses. We believe that it is not clever to burden student’s cognitive apparatus with excessive and redundant information that makes understanding of complex concepts more difficult. We want them to focus on meaning and logic that is behind programming constructs such as variables, conditionals, loops, etc. so they can use it to solve problems. We already designed a few games considering semantic method for teaching introductory programming. For example in game “Magical salvation” player has to use magical abilities to help sorting herbs (green ones in left cauldron, brown ones in right cauldron) for magical potion. After sorting first 20 herbs “by hand”, magician suggests using a spell that will do it for you. Payer is the encouraged to do it using conditional and loop. We realize that it will be challenging to write a scenario for the game because the path can not be predicted.

5. Conclusion Our model for educational game implements ideas from intelligent tutoring systems following the typical ITS design. It consist of four parts: domain model, which is derived using ACT‐R production rules, founded on knowledge space theory and is based on joint curriculum for introductory programming, student model, teacher model for selecting the optimal problems that supports scaffolding, and from communication model that is game itself. This paper describes work in progress so in future work we are going to design a game and compare its effectiveness with other methods for teaching how to program.

References Bloom, B. S. (1984). The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One‐to‐One Tutoring. Educational Researcher , 13 (6), 4‐16.

759

Matej Zapušek and Jože Rugelj Boulay, d. (1989). Some difficulties of learning how to program. Gomes, A., & Mendes, A. (2007). An environment to Improve Programming Education. Proceedings of the 2007 International Conference on Computer Systems and Technologies. New York: ACM. Rivers, K., & Koedinger, K. (2012). A Canonicalizing Model for Building. ITS'12 Proceedings of the 11th international conference on Intelligent Tutoring Systems , (pp. 591 ‐ 593). Shute, & Potska. (1996). Handbook of Research on Educational Communications and Technology. Vygotsky. (1978). Mind in society: The development of higher psychological processes. Cambridge: MA: Harvard University Press. Winslow, L. (1996). Programming pedagogy ‐ A psychological overview. SIGCSE Bulletin , 28, 17‐22. Wood. (1976). The role of tutoring in problem solving. Journal of Child Psychology and Child Psychiatry , 89‐100.

760

Masters Research papers

761

762

Cultivating Preschoolers Creativity Using Guided Interaction With Problem Solving Computer Games Georgios Fessakis and Dimitrios Lappas Learning Technology and Educational Engineering Lab., University of the Aegean, Rhodes, Greece gfesakis@rhodes.aegean.gr pse07120@rhodes.aegean.gr Abstract: Creative thinking is considered important factor for personal and social success in general. In addition, creativity as prerequisite to innovation is important for social and economic progress. The use of computer games for the development of creativity constitutes a modern challenge for the learning sciences. In this paper a case study is presented concerning the impact of a specific problem solving computer game («Crayon Physics Deluxe» of Kloonigames) to the creativity of preschoolers under the guidance of a teacher/guardian. Research analysis considers the MSFM creativity test results, and Self‐Efficacy measurement. Research results give evidence that the use of the game under the appropriate guidance of the teacher has positive impact to the fluency dimension of children’s creative thinking as well as their self‐ efficacy. Teacher guidance appears to be an important scaffolding mechanism if it encourages children to find several solutions and/or to reflect on their experiences. The presented research is interesting to creativity researchers, game developers who want to adopt research informed design principles, teachers who want to develop students’ creativity and parents who want to select beneficial games for their children. Keywords: creativity, young children, computer games, problem solving, crayon physics

1. Introduction Creative thinking is considered important in both the mental wellness and the prosperity of individuals (Torrance, 1995). From social view, creativity constitutes an essential precondition to innovation (European Commission, 1998). The development of creativity, therefore, is justified as a key aim for the modern educational systems (e.g. Partnership for 21st century skills, http://www.p21.org/). As it is claimed by modern psychology, creativity is not just an innate talent but it can be developed through appropriate education and practice methods. In addition a variety of Information and Communication Technology (ICT) applications can support the creative process, by strengthening various well known techniques (e.g. brainstorming) and introducing new ones (Sefertzi, 2000) Young children are in ideal and critical age of develop their creative potential (Wheeler, Waiter & Bromfield, 2002). Furthermore children are attracted and engaged of using computer games more and more while the use of developmentally inappropriate software could be harmful for their creativity (Haugland, 1992). So it is important to explore the feasibility of developing the creative potential of young children through the use of computer games. The present paper concerns a case study on the impact of an open‐problem solving digital game (Crayon Physics) on, (4‐6) years old, children’s creativity. In the paper the concept of creativity is defined, then the importance of creativity to preschool education is discussed, the relation of creativity to problem solving and ICT is reviewed after in order to document the research rational. Subsequently the research design and results are presented and finally the findings are discussed.

2. An educational approach of the creativity concept There are many definitions approaches available for the complex concept of creativity (e.g. Csikszentmihalyi, 1999; Runco, 1997, 2000; Sternberg & Lubart, 1996; Gardner, 1993). In the context of this study, synthesizing the main modern views, the term “creativity” means: the process of producing a work, of any kind (e.g. artifact, device, idea), that is considered remarkable and original, within the framework of a community. The originality is a fundamental requirement by definition of creativity. The scope of originality could be (i) individual (an individual creates something for the first time for him/her self), (ii) group (an individual creates something original in the framework of a group), and (iii) global (an individual creates something for the first time in human history). By the gradation of the originality scope, the creativity becomes a skill possessed by everyone. The term “democratic” creativity was coined by the National Advisory Committee on Creative and Cultural Education, in 1999, to state that all students can be creative in personal or group (e.g. class) level. The democratic view of creativity is more important for education systems and this is why it is also adopted in this

763

Georgios Fessakis and Dimitrios Lappas study. According to Torrance (1966, 1974, 2001) creativity depends mainly on divergent thinking ability which can be analyzed to four main characteristics of human responses to problems: (a) fluency: the number of relevant responses, (b) flexibility: the number of different categories or shifts in responses, (c) originality: the number of unusual yet relevant ideas, as determined by statistical infrequency, and (d) elaboration: the number of details used to extend a response. The above characteristics are the basis for most widely used measurements of creativity (e.g. MSFM test) (Moran, Milgram, Sawyers, & Fu, 1983). Education can affect creativity directly through the use of tested teaching techniques (Mansfield, Busse & Krepelka, 1978; Parnes & Brunelle, 1967; Rose & Lin, 1984; Taylor, 1972) or indirectly influencing factors that contribute to the development of creativity. These factors include intelligence, knowledge, thinking styles, personality and motivation (Sternberg and Lubart 1991) and self‐efficacy (Wood and Bandura, 1989). Self‐efficacy is important for creativity (Sternberg 1996) because persons with high self‐efficacy tend to engage in experimentation with persistence more often and thus the odds are that they will come up with more creative ideas and solutions.

3. Preschoolers, creativity, problem solving, and ICT Creative thinking is one of the most important abilities that children may develop during preschool age (Wheeler, Waiter & Bromfield, 2002). In addition according to Horakova (2004) children (3‐6) years old are in the ideal age to develop self‐efficacy which affects creativity. The flourishing of creative thinking that is observed at this age is fairly impressive. Resnick (2007) noticed and pointed out the distinction between the development of creative‐thinking skills that takes place in preschool education, and that of any other educational level, and proposed that kindergarten approach to learning should be extended to students of all ages since it is ideally suited to the needs of the contemporary society. Creativity in general, and in kindergarten in particular, has two main faces, the first concerns the process of producing art and personal expression (artistic creativity), while the second relates to the process of defining and/or solving problems (creative problem solving). One way of developing creativity is through discovering and solving problems. In addition, problem solving constitutes a well‐known, modern student‐centered pedagogical approach (Feldhusen & Treffinger, 1980). Even isolated learning assignments based on problem solving, have the potential to amplify students’ creativity (Davis & Rimm, 1985; Subotnik, 1988), as long as, creativity dimensions (fluency, flexibility, elaboration, and originality) are activated (Feldhusen & Treffinger, 1980). It therefore follows that, through the use of appropriately selected problems, it is possible to design learning activities that facilitate creative thinking development in general and more specifically for young children. In contemporary kindergarten ICT can be used for artistic creative activities as well as for creative problem solving. Clements’ (1995) studies on the influence of LOGO (Papert, 1980) programming language on child creativity, reported mixed results thus leaving the issue open. The development of open, exploratory, interactive, and developmentally appropriate for young children learning environments, that have occurred since then makes it necessary to revisit the questions through contemporary approaches (Fessakis, Gouli, & Mavroudi, 2013). Furthermore, Haugland (1992) has studied the influence of behavioristic educational software to children’s creativity and argues that the prolonged use of software of this type may destroy children’s creative potential, by prematurely strengthening convergent thinking through their exclusive exposure to problems with one, and only one, correct answer. This combined with Haugland’s (1992) assertion that software of this type weakens the intrinsic motivation for learning by substituting it with external reward, and furthermore, with the fact that when children are allowed to choose for themselves they show preference to software of closed type, reveals the responsibility that educators and guardians share in promoting the beneficial use of ICT.

4. Research rational In order to explore the feasibility to use of ICT to facilitate the development of creativity potential of preschoolers, a case study has conducted based on an awarded problem‐solving digital game, called “Crayon Physics Deluxe” (Purho, 2011). The specific game was chosen because it is engaging and incorporates open problem solving that could foster creativity. “Crayon Physics Deluxe” presents players with a series of problems of increasing complexity. In each level of the game an apple‐ball and a star appear simultaneously on the computer screen but on a different background scene. The player must guide the ball to touch the star. Obstacles, gaps and other challenges may appear in between, though. The player must think of and draw objects to push and carry the ball, to overcome the obstacles, and make it reach the star. Children can draw objects of whatever two dimensional shape they can imagine. The objects on the projected screen are subject to the law of gravity. There is not a restriction on the number of attempts and the player is permitted to delete

764

Georgios Fessakis and Dimitrios Lappas objects and retry the same level as many times she/he wish. If the apple finally reaches the star, then the star takes off and goes to the sky. The total number of stars on the sky constitutes the score of the user. Crayon Physics is considered age‐appropriate for ages 6 and older if no guidance is available in our case the teacher mediation can make the game accessible to younger children. Most of the puzzles can be solved in more than one ways giving the opportunity to the students to employ divergent thinking and express fluency in the specific problem genre. Indicative problem situations of the game are presented in Figure 1.

a) 1st solution

b) 2nd solution

Figure 1: Sample problem solving on “Crayon Physics Deluxe”

4.1 Methodology The case study methodology was chosen as suitable to an exploratory research of a complex situation under specific conditions. A case study concerns the in depth observation of a specific well bounded instance in realistic context. Case studies often enable the illustration of principles from the instances (Cohen et al, 2000). In the present research, the case of young children (4‐6 years old) attending a typical semi‐urban kindergarten in Greece, has been taken with the aim of investigating the extent to which, their involvement in the implementation of a computer game based, problem solving activities, under the guidance of a teacher, may impact the development of their creativity and self‐efficacy. Our research is an exploratory case study (Yin, 2011) which aims to exploration of research questions without prior stating of specific propositions.

4.2 Research questions The following questions were addressed: Q.1: Does the use of the digital game have an impact on the development of the children’s creativity? Q.2: Does the use of the digital game have an impact on the development of the children’s self‐efficacy?

4.3 Participants The participants were 10 children of the 19th public Kindergarten of Rhodes. Kindergartens in Greece serve children 4‐6 years old. The participants’ data are summarised in table 1, where children are referred to with pseudo names. The research conforms to the ethical guidelines of the European Union. The children were chosen randomly from the class and were also arbitrarily paired. Table 1: Research participants’ data Pair

Child

Sex

Age (Years:Months)

Kenta Yiorgos Efie Konstantinos Tsampika Sotiria Mechalis Nikoletta Kiki Yiannis

Female Male Female Male Female Female Male Female Female Male

5:6 5:3 5:3 5:11 5:9 5:10 5:7 5:1 4:3 4:10

P2 P3 P4 P5

765

Georgios Fessakis and Dimitrios Lappas

4.4 Process and research data collection The research was implemented on January 20, 21 and 24, 2011. Pairs P1 and P2 were treated on 20/Jan, pairs P3 and P4 on 21/Jan, and pair P5 on 24/Jan. The research has been conducted in the kindergarten teachers’ office where each pair used the software on a laptop, under the guidance of the researcher. The researcher (the second author of the paper), who is also a licensed kindergarten teacher, was not the regular teacher of the class. For each pair, the whole session has been organized as a sequence of four discrete stages as follows: 1st stage: Introductions and pre‐tests In the first stage the researcher, after a short introduction of him, conducted the tests to measure the children’s creativity as well as their computer games‐related self‐efficacy. The researcher recorded the answers provided by the children along with notes about their reactions. The Multidimensional Stimulus Fluency Measure test (MSFM) (Moran, Milgram, Sawyers, & Fu, 1983) was used for the assessment of preschool children’s creativity. For the application of the MSFM test, a process similar to the one described in Aguirre and Conners (2010) was followed. Children’s responses are scored in terms of Fluency and Originality. The usual method of measuring self‐efficacy is to present problems that are similar to the problems which children must solve in the intervention (Bong & Skaalvik, 2003). Thus, in the context of this research, the children were asked to answer the questions: a) “If I give you some plasticine do you believe that you can make a little man/woman out of it?”, in order to estimate the level of their self‐efficacy in problem solving, in general and b) “If I let you play a computer game, do you believe you can win?”, in order to estimate children’s level of self‐efficacy in regard to the use of computer games. The tests were individually answered by all of the study participants. nd 2 stage: Introducing the learning scenario In an attempt to make more meaningful the problem solving and motivate children a story based learning scenario was designed. The story is an adaptation of the fairytale “Snow White and the Seven Dwarfs” by the Grimm brothers. The adaptation concerns the end of the story; more specifically, in order to wake up Snow White, the prince has to collect the stars that had fallen on earth, with the apple. A digital narration of the fairytale was conducted with the use of the “Story Maker 2” software (Story Maker, 2003). The digital storytelling technique was enthusiastically received by children. During this stage, the researcher was taking notes about the reactions of the children. rd 3 stage: Problem Solving through the use of the computer game During this stage children solve problems playing the “Crayon Physics Deluxe” game, under the guidance of the teacher. The teacher’s role during this phase is crucial because she/he can formulate the children’s interactions and retain their engagement in the game in case of frustration. By guiding them through appropriate questions and suggestions to consider alternative solutions for the same problem and explain their thoughts, the teacher may foster children’s creativity. For the collection of research data in this stage, the screen of the computer along with children’s interactions with the software and their speech were recorded with the use of a computer screen capturing software. th 4 stage: Post‐tests of creativity and self‐efficacy Finally the teacher repeated the tests of self‐efficacy, creativity and problem solving given at first stage, in order to compare the results and document any significant difference.

5. Research results and findings Children reactions, willing of playing and engagement reveals that the game is very attractive for them. The table 2 shows that most of the pairs (P1‐P5) managed to solve 14 problems/levels and gather 14 stars in 44 to 71 minutes. The problems after the 14th level are too complicated for the children at least for their first introduction to the game. The pair P5 of younger children solved 10 levels in 38’ minutes and had more difficulties than the older.

766

Georgios Fessakis and Dimitrios Lappas Table 2: Levels solved by each pair Pair

Levels/Stars

P1 P2 P3 P4 P5

14 14 14 14 10

Duration of the session in minutes 63’ 52’ 71’ 44’ 38’

During the game use the teacher encouraged the children to experiment with alternative solutions. Furthermore he awarded the children for every successful choice and prompted them to explain their thinking. After each unsuccessful trial the teacher asked the children to describe what went wrong and analyse their decisions, a technique that could lead them to try new, improved ones. He also encouraged the collaboration among children. Finally, in cases of repeated unsuccessful efforts, the teacher would demonstrate possible solutions. Without the teacher's presence and guidance, the children would most probably have given up at one of the difficulties they would have encountered along the way; their search for alternative solutions would also be limited. The excerpt in table 3 is characteristic. Table 3: Characteristic dialog of teacher and children

What should we do here? Ah a stone! Where do we put it? Here (with his finger pointing over the right edge) Why there? So that it goes…here! (pointing to the star) You mean the apple? Yes! (smiling) If we put the stone over here, how is the apple supposed to reach the star? It will turn like this…(showing the movement of the catapult)

Researcher Yiannis Researcher Yiannis Researcher Yiannis Researcher Yiannis Researcher Yiannis

The main results of the research data analysis are presented in this section per research question. Q.1: Does the use of the software have an impact on the development of children’s creativity? All participated children underwent the MSFM creativity test before and after the use of the software and the scores of the test are shown on Table 4. Table 4: Results in MSFM creativity test

Yiorgos

Efie

Konstantinos

Tsampika

Sotiria

Mechalis

Nikoletta

Kiki

Yiannis

Fluency Before (FB) Fluency After (FA) Originality Before (OB) Originality After (OA)

Kenta

16 16 21 18

9 13 14 24

13 15 16 22

16 21 26 36

21 19 32 21

21 23 35 32

14 19 20 32

10 11 15 17

10 14 10 15

15 21 16 32

For the purpose of the statistical analysis of the results, four variables have been defined, namely the Fluency Before and After (FB and FA respectively) and the Originality Before and After (OB and OA respectively). Based

767

Georgios Fessakis and Dimitrios Lappas on the Shapiro‐Wilk tests results: a) (S‐W(FB)=0.916; p=0.322; α=0.05) b) (S‐W(FA)=0.955; p=0.727; α=0.05) c) (S‐W(ΟB)=0.918; p=0.338; α=0.05) d) (S‐W(ΟΑ)=0.905; p=0.246; α=0.05), we may consider, at a reasonable safety level (α=0.05), that all the variables follow the normal distribution. We can thus apply parametric tests of significance for the differences in mean values and for the variance, before and after the intervention. In particular, the statistical analysis of the data reveals that:

There is a statistically significant increase in the average fluency (from aFB=14.5 to aFA=17.2) since the t‐ test for two paired samples/Lower‐tailed test gives t(FB‐FA)=‐3.36, df=9, p=0.004. This should be probably attributed to the teacher’s attitude who insisted on the production of alternative solutions as well as on the software which could make the implementation of these solutions possible. The reduction of the standard deviation of the fluency of samples (from sFB=4.249 to sFA=3.967), although not statistically significant [Fisher's F‐test / Two‐tailed test: F(FB‐FA)=1.148, df1=df2=9, p=0.841], supports the value of fluency increase finding, given that means are affected by outliers.

When it comes to originality, the increase in the mean of samples (from aOB=20.5 to aOA=24.9) is not statistically significant, since the t‐test for two paired samples / Lower‐tailed test gives t(OB‐OA)=(‐1.69, df=9, p=0.063). The reduction in the standard deviation of the Originality variable (from sOB=8.141 to sOA=7.505) in this case as well, although not statistically significant [Fisher's F‐test / Two‐tailed test: F(OB‐ OA)=1.177, df1=df2=9, p=0.812], strengthens the means difference. Therefore, the intervention does not seem to make a significant change in the originality of the children’s solutions to the specific problems class. Any contrary finding would nevertheless be rather strange, since neither the total duration of the children’s involvement with the software, nor the children’s age, are conducive to the statistically significant increase in originality for the solutions of the specific problems class.

Finally, it was observed that a second solution to the same problem was suggested not only by students with high fluency score, but also by students with low fluency score (according to the MSFM test). Furthermore, original solutions were suggested not only by children of high originality score, but also by children of low score (according to the MSFM test). However, there is a quantitative difference in the number of multiple and original solutions that derived from children of high creativity score in comparison to the rest of the children. In other words, the essential difference among the children, as far as fluency and originality of solutions are concerned, is quantitative and not qualitative. Considering these results in relation to the research question of whether the software affects the creativity of children, we can claim that, according to the findings, the software use under appropriate teacher guidance has the potential of increasing children’s fluency early on. As long as the originality is concerned there was not significant increasing in this case study but it is open to examine the affection in the long run use of the game. Q.2: Does the use of the software have an impact on the development of self‐efficacy? Students underwent two Self‐Efficacy (SE) tests before and after the use of the software (Table 5).

Kenta

Yiorgos

Efie

Konstantinos

Tsampika

Sotiria

Mechalis

Nikoletta

Kiki

Yiannis

Table 5: Results in self‐efficacy test

SE in General Problem Solving Before (SPB) SE in General Problem Solving After (SPA) SE in Computer Games Before (SGB) SE in Computer Games After (SGA)

1 1 0 1

1 1 1 1

1 1 0 1

0 1 0 1

1 1 1 1

0 1 1 1

1 1 1 1

1 1 0 1

In table 5, we have four Self‐Efficacy variables which correspond to the rows of the table. More specifically the variables are: Self efficacy in general Problem Solving Before and After (SPB and SPA) as well as Self efficacy in computer Games ‐ Before and After (SGB and SGA). In order to test whether a statistically significant change exists before and after the intervention, we consider the variables as binary (0=No, 1=Yes), (adopting the less optimistic interpretation: “I do not know”=0) so as to be able to apply the McNemar test. Applying the McNemar test to table 2 we have the following results:

768

Georgios Fessakis and Dimitrios Lappas

The application of the McNemar test (Exact p‐value) / Lower‐tailed test to SPB and SPA variables (Q=3, Ζobserved=‐1.732, p=0.125), indicates an increase in general self‐efficacy from 7/10 to 10/10 which, however, is not statistically significant. It should be pointed out though, that the children of the sample had already had sufficiently high self‐efficacy results and a statistically significant change should, therefore, not be expected. Given that children is usual to show high self‐efficacy (Bandura & Schunk, 1981) the increasing of it in this case is interpreted as positive evidence.

On the other hand, as far as the computer game is concerned, the change in self‐efficacy appears statistically significant, as the McNemar test (Exact p‐value)/Lower‐tailed test in SGB and SGA variables, gives (Q=5, zobserved=‐2.236, p=0.031). We could therefore draw the conclusion, that the particular intervention described in this paper, seems to increase children’s self‐efficacy with regard to the use of the computer game. This is significant, because children with high self‐efficacy are not easily discouraged by difficulties and failures in an activity. Instead, they remain strongly involved in it because they believe that they will eventually manage to work things out.

6. Conclusion Since creativity constitutes an important factor for personal and social prosperity there is a strong interest of education community for the development of children’s creativity, starting from the preschool age. The extended use of digital games by the children opens new questions about their impact on the development of creativity. In the present research creative problem solving in a digital game environment, was experimentally tested. The research gives evidence for positive effect on the preschoolers’ fluency in solutions. A positive effect was also indicated in the improvement of the children’s self‐efficacy in relation to the use of computer games. Within the short duration of the study, we did not have a statistically significant increase in the originality of the solutions, which seems to require a better knowledge of the problems domain than the participant children had or could develop in the framework of the study. Finally, the non‐statistically significant increase in the number of original solutions produced by the participated children should not detract from the fact that we’ve had innovative solutions from children, regardless of the initial assessment of their creativity. Generally, the authors have the conviction that this particular kind of software under the appropriate pedagogical guidance may benefit the development of problem solving skills and the creative potential of children. The presented research is interesting to creativity researchers, game developers who want to adopt research informed design principles, teachers who want to develop students’ creativity and parents who want to select beneficial games for their children. In the future the research could be continued in the direction of long term impact of games to creativity and/or the design of specific software interactions for creativity cultivation.

References Aguirre, K., and Conners, F. (2010) Creativity and intelligence in preschoolers: preliminary findings. The University of Alabama McNair Journal. Vol. 10, pp. 1‐7. http://graduate.ua.edu/mcnair/journals/2010/Aguirre.pdf. Bong, M. and Skaalvik, E. M. (2003) Academic self‐concept and self‐efficacy: How different are they really? Educational Psychology Review, Vol.15, pp. 1–40 Clements, D. (1995) Teaching Creativity with Computers. Educational Psychology Review, Vol. 7, No. 2, pp. 141‐161 th Cohen L., Manion, L. and Morrison, K. (2000) Research Methods in Education (5 ed), London and New York: Routledge Falmer. Csikszentmihalyi, M. (1999) Implications of a systems perspective. In R. J. Sternberg (Eds), Handbook of creativity. pp. 325‐ 339. Cambridge: Cambridge University Press. Davis, G. A., and Rimm, S. B. (1985) Education of the gifted and talented Englewood Cliffs, NJ: Prentice Hall. European Commission (1998) Innovation Management Techniques in Operation, European Commission, Luxembourg. Feldhusen, J. F. and Treffinger, D. J. (1980) Creative thinking and problem solving in gifted education. Dubuque, IA: Kendall/ Hunt. Fessakis, G., Gouli, E., and Mavroudi, E. (2013) Problem solving by 5–6 years old kindergarten children in a computer programming environment: A case study. Computers & Education, Vol. 63, pp. 87–97. doi:10.1016/j.compedu.2012.11.016 Gardner, H. (1993) Creating minds. New York: Basic Books. Haugland, S. (1992) “The effect of computer software on preschool children’s developmental gains. Journal of computing childhood education”, Νο 3, pp. 15‐30. Horakova, S. (2004) Self‐efficacy at preschool children. Summary of the dissertation project. Charles University Prague. Philosophical Faculty. Department of Psychology. http://www.uky.edu/ ~eushe2/Pajares/tezeaj2.pdf

769

Georgios Fessakis and Dimitrios Lappas Mansfield, R. S., Busse, T. V. and Krepelka, E. G. (1978) “The effectiveness of creative training” Review of Educational Research, Vol. 48, No 4, pp. 517‐536. Moran, J.D. III, Milgram. R.M., Sawyers, J.K. and Fu, V.R. (1983) Original thinking in preschool children. Child Development. Vol. 54, pp. 921‐26. National Advisory Committee on Creative and Cultural Education (NACCCE) (1999) All our futures: creativity, culture and education, Department for Education and Employment, London. Papert, S. (1980) Mindstorms, New York: Basic Books Parnes, S.J. and Brunelle, E.A. (1967) “The literature of creativity”, Journal of Creative Behavior, Vol. 1, No 1, pp. 52‐104. Purho, P. (2011) Crayon Physics Deluxe. [Windows PC]. Played February 2011 Resnick, M. (2007) All I Really Need to Know (About Creative Thinking). I Learned (By Studying How Children Learn) in Kindergarten. Proceedings of the SIGCHI Conference on Creativity and Cognition, Washington, D.C. Rose, L.H. and Lin, H.T. (1984) “A meta‐analysis of long‐term creativity training programs”, Journal of Creative Behaviour, Vol. 18, No 1, pp. 11‐22. Runco, M. A. (1997) Creativity research handbook, (vols. 1‐3). Creskill, NJ: Hampton Press. Runco, M. A. (2000) Creativity: research on the process of creativity. In A. E. Kazdin (Ed.), Encyclopedia of psychology, Vol. 2, pp 342‐346. Washington, DC: American Psychological Association. Sefertzi, E. (2000) Creativity. A report for the EC funded Innoregio project. Sternberg, R. J. (1996) Investing in creativity: Many happy returns. Educational Leadership, Vol. 53, pp. 80–84. Sternberg, R. J. and Lubart, T. I. (1991) Creating creative minds. Phi Delta Kappan, Vol. 72, pp.608–614. Sternberg, R. J. and Lubart, T. I. (1996) Investing in creativity. American Psychologist, Vol. 51, pp. 677‐688. Story Maker 2. (2003). SPA Software. http://www.questaslimited.co.uk/SPA/demostorymaker.html Subotnik, R. F. (1988) “Factors from the structure of intellect model associated with gifted adolescents’ problem finding in science. Research with Westinghouse science talent search winners”, Journal of Creative Behavior, Vol. 22, pp. 42–54. Taylor, C.W. (1972) Can organisations be creative, too? In C.W. Taylor (ed.), Climates for Creativity, Pergamon Press, New York, pp. 1‐15. Torrance, E. P. (1966) Torrance Tests of Creative Thinking: Norms technical manual (Research Edition). Princeton, NJ: Personnel Press Torrance, E. P. (1974) Norms‐technical manual: Torrance Tests of Creative Thinking. Lexington, MA: Ginn and Company. Torrance, E. P. (2001) Experiences in developing creativity measures: Insights, discoveries, decisions. Manuscript submitted for publication. Torrance, E. P. (1995) Why fly? Norwood, New Yersey: Ablex. Wheeler, S., Waiter, S., and Bromfield, C. (2002) “Promoting creativity thinking through the use of ICT”, Journal of computer assisted learning, Vol. 18, pp. 367‐378. Wood, R. and Bandura, A. (1989) “Impact of conceptions of ability on self‐regulatory mechanism and complex decision making”, Journal of Personality and Social Psychology, Vol. 56, No. 3, 407‐415. Yin, R. K. (2011) Applications of Case Study Research. Applied Social Research Methods. SAGE Publications.

770

Haptic Physics Simulation Luciano Santos and Carlos Vaz de Carvalho Instituto Superior de Engenharia do Porto, Porto, Portugal 1050508@isep.ipp.pt cmc@isep.ipp.pt Abstract: Advances in informatics enabled the inclusion of more senses in the interaction between user and machine. One of those senses is touch, made possible through the use of haptic devices/interfaces. Simulators with a haptic command/interface (such as the Novint Falcon, used in the scope of this study) provide realistic feedback in terms of both graphics visualization and force feedback, enabling the user to feel and instinctively understand several aspects of the simulation. The usage of this technology in specific contexts such as physics learning can be quite meaningful because user actions will result in force reactions. This practical approach is contrary to conventional physics teaching where concepts are taught using mostly theoretical approaches rather than experimental ones. This study aims to demonstrate that virtual reality simulations with haptic capabilities are viable learning tools for physics education. The simulator presented in this article is divided in several modules, each addressing a specific area of physics. Developed modules so far relate to aerodynamics, friction and gravitation. The aerodynamics module allows the user to sense the forces applied to an airplane. The module portrays a 3D model of a Lockheed SR‐71b “Blackbird” and the user senses the thrust, drag, lift and weight as he moves the airplane inside the skybox representing the atmosphere. The gravitation module was developed as a game whose concept is to use a 3D model representing the Earth to divert asteroids into a collision course with the Sun. Gravitational forces are applied to the earth and to the asteroid, depending on the proximity between them. The asteroid’s trajectory will change and the user will feel the corresponding force. The friction module enables the user to drag a virtual cube over a flat surface and feel the corresponding friction force. The user is able to alter the material of both the cube and the floor which changes the static and dynamic friction coefficients. The use of these simulations in learning contexts is now being assessed. The first collected data is quite promising and it is expected to have a full set of results and conclusions in the next few months. Keywords: Haptics, simulation, physics, forces, learning

1. Introduction We live in a technological world that is in constant change and education needs to evolve in order to improve learning efficiency. Therefore there is a need to find innovative ways to scaffold students learning. But the way people gather information differs between individuals and depends on several factors. For instance, some people can be defined as “haptic” learners because they tend to learn better by doing hands‐on tasks (Gillespie & Okamura, 2008). One possible way of addressing these needs is the inclusion of information and communication technologies in the classroom. The aim of this work was to design, develop and validate a virtual reality simulator that allows the user to experience/feel several forces of physics such as friction forces, aerodynamic forces and gravitational forces and evaluate its use for education. As human‐machine interaction evolves, the need for inclusion of all the existing natural human senses (hearing, sight, smell, taste and touch) increases. Current human‐machine interaction already includes sight and hearing and haptics represents the inclusion of touch to the interaction. The term “haptics” come from the Greek word haptikós (“ἅπτω”) which refers to the sense of touch and in modern days refers to technology, devices and interfaces that provide tactile feedback. Haptic technology evolved from the initial mechanical approach to new areas of technology such as ultra‐sounds and magnetism. Haptic technology creates the sense of touch allowing users to manipulate virtual elements in a less constrained way with higher degrees of freedom (DOF). The use of haptics provides greater realism to the man‐machine interaction by adding another sense. So although anyone could learn by using haptic devices those who tend to be haptic learners tend to benefit more from this way of learning. A haptic device can have as many as six DOF referring to the movement of a 3D object in space and to the six types of movement that can be applied to it. Three of those types are related to translation: heaving (movement forward/back or back/forward), surging (displacement movement both up and down or the inverse) and swaying (displacement movement either left to right or right to left). The remaining three are related to rotation: pitching (movement of tilting either forward or backward), yawing (movement of swivelling either left or right) and rolling (movement of pivoting side to side). A device with a single DOF can be

771

Luciano Santos and Carlos Vaz de Carvalho for example the table top haptic mouse by (Suzuki & Yano, 2013). A device with two DOF can be for example the spherical interface proposed by (Watanabe, 2012). Most haptic devices nowadays have between three to six DOF. The main reason why haptic technology isn’t still part of everyday life is the price of hardware. Certain areas like Medicine or the Oil drilling industry already use Haptics but the equipment can cost thousands of euros (see Table 1). Only in recent years with the development of more affordable haptic devices such as the Novint Falcon the cost was reduced to a few hundred euros. With more devices becoming affordable the consumer starts to have access to this kind of devices for personal use and it is expected that the cost of this type of hardware will continue to decrease. Table 1: Samples of Haptics devices prices Haptics devices prices Haptic Device DOF Cost ($USD) SensAble PHANTOM Omni Premium 1.0 6 $ 20,450 SensAble PHANTOM Desktop 6 $ 13,000 Novint Falcon Pistol Grip Bundle ‐ White 3 $279.95 Novint Falcon standard bundle 3 $249.95

The Novint Falcon is a haptic device that has three degrees of freedom (3 DOF) and two types of handles: one is a grip similar to a ball that contains four buttons (Figure 1) and the other resembles a gun mainly used in action games like first person shooters (FPS). The device has a limited working area, but it is very resistant to damage, is capable of generating high forces and costs about 250 $USD making it the most affordable haptics device on the market. The Falcon has a refresh rate of 1 KHz, making both the physical and virtual reaction very realistic, since virtual movements generated on the simulator will be transmitted to the Falcon motors and transformed in physical movement almost in real‐time.

Figure 1: Novint Falcon The rest of the paper is organized as follows. Section 2, Simulation, explains computer generated simulations, haptic simulations and the use and development of haptic simulations for the purpose of physics learning. Section 3, Forces of physics simulator refers to the development of several haptic physics simulations referring to several existing types of forces of physics. Section 4, Conclusions, presents some concluding remarks.

2. Simulations A simulation can be defined as the process of designing a model of a real or imagined system and conducting experiments with that model (Smith, 2000). Most aspects of real life can be simulated and simulations are used in almost every area of knowledge, such as education, flight dynamics, medicine, etc. A major concern related to simulation is the fact that it is not equal to the real world but just an approximation. The use of simulation benefits education by enabling a task to be executed without the drawbacks that it would have in the real world, enabling the user to perform the task countless times without the fear of error. Some virtual reality simulations already exist with the intent of teaching physics like the ones from (Han & Black, 2011). Haptics are valued in simulations because they provide automatic feedback to the user, enabling a more realistic response time to a situation, as well as a better assimilation of the contents and environments of the simulation. This type of simulation can be used either as an education tool or for recreational purposes such as games (Farhadi‐Niaki, 2013). Several fields of study require students to perform physical tasks. Such tasks can be performed in a virtual environment with haptics technology, allowing the user to learn the same skills that would otherwise require superior amount of resources and equipment (ELI, 2007).

772

Luciano Santos and Carlos Vaz de Carvalho Haptic systems provide great possibilities in terms of physics learning, by adding another sense to the way we relate to technology. Some aspects of learning can now be perceived in a more immediate way when the user actually feels the forces that are in play. This allows certain concepts to be better understood in a more interactive way through the affinity between virtual reality and physical reality, allowing for a better understanding of physics. Haptics physics simulations have a multitude of possible uses for education purposes, allowing for concept teaching and specific training. A few works in this field already exist, for example, (Williams II, 2002) intended to teach high school physics.

3. Forces of physics simulator The Forces of Physics simulator is divided in several modules, each addressing a specific area of physics. Developed modules so far relate to aerodynamics, friction and gravitation. The aerodynamics module allows the user to sense the forces applied to an airplane. The module has a 3D model of a Lockheed SR‐71b “Blackbird” and the user senses the thrust, drag, lift and weight as he moves the airplane inside the skybox representing the atmosphere. The gravitation module was developed as a game whose concept is to use a 3D model representing the earth to divert 3D models representing asteroids into a collision course with sun where they will be destroyed. Gravitational forces are applied to the earth and to the asteroid, depending on the proximity between them. The asteroid’s trajectory will change and the user will feel the corresponding force. The friction module enables the user to drag a virtual cube over a flat surface and feel the corresponding friction force. The user is able to alter the material of both the cube and the floor which changes the static and dynamic friction coefficients. The simulation is multi‐language with its contents translated into Portuguese, Spanish, English and German.

3.1 Aerodynamics forces / flight dynamics forces The aerodynamics force can be divided in four components: thrust, drag, lift and weight. This module contains a representation of an airplane and as the user handles the haptic device, the model alters the position of the plane model, enabling the user to feel the aerodynamics force components decomposed according to the XYZ axis.

Figure 2: Aerodynamics module

3.2 Friction Friction forces can be divided in four components: pushing force, friction force, normal and weight. In order to simulate friction we used a 3D cube model with a handle that represents the physical world handle of the Novint falcon. The cube can be moved over a limited surface. The user can alter the materials of the cube and of the supporting surface using the arrows seen on the top part of Figure 3. The left part relates to the surface material and the right to the cube material. The alteration will result in changes in both the static and kinetic coefficients and consequently on the resulting friction force. Currently there are six types of surfaces: concrete, glass and rock, rubber, steel and wood. The cube surface is always dry, but the floor can be either dry or wet.

773

Luciano Santos and Carlos Vaz de Carvalho

Figure 3: Friction module

3.3 Gravitation Gravitation force is applied to both the active asteroid and the planet earth. As the distance between the objects decreases, the force increases, so the user manipulates the planet and, depending on the proximity of the objects, the user will feel the force accordingly. This module was developed as a game so every time an asteroid becomes active, it will start hurling towards the screen, and if the asteroid is attracted to the planet, the asteroid trajectory will be diverted. As the asteroid moves the user will be able to track its trajectory represented with green dots. The game objective is the destruction of the active asteroid that can only be accomplished if the user alters the asteroid trajectory so that it will collide with the sun. Each time the active asteroid is destroyed the amount of destroyed asteroids will increase as shown on the top left of Figure 4 next to the destroyed asteroid symbol, and subsequently a new asteroid will become active.

Figure 4: Gravitation module

4. Conclusion This article covers the use of haptic technologies and computer simulation for the purpose of physics education and respective evaluation by testing its use in high school classroom and testing the knowledge acquired by students during due to the use of the haptic simulator.

774

Luciano Santos and Carlos Vaz de Carvalho An application called Forces of Physics was implemented with the intent to assist users on their learning of concepts related to several types of forces of physics. The application doesn’t intend to replace books and notebooks used in education system, but instead it’s meant to complement it, by providing the students with hands‐on experiences. The application uses haptics as a new technological means for educational purposes, with the intent of developing and validating its use in education. The application will now be presented to high school students in order to compare it to the traditional way of teaching and learning.

Acknowledgements This article is based on work undertaken within the Alfa Gaviota (Grupos Académicos para la Visualización Orientada por Tecnologías Apropiadas) project funded by the European Commission ‐ Europaid.

References ELI. (2007). 7 Things You Should Know about... Haptics. Retrieved 4 6, 2013, from 7 Things You Should Know, EDUCAUSE Learning Initiative (ELI): http://net.educause.edu/ir/library/pdf/eli7029.pdf Farhadi‐Niaki, F. G. (2013). Usability Study of Static/Dynamic Gestures and Haptic Input as Interfaces to 3D Games. ACHI 2013, The Sixth International Conference on Advances in Computer‐Human Interactions, (pp. 315‐323). Gillespie, R. B., & Okamura, A. M. (2008). Haptic Interaction for Hands‐On Learning in System Dynamics and Controls. Control Systems Magazine. Han, I., & Black, J. B. (2011). Incorporating haptic feedback in simulation for learning physics. Computers & Education, 57(4), 2281‐2290. Smith, R. D. (2000). Simulation. Retrieved 4 13, 2013, from http://www.modelbenders.com/encyclopedia/encyclopedia.html Suzuki, H., & Yano, H. ,. (2013). 1 DOF Tabletop Haptic Mouse for Shape Recognition of 3D Virtual Objects. ACHI 2013, The Sixth International Conference on Advances in Computer‐Human Interactions, (pp. 309‐314). Watanabe, K. S. (2012). Bilateral control with 2‐DOF haptic spherical interface. IECON 2012‐38th Annual Conference on IEEE Industrial Electronics Society, (pp. 4394‐4399). Williams II, R. L. (2002). Haptics‐augmented high school physics tutorials. International Journal of Virtual Reality, 5(1).

775

Evaluating the Embedding of Games Based Learning in a Computing Subject at University Emilia Todorova and David Moffat Glasgow Caledonian University, Glasgow, UK Emilia.V.Todorova@gmail.com D.C.Moffat@gcu.ac.uk Abstract: Games Based Learning (GBL) has been an interest area for research in the education sector for the past few years. Current research has been seen to affect secondary and primary education systems, but GBL has rarely been seen on a Higher Education (HE) level. Furthermore, although the case for using games has been built through research in Serious Games, the actual integration of GBL in curricula is under researched at the moment. As part of a research project in Glasgow Caledonian University, a module within the Games Development and Design Suite was introduced to a game in order to measure the improvement of their skills and subject knowledge. As part of the research, the integration of the GBL method was also analysed in terms of its success and ease of embedding to the other teaching and learning techniques used. In terms of the modules used for the study, it can also be determined what can make the integration work better and what should be avoided if GBL is used in the Higher Education classroom Keywords: GBL, teaching, higher education, curriculum

1. Background Higher Education is facing many developments, most of which are related to technology. New ways of delivering education, such as MOOCs (Massive Open Online Courses), online, distant and blended learning are increasing in popularity and demand (Hiltz and Turoff, 2005). Traditional higher education institutions in response need to embrace this shift towards technology and make more use of it during the traditional classes in order to keep up with the trends, engage their students and continue to deliver good quality education. According to the Entertainment Software Association (2012) the typical student owns a laptop or a desktop a computer, at least. Students also own a variety of digital devices, such as smartphones and music players, game consoles and tablets. “Today, nearly every device with a screen plays games, providing interactive entertainment experiences for a wide and diverse population.” (Entertainment Software Association, 2012), which makes playing games possible not only in homes, but also on the move and in environments where game consoles/devices are not present. In the United States 49% of households own a games console and the average age of a gamer is 30 years old (Entertainment Software Association, 2012). It is evident from these findings, that gaming is not for children or teenagers anymore, but also adult learners are very likely to find games more attractive. One of the reasons why Games Based Learning can be a useful tool is that it allows a balance between conceptual (teaching about) and procedural (teaching how to do) knowledge, as Alessi (2000) notes. In addition to that, using games in education helps to address certain limitations or weaknesses of traditional education – motivation and engagement in learning. Games also have the ability to help the intake of knowledge and skills and can be used to support more traditional educational methods. (Sica, 2011) If we look closer on why exactly Games Based Learning can be indeed so beneficial when used in education, it has been established that games promote the active learning and “involve a direct focus on the learner’s active participation” (Sica, et al, 2011). Although Games Based Learning can enable more methods for teaching, promote engagement with the class and enable a better focus and relevance to subjects, there are many challenges to including Commercial‐Off The Shelf (COTS) games within the curriculum (Charsky and Mims, 2008). One of these challenges is finding the appropriate games that could fit within the learning objectives and aims which have been established. Another challenge is that there is still much more research to study the effects of including games as a learning tool, establish if games can really bring a positive result, as well as investigate the depth of influence games might have.

776

Emilia Todorova and David Moffat Furthermore, how to naturally embed games in the curriculum is currently an under researched area. This is one of the reasons why the use of GBL in Universities is very low at the moment. To add to this, the challenges described below also contribute to the low GBL activities in Universities. According to Charsky and Mims (2008), one of the most important things when introducing GBL to a curriculum is for the teachers to master the game first – playing the game in different styles, accomplishing different goals and trying out different techniques. This task alone can be time consuming and in a fast‐paced environment such as in a University might be more difficult to accomplish. This is also a challenge, because it is difficult to estimate the time needed for this task. Another challenge is that average games have been designed to include around 40‐60 hours of gameplay and some games (such as Civilization III) might need several hours of playing before one can have an understanding how the game can be played (Charsky and Mims, 2008). However the above concerns can be dealt with, as very clearly shown in a study by Blunt (2006) which shows some very promising results in studies taken in a University where a GBL approach has been applied for programmes in Business, Management and Economics. His results clearly show that the students who have played the prescribed games as part of their course have done better on average than the classes who undertook the same module but continued studying through the traditional methods. This paper will explore the notion of attaching GBL as a learning tool in the learning environment of a module in Artificial Intelligence. The paper will explore what risks have been identified when GBL intervention is included, what are the costs vs. the benefits, the method and results of the case study as well as guidelines and advice on integrating GBL in a module. These lessons have the intention of helping tutors who are not familiar with GBL or games to understand better how an intervention can take place and to help them understand if GBL is the right tool for their module. Research Question: Can the learning of Artificial Intelligence be complemented with the use of GBL and how can GBL be successfully included within a module curriculum?

2. Risk analysis One of the most significant issues in using games within education is that there are many barriers preventing academic tutors of involving games as part of their pedagogical approaches (Lean et al., 2006). In this section, the risks of involving games in the classroom will be reviewed. In the Results section, the risks will be reviewed again with more information on their occurrence. Table 1: Risks of using games in HE modules. I – stands for intervention risks; G – stands for games risks ID I1.

I2.

I3.

I4.

Risk Lack of Information About games and what makes them effective, as well as how best to include them in the curriculum Time Consumption The amount of time needed has been miscalculated and therefore more time is required than available. Unfamiliarity with games (Lean, et al, 2006) When the tutor is not familiar with games, or the game that is used within their classroom it will be much harder to allow for a smoother learning process.

Probability High

Admin and/or technical issues As in the integration of any technological tool for learning there might be technical issues occurring (for example problems with game installations).

Medium

High

Medium

777

Contingency Spending a good proportion of the time allowed for this to investigate the benefits of using a game and any good practices in HE. Allow a few extra weeks in case there are elements that have taken longer than anticipated. Even if the tutor is using ready materials, It is advisable that they play the game in great detail. If new games/materials are being used – they definitely need to be tested and played prior to implementation. Allow some extra ‘padding’ time, so if any technical issues occur, there will be time to fix them.

Emilia Todorova and David Moffat ID I5.

Risk Costs vs. Benefits. If the costs ,such as time and money, are not justified by the benefits, then the learning method will not be worth implementing.

Probability Low to Medium to High

Class Time Consumption The time spent in class for the intervention is too much and it impacts the students’ learning. Game is a Poor Fit with Learning Outcomes The selected game does not answer to the learning outcomes defined within the subject and therefore is unable to aid the enhancement of knowledge in the area. Game no longer appropriate This risk can take place if the GBL method has occurred in the modules already for more than 1 year. It is possible that if the programme content changes over time, the game will no longer be appropriate for it. Game is too complex. (Charsky and Mims, 2008) The game selected takes longer to learn than the allowed game time.

Low

G1.

G2.

G3.

Low

Medium

Contingency The more the tutor is unaware of what costs are involved before they begin the integration, the higher the chance of having more cost than benefit is. This is why, guidelines, case studies and prior research should be taken into account before proceeding. Create a small case study to examine the impact of the intervention AND/OR explore similar case studies. The criteria for the games need to be very detailed and clear. Before the selection of one game, a few must be tried and tested to make sure that the best possible fit has been found. The game should answer directly to the core learning outcomes, which are unlikely to change very often. Reviews of the effectiveness of GBL tools within the class should be conducted often. The tutor needs to play the game first in order to understand the level of complexity and judge if this will be appropriate for their course.

The risks in Table 1 can certainly jeopardise the successful running of a GBL module attachment. However, after the above risks have been identified, within the guideline in the Results section, how they have been avoided and if they have occurred – how they have been dealt with will be explained in detail to allow the study to be recreated with minimum risks.

3. Method To investigate if GBL can successfully be integrated within a higher education module, the impact the games have on students should be determined. For this to be established, a trial learning methodology was developed and included in the programme for a third year module “Artificial Intelligence for Games” (GAI). Their normal schedule included five contact hours separated in lectures (2 hours), tutorials (1 hour) and labs (2 hours) per week. The developed GBL intervention had to be attached within those five hours, and this was done through the labs. One hour was devoted to their ordinary programme and the rest to the games and to the GBL intervention. To understand better how the GBL inclusion works within the class room, the GAI class was split into two groups – treatment and control, where they were subjected to different learning methodologies. When the learning methodologies were developed they had to be subjected to the criteria illustrated in Table 2. Furthermore, the games also needed to adhere to certain criteria, as demonstrated in Table 3. Otherwise, there was a risk of not selecting an appropriate game for the study. As the study was done in class time, it was vital that it does not interfere with the learning process of the students and especially not lower their chance of achieving a good understanding of Artificial Intelligence. Table 2: Learning method (intervention) criteria Criteria

Response

2.1. The length of the intervention needs to be short, in order to not be invasive over the ordinary process. 2.2. The time used for the games needs to be short.

The intervention was implemented for a period of three weeks. The games were administered for maximum of one hour per week. The intervention was included within the practical exercises only and used maximum of half of the time available within the labs. The learning outcomes for both groups were the enhanced knowledge of Artificial Intelligence in both theoretical and practical capacity.

2.3. The Intervention should not use any extra time of the ordinary 5 hours of teaching. 2.4. Although the learning method for each of the groups was different, the learning outcomes and their fulfillment need to remain the same.

778

Emilia Todorova and David Moffat Table 3: Games criteria Criteria 3.1. Open‐source licensing 3.2. Free of charge. 3.3. Has some level of AI.

3.4. Code modifiable.

3.5. Easy to learn

Response All used games were under open‐source license. Both games are absolutely free to use. Unknown Horizons had two types of AI present – Settlement AI and Battle AI. Onslaught Arena had basic Battle AI. This criterion was not met fully. Unknown Horizons had a very complex code and it would have been very complicated to change it. Onslaught Arena, although free, did not allow the game to be run locally on the computers for more than a few seconds – meaning that any changes made could not be observed. Both games were easy to learn for both the students and a module tutor.

The criteria in Tables 2 and 3 were established prior to the study to provide clarity on how the project experiment will be conducted and also provide information for the module leader to agree upon before the inclusion, as it was very important that the students will not suffer any negative impact due to the study. Criteria 2.1 and 2.2 were necessary to allow a low risk intervention. The GBL intervention needs to serve only as an attachment in order to compliment the learning of AI, and not to be the primary learning method. To enhance this, criterion 2.3. has been defined. Criterion 2.4 was defined in order to allow for equal opportunities in both groups. As for the games, following the criteria can serve as a guideline on what games will be suitable for such integration. Only games fitting the criteria were selected and then tested.

3.1 The chosen games Unknown Horizons (Control Group) is a real‐time strategy (RTS) game with economic simulation aspects. The player is assuming the role of a settler who needs to establish a community, keep the people of this community happy and provide buildings and resources in order to do so. The game was perceived as able to improve some soft skills (such as problem solving) and also the field knowledge of the students as it incorporates two AI types – Settlement and Battle. The game is also fully open‐source, which means that students will be able to examine the code behind it and learn from its development. Onslaught Arena (Treatment Group) is an arcade game which has very simple controls and code. The player is assuming the role of a knight who needs to fight off creatures in a battle Arena. With each successful mission the game progresses to the next level where the creatures are stronger and have more advanced characteristics and occasionally the player needs to face bosses. The simple interface and the straight‐forward code should allow the students to easily understand the code behind the game and in return be able to understand how the game AI is acting. The game was perceived as a tool to enhance the understanding of AI within games.

4. Intervention There were all together 19 students in the study – 11 in the control group and 8 in the treatment group. The groups were split by their natural timetable. The game was released in the middle of the semester for three consecutive weeks and the students played the games and conducted the exercises for the second hour of their 2 hour lab sessions. The students did not have the choice to select which group they were in. Apart from the study material to go with the game, both groups had the same classes and covered the same knowledge in their ordinary programme. However, to understand better how a game can help enhance their experience in the class room, the groups were given two different types of resources,

779

Emilia Todorova and David Moffat The treatment group was given a tutorial during which included parts of the game code, as well as practical exercises where the students had to explain how certain methods work within the game, from both observing the code and the game play and were later asked to explain how they would improve the game and write their own methods based on the game. The control group, were given a theoretical tutorial, where they were asked to read the documentation supporting the development of the A.I. In this case, they had to play the game, while reading the description of the settlement and battle AI and understand how they work and why they react in certain ways. No practical exercises aside from playing the game were given. To evaluate the results the students were asked to fill out a questionnaire at the beginning and the end of the study, where they were asked to evaluate their skills. The questions concentrated on soft skills and field knowledge. For the purposes of this paper, the results will only look in the field knowledge responses.

5. Results This section will cover the results of the intervention and estimate the success of implementing the games. During the evaluation phase, the students were asked the question “Overall do you feel that playing the game has helped you understand the use of artificial intelligence in game development?” The responses from the Treatment Group were very similar to this one ‐ “Yes playing the game then going through the questions help me understand what is under the bonnet with regards to AI is programmed into games.” (7/8 students had a similar response). Most of the students felt that the combination of playing the game and working through a practical tutorial gave them the opportunity to understand how AI works. The students mainly mention that the tutorials was helpful to them and only vaguely mention the effects of actually playing the game – however as one student notes, “It made a lot more sense after playing through the game first.” Another student also noted that “manipulating the code was helpful”, supporting the statement that more interactivity with the code has been useful in the Treatment Group. The results in the control group are also positive, as the majority of students have commented in a similar manner on the same question. “Yes, I have been able to further my understanding of decision making and handling using an AI in a game.” Other interesting responses include: “Slightly since I have only started learning AI techniques this semester and have got to see them in action in a game.”; “Yes. It made me think about the particular types of AI that is used in this genre of game.” and more. In both cases, the treatment and control groups seem to have made very good use of the documentation supporting the games and they have indicated that without the tutorials they would have not learned as much. In both groups only one person has answered “No” to the question. However, the results from the self‐ evaluation of the students show a slightly different picture. At the beginning and at the end of the experiment the students were asked to rate their knowledge and skills in terms of “Artificial Intelligence for Games Development” and “Artificial Intelligence Programming Skills”. The results from the beginning and the end were analysed to see how the perception of the students has changed over the three weeks. Although, this only shows the students’ perception it should be said that student perception is a vital part of the student experience. This indicates that there are more ways in which GBL can improve engagement. GBL is thought to help this only because games are fun. However from this it can be observed that GBL can increase engagement, because students believe the games are enhancing their skills. In Figure 1 it is evident that the students playing Onslaught Arena felt more confident about their skills gain that the students playing Unknown Horizons. The data in Figure 1 has been derived from the surveys given to the students. The students were asked to rate their skills in terms of “Field knowledge and understanding” – theoretical knowledge; and “Field Knowledge Application” – practical knowledge. The students were given a scale between 1 and 5 and had to rate their skills accordingly, where 1 was equivalent to “No understanding” to 5, equivalent to “excellent understanding”. Their responses from the survey at the end were compared to the responses from the

780

Emilia Todorova and David Moffat beginning of the study, and the difference recorded. Hence any perception of improvement has been noted in the chart in Figure 1. Positive change indicates the number of students who have selected a higher response in the end, than in the beginning and no positive change indicates that their response has remained the same. For example: Student 1

Selection before intervention 2/5

Selection after intervention 4/5

Student 2

4/5

Figure 1: Field knowledge increase between control and treatment group Aside from the games and the way they make use of supporting materials, there are no other differences between the teaching and learning methodologies within the GAI course. This is why, it can be clearly seen which methodology has been more effective in the enhancement of field knowledge, which the students believe they have achieved. The treatment group and the control group were both expected to have achieved similar results in this category, however it can be seen that the methodology used within the treatment group has been more successful. The majority of the students within the treatment group indicated that their skills had improved, as opposed to the majority of students within the control group who indicated that there was no positive change. In the control group almost half of the students indicate that they had felt that their theoretical knowledge had improved, however only a small amount of students felt the same way for their practical knowledge. In the treatment group, a larger amount of students had indicated that their skills had improved in both theoretical and practical knowledge. This proves that in this case, the methodology used in the treatment group had been more successful.

6. GBL integration guideline In this section to integration of the GBL intervention will be discussed, specifically In terms of practical details. For the purposes of this, a cost/benefit analysis can be found below, after which a more detailed action plan with time estimate has been provided.

6.1 Cost/benefit analysis Within this case study the main cost was time. Specifically – preparation time and class time. However this can be further broken down, as demonstrated in the guidelines.

781

Emilia Todorova and David Moffat A significant cost is the class time. As no extra time is available for the GBL addition, part of the practical labs was taken out for this. Therefore, the class needs to be prepared to take out one hour of their ordinary lab exercises to undertake this exercise. There were no financial costs in this study; however such might occur if the games need to be purchased. As for the benefits, the games were seen to provide students with an alternative way to enhance their learning within AI. The games helped by demonstrating a practical example for their field. The engaging nature of the games was seen to encourage the students to participate more in the class. Including GBL in the learning process makes it more relevant to the students, which in turn makes them want to engage more in the class

Figure 2: Summary of costs and benefits. Most importantly, the module leader was happy with the GBL intervention and will continue to include it in their class in his module, which significantly contributes to the benefits within this case. In this case study, the benefits exceeded the costs. Although a good amount of time was spent researching GBL methods, and then finding the most suitable games which fit the criteria, the games were seen to benefit the learning process and to enhance engagement. However, this might be different for other cases.

6.2 Guideline This section will provide more practical guidelines in terms of dealing with risks and time/cost estimation for a GBL Integration. The risks identified in the Risk Analysis section have not all been met within this case study, however in Table 4, the risks that had occurred can be found with more detail on how they were dealt with. Table 4: Risks and their occurrence within the case study Risk ID I2 I4

How the risks were dealt with. The experiment had to start a few weeks later because of the installation process described below. After initially contacting the IT department for installing the games the time given for this was one week. However, the IT department did not finish the installations until three weeks after the request. Nevertheless, enough time was allowed as contingency and although the risk occurred, there was no negative effects.

In Table 5 a more detailed guideline with time cost indications has been included. The information has been derived from the conducted study.

782

Emilia Todorova and David Moffat Table 5: Guideline Activity Finding the suitable game(s) Trying the game(s)

Getting to know the selected game(s)

Developing tutorial materials

Description The game(s) should fit the criteria described in Table 3. As there will be a few games selected which fit the criteria, they will all need to be tested. The selected game(s) need to be played in great details in order to understand how best they can be integrated within the module.

Releasing the game in the classroom

To fully take advantage of the benefits a game can bring in the classroom, it is advisable that tutorials are also released with the games, in order to fulfill the intended outcomes. Practical tutorials which allowed for problem solving were seen to be more beneficial than theoretical ones. The games need to be installed in the lab(s) where they will be used.

TOTAL Hours per lesson:

How it was met in this study Took 5 hours intensive research. Took 8 hours. However if a number of games is selected, this can be a lengthy process. It might amount to 12‐15 hours. Took 4 hours. Depending on the complexity of the game, it can take longer. However, the tutor needs to be careful with Risk G3. It is likely that this process can take a few weeks, especially if there will be more than one game used. Took 2 hours for both groups. However depending on the complexity it can be slightly longer. Should not take more than 2‐3 hours per ‘lesson’. The time spent on this is not great, mainly because much of the preparation has already taken place. Making use of game documentation is also very helpful to decrease the time. Took 3 weeks due to IT department. This is why, tutors are advised to allow a month prior to actual start of the GBL intervention. The installation themselves should not take more than one day. Roughly 5 hours preparation per 1 contact hour.

7. Summary The case study has proven positive and can serve as a good example of the integration of GBL within a University module. The integration worked successfully and has shown how many of the barriers of introducing GBL in HE can be dealt with. The integration was not seen to be invasive of the module programme. Both the control and treatment groups did well and achieved the learning outcomes, however it is evident that the treatment group felt more benefits especially in terms of practical knowledge. This paper contributes to the knowledge and understanding of what barriers can be met when introducing GBL in universities and should help with advice on how the risks can be minimised, and the benefits maximized. The guideline at the end of the paper can serve as indication of how much time the activities for developing a GBL framework might take.

References Alessi, M.S. and Trollip, S. R., 2000, Multimedia for learning: Methods and development (3rd ed). Boston: Allyn and Bacon. Blunt, R. 2007, "Does game‐based learning work? Results from three recent studies", The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC)NTSA, Available at < http://patrickdunn.squarespace.com/storage/blunt_game_studies.pdf> Accessed 19th of May, 2013. Charsky, D. & Mims, C. 2008, "Integrating commercial off‐the‐shelf video games into school curriculums", TechTrends, vol. 52, no. 5, pp. 38‐44. Corbeil, P. 1999, "Learning from the children: Practical and theoretical reflections on playing and learning", Simulation & Gaming, vol. 30, no. 2, pp. 163‐180. DeKanter, N. 2004, "Gaming redefines interactivity for learning", TechTrends, vol. 49, no. 3, pp. 26‐31. Entertainment Software Asociation, 2012. Essential facts about the computer and video game industry. Available at < th http://www.theesa.com/facts/pdfs/ESA_EF_2012.pdf>. Accessed 19 of May, 2013 Hiltz, S. R., and Turoff, M., 2005 "Education goes digital: The evolution of online learning and the revolution in higher education." Communications of the ACM 59‐64. Lean, J., Moizer, J.,Towler, M., & Abbey, C. 2006 “Simulations and games. Use and barriers in higher education”. Active Learning in Higher Education 7, 3 227‐242. Available at <

783

Emilia Todorova and David Moffat http://peer.ccsd.cnrs.fr/docs/00/57/19/51/PDF/PEER_stage2_10.1177%252F1469787406069056.pdf > Accessed 22nd of May, 2013. Norman, D A. 1993, Things that make us smart: defending human attributes in the age of the machine, Perseus Books, Cambridge, MA Pappert, S. 1998 in Game Developer Magazine. Available at < http://www.papert.org/articles/Childpower.html> Accessed th 26 of October. Sica, L.S., Nigrelli, M.L., Rega, A. & Miglino, O. 2011, "The “Teaching to Teach with Technology” Project: Promoting Advanced Games Technologies in Education", Proceedings International Conference “The future of Education, pp. 169. Sica, L.S., Veneri, A.D. & Miglino, O. 2011, "Exploring New Technological Tools for Education: Some Prototypes and Their Pragmatical Classification", E learning/Book, vol. 1. Stapleton, A.J. 2004, "Serious games: Serious opportunities", Australian Game Developers‟ Conference, Academic Summit, Melbourne.

784

A Design Approach for Implementing 3D Educational Collaborative Virtual Environments on Virtual World Platforms Rosa Reis1, Benjamin Fonseca2and Paula Escudeiro1 ISEP, Porto, Portugal 2 UTAD, Vila Real, Portugal

rmr@isep.ipp.pt benjaf@utad.pt pmo@isep.ipp.pt Abstract: The collaborative virtual worlds can be used to achieve different purposes according to the intended use. The design of 3D collaborative virtual environments for learning has been an important research field for several years. However, the research in this specific field has shown that in most of the cases, the environments do not have appropriate technical characteristics. The design of the educational collaborative virtual environments goal is to produce environments that encourage users to achieve effective learning. In this sense, the current study presents a design model for development of 3D Educational Collaborative Virtual Environments. This model is based on the engineering software techniques and methods. It is supported by a spiral cycle that allows us to develop applications. The process is divided into a set of activities that are being carried out throughout each cycle, producing several work products, with the aim to provide each team member a set of guidelines and tools necessary for to make intelligent decisions about what they do. The model is composed by five steps, namely: Conception, Analysis, Design, Implementation and Evaluation. Each step contains a set of diagrams to support the developer team in their tasks. With this model, the applications are developed in a series of incremental releases, that is, the final system is constructed, based on the refined prototype. These steps include activities that enable to quantify the quality of Educational Collaborative Virtual Environments (ECVE). It is based on the Quantitative Evaluation Framework developed by Escudeiro (Escudeiro, 2007) and allow us to have a degree of freedom in the selection of quality criteria. Thus, we can obtain a single quantitative value of quality for any domain in analysis, i.e., we can adapt it in any domain and valence. Keywords: engineering software, assessment, lifecycle, collaborative virtual environments

1. Introduction Collaborative Virtual Environments (CVEs) are computer‐based 3D virtual worlds where users can meet and interact with each other in order to perform effectively collaborative work or have engaging experiences ‘being together’ [1]. When applied in education, these environments must be built taking into account some characteristics, such as instructions strategies, activities learning, communication, interactivity, type of media, contents, and metaphors. In most Education collaborative virtual environments (ECVE), these characteristics do not exist or have only some of them. This situation is mainly due to the lack of a organized systematic development process that assure the stimulation of new ideas, supported by the communication among the different elements of the development team or also because the developer team usually does not take account all phases of the software life cycle. In order to overcome these gaps, we decided to propose a design strategy that meets the purpose of the software process, i.e., that determines the order of the different system development phases and the evolution and the transition criteria between them. It is our intention that the design model covers all the development needs and provides a sufficient semantics of how to work properly all critical aspects of educational collaborative virtual environments. The model must be easy of use and it must aid to understand the system design; to stimulate new ideas, supported by a language that facilitates communication among the different actors of the development team; to promote the good practices, namely those that rules the engineering software. The present paper is structured in three sections. The section 2 describes our design strategy and in section 3, we draw some conclusions about the design strategy.

2. Design method proposal 3D Collaborative Educational Virtual Environments typically include complex information, and may allow sophisticated navigation behaviour. The proposed 3D Collaborative Educational Virtual Environments Design Method (ECVEDM) uses abstraction and software engineering techniques to, on one hand, allow a concise

785

Rosa Reis, Benjamin Fonsecaand Paula Escudeiro description of complex information items, and on the other hand, allow the specification of the complex navigation patterns and interface transformations. It is based on several assumptions, namely: promotes an organized structure by steps, where the activities are identified and also their rules; provides an adequate semantics of how to work properly all the critical aspects that have been highlighted in educational collaborative virtual environments; and encourages the creation of new ideas supported by the communication among the different elements of the development team. Given the complexity of collaborative virtual environments, it is important to do several revisions during the conception phase. The modification of some environment aspects leads us to different dynamic behaviours in order to have a life cycle model that allows modifications throughout its specifications. Also, it is necessary to look for the previous phases as well as for the improvement and exploitation of the solution space. In ECVEDM, 3D Collaborative Educational Virtual Environment is built in a five‐step process supporting an incremental or prototype process model. Each step focuses on a particular design problem giving rise to set of diagrams, Figure 1 summarizes the steps, products, mechanisms and design concerns in ECVEDM.

Figure 1: The ECVEDM method

2.1 Preliminary investigation and planning The preliminary investigation step allows the designer to specify a sequence of actions, namely the identification of the intended use of the system, objectives, profile of the users, selection of the tasks to support the interaction. The planning step includes a range of activities, which emphasize the system organization (including the team constitution and their responsibilities), the structuring of the tasks (WBS ‐ Work Breakdown Structure), project schedule, risk analysis and costs. These two steps are related, because they constitute a single phase, called the Conception phase. In this step, we delivered a document called Preliminary Draft Document that describes the work done in Preliminary Investigation and Planning steps. Its purpose is to determine the best way to conceive the environment and should contain information about the purpose of the project, team, tasks, feasibility study as well as the environment specifications. The specification of the environment is one of the most important parts of document, because the system's success depends on it. Here, we must carefully specify the objectives of environment, define the target population, specify the pedagogical model, define the instructional strategies, choose learning techniques, and describe the activities and the content of the design.

2.2 Analysis This step consists in modelling a real world system in order to be understood. Thus, we must analyse, specify and define the system.

786

Rosa Reis, Benjamin Fonsecaand Paula Escudeiro We propose two models: a requirements model and a analysis model. The first model aims to get the functionality of the system by using Use Case Diagram. This diagram is the basis of the design process and evaluation system, controlling a large part of the system under development (Jacobson, 2002). The analysis model consists in two diagrams: analysis diagram and storyboards (scenarios’ specification). The analysis diagram is drawn from the requirements model and the main objective is to facilitate the construction of the system. As the scenario becomes the main element of the environment, where users can develop their "actions" which will help them to acquire knowledge, it is our intent to identify subsystems. . So is easier identified the objects that compose the System. The storyboards have aim to give a global view of the ECVE, where we have clear idea of the speech, plot and other instructional questions. The design team and teachers share a continuous flow of assessments (corrections and/or suggestions) by sending and receiving versions of the storyboard. The organization of the storyboard is an important feature since it provides a clear view of what will be the ECVE.

2.3 Design The Design step comprises a set of steps that allow synthesizing, from the requirements, the representation of the structure data and programming the characteristics of the interface and procedures. This process is composed by two distinct phases, which consists in developing of an interaction model and navigation model. The interaction model is designed to visualize the interactions that the system contains, because if we look at "an interaction between a student (avatar) or students (avatars) and one environment (optionally including content resources, tools, instruments, systems computer services, real‐world events and objects) to held in response to a task intended to learning something "(Beetham, 2002) ‐ Learning Activity. The interaction model for ECVE becomes crucial. This model is composed by several diagrams, namely the Table of Action which identifies all participants of the system and their actions within the system and the Collaboration Diagram. The Table of Action is similar to the methodology used in X‐Tec (Escudeiro et al, 2006) and requires the creation of a matrix, where the tasks (actions) are described. The Collaboration Diagram is based on the Menchaca methodology (Menchaca et al., 2005), which involves the creation of social groups. Social groups are working groups, composed by students, who will perform some tasks in the environment. The navigation model defines a graphical view of the environment that allows the users to find information and encourage collaboration among the other participants. It aims to provide a graphical model showing the relationships between the various subsystems of the virtual space, i.e. a map navigation of the system.

2.4 Implementation All the specific environment features should be implemented (coded) according to the descriptions reported in the previous phases. It is not our intention to indicate the use of a specific programming language. The main goal is to assist in the development of an collaborative virtual environments, under a 3D virtual environments platform such as Second Life, Active Worlds, OpenSim, and others.

2.5 Assessment As the model is based on the spiral model, the system will be evaluated in each iteration. This assessment is based on the evaluation model, which is being developed. The goal of this step is to minimize the risks and the consequences of any change that could be require during the developing process. The ECVE developed will be assessed by using the evaluation model proposed by Escudeiro (Escudeiro, 2007), which is based on the framework called QEF (Quantitative evaluation Framework). We chose this framework because it allows us to identify a set of requirements for the ECVE. We are currently identifying these requirements and after this stage we will evaluate the systems and publish the results. The figure 2 shows the evaluation process of the ECVEDM.

3. Case study To a better understand the decisions taken and to verify the effectiveness of the design model of ECVEs on virtual world’s platforms, we decided to move forward with the creation of an educational collaborative virtual environment prototype. This prototype will help support a math class of 5th grade of Basic Education. The topic to be addressed is related to the Geometric Solids. The target audience is children 10 to 11 years old. We used the OpenSimulator platform.

787

Rosa Reis, Benjamin Fonsecaand Paula Escudeiro

Figure 2: Model of process evaluation We begin the conception of system with the identification of the essential needs for system development, where were defined the strategies of interaction and the tasks according to the learning outcomes. After, in the analysis phase, we identified the functionalities of the environment, in terms of operations and system conditions, putting aside the details of the platform on which will develop the environment. Based on the information of conception phase, we decided to divide the space into three main areas: information space, discussion space and learning space. This guideline provides help on identifying and detailing use cases and scenarios. We use the step‐by‐step description of the use‐case main scenarios. Then, gradually we added details to these scenarios, describing the use cases and describing scenarios through storyboard , as we can see in figure 3..

Figure 3: Use Case diagram and a learning scenario From the use case diagram and the scenarios descriptions we can identify a list of operations to draw the analysis diagram (see figure4) After building these artifacts, we have begun to the design phase in order to determine how we can facilitate the interaction between the users and their environment. We had the concern of building the activities which consider the challenge, curiosity, control and fantasy. Also, it was necessary to detail the interaction techniques associated with learning activities for drawing up the collaboration diagram (see table 1), which allowed us to organize the actions by different social groups. Each group partially defines how the elements interact with other groups.

788

Rosa Reis, Benjamin Fonsecaand Paula Escudeiro

Figure 4: Use case diagram and a learning scenario Table 1: Descriptions of functions services, restrictions of group each Tasks

Function

Services Restrictions

Social Group I (student) Social Group II (teacher) Observes the picture Choose/Write the correct option Ask for help Give suggestions Communicates with the Communicates with avatars environment and avatars Display questions Ssuggests new reading on the subject Reports on the choice Send emails Just done the test once. If the score for each activity is less than 50 points should suggest reviewing matters related with activity

4. Conclusion The development of ECVE is a complex task since it requires knowledge on different fields such as human‐ machine interface, design, education. In this sense, we decided to develop a model that (1) facilitates the communication between all team members and (2) whose flexibility can be used for the whole ECVE lifecycle or just for a set of processes. The model covers all phase’s lifecycle of software development, using different software engineering techniques. Also, the model is supported by an evaluation model, which is applied during the assessment step. The main objective is to structure the process of assessment based on the Quantitative Evaluation Framework [3], because it allows to measure quality of the final product, and evaluate the systems quality at any moment during its lifecycle. However, this study still needs to be refined to highlight the benefits of ECVE development through a model that structures all process of construction environments, where the teachers can using in their classrooms..

References Beetham, H. (2002) “Developing learning technology networks through shared representations of practice, Proceedings of the 9th International Improving Student Learning Symposium, 421‐434, Oxford: OCSLD Escudeiro Paula (2007) X‐Tec Model and QEF Model: A case study, in T. Bastiaens & S. Carliner (Eds.), Proceedings of World Conference on E‐Learning in Corporate, Government, Healthcare, and Higher Education 2007, Chesapeake, VA: AACE, 2007, , pp. 258‐265. Escudeiro, P., Bibarra, J.,(2006) X‐TEC: Techno‐Didactical Extension for Instruction/Learning Based on the Computer, Actas da Conferência SITE 2006, Orlando, FL, E.U.A. Jacobson, Ivan (2002) Object Oriented Systems Engineering, Addison‐Wesley, 1992 Improving Student Learning Symposium, 421‐434, Oxford: OCSLD. Menchaca, R., B. Ballares, R. Quintero and C. Carreto (2005) Software Engineering, HCI Techniques and Java Technologies Joined to Develop Web‐Based 3D Collaborative Virtual Environments, Proceeding CLIHC '05 Proceedings of the Latin American conference on Human‐computer interaction. Reis, R., Benjamin F. and Escudeiro, P.(2012) Assessment Model for Educational Collaborative Virtual Environments, Proceedings of 7th International Computer Science & Education (ICCSE), 2012, 1555 – 1559.

789

EMOTE: Embodied‐Perceptive Tutors for Empathy‐Based Learning in a Game Environment Sofia Serholt1, Wolmet Barendregt1, Tiago Ribeiro2, Ginevra Castellano3, Ana Paiva2, Arvid Kappas4, Ruth Aylett5 and Fernando Nabais6 1 Department of Applied IT, IT faculty at the University of Gothenburg, Gothenburg, Sweden 2 GAIPS, INESC‐ID and Instituto Superior Tecnico, Lisboa, Portugal 3 Centre for Human‐Computer Interaction, University of Birmingham, Birmingham, UK 4 School of Humanities and Social Sciences, Jacobs University, Bremen GMBH, Germany 5 School of Mathematical and Computer Science, Heriot‐Watt University, Edinburgh, UK 6 YDreams Robotics S.A, Portugal sofia.serholt@ait.gu.se wolmet.barendregt@ait.gu.se tiagrib@gmail.com g.castellano@bham.ac.uk paiva.a@gmail.com a.kappas@jacobs‐university.de r.s.aylett@hw.ac.uk fernando.nabais@ydreamsrobotics.com Abstract: Significant work has been devoted to the design of artificial tutors with human capabilities with the aim of helping increase the efficiency achieved with a human instructor. Yet, these systems still lack the personal, empathic and human elements that characterise a traditional teacher and fail to engage and motivate students in the same way a human teacher does. The EU‐funded project EMOTE (EMbOdied‐perceptive Tutors for Empathy‐based learning) has recently started, and will continue until the end of 2015. The project aims to design, develop and evaluate a new generation of virtual and robotic embodied tutors that have perceptive capabilities to engage in empathic interactions with learners in a shared physical space. In this paper we wish to discuss the approach we are taking in the project as well as how the project may contribute to knowledge relevant for the Games‐Based Learning community. Keywords: geography, embodied tutors, empathy‐based learning, embodied pedagogical agents, games‐based learning, sustainable development

1. Introduction Animated pedagogical agents, embodied as a 2D or 3D character, with social cues integrated in learning environments may engage and motivate students and help them to achieve greater learning efforts (Chen et al. 2012, McQuiggan and Lester 2007). Recent research on comparing robots with their virtual representations has furthermore demonstrated that the robotic embodiment was preferred by users in terms of social presence (Kidd 2003), enjoyment (Pereira et al. 2008) and performance (Saerbeck et al. 2010). Moreover, embodied robotic tutors that are not only able to display social cues but also have the ability to perceive emotions experienced by learners, and to incorporate these into pedagogical strategies may be even more beneficial for learning (Burleson 2006). In the EMOTE project, that will run from December 2012 to December 2015, the focus will be on embodied robotic tutors that are able to understand learners’ affective and motivational states, adapt to the learners’ needs and to display appropriate responses in order to engage and motivate children when learning geography in a game‐like environment. Therefore, automatic recognition of the user’s affective state is of primary importance for a virtual agent or robot to establish an affective loop with the user, through generation of an appropriate response (Castellano et al. 2010).

790

Sofia Serholt et al.

1.1 The set‐up and geography games The robotic tutor, an Aldebaran NAO torso, will interact with the students in a shared physical space with the support of maps and charts on a MultiTaction 55” multi‐touch table. The target group will be children between the ages of 11 and 13 in geography or sustainable development classes. The tutor will shift between guided and discovery learning approaches depending on the topic that is taught and each student’s needs. The multi‐ touch table will comprise two separate educational scenarios on geography‐related topics, of which the first will be a questlike virtual learning environment focusing on map exploration. A game developed by Brøderbund in 1985 titled ‘Where in the world is Carmen Sandiego?’, focusing on world geography, flags, currencies, etc. incorporates some similarities. However, EMOTE’s set‐up with a multi‐touch table is more suitable for spatial activities, such as map reading. Another recent geography game relevant for EMOTE, designed to adapt “to individual learners, their prior knowledge, abilities, preferences, and learning progress” (Kickmeier‐Rust and Albert 2009) is 80Days (www.eightydays.eu). Aside from adaptivity within the game, EMOTE will also explore the impact of adapting the robotic tutor’s behaviour to the emotional state of the player. The second scenario will focus on sustainability and will be based on an existing game EU‐funded serious game called EnerCities, which deals with sustainable energy awareness. Both scenarios will be designed to gradually encourage students to apply self‐directed learning strategies. After having experienced the interaction with the embodied tutor through the multi‐touch table, the students will be able to work with a virtual version of the tutor in a different setting on a handheld device.

2. Work in progress Currently, researchers in the participating countries are working alongside geography teachers through interviews, workshops and prototype studies, in order to develop the content and structure of the two scenarios, as well as the empathic behaviour of the tutor. First, several interviews were held to understand the teachers’ needs for and feelings towards a robotic tutor in the classroom. Teachers were generally rather welcoming towards a robotic tutor if it could provide their pupils with an individualised learning approach that they cannot give themselves due to time constraints. However, they also wanted to have a clear picture of the activities the child and the tutor had gone through, meaning that the tutor/application should log interactions and provide teachers with an overview of students’ progress. In addition, a concern was raised regarding the importance of ensuring that the amount of time spent with the tutor is evenly distributed among the students in order to avoid conflicts, while not placing additional administrative work load on the teacher. Furthermore, teachers generally advocated for a multi‐player mode where 2‐4 students may participate. In such a setting, the game could be integrated into a thematic, collaborative, class project on sustainable development addressing different aspects of environmental, economic and social sustainability. While most of the teachers usually would place themselves outside the game as observers or moderators of discussion, they also recognized the possibilities for the tutor to function as an interactive learning partner or peer‐learning agent actually playing the game on a student level, or even taking the role of a tutee in need of help from the other players, consequently providing the students with increased influence and responsibility in completing the game. We are planning to study the effects of these different tutoring approaches in the remainder of the project. After the initial interviews, we gathered the official learning aims for geography and sustainable development education in each of the participating countries. Based on the interviews and the learning aims, a first learning scenario was defined. In this scenario the actions of the robotic tutor and the multi‐touch table were described in detail as well as some envisioned reactions of the children on the tutors’ instructions. The learning aims supported by this scenario as well as the scenario itself were then discussed once more with the teachers.

2.1 Next steps A subsequent step will be to refine the scenario in collaboration with the teachers, especially focusing on how a real teacher would behave in similar situations. Our goals will be to design the teaching strategy (guided versus discovery learning) by asking teachers to design responses based on students’ different levels of proficiency in the subjects; identify common errors (in order to detect them computationally) by asking teachers to pinpoint children's difficulties, to ensure that the system is easy to understand, and to design

791

Sofia Serholt et al. assessment activities that can evaluate learning outcome. Thereafter, we will develop a visual layered paper prototype which the teachers will use with students in a mock‐up study. At least two sessions comprising students in need of varying amounts of pedagogical support will then be videotaped and analysed. The goals of this mock‐up study are to find out how the teaching strategy is expressed in reality, to verify and expand on the common errors made by the children through actual observation, to evaluate that the planned tasks are feasible and do not introduce unnecessary confusion, and to get a corpus that will be annotated for empathy and teaching strategy in order to design the concrete behaviours of the robot.

3. Evaluation Evaluation of the tutors and the application is no easy task; we will have to consider effects and affects, usability, user experience (UX) as well as game experience. Therefore we will start evaluating prototypes of the tutors and applications already during the first year of the project, especially concerning usability and UX. Towards the end of the project we will evaluate the tutors and the games in a small scale longitudinal study in which we explicitly intend to evaluate ecologically valid classroom use and the added value of the robots from a teacher perspective. The determination of proper evaluation methods for the longitudinal study based on relevant literature and research will begin in the second year of the project. It is believed, however, that an important basis for evaluating our set up will be to utilize teachers’ existing experience and expertise in assessing teaching materials and learning outcome in general. Thus, we intend to explore methods of including teachers in the actual assessment of students’ learning outcome. We hope that by paying special consideration to the users, both students and teachers, these findings may provide insights for future work on game‐based learning materials aimed for authentic school settings.

4. Conclusions This paper presented some initial steps in designing, developing and evaluating embodied empathic virtual and robotic tutors in a game‐based virtual learning environment. Also, the EMOTE project aims to investigate whether the use of collaborative serious games can facilitate students’ awareness and understanding of sustainable development. Most importantly, the project intends to contribute with knowledge about the role that an empathic tutor able to alter between different pedagogical strategies can play when children collaboratively or individually engage in game‐based learning activities through a multi‐touch table.

Acknowledgements We would like to thank the teachers at Leteboskolan, Horred. This work was supported by national funds through FCT Funda ção para a Ciência e a Tecnologia, under project PEst‐OE/EEI/LA0021/2013 and by the EU 7th Framework Program project EMOTE (FP7/2007‐2013) under grant agreements no. 215554 and 317923. It does not represent the opinion of the EC, and the EC is not responsible for any use that might be made of data appearing therein.

References Burleson, W. (2006). Affective Learning Companions: Strategies for Empathetic Agents with Real‐Time Multimodal Affective Sensing to Foster Meta‐Cognitive and Meta‐Affective Approaches to Learning, Motivation, and Perseverance. PhD, MIT. Castellano, G., Leite, I., Pereira, A., Martinho, C., Paiva, A. & Mcowan, P. W. (2010). Affect Recognition for Interactive Companions: Challenges and design in real world scenarios. Journal on Multimodal User Interfaces, 3, 89–98. Chen, G.‐D., Lee, J.‐H., Wang, C.‐Y., Chao, P.‐Y., Li, L.‐Y. & Lee, T.‐Y. (2012). An Empathic Avatar in a Computer‐Aided Learning Program to Encourage and Persuade Learners. Educational Technology & Society, 15, 62‐72. Kickmeier‐Rust, M. D. & Albert, D. (2009). Emergence in digital educational games: A world of incidents in a universe of rules. In: PIVEC, M. (ed.) 3rd European Conference on Games‐based Learning (ECGBL). Graz, Austria. Kidd, C. (2003). Sociable Robots: The Role of Presence and Task in Human‐Robot Interaction. McQuiggan, S. W. & Lester, J. C. (2007). Modeling and Evaluating Empathy in Embodied Companion Agents. International Journal of Human‐Computer Studies, 65, 348‐360. Pereira, A., Martinho, C., Leite, I. & Paiva, A. (2008). iCat, the chess player: the influence of embodiment in the enjoyment of a game. In: PADGHAM, PARKES, MÜLLER & PARSONS, eds. 7th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2008), 2008 Estoril, Portugal. Saerbeck, M., Schut, T., Bartneck, C. & Janse, M. D. (2010). Expressive Robots in Education ‐ Varying the Degree of Social Supportive Behavior of a Robotic Tutor. CHI 2010. Atlanta, Georgia, USA.

792

Exploring Learning Effects During Virtual Sports Through Biomechanical Analysis (a Work in Progress) Pooya Soltani and João Paulo Vilas‐Boas Porto Biomechanics Laboratory (LABIOMEP), Faculty of Sport, University of Porto, Porto – Portugal 111114024@fade.up.pt jpvb@fade.up.pt Abstract: In this work‐in‐progress, we are comparing the kinematics of movement during playing virtual swimming exergame between novice and experienced players in order to detect possible learning effects. Ten participants performed a 100‐meters front crawl virtual swimming using Xbox and Kinect. A 12 camera Qualisys motion capture system, operating at 200 Hz, tracked the position of 22 reflective markers. Preliminary results indicate that novice players showed a pattern more similar to the real swimming spending more time on accuracy of movement while the expert player used an adaptive circular pattern in order to win the game. Keywords: Exergame; virtual sport; learning; biomechanics

1. Introduction Since 2000, new types of video game consoles such as Microsoft Xbox and Kinect, Nintendo Wii or Sony Move often tagged as Exergames have been introduced in which users have to interact physically with the characters inside the game. However, claiming that Exergames can promote an active lifestyle, they don’t actually increase levels of physical activity but the motivation in performing some movements (Peng et al., 2013). In most of Exergames, the goal is to mimic the movements of avatars and get points based on the movements. One of the popular game genres is virtual sports. They encourage participants to do the activity as they are really playing the real sport. These systems use several methods to detect the movements. Adaptations to active video games may happen after playing for some time. In fact most of the users start playing technically rather than emotionally which means that they will learn how to win the game with less effort (Pasch et al., 2009). One of the causes is due to specifications of the platform used. For example, Nintendo Wii uses a sensor which detects the movement with an internal accelerometer. Although, the optimum goal is to perform the movements completely in virtual sports, there is always a chance of “cheating” due to learning because the user can easily fool the accelerometer by using one’s wrist to manipulate the controller (Lavac et al., 2009). This potential often discovered through experience or learning others’ styles of playing. On the other hand, as there is no exact force confronting players’ actions, there are some concerns regarding efficacy and safety during playing Exergames especially when high exposure to the game is expected. Characterizing Exergames biomechanically allow designers to create more realistic games for a more meaningful experience; will allow hardware developers to identify the limitations of their platforms (accelerometer sensors, Infrared sensors, EMG based sensors) in order to minimize the effects of learning; reducing musculoskeletal injuries (accidents due to limitations of players both in terms of space and muscle activity), and creating additional resources (i.e. warm up and cool down periods). Therefore the purpose of this work‐in‐progress is to compare quantity of movements between novice and experienced players to characterize learning occurring during playing virtual sports.

2. Methods Ten healthy subjects (6 men and 4 women; Age: 22‐31) played “Michael Phelps: Push the Limit” virtual swimming game using Xbox 360 and Kinect. Subjects were considered as “novice” if their ranking during the game was 1st to 4th and “expert” if their position was 5th to 8th. Subjects were asked to refrain from exercising the upper body at least 48 hours before the testing sessions. All testing was conducted on the dominant arm. Virtual Swimming: Subjects had to stand in front of the Kinect sensor and bend forward and as soon as they saw the “Go!” command, they had to return back to normal position with arms in front. After that, they had to swing their arms in order to move the avatar in the game (100 meters front crawl swimming). At the end of the

793

Pooya Soltani and João Paulo Vilas‐Boas event, they had to drop both hands and then raise one of them to finish the race. In order to prevent the player from swimming too fast or too slow, there is a spectrum on the screen with a blue zone in between which indicates if the speed is at the moderate level. At the middle of the second lap, there is a possibility to swim as fast as possible called “Push the Limit”. Collection of Kinematics data: Dominant arm’s movements were recorded at 200 Hz with a 12‐camera Qualisys motion capture system (Qualisys AB, Gothenburg, Sweden). Before each experiment, the cameras were calibrated to the measurement volume. 22 spherical reflective markers were placed over the skin. Collection of data was started when the subjects placed at “ready” position over a period of one minute. The data were collected in acquisition software (Qualisys Track Manager). Swimming technique and definition of cycles: The event was divided into five phases as suggested by Maglischo (1993). Based on the speed (Normal Vs. Fast), novice and expert players were compared. The swimming cycle began with stretching the right arm forward which is considered as “Entry and Stretch”. As the so called propulsive phase of left arm finished, a phase called “Downsweep” starts. Some characteristics of this phase are different than real swimming due to the position of the body. Following this phase, the hand moves in a circular sweep termed as “Insweep”. Hand direction changes to out, back, and up in a term called “Upseep”. This phase finishes as the hand passes the thigh where its direction changes from back and up, to up and forward which is names as “Recovery” phases. Statistical Analysis: All data was statistically analysed using SPSS version 20. Comparisons between expert and novice players on selected parameters were conducted using ANOVA. Statistical significance was accepted at p ≤ 0.05.

3. Results Virtual swimming performance, number of cycles, start, average and max velocity and angular distance covered during the event are presented in Table 1. A sample movement pattern created by a marker placed on the dominant hand in line with the index finger for both novice and expert players were shown in Figure 1. As shown especially in superior view, novice players performed closer to real swimming following swimming phases in their performance while the expert players were creating a circular pattern. Time Motion Analysis and percentage of time dedicated to each phase is shown in Table 2. Expert players had lower average velocity (2.48±0.20 Vs. 3.27±0.22m.s‐1) and lower covered distance (113.28±11.54 Vs. 165±12.62m). However, the differences in performance level were not significant for front crawl parameters. (Performance Level: F (1, 7) = 1.008, p = 0.648). Novice players had significantly lower angular distance covered (F (1, 8) = 8.085, p = 0.021)

NOVICE 7.14±1.74

794

Angular distance covered (m) 165.00±12.62 NOVICE

EXPERT 5.91±1.10

Numbers are expressed in Mean±SD.

113.28±11.54 EXPERT

NOVICE 3.27±0.22

Max velocity: push the limit phase (m.s‐1)

Average velocity (m.s‐1) EXPERT 2.48±0.20

EXPERT 5.06±1.64

NOVICE

NOVICE 46 [11]

3.17±0.39

EXPERT 36 [8]

EXPERT

NOVICE 57.00±3.74

2.51±0.52

EXPERT 48.40±0.89

NOVICE

NOVICE 5

4.87±1.52

EXPERT 5

Start velocity (m.s‐1)

Total number of cycles [fast]

LEVEL OF EXPERIENCE FRONT CRAWL

Number of players

Total time to complete the event (s)

Average velocity [normal] (m.s‐1)

Table 1: Kinematics of 100‐meters front crawl virtual swimming between novice and expert players.

Pooya Soltani and João Paulo Vilas‐Boas

Figure 1: Sample hand pathways in one complete cycle in front crawl Table 2: Time motion analysis of different phases of virtual swimming between novice and expert players FRONT CRAWL EXPERT NOVICE

PHASES (% TIME) [FAST SWIMMING] ENTRY 25±5 [25±1] 30±7 [30±6]

DOWN+CATCH+INSWEEP 28±5 [25±3] 29±4 [27±4]

UPSWEEP 19±7 [20±2] 16±2 [19±3]

RECOVERY 26±3 [27±2] 21±5 [22±5]

Numbers are expressed in Mean±SD.

4. Discussion The purpose of this study is to detect the learning effects occurring during playing virtual sport exergames through biomechanics. The preliminary results of this work‐in‐progress shows that in general, the movement pattern of novice player was closer to real swimming. Expert player focused more on completion of the circular movement. Expert players finished the event in a shorter time and generally had lower stroke cycles to complete the event. This was because novice players wanted to adapt their movements to the spectrum and sometimes, they were swinging their arms slower to decrease the speed of avatar. The start velocity was higher in novice player and it was due to the fact that they were bending their backs more than expert players which took more time to extend. Average velocity was lower in expert players and it was because of the consistency of their movements. As expert players had less numbers of cycles, angular distance covered was also lower in expert players. The expert players dedicated more time to recovery phase as they were creating a circular pattern while novice players spent more time to down+catch+insweep phase which shows that they were following the real swimming pattern. In our observation, as novice players increase their speed (Push the Limit), they switched from doing the movements correctly to do the phases faster. Expert player prioritized their movements to a more circular pathway. As expert players had more experience in playing the game, they perceived that accurate real swimming movements are not necessary to play the game. Previously virtual reality is being used in teaching several training programs (Snyder et al., 2011). As these gaming environments provide same scenarios (Miles et al., 2012) and because of time constrains, virtual sport exergames might be a good alternative to be considered in accompany with traditional training. Biomechanical analysis of the games may provide an image on how reliable these systems are in terms of teaching and structuring practice. Exergames may benefit players physiologically but since the chance of cheating exists due to limitation of platforms; detailed characterization of games may provide valuable information toward the feedback these

795

Pooya Soltani and João Paulo Vilas‐Boas games provide. Considering learning effects of exergames, our preliminary results showed that expert player even had more movement compared to the novice player. However, more data with more subjects are needed for a better conclusion. Expert player adapted his movement in order to win the game while the novice player was more focused on the accuracy of movements. It seems that the Kinect sensor is not able to detect the delicate movements which may be a good reason to switch from performing real movement to adaptive movements in exergames.

References Levac, Danielle, Pierrynowski, Michael R., Canestraro, Melissa, Gurr, Lindsay, Leonard, Laurean, & Neeley, Christyann. (2010) "Exploring children’s movement characteristics during virtual reality video game play" Human Movement Science, Vol. 29, No. 6, pp 1023‐1038. Maglischo, E. W. (1993) Swimming even faster. Mayfield Publishing Co., Mountain View, Calif.; United States. Miles, Helen C., Pop, Serban R., Watt, Simon J., Lawrence, Gavin P., & John, Nigel W. (2012) "A review of virtual environments for training in ball sports" Computers & Graphics, Vol. 36, No. 6, pp 714‐726. Pasch, Marco, Bianchi‐Berthouze, Nadia, van Dijk, Betsy, & Nijholt, Anton. (2009) "Movement‐based sports video games: Investigating motivation and gaming experience", Entertainment Computing, Vol. 1, No. 2, pp 49‐61. Peng, Wei, Crouse, Julia C., & Lin, Jih‐Hsuan. (2013) "Using Active Video Games for Physical Activity Promotion: A Systematic Review of the Current State of Research", Health Education & Behavior, Vol. 40, No. 2, pp 171‐192. Snyder, Christopher W., Vandromme, Marianne J., Tyra, Sharon L., Porterfield, John R., Jr., Clements, Ronald H., & Hawn, Mary T. (2011) " Effects of virtual reality simulator training method and observational learning on surgical performance", World Journal Of Surgery, Vol. 35, No. 2, pp 245‐252.

796

siLang: Culturally Oriented Language Skill Development in Line With Workplace Needs Hariklia Tsalapatas1, Olivier Heidmann2, Rene Alimisi2 and Elias Houstis2 1 Univeristy of Thessaly, Greece 2 CERTH/IRETETH, Greece htsalapa@uth.gr olivier.heidmann@gmail.com ralimisi@gmail.com enh@uth.gr Abstract: This paper introduces siLang, a serious gaming approach for situated learning of vehicular languages. Otherwise referred to as lingua franca, these languages are commonly understood and used across communities. They facilitate effective communication by individuals not sharing a mother tongue. In the context of ever increasing international collaboration, this work takes a new approach in language learning building on the added‐value introduced by vehicular languages as cross‐cultural communication tools. siLang takes into account cultural aspects by exposing learners to the use of vehicular languages by native as well as non‐native speakers. The project integrates into a serious game culturally‐ influenced communication norms and language transfer effects including diverse pronunciations, expressions, and idioms through scenarios that draw inspiration from real‐life. The learning game is supported by good practice recommendations targeting language instructors facilitating the effective integration of proposed situated, game‐based learning into language education with applications in wider learning settings. The development of the serious game is work in progress. Validation activities will take place in Greece, Norway, Italy, Portugal, and Estonia involving undergraduate students, professionals, and vocational workers. Keywords: serious games, foreign language learning, situated learning, enculturation, ‘foldback’ game structure, narrative immersion

1. Introduction: Serious games in various domains Serious games serve purposes within various sectors: health, military, business, socio‐political, and education. They offer educational, training, and strategy development opportunities that stem from simulation‐based scenarios, ‘habituation’, and enculturation (Ulicsak and Wright, 2010, p.54). In the health sector, serious games are used for training practitioners, including perspective doctors, nurses, surgeons, or others. Some games, such as the Safros project, simulate human organ functionality offering opportunities for virtually identifying malfunctions. Others provide a virtual but realistic environment for simulating the execution of surgeries providing practice to trainees on a series of medical scenarios; examples include the Dental Implant Training Simulation and the Hollier Simulation (ibid). In business, serious games are recognized as tools that have the capacity to influence change in operational processes. There is an increasing interest on serious games for internal training and professional education (Meister and Willyerd, 2010). It is foreseeable that in the future games may dominate professional skill development practices. Other serious games are not directly linked to formal or professional education but help individuals develop skills useful for their fruitful engagement in public and social life. An example is Climate Change (Ulicsak and Wright, 2010). In the military, serious games offer alternative training for dangerous, expensive, time‐consuming, hard to plan, and not easily repeatable or reviewed practices (Ulicsak and Wright, 2010). Some simulate reality, for example flying a helicopter; others focus on strategic planning, decision making, and communicative norms in military life. Examples include ARMA 2, TLTCS, and more (ibid). Serious games are gradually emerging as tools for language learning and teaching (Meyer and Sorensen, 2009; Johnson, 2010; Amoia et al, 2012). While they are not yet part of formal learning practices they are warmly accepted in informal educational settings. Advantages of serious games in language education include the realistic environment they provide and access to authentic. ‘Intrinsic serious game design’ for language learning moves beyond drill‐ and‐ error tasks to support ‘fruitful thinking, real language interaction, and learner engagement (Meyer and Sorensen, 2009, p. 291). Serious games offer the potential to ‘enculturate’ the

797

Hariklia Tsalapatas et al. user in a community of practice or realistic environment that is rich in authentic content and resources. However, learning outcomes from the use of serious games in language learning are not guaranteed. As Mayer and Bekebrede (2006) advocate games may trigger mechanisms towards learning but this does not suffice; the educational design, the embedded pedagogy, and the role of the teacher play an important role in achieving significant learning outcomes. siLang takes into account theoretical debates and recent developments in the area of serious games to enhance language competencies relevant to the workplace. The project applies situated methodologies within language learning games aiming to enrich communication skills related everyday as well as business contexts in diverse cultural environments through effective use of vehicular languages.

2. Foreign language learning in the context of siLang Upgrading training practices is important in order to keep pace with changing career contexts and a rapidly evolving labour‐market (Cedefop, 2009). Business‐related mobility is the norm for professionals, academics, vocational workers, and higher education students. Individuals are often called to communicate in a foreign language in cultural settings that diverge significantly from their own and from one country to another. This takes place in the context of not only physical but also virtual mobility enabled by on‐line tools and communities (Tsalapatas et al, 2013). To meet work objectives, professionals often rely on vehicular languages, such as English, German, or French. Existing training programs that exploit new technologies mainly focus on helping professionals achieve working level command of a particular foreign language in a short period of time, build basic vocabulary for everyday activities in a foreign tongue, or to use terms related to a particular profession, such as engineering, tourism, and more. However, most widely accessible language learning approaches fail to integrate the diverging use of a lingua franca by non‐native speakers (Tsalapatas et al, 2013); siLang addresses this gap in professional language education by exposing users to the diverging use of a lingua franca in varying cultural and work environments, which is affected by transfer effects, localized expressions, and communication norms. siLang follows an end‐user centered implementation approach. To ensure that outcomes meet the learning and teaching needs of the targeted professional and vocational community the project starts by analyzing typical strengths and challenges of the training schemes and offerings in the countries represented in the project consortium (Greece, Estonia, Italy, Portugal and Norway) as well as typical issues that non‐native speakers encounter when communicating a foreign tongue. siLang proposes a learning intervention that exploits serious gaming technologies and uses a didactical framework which builds on situated learning, problem‐based approaches and role‐playing practices (Tsalapatas et al, 2013). Theory is validated in practice through the design and technical implementation of the siLang serious game and corresponding educational supporting resources and services. In the following section, the siLang ideas underpinning the design of the siLang serious game are brought into focus. The siLang methodology is broad and can be applied for building communication capacity in any language. For validation purposes the implementation focuses on learning English as a lingua franca.

3. siLang serious game design and structure The siLang serious game design is based on story‐telling approaches that combine focused, realistic language learning scenes into complete learning scenarios. Storylines are situated in real‐life experiences and immerse learners into the diverse uses of a ‘lingua franca’ in various cultural contexts. They aim to expose users to typical daily and business communication taking into account localized expressions, communication norms, and transfer effects manifested, for example, in pronunciation or syntax. Examples of siLang learning scenes include communication in international settings, presenting ideas during a meeting, starting and finishing a meeting, greetings, scheduling/ timetabling, distance collaboration through popular communication services, and more. An example scenario that combines multiple scenes may be: a professional receives information via email on a business meeting, travels to the business location, uses local transport to go from the airport to the hotel, checks into the hotel room, and orders dinner. These activities are related to communication in the context of business travel but do not directly refer to business activities. The scenario may continue with professional communication–related plots including the user introducing herself during a meeting, presenting interests, noting tasks for implementation, and closing the meeting.

798

Hariklia Tsalapatas et al. The game deploys ‘narrative immersion’ (Adams, 2010); it creates a feeling of acting inside a story. Scenarios are developed using flexible ‘foldback’ game story structure that links multiple scenes. The siLang plot can ‘branch several times but it will fold back to one inevitable event before branching again and folding back to another inevitable event’ (Adams, 2010, p.174). Each scene exposes the user to a certain language task and challenge. Once a task is completed another scene ‘unlocks’ based on the user’s preferences and the plot unfolds. A central ‘hub’ is used to set ‘plot‐critical situations’ that the learner cannot ignore or alter (Adams, 2010). Foldback scenario design offers users a sense of agency and control of the game while it significantly eliminates the cost and complexity of developing a fully branching plot (Adams, 2010). Learners receive continuous feedback on their performance through a ‘digital agenda’ that documents the storyline path already completed. More details are provided through the digital agenda on the learner’s achievements in each individual learning scene in relation to learning objectives. Additional feedback can be provided by the teacher when the learning game is used in the context of structured class activities. Collaborative use of the game through comparison of learner responses may further enrich classroom activities through gamification. The siLang serious game is under development as an animated application. Non‐native speaker voices representative of a particular cultural community contribute to immersing learners to a particular cultural environment. For example, a learner that uses the game to prepare for a business trip to Italy is exposed to Italian pronunciation and cultural norms. The siLang serious game will support both traditional keyboard and emerging touch screen interfaces aiming a wide adoption.

4. Instead of conclusion This paper presents work in progress related to the design and implementation of siLang serious game for developing communication and cultural awareness skills in relation to vehicular languages. The siLang serious game encompasses contextually‐based and habitual learning experiences with a focus on professional and workplace needs. This paper does not aim to present final outcomes but rather the theoretical framework that underlines the siLang proof‐of‐concept language learning game under development. The siLang serious game, services, and methodologies will be evaluated in Greece, Norway, Portugal, Estonia, and Italy involving groups of learners, and specifically vocational workers, professionals and high education students, and theirs trainers. Validation activities will take place in an on‐going manner providing insight on the relevance and impact of the siLang approach and implementation towards business related communication skill development.

Acknowledgements This work is funded with the support of the Life Long Learning Programme of the European Commission and specifically KA2: Languages and runs from 2012 to 2014. This publication reflects the views only of the author and the Commission cannot be held responsible for any use which may be made of the information contained therein.

References Adams, E. (2010) Fundamentals of Game Design (2nd Edition.) New Riders Publishing. ISBN 0‐321‐64337‐2. Amoia, M., Bretandiere, T., Denis, A., Gardent, C., Prez‐ Beltrachini, L. (2012) ‘A Serious Game for Second Language Acquisition in a Virtual Environment’. Journal on Systemics, Cybernetics and Informatics (JSCI) 10, 1, 24‐34. Cedefop. (2009) ‘Continuity, consolidation and change. Towards a European era of vocational education and training’. Cedefop Reference series; 73 Luxembourg: Office for Official Publications of the European Communities. Johnson, W.L. (2010) Serious Use of a Serious Game for Language Learning, International Journal of Artificial Intelligence in Education. V. 20, p. 175‐195, DOI 10.3233/JAI‐2010‐0006, IOS Press Mayer, I. and Bekebrede, G. (2006) Serious games and simulation based e‐learning for infrastructure management. In Affective and emotional aspects of human‐computer interaction: emphasis on gamebased and innovative learning approaches. (ed) M Pivec. Amsterdam: iOS Press Meister, J.C. and Willyerd, K. (2010) The 2020 workplace: How innovative companies attract, develop, and keep tomorrow’s employees today. New York: HarperCollins Publishers. Meyer, B. and Sorensen, B. (2009).’Designing Serious Games for Computer Assisted Language Learning – a Framework for Development and Analysis' [online] http://link.springer.com/chapter/10.1007%2F978‐1‐4020‐9496‐5_5?LI=true Tsalapatas, H., Heidmann, O., Alimisi, R., Houstis, E. (2013). ‘A serious game‐based approach for situated learning of vehicular languages addressing work needs and cultural aspects’. 7th International Technology, Education and Development Conference, Valencia, Spain. 4‐5 March, 2013. ISBN: 978‐84‐616‐2661‐8. Publisher: IATED Ulicsak, M. and Wright, M. (2010). ‘Games in education: Serious Games’ [online] http://media.futurelab.org.uk/resources/documents/lit_reviews/Serious‐Games_Review.pdf

799

Developing Ethical Decision Making Skill of Novice Volunteers in Natural Disaster Response Didin Wahyudin1, 3, Shinobu Hasegawa2,1 and Tina Dahlan3 1 School of Information Science JAIST, Nomi‐Ishikawa, Japan 2 Center for Graduate Education Initiative JAIST, Nomi‐Ishikawa, Japan 3 Department of Psychology, Indonesia University of Education, Bandung, Indonesia didin.wahyudin@jaist.ac.jp hasegawa@jaist.ac.jp tinadahlan@upi.edu Abstract: In recent years, many natural disasters caused heavy damage. In such situations, natural disaster response did not perform properly to an appropriate standard. This often occurred when first responders were involved, especially novice volunteers who did not have the accurate decision making skill. One of the main issues is the lack of regular training to develop such skill. It has been pointed out that exercise of the non‐technical abilities, such as decision making and situation awareness has an enormous impact on effective and successful disaster response. Furthermore, full ethical situations frequently appear in response action. However, such live practice is difficult to arrange. It has been claimed that the novice volunteers will not improve their skill from live training as much as an expert. This may be due to the feedback limitation to recognize the situation from typical events, as well as the fact they are untrained. The purpose of this research is to promote a mobile game based learning for developing ethical decision making skill. The system proposed can be used to cultivate volunteers’ skill at all times during official disaster management training inside and outside of class. First of all, we conducted a preliminary survey to measure the awareness of the ethical decision making skill of the novice volunteers from high school and university organizations in Indonesia. We asked them to answer three categories of questions. Each category encompassed six components of moral intensity in ethical decision‐making: (1) Magnitude of Consequence, (2) Social Consensus, (3) Probability of Effect, (4) Temporal Immediacy, (5) Proximity, and (6) Concentration of Effect. Based on these preliminary surveys, we have designed a training system called Magnitude which enables the novice volunteer to develop their ethical decision making skill at all times during official disaster management training inside and outside of class, and expect them to improve their performance in disaster response activities. Keywords: mobile game based learning, ethical decision making skill, novice volunteer, disaster response

1. Introduction Indonesia is geographically located within the areas that have been victims of catastrophic disaster effects frequently so‐called “the Ring of Fire’, that is the Ancient Australia‐Indian Continent, the Pacific Ocean Floor, and the Eurasian Plate. These areas are at risk of natural disasters persistently, for instances, earthquakes and tsunamis, and volcano eruptions, which can be devastating. Many facts on disaster responses in Indonesia show that these activities are still sporadically performed. It must be admitted that many of the actions undertaken aim to decrease the impact of the risks posed by the disaster. For example, disaster rescue training for Boy Scouts and youth member of Red Cross Society. However, A volunteer must be trained as much as possible in order to have broad knowledge of disaster response. In other words, the volunteer should be able to conduct training programmes independently to increase their skill. It is argued that decision making in emergency response is a vital skill for first responders. The decisions can impact the capability of response agencies to do their work (FEMA 2010). This research aims to contribute an alternative to the existing volunteer training system. It can be used not only during official disaster management training classes but can also be played and practiced in other places at all times. Furthermore, the research endeavours to supply a mobile game based learning (mobile GBL) to develop ethical decision making skill, in natural disaster response, with high motivation and playful learning.

2. Literature review 2.1 Ethical decision making skill in disaster response Occasionally, we have to face a disaster situation that becomes a portion of individuals’ lives. Anyone can be affected by accidents involving an earthquake and tsunami, flooding and a landslide, or other life‐threatening disasters. However, disaster response and rescue are sometimes unprepared. The short time span of disaster

800

Didin Wahyudin, Shinobu Hasegawa and Tina Dahlan response necessitates the utilization of all resources and requires accurate decision making (Carillo 2011). Thus, the ability to distinguish present and probable difficulties can positively affect the victims (FEMA 2010). One key issue in the preparation of disaster response is the training of decision makers and emergency responders in dealing with the circumstances. It has been pointed out that exercise of the non‐technical abilities, such as decision making and situation awareness has an enormous impact on effective and successful disaster response. In addition, the simulation approach for training and studying of ethical aspect in organization emphasize the significance of institutionalizing and legitimizing ethical decision making (LeClaire and Ferrell 2005).

2.2 Mobile GBL and related work Serious games have been fostered in the education field for a long time, such as, in the domain of personal training in military, business and emergency. It is offer a situation with minimum cost that can impress learners to explore many choices in a virtual setting and review their activities. For example, training for management of disaster and emergency using Play2Train, a virtual learning in Second Life environment. Hewitt, Spencer, Mirliss, and Twal (2010) stated “virtual world can provide an engaging, learning‐intensive alternative to face‐ to‐face scenario exercises”. Such game also delivers the direct advice affecting the learners can receive suggestion to change their action immediately (Caird‐Daley, Harris, Bessel and Lowe 2007). Increasing of mobile technology motivates scholars and researchers to stipulate learning technique using game. An affordance of such system is that mobile games can directly suitable into the desired learning situation. Klopfer (2008) described “mobile games provide many opportunities to consider the game play thoughtfully, discuss it with others, and reflect on its significance, without requiring substantial investments in game‐play time”. In summary, learners can engage in mobile games for a few minutes a day or week, and It is enable learners to grasp educational subjects within playful learning anywhere and anytime.

3. Preliminary survey To identify the importance of the system, we organized a preliminary survey. We selected a number of the novice volunteers from high school and university organizations in Indonesia. Using demographic data, we placed the respondents into two groups. The first group consisted of eighteen volunteers who had been part of the organization for more than 2 years, some of whom had actual volunteer experience. Another group was comprised of fourteen volunteers who joined less than 1 year ago.

3.1 Questionnaires We provided eighteen questions inspired by Liang, Xiaomeng, and Maijing (2011) research finding. These divided into three categories containing six components of the moral intensity (Jones 1991). Therefore, we asked a psychology expert to review ethical aspects of the questions. The first category had scenarios that must be answered with disagree. For example, a question contained Probability of Effect: floods hit a suburb of South Bandung, Indonesia. There is a collapsed bridge under which a dead boy trapped. Mr. Pandu, a novice volunteer, considering to cross the river to evacuate the boy, without wearing a life jacket”. In this scenario, he knew that the victim trapped under the collapsed bridge was dead, so it was not urgent to evacuate immediately. The second category is the modified versions of the first category that must be answered with agree. We changed the situation that the little boy is unconscious. If evacuation finished immediately, he could save the victim from the effect of the river current. The last category of question needs answer reflecting three situations: Would Act, Would Not Act, and Unsure. For example, Mr. Pandu is a special person on the disaster response team. He had a lot of skills not owned by the other. One day when a catastrophe disaster occurred, as a member of advanced team of disaster responder, he asked to go into the disaster area soon. At the same occasion, he also had the responsibility to care his family members including his sister and beloved mother who suffered injuries. What is the best decision? In this scenario, we assume that Mr. Pandu faced into two dilemmas. If he decided to take care his family members, there were a lot of problems appeared in disaster response. His expertise was necessary for accurate and quick response. However, if he decided to go and left his family, there was also a problem. Psychologically, he intended to help his family in advance, and it caused interruption of his concentration in disaster response activity.

801

Didin Wahyudin, Shinobu Hasegawa and Tina Dahlan

3.2 Results Data collected from the first and second categories did not show the difference between two groups. Experienced and inexperienced respondents possessed the tendency to answer questions with a similar result. The most of experienced respondents answered the question correctly, and only few of them faced to confusion. It explains that the experienced respondents understood the problems clearly, so they made the right decisions. However, the majority of inexperienced respondents confused when making the decision (Table 1). The survey result underscores the assumption that the proposed mobile GBL is adequate to fulfil necessity of ubiquitous training system for developing ethical decision making. Table 1: Data collected from the third question category First Group

Second Group Unsure 0

Respondent = 14 Would Would Act Not Act 2 4

Unsure

1. Magnitude of Consequence

Respondent = 18 Would Would Act Not Act 12 6

2. Social Consensus

3. Probability of Effect

4. Temporal Immediacy

5. Proximity

6. Concentration of Effect

Question (Moral Aspect)

4. Magnitude model 4.1 Framework

Figure 1: Mobile GBL framework for developing ethical decision making skill Prensky (2001) defined many genres that have the potential to support the learning of decision making skill. Based on his opinion, the proposed game called Magnitude combines various genres that support learning this skill including strategy, and role‐playing (Figure 1). We use unity3D, a mature game engine on the market, for developing applications. On the server side, a system administrator (training manager) can create game scenarios described in the following section, and collect and analyse assessment data. On the client side, the player can update the level of the game, generate a dynamic game level, and send the assessment result to

802

Didin Wahyudin, Shinobu Hasegawa and Tina Dahlan the server. Magnitude consists of the branched stories supplying flexible game scenarios. For example, if the player makes a wrong decision, he will be faced a basic topic scenario.

4.2 Example scenario The game seats its learner in disaster response situation. Player asked to play the role of a member of the disaster first responder team. For instance, situation when floods disaster hit a suburb of a city. As a volunteer, player has a duty to rescue the victims in this city spontaneously. There is a little boy trapped under the collapsed bridge. Player should find complete information about bridge structure, victim information and condition. He/she may gather reports from news highlight. There is much information. Some of them are informative, and it is may guide the player to make a comprehensive analysis. The others perhaps contain jumble information. Player will be forced to build widespread analytical thinking when he/she confronted the information from a reliable source and another. In this scenario example, the player challenges into two conditions. First, he/she should cross, without wearing a life jacket, the river to evacuate the boy. This action may risk his/her safety. Second, he/she can wait until the other volunteers arrive with a life jacket, but this may result in the victim losing boy’s life.

5. Conclusions and future work According to the result of preliminary survey, we argue that the hypothesis of the importance of mobile GBL for developing ethical decision making has been proved. This research may contribute the original model of natural disaster response training to improve ethical decision making skill. Furthermore, we continue this work focusing on the development of the system and evaluation of its effectiveness.

Acknowledgements We greatly appreciate the support of the Indonesia Government Scholarship and Indonesia Search and Rescue (SAR) organization and thank them for their contribution of knowledge to this research.

References Caird‐Daley, A.K., Harris, D., Bessell, K. and Lowe, M. (2007) “Training Decision Making Using Serious Games”, Human Factors Integration Defence Technology Centre, Report No: HFIDTC/2/WP4.6.1/1. Carrillo, G. (2011) Introduction to Disaster Management, Virtual University for Small States of the Commonwealth (VUSSC), Canada. FEMA. (2010) Decision Making and Problem Solving, Federal Emergency Management Agency, US. Hewitt, A.M., Spencer, S.S., Mirliss, D. and Twal, R., (2010) “Incident and Disaster Management Training: Collaborative Learning Opportunities Using Virtual World Scenarios”, Advanced ICTs for Disaster Management and Threat Detection: Collaborative and Distributed Frameworks, IGI Global, London, pp. 179‐200. Jones, T.M. (1991) “Ethical Decision Making by Individuals in Organizations: An Issue‐contingent Model”, The Academy of Management Review, Vol. 16 No 2 (April, 1991), pp. 366‐395. Klopfer, E. (2008) Augmented Learning Research and Design of Mobile Educational Games, The MIT Press Cambridge, London, England, pp. 228. LeClair, D.T., and Ferrell, L. (2000) “Innovation in Experiential Business Ethics Training”, Journal of Business Ethics, Kluwer Academic Publishers. Netherlands, pp. 313–322. Liang, L., Xiaomeng, Z. and Maijing, S. (2011) “Development of ethical decision making scenarios that focus on emergency rescue” 8th International Conference on Service Systems and Service Management (ICSSSM), Tianjin, China, 25‐27 June 2011. Prensky, M. (2001) Digital Game‐Based Learning. McGraw‐Hill, New York.

803