OpenLivingLab Days 2019 Conference Proceedings

Page 1

Proceedings of the OpenLivingLab Days Conference

Co-creating Innovation: Scaling-up from Local to Global



ENoLL Office Pleinlaan 9 B-1050 Brussels Belgium T: +32 2 614 85 47 3 E:

This report is a compilation of the papers presented between the 3rd and 5th of September 2019, in Thessaloniki, Greece, as part of the OpenLivingLab Days 2019 conference. The publications here contained are a result of the double-blind review and evaluation procedure launched on February of 2019 as part of the “Call for Papers” responding to the theme of the OpenLivingLab Days 2019 conference: “Co-creating Innovation: Scaling-up from Local to Global”

The “Call for Papers” encouraged contributions from three different paper categories to stimulate a diverse participation of actors: Research Papers providing consolidated scientific research; Innovation Papers showing case studies from a practitioner perspective; and Research in-Progress works, presenting relevant preliminary results.

ISBN (e-book): 9789082102796 © 2019 ENoLL - European Network of Living Labs All rights reserved


Review Panel Chair Prof. Panagiotis Bamidis. Aristotle University of Thessaloniki.

Steering Committee Top Selected Papers Chair: Dr. Dimitri Schuurman. imec.

Health and Wellbeing Chairs: Ömer Onur. Başakşehir LL Istanbul. Prof. Panagiotis Bamidis. Aristotle University of Thessaloniki.

Smart Cities and Regions Chair: Dr. Ali Padyab. Luleå University of Technology.

Living Lab Sustainability Chair: Dr. Joëlle Mastelic. Haut Ecole Spécialisée de Suisse occidental.

Theoretical and Methodological Challenges Chair: Prof. Tuija Hirvikoski. Laurea University of Applied Sciences.

Doctoral Consortium Chairs: Prof. Anna Stålbröst. Luleå University of Technology. Dr. Brigitte Trousse. Inria, University Côte d'Azur.

Evaluation Committee: Abdorasoul Habibipour – Botnia Living lab Anja Overdiek –The Hague University of Applied Sciences Antonis Billis – Knowle West Media Centre Aya Rizk – Luleå University of Technology Balatsas-Lekkas Angelos – VTT Technical Research Centre of Finland Benjamin Nanchen – HES-SO Valais-Wallis Bianca Ceccarelli – LifeTechValley/University Hasselt David Jamieson – Northumbria University Eric Seulliet – La Fabrique du Futur Francesco Molinari – XR8 sas & Lunigiana Amica Gareth Priday – Australian Network of Living Labs (ALLIN) Hanna-Greta Puurtinen – TAMK Living Lab Isis Gouedard – HES SO Valais Joelle Mastellic – Energy Living Lab Josep M. Pique Huerta – La Salle - Ramon Llull University Judith Urlings – Happy Aging LifeTechValley Masataka Mori – Miratuku Ömer Onur – Başakşehir Municipality / Başakşehir Living Lab Rob Wilson – Newcastle Business School Stefano Tarantola – Joint Research Centre-European Commission Suvi Konsti-Laakso – Lappeenranta-Lahti University of Technology Athanasios Priftis – University of Applied Sciences in Geneva (HES-SO)

ENoLL Office Contributors Clara Mafé

Thess-AHALL Contributors Despoina Mantziari

Table of Contents Top-6 Papers selected by the Evaluation Committee A Creative Citizens Model for Smart Urban Planning by Helen Manchester and Carolyn Hassan ............................................................................................ 14 Agile Piloting for Smarter Cities: 3 Cases of Engaging Ecosystems and Communities in Co-creation by Kaisa Spilling, Janne Rinne and Matti Hämälainen ........................................................................................................... 28 Co-Creating Technology for Societal Change: A Mobile App Addressing Homelessness by Rachel Burrows, Antonette Mendoza, Sonja Pedell, Leon Sterling, Tim Miller and Alexi Lopez-Lorca .......................................................... 41 Living Lab Activities for Social Problem-Solving R&D Projects in Korea by Ji Eun Seong and Ji In Park ................................................................................. 61 Living Labs and Circular Economy. The case of Turin by Federico Cuomo, Nadia Lambiase and Antonio Castagna .............................................................. 83 To Get Things Right for Children. Implementation of a Public Social Living Lab Model for Coordinated Support for Children in Need by Angelika Thelin, Torbjörn Forkby and Mats Anderberg ................................................................ ..99

Health and Wellbeing Co-creating innovative tools with and for people with Intellectual Disabilities: The case of DS Leisure e-Training Platform by Maria Metaxa, Foteini Dolianiti, Ioanna Dratsiou, Evangelia Romanopoulou, Dimitris Spachos, Theodore Savvidis, Vasiliki Zilidou, Maria Karagianni & Panagiotis Bamidis .. 115 Creating an anonymous, at-home screening for sexually transmitted diseases sent by letter mail: the cross-border development of a standardized urine collection device and associated testing service by Judith H.J. Urlings, Bianca Ceccarelli, Claire A.G.J. Huijnen, Paulette J.J. Wauben, Joke Donné, Ronald Van den Bossche, Alejandra Rios-Cortes, Koen Beyers and Vanessa Vankerckhoven ................................................................ 127 IoT –based Smart living Environments for ageing well in Greece by Sofia Segkouli, Stefanos Stavrotheodoros, Nikolaos Kaklanis, Konstantinos Votis, George Dafoulas, Christina Karaberi and Dimitrios Tzovaras .......................... 139 Participatory design and validation of an innovative training program to maintain Autonomy of older adults with Alzheimer’s Disease by Despoina Mantziari, Antonis Billis, Giorgos Arfaras, Maria Karagianni, Vasiliki Zilidou and Panagiotis Bamidis ............................................................................................. 152 Responsible Design for an Older Future by Gareth Priday and Sonja Pedell ............................................................................................................................. 170

Smart Cities and Regions A Case Study of a Living Lab through a Bus Improvement Committee in the Yeongjong area of Incheon City by Min-Ho Suh, Junghyun Park, Minki Kim and Won-Kyun Joo .................................................................................................... 189 Adapting Living Lab approaches to marginal contexts and urban regeneration: the case of Mapping San Siro Lab by Elena Maranghi and Francesca Cognetti ............................................................................................. 205 Intelligent Living Lab: Supporting data-centric decision-making using ICT tools by Minki Kim, Junghyun Park, Min-ho Suh, and Won-Kyun Joo ............. 200 Open Innovation Camp (OIC) – A Tool For Solving Complex Problems Rapidly by Teemu Santonen, Julia Nevmerzhitskaya, Aletta Purola and Harri Haapaniemi ......................................................................................................... 226

Sustainable Living Lab Processes, Business Models and Goals Building a platform of social entrepreneurship and living together by Athanasios Priftis, Leonor Afonso, Theo Bondolfi and Jean-Philippe Trabichet ............................................................................................................................. 244 Business model review for Living Labs: Exploring business challenges and success factors of European Living Labs by Justus von Geibler, Julius Piwowar and Linda Weber.................................................................................. 254 Facilitate innovation and collective intelligence through play by Yves Zieba and Isis Gouédard............................................................................................... 273 Launch Process of a Living Lab and Required Leadership for Practitioners by Masataka Mori and Kyosuke Sakakura......................................................... 280 Living Labs need sustainable business models: the Funding Mix Framework to bridge the gap between theory and practice by Edoardo Gualandi and Flavia Fini ..................................................................................... 295 Sustainable person-centered Living Lab for regional management as extension of Japanese dementia care activities by Atsunobu Kimura, Mizue Hayashi, Fumiya Akasaka and Masayuki Ihara ................................................. 312 The value of participatory approaches in developing energy services by Joelle Mastelic and Stéphane Genoud .............................................................. 322

Theoretical and Methodological Living Lab Challenges Blockchain, a promising way for scaling up co-creation of innovation from local to global by Eric Seulliet .......................................................................... 338 Cross-cultural Differences in Living Lab Research by Nele A.J. De Witte, Ingrid Adriaensen, Leen Broeckx, Vicky Van Der Auwera and Tom Van Daele ............................................................................................................................. 352 Developing a test and validation protocol based on quasi-experiment and analogue observation to evaluate the performance of a Living Lab output by

Benjamin Nanchen, Emmanuel Fragnière, Patrick Kuonen, Joëlle Mastellic, Randolf Ramseyer and Henk Verloo.................................................................. 368 Identifying Living Lab orchestrators’ individual-level skills by Anne Äyväri, Tuija Hirvikoski and Heidi Uitto ........................................................................... 382 Improving Quality in Higher Education by using Living Lab Methods by Karin Axelsson, Yvonne Eriksson and Anders Berglund ............................................ 394 LivingLab 65+ - Co-creation with retirement and nursing homes by Veronika Hämmerle, Stephanie Lehmann, Cora Pauli and Sabina Misoch ..................... 408 Living Labs for small retailers – in search of methodology and tools by Heleen Geerts, Gabriela Bustamante Castillo and Anja Overdiek .................... 416

Doctoral Consortium Papers Barriers for Test and Adoption of Digital Innovations by End-users in a Living Lab Context by Abdolrasoul Habibipour ............................................... 434 Beyond participation: exploring citizen stakeholder empowerment in the cocreation of innovation by Shelly Tsui .............................................................. 450 Can Open Innovation offer a new perspectives for development of ecosystemic business models? by Julia Nevmerzhitskaya .......................... 459 The Roles, Functioning and Culture of Urban Innomediaries by Jimmy Paquet-Cormier................................................................................................... 467 Urban Living Labs as a smart city approach: how does socio-technical innovation transform urban development? by Hui Lyu ................................ 475

Acknowledgments This publication is a collaborative effort of several individuals representing the European Network of Living Labs and its network members.


Top 6 papers selected by the Evaluation Committee 12


A creative citizens model for smart urban planning

Helen Manchester1 and Carolyn Hassan2 1

Associate Professor Digital Inequalities and Urban Futures, School of Education, University of Bristol, United Kingdom 2 Director, Knowle West Media Centre, United Kingdom Category: Research in-progress Topic: Smart Cities & Regions

Abstract Recent moves, led by the Living Labs movement and others, have begun to place the citizen at the centre of Smart City discussions. But questions around what theories and forms of learning are required for citizens to play a role in the development of digital, urban futures are rarely asked. This paper adopts ethnographic methods to study the assumptions about learning in a Europe-wide smart city project that included a component of work led by Bristol Living Lab (KWMC). Our paper provides important messages for Living Labs and others keen to include citizens in smart city development. It suggests that the current ‘banking’ models of learning adopted in relation to citizen participation are not fit for purpose and that new models are needed. This needs to recognise citizen learning as situated in social and material contexts and embedded in unequal relations of power, knowledge and resources. We make the case for smart city initiatives to offer city inhabitants critical, creative learning opportunities that begin to address the inequalities that constitute the contemporary smart city. Keywords: Smart City, citizen, learning, digital inequalities, critical, creative citizen


1 Introduction The educational challenge implicit in citizen involvement in smart cities is visible in urban theory literature which argues that there is a distinct mismatch between the rhetoric of the potential of Smart Cities worldwide to create more inclusive, democratic or more innovative cities and the actual practice of Smart City planning (Hambleton, 2014; McFarlane, 2011; Campbell, 2012; McFarlane and Söderström, 2017). In addition, whilst citizens are increasingly placed at the centre of Smart City visions problems with questions of governance, citizenship models and relationality of power have been noted (Hollands, 2015; Joss, Cook & Dayot, 2017; Gabrys 2014; Cardullo and Kitchin, 2018). Batty, for example, asks whether, “Smart cities are equitable cities…. that… the sort of infrastructure, expertise and data that will characterise the smart city will enable equity to be easily established and such cities will improve the quality of urban life.” (Batty et al, 2012 p. 516) This paper draws on learning theory, which has been little utilised in relation to living labs, smart cities and collaboration in smart city work, to explore an emerging model of learning that might inform more equitable design of future, digital cities. In the first section of the paper we utilise learning theory to explore a model of digital learning that we believe can have distinctive effects on participation and inclusion in conversations about digital, urban futures. In section 2 we go on to introduce our research questions and to briefly discuss the collaborative ethnographic methods we used to explore the approach of the Bristol Living Lab within one European Smart City project in the city of Bristol, UK. In Section 3 our findings explore how placing creative, collaborative models of learning at the heart of Smart City planning might enable more inclusive approaches to learning about and designing urban futures. 1.1 The Creative Citizens Model In 2016 we were brought into a major city-wide, project in Bristol in which the question of how to build the capacity of citizens to engage with Smart City developments was central. Our role was to ‘document and explore the role of citizens in co-designing digital futures’ in the city. Located in a School of Education we came to the project with a particular set of assumptions about both learning and how learning might play a role in citizens’ shaping of a city. First: our assumption was that learners are active. Theories of critical digital literacies have long suggested that citizens should be considered active learners when engaging with new technological developments (Potter & McDougall, 2017; Eynon, 2015). Seeing the citizen as an active learner places value on attending to how their diverse knowledges, creative approaches and critical social actions might contribute to positive future urban development. Here, learning can be understood as a tool to enable people to understand new and emerging digital technologies in order to change the Smart City, not adapt to it (McFarlane and Söderström, 2017).


Second: we recognised literacy practices as being ideological not neutral. Current Smart City discourse presents the introduction of a particular set of digital literacy skills as an unalloyed good (Tapscott, 1998; Jenkins, 2007). The long history of literacy studies (Street, 2003), however, would suggest the importance of situating literacy practices in social, cultural and historical contexts, paying attention to relationships between literacy, power and knowledge and attending to the inequalities and knowledge hierarchies that these produce (Gee, 2000; Buckingham, 2006; Selwyn and Facer, 2013). These literatures suggest that models of learning in Smart City projects must involve a deeper understanding of the already existing cultural knowledges and experiences of the city (Barton and Hamilton, 2012; Erstad & Sefton-Green, 2013). Third: As the digital increasingly becomes the ‘stuff’ of everyday life (Miller, 2010) the development of knowledge hierarchies becomes invisible, embedded in hidden infrastructures and algorithms that permeate the instrumentation of the city (Star, 1999). This perspective suggests that learning is distributed across digital and material spaces (McFarlane, 2011). This requires us, as researchers, to consider that learning practices in the Smart City also include the material infrastructures being embedded to make smart cities work, the place-based memories held by the communities involved, and the new material goods such as electronic cars that suddenly appear on street corners. Our position on entering this Smart City project, then, was one that understands learning as situated, ideological and material, drawing attention to why and how particular ideas about citizen learning become dominant over others and how these ideas might be questioned and reframed (Gee, 2000; Street, 2013). It sees the learner as creative, active and critical, as wanting (rather than needing) to learn certain things in relation to making urban futures. It recognises that, given the historical patterns of inequality that produce patterns of ownership, access and control of technologies, there are obstacles to city inhabitants finding routes to influence policy and technology shaping the development of the city.

2 Introducing the Mimeo Project and research aims The Mimeo project took place in Bristol, a medium sized city in the UK. The city partnered with two other European cities to successfully bid for a large European Union Horizon 2020 project. The project lasts for 5 years (2016 – 2021) and is in its implementation phase as we write in 2019. The project is managed by the local municipality with around 16 partners including small and medium sized enterprises, community organisations as well as academic partners working on technology implementation and other evaluation. The increasing focus on the need for citizens’ involvement in Smart Cities is stated in the overall objective of the project: “To increase the quality of life for citizens across Europe by demonstrating the impact of innovative technologies used to co-


create smart city services with citizens, and prove the optimal process for replicating successes within cities and across cities” (Mimeo Project documentation). The ‘co-creation’ approach is intended to enable innovative, replicable Smart City services to emerge in dialogue between citizens and project team that tackle familiar urban problems such as traffic congestion, poor air quality and unsustainable energy use. However, the project proposal documentation had no specific aims relating to citizen engagement or learning. Further, it is worth noting that while the development of technology accounts for half of the budget, citizen engagement in the process has just over 10% of the budget. The management of citizen participation in the project is led by a media arts organisation and ‘Living Lab’ in the city (KWMC). The organisation is not based in the Mimeo case study area but is known for its innovative work around digital inclusion in the city. KWMC have been working closely with the municipality and the universities in the city for a number of years to develop a framework that creates a smart city where ‘the power of technology is harnessed to tackle the issues that people care about.’ (KWMC, 2016). Their approach rejects an emphasis on the role of data, hardware and software in Smart City projects by seeking to include the knowledge and lived experience of the citizens within the community to address actual needs (Soderstrom, Paasche & Klauser, 2014). The stated goal of their approach is to collaborate with citizens to ‘tackle local issues by enabling participatory processes, bottom-up sensor infrastructures and collecting, making sense and sharing relevant open data’ (Balestrini, Creus, Masfarre & Caniguearal, 2016, p. 3). The Mimeo project is working in a ‘case study’ area in each city on the implementation of specific smart infrastructure. In Bristol a vibrant and diverse area of the city was selected. The population living in this area (N= 50,000 approx.) has the highest percentage of black or minority ethnic citizens (BME) in residence (44% whereas the city average is 16%). 51% of all accommodation in this area is flats, whereas the city average is 20%. The district also has the lowest levels of car availability in the city with (46% of households with no car, city average 29%). According to the Department of Energy and Climate Change two areas within this district are in the top 10% of households in the country experiencing fuel poverty. Almost one third of the neighbourhoods within the district are classified as amongst the 10% most deprived neighbourhoods nationally. The number of recipients of out of work benefits is significantly higher than the city average (12.1%), with the rate in one neighbourhood amongst the city’s highest at 26.2%. These multiple deprivations resulted in the area being granted European Union Objective 2 status and 'New Deal for Communities' status by the UK government1.


Statistics from 444ac- a3d8-4417-b967-b1c19ec3512f accessed 14th Feb, 2018


Mimeo is directed towards developing a suite of new technological designs including innovations around smart homes and retro-fitting of homes to become more energy efficient, the installation of electric bikes and vehicles and an electric vehicle charging infrastructure, an on demand electric bus service, the development of a Smart City platform and an energy demand management service. The challenge of diverse partners with competing motivations and desires meant that, although attempts were continually being made to embed citizen engagement and co-design into the project, this was challenging. KWMC recognised this and created a specific stream of work within the Mimeo project called ‘Created by Us’ in order to achieve their aims related to increasing the role of citizens. The focus was on ‘Citizen Sensing’: a process involving people building and using small and generally low-cost sensor technology to help them to collect data about issues of importance to them. It was hoped that the ‘Citizen Sensing’ element of the work would also feed into the design of some of the larger scale technological infrastructure projects. Our own position on the project was as researchers tasked with researching the citizen engagement activities on the project. Our research aimed to explore the way that citizens, and their learning, were imagined and included in the Mimeo project. Instead of only offering description or critique our research was designed to uncover and explore the learning models and practices of citizen involvement in the Mimeo project. Our purpose was not to provide a framework for citizen engagement (as other projects have tried to do, see European Union, 2017) but rather, through in- depth empirical work, to illuminate approaches to digital learning that might support the participation of those often marginalised from conversations about digital, urban design. The key project partners and associated partners brought into the Mimeo project in Bristol included a diverse array of organisations. In the process of seeking informed consent confidentiality and anonymity were not promised to those involved. However, in reporting our findings names are not used and labels given when quoting individuals are deliberately ambiguous in order to avoid, as far as possible, directly identifying actors. 2.1 Question, methodology and methods In this paper, we explore the following question through detailed analysis of the citizen sensing element of the project, led by KWMC: •

How might a Creative Citizens model offer opportunities for critical, digital learning in Smart City planning?

Our research recognises the need to ground critical Smart City scholarship within specific places, foregrounding the distinctive knowledges, concerns and challenges of marginal, often excluded citizens (McFarlane & Söderström, 2017). Our engagement in the field began as the project began and will finish in October 2019 when the major intervention in the city is due to be completed. By exploring learning in a Smart City in a specific location and time we are attempting to understand ‘situated bodies of practices, into which human actors are differently


enrolled’ (Cowley et al, 2018, p.55). In order to understand citizen learning on the project we recognise a need to account for the conditional and changing local situatedness of people, policies, technologies and places that create the dynamic landscape in which the project operates. We therefore sought to explore how citizen learning was constituted in relation between multiple and diverse actors ‘rather than as only imposed by state or corporate actors from ‘above’ (Cowley et al, 2018, p. 55). We conducted participant observation, working closely with project partners. The data generated includes observations of nine whole project team days, twentyseven other meetings, including engagement and communication group meetings and ten workshops held with local organisations and citizens, producing in- depth fieldnotes from meetings and events. Details on how citizen learning was discussed by these project partners were carefully noted and any changes over time specifically recorded. We also conducted twenty face-to-face interviews with project partners and organisations, which were transcribed, including those who were not formal partners on the project but were brought in as the project evolved, for instance, anchor community organisations in the locality. Discourse analysis of key texts, project documentation and online communications was also undertaken. As we were particularly looking at the citizen engagement package in the Bristol case study site, we spent time with, and worked alongside, the KWMC staff, joining their team meetings and recording their approach throughout the project. In all of our work we were looking for where accounts of learning became visible and we followed the actors, attending smaller meetings and having informal conversations as and when interesting data emerged. Data were analysed iteratively and analysis was carried out using Nvivo software, following a thematic approach. As the themes emerged from the data analysis, the approach to the research could be iteratively developed, in collaboration with KWMC and others involved in citizen engagement activities. Interview schedules were adapted to reflect findings as they emerged and to build on our understandings of citizens and their learning as the project was rolled out.

3 Findings In this section we use our data to illustrate, unpick and discuss the work of the Bristol Living Lab (KWMC) and their particular approach to learning that was evident in their ‘Citizen Sensing’ work. For a full discussion of the learning models that circulated within the wider project see Manchester and Cope (2019). 3.1 Creative Citizens Model The Creative Citizens learning model operated in the citizen sensing element of the Mimeo project run by KWMC. As discussed earlier this approach situates learning in historically constructed, unequal relations of power, knowledge and resource, it recognises the significance of learner diversity and is focused on how


inequality might be challenged through working alongside individuals and communities to discover what animates them, but also in thinking about how they might be connected into powerful processes at work in the city. In initial conversations with anchor community organisations in the area it became obvious to KWMC and ourselves that there was a disconnect between the everyday lives of many of the inhabitants in the case study neighbourhood, and the ‘smart’ technologies being introduced and discussed on the project. This key tension was discussed with one of the project managers at KWMC who suggested that, “…the technology doesn’t mean anything to the community and so [we are] trying to steal that space where technology is developed based on its meaning and its purpose to real people. But these real people having a part in that is challenging.” (Interview project manager, media arts organisation). As KWMC did not have a long history of working in the targeted area they recognised that they needed to understand the histories and knowledges held in the targeted community. A first step involved inviting local anchor organisations to discuss their possible participation. Many of the community practitioners were very dubious about the effectiveness of using digital media to tackle inequalities and felt that there were a range of other priorities in citizens’ lives that should be given priority (fieldnotes, March, 2017). Considering historical relations of power, and negative feelings towards both the municipality (in a context of UK austerity) and the EU, meant that drawing these local knowledges from local civil society organisations and inhabitants into Mimeo needed to be carefully managed. As one of the project managers observed: “Quite rightly some of the organisations were apprehensive even before it started but for me having seen how those negotiations have been done and how careful you have to be this has been a great learning process. And I think if you want to test out anything with people then you have to build in the groundwork to introduce the project properly, finding and respecting the key stakeholders who are already in the community and working from there”. (Interview project manager, media arts organisation) Connecting with community anchor organisations in the area, which had been overlooked in the bid writing, here helped KWMC to acknowledge the huge amount of knowledge, understanding and research around the key assets, concerns and challenges faced by those living in the area. This included integrating the findings from a timely publication of a community research led project that had identified a key set of priorities around quality of life for inhabitants of the area. For instance, when asked which services were important to people’s wellbeing, over half of respondents in the area pointed to parks and green spaces. These were also identified as a key focus for municipality investment (Up Our Street, 2017). This community-led research questioned the assumption in the Mimeo project that ‘smartness’ will deliver a better quality of


life. Instead, these findings suggested more green spaces (we might argue the opposite of smartness) was what was truly desired. Starting with the everyday issues of concern to residents and providing diverse opportunities for engagement and learning, meant that this work stream began to switch away from a focus on the technology and the narrow, technocratic deliverables of the Mimeo project. As a result, this work was silo-ed, largely because the inhabitants’ concerns did not connect with the technological developments listed in the project deliverables. Despite this, the experimental work led by KWMC continued and the ‘Citizen Sensing’ strand of activity began to take shape. Citizen Sensing involves collaborating with citizens on the development of low-tech sensor infrastructures and supporting them to make sense of and share the knowledge gained through the sensor technology development. Initial activity designed by KWMC in the project area involved in depth, informal on the ground work involving artists and community development workers ‘hanging out’ and beginning conversations in local chicken shops, cafes and nailbars. Several issues emerged including concerns around damp homes and poor air quality. Both issues connected with wider concerns that had been expressed in the community-led quality of life research (Up Our Street, 2017). Initially the damp homes issue gathered momentum, partly because it connected with multiple local concerns including health issues, social stigma and poorquality housing, demonstrating the complexity of connections between issues faced by marginalised communities. In order to challenge knowledge hierarchies in the city it was deemed essential to connect these emerging local concerns with powerful processes and people in the city. This was possible as these concerns around health, social and housing inequalities were also shared by many local organisations and by policy makers in the municipality. In addition, computer scientists and technologists, open data and other experts were interested in exploring how sensor data might be utilized by citizens to understand the issues of concern in more depth. Out of this dialogue, the ‘Damp Busters’ project emerged. In developing this work, the importance of connecting lay knowledges with expert knowledges in order to generate dialogue between them became clear. The mutual exchange and learning required was challenging, particularly when innovative technologies, often seen as ‘irrelevant’ in the area, were at the heart of the inquiry, It’s great to have different partners from different worlds, it’s a challenge as well, but the level of expertise is amazing - to speak to someone who is creating a cutting-edge network the like of which has not been seen before is a challenge but it’s also exciting to connect people up who might not have had the chance to access those sorts of technologies. (Interview, project manager, media arts organisation)


The role of KWMC was vital here in ‘holding’ (in the psychoanalytical sense of ‘holding’ emotions and doubt so that they are manageable, see Bion, 1984; Bibby, 2009) the collaboration at the early stages, which involved supporting the translation of knowledge and ideas across the different groups involved. The importance of offering multiple and varied opportunities for participation and involving a diverse team in order to facilitate the inclusion of citizen knowledge and to challenge unequal relations of power quickly became obvious. Artists, technologists and engagement specialists designed and offered a wide range of opportunities for learning and participation including workshops, hack days and working on design briefs together. Participation across the activities varied, as a KWMC project manager explained, We didn’t get hung up on everyone attending everything, it wasn’t that we were going to do lots of hackdays. We offered a range of different activities to keep people on board. Learners were seen here as active in understanding what they might want or need to learn and how Citizen Sensing might help them to do so. Digital learning opportunities involved diverse groups of people coming together in practice based, material encounters bringing the technology design into relation with the everyday lives of those living with damp. Learning here was understood as both relational and material, involving engagement with artefacts and policy agendas. For instance, social tenants worked alongside the tenancy officer in the municipality to identify and target private landlords who ran damp properties in order to challenge those who were not acting ethically. In addition, a focus on critical engagement with the actual technologies was visible in events throughout the damp project that offered people opportunities to play with sensor technologies and other digital, material devices. Technologists as well as residents in the area, academics, civil servants and others took part. The events were designed to share knowledge and expertise and to demystify terms like ‘data’ and ‘sensor technologies’. In this way an open design process was created where different knowledges were put into conversation in order to open up new questions around digital, urban futures (Storni, Binder, Linde & Stuedahl, 2015). Autonomy and creativity were encouraged in these workshops where the processes and the digital tools and interfaces were designed as ‘tools for conviviality’ to encourage engagement among and between people and the city environment (Illich, 2002). The Citizen Sensing group built a prototype sensor together using open technology. The tool was designed in response to a real concern raised by those involved in the design process. This ‘making together’ approach (Ingold, 2013) enabled those who were not technology experts to see how things are put together and to build knowledge about sensors and their design. The co-design process also supported inhabitants of the area to ask questions around data and ethics that related to their everyday lives and concerns. As a KWMC project manager suggested, it was important that the interface itself was user friendly, however,


“what was really important to the citizens was ‘I want to choose when it’s switched on, what happens with the data, I want to know who the data is being shared with”. The interface that was co-designed in order to house the sensor technologies in people’s homes was in the shape of frog. The frog, who loves to live in damp places, became an accessible material symbol for the project that linked the digital data being collected with the everyday lives of those experiencing damp. It was designed to be as attractive and easy as possible to use. Notably, the intention here was not to make the infrastructure invisible to citizens, rather, tenants in the five households involved in the testing phase were trained so that they all understood how the technology worked and what data it was collecting. Through the co-construction of a data agreement, issues related to data ownership were experimented with, allowing different voices to be heard in the process, and raising awareness of the various concerns of all of those involved from the sensor developers, through to the landlords, municipality officers and residents themselves. In terms of the sustainability of the work a national sustainable energy organisation worked alongside 16 people to support them to become ‘Community Damp Busters’- a knowledgeable local team of people able to support others with damp issues in their neighbourhood. Interestingly, the focus of the action very firmly moved away from the frog sensors and their deployment to the issue itself. As the KWMC project manager explained, “Although the frog became a symbol for the project the real focus is on making a difference. If we had more information, like if we knew who owned the property, then we could ask are these buildings fit for purpose? The key thing to keep momentum going is to empower people to make changes.” The Damp Busters project offers us insights into how practices of situated, critical learning might be adopted with citizens on a Smart City project. This involved accounting for the everyday lives and unequal relations of power, knowledge and resources in the area. The approach stresses the need to provide multiple and varied opportunities for participation, including those that are creative, encourage autonomy and involve linking citizens into powerful processes and people in the city. Direct engagement in building technologies together also has the effect of making visible technology design processes and, in particular, ethical issues that may be significant barriers to the implementation and sustainability of Smart City infrastructure.

4 Conclusion Despite a genuine desire to include citizens in the Mimeo project it remained difficult for the team to do so. Writing a successful, ambitious and innovative bid had created a constant tension running through the project in balancing the rollout of the technological infrastructure with what could be delivered on the ground that might be of benefit to local residents. Our argument in this paper is that in


order to include citizens in Smart City planning much greater reflexivity is needed around models of learning, and assumptions about citizens and their capacities and interests. In particular, early discussion of different learning models and the collaborative design of developmental practices with all partners is necessary. Our argument, drawing on extensive literature in education studies, is that a model of learning that foregrounds collaborative, creative, critical learning for all involved is necessary in order to include citizens in powerful urban design processes. This requires significant commitment from policy makers, technology companies, funders and citizens themselves and the work needs to be brought out of the silos of sub projects and ‘engagement activities’ and into mainstream Smart City policy and practice. It is only in this way that we can begin to weaken the hold of neoliberal, technocratic approaches in the critical imaginations of those involved in smart urban planning (Cardullo and Kitchin, 2018; Joss et al, 2017).

Funding details The research was funded by the European Union Horizon 2020 fund under grant number 691735.

References Balestrini, M., Creus, J., Masfarre, G. & Caniguearal, A. (2016) A Future in Common: Understanding and Framing Commoning Strategies for Bristol Accessed, June 2018 phase2_d2_v6.pdf Barton, D., & Hamilton, M. (2012). Local literacies: Reading and writing in one community. Routledge. Batty, M., Axhausen, K. W., Giannotti, F., Pozdnoukhov, A., Bazzani, A., Wachowicz, M., & Portugali, Y. (2012). Smart cities of the future. The European Physical Journal Special Topics, 214(1), pp.481-518. Bibby, T. (2009) 'How do children understand themselves as learners? Towards a learner-centred understanding of pedagogy' Pedagogy, Culture & Society 17 (1): 41- 55 Bion, W. (1984) Learning from Experience, London: Karnac Books Buckingham, D. (2006) Is there a Digital Generation? In Buckingham, D & Willet, R. (eds.) Digital Generations: Children, young people and digital media. Campbell, T. (2012). Beyond Smart Cities: How Cities Network. Learn and Innovate, Earthscan, New York (NY). Cardullo, P. & Kitchin, R. (2018) Smart Urbanism and smart citizenship: The neoliberal logic of ‘citizen-focussed smart cities in Europe. The Programmable City Working paper 39,


Cowley, R., Joss, S., & Dayot, Y. (2018). The smart city and its publics: insights from across six UK cities. Urban Research & Practice, 11(1), pp. 53-77. Erstad, O., & Sefton-Green, J. (Eds.). (2013). Identity, community, and learning lives in the digital age. Cambridge: Cambridge University Press. European Union: Market Place of the European Innovation Partnership on Smart Cities and Communities (2017) Inclusive smart cities: A European manifesto on citizen engagement. Retrieved October 19th, 2018, sive%20S mart%20Cities_0.pdf Eynon, R. (2015) The quantified self for learning: critical questions for education, Learning, Media and Technology 40 (4) pp.407-411. Gabrys, J. (2014). Programming environments: environmentality and citizen sensing in the smart city. Environment and Planning D: Society and Space, 32(1), 30-48 Gee, J. (2000) Chapter 3 : Identity as an Analytic Lens for Research in Education. Review of research in Education. Hambleton, R. (2014). Leading the inclusive city: Place-based innovation for a bounded planet. Bristol: Policy Press. Hollands, R. G. (2015). Critical interventions into the corporate smart city. Cambridge Journal of Regions, Economy and Society, 8(1), 61-77. Ingold, T. (2013). Design and anthropology. Ashgate Publishing, Ltd.. Jenkins, H., with Purushotma, R., Robison, A.J. Weigel, M., and Clinton, K. (2007) Confronting the challenges of participatory culture: Media education for the 21st century (Part One) Nordic Journal of Digital Culture, 1:2, pp. 23–33 Joss, S., Cook, M., & Dayot, Y. (2017). Smart cities: Towards a new citizenship regime? A discourse analysis of the British Smart City Standard. Journal of Urban Technology, 24(4), 29-49. Knowle West Media Centre (2016) The Bristol Approach in action: working together to create tech for the common good. Accessed June 2018 McFarlane, C. (2011) Learning the City: knowledge and translocal assemblage. Oxford: Wiley Blackwell. McFarlane,C. & Söderström, O. (2017) On alternative smart cities, City, 21:3-4, 312- 328, DOI: 10.1080/13604813.2017.1327166 Manchester, H. and Cope, G. (2019) Learning to be a smart citizen. Oxford Journal of Education 45: 2, 224-241. Miller, D., (2010). Stuff. Cambridge: Polity. Potter, J. & McDougall, J. (2017) Digital Media, Culture and Education: Theorising third space literacies. London: Palgrave MacMillan.


Selwyn, N. & Facer, K. (2013) The Politics of Education and Technology. London: Palgrave Macmillan. Söderström, O., Paasche, T., & Klauser, F. (2014). Smart cities as corporate storytelling. City, 18(3), pp. 307-320. Storni, C., Binder, T., Linde, P., & Stuedahl, D. (2015) Designing things together: intersections of co-design and actor–network theory, CoDesign, 11:3-4, 149-151, DOI: 10.1080/15710882.2015.1081442) Street, B. (2003). What’s “new” in New Literacy Studies? Critical approaches to literacy in theory and practice. Current issues in comparative education, 5(2), 77-91. Street, B. (2013) (2nd edition) Social Literacies: Critical Approaches to Literacy in Development, Ethnography and Education. Routledge: Oxon Tapscott, D. (1998). Growing up digital: The rise of the net generation. New York: McGraw-Hill. Up Our Street (2017) Understanding wellbeing through community research in Easton and Lawrence Hill (Research Report). Accessed, June 2018:



Agile piloting for smarter cities: 3 cases of engaging ecosystems and communities in co-creation

Kaisa Spilling*1, Janne Rinne1, Matti Hämäläinen1

*Corresponding author Virium Helsinki

1 Forum

Category: Innovation paper

Topic: Open Track

Abstract Agile Piloting Programme is a proven method that supports and facilitates smart city development and engage a wider stakeholder network to co-create new services. During 2016-2019, more than 50 pilots have run on the different platforms in Helsinki. The thematic piloting rounds have ranged from climate positive and resource wise solutions to innovative local services and wellbeing. The model has been adopted in different domains of smart city and used widely in the network of six biggest cities of Finland. In this paper we briefly present the model and give examples of three cases that highlight different aspects on how co-creation and experimentation has been applied in different city platforms: Smart Kalasatama (Health & Wellbeing), Jätkäsaari Mobility Lab (mobility and transport) and Helsinki schools (education). Keywords: Smart Cities, Living labs, Open innovation, Experimentation, Startups, Ecosystems


1 Introduction Smart city development has become the mainstream. It is not only about technology. Cities are looking for new ways of working together with companies and other stakeholders to solve the problems of rapidly growing and evolving cities. The new solutions require more agile ways of operating and collaboration. Agile piloting was developed as a method to activate the innovation ecosystem, and it opens the city infrastructure, data and services as an urban lab for experimentation. It also offers companies and start-ups an authentic real-life environment to test and develop their services, together with residents who may participate in the process as experts of everyday life. Lean development, experimental culture and a permission to fail are what it takes to create the urban futures. From 2013, Forum Virium Helsinki — a city owned innovation company — has been orchestrating the innovation platform activities in Kalasatama, the model district for smart city development in Helsinki. One of the methods created is Agile Piloting Programme, a proven method that supports and facilitates smart city development and engages a wider stakeholder network to co-create new services. During 2016-2018, Smart Kalasatama Agile Piloting Programme has run and facilitated 21 agile pilots. By 2019, more than 50 pilots have run on the different platforms in Helsinki. The model has been adopted in different domains of smart city and used widely in the network of six biggest cities of Finland and the city of Stavanger in Norway. Conventional smart city development often focuses on long term infrastructure projects and is driven by large players, such as construction companies, energy industry, etc. The programme model for Agile piloting was developed in order to accelerate smart city development and enable the participation of smaller players — such as start-ups and SME’s — and to share any learning outcomes with a much wider audience. Agile piloting also provides a means to engage citizens and the users of the services as pilot initiators, co-developers, users and to provide learnings about what smart city development is all about. The model has been documented in CookBook – Recipes for Agile Piloting, a guidebook for city innovators. Experimentation and small, agile pilots provide useful means to approach an uncertain future. Agile piloting is a good method for creating something new in order to uncover the best solution and to gain insight on how users experience the service. Experimentation produces new understanding. The aim is to learn as much as possible, to speed up the development of solutions, and to create new business models. All parties involved gain value from the different learnings generated during experimentation; the company carrying out the pilot and platform facilitating the process, as well as the city and the wider network. In this paper, we briefly present the model and give examples of three cases that highlight different aspects on how co-creation and experimentation has been applied in different city platforms: Smart Kalasatama (Health & Wellbeing), Jätkäsaari Mobility Lab (mobility and transport) and Helsinki schools (education).


1.1 Agile Piloting programme The Agile Piloting programme invites Start-ups and SMEs by open call to test and co-develop their services in real-life environment during a 6-month period. The programme procures pilots for a small compensation (e.g. 1,000–10,000€). The pilots are typically financed with external project funding, but the city, corporate partners or any other actor can act as a funding partner. The programme offers tools to innovate, co-create and experiment new services together with the startups, the city and other stakeholders. Learning is in the heart of the process. The programme engages the different actors of the quadruple helix, e.g. the companies, residents, city officials and researchers, and creates value for all participants. Small start-ups have innovative ideas, but they need support for executing pilots in real-life environment and getting feedback from users. Cocreative process is in the core, and the programme supports the pilots by providing them with methods for co-creation, access to local communities and city infrastructure, new networks, market reference and visibility. The startups have been satisfied: agile piloting accelerates learning, and positive visibility in media for the pilots has been appreciated. Engaging corporate partners to the process allows them to offer technologies or platforms and connect with startups, the city and residents. The residents gain understanding of future solutions and have the opportunity to explore and affect new services. For the city, the pilots are a way of getting a sneak peek into the future, and they work as an anticipation method to forthcoming changes. 1.2 How it works: the process Running a piloting round requires intensive facilitation work from the Living Lab. Facilitation of the experimentation process is about leading networks, communications and hands on work. The most effective use of resources is achieved when pilots are run together as a programme. There are many synergies when the various stages of instruction, facilitation and evaluation are carried out simultaneously with several pilots. A crucial task is the creation of trust among the various players. In addition to the managerial activities related to innovation processes and resources, mediation among stakeholders and activities is needed to create trust and shared meanings enabling shared learning, a shared vision and shared value creation among the multiple actors needed in living labs. (Äyväri, Jyrämä, Hirvikoski, 2018) The facilitator organisation, the living lab or the city engages the various stakeholders in the process in the different stages: defining themes, open call, selection of pilots, experimentation and evaluation. The process is described more in detail in the Cookbook for Agile Piloting Cookbook (Mustonen, Spilling, Bergström. 2018) 1) Stakeholder mapping and defining themes The key questions to start with when initiating a piloting programme are the following: • Who owns the programme? What are the learning goals?



Who are the central cooperation partners who will participate in defining the theme, in funding the pilots and in co-development? Cooperation partners can also provide an environment or other resources for experiments.

The first step when planning an agile piloting programme is defining the themes and the challenge together with the collaborators. This stage is an important one as it is a way to set the common goals and a means to get the essential parties committed. 2) Open Call A piloting programme starts by an open call for pilots. Typically, in open calls for agile pilots, the solutions can be based on existing data, such as open data and use technology in an innovative way. The piloting programme can be used to enhance an existing solution with a new user group, or to add new features to it. Another aim is to test and validate new business models or new ways for involving partner companies. The selected pilots should be linked to the local platforms or the collaborator partners activities. Information about the call is shared through a broad range of channels. The collaborator network’s channels are used widely for communications. The collaboration network is also engaged in the selection of the pilots. 3) Selection of pilots The pilots are selected in collaboration with an expert jury. The living lab is responsible for making agreements with start-ups about the pilots. A common kick-off is arranged for the selected pilots. The key stakeholders are engaged from selection and the kick-off to participate to the process. 4) Experimentation The experimentation phase is an up to 6-month long process, facilitated by the living lab that helps with user recruitment, stakeholder collaboration, integrating to the infrastructure and co-design. The multiplayer network requires active communications with stakeholders about the content and progress of the experimentation. Best way to succeed is to collaborate with partners and engage them to actively communicate through their own channels, and encourage all parties to share their experiences. The living lab facilitates the experimentation process and supports the pilots by recruiting users / professionals, organising workshops, helping to integrate to city infrastructure and service.

5) Evaluation In order to maximise the learnings from the process it is essential to evaluate the process itself, as well as the pilots. Evaluation is an ongoing process, the model consists of several tools to support this such as reporting in the form of light questionnaires in the different phases of the experimentation, and physical facilitated events that gather the relevant stakeholders. The results of the pilots


are documented, and the learnings and best practices disseminated widely - in the best cases, the services are scaled up wider. The overall learnings from the piloting programme can be scaled up and taken forward in all participants own activities.

2 Case 1: Kalasatama Wellbeing - co-creation to engage wider ecosystem 2.1 Smart Kalasatama - smart & sustainable district as a Living Lab Smart Kalasatama in Helsinki, is a former brownfield area, developed to a residential district. By 2035, the district will offer a home for approximately 25,000 residents and jobs for 10,000 people. Currently, there are 3,500 people living in the area. It is a vivid Smart City innovation platform to co-create smart sustainable urban infrastructure and services. Smart Kalasatama is developed flexibly and through piloting, in close collaboration with 200+ stakeholders including residents, companies, city officials and researchers. The Smart Kalasatama Innovator’s Club, gathers the area’s companies, organisations and residents, who take part in defining needs and participate as users in pilots. The vision of the smart district is that smart services save one hour of citizen’s time every day. Building the common vision with the stakeholders served to identify and focus the thematic areas for experimentation. In Kalasatama, the agile pilots have been exploring different areas of smart and sustainable everyday life, such as energy and resource efficient services, sharing economy, health and wellbeing. 2.2 Co-creating wellbeing The Kalasatama Health and Wellbeing Center opened in February 2018. The new centre provides a range of public health and social services available under the same roof. The spaces are flexible and multi-purpose. Already during the construction of the centre, Smart Kalasatama team initiated collaboration with the City of Helsinki in order to define a framework of development and the themes to be taken up further (Hirvikoski, Lehtonen & Äyväri 2016). The new centre opened possibilities for piloting together with professionals and using the Health and Wellbeing centre as a platform for experimentation. The recent Kalasatama Wellbeing piloting programme, in 2018, accentuated the role of corporate partners on the side the City. The partners beyond the city (Kalasatama Health and Wellbeing Centre) were the corporations all active in the district2. Laurea University of Applied Science served as a research and development partner. 2.3 Co-creation throughout the process

2 Construction company SRV Group is an active developer with a commercial center and residential

buildings in the district, Kesko retail group has wide occupational health services, with their new campus located in the heart of the district and IT consultancy CGI Finland, has a range of solutions and services for cities.


The process started by defining the challenge to be solved together, gathering the corporate partners, the city and the living lab team. The thematic framework for the Kalasatama Health and Wellbeing centre as a platform for experimentation defined with the professionals of the city of Helsinki, served as a starting point (Hirvikoski & al 2016). The challenge formulated together was seeking to answer the following question: how can the residents to better take care of their personal wellbeing and health on a daily basis? The open call was launched in January - February 2018 and engaged over 30 start-ups offering their pilots. The expert jury selected the top 10 pilots evaluated with a set of common criteria. These start-ups were invited in a Co-creation Jam, organized to give a better understanding of the collaboration partners aims, the start-ups’ goals and enable to finetune their pilot ideas. The event also gave possibilities to find opportunities for collaboration with the other teams. The Cocreation Jam concentrated on the pilot proposals business model and value proposition, experimentation goals and user experience. The pilots were selected in a pitching session followed by the event. An essential ground for the selection was to identify the real-life platform for each pilot, to enable a quick start for the experimentation phase. The five pilots selected, presented digital services focusing on healthy nutrition and wellbeing, in particular stress management, helping the residents to take better care of their health in daily life. 2.4 Engaging wider ecosystems Agile piloting programme targets to create service ecosystems around the piloting themes, and a wider audience can be invited to the process. These can include the representatives of other sectors of the City, and even non-health and welfare companies, all in all actors supporting the activities and interested to explore the possibilities for cooperation. The pilots serve as a neutral ground for co-creation with a wider ecosystem. Within the 6-month long Kalasatama Wellbeing programme, the start-ups and collaboration partners were gathered in two common meetups. In the first one, the aim was first to strengthen collaboration across pilots and with other partners, help to overcome boundaries in the starting phase. In the next one, the aim was to share learnings from the pilots, as well as to discuss potential to scale the learnings and focus on next steps. The thematic co-creation workshops arranged served to engage additional external partners. The end event of the programme gathered a wide audience to discuss the pilot experiences as well as more generally innovation platform activities in the field of health and wellbeing. A total of 450 end users and 20 professionals participated in the piloting programme. The corporate collaboration partners brought dynamics to the programme by providing optional test beds, potential partners and customers for the start-ups, which was especially appreciated by the start-ups. Furthermore, the programme tested in practice how the collaboration with a university partner can support the living lab phase within the Health and Wellbeing centre, and thus make more intense piloting possible. The students from Laurea University of Applied Sciences were a valuable asset to support the living lab work in practice.


Evaluation played a central part of the programme in order to generate learnings for all parties involved. The pilots were evaluated from the perspective of the end users and the professionals process. On the other hand, the programme was evaluated from the perspective of the start-ups and the collaboration partners. Bringing in corporate partners demands more from the facilitating organisation, but is valued both by the start-ups and the city. The best results occur when the corporate partners have a clear role and can for example provide their technologies or platforms or act as an alternative test bed for pilots.

3 Case 2: Jätkäsaari - Engaging residents to mobility innovation Recently, alongside infrastructure development and regulation, urban mobility challenges have been approached through experimentation and citizen-centric innovation. In Jätkäsaari Mobility Lab, inclusive co-creation and facilitated experimentation are used to boost mobility innovation and to tackle local mobility challenges. Here, we describe how the model for agile piloting, originally developed in Smart Kalasatama, has been applied and adapted in the domain of smart mobility. 3.1 Tackling a real-world mobility challenge Helsinki is one of the busiest passenger ports in Europe, due to extensive ferry traffic between two capital cities, Helsinki in Finland and Tallinn Estonia. The former logistic port of Jätkäsaari has moved to the outskirts of Helsinki, but the passenger port remains in Jätkäsaari, with about seven million passengers passing through the neighbourhood annually. Jätkäsaari is also the largest district development project in Finland, converting the brownfield areas of the former logistics port into homes of almost 20,000 people. Jätkäsaari is connected to the city centre with only two main bridges and suffers from traffic peaks from the port and construction-related traffic. As Jätkäsaari is undergoing serious transformation and as its mobility challenges are very concrete and real, it has been selected as the testbed for novel mobility services, technologies and solutions in Helsinki. There is a long history of mobility-related R&D projects in the area. What differentiates Jätkäsaari from many mobility labs in Finland and abroad, is the involvement of the residents in the co-creation and living lab activities. 3.2 Citizen-centric mobility experimentations The approach applied in Jätkäsaari follows the model of agile piloting developed in Smart Kalasatama. The role of residents in the co-creation is highlighted and they are deeply involved in the key phases of the process - in defining the mobility challenges to be tackled, in co-designing the pilots, in using the piloted services and in giving continuous feedback about them. The local mobility challenges are very tangible and have an impact on the daily lives of the citizens. For this reason, the local community and residents were invited in the process from the very beginning, in defining the mobility issues that the agile piloting would address later.


The content and scope of the piloting round was defined in collaboration with residents, city professionals (urban planners and traffic planners), and private companies. To this end, local mobility needs, development areas and pain points were addressed in four open co-creation workshops and via an online questionnaire targeted at residents during December 2019 - January 2019. The results of the web survey (n=150) gave input to the iterative workshops, that identified the mobility pain points and potential solutions to them. An open call was formulated on the basis of the workshop results. In January-March 2019, the open call was published, with the aim to find new and innovative mobility solutions. Companies were invited to offer pilots that would offer practical solutions to the challenges faced by the residents or to reduce congestion, emissions or other negative impacts of traffic. An expert jury scored the proposals according to the evaluation criteria: novelty and innovativeness, feasibility, and impact. Traffic planners and urban planners from the City were represented in the jury. A total of four pilots were ultimately selected for implementation: 1. A local cargo bike sharing scheme, 2. Shared hobby and sports transports, 3. Local cargo bike logistics, and 4. Smart pedestrian crosswalk. The open kick-off event was organised on 27 May 2019, with co-design sessions open to residents, planners and other local stakeholders. The aim was to gather further ideas for the implementation of the pilots. Once all pilots will be running and open to end-users, the residents will have an active role as co-designers, end-users and feedback providers. The active piloting period with the end-users takes place June-October 2019. The results of this piloting round will be available and shared with the wider ecosystem by the end of the same year. 3.3 Insights to engaging the residents In Jätkäsaari, residents do have high interest in their living environment and strong opinions about its development. The neighbourhood is under construction and the residents want to influence how the whole neighbourhood will turn out to be. However, having them involved in practical terms requires a significant effort from the facilitator of the pilots — in this case Forum Virium Helsinki. A further challenge is to ensure that the process would be as open as possible, and not biased to specific individuals, organisations or interest groups. Utilising existing communication channels, networks, communities, and “gatekeeper organisations” is the key for engaging with the diverse local community. In Jätkäsaari, Forum Virium Helsinki has teamed up with local key actors. Formal neighbourhood association (Jätkäsaari-Seura), informal social media communities (Jätkäsaari-liike), Jätkäsaari district development project of the City of Helsinki, and Port of Helsinki have been valuable partners in the process. Forum Virium Helsinki has presented the upcoming pilots and related co-creation activities in various local events. Some of them have been addressed specifically agile pilots and related living lab activities. In engaging with new residents, the


existing local events organised by others have been very valuable, for example the participatory events of The City, meetings of Jätkäsaari association, and local fairs organised by the Port of Helsinki. The communication of the pilots has been diverse, ranging from websites to posters in the library and local grocery stores, and from social media campaigns to a stand in the events. A dedicated living lab website for piloting and experimentation will be launched during summer 2019. This website will be an arena to communicate topical issues about pilots and their results, a low-step channel for residents to get involved and to exchange ideas about the piloting and living lab activities. The goal is to make the involvement easier for the residents and to make the facilitation of the participation more structured and organised.

4 Case 3: EdTech co-creation with Schools Education Technology is one of the fastest growing fields of business today, already worth more than $6 trillion per year, and growing at the rate of ~7% annually. Looking at these numbers, it’s no wonder that there is great interest in developing new products and services that could take a slice of this pie. However, the field of education is quite protective by nature, and the education community does not react well to outsiders who might be suggesting changes to the prevailing practices. As the world around is changing rapidly through continuous advances in digital technologies, the schools cannot stay inert to the change, but will need to adapt. At the same time, the schools cannot expose their students to the numerous edtech fads that might jeopardise their learning outcomes for the price of convenience or cost-effectiveness. The city of Helsinki, among many other regions, has recognized the situation, and made commitments to speed-up and facilitate the diffusion of innovations into the public-school world. One revelation along this track has been that, there is a clear need for a platform, on which the teachers, students, and edtech companies can co-create, co-develop, and test novel educational tools and methods — and do so while developing corresponding pedagogical practices. Whether the process is driven by the schools or by the business, it is clear that there needs to be clear benefits for both parties; otherwise nobody would be willing to commit to it. 4.1 Towards a new EdTech Co-Creation Platform In 2017, an agile piloting programme begun in order to establish a new, permanent edtech co-creation platform. The plan was to initiate a transparent model for edtech start-ups to co-develop or pilot their products at public schools in Helsinki. Before the pilot programme began, there were already numerous pilots running at Helsinki schools. Due to uncertainty in regulations and policies, these pilots were mostly conducted under-the-radar, without any formal approval from, nor communication with the educational authorities. In order to tackle the lack of transparency, the programme structure was designed to be as flexible and inclusive as possible. The main objective of the piloting programme was to make the piloting more transparent within the Helsinki education community, primarily in order to spread the results and to build up a


culture of co-creation, but also to increase the impact of any pedagogical discoveries or revelations. A plan was made for a piloting programme structure, in which the facilitator organisation was to offer facilitation and formal support in order to achieve higher quality and better outcomes. Following the format of Agile Piloting (Mustonen et. al. 2018) and the Espoo KYKY Model (Sutinen et. al, 2016), the pilot programme structure was following a standard format, while in the spirit of lean development, most of the process facilitation was planned to take place at the beginning of the program. Rather than acting as gatekeepers, the organisers took the role of matchmakers between the schools and businesses, and tried to make sure that any proposed pilot projects had a clear match with an identified need from the educator community. Once the screening and matching had been completed, the facilitators assisted with the planning of the pilot project; after which the co-creators were given almost complete autonomy over their pilot projects. During the piloting programme, the projects had to be initiated by the co-creation platform rather than waiting for either of the parties to make the first move (which would be the expectation for the more permanent platform in the long run). For this reason, a one-day workshop was organised in order to identify specific needs from the user community (i.e. schools), and to transform those needs into formal challenges that would be used to invite edtech companies to participate in the program. About a dozen teachers were invited to as experts to this workshop that took place in two stages. First, a discussion was facilitated around the topic of trends in education and digitalisation. Based on this discussion, a few dozen explicit and latent needs were identified for the education sector. Later on, these needs were categorised and screened in order to forge them into specific challenges. Co-creators were then selected through an open call, followed by a workshop in which facilitators helped the participants to create solid plans for their Agile Pilots. After this workshop, the facilitators stepped back, and allowed the co-creators to focus on conducting their pilots. After the piloting period was over, a facilitated wrap-up meeting was held between the co-creators. At this meeting, the experiment was evaluated on two levels. First, the execution of the pilot project was analysed, making note of how accurately the piloting plan was followed, what type of facilitation took place during the process, how well the schedule held, etc. Second, the outcomes of the pilot were evaluated from participants perspectives; did the results satisfy the expectations, were any new discoveries made, etc. During the first two rounds of the piloting programme, 16 edtech start-ups were running a total of 27 co-development or pilot projects in collaboration with 17 schools, involving roughly 50 teachers and more than 1500 students. The piloting programme evolved to a model labelled as Easy Access Co-development, which is expected to become the core of the more permanent edtech co-creation platform.

5 Scaling up learnings Piloting is a way to start anything - a systemic change, new services and activities. A city makes the most out of piloting by linking the themes to the strategic aims.


It also provides a means to scale up: after running several piloting programmes, and tens of pilots we have explored that the learnings from the pilots keep on living and are available for the stakeholders to take further in their own activities. In some cases, the pilots as such or in a format incorporating learnings are scaled up to other districts. Agile piloting in smart city development is a fast way to gain more insight into problems and make them visible as well as to engage a range of stakeholders. It enables the whole urban community to learn as much as possible during the intensive piloting period. Piloting reveals the challenges in scaling up smart solutions. Most of the solutions share the same challenges, such as lack of business ecosystems, legislative and regulatory issues, lack of interoperability and the slowness of change in behaviour of the users in adapting new services. (Mustonen & al, 2018). 5. 1 Added value by engaging diverse actors Although the approach of agile piloting has been adapted in varying contexts, the engagement of diverse actors remains as one of its key elements. Co-creation with wider network - private companies, citizens, civil servants, research partners - adds value but does not happen by itself. Implementing a high-quality piloting programme requires resources, in terms of working time, expertise and other resources from the facilitating organisation. This should not come as a surprise to the facilitating organisation. The experience showed that engaging corporate partners as collaborators in the piloting programme brought new dynamics to the process, and also engaged more mature start-ups to participate. Both start-ups and corporate partners appreciate an efficiently facilitated process: it saves time and resources for other actors, which is a major benefit to start-ups in particular. The co-creation workshops and user feedback provide valuable lessons learned and follow-up ideas concerning the development of services and business models. Piloting in real-life environment also provides unanticipated learnings that were proven particularly valuable. The start-ups gave positive feedback on the large number of contacts that the programme allowed them to establish and especially appreciated the visibility given to the pilots. Running a piloting programme, is not only effective use of resources for the living lab, but it also creates synergies among the start-ups participating in the programme. The facilitator supports collaboration by the various activities arranged during the different steps in the piloting programme. City professionals are crucial stakeholders for many reasons. In a health and wellbeing context the professionals are an essential and truly valuable user group - that often acts as a gatekeeper organisation to the end users. In mobility context, the planners participated in the definition of the mobility challenges and in the selection of the pilots, bringing necessary expertise and realism in the most highflying ideas from the residents. Bringing civil servants on board in early stages increases the commitment and ownership in the process.


Even when part of the costs of the pilots are being covered for the start-ups, money is not the primary reason for taking part in agile piloting. According to our experience, what is valued the most are the real urban testing environment and real end-users, facilitated piloting process, support in communication, and access to networks of key stakeholders and potential cooperation partners. 5.2 Trust and clear rules of the game Developing a trustful communication with the residents takes time and effort. It is crucial to identify the existing networks, organisations and gatekeepers and cooperate with them. Identifying the key local actors is efficient timewise, and cooperation with them helps to understand the local context, wishes and concerns of the locals, and it also increases the commitment of the local community to the process. When collaborating with the residents, it is important to speak their language and not to confuse them with “innovation jargon�. It is important that they understand that pilots provide something practical and useful that they can understand and relate to. Communicating the results of the pilots back to active residents is necessary in order to foster the relationship and commitment to future pilots as well. Working with the school or health care centre provides a different living lab environment, where most often the communications to the end users runs through the teachers and healthcare professionals, if not agreed otherwise. There the key is clear rules of the game with the professionals whose working environment and processes are central.

6 Summary Agile piloting provides a flexible framework for co-creating urban futures. Experimentation also provides learnings for the living lab - the model is an adaptive process that can be modified and further developed to better accelerate smart city development and provide value for the different stakeholders. This has been proven by the wide adaptation in the cities in Finland, with over 100 pilots run and interest from other Nordic countries. For us the Agile Piloting programme model has been a learning process on how to best accelerate the creation of smart urban services and engage the urban community. It has also provided means for creating wider ecosystems that take the learnings of the pilots further in their own activities. Piloting in different reallife environments of a smart city has different requirements in terms of users, city professionals like teachers or healthcare professionals and city infrastructure. What is common for all of them is that co-creation is a valuable means for developing better future services and agile pilots to provide a neutral platform for co-creation. The three case examples of this paper demonstrate that the model for agile piloting can be applied in very different domains. It is essential for the facilitator organisation to know the thematic and local context and adjust the activities


accordingly. As highlighted in the three case examples, different actors may have very different perspective towards piloting. The facilitator organisation should be aware of the different perspectives, expectations and concerns and recognise these in the process. We conclude that Agile Piloting has a lot of potential to spark innovation and to solve concrete challenges in different contexts. However, the programme for agile piloting is not a rigid model that fits automatically everywhere, but rather a flexible framework that should be - and has been - adapted in varying ways in different settings.

References IBIS Capital (2013) Global e-Learning Investment Review Hirvikoski T., Lehto P., Äyväri A., (2016). Development and experimentation platform for social, health and wellbeing services in the context of Kalasatama health and wellbeing centre., Laurea-ammattikorkeakoulu. ISBN 978-951-799-441-5 Mustonen, V., Spilling, K., Bergström, M., (2018). Cook Book – Recipes for agile piloting. Helsinki: Forum Virium Helsinki Rill, B. R., & Hämäläinen, M. M. (2018). The Art of Co-Creation. Palgrave Mcmillan. ISBN 978-981-10-8500-0 Sutinen P., Erkkilä K., Wollstén P., Hagman K., Hirvikoski T., Äyväri A., (2016). KYKY Living Lab handbook for co-creation by schools and companies., Espoo: City of Espoo. ISBN 978-951-857-732-7 Äyväri A., Jyrämä A., Hirvikoski T., (2018). The Circle of Mediators: Towards a governance model for tackling sustainability challenges in a city. , European Network of Living Labs. URN:NBN:fi:amk-2018090414850 Äyväri A. (2018), Kalasataman terveys- ja hyvinvointikeskus kehittämis- ja kokeilualustana: tavoitteet, palvelut ja prosessit yritysten ja alustan näkökulmasta, Laurea-ammattikorkeakoulu


Co-Creating Technology for Societal Change: A Mobile App Addressing Homelessness

Rachel Burrows*1,2,3, Antonette Mendoza2, Sonja Pedell3, Leon Sterling2,3, Tim Miller2 and Alexi Lopez-Lorca4


*Corresponding author Ltd, Cambridge Science Park, United Kingdom 2 University of Melbourne, Australia 3Swinburne University of Technology, Australia 4 Semantic Web Company, Vienna, Austria Category: Full Research Topic: Open Track

Abstract Living Lab projects often involve the collaboration of diverse stakeholders. This is particularly true with new technology that aims to tackle the systemic and societal problem of homelessness. In this paper, we present a mixed-method approach to understand the perspectives of key stakeholders. We discuss our findings and their implications for the development of a mobile app that aims to help people who are homeless. We measure usage of the mobile app which currently attracts over 10,000 users each month in Australia. We also conduct semi-structured interviews with 30 participants who are either homeless, exhomeless or service providers. Our study provides insights and an approach that may help others in developing similar systems. We discuss barriers and enablers of success relating to (i) organisational concerns from service providers, (ii) maintaining awareness of the system in the homeless community, and (iii) supporting user needs in software design. We propose and demonstrate our emotion-led approach to bring a novel perspective on the concerns from key actors influencing the adoption of new technologies. Keywords: Thematic Analysis, Homelessness, Emotion-led Design, MixedMethod, Technology innovation, Living Lab approach, Socio- technical systems


1 Introduction Homelessness is a systemic societal problem that requires a holistic approach to progress towards a solution. In Australia, many organisations and communities aim to help people experiencing homelessness transition into stable living situations. Existing service providers may help with a range of needs from temporary accommodation, meals, or finding a job. Despite efforts, the number of people in Australia experiencing homelessness has increased by 14% in the five years leading up to 2016 according to the Australian Bureau of Statistics. This urgent situation has led to new initiatives to improve the access that people experiencing homelessness have to relevant information about services. One way to achieve this is through the development of new technology that helps people experiencing homelessness access the help that they need. Unfortunately, introducing new technology to help those who are homeless brings unique challenges. Existing research outlines challenges that are faced by technology developers (Woelfer et al 2009, 2011a, 2011b, 2012; Hersberger 2003, 2012; Chatman et al 1996; Le Dantec et al 2008; MunĂľz et al 2004; Humphry 2014; Deng et al 2016). For instance, Woelfer (2009) and Chatman (1996) suggest this group of users may not have the necessary devices or motivation to access information online. Hersberger (2003) suggests that this group of users suffer from information overload. New technology may, therefore, risk contributing to their confusion. Finally, Weise et al (2017) outline how new technology may be resisted due to incompatibility with existing service provider protocols. Due to the sensitive nature of this application domain, and the complexity of the socio- technical system, we use an emotion-led approach to guide our analysis and understanding. New emotion-led approaches to software design and evaluation are now being increasingly utilised. These approaches aim to understand how we can resolve concerns of all stakeholders and address their needs. People will reject technology if it does not address their emotional concerns (Dix et al 2003; Krumbholz et al 2000; Norman 2005; Pedell et al 2014; Miller et al 2015). Emotional experiences with technology are formed and then change over time based not only on the actual technology engagement experience but also are layered with associated experiences (Saffarizadeh et al 2017; Alatawi et al 2018). This need is heightened when creating technology for vulnerable users. For instance, users may wish to feel in control, connected, hopeful, cared for, or empowered, among others (Toscos et al 2013; Pedell et al. 2014; Saffarizadeh et al 2017). This work reacts to questions and uncertainties around new technology for homelessness. There is a lack of understanding over if and how new technology can be introduced into this complex system of service providers. While there are some studies that focus on designing for vulnerable user groups, and even those who are homeless, these studies do not focus on emotional experiences and also do not evaluate a deployed system. Our work has contributed to the development of a web app that helps homeless Australians find information about services that can help them. The app has been deployed since 2016, contains information about services providers, and currently attracts over 10,000 users each month.


There are 16 service categories including food, housing, everyday needs, money help and counselling. The objective of this work is to (i) provide an in-depth understanding of the key barriers and enablers to success, and (ii) provide advice for living lab projects who are facing similar challenges. The remainder of this paper is structured as follows: We outline relevant literature on homelessness, and approaches for designing technology that helps those experiencing homelessness. We then outline our research method and present findings. Quantitative user engagement data was collected to understand usage trends. We also conduct semi-structured interviews with participants who were either homeless, ex-homeless, or service providers. Seven themes emerged that are associated with emotional viewpoints or experiences of key actors. These themes represented both barriers and enablers of success depending on how they were addressed. We finally discuss and provide advice in relation to three challenges that may be faced by similar initiatives. 1.1 Homelessness and Service Providers The Australian Bureau of Statistics defines homelessness as: “ a dwelling that is inadequate; or has no tenure, or if their initial tenure is short and not extendable; or does not allow them to have control of, and access to space for social relations.� There are many government services across Australia whose purpose is to help those experiencing homelessness to find the help that they need. Service providers allocate an extensive range of support, and become largely responsible for the diffusion of new information to those who are homeless via a mix of government funded organisations and grassroots organisations (Woelfer et al 2009). There are many opportunities to support existing service providers by developing technology that facilitates online help-seeking activities. The new means of accessing information can often be met with resistance by existing service providers, though. One study by Woelfer et al (2011) suggests that we should take a precautionary stance when it comes to providing access to complex service information online and has even suggested that new technology may not be an appropriate solution. This is because most information for homeless people is currently exchanged in face-to-face situations (Hersberger et al 2013; Le Dantec et al 2008). The traditional way of interacting with these services allows service providers to tailor advice and recommendations on a case by case basis. It also allows service providers to retain control over what information is provided to each person. This control is often preferred by service providers since the new means of accessing information creates new expectations upon the service providers. Such new expectations may be hard for the service providers to meet (Le Dantec et al 2008; Weise et al 2017). Work by Hersberger et al (2013) and Chatman et al (1996) suggests that those who are homeless are already overwhelmed by information provided by services and are unlikely to own a smartphone to access this information.


Those experiencing homelessness represent a unique user group. The major causes of homelessness according to the Australian Bureau of Statistics include family violence or financial difficulties. In fact, only 6% of people who are considered homeless are characterised as rough sleeping; many live in severely overcrowded homes or transitional accommodation. Additional problems accumulate over time, such as drug and alcohol abuse, creating a viscous cycle and a worsening situation (Woelfer et al 2009). Despite the scepticism around communicating complex service information online, there is a counter body of evidence that suggests it would be beneficial to many. A mobile phone is often viewed as a necessity and a critical lifeline (Le Dantec et al 2008). A more recent study estimated most people experiencing homelessness prioritise retaining their mobile phone to keep in touch with family, friends and necessary service providers (Humphry 2014). Even those without a smart phone may have access to the internet via alternative means, such as the library or with the help of case workers. Work by Woelfer et al (2012) indicates that the use of social media by young homeless people is extensive and that younger adults utilise numerous online information seeking strategies.

2. Method 2.1 An Emotion-led Approach People will reject technology if it does not address their emotional concerns (Dix et al 2003; Krumbholz et al 2000; Norman 2005; Wood et al 2006; Pedell et al 2014; Miller et al 2015). For this reason, there is a growing body of work that calls for an increased focus on information about the emotions of stakeholders to improve the design and evaluation of new technology. Emotional experiences may be related to aspects of the software design, such as a particular feature that is displayed. Work on socio-materiality and technology affordances (Orlikowski et al 2008; Vaat et al 2013; Leonardi 2013) shows how aspects of design can trigger positive and negative emotional perceptions. Emotional experiences associated with a particular technology are also influenced by external factors, including other individuals or organisations that are associated with the engagement experience. Misplaced expectations may still be attributed to the technology itself due to multiple experiences becoming aggregated and associated with each other (Wood et al 2006). These experiences could include those occurring during the progression towards a goal (Clore et al 2008; Luce et al 2001). Consequently, in our case study, participants may interact with a service provider and their reaction is attributed to the technology that facilitated the interactions. While there are a variety of popular psychological frameworks that characterise emotions, their content and utility for system design and evaluation will vary. In fact, the current range of emotional concerns that are considered has already been criticised as being ‘narrow’ (Dunne et al 2001). Some psychological frameworks are grounded in primary (also referred to as basic) emotions such as fear, anger, or joy (Ekman et al (1992); Schwarz and Clore (1983)). Other emotional frameworks contain different types of emotions, including those that are more reflective. For instance, some emotions are characterised by having


relatively lower levels of arousal and involve relatively higher levels of reflective, cognitive processes; examples include the characterisation of shame and resentment (Martin and Tesser (1996); Desmet and Hekkert (2007); Plutchik (2003)). Due to the sensitive nature of this domain, we utilise an emotion-led approach. This means we characterise results according to the way stakeholders wish to feel when using technology and place emphasis on emotional concerns when analysing results. Similar to work by Deng et al (2016), we do not rely on a single classification or framework to interpret our results. This is because it is important to inquire about the wide ranging and often subtle feelings and perspectives of stakeholders that may not be accurately represented by a generalisable psychological framework. Our research has contributed to the design and evaluation of a web app for homelessness. In this paper, we present a mixed-method study that investigates the challenges for developing technology for homelessness. The quantitative phase provides an overview of technology use collected from all users, spanning the two years since deployment. Quantitative data enabled us to discover usage trends over time. The qualitative phase is centred around 30 in-depth semistructured interviews that were conducted with a range of stakeholders. Seven themes emerged from the thematic analysis as a basis for further discussion. 2.2 The Web App The web app aims to tackle the problem of homelessness by assisting those who are homeless in finding useful information. It provides information from over 350,000 services in Australia. The listed services range from help with health issues, food, shelter through to legal and financial advice. Figure 1 shows three example screenshots. Our recent work has already focused on evaluating specific design decisions (Burrows et al 2019a 2019b). Differently, this paper takes a different perspective to understand the impact and role that Ask Izzy has as part of a broader socio-technical system. A typical use involves the browse/search page. A user is presented with 16 help categories. The user can choose to give their location and is guided through a series of category-specific questions. Based on these answers, the user is presented with a service list compiled via a service filter process, detailing results of services that match their criteria - and potentially ordered by relevance.


Figure 1. Screenshots of second version of Ask Izzy.

A user can select a particular service to view its detailed service page. The detailed service page displays information about how to connect with the particular service, how to get there, who it is for, and what clients should expect. 2.3 Data Collection and Analysis


Usage data was collected over a period of two years. We also collected data on how often users returned to the web app. An overview of the number of people that used the app is shown in Table 2. These time windows were selected in order to capture data that was representative of the normal use while spanning enough time to account for seasonal variations. A key concern for this system was that the people would cease to continue to use it in the long term. We therefore capture usage data over a longer-term. For instance, demand for services typically spikes in winter and around New Year (Australian summer). The returning behaviour for a user captures the number of days since their last session. For this finer-grained analysis, we present aggregated data based on percentage change to preserve anonymity. Returning behaviour is compared by taking four 12-week time windows and then analysing how often users returned to the web app for seven days after the initial session. A session starts when a user engages with the web app and ends after the default of 30 mins of inactivity.

Table 1. Interview Participants

Stakeholder group


Sample Coverage

Homeless and Ex-Homeless


Adult, Youth, Family Violence, Veteran, Mental or Emotional Difficulties, Drugs and Alcohol Problems, With Children


Official Service Providers including Government funded providers and Charities.


A representative from the software company

Service Providers Software Owner

Two researchers conducted a series of semi-structured, one-hour interviews with 30 participants, six months after the deployment of the web app. We took care to ensure the recruitment procedures and interview locations were appropriate; participants were recruited via existing service providers who were also able to provide a familiar environment for the discussion to take place. Participants who were homeless had some experience with the app. This ranged from a single use to frequent use over the 6-month period. Service providers were aware of the web app and therefore played a role in raising awareness with the homeless people with whom they were in contact. Table 1 gives an overview of the participants and different stakeholder groups that were interviewed. Participants were selected to represent a range of people who have a stake in the success of the application and who have had first- hand experience with the web app. This included those who were homeless, ex-homeless, service providers, and the software company. A semi-structured interview was chosen to give flexibility to the conversation. It allowed participants to diverge and discuss contextual factors that may be unexpectedly related to their perceptions of the web app. With regard to the


software design, we asked what they liked, disliked, and what they would change in the mobile app. We also asked how using it made them feel. We discussed interactions and experiences outside of the application including how they heard about it and if they had recommended it or supported others in using it. We then asked what they thought were the barriers to uptake. If they chose not to use the web app, we asked for the reason. The qualitative data comprised of audio files recorded from the semi-structured interviews. All transcripts and audio recordings were imported into the NVivo tool (Bazeley 2013). The results were analysed by two authors following a thematic data analysis process (Braun 2006) in order to identify, analyse, and report the themes from the data. Codes (i.e. quotes) were extracted from the transcripts that were related to emotional experiences or concerns. The codes of the transcripts were grouped individually into themes and then later discussed and merged to form a final agreed set of themes. Following guidelines for thematic analysis, and similar to comparable approaches (e.g. Deng 2016), no specific framework of emotions was used to categorise the elicited codes. Any text phrases that were thought of as representing the way the participant would or would not like to feel were marked and extracted. We retained information about the role of each participant in order to contrast views of service providers with those who are homeless or ex- homeless. Each theme we analysed according to different stakeholder groups.

3 Results We now describe key findings from the quantitative and qualitative phases of the study. The subsequent section discusses implications of these findings. 3.1 Quantitative Data about the numbers of users is shown in Table 2. This gives an overview of the numbers of users that were visiting in a two-year period. Each row shows the weekly mean, median, standard deviation, range, minimum and maximum number of users. Figure 2 shows information about the number of users from four different time windows over the entire two-year data collection period. Figure 3 shows two charts that capture returning behaviour.

Table 2. Descriptive Statistics


Weekly Measure Mean Median Std. Dev. Range Min. Max. Total

Year 1 number of users 817 836 219 895 473 1368 42464

Year 2 number of users 2287 2328 622 2557 1046 3603 118949

Returning behaviour is a popular measure to understand how often users repeatedly visit the app. There is no ‘ideal’ time period, however by looking at the patterns and changes over time it is possible to see how using the app is used sporadically, routinely, or how new features that are designed and implemented change the patterns of engagement. To give an example, let’s consider the chart on the right. We can see that, in time window 4, the number of users that returned to the web app after 7 days has increased by 800% when compared to the 1st time window. Time window 1 is used as a baseline for comparison and therefore not shown on the charts (as it would appear as 0%). If a user has more than one session in the same day, the extra sessions are counted as ‘0 Days’.

Figure 2. Overview of User Numbers


Figure 3. Percentage Change in Number of Sessions.

The chart on the left is showing users that belong to an active user base (multiuser sessions) whereas the chart on the right is showing all users. New users are also counted as ‘0 Days’; and for this reason it is popular practice to report multi-session users1 separately. A focus on multi-session users can therefore be seen in the left-hand chart in Figure 3. Multi-session users are those who have had more than one session within a particular time window and are considered to represent an active user base. For more information about we refer the reader to google analytics documentation3. 3.2 Qualitative Table 3 describes seven themes that emerged from analysis. They represent barriers or enablers to the uptake and use of the app. Each theme is represented and articulated as a positive goal to be addressed. These goals therefore can be interpreted as enablers of success if they are adequately addressed, or, barriers if they are not. After reading the transcripts, 107 codes were extracted and subsequently grouped into the 7 themes to represent how users would like to feel. These themes were Empowerment and Control, Assurance, Cared For, Identity and Belonging, Clarity, Unashamed / Without Stigma and Hopeful. We give a brief description of these themes in this section and their implications are discussed in the following section. Figure 4 visualises the key groups of people who are either directly or indirectly influential in the success of Ask Izzy. The figure depicts system entities who are either individuals, organisations or communities in this socio-technical system. It depicts the collaboration and the communication between these entities (represented by the arrows). Finally, it depicts the technology engagement. Technology engagement is also integral to the flow of information as we are able to monitor and react to user engagement from in-app measures.


Google Analytics:


Table 3. Emergent Themes

Number of Codes

Empowerment and Control


Cared For




Description New means of accessing information empowers information-seekers and increased choice provides a greater amount of control over how they interact with service providers. Assurance is a theme related to the levels of trust and confidence in information received via the system or a person. If participants sawevidence that individuals were working to provide them with help then this provided them with a sense that they were cared for and cared about.

Identity and Belonging


Visual imagery within the application was important to signal that this application was created inclusively for the cultural needs of multiple user groups.



This theme relates to the need to present relevant information without overwhelming users.

Unashamed and Without Stigma


The stigma around being homeless reduces the motivation of people to seek help.


Hopefulness encourages further help-seeking behaviour. It is also important to be realistic about the extent and existing service provider can help.


4 Discussion Overall, the project had a positive impact and was used by many homeless people to seek help. The quantitative findings indicate that overall use and the percentage of returning users are increasing. The average number of users in the first year of deployment was 817, this increased to 2287 in the second year each week. Over 10,000 users are now accessing the web app each month which is evidence that a large proportion of people are choosing to seek information online via the web app. In this section we discuss the barriers and enablers of success in this project and reflect on the implications with respect to the living lab methodology. We break down the discussion into three sections. Firstly, we discuss how the themes were associated with practices and protocols regarding


how members of the public interacted with service providers. Secondly, we discuss how these themes provided insights on how to maintain momentum and awareness of the system. Finally, we illustrate how themes were associated with aspects of the software design.

Figure 4. Collaboration Pathways between Organisations, Communities, and Individuals

4.1 Organisational Implications: New ways of discovering services The system forged a new means for those who are homeless to discover and interact with service providers. This was seen as empowering by those who are homeless. Differently, service providers, were concerned about the lack of control they had about how people were passed between coordinating organisations. This challenge was further debated when designing the information that was presented in the web app itself. On one hand, if the service list within the application presented services which were usually only recommended in-person (by a qualified service provider), then there was a risk that a client would attempt to access a service that was inappropriate to their situation. On the other hand, if these services were removed from the list, then the reduction in options would limit the choice and control for the client with regard to the ways of interacting and accessing services. If we consider Figure 4, it is possible to understand how two key information flow links could easily break. Service providers had great influence on adoption success as they would cease to recommend the web app if they perceived the presented information as risky. Similarly, clients recommending to other clients


were at risk of ceasing to recommend the app if they did not feel their needs were addressed. One service provider said: “we like to think that information only exists in hard copy form and that’s been a challenge we’ve had for a long period of time. But even in a meeting we had recently the number of people that identified the [web app] and how much they use it and how useful it is” Results indicate that a large number of people who are homeless chose to find service information online. One person who used to be homeless stated: “I would go to internet cafes to research information on homelessness and where I can get help from. I had no idea, I had no idea” It is possible to see that the number of users who returned to the app also increased over time. Further barriers were removed as the software company provided free battery packs so that those who are homeless could charge their phones and access the website for extended periods of time. Additionally, the website was unmetered which meant that accessing the website did not cost any money. 4.2 Maintaining Momentum Projects like Ask Izzy rely on the continued sharing of information in order to maintain user engagement momentum. This is because new people are constantly becoming homeless, or finding a home. Similarly, service providers often undergo organisational changes, meaning new people take on the recommending roles. These are just two examples of how the people who could recommend or benefit from Ask Izzy are constantly changing. Communication pathways need to be maintained so that those who are homeless are aware and made aware of Ask Izzy. If service providers do not feel assured that the data is accurate, or if they feel that the amount of control that is given to clients may have adverse consequences, then they may choose not to recommend the application. Even if service providers are willing to promote the web app, there is still uncertainty around who will be responsible for ensuring that this awareness is maintained.

One service provider stated: “I guess your question was who should be responsible for it, I mean I guess it goes to who owns [the app] and I would think that the perception [is that


the software company] owns it, so it’s their thing. And that’s problematic in terms of ongoing funding” The participants in our study, including service providers and those who had experienced homeless outlined that a broader group of individuals and organisations could contribute to maintaining awareness of Ask Izzy. Based on results from interviews, these people could be: (1) service providers and case workers who are client-facing, (2) prominent members of Aboriginal and Torres Strait Islander communities (e.g. Elders), and (3) the general public. In many instances, the interviewees suggested increasing awareness in hospitals, police, and public transport workers as these groups frequently come into contact with people needing help in their day to day work. A participant who had been previously homeless said: “it would be good to do some advertising on the Metro or trams or something, some advertising, when people are sitting on the train or the tram and they’re approached by people who are asking for money” Peer-to-peer recommendation of the application was often described as trusted source of information. People felt assured and cared for and trusted the advice of others in a similar situation. Consequently, awareness of the web app was effectively communicated socially at events. “some of the services or they go and have a meal there or catch up with people they know … So there was a lot of this talk bubbling around and I think that’s for other people that were homeless it tends to be quite effective” In this quote the participant was describing how the tips and strategies from other people who had a lived experience of homelessness were effective. In these cases, certain services, such as food vans became locations where information was routinely exchanged. Interestingly, there is an increasing number of users who return to the web app after 7 days. This may be explained by people accessing the web app after hearing about it via peer-to-peer recommendation at a repeating gathering, meeting or event. There is a possibility that an event or activity that occurs weekly triggers motivation to use the web app. In fact, quite often an expert user had initiated interaction to help another person access a particular service. Young adults often asked questions to their case worker whose hands-on guidance would take the stress away from searching for appropriate courses of action. One participant who used to be homeless explained how he often helped others who didn’t have access to a smart phone and WIFI. Interactions and raising awareness about the web app often occurred during activities that were social. By understanding how multiple stakeholders are able to collaborate we are able to understand the ongoing success of Ask Izzy as dynamic and constantly changing.


4.3 Reconciling Diverse Viewpoints and Trade-offs in Software Design The themes elicited from the interviews were also related to aspects of the software design. In this application domain, the interaction with the web app is one small part of a much longer help-seeking journey. However, it is important to consider how the initial interaction with the web app creates expectations that may have consequences on future actions. If the expectation created in design is not realised by the service provider then this may have negative consequences on both service providers and the success of the web app itself. As an example, consider the theme of hopefulness. In this circumstance, the overriding expectations and hope of finding what is needed (i.e. a home) can vary greatly. One service provider stated that they “spend a lot of time managing client expectations”. Consequently, avoiding disappointment remains an ongoing challenge. The engagement with the web app becomes a trigger where hope is created. The expectation that is initially created is then acted upon and then changes with each subsequent interaction with service providers. For many individuals experiencing homelessness will be a number of years. Therefore, the more this need is addressed in design (the greater the initial hope), the greater the risk of negative consequences in the long-term (expectations not met). One service provider stated that “you’re actually setting the expectation that there’s hope there when actually that’s misplaced”. Someone who used to be homeless recalled her experience after finding a service advertised to only find that the service was not able to help: “It’s so depressing when you’re trying to find accommodation and you read something and say oh my god, they’ve got support to transitional accommodation. I remember this happened with me when I was homeless and I’d get really excited, oh great, I’ve found a place, maybe they can help me, hang up after hang up, after hang up.” In this sense, one of the greatest challenges for technology design is to provide clarity and hope about what can be achieved by seeking help without reducing motivation to seek help. Again, taking a holistic approach allowed technology developers to consider the consequences that specific design features would have on future actions and interactions with service providers.

5 Conclusion This paper investigates the key challenges of developing new technology to help those who are homeless. We believe the lessons from our project may be useful to others who are creating new technology for socially-complex projects. We conducted a mixed-method study based on the development of a web app. We contrast the different perspectives of people who were homeless, ex-homeless, service providers and a representative from the software development company. We found that an emotion-led approach was useful to understand stakeholder concerns in the context of the broader socio-technical system. People who are


homeless wanted to feel empowered, in control, assured, cared for, unashamed and hopeful while interacting with the web app. In particular, we discuss the technology design and the impact it had on existing organisational practices and protocols. We also discuss the challenge of maintaining momentum and awareness of the web app. We explain the importance of being mindful so as not to raise unrealistic expectations. Finally, we give an example of how our approach was used to inform design trade-offs.

Acknowledgements This research has been made possible by the support and collaboration with Infoxchange a not-for-profit social enterprise developing technology for social change. This project was funded by the Australian Research Council Discovery Grant DP160104083 “Catering for individuals' emotions in technology development”.

References Alatawi, E., Mendoza, A., and Miller, T. 2018. “Psychologically-Driven Requirements Engineering: A Case Study in Depression Care,” In 25th Australasian Software Engineering Conference (ASWEC), IEEE. Ask Izzy Website. “”: accessed 15/01/2019. Bazeley, P., and Jackson, K. 2013. Qualitative Data Analysis with NVivo, Sage Publications. Burrows, R., Mendoza, A., Sterling, L., Miller, T., Pedell, S. 2019. Evaluating Ask Izzy: A Mobile Web App for People Experiencing Homelessness. In: Proceedings of the 17th European Conference on Computer-Supported Cooperative Work. Burrows, R., Lopez-Lorca, A., Sterling, L.; Miller, T., Mendoza, A.; Pedell, S. 2019. Motivational Modelling in Software for Homelessness: Lessons from an Industrial Study. In Proceedings of the 27th IEEE International Conference on Requirements Engineering. Chamberlain, C., Johnson, G., and Robinson, C. 2014. Homelessness in Australia, UNSW Press. Chatman, E. A. 1996. “The impoverished lifeworld of outsiders,” Journal of the American Society for information science (47:3), pp. 193–206. Clore, G. L., and Ortony, A. 2008. “Appraisal theories: How cognition shapes affect into emotion.” Deng X. N., Joshi K. D., and Galliers R. D., 2016. “The duality of empowerment and marginalization in microtask crowdsourcing: giving voice to the less powerful through value sensitive design” MIS Q. 40, 2 (June 2016), 279302. Desmet, P., and Hekkert, P. 2007. “Framework of product experience,” International Journal of Design (1:1). Dix, A., Finlay, J. E., Abowd, G. D., and Beale, R. 2003. Human-Computer Interaction (3rd Edition), Prentice-Hall, Inc.


Ekman, P. 1992. “An argument for basic emotions,” Cognition & Emotion (6:3-4), pp.169–200. Griffiths, P. E., and Scarantino, A. 2005. “Emotions in the wild: The situated perspective on emotion,”. Hersberger, J. 2003. “A qualitative approach to examining information transfer via social networks among homeless populations,” The New Review of Information Behaviour Research (4:1), pp. 95–108. Hersberger, J. A. 2013. “Are the economically poor information poor? Does the digital divide affect the homeless and access to information?” in Proceedings of the Annual Conference of CAIS/Actes du congrès annuel de l’ACSI. Hou, Y., Lampe, C., Bulinski, M., and Prescott, J. J. 2017. “Factors in Fairness and Emotion in Online Case Resolution Systems,” in Proceedings of the CHI Conference on Human Factors in Computing Systems, ACM. Humphry, J. 2014. Homeless and Connected: Mobile phones and the Internet in the lives of homeless Australians. Krumbholz, M. a., Galliers, J., Coulianos, N., and Maiden, N. 2000. “Implementing enterprise resource planning packages in different corporate and national cultures,” Journal of Information Technology (15:4), pp. 267–279. Le Dantec, C. A., and Edwards, W. K. 2008. “Designs on dignity: perceptions of technology among the homeless,” in Proceedings of the SIGCHI conference on human factors in computing systems, ACM. Leonardi, P. M. 2013. “Theoretical foundations for the study of sociomateriality,” Information and Organization (23:2), pp. 59–76. Lowry, P. B., Gaskin, J., Twyman, N., Hammer, B., and Roberts, T. 2012. “Taking fun and Game seriously: Proposing the hedonic-motivation system adoption model (HMSAM)”. Luce, M. F., Bettman, J. R., and Payne, J. W. 2001. “Emotional decisions: Tradeoff difficulty and coping in consumer choice,” Monographs of the Journal of Consumer Research (1), pp. 1–209. Majchrzak, A., Faraj, S., Kane, G. C., and Azad, B. 2013. “The Contradictory Influence of Social Media Affordances on Online Communal Knowledge Sharing,” Journal of Computer- Mediated Communication (19:1), pp. 38– 55. Massimi, M., Dimond, J. P., and Le Dantec, C. A. 2012. “Finding a new normal: the role of technology in life disruptions,” in Proc. of the ACM Conference on Computer-Supported Cooperative Work, ACM. Miller, T., Pedell, S., Lopez-Lorca, A. A., Mendoza, A., Sterling, L., and Keirnan, A. 2015. “Emotion-led modelling for people-oriented requirements engineering: the case study of emergency systems,” Journal of Systems and Software (105), pp. 54–71.


Muñoz, M., Vázquez, C., and Vázquez, J. J. 2004. “A comparison between homeless, domiciled and vulnerable populations in Madrid,” Population (59:1), pp. 129–142. Norman, D. A. 2005. Emotional design: Why we love (or hate) everyday things, Basic books. “Census of Population and Housing: Estimating homelessness, 2016,” Orlikowski, W. J., and Scott, S. V. 2008. “10 Sociomateriality: Challenging the Separation of Technology, Work and Organization,” Academy of Management Annals (2:1), pp. 433–474. Pedell, S., Keirnan, A., Priday, G., Miller, T., Mendoza, A., Lopez-Lorca, A., and Sterling, L. 2017. “Methods for Supporting Older Users in Communicating Their Emotions at Different Phases of a Living Lab Project”. Pedell, S., Miller, T., Vetere, F., Sterling, L., and Howard, S. 2014. “SociallyOriented Requirements Engineering: Software Engineering Meets Ethnography,” in Perspectives on Culture and Agent-based Simulations, Springer, pp. 191–210. Plutchik, R. 2003. Emotions and life: Perspectives from psychology, biology, and evolution. American Psychological Association. Saffarizadeh, K., Boodraj, M., and Alashoor, T. M. 2017. “Conversational Assistants: Investigating Privacy Concerns, Trust, and Self-Disclosure,”. Schwarz, N., and Clore, G. L. 1983. “Mood, misattribution, and judgments of wellbeing: Informative and directive functions of affective states.” Journal of personality and social psychology (45:3), p. 513. Toscos, T., Connelly, K., and Rogers, Y. 2013. “Designing for positive health affect: Decoupling negative emotion and health monitoring technologies,” in Pervasive Computing Technologies for Healthcare (Pervasive Health). IEEE. Vaast, E., and Kaganer, E. 2013. “Social media affordances and governance in the workplace: An examination of organizational policies,” Journal of Computer-Mediated Communication (19:1), pp. 78–101. Vines, J., McNaney, R., Clarke, R., Lindsay, S.,McCarthy, J., Howard, S., Romero,M., and Wallace, J. 2013. “Designing for -and with- vulnerable people,” in CHI’13 Extended Abstracts on Human Factors in Computing Systems, ACM. Website, Homelessness Australia 2018. “/ accessed 15/01/2019”. Weise, S., Coulton, P., and Chiasson, M. 2017. “Designing in between Local Government and the Public–Using Institutional Analysis in Interventions on Civic Infrastructures,” Computer Supported Cooperative Work (CSCW) (26:4-6), pp. 927–958. Woelfer, J. P., and Hendry, D. G. 2009. “Stabilizing homeless young people with information and place,” Journal of the American Society for Information Science and Technology (60:11), pp. 2300–2312.


Woelfer, J. P., and Hendry, D. G. 2011. “Designing ubiquitous information systems for a community of homeless young people: precaution and a way forward,” Personal and Ubiquitous Computing (15:6), pp. 565–573. Woelfer, J. P., and Hendry, D. G. 2012. “Homeless Young People on Social Network Sites,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’12, New York, NY, USA: ACM. Woelfer, J. P., Iverson, A., Hendry, D. G., Friedman, B., and Gill, B. T. 2011. “Improving the safety of homeless young people with mobile phones: values, form and function,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM. Wood, S. L., and Moreau, C. P. 2006. “From fear to loathing? How emotion influences the evaluation and early use of innovations,” Journal of Marketing (70:3), pp. 44–5.



Living Lab Activities for Social Problem Solving R&D Projects in Korea: Achievements and Challenges from Case Studies Jieun Seong*1, and Ji In Park2

1 Science

*Corresponding author and Technology Policy Institute (STEPI), South Korea 2 MSIT (Ministry of Science and ICT), South Korea Category: Full Research Topic: Theoretical & Methodological Challenges

Abstract The purpose of this study is to investigate the role of “living lab” activities in social problem- solving R&D projects conducted in Korea and to derive its achievements and tasks. This study analysed the representative cases in which living lab activities are prominent among social problem-solving R&D projects in Korea. The analysed cases are ‘develop portable fundus camera for eye disease screening test to resolve health inequalities’ and ‘auto-sensing integrated system development in rural pedestrian crosswalk’. Through this study, we identify the contents of the living lab activities of these social problem- solving R&D projects and present policy issues. The characteristics derived from the analysis are as follows: 1) living labs are being introduced as a methodology for user and demand-oriented research innovation in Korean R&D projects. 2) these projects conduct new policy experiments that try to overcome the limitations of the Korean innovation system, such as top-down approach in policy making led by central government; R&D planning focused on technology providers’ convenience; and industrial innovation emphasizing economic growth. Keywords: Korean living lab, R&D innovation model, social problem-solving R&D project, R&SD (Research & Solution Development), Case Studies, Achievements and Challenges


1 Introduction “Living labs” have been introduced and implemented in recent years in Korea both as innovation models led by social actors (e.g., local communities and users) and as venues for field-based innovation. Not only the central government, such as the Ministry of Science and ICT (MSIT, renamed from Ministry of Science, ICT and Future Planning (MSIP) in 2017), but also local municipalities and other entities are adopting living lab projects for a variety of efforts including the development of products and services, establishment of public infrastructure, reform of local communities and tackling of local problems. For instance, the “Social Problem-solving R&D Project” led by the MSIT and the “Energy Technology Acceptability Enhancing and Commercialization Promotion Project” led by the Ministry of Trade, Industry and Energy (MOTIE) are efforts to improve field applicability and demand-oriented ness through living lab approaches. Similarly, a variety of IoT living lab projects led by municipalities and public institutes, such as the BukchonHanok Village IoT living lab in Seoul and the Seongnam Senior Experience Center’s senior living lab, are examples of other efforts to apply modern technology, such as IoT, to enhance public services and solve local problems (Seong, Han & Park, 2016). This study aims to analyze samples cases of living labs in social problem-solving R&D projects that are implemented in such a way as to promote innovation in national R&D projects. To this end, we looked at case studies involving the development of “fundus camera technology” and “automated pedestrian detection system” (in particular, the general overview, objectives, planning and management systems, achievements and challenges of such projects) and were able to draw policy suggestions. As for research methods, in-depth interviews with major participants were conducted along with literature reviews of policy reports, academic papers, newspaper articles, and other sources.

2 Theoretical background 2.1 Innovation Models in Korea & Living Labs Innovation models in Korea thus far have been based on the linear model. Embodied in this model is the perspective that innovation will occur when R&D results from research labs become commercialized. Therefore, in this model, R&D activities themselves constitute the critical source of innovation (Seong & Song, 2007). Traditional ICT policies, largely built upon TDX, CDMA and other cutting-edge technologies, are also based on this linear model, wherein users are assigned the role of merely informing tech-providing companies about their needs or serve as passive bystanders who adapt themselves to developed products (Song and Seong, 2013). However, such linear models focusing on R&D have gradually come up against their limits. Structural limitations imposed by an ageing society, fierce competition from emerging economies, coupled with stagnant economic growth and other


factors, have made it difficult to steadily expand human and financial input, thereby reducing the efficacy of traditional input-driven innovation models (Song and Seong, 2013). A living lab is a user-driven innovation arena where users function as actors rather than objects of research innovation and activity, and the significance of a living lab lies in that it is an open innovation model led by end-users who actively engage in problem solving. As such, living lab activities are most prevalent in areas related to everyday living, such as energy, housing, transportation, education, and health care, where user experience and intuition matter the most. Recently, Korea has seen a wide variety of actors (e.g., the central government, municipalities, and social economy organizations) considering living labs as an alternative model for technological and social innovation. MSIT, the Ministry of Health and Welfare (MOHW), and other government agencies are examining living labs as a novel way to induce social problem-solving innovations that utilize ICT and other technologies. On the other hand, the municipal governments of Seoul, Daejeon and other cities are beginning to see living labs as new social innovation models distinguished from conventional approaches. Finally, public welfare institutions (e.g., nursing homes and hospitals) and social economy organizations are paying attention to living labs as experiments for realizing welfare goals and improving welfare service delivery systems. 2,2 Introduction of Living Labs as Implementation Mechanisms for National R&D Projects The MSIT’s social problem-solving R&D project, a national R&D initiative, is a new approach to tackle societal challenges such as the diminishing quality of life and social polarization (MSIP, 2016). Unlike traditional R&D projects that focus on the provision of technology and industrial innovation, this project pursues values such as socially responsible technology as responses to social demands, while integrating technology, law, organizations, culture, and other fields to solve problems facing end- users (Song & Jeong, 2016). However, more than anything, the project fosters open-innovation activities, wherein the introduction of a living lab approach allows end-users and researchers to work together to develop, substantiate and evaluate products in actual living spaces. As such, this project emphasizes civic participation, and based on that, explores social issues closely related to everyday living. Furthermore, by encouraging participation by social innovation organizations to effectively deliver developed products and services, the project seeks to raise their efficacy.


Table 1. The Characteristics of Korea’s Social Problem-solving R&D Projects AS-IS technology acquisition


Primary objective

Characteristics Planning

Management Characteristics by stage

Growth-based, Human-centric, pursuing better focusing on national quality of life as well as economic development economic development R&D, R&BD → R&SD (Research & Solution Development) Acquire scientific and technological Solve social problems competitiveness · Problem -solving convergence* · Technological *technology + humanities and convergence social science + regulation · Provider -centric R&D · Recipient -centric R&D Research divisioncentric

Research division and policy division cooperation-centric

R&D progressioncentric management (program manager)

Problem-solving and change management (solution consultant) Extent of social problem resolution through the production and delivery of products and services or by systemic transition Exploration of social problems and systemization of service delivery

· Research results such as papers or patents. Evaluation

· Verification of research results, dissemination

Main impetus

TO-BE social problem-solving program

Technological development

The ‘social problem-solving R&D project’ is being implemented across three sectors: living environment, disaster safety, and social gap reduction. The living environment sector focuses on everyday-living problems facing citizens—in particular, health and environmental issues. The disaster safety sector, on the other hand, deals mainly with disasters (e.g., toxic gas release and explosion accident) that can occur at the regional and national levels and seeks to develop products and services that can tackle these problems. Finally, the social gap reduction sector centers on socioeconomic gaps that are apparent in everyday living, seeking to develop low-priced quality products and services to help protect the underprivileged. Those projects introduced living-lab methodology as a new way of R&D and tried to promote open innovation activities in which end users and researchers jointly develop, demonstrate and evaluate products in real life spaces. The Living Lab is an infrastructure that enables professionals and end users to continuously improve their products, services, and demonstrations with customer interactions.


Table 2. List of Korea’s Social Problem-solving R&D Projects

Source: Seong et al. (2016). Revised.

The purpose of this study is to investigate the living lab activities in the social problem-solving R&D projects conducted in Korea and to derive its achievements and tasks.

Figure 1. Living Lab as methodology of ‘Social problem-solving R&D Project / Source: MSIP (2016)


3 Case Studies on Living Labs in the ‘Social Problem-solving R&D Project’ 3.1 Case Study on the R&D of a Portable Fundus Camera 3.1.1 R&D goals This project aims to develop and distribute portable fundus cameras that can be used for early diagnosis of ocular fundus lesions occurring in people without proper medical care. According to The Korean Retina Society (KRC), the number of patients with age-related major retinal diseases categorized into four types rose from 382,247 in 2009 to 510,413 in 2013, a 35.1% increase over five years(Health Chosun, 2012). The primary cause of blindness among these patients is diabetic retinopathy, which can be prevented by early diagnosis (Seong et al., 2016: 196). In particular, medically vulnerable demographic groups (e.g., seniors, those with mobility problems, and residents of remote areas) are in ever greater need of visiting/portable fundus examinations with retinography. As such, the research team is focusing on developing a portable design and intuitive GUI software to achieve convenience. To realize this goal, the team is operating a user-driven living lab to apply the outcome of interactions among a variety of actors in subsequent product development. In operating the living lab for this fundus camera project, a wide variety of stakeholders participated, including technology developers; experts in living lab management, validation and authorization; and organizations related to commercialization and distribution. The living lab management team and these participants together operated the living lab throughout the exploration and experimentation phases. The first living lab came up with a product that reflected the preferred functionalities of users surveyed. The second living lab, on the other hand, mainly aimed to evaluate the performance and usability of prototype cameras. These prototype models were constantly modified based on the pros and cons discovered and evaluated through use in real-life settings. 3.1.2 Building a system of Living Lab Infrastructure design As part of preparations before the launch of the living lab, the team engaged in various activities including the handling of legal procedures related to clinical trials and the development of guidelines for participants. First, the project had to pass IRB review, and the authorization for the manufacturing of medical equipment also had to be obtained. MOUs were signed with medical cooperatives, clinics, and university hospitals for the operation of the living lab. Next, product manuals and questionnaires were issued. The manual was offered in three different versions: an equipment manual, a usage manual, and a quick manual for nonexperts containing explanations on how to make a simple diagnosis. At the same time, the questionnaires were produced for two phases: specification design and evaluation during real-life use. Questions related to the former included: (a) What are the usage requirements? (b) What type of assistance is needed? (c) How should the living lab be structured? As for the second phase, a


three-day survey focusing on user convenience was conducted to evaluate the proficiency obtained by repeated use of the product. Survey results were also uploaded to a website for real-time feedback. End –user organization The living lab proceeded in two phases. As each phase had different characteristics, different user groups were formed for each. The first living lab was related to the design and exploration of product concepts, so opinions from core users were collected to determine specifications. These core users were a group of some 200 ophthalmologists specializing in retinal and vitreous diseases and therefore were familiar with fundus photography as well as constituting the real force behind the market for fundus cameras. By asking them “What functionalities need to be added or eliminated?” in specific, detailed questions, the team was able to determine development direction and detailed specifications. Not only was this survey useful in exploring the needs of real users before product development, but it also provided the extra benefit of promoting the product to be developed. The participants of the second set of living labs consisted of an expert group and a non-expert group. The reason behind such classification was to differentiate the way these living labs were operated to reflect the participants’ knowledge of fundus cameras and their skill levels. The expert group was subdivided into small eye clinics and large general hospitals. The non-expert group, on the other hand, consisted of medical cooperatives. Although the members of this latter group did not have prior experience in fundus photography, they had a deep understanding of the need for early diagnosis of eye diseases and telemedicine, thereby serving as a well-organized civic group that is best aware of the purpose of product development and that actively participates in the process. The non-expert group played the dual role of end-users and a service delivery system for endbeneficiaries. All of these groups shared the fact that they were all potential buyers as endusers. However, depending on their expertise and familiarity with related products, each provided its own unique input on the product. The general hospital group often had their own fundus photography equipment, so their experience allowed them to compare the research team’s prototype with other existing equipment on a practical level. The eye clinic group, on the other hand, consisted of private and small-sized clinics and was regarded as comprising the majority of potential buyers. As such, the living lab for this group acted as a market tester. The group was also considered an appropriate test bed for evaluating the usability of a portable fundus camera as the members of the group came into contact with a large number of patients. The medical cooperative group, on the other hand, were composed of the uninitiated who were able to provide fresh perspectives and assist in adding user-friendliness to the product by participating in the development process and the drafting of the manual. Table 3. Organization of actors of fundus camera Living Lab



Actor’s characteristics Potential buyers

General Hospitals

Expertise (fundus camerarelated skills) Ample experience in fundus photography and analysis

Actor’s Roles in the Living Lab Evaluate photo quality Share prior experience in fundus photography Evaluate usability

Possess portable fundus cameras / ample experience Expert group

Potential buyers

Ample experience in fundus photography

Most suitable group for evaluating Highly skilled in fundus photography Eye Clinics marketability as they deal with numerous patients No prior experience Potential buyers Nonexpert group

Medical Co-ops

Best aware of the necessity of early diagnosis and telemedicine

Share prior experience in fundus photography Evaluate marketability and usability

Provide perspectives of the uninitiated Offer advice on the product manual

Source: Seong et al. (2016); Seong & Han & Jeong (2018)

3.1.2 Exploration: co-design This phase involved an online and offline survey conducted during December 2015 involving 83 ophthalmologists with ample experience in handling fundus cameras. 95.3% of the respondents said they were currently using fundus cameras, while only 2.3% said they were using portable ones, indicating significantly lower usage rates of the latter. As for reasons why they did not use a portable version, the respondents cited high prices (30.1%), low picture quality (29.2%) and difficulty of use. When asked a specific question, namely “What functionalities need to be added or removed?” they chose “high resolution,” “lightness” and “photography of the eye in a non-mydriatic state.” Based on these responses, the team abandoned its original plan of developing a budget-priced device using existing camera modules, and instead opted for developing its own unique product (Table 1). Reflecting the feedback from these experts, the team adjusted its target price range and developed a prototype that features a 1-inch CCD and a dual-light source that combined IR LED and white LED modules. 3.1.4 Experimentation: co-production This phase was conducted to evaluate the performance of the prototype, with emphasis on image quality, usability, device learning curve, and differentiation from existing cameras. The evaluation was carried out over one month during September 2016 on the three end-user groups and 301 subjects. As for image quality, a group of experts assessed videos taken by living lab participants. As


for usability, doctors, nurses, optometrists, and other professionals assessed difficulty of use. Finally, as for the learning curve, in order to determine whether users can easily learn how to manipulate the camera with basic training, medical doctors not specializing in ophthalmology were repeatedly tested to evaluate their camera use ability. The experiments provided feedback on a variety of aspects including the level of difficulty in using the camera, accessibility of the software, appropriateness of the weight, comparison with existing devices, and the addition of an LED display. The level of difficulty felt when using the camera varied greatly depending on the user group; those with experience in using a portable fundus camera mostly felt that the prototype was similar to existing cameras in usability, whereas those with no prior experience found it difficult to use. On the other hand, those who are not eye experts had to undergo much trial and error during early experiments as they found it challenging to recognize a target point. After three days of usage, their manipulation of the camera gradually improved, but these subjects still found the camera more challenging to use than fixed-type cameras. As a solution, the team is currently developing its own original display and guidance software that provides a fixation point. Moreover, a variety of design choices were made to enhance usability (e.g., a safety strap to prevent falls). The feedback also indicated a need for a separate device manual for non- skilled users, a training program, and the development of indicators to enhance image quality. In particular, the participants raised the need for a guidebook containing medical knowledge that would allow non-experts to make a simple diagnosis. 3.1.5 Achievements In this case, the living lab, wherein a variety of actors participated in repeated experimentation, acted as a catalyst in upgrading the product and making it more user-friendly. The living lab was conducted in three phases. First, institutional and systemic designs were devised to facilitate smooth operation of the living lab. Next, the exploration phase driven mostly by expert groups was conducted to determine product concepts and create prototypes. Finally, the team evaluated usability through the living lab by expanding user groups to include small eye clinics and medical cooperatives. In short, problems and basic concepts for products and services were defined through interactions with core users before determining specifics by expanding the scope of end-user groups and collecting feedback. Participation by different occupational groups allowed the team to collect different feedback from groups varying in expertise and familiarity with related products. Such varied participation aided in the development of a product that features a user-friendly design and technology while increasing the attractiveness of the product. Moreover, direct feedback from potential buyers (e.g., eye disease experts, hospital professionals and medical cooperatives well aware of the need for the product) helped to promote it and raised the commercialization possibilities of the R&D results. On the other hand, the learning effect from these various groups led to the creation of a number of alternatives to cope with problems likely to arise during


telemedicine. For instance, a more detailed manual for uninitiated users, a training program, and the need for developing indicators to improve image quality— these were ideas ophthalmologists had not conceived of, and would serve as complements that can encourage a broadening of the user base. In addition, efforts such as developing standard fundus photograph samples and a guidebook for non-experts served as opportunities to create services that would allow regular users to make simple diagnoses of eye diseases. The living lab for the development of portable fundus camera technology is an effort to reduce disparity in healthcare access. By developing a high-quality portable fundus camera at an affordable price, the lab aimed to expand the use of such cameras. In addition, by lowering the technical barriers surrounding fundus photography (which had been the exclusive domain of ophthalmologists) and by making it accessible to ordinary citizens, the living lab transformed fundus photography into an open service. Moreover, by running separate training programs that can promote simultaneous distribution of product and services, the team also strived to reduce gaps in medical care in a tangible way. 3. 2 Case Study on the Development of an Automated Pedestrian Detection System 3.2.1 R&D goals Unlike major cities, many rural areas in Korea have limited traffic safety infrastructure, which puts vulnerable pedestrians (e.g., children, the elderly, and the disabled) at a high risk of accidents at crosswalks. To prevent such risks, the case in question aims to develop an automated pedestrian detection system that can enhance convenience and safety for these vulnerable pedestrians. Technically, the system aims to achieve a 99% video detection rate for pedestrians; it will be connected to existing facilities and apply object recognition and tracking technology in a detection zone. To develop a commercially viable system, the R&D team worked with traffic engineers and a company specializing in video detectors to operate a living lab in areas with limited traffic safety infrastructure. The living lab was executed in three phases. In the first phase, living lab sites and participants were selected and analyzed with VISWALK, a pedestrian simulation software tool, in order to set up a basis for comparison. In the next phase, the Living Lab Experience Community (LEC) consisting of local residents, public servants, facility managers and other experts was established to verify the living lab and collectively proceed with the entire process of planning, examination, experience, application, enhancement, and verification. In the final phase, an integrated auto-detection system was installed and operated to conduct research on its continuous enhancement. The feedback from the LEC and a log analysis were also considered as basic tools for the distribution and upgrade of the system. 3.2.2 Building a system of Living Lab Infrastructure design Regarding the selection of living lab sites, Jeollabuk-do Province was chosen as the optimal region as roads in the province show typical characteristics of rural roads, posing a high risk of pedestrian traffic accidents. Moreover, the provincial


government expressed a strong willingness to support the living lab project and relevant organizations in the province were well coordinated. As for living lab participants, there were two general considerations used as selection criteria. First, a variety of factors were considered including the presence of facilities that can induce pedestrian traffic, opinions of local residents, the way traffic signals are operated, and the possibility of camera-based autodetection. In addition, local residents informed the research team of sites with a history of pedestrian accidents or sites that are accident-prone, allowing the team to include bus stops, roadside residential areas, senior welfare centers, commercial facilities, factories, churches, and schools as its living lab sites. Next, based on this feedback, the team also considered other factors such as the amount of pedestrian traffic, the diversity of vehicular traffic, the regularity of road geometry, and the frequency of traffic accidents, before selecting four areas in Jeonju City, Jeollabuk-do Province as living lab sites. LEC Organization In order to raise the efficacy of the system, an LEC platform was established so that all participants could regularly share and exchange opinions. Researchers, administrative agencies, police stations, research institutes affiliated with universities and administrative agencies, the Korea Road Traffic Authority (which oversees traffic systems), and local residents—all participated actively in the planning, development, and installation process. The LEC was classified into system developers, living lab operators, and local resident representatives, depending on the role they played. The system developers consisted of a group of experts involved in the development and enhancement of the automated pedestrian detection system. The Korea Road Traffic Authority took charge of the operation of traffic signals and system certification. Police stations provided guidance on development direction and precautions based on the current state of the living lab sites. Tech companies involved examined, based on existing pedestrian detection technologies, the overall conditions related to the operation of a network that is tailored to the local context. The living lab operators consisted of seven people: four field employees from administrative agencies and three academics specializing in traffic-related matters. The former provided advice on administrative matters related to the installation and operation of the system and took practical measures, whereas the latter offered opinions on the overall system operation, including system installation, signal operation, and future planning. Resident representatives consisted of a group of 14 volunteers capable of using the Internet. Those interested in solving social matters and recommended by local administrative agencies were included in this group, and other members were selected in such a way as to ensure diversity in gender, age, and occupation. By sharing opinions on the inconveniences and issues related to system installation, these resident representatives helped the research team


handle a range of challenges. They were also responsible for promoting the system to other local residents after installation. Table 4. LEC Organization Actor’s Roles in the Living Lab

Actor’s characteristics

Expert group

Working group

End user group

Secure institutional compatibility through collaboration between system developers and working-level people related to laws and institutions Secure the effectiveness and professionalism of the operation through collaboration with working- level traffic system experts and professors specializing in traffic- related fields Resident representatives from four areas interested in solving social issues

System development

Possess IT literacy

Promotion of the system to local residents

Product certification

Participant System developers Local police officers Korea Road Traffic Authority

Living lab operation

Academic faculties

System risk management

Administrative agencies

Living lab participation and feedback

Local communities

3.2.3 Exploration: co-design As transportation-related living labs are closely related to daily safety, it is difficult to test prototypes immediately in real situations. Thus, as a complementary alternative, pre-simulations were conducted on four sites using the VISWALK simulation program. The analysis showed a reduction of 70% in average traffic delay times in three of the sites as well as the reduction of 40-50% in the average pedestrian crossing time.

Figure 2. VISWALK simulation screen / Source: Seong et al. (2016)


This result confirmed that the introduction of the system would bring positive benefits to both drivers and pedestrians. The details were shared with the LEC by having them posted on the project website, wherein LEC members were able to discuss operation-related issues in real-time.

Figure 3. LEC organization (top: Inhabitants briefing session, Bottom: Expert meeting) / Source: Seong et al. (2016)

The next step involved receiving real-time feedback from experts and residents before making a prototype. Minimum legal requirements exist for pedestrian crossing systems. As such, the team sought cooperation for system installation through meetings with experts while receiving demands and a variety of ideas from the policy agency, which is the final authority, and working-level public servants from municipal governments. In addition, to prepare for any risks that may arise during system installation and operation, the project team collected opinions from resident representatives and tried to apply those ideas immediately. The team also held information sessions to explain the system under development to them and encourage its use; conducted public hearings to help prevent the invasion of privacy related to the installation of cameras; and had opportunities to identify system requests. 3.2.4 Experimentation: co-production System log analysis The controller on the automated detection system allowed detailed records of signal operations before and after the system installation to be compared and analyzed, thereby enabling quantification of the effects of the installation. However, at the present stage, as the stability of the system has not yet been verified, a green (walk) signal had never been assigned; thus, the number of


pedestrians detected were used instead as an indirect alternative to the number of walk signals assigned. The log data for the first living lab was collected from the four sites during September 2016. The analysis of the data indicated that in outer suburbs the number of walk signals assigned were reduced to approximately 10-23% of the normal figure. This means that a pedestrian crossing that assigns walk signals 720 times per day can function with signal levels roughly 10-20% of that number, while providing both pedestrians and passing vehicles with improved safety and time savings. It can thus be concluded that in outer suburbs, the system can dramatically enhance safety and convenience for pedestrians. However, our analysis suggests that in inner cities, where the amount of pedestrian traffic varies greatly between weekdays and weekends, the efficacy of operating the system is relatively lower. For instance, if there are an excessive number of walk signals, that can disrupt the flow of vehicle traffic. Thus, we drew the conclusion that at inner city sites, the options of adjusting signal intervals or flexibly operating the system only during certain hours need to be considered. Feedback of LEC Above all, it was shown that the product should be improved for better pedestrian safety. For example, some demanded audible guidance and electronic message boards that could allow pedestrians to easily recognize whether the automated detection system was in operation. It was also apparent that the wait time should be flexibly controlled according to the type of pedestrian (e.g., the elderly) and the number of pedestrians crossing. There was also demand for enhancing the reliability of the product. To this end, the product should be carefully designed so pedestrians can see clearly whether it is in operation. It was also requested that the camera display the detection range and a signal indicate whether the detection function is activated. Moreover, video needs to be record when the system detects pedestrians in order to analyze user behavior patterns during the living lab. However, this requires further consideration since users may feel uncomfortable about being filmed and since the act of filming pedestrians may also be against the Personal Information Protection Act and privacy laws. Thus, it was necessary to stipulate that "Video images will be deleted within one day unless there is an accident." pursuant to the Personal Information Protection Act. In addition, the living lab management needed to be revised in such a way as to consider the time and place in which users actually use the system in order to accurately identify residents' behavior patterns. There was also a request for the enhancement of the product’s feedback system. For example, a supplementary manual control system was needed for unexpected accidents. Some also suggested that if the product is equipped with real-time recording and partial storage through the application of the camera technology as well as a vehicle control feature, it could even play a role in reducing accidents caused by drivers running red lights. There was also a suggestion for establishing national technical standards for the system. Although there is currently no product based on the technology in question, it will be


necessary to establish technical standards for utilization and diffusion of the system in the future. 3.2.5 Achievements At end of the first year, the results of the living lab were as follows. The living lab research community, LEC, was formed to build a system through which anyone could participate in product development and installation in a variety of ways. In particular, feedback from local resident representatives created an environment in which users could take advantage of the product with enhanced convenience and safety. The installation costs of the automated pedestrian detection systems at the four sites were about 25% of the costs for traditional systems. The new system therefore offered significant potential advantages in commercialization and distribution. Log records confirm that the system has reduced travel time for both pedestrians and vehicles. When it came to site selection, this case tested the system at as many varied crosswalks as possible. The development team adopted an approach that gradually narrowed down crosswalk types to just one that provided the system with the best efficiency, and this approach proved to be very appropriate. Moreover, the living lab sites were recommended and selected by utilizing the experience of the living lab participants. As such, the sites were situated in context by local participants, which was not achievable through a field survey by outside researchers. The case of the automated pedestrian detection system development constitutes an urban transport arena, and serves as an urban transition lab. The constant contact and communication between new technology and local communities develops new meaning and culture within the context of traffic technology by inviting change in the daily behavior of civil society. In the end, it is an effort to establish a technical system that promises to reduce the risk of accidents on the roads in rural areas, and to find fundamental solutions through steady interaction with relevant local actors. This case has been praised as a successful alternative approach to traffic problems and is now spreading to other municipal governments (Seong & Han & Jeong 2018). 3.3 Cases Summary and Learning from the Cases The social problem-solving R&D project led by the MSIP has the goal of solving social problems and has used living labs as an effective management method to enhance the efficacy of technology development. From examining the aforementioned two cases, the following characteristics have become apparent. First, the purpose of a living lab was clearly defined, and focus was put on organizing the end-user community to enable living lab participants to play a central role. In the case of the fundus camera, the product concept and specifications centered on core users were first determined, before completing product development by analyzing their opinions from much richer and more diverse perspectives. As such, this can be viewed as an exemplary case wherein R&D activities helped solve a medical gap issue and secure new markets. On the other hand, in the case of the development of crosswalk technology for rural


areas, there was active participation by residents in tackling problems, and it served as an Urban Transition Lab. At the same time, the LEC formed a new innovation ecosystem and created a common vision among participating members, while striving to solve urban problems through experiments. Second, ways to improve the delivery system and legal/regulatory systems were explored concurrently in order to commercialize the product. In the case of the fundus camera, the team tried to deliver its products and services to those who actually needed them by incorporating social innovation groups as participants. In the case of the crosswalk technology, the team is collaborating with related organizations to build a safety evaluation system. Third, operating living labs for each stage of technology development (from product planning to field testing) created concrete participatory spaces for endusers, maximizing the technology development. This confirms the effectiveness of user-driven open-innovation, while at the same time providing idea- based SMEs with possibilities for new (Seong & Han & Jeong 2018). The following aspects with regard to the planning and operation of living lab can be extrapolated from this case study. First, in planning living lab, the concrete presentation of social issues and user demand to be addressed and organization of end users are important. The main challenge of the living lab project is to concretize abstract social problems to a level at which they can be tackled through R&D projects. Both cases emphasize this process, and in particular, various efforts were made to organize the end user group and encourage their participation.


Table 5. Comparison of cases. Source: Seong & Han & Jeong (2018) Development of portable fundus camera technology Purpose of development Set up institutional infrastructure and select sites

Reduce healthcare disparities

Solve regional traffic problems

Contact potential buyers, etc. and pass an IRB review

Request cooperation from traffic system-related institutions, regarding sites with severe traffic problems

Organize expert groups for product planning Living lab design

Organization of end-user groups

Exploration Living lab

Experimentation Living lab

Living lab outcomes and commercialization strategies

Development of an automated pedestrian detection system

Expand user groups to include potential buyers for product testing in actual settings Establish product specifications Utilize surveys, interviews, etc.

Organize working-level experts for product planning Expand user groups to include local residents for product testing in actual settings

Pre-simulation analysis Share analysis findings with LEC and receive feedback

Produce a minimum viable product (MVP)

Proto type installation and log analysis

Feedback from potential buyers

Apply site-specific product specifications

Enhance service delivery system with participation from social innovation organizations

Spread solutions through the establishment of an LEC

Develop a commercialization model that includes user education services and explore new markets

Spread policies for problemsolving by establishing an urban transport area. Develop a risk-management model and a certification system related to traffic laws

Second, the construction of a cooperative governance between the public, industry, academia, research institutions, and government institutions is necessary. To this end, the comprising parties must consist of civil groups with sufficient representation and public nature, strong-willed governments with a commitment to problem solving, industry members with capacity for innovation and high commitment to social values, and interdisciplinary research and academic institutions that can handle the problem in a holistic manner. The two cases studied differ in the nature of participatory actors’ activities and degree of cooperation, but the effort to create a kind of cooperative body among the actors with the above characteristics is a common factor.


The third major element is the challenge to select the appropriate space or site for living lab. As seen in the case, there are noteworthy efforts to choose an effective site to implement living lab, with consideration to areas with a concentration of transportation and safety issues, areas where the residents are committed to addressing the problems, and areas where there is a history of civil participation and action. Fourth, appropriate methodology for developing and implementing, depending on the problem type and situation being addressed, is essential. In the case of fundus camera development, the project was implemented across two groups, one comprised of optometric professionals and the other of non- professionals; the professional group was further divided into doctors, nurses, and optometric photography technicians. In the case of automatic pedestrian sensor system development, the VISWALK simulation and various other methods were employed to suit each situation. Fifth, the improvement of legal frameworks with relation to planning and operating living lab is important. Since the cases surveyed here dealt with medical equipment and transportation systems that may have serious consequences on health, important issues were raised with regard to the safety and ethics of technology, products, and services. The important challenges of living lab also include IRB certification, improvement of transportation law, etc., as well. Sixth, various communication methodologies and educational programs including dedicated mobile applications, quick display, manuals and guidebooks, etc. for the continued interaction between the professional developer groups and the end user groups in the process of developing, testing, and assessing technology were developed.

Figure 4. Learning Elements in the Cases’ Living Lab Activities


4 Policy Challenges from the Cases Korea has pursued science and technology innovation activities focusing on economic growth and industrial development. For the rapid growth, Korea has taken a strategy to develop capable subjects and areas that can grow fast first. In recent years, inclusive innovation has been emphasized in Korea to reduce social disparities and strengthen social integration. To this end, that the paradigm shift from R&D-oriented policy to consumer-oriented problem-solving innovation policy is emphasized. There is an attempt to integrate the subjects, fields, and areas previously excluded in the process of science and technology innovation. The emergence of new R&D categories in 2010, such as 'Social Problem- Solving R&D Project', reflects the situation of Korea nowadays. In particular, these projects introduced living-lab method as a propulsion system and tried to promote open innovation activities in which end users and researchers jointly develop, demonstrate and evaluate products in real life space. The living lab is an infrastructure that enables professionals and end users to continuously improve their products, services, and demonstrations with customer interactions.

Figure 5. Significance of Living Labs Activities for Social Problem-Solving R&D Projects

First, living labs are a new concept in Korea and are currently in their early stages. Since various experiments are being conducted, it is necessary to examine and analyze them in depth through a systematic framework. It is also important to monitor and evaluate on-going living lab projects, share experiences and achievements, and explore models suitable for Korean society. Second, the introduction and spread of living labs requires strategic niche management. If a new system undergoes testing through various living lab prototypes and is successful, it should expand to become a bigger experiment.


To this end, it is important to form a common vision among interested parties through living lab activities and to draw a consensus on the transition. Third, living labs need to be used as an important concept in changing the paradigm of innovation policy from a technology-based approach to a user- and demand-based. Only by shifting the focus of the innovation activities and policies from acquisition of technologies to consumer needs and problem- solving can living labs become a platform that links the creativity of ordinary citizens with the official innovation process. It is imperative to bring innovation to the entire range of existing R&D systems, such as planning, budgeting, evaluation, as well as infrastructure and ecosystem development, through the concept of living labs. Finally, flexible and adjustable policy management is vital for the diffusion and utilization of products and services developed through the production of ultradisciplinary knowledge in the process of operating living labs. The whole system needs to be critically evaluated to create an environment in which policy errors can be corrected based on long-term socio-technical systemic perspectives (Seong & Han & Jeong 2018).

References Health Chosun (2012), 1 out of 4 patients with retinal blindness had 'diabetic retinopathy', html National Science and Technology Commission (2012), “New Science and Technology Program Promotion Strategy”. MSIP ((2016), Guidelines for Solving Social Problem-Solving R&D. Schuurman, D., Marez, L.D., & Ballon, P. (2016), “The Impact of Living Lab Methodology on Open Innovation Contributions and Outcomes”, Technology Innovation management Review, 6(1): 7- 16. Seong, J. & Song, W. (2007). Theory and Application of Total Innovation Policy, Journal of Technology Innovation, 10(3): 555-579. Seong, J. & Song, W & Park, I (2014), “Living Lab as User-Driven Innovation Model: Case Analysis and Applicability”, Korea Technology Innovation Society, 17(2): 309-333. Seong, J. et al. (2016). Selected as a specialized agency for the construction and operation of 'Citizen Research Mentor Team' for social problem-solving technology development project. Ministry of Science and ICT. Seong, J. & Han, G. & Park, I, (2016), Status and Tasks of Living Lab Activities in Korea. STEPI Insight, 184, STEPI. Seong, J. & Han, G. & Jeong, S, (2018), Analysis of Living Lab Cases in R&D Initiatives for Solving Societal Problems and Challenges. Journal of Science & Technology Studies, 18(1): 177-217.


Song, W., and Seong, J. (2013), Science and Technology Innovation Policy to Solve Social Issues, Paju: Hanwool Publishing Co. (Text in Korean). Song, W. & Jeong, S. (2016). Status and Tasks of the Research and Development Project for Solving Social Problems. STEPI Insight, 185, STEPI.



Living Labs and Circular Economy: the case of Turin

Federico Cuomo1, Nadia Lambiase2 and Antonio Castagna3 1

University of Turin-City of Turin, Italy 2 University of Turin, Italy 3 Managerial Trainer Category: Research in-progress Topic: Smart Cities & Regions

Abstract This paper aims to present the case of the Torino Living Lab on Sharing and Circular Economy in an attempt to highlight possible future scenarios for policies to stimulate urban innovation in the environmental and social fields. The case study is analysed in three phases. First of all, it is described the approach of the local public administration to the tool of the Living Lab as a stimulus to innovation. In the second part, the Turin Living Lab on Sharing and Circular Economy is deepened and potentialities and weaknesses are highlighted. In the last section we focus on understanding how the selected case can open possible fields of comparison between administrations in order to improve globally by sharing their local experiences. Keywords: Living Lab, Sharing Economy, Circular Economy, Regeneration, Turin


1 Introduction Last Autumn the Municipality of Turin launched the Torino City Lab as an initiative-platform aimed at creating simplified conditions for companies interested in conducting tests of solutions for urban living. With this action, the City officially committed itself to become a promoter of public and private initiatives aimed at improving the urban ecosystem and proposing ideas in different fields of innovation: from IoT (Internet of Things) to collaborative and circular economy activities. Adopting the perspective of a public actor not only as a regulator, but also as an hub of boost to local development, the Torino City Lab as permanent platform in the urban area was created for social, economic and administrative conditions. As reported in the Giorgio Rota Report of 2018, the area of the City of Turin is characterized by a high rate of small and medium-sized bodies operating in the tertiary sector with less than 10 employees. Moreover, concerning capital corporations of the tertiary sector, Turin has an average of 14 employees per company. This aspect distinguishes Turin from the other large cities in the CentreNorth, where the size of the company is usually larger (Centro Einaudi 2018). Even more significant is the low number of new companies registered in the Turin metropolitan area, which in 2018 stood at around 13 thousand units, recording the lowest result in the last decade (Camera di Commercio di Torino 2018). This stagnant entrepreneurial context, increased due to the economic crisis, is more frequently leading small and medium-sized enterprises to seek the support of public players, especially in the taking off phase of their market. This dynamic of local market is being progressively combined with the expressed commitment at national level to develop practical policies to foster and accompany development of start-ups considered innovative, as regulated in the Decree Law 179/2012, known as "Decree Growth 2.0" (Ministero dello Sviluppo Economico 2016). Therefore, the City is planning to become a laboratory in which companies can establish direct contact with the final users of their products. The main purpose of the action consists in covering the weaknesses of local companies in precommercial phase. According to the local actor perspective, this initiative would contribute to regenerating of the local entrepreneurial landscape. At the same time, this mode of partnership with private sector could also attract investments from Italian or international companies to the territory. Starting from this framework, the following contribution presents the case of the Torino Living Lab on Sharing and Circular Economy as a possible development basis for innovative environmental policies on a local and global scales. In order to provide the best possible structure for the research, the contribution is divided into five sections: (I) the description of the new permanent laboratory proposed by the City of Turin; (II) the past experiences of living labs in Turin; (III) the approach and the birth of the Torino Living Lab on Sharing and Circular Economy; (IV) the presentation of the experimentations admitted to the Living Lab; (V) the interpretation of the case study as a basis for future projects on local and international scales.


2 The Torino City Lab: a permanent platform for experimentation To achieve the objectives described in the previous paragraph, the Torino City Lab presents itself as a platform that aims to generate four main outputs in the urban ecosystem. First of all, the Lab ensures the access to public spaces through streamlining the administrative process. The initiative is promoted by adopting a new strategy on the part of the local authority, which is capable of acting by making all its sectors work with an integrated perspective. More specifically, the Innovation Area of the City is committed to working in agreement with the Environment and Green Spaces Area to ensure simplified procedures for experimenters. This cooperative management is born from the desire to quickly coordinate all the local offices and to meet all needs of experimental bodies. Secondly, the Laboratory is addressed to connect local actors operating in economic sectors considered to be innovative. Through experimentation activities, small and medium-sized companies have the opportunity to create partnerships with large public and private multiutilities that manage the services sector in the area. In addition, the experimenting subjects have the opportunity to deal directly with the world of research. Adopting this approach, a start-up might have the chance to collaborate with big local multiutilities as SMAT (management company of hydric sector), IREN (management company of energy sector), AMIAT (management company of waste) but also with the University and the Politecnico of Turin. As a third point, the Torino City Lab aims to make it possible to test products and ideas that might be exported on a larger scale. From this point of view, every project in the Lab are not planned to fill out only local needs, whereas they should be designed to be reused and fitted on wider scales. This designed process is addressed to match the transnational co-creation strategy (Santonen, Creazzo, Griffon, Bòdi, Aversano 2017). Finally, the Torino City Lab is based on the involvement of citizens as final users and aims to adapt the experimentations to the needs expressed by peoples. For this reason, in addition to the permanent chance to propose to the City innovation ideas, the public administration works to open specific calls based on identified challenges to fill out emerging needs of urban areas or European Union directives. This last aspect puts the Laboratory in its own right in the category of Living Lab, providing the urban territory to create a public-private-people partnership, achieving the innovation model of the quadruple helix. As described in recent literature, this pattern shapes the collaboration of four main actors: public authorities, industry, academia and citizens (Varmland County Administrative Board 2018). Following this purpose, the City of Turin decides to plan in detail the Torino City Lab, in order to avoid wasting energy and to set specific objectives.


As shown in the Table 1 below, the City identifies specific mission, vision and values to be pursued in the development of the platform. Table 1. The features of the Torino City Lab. Source: City of Turin4 Mission ●

Facilitate testing operations in real conditions of innovative solutions of public interest. Offer constant support to facilitate access and then facilitate the conduct of trials, in relations with Internal Services and Utilities.

Vision ●

Values Positioning Torino at European and international level as a place where innovation is easier and is a shared challenge for the territory.

Agility in the execution of activities.

Transparency of the process.

Openness of the partnership.

Attracting companies from Europe and the world to engage new trajectories of economic development in sectors with high added value and to serve the citizens of tomorrow.

This platform enables the City to promote new challenges in environmental fields, which are difficult to address with classic regulatory tools, involving a huge variety of public and private actors as well as citizens. Eventually, within the City Laboratory the Municipality decides to promote one of the policy areas considered most important both to offer new opportunities for local development and to match European inputs in environmental policies: The Circular Economy.

3 Turin Municipality and Living Labs As previously described, the Torino City Lab foresees the chance of hosting actors from different innovative fields. However, the City has decided to adopt the tool of living labs, paying particular attention to environmental impacts and the promotion of sustainable development. For this reason, the several steps that have marked the construction of the Torino City Lab have always combined innovation with environmental sustainability to achieve a long-term goal: becoming a Circular Economy Hub. At the end of 2015, for the first time the European Commission designed a Circular Economy Action Plan. On the one hand, it claimed the necessity to change the economic model to face the lack of resources in a sustainable way. On the other, it sets up almost 10 billion to boost the transition towards a new plan of development, financing projects based on redesign, reuse and recycle values (European Commission 2019). Despite this, European research institutes, 4


as the Ellen MacArthur Foundation, have stressed that the Circular Economy cannot be supported neither exclusively through top-down investments of funds, nor merely introducing regulative limits to the industrial processes. From this point of view, the Circular Economy theory has been based on promoting the model of the 3 R (Reduce, Reuse, Recycle) starting from the capability to choose and act of the purchasers and end users of all services: the citizens (Yang, Zhou, Xu 2014). The reference idea started from the assumption that the citizen can represent the engine of change. In this sense, the end user goes to affect not only reducing their consumption or reusing as much as possible finished products, but also on the systems of product design and durability of materials with their choices of purchase. Adopting this perspective, in recent years the City of Turin began to imagine the Living Lab tool as a potential stimulus to the Circular Economy. In this sense, the Public Administration has started to experiment laboratories in several fields directly linked to the Circular Economy topic. In 2016, the City launched the first Living Lab in its history in the Campidoglio district, providing the urban area for an experimentation of technologies and innovative ideas related to the Smart Cities sector. For a year the neighbourhood became the home of 32 experiments that changed the area ecosystem through data sharing technologies and air quality monitoring systems, urban farming and projects against food waste. In 2017, the local Public Administration decided to open another Living Lab spread over several suburbs of the city, focusing on the more specific issue of IoT. As in the previous laboratory, technological innovations were brought into contact with citizens with the clear aim of boosting new companies committed to environmental sustainability and improving the quality of life in urban contexts. Therefore, IoT technologies were selected with reference to specific areas of application related to the environment and the daily lives of citizens: the quality of the urban ecosystem (air and noise monitoring systems); mobility; energy efficiency; security and management of buildings; culture and social inclusion. 3.1 Turin Living Lab on Sharing and Circular Economy Although both the described laboratories had a close link with environmental sustainability, neither of them had been specifically focused on the Circular Economy paradigm. The real chance of implementing a Living Lab on the Circular Economy was opened in the summer of 2017, when the City of Turin received a budget of 18 million euros to implement the AxTo (Actions for the Turin suburbs) programme through a Presidential Decree from the Council of Ministers aimed at fostering urban regeneration. Through this broad programme, the City was committed to implementing 44 specific interventions in the selected suburbs of the City, focusing on five areas of action: Public Space; House; Work and Innovation; School and Culture;


Community and Participation. Within the third pillar, which combined the challenge of stimulating businesses and employment with innovation, the idea of planning a Living Lab on Sharing and Circular Economy was born. Therefore, this specific laboratory was inserted as action 3.02, focused on innovation in the suburbs as mechanism capable of dealing with the crisis of local businesses (Comune di Torino 2018). The reference pattern was based on creating open-air laboratories that give a chance to companies engaged in sectors such as sharing economy, internet of things, digital manufacturing, circular economy, environmental sustainability and food. In addition, innovation was also interpreted from the point of view of recovering the craft heritage of the reference areas to keep alive sectors of the craft industry that are strongly linked with circular economy (shoemakers, carpenters, hardwars). For these reasons, four main goals were identified by the City: boosting the local private sector; stimulating new ideas of business; creating a network of sustainable development composed by entrepreneurs; planning conditions to host in public spaces innovative experimentations. To achieve these goals, the project was planned by the Development and Innovation Area of the City over 18 months, from May 2018 to December 2019, with a maximum time allowed of 9 months for each experimentation. In the spring of 2018, an external “Managing Authority� of technical support to companies and communication with citizens was identified through a public call for tenders. This initiative of the City was addressed to non-profit companies, associations and foundations specialised in development strategies and activities of territorial promotion. In the same period, the City publishes the call for the selection of private experimenters, open for two months (May-July 2018) to companies in partnership with community associations or Universities and research institutions. The contribution made available by the City amounted to 100 thousand euros. Each testing action could receive a grant up to a maximum of 15 thousand euros, equal to 50% of the total eligible investment to cover the costs of experimentation. The identified areas of experimentation of the Laboratory were concentrated in the North and South suburb neighbourhoods. However, proposals that provided actions spread throughout the city area were also allowed.


Figure 1. The map of admissible areas of the Living Lab / Source: City of Turin5

To objectively examine proposals that were could be deeply diversified from each other, the City set up an ad hoc evaluation committee to select the projects. This evaluation body was composed bringing together experts in the different fields of activity. Accordingly, to the committee effort, the evaluation process was based on five criteria considered decisive for access to public funding: (I) Technical Feasibility; (II) Uniformity; (III) Level of Innovation; (IV) Level of Engagement; (V) Economic Sustainability. The type of contract chosen by the City to start the testing phase with the selected subjects was the Partnership Agreement. The latter was considered by the public administration as the most suitable to start the Living Lab because it clarified the conditions and facilitates the administrative procedures for the transfer of grants. 3.2 The methodology From the point of view of small companies, planning projects in the context of the circular and collaborative economy, activating partnerships with other private subjects, cooperatives and universities are not simple actions. These difficulties can represent obstacles to innovation. For this reason, the City of Torino wanted to include a Management Authority that could apply an accompanying methodology. In the design phase of the Living Lab on Sharing and Circular Economy, the experts of the Authority met the proponents with the aim of informing and encouraging new networks. These preliminary meetings were an opportunity to share ideas and encourage their development. This action enhanced the 5


strengths and highlighted the weaknesses of projects in terms of technical feasibility and economic sustainability, urging subjects to improve specific aspects. In the evaluation of projects phase, the team constructed a summary table in which the strengths and weaknesses of presented projects were indicated with reference to each specific assessment criteria. They also guaranteed their presence during the discussion, with the objective of facilitating the work of the evaluation commission. In the system and planning phase, the proponents followed a structured training course in four workshops with the aim of validating the idea; identify monitoring, evaluation and impact indicators; find consistent community engagement methodologies; develop the action plan. Design thinking and systemic design were the basis of the working method. System map and Social business model canvas were the main tools used. The output of the entire activity was a roadmap for each of the eight projects admitted to the trial. In the testing phase the team specialists met the proposers, individually or with the whole project team, on at least four different occasions with the coaching methodology. On the one hand, this initiative was implemented to reaffirm the value of the partnership between public and private subjects. The meetings could be experienced as a form of bureaucratic control. However, the difference between monitoring as a form of support instead of control has not been easy to perceive and it has made necessary to build a relationship of trust with the proponents. On the other, emphasizing weaknesses has represented a value for the project, as well as insisting on the identification of precise indicators of success or failure. Form the point of view of the City, the accompanying methodology carried out by the Authority has given rise to some points for reflection. First of all, coaching activities have allowed to gather information and data useful for evaluating the effectiveness. The collected data have concerned the individual projects, the identification of the critical phases, the prevailing orientations of the proposers, the blind spots of the projects, the ability to involve and engage the population, with the aim of identifying suitable forms of support and accompaniment for future living labs. In addition, information has been collected on the entire experimentation process, the ability to facilitate the activation of territorial networks, the visibility and the communicative impact, with the aim of developing processes capable to promote innovation in the circular and collaborative economy. The accompaniment has also made it possible to provide adjustments during the course of the Living Lab, in order to deal with difficulties that have arisen in the process phase. Furthermore, the in-progress accompaniment has facilitated the activation by the Public Administration, in terms of authorizations and definition of agreements with the involved public departments.


Table 2. The methodology pathway of the Living Lab (Adapted by authors with permission of SocialFare)

The latter aspect has represented a tough step. Social innovation and economic innovation require the presence of an ecosystem capable of accepting and enabling changes, both from the point of view of norms and local regulations and authorization processes. For instance, one of the companies needed to conclude an agreement with the city to experiment with hydroponic production in a city park; another needed to collect plastic with the contribution of citizens, avoiding that it ends up in the waste stream; a third decided to produce and distribute hot meals in the homes of the first night shelter for homeless people also using food from charitable collections from the city markets. In these cases, Authority has activated various services to cooperate within the short time frame of implementation. Another aspect to emphasize has been the support of the Authority for the dissemination of the 3.02 program aimed to insert and position the City of Turin in a European debate on the circular and collaborative economy, identifying opportunities for meetings, workshops, conferences, where to present the case of the experiments conducted on the territory of the City.

4 The experimentations The time frame for developing the eight selected projects is from January to September 2019. To have a better understanding of the characteristics of the Living Lab it is necessary to present the projects admitted to experimentation on the urban territory. Abbasso Impatto (Lower Impact)


Abbasso Impatto is a project conceived and developed by the Verdessenza cooperative, based on the collaborative economy and built on the model of Solidarity Purchase Groups (collective purchasing groups). The objective is to reduce the environmental impacts in the consumption of catering and hospitality establishments and to guarantee sustainable prices for supplies thanks to collective purchasing. The experimentation area identified is the San Salvario district of Turin. Verdessenza is primarily concerned with assessing the needs of catering and hospitality establishments, and simultaneously identifies and selects the groups of products and services to be offered to them. In order to carefully choose the suppliers of the necessary products and services, it draws up the Minimum Environmental Criteria through which to select the suppliers that guarantee a production process more attentive to socio-environmental sustainability. Edilizia Circolare (Circular Building) Edilizia Circolare, a project conceived and developed by the Emmegi company, was born with the aim of applying the concept of reuse and recycling in the construction sector. The first step is the establishment of a team of professionals for reuse, made up of architects, designers, companies and artisans. This will be followed by the identification and collection of potentially reusable materials that will be donated by citizens, businesses and local artisans, to be transformed and come back to life in new furnishings and finishes. Throughout the project, workshops and focus groups will be organized for both professionals and DIY (do it yourself) enthusiasts. The final product of the trial will be the restyling of a room located in Via Montevideo 41, entrusted by the Municipality of Turin to the Paradigma Social Cooperative, which will host a cafĂŠ for members and new laboratories. Suolo sostitutivo (Replacement soil) The project, conceived and developed by Horizon srl, aims at the re-use, in the context of territorial planning, of inert material, following an appropriate treatment, coming from excavations carried out in the city for infrastructural works. Normally, in fact, this material is classified as waste and is stored in landfills. The main objective of the project is the development of a technical protocol for the constitution of a soil capable of replacing the natural one, suitable to sustain a plant substrate over time. Large volumes of inert materials that are difficult to dispose would be transformed into secondary raw materials, in line with the principles of the circular economy and with the current provisions of the European community regarding waste reduction and re-use and recycling of waste materials in order to guarantee the conservation of ecosystems.


The information taken from the experimentation can then be used to redesign or convert parts of the industrial waste generation process and to develop a mixture that can become a marketable product. UrbanAquaFarm UrbanAquaFarm, an experimental project proposed by Carlo Prelli Service, wants to develop and test innovative systems for horticulture. Within the framework of the "Orti Urbani Torino" system, a pilot project is proposed that creates a collaborative system of production and consumption of plant products based on "hydroponic" culture techniques. Specifically, the project will build prototypes and experiment with circular horticulture practices, in the area of the "urban gardens" inserted in the Parco dei Laghetti in the north of the city, inaugurated during the 2018 spring in an area currently undergoing redevelopment. During the implementation of the project, specific dissemination activities will be carried out, as well as additional parallel and collateral initiatives, directly applicable to the entire system of "Urban Gardens" of the city, such as training courses on the "Hydroponic" system, vocational training seminars, visits guided tours for students and groups of citizens. Humana, RicuciTo project The project arises from the fact that 5,000 of the 20,000 tons of used clothes stored each year in the Pregnana Milanese warehouse have any market value. Humana has chosen to build a pilot project capable of dealing with one of the main components of this quota, denim. To do this it has built a partnership with the design course of the Polytechnic of Turin, which has allowed 240 students of the second year of Design to work on the problem, and with the social cooperative "Il Gelso" which has also activated the laboratory of the District House Lorusso and Cotugno of Turin where three women work. Izmade, Beautiful Precious Plastic project Izmade creates design objects and furniture, working wood and metal. Beautiful Preciuos Plastic aims to include the Precious Plastic open source machine in the laboratory, so as to expand the range of materials and objects. The machine consists of a shredder, an extruder, an injector and a press. To carry out the process it is necessary to engage the local population that is called to participate in a dedicated collection of plastic and that will then be able to access the workshops dedicated to specialists and amateurs in the multifunctional centres of Barriera di Milano area. Magma, the Balon's Marketplace project The project plans to create a marketplace dedicated to vintage goods and antiquities marketed in the traditional BalĂ´n market in Turin, which has been held since 1857. The markeplace has two main objectives: 1) to increase the commercial capacity in a traditional sector, opening it to a foreign clientele or resident far from Turin; 2) supporting evolution and transparency in a traditional sector closed to innovation and comparison with wider and more developed markets.


Stranaidea, project CON il cibo 2 The project is the evolution of a previous experimentation with which the Stranaidea Social Cooperative had already begun to distribute hot meals in one of the three-night shelter facilities it manages. The objectives of the project are to guarantee at least one hot evening meal to the guests of the facilities managed directly by the cooperative and to encourage the empowerment of guests by involving them in food collection preparation and distribution. The cooperative is trying to activate a negotiation with the City, the client of the service, also in agreement with other subjects that manage other reception centers that could benefit from the service.

5 Towards a circular economy Hub for the Municipality of Turin Cities are in a critical position with respect to the transition to the circular economy. On the one hand they have a very high impact on the environment for all the activities they carry out. On the other hand, their characteristic of having a high concentration rate of resources, capital, data and talents in a relatively narrow geographical area can be an opportunity. In this perspective, it becomes interesting to think about the city in terms of a peculiar ecosystem of social and economic innovation, able to face the transition from a linear to a circular economic model (Ministry of the Environment for the Protection of the Territory and the Sea, Ministry of Economic Development, 2017). In order to increase awareness of its existence, there are two fundamental ingredients that those who administer a city have to consider: a) intention to invest in and with the city community; b) establish clear and shared governance mechanisms. Within this perspective is moving forward the ongoing experimentation of the Living Lab on Sharing and Circular Economy of the City of Turin, for which the Municipal Administration has chosen to be flanked by a Managing Authority. The objective of this choice is to experiment a methodology and a practice aimed at building the structure of a territorial hub of Circular and Collaborative Economy of Turin and at formulating the definition of a city policy concerning the circular economy issues. Specifically, a territorial hub is meant to be an ecosystem of public, private and civil society subjects that interact with the aim of bringing to value, economic, social and institutional the environment in which they operate. In order to function at its best, this ecosystem has to plan a governance structure capable of effectively putting the various actors involved into relation. Therefore, the action of the Municipal Administration in assuming the management of direction and involvement plays a key-role. As regards the strategic political address, it is desirable to create an inter-council Control Room (environment, innovation, work, social inclusion, culture and education). Concerning the involvement, it is necessary to consider that the boundaries of the Circular Economy, as literature is defining it (Lacy P., Rubqvist J., Lamonica B., 2016), is very broad and extends well beyond the only field of recycling. Rather it includes all the phases of the realization of a good and service, and therefore: the


conception and design, the production, the distribution, the modality of fruition and of transformation. The circular economy business models are different, and they cover several phases of the production cycle: circular chain from the beginning (biobased material and energy, and/or second raw material as productive inputs); recovery and recycling; upcycling; extension of product life; sharing platforms; product as a service. Moreover, could be considered crucial the creation of a subject as a Circular Economy Manager Group, which acts as the executive branch of the Control Room. This body would operate with the mandate to coordinate and bring to system the various initiatives, practices and projects that already are moving in the city in the various sectors of the circular economy. Several are the key actors to be involved in the Hub governance structure. First of all, the different universities in the area: The University of Turin, through the new Doctoral School Innovation for the circular Economy; the Polytechnic of Turin, thanks to the multiple skills related to Systemic Design; the University of Gastronomic Sciences of Pollenzo which carries out research related to food and circular economy. Secondary, strategic players that must be involved in the Hub's governance structure are trade associations such as the Chamber of Commerce, Industrial Union, Confindustria, Confartigianato, CNA, Confcooperative, Legambiente. Thirdly, the various utility companies are key-actors that have to be involved. In this regard, the experience carried out by Maribor is interesting. In June 2018 Maribor became the first city in Slovenia to define a strategy for transition in the circular economy, in close synergy with the Charter of Sustainable Development Goals. This strategy was conceived and developed by Wcycle Maribor - Institute for the circular economy - founded by five utility companies in the city. The objective is to implement a management system for all flows of materials and resources available in the city, capable of generating cross-sectoral cooperation between seven different fields: urban waste; construction and demolition waste; mobility; water; power; territorial planning; collaborative economy. The five companies aim to achieve the highest rate of reuse of material, energy and water, sharing information and activities. Finally, the concept of hub recalls the network paradigm (Buchanan M., 2003). The constituting Hub is to be understood both as a new network of territorial actors and at the same time connection of existing ones. Additionally, the Hub has to be meant as a node of a wider network, connected to supra-local scales of action. With regard to the connection with the supra-local networks, the work of the Circular Economy Manager Group is strategic, which makes it possible to relate what is moving, developing and is learned at the Hub level with the rest of the Italian and international circular economy context. To sum up, adopting the City perspective it will be important to identify one or more significant places in the urban area, able to rise to physical places of networking and exchange, in which companies, citizens, schools and universities can directly interact. Some of these can be identified starting from the same


experiments in progress through the Living Lab on Sharing and Circular Economy. At the same time, the City is thinking of exploiting disused spaces, such as the Remida centre, to build physical clusters for the new Hub. Therefore, the next step will be to find a suitable home for the Hub, making the most of the huge architectural heritage left unused by the crisis of local industry.

6 Conclusions This paper has addressed the practical case study of the Living Lab on Sharing and Circular Economy of the City of Turin. Deepening the Turin policy of using Living Labs as a stimulus for innovation, the case study suggests three main points of discussion. First of all, the Living Lab on Sharing and Circular Economy has shown how there is an emerging network of local businesses, associations and committees of citizens increasingly active in the field of sustainable entrepreneurship. Through the facilities made available by the City and the guidance of the Managing Authority, these bodies have managed to get in direct contact with citizens. This direct approach provided through the Living Lab could be a tool to improve their innovative ideas and modify them to better match the needs of citizens. Secondly, the case study has suggested how the Living Lab methodology allows to build the foundations to turn cities into innovation hubs. In the recent history of the City of Turin, no classic regulatory or business incentive tool has ever managed to bring together the City, research institutions, businesses and citizens on such a key-issue as the Sharing and Circular Economy. This Living Lab will therefore be the starting point for transforming the Torino City Lab into a real Hub of Circular Economy. Finally, the case presented has highlighted how local action is directly linked to global policies. The need to find sustainable business solutions that preserve the environment by working on reuse, material recovery and recycling is shared by most of the world's institutions. However, shared policies on paper often find barriers insurmountable in practice. Nevertheless, living labs could represent practical tools to connect cities and to scale from local to global policies in support of the sharing and circular economy. To sum up, the case of study has clarified that the greatest challenge is to bring together different actors on both a local and global scale to promote real changes in environmental regeneration and citizen services policies. A challenge that can only be faced by cities through dialogue and positive exchanges addressed to the planning of future living labs.

References Buchanan M., (2003), Nexus. La rivoluzionaria teoria delle reti. PerchÊ la natura, la società , l’economia, la comunicazione, funzionano allo stesso modo, Mondatori.


Centro Einaudi, (2018), Servizi: uscire dal Labirinto. Diciannovesimo Rapporto “Giorgio Rota” su Torino. Torino, Centro Einaudi Comune di Torino, (2018), AxTo Azioni per le periferie torinesi. Schede descrittive delle azioni, Torino, Comune di Torino Lacy P., Rubqvist J., Lamonica B., (2016), Circular economy. Dallo spreco al valore, Egea. Santonen, T., Creazzo, L., Griffon, A., Bòdi, Z., Aversano, P., (2017), Cities as Living Labs-Increasing the impacts of investment in the circular economy for sustainable cities, Luxembourg, European Union Yang, Q., A., Zhou, J., Xu, K., (2014), A 3R Implementation Framework to Enable Circular Consumption in Community, International Journal of Environmental Science and Development

Web-Site References Abbasso Impatto European Commission, (2019, March 4), Report from the Commission to the European Parliament, the European Economic and Social Committee and the Committee of the Regions on the implementation of the Circular Economy Action Plan. Retrived from http: // Camera di Commercio di Torino, (2018), Nati-mortalità delle imprese torinesi nel 2018. Retrived from Circular Glasgow Comune di Torino, Centro Riciclaggio Creativo Remida centro_remida/index.shtml Edilizia circolare European Project Urban Wins Ministry of the Environment for the Protection of the Territory and the Sea, Ministry of Economic Development (2017) Towards a circular economy model for Italy. Framework document and strategic positioning. Retrived from Ministero dello Sviluppo Economico, (2016, February 4), Scheda di sintesi della policy a sostegno delle start-up innovative. Retrived from cy_startup_innovative_0 4_02_2016.pdf Politecnico di Torino one SocialFare


Slovenia Times aribor Suolo sostitutivo Tavolo del Riuso UniversitĂ degli Studi di Torino, Scuola di Dottorato Innovation for the circular economy UrbanAquaFarm Varmland County Administrative Board, (2018, June 12), A quadruple Helix Guide for innovations. Retrived from


To get things right for children. Implementation of a public social living lab model for coordinated support for children in need Angelika Thelin*1, Torbjörn Forkby1 and Mats Anderberg2

*Corresponding author of Social Work, Linnaeus University, Sweden 2 Department of Pedagogy and Learning, Linnaeus University, Sweden 1 Department

Category: Full Research Topic: Health and Wellbeing

Abstract There is a large need in Sweden and internationally for the development of knowledge-based approaches to improve children’s well-being, promote learning, school attachment and self-efficacy early in life. This includes both the articulation of comprehensive policy frameworks and the implementation of targeted interventions. One response to this is presented by the Scottish model Getting It Right for Every Child (GIRFEC). Central pillars are to improve children’s well- being and learning through early intervention, universal service provision, and multi-agency coordination. The model has gained substantial interest in Sweden, where the most challenging implementation is taking place in the county of Kronoberg, including eight municipalities and several health service organizations. This research paper is based on material from the ongoing evaluation that aimed to establish an interactive research in support of the implementation process. The paper describes the early process that followed the implementation decision and discuss how it might be understood as a public collaborative social living lab and what this demands from the researchers. Emphasis is put on the researcher’s role to balance between partaking in the innovative work and standing aside and giving critical reflections. Keywords: coordinated interventions, children, well-being, implementation, social living lab, on-going evaluation


1 Introduction Social innovations share the commonality of being social to their both ends and means (Young Foundation 2012). To work with social innovations through interorganisational and multi-actor collaboration has been presented as a way forward to meet financial as well as democratic challenges in the public welfare sector (Nicholls, Simon & Gabriel 2015, Sørensen & Torfing 2015). Social innovation can be applied with the aim to create something entirely new, but also in adaptive processes when policy or interventions are transferred between contexts (Sørensen & Torfing 2015). The social living lab methodology offers a broad framework for how to address societal challenges on local and global level through innovative and co-productive processes between users, professionals, researchers and other stakeholders (Garcia Robles et al. 2016). It suits therefore well with policy intentions to establish trust-based processes for enhanced quality in the public sector. To integrate research in these innovative processes are furthermore in line with the objectives for evidence-based practice in the welfare sector (Greener & Greve 2014, Nicholls, Simon & Gabriel 2015, Sørensen & Torfing 2015). However, research provides limited empirical examples of social innovation processes in the public sector that integrated research (Garcia Robles et al. 2016, Nicholls, Simon & Gabriel 2015, Sørensen & Torfing 2015). This paper discusses how an implementation of a knowledge-based praxis model (GIRFEC) could be facilitated by the use of a public social living lab methodology, and what this implies for the researcher role. Interactive research was connected to the implementation process from the beginning. The intention with the implementation is to bring about early and coordinated interventions for children’s and youth’s wellbeing and learning. The implementation (here labelled BBiK6) takes place in the county of Kronoberg involving eight municipalities and health care services in the south of Sweden. 1.1 The Swedish context Sweden has a universal child welfare model (Robertson 2011), realized as the offering of comprehensive public assistance generally to children and families. For example, public health and dental care are free of charge for all children from birth until 20 years of age. Preventive health care is integrated in all schools and child benefits or grants are allowed all families until the end of upper secondary school. In spite of this, the trust for the public social policy is challenged due to perceived inefficient in public sector, inflexible adaption to meet needs, and lack of cooperation between welfare actors (Bringselius 2018). The relative income poverty among single households with children has increased in the last decades, mental health problems among children and youth has developed in a problematic way, and it is hard for young adults to find housing and employments. In this context, child welfare services are criticized for spending too much time on documentation and assessment, and too little time to give voice and support to children and families in need. It has also been argued for more influence from evidence-based methods in the educational system and in the 6 Acronym from ”Barnets Bästa Gäller! I Kronobergs län” – Get It Right for Every Child! In Kronoberg.


social services (Grefve 2017, The National Agency for Education and the National Board of Health and Welfare 2018, The Swedish Agency for Public Management 2014). Turning the UN Convention of Children’s right into a law also present a challenge for the practical work. 1.2 The praxis model in Scotland Child welfare systems in the Western world can broadly be divided into two categories: universal systems exemplified by Sweden and residual systems found for example in the United Kingdom (Robertson 2011). The former aims to provide support for families in a more comprehensive manner, and at prevention rather than managing the consequences from risks. The latter emphasises the importance of parents’ rights and the integrity of family life. Public assistance is concentrated towards protection of children at more severe risks. Scotland has been described as having a hybrid child welfare approach (Robertson 2011, Stafford et al. 2008), highly concern with the families, parents and individual’s discretion at the same time as stressing assets and resilience of communities for achieving wellbeing among children (Coles et al. 2016, Peterson 2015). An expression of the latter intention is the policy framework Getting It Right for Every Child (GIRFEC). This was worked out for over a decade, and were to be fully implemented in 2016 (Coles et al. 2016). In the short run, the aim of the policy framework was to provide better quality of life for children by focussing on early, universal and, when needed, coordinated efforts for those at risk. In the end, the aim was to reduce inequalities stemming from poverty and narrow life expectancies. Children and youth were to be offered the opportunity to develop, reach their full potential, and become successful, confident, and responsible citizens. More specifically GIRFEC consists of a praxis model with the following key elements: (1) the Named Person or the Named Person Service and Lead Professional, (2) three assessment tools – the SHANARRI well-being indicators, the Resilience Matrix, and the self-assessment tool My World Triangle and (3) the coordination document Single Child’s Plan (Coles et al. 2016). The “named person” is the appointed actor for every child targeted through universal provisions from health and education services. For those children and youth in need of more extensive support, a Lead professional (usually a social worker) could be appointed. The responsible professional is to make sure that needs, risks and resilience are assessed and suitable measures are undertaken if needed. The aim of the assessment tools and the single child’s plan is to facilitate common understanding of notions of well-being in all concerned agencies, based on a holistic, ecological child development theory (Coles et al. 2016, Peterson 2015). The policy framework intends to stop agencies operating independently of one another, only concerned with one or the other aspect of the child’s life. It is stressed that success of the overall work must be searched in terms of children’s and youth’s wellbeing, not in scrutinizing professional processes (Coles et al. 2016). GIRFEC considers children’s’ perspectives and rights, aims to ensure that children and families are listened to and are included in as well as understand


decisions affecting them. Positive outcomes particularly in respect to children’s protection and changes in professional practice and culture have been reported from the implementation of the model. At the same time, different views on professional roles and the balance between intrusions in family life, supporting well-being and protecting children have been noted (Coles et al. 2016).

2 Previous research on implementation and interactive research Implementation research has pointed at several challenges involved in the process of reforming policy, or making a new model/method to work: ● Even in the less complicated processes, focus on the original intentions is often lost (Alexanderson, 2006; Pressman & Wildavsky, 1983). ● It is difficult to overcome conflicting goals, governance forms and views on what can be considered as quality between organisations (Darlington & Feeney, 2008). ● There are often different views on what should be done for individuals among users, different professionals and organisations (Danermark & Kullberg, 1999). ● There are seldom any objective and knowledge-based standpoint available to argue from in resource-demanding, complicated or complex processes (Julian et al., 1995; Rogers 2008). ● Legal and financial obstacles can prevent good ideas and collaboration between organisations (Huxham & Vangen 2005). All these challenges need problem-solving processes aiming to establish longterm trust, confidence in each other's work, and commitment to a common goal. These would often be realised as multi-actor negotiations about overstepping boundaries protected from different stakeholders (Gieryn, 1983; Lamont & Molnár, 2002). It furthermore points to the need of provoding the parties with required knowledge, inspiring their will to participate and giving them adequate conditions for doing so (Lundquist 1987:47, Logan & Graham 1998). Dynamic methods for collaboration between the academic world and community actors, in everything from joint problem formulation, knowledge-generating processes to actual change processes is required to meet many of today’s challenges (Nowotny et al. 2001, Svensson et al. 2002). The so-called fifth generation of ongoing evaluations or interactive research has underlined that implementation must acknowledge also local contexts and individual idiosyncratic interpretations (Sjöberg et al. 2009, Svensson et al. 2007, Weiss 1979). At the same time, the researcher’s role is to regularly “inject” more general scientific feedback into the practice – leading to new questions, analysis and organisational action supporting local development. The researchers are supposed to act as a “critical friend” (Sjöberg et al. 2009), however thereby putting him/herself in front of other questions: ●

How to maintain an independent and critical role when becoming involved in the development processes and acquainted to the participants? ● How to keep focus on long-term results, when the organisations work under the pressure of efficiency?


How to capture the views and wishes of different stakeholders if identified as part of the processes? ● How secure sufficient resources for the rather resource demanding interactive research? Complicated and complex development processes will take (sometimes extensive) resources. Nevertheless, an iterative message from research is to establish partnership, integrating practice and research when striving for increased quality and efficiency in public sector for the best of the citizens’ wellbeing. In addition, these co-productive processes should also include users, citizens, different actors and organisations in the public, private or voluntary sector, as well as hybrids inbetween. However, there is still a lack of knowledge about how to establish these collaborations in different forms of practice and contexts, as well as what different forms of collaborations might bring about – for example in relation to coordinated efforts for children’s and families’ wellbeing and needs.

3 Method and materials The on-going evaluation of or interactive research in BBiK started in December 2018, and included three researchers in social work and pedagogy (the authors of this paper). Additional financial support has been granted from the Kamprad Family Foundation for entrepreneurship, research and charity for three more years, something that makes an expansion of the scope and research group possible. The details of how the interactive research should be formed was held quite open at the beginning. Generally, the role of the researchers was to introduce scientific perspectives and knowledge as well as ”disturb” the implementation by raising questions and problematize the process based on collected material, acting as critical friends (Sjöberg et al. 2009). Since working close to an evolving and changing implementation process, we have to continuously reflect on and reconsider our “temporary” understanding, position and research methods to adapt. This approach serves to make the research flexible enough to contribute to ongoing knowledge and development processes, without creating unnecessary uncertainty or vagueness through jumping between research positions. Changes in the research direction should instead depend on the development and needs arisen during the implementation process, identified through analysis of connections between the collected material and scientific perspectives and knowledge. The findings in this paper is the results of such analysis in the early implementation processes. Following empirical data have been collected and considered for the purpose of writing this paper: ● One focus group interview with the three regional process leaders, lasting 1,5 hours. ● Eight hours reflexive dialogues between researchers and the regional process leaders. ● One group interview, 1,5 hours, with three local facilitators from the biggest municipality.


Observation of two steering committee meeting. Written documentation prior to and memoranda after these meetings. ● Participant observations of one operational group meeting and one “creative workshop” at the regional level. ● Participant observation of one operational group meeting in a smaller municipality. The researchers and authors of this paper have met several times and considered the collected material through a dialogue. The dialogue has been about how the overall implementation process can be understood through a connection between the collected material, scientific perspectives and knowledge; what strengths and challenges can be identified; and what does this imply for the interactive research design. Between the meetings, the researchers have search and shared information about possibly relevant scientific perspectives and knowledge. The section following below share the results of this dialogue. It describes how the researchers understand the early implementation process, but present above all scientific perspectives and knowledge that has been found relevant to consider in relation to the development. The paper then ends in conclusions about what lessons have been learned through this intellectual work.

4 Findings 4.1 The early implementation processes In the County Kronoberg, the first step of the implementation process in practice consisted of setting up a supporting organisation. In 2018, at the time for the decision to implement GIRFEC from the end of the year, a regional steering committee with representatives from the involved stakeholders’ top officials was set up. After this, an operative regional group consisted of first line managers from all municipalities and the health services within the county council were given mandate to work with the implementation process, together with the three regional process leaders. From the beginning of 2019, local operational groups with both first line managers and professionals in each municipality and the local health care sector have been formed. These local groups are to include the first line managers that are part of the regional operational groups. All groups that have been formed are interdisciplinary with representatives from the school, social services and health care. However, only the regional steering committee and one local operational group has so far succeeded to include the police, since they could not prioritize this before emergency matters. One parallel disciplinary group has been formed within the health care sector, due to a felt need to strengthen their internal collaboration first (among psychiatry for children and youth, maternity welfare, childcare centre), and an un-familiarity of prioritizing broader collaborative network outside their own sector. Early in the implementation process, the steering group wanted an examination of the County’s preconditions for implementing the GIRFEC model. However, they soon changed this position and called for more of direct action. Argument such as that action needed to be taken in order to keep practitioners focused,


motivated and encouraged were put forward. The risk that BBiK would enjoy decreased attention over time, be seen as a diffuse and too complex process had to be prevented by immediate action. Some also argued that action was needed on behalf of the children and youth who suffered unnecessarily (partly) because of bad organisation. The demands for quick and targeted change processes came, in the regional process leaders’ interpretation, to be translated into working with the ideological and theoretical foundation of the model. Something that could evolve as a common glue holding the actors together in future work. The chosen way of doing this was to arrange “creative workshop” for the regional operational group of first line managers, followed by discussions in the local operational groups, occasional tests in local practice, followed by feedback back the regional level. The first process started with a processing of well-being indicators of children, expressed in existing assessments tools compared to those in the SHANARRI model. The workshop then turned to creative work inspired by The Logical Framework Approach (SIDA 2006), which ended in an outline of a new common well-being assessment tool for the involved organisations in BBiK. While not far from assessment tools already in use in their organisation, the new outline had a clearer focus on the child as being the subject. This outline was then send to the local level for discussion and tests. The regional facilitators have also encouraged the local operational groups to take into consideration the views of children and families on the assessment tool. One municipally has returned feedback from views of some five years old children, and some others are still discussing how to go on with including children, parents, private and non-profit organisations in the process. A final version of the well-being indicators is to be decided upon by the steering committee and implemented after the summer of 2019. The implementation idea is to go through similar processes also with the rest of the central elements of the GIRFEC praxis model. Parallel interviews and observations at the local level so far indicate that BBiK locally primarily is associated with other efforts than those related to GIRFEC. Still there is an obvious lack of knowledge about GIRFEC. Whereas GIRFEC is a comprehensive coordination tool for investigation, planning and evaluation of any possible individual efforts – leaving room for any kind of intervention depending on needs. The efforts in focus at the local level is so far primarily about predetermined, fixed organized and already existing coordinated actions developed through the last five years. 4.2 Challenges for the interactive research design As a response to the steering group early wishes of an examination of the County’s preconditions for implementing the GIRFEC model, the interactive research found inspiration from The Ottawa Model of Research Use (OMRU) (Logan & Graham 1998). The model’s aim is to promote research use in health care practices and provide six themes for the researcher–practitioner interface: 1) organisational terms/the practical context, 2) those who shall receive and


implement the method, and 3) the new method/model in itself. These themes are used to clarify the objectives of the development process and identifying, preventing and managing obstacles and difficulties expected in the implementation process. Dialogues and knowledge-processes are facilitated in which researchers and practitioners target the 4) implementation strategies when the method/model is to be put into practice, 5) the actual implementation, and follow-up data 6) the use of the evidence, and health-related and other results from the process. However, the changed position of the steering group and the decision to implement the GIRFEC praxis model immediately forced the interactive research to jump straight into an implementation process without prior assessment. This meant that the implementation stages explained by the OMRU methodology had to be sidestepped, even though they could have a profound importance for whether or not to succeed. Even though the Ottowa model is dynamic, it still follows a systematic logic starting with the investigation, followed by planning and then action. Barriers and supporting factors are identified in a time where they still can be addressed, providing direction for adjusting chosen strategies. This way of working therefore presupposes a predetermined work process that the actors involved follow. The process leaders’ draw to move back in the implementation circle can be interpreted as a kind of soft resistance that forced the process into a slower pace, and in a way trying to bring about an imitation of the original development of the GIFREC model. The workshops as a strategic implementation tools, were set out to organise multidisciplinary and multisectoral creative work, aiming at a common language, understanding and trust between all involved actors. At the same time, this introduce an ambiguity of whether it is GIFREC that is implemented in its original form or whether the process now rather is aiming at change through imitation of innovative solutions from elsewhere through a process of collaborative adoption and adaptation. The later – especially since the process also have interactive research connected to it – could then be understood in terms of a public collaborative social living lab. 4.3 Implementation through a public social living lab Living labs integrate innovative processes and research, have a user-centred perspective as their starting point, and use local experiences and real contexts as their basis (Garcia Robles et al. 2016). They can be organised in many different ways and involve various methods, while characterised by an overall co-creative approach. Input from research is deemed to be useful, relevant and applicable in practice. While clearly influenced philosophically from pragmatism, the need of incorporating rigour research should not be downplayed (Mulgan 2015). The application of social living labs usually emphasises power relations and evolves as creative organic/dynamic processes (Hughes, Wolf & Foth 2017, Scholl & Kemp 2016), as parts of a neverending history of development (Sørensen & Torfing 2015). The products from social innovations consist of ideas addressing social needs, which are translated into practice, implemented and


result in changes (Nicholls, Simon & Gabriel 2015). They can be portrayed as: “a conflict-ridden attempt to find joint solutions to shared problems through provisional and disputed agreements” (Sørensen & Torfing 2015:155). The goal is to develop innovations with high degree of acceptance and credibility among involved parties (Scholl & Kemp 2016), at the same time as previous research has shown that innovations are likely “to create value for some and destroy it for others” (Nicholls, Simon & Gabriel 2015:5). The expansion of a public social innovation agenda has been understood as an advancement of New Public Management (NPM). Previous trust in NPM, individual entrepreneurial capacity or designated development units might have enhanced efficacy to some point but has also increased fragmentation, distrust and auditing and resulted in administrative overload. Innovations have furthermore seldom been shown to be the result of the efforts of a single actor, but of the meeting between different actors, organisations and collaboration (Sørensen & Torfing 2015). Attention has begun to turn to New Public Governance (NPG) and towards innovation processes through multi-actor, and inter-organisational collaboration in networks and through partnership. The underlying ideas are based on collaboration around knowledge, ideas, resources and praxis and through this enhance trust, stimulate mutual learning and innovation processes, not on the present salience given to competition. The facilitation from an adaptive, pragmatic, distributive, horizontal and integrative leadership with focus on metagovernance has been pointed out as helpful in such processes (Sørensen & Torfing 2015). One strength with understanding the interactive research in focus of this paper in terms of a public collaborative social living lab is that it puts action in the centre of the process. Starting in action might be a more successful approach for upholding motivation, focus and encouragement in necessary developments for increasing citizen’s wellbeing under rather challenging circumstances. Through the social living labs methodology, public, private and civil functionaries can possibly attain executive power, take majority decision on strategies, test proposed solutions, and sort out those working. The inclusion of different experiences, forms of knowledge give a creative tension potentially useful for successful innovative work, when giving structure and process facilitation. The deliberative solution finding process can result in jointly held strategies that gain high degree of acceptance and credibility broadly in the communities.

5 Discussion of the researchers’ position The development described above has placed the interactive or on-going research at a crossroad. Should it take a critical stance to the change in direction, pointing out the risks of losing the initial praxis models knowledge base and creating even more resource demanding, complicated as well as complex processes then was initially planned for? Should the researchers continue to contribute to the process by trying to make the practitioners slowing down the process, and stay at implementing the original praxis model? This would make


the process more rational through adapting a predetermined work process that all actors follow. A more manageable implementation could then be carried out through pilots. An alternative is however to take a more critical stance towards the initial planed implementations process and interactive research, understanding it as not being dynamic, integrated, collaborative as well as efficient enough to meet the complex and changing challenges in society and practice. In order to enhance success in implementation processes in current welfare systems in Europe, it can be argued that there is a need to build on historical learning in the new context. When a new way of working is to be established, there will necessarily be processes both of a) understanding the model and the implementation soil, b) initiate common knowledge-processes both aiming to learn the new and delearn malfunctioning routines, c) construct the new and de-construct various hinders. These implementation processes will by necessity involve social innovation processes and be traceable back to several knowledge sources and historical development processes. In the welfare sector, such a process is furthermore to be done under a situation where citizens as well as politicians, managers and practitioners might be of the opinion that there is no time and not enough resources to carry out a slower and firmly established development process, which starts in examinations and thinking, before action and finely hopefully positive outcome. But if the implementation is understood as a public collaborative social living lab – what position do we as researchers see as most fruitful within it? Should we continue by following the process less co-creative, being prepared to both inspire and disturb the development with external scientific perspectives and knowledge as well as describing and analysing failures and successes? This approach would make us researchers to maintain an independent and critical role, even when we are involved in the development processes and closely acquainted to the participants. It could help in keeping focus on long-term results, when organisations work under the pressure of efficiency. The researcher would not put all their energy in practitioners’ current interests or in increasing the credibility giving for politicians. The sight would not be narrow down focussing on low hanging apples in a strive to enhance efficiency and positive short-term outcome to the social problems highest on the political agenda of today.

6 Conclusions The integration of social living lab methodology and ongoing evaluation could potentially embrace a pragmatic methodological in research, feeding the process with knowledge and research activities that involved parties find to be useful, relevant and applicable for enhancing quality and efficiency in the public sector. The question then turns into how researchers can be part of public collaborative social living labs in a way so that they become as efficient as possible, at the same time as they secure multi-actor and inter-organisational collaboration on an innovative as well as knowledge-based foundation.


Rather than focusing on describing and analysing problems in the process, to examine the processes of finding and experimentation of solutions within contextual limits and challenges becomes central. Such examination includes considering the value of feeding and disturbing the development process with scientific perspectives and knowledge. The researcher must always be aware of his/her role of contributing from a relative alien position. It furthermore becomes important to make sure that research capture different stakeholders’ wishes and views. The critical scientific approach will still have to include the articulation of contradictions, differences in interest, conflicts and ethical dilemmas when needed. There is also a need for posing question about what the “new� in the creative processes running; asking for verification for unjustified claims; investigate the theoretical base for different innovations; and also to strictly evaluate its short run outcome and search for long terms results. However, there seems to be no good reason for letting some actors focus on problems and others on solutions. That is one important reason why to look for this integrated model between on-going evaluation and the social living lab. Wellbeing of children, or anyone for that sake, would be helped by deep collaboration holistic views and of joint and never-ending problem formulations, articulation of objectives and knowledge-generating processes. The early implementation process in Sweden has made the connected interactive research change from a more rational implementation model to focus on how to incorporate collaborative social living lab theory and methodology. The next question to consider will be if, and in that case how, this changed understanding of the overall development process also calls for changed methods planned for or already put into action through the early interactive research design.

Appendix 1: The research questions in the qualitative interviews 1) Describe how BBiK has evolved? What key events have led to here you are today? 2) What are the strengths and weaknesses with the current content of BBiK? 3) How does BBiK related to Scotland's praxis model in GIRFEC? 4) What difficulties and enabling factors can you see in the history of developing BBiK? 5) What strategies do you have to manage obstacles that expect for the future? 6) How do the local and regional development process relate to each other in BBiK? 7) What thoughts do you have about how you want to cooperate with the academy in the continuing process?


References Alexanderson, K. (2006). Om evidens i socialt arbete: möjligheter och begränsningar för ett evidensbaserat arbetssätt inom socialtjänsten. [About evidence in social work: opportunities and limitations for an evidence-based approach in the social services]. Falun: Dalarnas forskningsråd. Bringselius, L. (ed.) (2018). Att styra och leda med tillit. Forskning och praktik. [To manage and lead through trust. Research and practice]. Stockholm: Tillitsdelegationen. Coles, E., Cheyne, H., Rankin, J., & Daniel, B. (2016). Getting It Right for Every Child: A National Policy Framework to Promote Children's Well‐being in Scotland, United Kingdom. Milbank Quarterly, 94(2), 334-365. Danermark, B. & Kullberg, C. (1999). Samverkan. Välfärdsstatens nya arbetsform. [Cooperation. The new working method in Welfare states]. Lund: Studentlitteratur. Darlington, Y. & Feeney, J. A. (2008). Collaboration between mental health and child protection services: Professionals' perceptions of best practice. In Children and Youth Services Review, 30(2), 187-198. Garcia Robles, A., Hirvikosky, T., Schuurman, D. & Stokes, L. (2016). Introducing ENoLL and its living lab community, European Network of Living Labs Brussels Gieryn, T. F. (1983). Boundary-work and the demarcation of science from nonscience: Strains and interests in professional ideologies of scientists. In American Sociological Review, 48(6), 781-795. Greener, I. & Greve, B. (2014) Evidence and evaluation in Social Policy. Broadening Perspective in Social Policy. Wiley-Blackwell Grefve, C. (2017). Barnets och ungdomens reform. Förslag för en hållbar framtid. Slutrapport från den nationella samordnaren för den sociala barn- och ungdomsvården. [The reform for children and youth. Suggestions for a sustainable future. Final report from the National Coordinator for Social Child and Youth Care]. Government Offices, Ministry of Social Affairs Hughes, H., Wolf, R., & Foth, M. (2017). Informed digital learning through social living labs as participatory methodology: The case of Food Rescue Townsville. In Information and Learning Science, 118(9/10), 518-534. Huxham, C. & Vangen, S. (2005). Managing to collaborate: the theory and practice of collaborative advantage. London/New York, NY: Routledge. Julian, D.A., Jones, A. & Deyo, D. (1995). Open Systems Evaluation and the Logic Model: Program Planning and Evaluation tools. In Evaluation and Program Planning, 18(4), 333-341. Lamont, M. & Molnár, V. (2002). The study of boundaries in the social sciences. In Annual review of sociology, 28(1), 167-195.


Logan, J. & Graham, I. D. (1998) Model of Health Care Research Use Model of Health Care Research Use. In Science Communication, 20(2), 227-246 Lundquist, L. (1987). Implementation Steering. An Actor-Structure Approach. Lund: Studentlitteratur Mulgan, G. (2015) Foreword: The Study of Social Innovation – Theory, Practice and Progress. In Nicholls, A., Simon, J. & Gabriel, M. (eds.) New Frontiers in Social Innovation Research. Palgrave Macmillan Nicholls, A., Simon, J. & Gabriel, M (2015) Introduction: Dimensions of Social Innovation. In Nicholls, A., Simon, J. & Gabriel, M. (eds.) New Frontiers in Social Innovation Research. Palgrave Macmillan Nowotny, H., Scott, P. & Gibbons, M. (2001). Re-thinking science: knowledge and the public in an age of uncertainty. Cambridge: Polity Press. Peterson (2015). Personalizing Education at Scale: Learning from International System Strategies. Cambridge, Massachusetts: Harvard Graduate School of Education. Pressman, J. & Wildavsky, A. (1983). Implementation. Berkley: University of California. Robertson A. (2011) The Fork in the Road: A Scottish Perspective on Relationship Building as a Catalyst toward Prevention in Child Welfare [dissertation]. Urbana- Champaign: University of Illinois. Rogers, P. J. (2008). Using programme theory to evaluate complicated and complex aspects of interventions. In Evaluation, 14(1), 29-48. Scholl, C. & Kemp, R. (2016). City Labs as Vehicles for Innovation in Urban Planning Processe. In Urban Planning, 1(4), 89-102 Sjöberg, K., Brulin, G. & Svensson, L. (2009) Lärande utvärdering – följeforskning: En syntes. [On-going evaluation – interactive research: A synthesis] In Sjöberg, K. (ed.) Lärande utvärdering genom följeforskning. [On-going evaluation through interactive research]. Lund: Studentlitteratur Stafford, A., Vincent, S. & Morgan-Klein, N. (2008) Child protection: an overview. In Stafford, A. & Vincent, S., (eds). Safeguarding and Protecting Children and Young People. Edinburgh, United Kingdom: Dunedin Academic Press SIDA (2006) Logical Framework Approach – with an appreciative approach. [Sida is a government agency working on behalf of the Swedish parliament and government, with the mission to reduce poverty in the world] Svensson, L., Ellström, P.-E., & Brulin, G. (2007). Introduction – on interactive research. In International Journal of Action Research, 3(3), 233-249 Sørensen, E. & Torfing, J. (2015) Enhancing Public Innovation through Collaboration, Leadership and New Public Governance. In Nicholls, A., Simon, J. & Gabriel, M. (eds.) New Frontiers in Social Innovation Research. Palgrave Macmillan The National Agency for Education and the National Board of Health and Welfare (2018). Uppdrag att genomföra ett utvecklingsarbete för tidiga och


samordnade insatser för barn och unga. Delredovisning. [Assignment to develop early and coordinated efforts for children and youth. Sub-report] The Swedish Agency for Public Management (2014). Evidensbaserad praktik inom socialtjänsten. Utvärdering av överenskommelsen mellan regeringen och SKL. Slutrapport. [Evidence-based practice in the social services. Evaluation of the agreement between the Government and the Swedish Association of Local Authorities and Regions. Final report] Weiss, C. (1979) The many meanings of research utilization. In Public Administration Review 3, 426-31 Young Foundation (2012) Social innovation Overview – Part 1: Defining Social Innovation. A deliverable of the project: “The theoretical, empirical and policy foundations for building social innovation in Europe” (TEPSIE). Brussels: European Commission, DG Research.


Health and Wellbeing 113


Co-creating innovative tools with and for people with Intellectual Disabilities: The case of DS Leisure e-Training Platform Maria Metaxa1, Foteini Dolianiti1, Ioanna Dratsiou1, Evangelia Romanopoulou1, Dimitris Spachos1, Theodore Savvidis1, Vasiliki Zilidou1, Maria Karagianni1 and Panagiotis Bamidis1 1 Lab

of Medical Physics, School of Medicine, Faculty of Health Sciences, Aristotle University of Thessaloniki, Greece Category: Research in-progress

Abstract Participation is a key factor and a central concept when considering interventions for supporting people with Intellectual Disabilities (ID). Building upon the potentialities of the Long Lasting Memories Care service -an integrated ICT platform for cognitive and physical training- for enhancing the daily performance of people with Down Syndrome (DS), and taking into consideration the importance of participatory design for removing the barriers that undermine inclusion, DS Leisure - an e-Training Program for improving Quality Of Life Through Inclusive Leisure- was introduced by Thessaloniki Active and Healthy Ageing Living Lab (Thess-AHALL). The present work presents the design, development and validation steps of the DS Leisure, where all Thess-AHALL actors -including people with DS and ID, their families, educators and Lab of Medical Physics- were actively involved in the process as co-creators. The aim of this paper is to evaluate the usability and adequacy of the Games and Virtual Scenarios included in the e-Training platform, as tools for training people with DS and other IDs on inclusive leisure. Eleven specialists with previous experience in the field participated in the survey. Findings suggested that both Games and Virtual Scenarios are easy to use, consistent and provide a sense of control to the user, although support from a trainer may be required. Additionally, the adequacy of the Games and Virtual Scenarios was positively rated, as they were considered to be constructive and interactive educational experiences for everyday skills enhancement. Overall, the positive feedback received sets the ground for further research to reveal whether designing withand not just for- people with ID is an approach that results to successful outcomes. Keywords: assistive technology, human- centered design, participatory, cocreation, intellectual disability, inclusion


1 Introduction Intellectual Disability (ID) is characterized by significant limitations in both intellectual functioning and adaptive behavior, with the latter covering a wide range of everyday social and practical skills (American Association on Intellectual and Developmental Disabilities, n.d.). Among the genetic causal factors related to ID, Down Syndrome (DS) is the most common one, affecting more than 5 million people worldwide (ECNP, n.d.). DS is a chromosomal abnormality that is associated with consequences in the areas of cognitive, linguistic, speech-motor and social development (Chapman & Hesketh, 2000). Many studies have empirically supported the role of social relations and active leisure experiences in happiness and well-being (Holder & Coleman, 2009; Holder, Coleman & Sehn, 2009). Although leisure has been associated with emotional and psychological benefits for people with ID (Williams & Dattilo, 1997; Caldwell & Gilbert, 1990), leisure participation for people with DS is restricted to solitary and sedentary activities (Oates, Bebbington, Bourke, Girdler, & Leonard, 2011). This suggests the need for ongoing support of people with DS and other IDs to develop and enhance the skills necessary for effective use of leisure time and to facilitate full inclusion (Buttimer & Tierney, 2005). 1.1 Assistive technology for Inclusion It is a fact that barriers in performing daily activities can be reduced or removed by the use of Assistive Technology (AT) (Reed, 2007). Such technologies as computer games or computer-based activities enable people with ID to make use of their leisure time and to participate in social activities (Erdem, 2017). In regard to people with DS, AT contribution is crucial in independent living and could contribute significantly in building self-confidence, independence as well as in achieving high quality of life (Reed, 2007; Feng, Lazar, Kumin & Ozok, 2010). Commonly, the integration of serious games in learning can be an effective approach for people with ID, as long as these games are specifically designed considering their unique characteristics and cognitive abilities (Cano, FernándezManjón & García- Tejedor, 2018). 1.2 Living Lab, Participatory design and Co-creation Living Labs are user-centered experimentation environments where users are part of the innovation’s co-creation and co-production process (Ballon, Pierson & Delaere, 2005). This approach advocates that “the user is not simply a source of information or evaluator of the final product, but an active contributor of design ideas and a decision-maker in the process, often referred to as “co-creator” or “co-designer” (Sanders & Stappers, 2008). Regarding the Living Labs operation, it is important that the roles of each participant- actor are distinct. In particular, the involved actors are: (a) providers, who “enter into Living Labs networks to co-develop new products”, (b) users including “both current and potential clientele of products and services” (c) utilizers, namely, “non-producers that seek efficiency gains and new knowledge from the Living Labs” and, (d) enablers, namely, “organizations that provide supportive technology, and other necessary resources to the use of participants” (Leminen & Westerlund, 2012).


In this vein, participatory design has been implemented as an approach that can help remove the barriers that undermine inclusion of people with DS when technology- enhanced learning tools are designed without accessibility in mind (Buzzi, Buzzi, Perrone et al., 2018). For example, Engler and Schulze (2017) involved people with DS, their families and trainers in the POSEIDON project, which aimed to increase their independence and autonomy by providing support in the areas of time management, mobility and money handling, and as it is proved, there is great benefit to increase inclusion of people with DS into society.

2 Previous work of Thess-AHALL Thessaloniki Active and Healthy Ageing Living Lab (Thess-AHALL) (, operating since 2014, is the Living Lab governed by the Laboratory of Medical Physics, Medical School of the Aristotle University of Thessaloniki ( The lab’s main strength lies with the health and social conditions’ improvement and independent living facilitation, by actively pursuing co-creation and co-design with end-users and relevant community stakeholders. In this manner, co- created work of Thess-AHALL includes Long Lasting Memories Care (LLM Care) service – an integrated ICT tool- which combines state-of-the-art mental exercises with physical activity. In this context, wFitForAll platform has been developed by the Lab including the physical training exercises (Bamidis et al., 2015). In contrast with mono- therapeutic interventions, LLM Care suggests a holistic approach against cognitive deterioration. Initially, the LLM Care service was exploited to improve or maintain the quality of elderly’s people life. Bamidis and his colleagues (2015) performed a LLM Care- based intervention to 322 older adults aged ≥55 years with or without neurocognitive disordered (NCD). The results of the intervention showed significant improvement to participant’s global cognition (p= 0.002). Following this, LLM Care service appears to be a vital tool to people with DS and other IDs providing important potential towards health, social and daily living improvement (Romanopoulou, Zilidou, Savvidis, et al., 2018). In this direction, Thess-AHALL co-creation network was exploited for the “Training Program for Improving Quality of Life through Inclusive Leisure for Persons with Down Syndrome”, called DS Leisure Project aims at increasing DS peoples’ competences in terms of attitudes, skills and knowledge enhancing their daily performance. This project is based on the wFitForAll Platform and includes training Games and Virtual Scenarios (VS). Table 1. DS Leisure design steps and how they were applied. Design steps

Involved Actors

How this step was applied in the project


Users: People with DS

Step 1: LLM Care implementation

Utilizers: Organizations for people with disabilities, Educators & Parents Enablers: Lab of Medical Physics (AUTH) Users: People with DS

Step 2: Co-Creation Working Session

Utilizers: Organizations for people with disabilities, Educators & Parents Enablers: Lab of Medical Physics (AUTH)

Step 3: Design and Development of the Training Materials and the DS Leisure e- Training Platform

Co-Validation of the Training Materials

Enablers: Lab of Medical Physics (AUTH)

Participants: 14

Utilizers: Organizations for people with disabilities, Educators & Parents Enablers: Lab of Medical Physics (AUTH)

At the Co-Validation Working Session, the Training Materials were presented and, through participatory and interviews, all participants (Users, Utilizers and Enablers) discussed with regard to the Activities extracted from the Co-creation Working session, such as the content and structure of Methodological Guide.

Participants: 11

Step 5: Validation of the eTraining Platform (Games & Virtual Scenarios)

Participants: 22 In the context of LLM Care’s new innovation services, Thess- AHALL organized a Co-Creation Session in Thessaloniki by actively involving Users and Utilizers into the design and selection process of the Training Materials, Assistive Technology, Contents and Methods of the DS Leisure Project in the wFitForAll. Based upon the Users need emerged from the Co-Creation Working Session, design and development of the Training Materials and the DS Leisure e-Training Platform were carried out on the wFitForAll Platform by the Enablers.

Users: People with DS

Step 4:

Participants: 46 wFitForAll Platform of LLM Care –a Thess-AHALL’s service- appears that may enhance health and daily living skills of people with Down Syndrome (Romanopoulou et al., 2018).

Finally, specialists were asked to fill in a questionnaire regarding the usability and adequacy of the Games and Virtual Scenarios of the DS Leisure eTraining Platform.

Utilizers: Specialists

Considering the importance of the participatory design, the involved actors and the steps of the DS Leisure Project design process by Thess-AHALL are described in Table 1 above.


3 Purpose of the study Among the five steps presented in the previous subsection (see Table 1), this work mainly focuses on the validation of the Games and Virtual Scenarios (VS) of the DS Leisure e-Training Platform (step 5). In particular, this study focuses on the usability and adequacy of the Games and VS as tools for training people with DS and other IDs on inclusive leisure.

4 Materials & Methods 4.1 Participants In order to validate the e-Training Platform (Games and VS), 11 participants took part in the survey (10 female and 1 male). All of them were specialists with previous experience in working with people with ID and came from a range of professional fields, such as psychologists, special educators and LLM Care trainers. 4.2 Games and Virtual Scenarios The results of the Co-Creation session have highlighted the need to enhance users’ memory performance and money management, as well as to get them aware of the everyday life challenges. Based upon these specific needs, DS Leisure e-Training Platform was enriched with two Games and ten VS, for which specialists’ validation considered necessary. In the first game, called the “Memory Game”, the user is asked to invert upsidedown cards displaying different leisure activities (e.g., cinema, party) and find pairs of matching images. In the second game, called the “Money Game”, banknotes and coins appear on the screen and the player has to make the right selection in order to reach a specific amount of money and fill his or her wallet. VS represent daily living leisure activities, such as going to the cinema or a concert, making a restaurant reservation, organizing a party, watching a football game, using the public transportation etc. Here, the goal is for the user to choose among different options and follow the path of the right decisions in order to correctly fulfil the activities.


4.3 Assessment & Procedure The questionnaire comprised of both closed-ended and open-ended questions was devised in order to assess the usability and adequacy of the Games and VS included in the e-Training Platform. In the closed-ended questions, participants were asked to determine in a 5-point Likert scale their degree of agreement to a set of statements regarding different usability and adequacy dimensions. In the open-ended questions, participants were asked to provide feedback regarding the strong and weak features of the Games and VS as well as their suggestions for improvement. The questionnaire was administered through an online survey tool with an introductory e-mail. The e-mail explained the general purpose of the study and the particular goals of the Games and the VS. It also contained the online links to the e- Training Platform and the online survey. Participants were asked: a) to visit the e- Training Platform and play the Games and the VS, b) to fill in the questionnaire.

5 Results & Discussion 5.1 Validation of the Games This subsection presents the results regarding the usability and adequacy of the games included in the e-Training platform. As Figure 1 shows, perceived ease of use was rated as equal or above neutral (i.e., 3) with the majority expressing an agreement (n = 5) or a strongly agreement (n = 3) that games were easy to use. Most respondents (n = 6) strongly believed that games were consistent and agreed that people with ID feel in control over the games. On the other hand, respondents did not express a clear view as to whether people with ID need the support of a trainer to use the games, and the majority (n = 7) gave a neutral rating.

Figure 1. Responses regarding with the Games usability by people with DS and other IDs.


Regarding game adequacy, on the whole, respondents gave positive ratings, as shown in Figure 2. The majority (n = 8) argued or strongly argued that the games serve the special needs of people with ID, they enhance the performance of people with ID and they offer direct interaction. Additionally, with the exception of two respondents, feedback appropriateness was considered to be above neutral.

Figure 2. Responses regarding with the Games adequacy.

Among the open-ended responses, strong game features were identified by all 11 participants. Specifically, the entertaining character of the games was mentioned by the majority (n = 8), followed by the visual stimuli (i.e., images, colors) (n = 3), the learning goals (n = 3) (e.g., “people can learn how to organize activities�) and the interactivity (n = 2). Self-paced training (n = 1), ease of use (n = 1) and feedback provision (n = 1) were, also, identified as positive game attributes. On the other hand, game weaknesses were mentioned to a lesser extent. Participants (n = 3) emphasized on the lack of auditory stimuli -namely, sounds, music, audio feedback and audio instructions- while there existed some doubts regarding the ease of understanding (n = 2). Short game-play duration (n = 1) and lack of different difficulty levels (n = 1) were also mentioned. 5.2 Validation of the Virtual Scenarios Considering the VS usability (Fig. 3), a contradictory result is that they are not considered to be difficult for people with ID (n = 9), though support of the trainer appears to be still recommended by the evaluators (n = 6). Furthermore, most of


them argue that VS are not lacking in consistency (n = 8), as such give full control to the end- user (n = 8).

Figure 3. Responses regarding with the Virtual Scenarios usability by people with DS and other IDs.

As shown in Figure 4, most of the participants agree or strongly agree that VS respond to everyday living (n= 8), as well they are constructive educational experience (n= 9). In fact, everyday skills as money handling and use of public transportation were both rated as equal or above neutral (Fig. 4). Regarding the open-ended responses about the VS, the evaluators mentioned content correspondence to the reality (n = 3), decision- making enhancement (n = 2) and design simplicity (n = 2). On the other hand, the lack of enough auditoryvisual stimuli has been identified as weakness, as it is possible to make the VS hard to understand (n = 3). Hence, the support of an educator is claimed to be a necessity (n = 2). Overall, the results from the specialists’ evaluation revealed some interesting insights. Among the open-ended responses, both positive and negative suggestions were reported. Moreover, most of the qualitative responses argue that Games and VS were accessible, pleasant and easy to use by people with disabilities (n = 6).


Figure 4. Responses regarding with the Virtual Scenarios adequacy.

Additionally, no technical obstacles were reported by the facilitators during their experience with the e- Training Platform (n = 11). Besides that, educational contribution of the games and the VS to social inclusion considered to be a significant advantage (n = 6), as they “enhance skills targeted to the needs of people with disabilities for autonomy and social engagement” and “support everyday living”. Finally, suggestions such as graded difficulty of levels (n = 3) and auditory provision of instructions (n = 2) were stated.

6 Conclusion It appears that the technology can significantly contribute to the process of providing everyday skills to people with ID. However, in contrast with most of digital games and web-based applications for people with DS, which are designed with no consideration of their special needs (Feng et al., 2010), this study innovation lies with the human-centered design (Maguire, 2001; Sanders & Stappers, 2008; Feng et al., 2010) followed by Thess-AHALL. As it was considered crucial to this survey considering not only the obstacles to be faced, but also the special needs of the people with ID, Thess-AHALL actively involved people with ID in the design process by choosing the Training Materials, AT tools, Contents and Methods of DS Leisure e- Training Platform. In addition, the special of this survey as opposed from previous approaches is the objective of the e- Training Platform itself which focuses on leisure activities that are yet considered to be restricted for people with DS (Oates et al., 2011), as well as the original material that was developed in a Living Lab network for this purpose.


Furthermore, this work presented the dynamic process that led to the DS Leisure e- Training Platform, offering an example of how sustainability –a key principle of Living Labs (Bergvall- Kåreborn, Eriksson, Ståhlbröst & Svensson, 2009) - can be achieved. Taking into consideration that innovation and evolution are two interdependent phenomena (Yurchyshyna, Khadraoui, Opprecht & Léonard, 2011), Thess- AHALL continuously builds upon and enriches the accumulated knowledge, by bringing new features to the LLM Care service and deploying its components (i.e., wFitForAll platform) to develop new products (i.e., DS Leisure e-training Games). Specifically, beginning with the Living Lab’s LLM Care service implementation to elderly people that subsequently applied to persons with DS and ID, DS Leisure Project co-creation followed. Regarding the results of this study, the validation of the Games and the VS by specialists revealed successful outcomes. Although, there is a remaining question concerning the positive effect –whether it is affected by the way the skills were trained or due to the fact that the Training Materials were co-created with the people with DS- that is yet not clear. In our point of view, both factors are considered to have been equally crucial to this matter, but more research is needed to clarify this question. In view of that, further research is also in the process of investigating whether or not autonomy and quality of life of people with ID may be enhanced by the DS Leisure e- Training Platform.

Acknowledgements This research was supported by DS LEISURE project funded by the European Commission within the ERASMUS+ 2017 Programme, as well as the business exploitation scheme of the ICT-PSP funded project LLM, namely, LLM Care which is a self-funded initiative at the Aristotle University of Thessaloniki (

References American Psychiatric Association (APA) (n.d.). What is Intellectual Disability? Retrieved at 30 May 2019 from: intellectual-disability Ballon, P., Pierson, J. & Delaere, S. (2005). ‘Open innovation platforms for broadband services: benchmarking European practices’, Proceedings of the 16th European Regional Conference, Porto, Portugal. Bamidis, P. D., Fissler, P., Papageorgiou, S. G., Zilidou, V., Konstantinidis, E. I., Billis, A. S., ... & Tsilikopoulou, G. (2015). Gains in cognition through combined cognitive and physical training: the role of training dosage and severity of neurocognitive disorder. Frontiers in Aging Neuroscience, 7, 152. Bergvall -Kåreborn, B., Eriksson, C. I., Ståhlbröst, A., & Svensson, J. (2009). A milieu for innovation: defining living labs. In ISPIM Innovation Symposium: 06/12/2009-09/12/2009.


Buttimer, J., & Tierney, E. (2005). Patterns of leisure participation among adolescents with a mild intellectual disability. Journal of intellectual disabilities, 9(1), 25-42. Buzzi, M. C., Buzzi M., Perrone E., & Senette C. (2018). Personalized technology- enhanced training for people with cognitive impairment. Universal Access in the Information Society, 1–17. Caldwell, L. L., & Gilbert, A. A. (1990). Leisure, health, and disability: A review and discussion. Canadian Journal of Community Mental Health, 9(2), 111122. Cano, A.R., Fernández- Manjón, B., & García- Tejedor, Á. J. (2018). Using game learning analytics for validating the design of a learning game for adults with intellectual disabilities. British Journal of Educational Technology, 49(4), 659–672. Chapman, R. S., & Hesketh, L. J. (2000). Behavioral phenotype of individuals with DS. Mental Retardation and Developmental Disabilities Research Reviews, 6(2), 84 – 95.;2- P ECNP (n.d.). Down Syndrome and Other Genetic Developmental Disorders Network. Retrieved at 1 June 2019 from: networks/List-ECNP-Networks/Down-syndrome Engler, A., & Schulze, E. (2017). POSEIDON - Bringing Assistive Technology to People with Down Syndrome: Results of a Three Year European Project. Studies in health technology and informatics, 236, 169-175. Erdem, R. (2017). Students with Special Educational Needs and Assistive Technologies: A Literature Review. The Turkish Online Journal of Educational Technology, 16(1), 128–146. Feng, J., Lazar, J., Kumin, L., & Ozok, A. A. (2010). Computer Usage by Children with Down Syndrome: Challenges and Future Research. Journal ACM Transactions on Accessible Computing (TACCESS), 2(3), 144. Holder, M. D., & Coleman, B. (2009). The contribution of social relationships to children’s happiness. Journal of happiness studies, 10(3), 329-349. Holder, M. D., Coleman, B., & Sehn, Z. L. (2009). The contribution of active and passive leisure to children's well-being. Journal of health psychology, 14(3), 378-386. Leminen, S., & Westerlund, M. (2012). Towards innovation in Living Labs networks. International Journal of Product Development, 17(1-2), 43-59. Maguire, M. (2001). Methods to support human- centred design. International Journal of Human-Computer Studies, 55, 587-634. doi:10.1006/ijhc.2001.0503


Oates, A., Bebbington, A., Bourke, J., Girdler, S., & Leonard, H. (2011). Leisure participation for school-aged children with Down syndrome. Disability and rehabilitation, 33(19-20), 1880-1889. Reed, P. (2007). A resource guide for teachers and administrators about assistive technology. Oshkosh: Wisconsin Assistive Technology Initiative. Romanopoulou, E., Zilidou, V., Savvidis, T., Chatzisevastou, L. C., & Bamidis, P., (2018). Unmet Needs of Persons with Down Syndrome: How Assistive Technology and Game- based Training May Fill the Gap. In Hasman et al. (Eds.) Data, Informatics and Technology: An Inspiration for Improved Healthcare, IOS Press, doi:10.3233/978-1-61499-880-8-15 Sanders, E., & Stappers, P.J. (2008). Co-Creation and the New Landscapes of Design, CoDesign, 4 (1), 5- 18. Williams, R., & Dattilo, J. (1997). Effects of leisure education on selfdetermination, social interaction, and positive affect of young adults with mental retardation. Therapeutic Recreation Journal, 31(4), 244-258. Yurchyshyna, A., Khadraoui, A., Opprecht, W., & LĂŠonard, M. (2011). Innovation and evolution of services: role of initiatives. In: Tambouris E., Macintosh A., de Bruijn H. (eds) Electronic Participation. Lecture Notes in Computer Science, vol 6847 (pp. 262-273). Springer, Berlin, Heidelberg.


Creating an anonymous, at-home screening for sexually transmitted diseases sent by letter mail: the cross-border development of a standardized urine collection device and associated testing service

Judith H.J. Urlings1,2*, Bianca Ceccarelli1,2, Claire A.G.J. Huijnen3, Paulette J.J. Wauben3, Joke DonnĂŠ4, Ronald Van den Bossche4, Alejandra Rios-Cortes4, Koen Beyers4 and Vanessa Vankerckhoven4 *Corresponding author Aging | LifeTechValley, Diepenbeek, Belgium 2 Hasselt University, Hasselt, Belgium 3 Centre of Expertise for Innovative Care and Technology (EIZT) at Zuyd University of Applied Sciences, Heerlen, The Netherlands 4 Novosanis, Wijnegem, Belgium 1 Happy

Category: Innovation Paper Abstract Novosanis' Colli-Pee is a unique device for standardized and volumetric selfcollection of the first-void fraction of urine. The aim of the present living lab project was to optimize the design of the Colli-Pee for postal mailing. Novosanis wishes to combine the Colli-Pee device with a sexually transmitted infection (STI) testing service via letter mail. Possible fitting of the product and associated service in the existing health care systems of Belgium and The Netherlands was explored by interviewing professional stakeholders. Additionally, a co-creation session and large scale live-test were performed with voluntary end-users in both countries. Both the outcomes of the innovation project as well as learnings for the living labs are discussed in this paper. Keywords: Cross border, Healthcare innovation, Product design, Business model design, Sexual health care


1 Introduction The first-void or first-catch urine (first 20 ml of the urine flow) is being used more and more for the detection of sexually transmitted diseases (STIs). A large research project from The Netherlands has shown that the prevalence of chlamydia can be significantly reduced by active screenings, as larger groups can be included in this type of screening (van den Broek et al., 2010). Online STI self-tests allow for home-based sampling and returning the sample kit to the lab via letter mail. This anonymous and non-invasive procedure might help patients to overcome feelings of shame or stigma, and might in turn lead to increased participation in screenings (von Karsa et al., 2015). Typically, a standard urine cup is used and the patient is asked to collect first void. However, this cup does not allow collection of a standardized first-void urine fraction. The first-void of urine can easily be missed, or the interruption of the urine stream is not successful. Additionally, the traditional urine containers do not allow efficient letter mail, except as a postal package. Novosanis' Colli-Pee is an unique device for standardized and volumetric self-collection of the firstvoid fraction of urine. It is a patented technology that automatically separates the first-void from the mid-stream urine without active user involvement. Novosanis wished to optimize their Colli-Pee urine detection device with respect to user-experience as well as letter mail delivery. The aim of the present project was to design a new generation of the Colli-Pee device that is optimized in architecture and materials to be optimally suited for postal mailing. Novosanis wishes to combine the Colli-Pee device with an STItesting service via letter mail. Possible fitting of the product and associated service in the existing health care system is explored by interviewing professional stakeholders. Both Belgian and Dutch end-users and stakeholders where involved in various stages of the design process: â—? Co-creation sessions o to gain knowledge on end-users wishes, needs, barriers and habits to STI screening. o to have various alternative Colli-Pee 3D print prototypes evaluated by end-users. â—? Live-tests: o to gain knowledge on end-user experience in the process of ordering, using and returning the sample kit. â—? Business-model interviews: o to gain understanding of the current practices in the field of STI detection and treatment. o to collect opinions of professional stakeholders on the Colli-Pee device and associated STI test service via letter mail. Those stakeholder involvement activities were carried out by two existing Living Labs: EIZT (Centre of Expertise for Innovative Care and Technology), associated with Zuyd Hogeschool in the Netherlands and Happy Aging, associated with Hasselt University in Belgium. Both Living Labs have extensive experience with like-wise innovation trajectories. Therefore, the network and panels of the Living


Labs – carefully curated over the course of several years- were crucial for carrying out the described project. The main purpose of the present paper is to disseminate learnings captured from this cross-border innovation project to the wider community of European living labs.

2 Methods 2.1 Background The current innovation project takes place within the framework of CrossCare. CrossCare is an Interreg Vlaanderen-Nederland project that is ran by six Living Labs: three located in Flanders (Belgium), three located in The Netherlands. Small and medium sized companies (SME) can propose an innovation project to CrossCare. If the project is accepted, the SME receives both financial as well as living lab support to further develop its product or service and business model. The typical duration of a CrossCare innovation project is between 12 and 24 months. Every innovation project is coached by two Living Labs: one in the Netherlands, one in Belgium. In the current project, the existing Colli-Pee device is further developed and the feasibility of an associated STI screening via letter mail – known as ‘Pee-post’ is explored. 2.2. Co-creation session Before the start of the session, all participants signed written informed consent forms, guaranteeing mutual confidentiality. This means that information about and input from participants is kept confidential and only shared in an anonymous format, but also that participants keep information that they receive on product design and the business model confidential. The co-creation session consisted of three parts: The Round Robin exercise, getting to know the Colli-Pee and the User experience/user acceptance quadrant exercise. All parts are described in more detail below. 2.2.1 Round Robin The Round Robin is a concept ideation technique in which ideas evolve as they pass from person to person (Luma Institute, 2012). The method is used to create fresh new ideas based on shared authorship. Passing a first idea to the next author can make the idea grow in unexpected ways. In our case, the Round Robin assignment was built up out of 3 questions: ● Where/how would you apply for an STI-test? ● What would the test look like? ● How would the results be communicated to you? Every participant started with an empty sheet with the 3 questions. He or she filled in question one, then passed on the paper to their left-hand neighbor. This second participant read the answer to question 1 and formulated his answer to question 2, given the framework outline in answer 1. The process continued until the three questions were answered. The assignment was followed by a plenary feedback session.


2.2.2. Getting to know the Colli-Pee As a second step in the co-creation process, the company Novosanis presented 4 different prototypes of the Colli-Pee device. Prototypes differed in terms of materials used, assembly needed by the user and recyclability. For the sake of confidentiality, the different prototypes presented are not discussed in detail here. The Colli-Pee is typically delivered to the end-user in a standard size cardboard box that meets the maximum size for letter mail delivery. This box contains the Colli-Pee, and a plastic sealable bag to wrap the urine sample for delivery to the laboratory. Participants of the co-creation sessions were also asked to view the user manual of the Colli-Pee that is presented on the inside of the delivery box in images. Secondly, the available manual-video is shown to the participants. Participants are asked to provide feedback, both on the device-prototypes, as well as on the instruction video. 2.2.3 User experience and user acceptance The group of participants was presented with one A0 size poster divided into quadrants. On the horizontal axis ‘user experience’ is represented. This scale runs from ‘makes the product easy to use’ to ‘makes the product difficult to use’. On the vertical axis ‘user acceptance’ is represented. This scale runs from ‘is important for in my choice to use the product’ to ‘is not important for me in my choice to use the product’. Eight product characteristics were pre-identified by Novosanis and summed up below. Every participant individually noted each characteristic on a post-it note and placed it on the desired quadrant on the poster: ● Pricing of the device and associated service ● The time the total testing procedure takes (application to test result) ● Recyclability of the device and packaging ● Material of the Colli-Pee ● Hygiene while using the device ● Chance of leaking ● Ergonomics of the device ● Ease of use 2.3 Live-test Eligible participants for the live-test (adults between 18-45 years old) were recruited through the CrossCare and Living Labs newsletter, social media accounts and websites of the Living Labs and through personal communication. Participants were recruited in two gender (male, female) and three age (18-24, 25-34, 35-45) groups. As the Belgian living lab, Happy Aging recruited 60 participants, equally distributed over the six groups. At the point of writing this paper the Dutch live-test was still on-going. Forty-one participants were already sent a Colli-Pee parcel (19 male). A total of 36 urine samples were returned, and 34 participants completed the live-test, including the evaluation survey up till now. No further recruitment is ongoing.


In the Belgian part of the study, participants applied for participation through an online web-form (Google Forms) and gave digital informed consent there. In The Netherlands, participants were sent an informed consent form via e-mail. Participants were ensured that no analyses except for volume determination would be performed on their urine sample. Age, gender and contact details including postal address was asked from each participant. Every participant gave consent to receive a urine-collection kit with the Colli-Pee device. After entrance into the study, participants received a confirmation mail (in Belgium with a digital link to the informed consent form for safekeeping by the participant), and a link to the online video-manual on the Novosanis YouTube channel. The Colli-Pee parcel was sent to the participant by the living lab. In Belgium, both the living lab and the participant could track their individual parcel through the online portal of the postal company. Instructions on how to return the parcel to a postal drop-off point once the Colli-Pee was used were included in the mail. In The Netherlands, the Colli-Pee parcels were sent in an envelope especially designed for medical transport. This procedure was adopted after consulting with the Dutch postal company and did not allow tracking of the parcel. For returning of the urine sample, Dutch participants could drop the return envelope in any of the orange mail boxes available in all neighborhoods. Once the parcel was delivered at the participant, a follow-up mailing was sent with again a link to the video-manual and a link to the evaluation survey. Individual participation codes (owned by the living lab researcher) ensured anonymous participation in the survey. If no response was recorded after 2 weeks, a reminder e-mail was sent. As a last reminder, a phone-call was made to the participant 5 days after the original reminder. 2.4 Business model interviews To gain insight on how the Colli-Pee and associated testing service can be embedded in the current care system, the living labs administered interviews with professional stakeholders, active in primary care in both The Netherlands and Belgium. Business model interviews were prepared through the method described by Albert & van der Auwermeulen (2017). Interviewees were recruited from the Living Lab partner organizations, both in Belgium as well as in The Netherlands. To increase the chance of participation, interviews were conducted at a time and location preferred by the interviewee. In Belgium, one general practitioner, one pharmacist and a manager from a healthcare insurance company participated in the interviews. In the Netherlands, one general practitioner, one pharmacist and one medical doctor working in a center for sexual healthcare participated. Due to the confidential nature of these interviews for both the interviewee and the company, the results of the interviews are not extensively reported in this paper.


3 Results 3.1 Co-creation session 3.1.1 Session results Belgium The co-creation session was visited by 8 eight participants (3 male), between 18 and 49 years old in Belgium. In the Belgian session, from the Round Robin exercise we learned that the family physician is considered as the trusted party to turn to for questions regarding sexual health. Trust in the family physician is higher than the level trust felt for commercial parties. Although in Belgium a patient is free to visit another general practitioner than the one that holds a person’s medical health record, most participants preferred to visit their personal family doctor. If participants would buy a test in a physical store (i.e. pharmacist, drugstore), one would assume that the results are readily available, similar to a pregnancy test. Participants would prefer a complete test-kit, rather than a separate test for a single pathology. The majority of the participants expect that a urine and blood sample are necessary for a full STI screening. An important insight is that participants expect to receive extensive additional information from their general practitioner. An informative conversation should cover both the test-procedure as well as information on STI´s and reliable preventive measures. Despite the fact that the family doctor is regarded as the preferred partner in STI testing, the majority of the participants would prefer to provide a self-sampled urine or a vaginal swab rather than have the procedure done by the doctor if this was a possibility. Most participants would prefer to receive the test-results in a face to face meeting. In case of online communication, the possibility should be given to have an (online) consultation with a doctor, especially in case of a positive screening. In such a consultation, treatment, future prevention and the need for notification of sexual partners should be discussed. From the user experience and user acceptance exercise, various important characteristics of the Colli-Pee and associated service were derived. First, the cost of the total service influences the user acceptance. If the cost exceeds the cost of a test by the general practitioner, then testing in the doctor’s office is preferred. Secondly, the duration of completion of the entire procedure was named as a factor in user acceptance. About half of the participant thinks a quickly available test results will influence their choice for an online test. Thirdly, half of the participants think choice of materials and recyclability influenced their user experience. 3.1.2 Session results The Netherlands Nine people took part in the Dutch co-creation session (2 male). In the Round Robin exercise Dutch participants indicate that they would contact their general practitioner or a center for sexual health for an STI screening. Others would prefer to buy a test at a drugstore or online. Participants expect that either the general practitioner, the assistant of the general practitioner or the patient him/herself would carry out the test. One would expect testing of a blood and


urine sample, either in the doctor’s office or in an external laboratory. For a test bought online or at a drugstore, one would expect immediate results. Participants have very diverse expectations towards the way they receive the test results. Various options that are named are: 1. To be contacted by telephone by the general practitioner, 2. To call a ‘test-results line’ or log in to a ‘test-results website’, 3. In a physical consultation with the doctor, 4. Directly read the results of the test. The user experience vs user acceptance quadrant is filled in per presented prototype. No large differences on characteristic placement arise between the four prototypes. Choice of materials and recyclability is important for the user acceptance. In contrast to the session in Belgium, time needed to complete the test and costs of the test are not regarded as top 3 characteristics in terms of user experience and user acceptance. Based on the learnings and insights gathered during the two co-creation sessions and the business model interviews, Novosanis optimized the design of the Colli-Pee and the entire Pee-post service for Belgium and The Netherlands separately. Approximately one year after the co-creation session, the product and service were ready for live-testing. 3.2 Live test 3.2.1 Test results In starting up the live-test important knowledge was gathered with respect to delivery options in both countries. Although the Colli-Pee parcels were optimized for letter mail, this type of delivery has an important disadvantage in Belgium. The postal service does not allow tracking of letter mail, therefore Colli-Pees sent by letter mail can´t be tracked. Delivery as a parcel allows for tracking of the shipment but has the disadvantage that it is not delivered in the letter mailbox at home. Additionally, for returning the parcels to the lab, the participant needs to hand in the parcel at the postal service centers for scanning, rather than drop it in the regular (red) letterboxes available in many neighborhoods. 3.2.2 Live test results Belgium Via Happy Aging, 60 adults between 18 and 45 years (20 in each category: 1824, 25-34, 35-45 years old) of age participated in the project. Sixty Colli-Pee parcels were sent out, 58 arrived at the participants home. Two participants did not pick up the parcel at the postal office (where the parcel is available after it could not be delivered at the home address) within two weeks, after which the parcel was returned to sender. Nine participants did not return a urine sample to the lab. A total of 12 participants did not complete the evaluation questionnaire despite one reminder per e-mail and one reminder per telephone. Twenty-six participants completed university education, 18 completed a professional bachelor level education. The quality of the information that participants received prior to participation in the project was rated as 7.36 on a scale from 1 to 8. The Colli-Pee parcel was delivered in the letter mailbox of the participant in 24 cases. In 15 cases, the participant was at home at the time of delivery and the parcel was handed over


by the postman. In all other cases, the parcel was available at the postal service center or delivered at a ‘safe location’ (e.g. on the terrace of the house) that could be communicated by the participant to the postal company online by using the tracking code of the parcel. None of the participants reported difficulties in receiving the Colli-Pee. Three packages were slightly damaged in transport: in one case the seal of the package was broken, in one case a corner of the box was damaged and in one case, the buffer solution was leaking from the sample tube. Information on how to use the Colli-Pee and how to return the urine sample was presented in a cartoon format on the inside of the box. The quality of this information was rated as 6.4 on a scale from 1 to 8. The anonymity of the parcel was rated as 7.34 on a scale from 1 to 8. Surprisingly, one in three participants did not view the instructional video before using the Colli-Pee. Two participants reported that they would rather not use the Colli-Pee again to collect a urine sample. 3.2.3 Live test results Netherlands In the Dutch part of the live test, coordinated by the living lab EIZT, bags were used for mailing instead of boxes, as strongly advised by the Dutch mail. Fortyone participants were included and were sent a Colli-Pee parcel. Of those, 34 participants (13 male) completed the procedure, including the evaluation questionnaire. Fourteen participants were over 35 years old, 15 were between 25 and 34 years old, 5 were between 18 and 24 years old. Twenty participants completed university education, nine completed professional bachelor level education. The quality of the information that participants received prior to participation in the project was rated as 7.29 on a scale from 1 to 8. All Colli-Pees were delivered in the letter mail box at the home of the participant. One participant had to request for another Colli-Pee as the first one was not delivered. As no tracking of parcels was possible in the Dutch part of the study, the whereabouts of this parcel is unknown. None of the participants reported any damage to the package after transport. Information on how to use the Colli-Pee and how to return the urine sample was presented in a cartoon format on the inside of the bag. The quality of this information was rated as 7.18 on a scale from 1 to 8. The anonymity of the parcel was rated as 7.09 on a scale from 1 to 8. In this case, more than half (20) of the participants did not view the instructional video before using the Colli-Pee. Two participants reported that they would rather not use the Colli-Pee again to collect a urine sample, four were doubtful on whether they would use the Colli-Pee again. 3.3 Living Lab Learnings It was striking that large percentages of participants of both the co-creation session as the live test completed higher education. This might have influenced the results of the session. For example, it was found that a large proportion of the participants would turn to their general practitioner for an STI screening. This


preference for care delivery through the general practitioner might be different in other socioeconomic status (SES) groups. Previous Belgian research for example has shown that lower SES is associated with lower participation in cervical cancer screenings by general practitioners (Lorant, Boland, Humblet, & Deliège, 2002). Therefore, in future projects sufficient attention should be given to the recruitment of a variety of end-users, not only in terms of age and gender, but also in terms of educational level, employment or native background. The user experience vs user acceptance quadrant method is not easy to use in a group discussion. Evaluating product characteristics on both scales at once makes the assignment unnecessary complex for participants, and leads to a situation where one of both aspects gets more attention than the other. For future co-creation sessions we would prefer to split the assignment into 2 steps. First, have all pre-identified characteristics sorted on their importance in making the device and service easy to use. In a second step, all characteristics can be sorted in order of importance for the device and service to be accepted by the end-user. Alternatively, prices could be allocated to product characteristics. Participants could in turn be given a fixed amount of fake money and be asked to ‘buy’ specific product characteristics. A relatively large percentage of participants was lost to follow-up in the live-test procedure in Belgium, despite strict monitoring of the participant (following the parcel online, written and personal reminders for the participant). In the Belgian part of the study, from a total of 60 participants, ten participants did not complete the evaluation survey even though some of these participants did return a urine sample. As the living lab project ran during the months July and August the low response rates could possibly be due to the summer holidays. Additionally, the obligatory effort of going to the postal office to return the sample package might have influenced the lost-to-follow up rate. If samples could be returned via the standard postal boxes, this might have reduced the effort needed to participate in the study. One important aspect of an anonymous STI screening procedure has not been simulated in our live-test, i.e., receiving the test-outcome for example through an online platform. This aspect may however have a large impact on the useracceptance of the service. Additionally, the follow-up on a positive screening by the patient could not evaluated in the current live-test as no real analyses were performed on the urine samples. For example, if a patient receives the outcome that an STI is detected, he or she should still turn to his or her general practitioner for medical treatment or further follow up. Cultural and organizational differences between countries matter in innovation projects. In the case currently presented, user-preferences for obtaining an STI screening differed between Dutch and Belgian end-users. For example, Dutch participants preferred the well-known centers for sexual health to turn to, in addition to their general practitioner. Belgium however, does not have such an extensive range of sexual health centers. Additionally, the cost of the test and the time needed to complete the test was regarded as more important in Belgium than in The Netherlands. Lastly, letter mail delivery options differ between


countries. These contextual factors influence the feasibility of an innovation like the Colli-Pee to a large degree. With respect to the business model interviews, it was learned that it is difficult to motivate professional stakeholders (e.g. general practitioners, pharmacists) to participate in a focus group if no financial compensation is foreseen. Therefore, we changed our approach from an original focus group set-up to individual interviews. Flexibility in terms of timing and location for the interview is necessary for Living Labs. Additionally, we learned that almost none of the consulted stakeholders held a negative attitude towards the product or associated service. It is the fitting of the product within the strict regulations in the healthcare sector (for example related to reimbursement by health care insurers) that most likely has a large influence on the market adoption.

4 Conclusion The present Living Lab project has involved both end-users as well as professional stakeholders in the development of the Colli-Pee device and the associated STI screening service via letter mail, marketed as ‘Pee-post’. Two co-creation sessions in two neighboring countries have raised interesting insight on citizens’ preferences with regard to sexual health care. Extensive livetesting in both Belgium and The Netherlands has shown that participants valued the anonymity of the service and found the product easy to use with the provided instructions. Almost all participants would consider using the Colli-Pee again. Additionally, the business model interviews with professional stakeholders led to additional knowledge on how a new product and service could be integrated into the existing primary health care systems. Belgium and The Netherlands – although neighboring countries – differ to a large extent both on user expectations as contextual factors (existing care systems, postal service etc.). On a living lab level, we conclude that other factors, next to gender and age, should be considered in recruiting participants to avoid bias because of educational level, socioeconomic status etc. Secondly, living lab experiments, in contrast to lab experiments, do not take place in controlled environments. Therefore, a relatively large drop-out and lost to follow up rate of participants should be anticipated. In addition to that, sufficient should be anticipated for the living lab manager to follow up on participants to complete the full study, including the evaluative survey. Thirdly, the neutral role of the Living lab is crucial in both setting up anonymous live tests as well as in facilitating (non-commercial) contacts with various stakeholder from the wider network of the Living Lab, such as, in this case, health care professionals and postal service providers.

References Albert, J., & Van der Auwermeulen, T. (2017, March). Why classic Business Modelling doesn't work for complex business domains–A new Business


Modelling approach Symposium (pp. 1-18).






Lorant, V., Boland, B., Humblet, P., & Deliège, D. (2002). Equity in prevention and health care. Journal of Epidemiology & Community Health, 56(7), 510-516. Luma Institute, L. (2012). Innovating for people: Handbook of human-centered design methods: LUMA Institute, LLC. van den Broek, I. V., Hoebe, C. J., van Bergen, J. E., Brouwers, E. E., de Feijter, E. M., Fennema, J. S., de Coul, E. L. O. (2010). Evaluation design of a systematic, selective, internet-based, Chlamydia screening implementation in the Netherlands, 2008-2010: implications of first results for the analysis. BMC infectious diseases, 10(1), 89. von Karsa, L., Arbyn, M., De Vuyst, H., Dillner, J., Dillner, L., Franceschi, S., Suonio, E. (2015). European guidelines for quality assurance in cervical cancer screening. Summary of the supplements on HPV screening and vaccination. Papillomavirus research, 1, 22-31.



IoT –based Smart Living Environments for ageing well in Greece

Segkouli Sofia*1, Stefanos Stavrotheodoros1, Kaklanis Nikolaos1, Votis Konstantinos1, Dafoulas E. George2,3,4, Karaberi Christina2,4, Tzovaras Dimitrios1 *Corresponding author Technologies Institute-ITI, Centre for Research and Technology Hellas-CERTH, Thessaloniki, Greece 2 CitiesNet- Intermunicipal Development Company, Digital Cities of Central Greece SA, Greece 3 Faculty of Medicine, University of Thessaly, Larisa, Greece 4 E-Trikala S.A Developmental Municipal Company of Trikala, Greece

1 Information

Category: Research-in-progress Abstract This paper aims to provide an overview of the Greek Deployment Site, one of the nine (9) deployment sites of the Large Scale ACTIVAGE project, a European Multi Centric Large-Scale Pilot for Ageing well. Main goal of ACTIVAGE is to provide an IoT Ecosystem Suite (AIOTES) resolving interoperability at different layers between heterogeneous existing IoT Platforms for Active and Healthy Ageing (AHA). Greece is one of the most “aged” countries in the EU. Specifically, according to a research made in 2014 [1], people at 65 years and over account for 20.2% of the total population. This large scale IoT pilot connects in one large scale pilot site three of the most innovative Greek regions (i.e. Municipality of Pilea-Hortiatis, 10 Municipalities in Central Greece, Municipality of Metamorfosis) that are representative of different, complementary, geopolitical and socioeconomic realities. This paper highlights the initial goals, achievements, technical solutions, critical technological, organizational, privacy and security challenges and also the best practices that have been initiated by the Greek Large-Scale Pilot, in order to address successfully a) pilots’ performance and b) new business models’ acceptance and ecosystem sustainability. To this end, a ‘reference evaluation framework’ has been initiated as a key aspect of the ACTIVAGE project for the smart living ecosystems’ assessment. It also stresses the obstacles that have been faced so far by the various stakeholders involved (end- users, healthcare professionals, relatives, social environment, caregivers) and the lessons learned during the Greek pilot recruitment, installations, training and running. Last but not least, the Greek Deployment Site has been assigned to coordinate the ethical and legal activities of the ACTIVAGE consortium. Thus, an outline is provided about the study of the ethical and legal requirements in depth and the optimum coordination of data management as it is experienced by the Greek deployment Site (GR DS) in compliance to the new regulation (GDPR). The ultimate goal was to address


trustworthiness, privacy, data protection and security in project level and also in each DS internally. Keywords: IoT Ecosystem, Ageing well, Active and Healthy Ageing (AHA), data management, data privacy and security


1 Introduction ACTIVAGE is a large scale IoT pilot aiming at setting –up and deploying a number of Use Cases (UCs) to provide added value services in elderly, across several European countries (Spain, France, Italy, Germany, Finland and United Kingdom). The project vision was to implement a strategic reference sustainable ecosystem, the IoT Ecosystem Suite (AIOTES), of smart living solutions to deliver interoperability at different layers between heterogeneous platforms and enable the interconnectivity of heterogeneous IoT devices with the ultimate goal to provide smart living services and ensure the autonomous living of seniors. Active and independent ageing is among the priorities of EU initiatives related to older adults [2]. AHA services based on the Internet-of-Things are promising and various IoT solutions are deployed in order to sense, measure and control indoor and outdoor activities to support the independent living of seniors [3]. Nevertheless, the existing infrastructures in the view of global IoT landscape lack the critical component of inter-connection which is expected to bring significant added value. Moreover, various interoperability barriers exist in the current IoT ecosystems, as the lack of IoT protocol interoperability (systems are often vendor locked by design), interconnected smart objects of different owner’s demand data sharing that raises serious privacy concerns, large-scale integration imposes rules that sometimes are not equally accessed by all users [4]. Therefore, the heterogeneity of IoT devices and sensors along with the communication technologies and interoperability in different layers is still a challenge for expanding IoT solutions in a global level [5]. ACTIVAGE aims at exploiting the plethora of IoT platforms into a dynamic ecosystem of connected devices, and overcome the fragmentation of architectures, and applications towards integrated environments and open systems in order to offer solutions to different stakeholders (users and their families’ service providers, public authorities). In line with this main concern, the Greek pilot of ACTIVAGE project envisaged to use unified IoT solutions at service level, involving different municipalities in terms of various pilots’ use cases used as an IoT enabled single pilot to provide smart living services and maximize the effects of IoT technologies in elderly autonomous life. The smart IoT environments were tested and validated in controlled and realistic environments before their large-scale deployment in the selected sites. Specifically, the smart home of CERTH/ITI was used for living lab testing, providing a fully controllable environment similar to the real cases.

2 Related Work Relevant research work demonstrated that Internet of Things (IoT) innovation could have tremendous opportunities for seniors willing to live autonomously at homes. Moreover, IoT ecosystems can provide healthcare services indicating great potential to support effective self-care and the independent living of elderly [6].


For example, for the We-care project [7] an IoT solution for the elderly living assistance has been developed that monitors patients’ vital information, and additionally provides mechanisms to trigger alarms in emergency situations. Mighali et al. [8] in the context of City4Age, presented a non-intrusive IoT-based system that collects information related to movement and body mobility and through analysis detects automatically behavioral changes in elderly people for the prevention of Mild Cognitive Impairment. Similarly, Wang et al. [9] designed a situation-aware abnormality detection system for the elder people that utilize a u-tiles sensor network to detect their position and trace. Zhang et al. [10] created a Smart Assistive Living platform that aggregates sensor information about environmental, cognitive, physical and physiological factors and integrate them to support the delivery of telehealth and telecare services to older people with mild dementia. Other solutions provide activity-monitoring functionalities with the utilization of wearable devices. For example, Popleteev [11] presented a system that uses a wearable magnet for activity tracking and indoor positioning, while BelmonteFernández et al. [12] presented a similar tracking system that uses a Smart-watch wearable device that acquires the Wi-Fi strength signals of surrounding installed Wireless Access Points. Although all the aforementioned solutions follow the IoT paradigm and provide useful insights of to improve the autonomous living of elder people, they have not been tested in a large-scale environment. Scale. Providing an open source, interoperable system based on AAL services that will offer high quality AAL services supporting home care and health of elderly through interoperability and low cost, for numerous elder users, is still a challenge.

3 Proposed framework The Smart Home Scenario is directly related to the following use cases: (UC1) Daily activity monitoring, (UC2) Integrated care and (UC3) Monitoring outside. It targets the older adult population that lives independently in 3 different Greek regions. A number of qualified Key Performance Indicators (KPIs) have been defined in terms of the project in order to assess the progress of pilots’ operations in respect to the Deployment progress, preparation, the installations, the experiment running, the open calls etc. Each DS of the ACTIVAGE project has to report periodically a number of KPIs in order to allow the progressive assessment of their status. Also a number of questionnaires and scales have been defined for administration in elderly and their caregivers to provide qualitative information about the impact of IoT technologies in daily life. The key stakeholders of the proposed framework are the following: ● Elderly end –users ● Caregivers: relatives or health professionals ● Researchers and Developers ● Entities willing to provide AHA services (i.e. municipalities)


Despite the typically of the set –up, the innovative point is that for the first time AHA services are envisaged to be provided through IoT technologies in such a wide scale. Concerning the GR DS, the target values that have to be achieved by the end of the project in respect to recruitment and installations KPIs and also the current values so far are quoted below: Table 1: Key Performance Indicators, Current and Target Values in Greek Deployment Site Elderly Recruitmen Greek t Status Deployment Site (Current Values) DCCG 108 Municipality of 70 Pylaias – Chortiati Municipalityof 68 Metamorhosis

Caregivers Recruitment Status (Current Values) 83

Elderly Target Values

Installations (Current Values)

Installations (Target Values)












The inclusion criteria for participation acceptance in the research were the following: ● elderly above the age of 65 ● living alone ● have no serious medical or clinical condition that acquires 24-hour monitoring by another person In terms of the Smart Home environment, an IoT-based system that is consisted of commercially available devices was installed in each user’s home. This system uses the universAAL IoT platform and collects data from 6 wireless devices that are either wearables (e.g. panic button) or placed on specific places/rooms of the home. The architecture of the technical solution used in the Greek deployment site is depicted below:


Figure 1: Greek Deployment Site Monitoring Platform

All data collected from the devices and also the measurements (Table 2) are transmitted to the cloud server via a gateway (Raspberry Pi) in real-time. Table 2: Devices, sensors utilized in the Greek Deployment Site No. of sensors 1

Wireless sensor

Sensor placement


Panic button


Alarm (binary)

1 x bedroom 4

1 x living room 1 x kitchen

Motion sensor

Presence (binary) Temperature (°C) Luminance (Lux)

1 x bathroom


Door opening/closing detector

Central door

Open/close (binary) Temperature (°C)


Personal Health System

Tele BP/glucometers

Blood Pressure and/or Glucose measurement

Then, this data is automatically analyzed in order to enable: a) daily behavioral activity monitoring, b) health status monitoring, c) alarms/notifications provided to caregiver in urgent cases. All the information collected from the smart home installations is transmitted to the corresponding caregivers through the ACTIVAGE Smart Home monitoring platform. In particular, a dashboard for each installation indicates in real time all the data as received from the sensors (Figure 1).


Figure 2: Sensors' data displayed in the monitoring platform of the Smart Home

Moreover, rules are used within the monitoring platform to alarm for abnormal events displayed accordingly in the dashboard (Figure 3).

Figure 3: Notifications displayed in the ACTIVAGE monitoring platform

Additionally, the ACTIVAGE monitoring platform provides access to all collected data from each sensor. This data could be visualized through diagrams with environmental measurements (e.g. temperature). A sample of data is depicted concerning the temperature of a smart home as sent during a concrete period (Figure 4).


Figure 4: Diagram of temperature measurements

4 Methodology, challenges and barriers Within ACTIVAGE, a concrete methodology has been followed for the assessment of smart living solutions for ageing well. In particular, specific indicators related to QoL, accessibility and usability and also services’ models introduced during pilots’ performance. Among the main objectives of ACTIVAGE was to monitor and track specific metrics at a local and global level towards a coherent and ‘’Glogal’’ evaluation framework. This framework is based on the main Triple Win indicators of the European Innovation Partnership on Active and Healthy Ageing (EIPonAHA): Impact on QoL, Sustainability, Innovation & Growth. This evaluation framework is defined to be communicated to different stakeholders through the ACTIVAGE Evidence Open Database, a global data repository consisted by three user interfaces: o The LSP dashboard, which tracks real time data by each deployment site o The ‘ACTIVAGE Public Evidence website’, an open data base which will be available also after the project’s end o The ‘AHA –ADVISOR’, an interface provided as a web-based ICT platform which offers valuable services enabled by IoT technologies for active and healthy ageing. In line with these objectives, the GR DS is dedicated to enable new IoT based services to support seniors in different Greek regions and also exploit these solutions to create IoT business cases for healthy ageing.

4.1 Uses Cases implementation in IoT context


The GR DS has to implement 3 use cases in respect to the domain of AHA, AAL and e-Health, which were mentioned in the previous section. Concerning the UC1 & UC3, the health personnel from each Municipality has to recruit a sufficient number of potential users, in order to accelerate the initiation of installations. The UC2 services have been initiated since the procurement of the routers for the tele BP/glucometers had been completed. In terms of the ACTIVAGE project, the IoT Ecosystem has been validated and tested in realistic environments having as a main target to collect technical values the KPIs. Concerning the GR DS, relevant KPIs have been defined for users’ recruitment, installations, and also for the scales/questionnaires administered in elderly users and their caregivers. Three evaluation periods (baseline, intermediate and final) have been defined during the pilot and the evaluation tools used are the following: o CarerQoL-7D to measure the impact on quality of life of the caregiver [13] o EQ5D-3L to measure the impact on quality of life of the elderly end user [14] o UT-AUT to evaluate the end user’s acceptance of the service [15] o Global Questionnaire which is a common evaluation tool designed for all 9 Deployment sites that provides input on QoL and Acceptability of ACTIVAGE IoT platform by users (16) o ADL and IADL to measure the end users’ level of independency on everyday chores [17] [18] o UEQ to measure the end users’ level of empowerment [19] o UCLA to measure the end users’ level of isolation and loneliness [20] o FES-1 to evaluate the end users fear of fall [21] 4.2 Users’ Recruitment, consent procedure In respect to the GR DS, the recruitment process of the users was and still is among the most challenging issues for personnel assigned to conduct this task. Timing and synchronization of user recruitment and pilots’ installations was among the most critical factors for the pilot progress. From the very beginning of the project within the GR DS concerns and afterwards when the new regulation came into force (May 2018) [22], the consent procedure has been addressed as an integral part of data management. Therefore, what considered important was to produce consent forms in compliance to GDPR and met all requirements in respect to privacy –related data management. 4.3 Personnel/Developers Training Before and during the installation phase, the GR DS leader (CERTH) played a supportive role concerning the main functionalities and technical requirements that have to be followed in respect to devices’ configuration and installation. Therefore, training sessions have been scheduled in order to provide the proper guidelines and adequate assistance in installations’ procedure. Moreover, a


manual has been prepared and handed out to developers in order to proceed smoothly in the procedure. Additional on-line training sessions took place in order to train the health personnel responsible for the recruitment the evaluation about the official forms and evaluation tools distribution and data gathering that takes place throughout the pilot. 4.4 Technological challenges The main role of the Greek DS is to support the ACTIVAGE development and infrastructure of IoT services/tools (existing and new). In particular, the Greek DS was assigned to define and implement an infrastructure for data storing processing; including cataloguing, indexing and searching of all ACTIVAGE IoT data (services, applications, devices). The Greek pilot site and more specifically CERTH, was the responsible for developing large scale data Analytics mechanisms to extract maximum value from both historical and real-time data stored in the Data Store will be analysed and integrated. The focus is particularly on predictive data analysis mechanism – finding patterns and associations in the data, which enable us to predict issues, spot anomalies, and understand relationships between different important factors. CERTH developed a multi-functional platform with multi-criteria analysis and detection of spatiotemporal patterns in order to ensure the optimization of IoT management operations. 4.5 Open calls Smart living spaces for ageing well supported by the IoT ecosystems were a great business and market opportunity to be extended in the course of ACTIVAGE and also after the end of the project. From this perspective open callers were anticipated to integrate and /or exploit existing the work, project functionalities and services, retrieving new offerings adapted to specific user requirements. However ethical and legal points of data management as well as technical issues (technologies integration etc.) have to be solved. 4.6 Ethical and legal risks IoT technologies and ecosystems could offer the potential of big data use and data access with tremendous potential for personalized services. From the beginning of the project it was clearly realized that connected smart things and their services should be adapted to an ethical and legal IoT environment. In respect to the core principle of data protection, Ethics by Design [22], a data policy framework has to be defined according to devices, platforms’ design and development from the design phase of the project. In addition, it is of high importance that as the project was running and on May 2018 a new regulation came into effect, General data Protection Regulation (GDPR) and ethical and legal issues of a vast amount of data handling had to be managed properly. In line with the new regulation (CDPR) and due to the need of ‘special categories’ (health data, known as sensitive data) of data processing in terms of the ‘UC2


Integrated care’ on a large scale and through a systematic monitoring and use of IoT technology, the GR DS had to plan and conduct carefully a number of organizational activities: ● DPO and controllers, processors’ assignment to monitor GDPR oriented actions ● Data flow processing ● DPIA documentations of each municipality adapted to the needs and the concerns of the use cases ● Initiate a Data Protection Methodology (e.g. personal data collected for specific, legitimate and explicit purposes, processed in consistency with the purposes defined from the DoA of the project, access is permitted in authorised persons) Moreover, a number of technical mitigation measures and Privacy Enhancing Technologies (PETs) have been initiated to minimize the risk of potential data breach and ensure data protection and humans’ rights. In particular a number of encryption algorithms have been used (RSA RC4, DES, 3DES), RSA RC4 (40, 56, 128, 256-bit keys), DES (40, 56-bit keys) and 2-key 3DES ή 3-key 3DES (112 or 168-bit keys) as well as software components based on technologies such as blockchain [23] etc., to enhance the security and privacy in AIOTES architecture. Moreover, authentication mechanisms have been utilised for resource accesses in universAAL based on user roles and associated access rights in order to ensure that personal data will be submitted to the right data subject [24]. Data anonymization mechanism is reversible only for authorised entities.

5 Conclusion and Future Steps From the current status perspective of the GR DS, more than 90% of the recruitment has taken place and all installations, thus baseline evaluation will be completed before July 2019. Intermediate evaluation that will take place during August will provide a more complete feedback in respect to both the local and the global KPIs. Despite the technical problems that all the stakeholders of the GR DS (developers, health personnel, psychologists etc.) have to overcome, IoT technologies have been proven so far a very challenging and effective solution for people living alone and need for daily support and guidance. In terms of ACTIVAGE, our study has been devoted to address elderly requirements and different demands in relation to care requirements through IoT –based remote monitoring. However, despite the importance of IoT provided services for elderly care and healthcare, personalization issues have to be further addressed in future oriented IoT based systems for elderly. Consequently, what is of outmost significance is the dynamic nature of seniors’ needs, hence personalized monitoring should be among the most challenging aspects of future oriented IoT based systems for elderly remote monitoring and well-being.



This paper is part of the ACTIVAGE project that has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement No 732679.

References [1] europe/

[2] Christensen, K., Doblhammer, G., Rau, R., & Vaupel, J. W. (2009). Ageing populations: the challenges ahead. The lancet, 374(9696), 1196-1208. [3] Vermesan, O., & Bacquet, J. (Eds.). (2019). Next Generation Internet of Things: Distributed Intelligence at the Edge and Human Machine-toMachine Cooperation. River Publishers. [4] Arbel, I., Baker, K., Bailey, T., Bouraoui, M., Chartier, I., Contin, L., ... & Vermesan, O. (2015). AIOTI WG5-Smart Living Environment for Ageing Well. Alliance for Internet of Things Innovation (AIOTI), Report v1. 0. [5] Rechel, B., Grundy, E., Robine, J. M., Cylus, J., Mackenbach, J. P., Knai, C., & McKee, M. (2013). Ageing in the European union. The Lancet, 381(9874), 1312-1322. [6] World Health Organization. (2011). mHealth: new horizons for health through mobile technologies. mHealth: new horizons for health through mobile technologies. [7] Pinto S., Cabral J., Gomes T. (2017). We-care: An IoT-based health care system for elderly people. IEEE International Conference on Industrial Technology, ICIT, IEEE, pp. 1378-1383 [8] Mighali V., Patrono L., Rodrigues J. J. P. C., Solie P., Stefanizzi M. L. (2017). A smart remote elderly monitoring system based on IoT technologies. Ninth International Conference on Ubiquitous and Future Networks ICUFN 2017. [9] Wang J., Cheng Z., Zhang M., Zhou Y., Jing L. (2012). Design of a situation – aware system for abnormal activity detection of elderly people. Proc. Eighth Int'l Conf. Active Media Technology (AMT), pp. 561-571. [10] Zhang, Q., Su, Y., Yu, P. (2014). Assisting an elderly with early dementia using wireless sensors data in smarter safer home. Service Science and Knowledge Innovation, pp. 398–404. [11] Popleteev, A. (2015). Activity tracking and indoor positioning with a wearable magnet. In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers, pp. 253–256 [12] Belmonte-Fernández, Ó., Puertas-Cabedo, A., Torres-Sospedra, J., Montoliu-Colás, R., Trilles- Oliver, S. (2016). An Indoor Positioning System Based on Wearables for Ambient-Assisted Living. Sensors 17(1), 36.


[13] Brouwer, W. B. F., Van Exel, N. J. A., Van Gorp, B., & Redekop, W. K. (2006). The CarerQol instrument: a new instrument to measure care-related quality of life of informal caregivers for use in economic evaluations. Quality of Life Research, 15(6), 1005-1021. [14] Van Reenen M., Oppe M.(2005). EQ-5D-3L User Guide Basic information on how to use the EQ- 5D-3L instrument, Version 5.1. EuroQoL R. F. [15] Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS quarterly, 425-478. [16] ACTIVAGE public deliverable (2017). D 6.1 Consolidated list of KPI and coordinated methodology for evaluation documents/ [17] Katz S. (1983). Assessing self- maintenance: Activities of Daily living, mobility and instrumental activities of daily living. J Am Geriatr Soc. Dec;31(12):721-7. PMID: 6418786 [18] Graf C. (2008) .The Lawton Instrumental Activities of Daily Living (IADL) Scale. Best Practices in nursing care to older adults. The Hartford Institute for Geriatric Nursing. April, VoL.108(4):59 [19] Wanner, M., Hartmann, C., Pestoni, G., Martin, B. W., Siegrist, M., & MartinDiener, E. (2017). Validation of the Global Physical Activity Questionnaire for self-administration in a European context. BMJ open sport & exercise medicine, 3(1), e000206. [20] Russell, D., Peplau, L. A., & Ferguson, M. L. (1978). Developing a measure of loneliness. Journal of personality assessment, 42(3), 290-294. [21] Yardley, L., Beyer, N., Hauer, K., Kempen, G., Piot-Ziegler, C., & Todd, C. (2005). Development and initial validation of the Falls Efficacy ScaleInternational (FES-I). Age and ageing, 34(6), 614- 619. [22] hics/h2020_hi_ ethics-data-protection_en.pdf. [23] Theodouli, A., Arakliotis, S., Moschou, K., Votis, K., & Tzovaras, D. (2018, August). On the design of a Blockchain-based system to facilitate Healthcare Data Sharing. In 2018 17th IEEE International Conference On Trust, Security And Privacy In Computing And Communications/12th IEEE International Conference On Big Data Science And Engineering (TrustCom/BigDataSE) (pp. 1374-1379). IEEE. [24] Raschke, P., KĂźpper, A., Drozd, O., & Kirrane, S. (2017, September). Designing a GDPR- Compliant and Usable Privacy Dashboard. In IFIP International Summer School on Privacy and Identity Management (pp. 221-236). Springer, Cham.


Participatory design and validation of an innovative training program to maintain Autonomy of older adults with Alzheimer’s Disease

Despoina Mantziari1*, Antonis Billis1, George Arfaras1, Maria Karagianni1, Vasiliki Zilidou1 and Panagiotis D. Bamidis*1

1 Thessaloniki

*Corresponding Author Active & Health Ageing Living Lab, Medical School, Aristotle University of Thessaloniki, Greece Category: Full Research

Abstract Dementia and its most common type, Alzheimer’s disease, constitute one of the most challenging global health priorities. Nowadays, several dementias- friendly initiatives insist on the support of Autonomy of older adults with initial/mild Alzheimer’s and their caregivers. AD-Autonomy project aims to enhance (ICT) skills/competences/attitudes of PwAD and their carers through an innovative training program. The project adopted the participatory design approach in two milestones, a) the co-creation of the training programme methodology and b) the co-validation of the training material and the eLearning environment. This paper focuses on Thess-AHALL’s co-design sessions with end-users, as one of the five pilot sites of the program. Keywords: Alzheimer’s Disease, autonomy, e-learning, assistive technologies, co- creation


1 Introduction According to the World Health Organization (WHO 2019), around 50 million people have dementia in 2019, worldwide, while there are 10 million new cases every year. As dementia is an umbrella term for a variety of brain disorders, affecting cognitive function and memory, Alzheimer’s disease (AD) is the most common type of dementia, representing around 60%-70% of the total number of cases in the world. Accordingly, the joint OECD and EU report on the 2018 State of Health among the EU countries estimates that 7% (9.1 million people) of the population aged over 60 are living with dementia in EU member states, compared to 5.9 million in 2000. The total number of persons with dementia (PwD) is projected to double by 2030. Countries in the European south that have higher rates of older adults, generally have a greater proportion of persons with dementia (Italy, Greece, France, Spain). It is estimated that around 8% of population aged over 60 is living with dementia in these countries (OECD/EU 2018). As dementia progresses to Alzheimer’s, person’s abilities of memory, thinking, orientation, comprehension, calculation, learning capacity, language and judgement are significantly affected. Moreover, the cognitive function impairment, in most of the cases, is followed by loss of emotional control, changes in social behaviour and limited self-esteem, which, in combination with the social stigma and discrimination, experienced by people with AD, often lead to depression and social isolation. Also, WHO includes dementia among the highest costly chronic diseases, without treatment currently available to cure or change its progress, estimating that its societal cost amounted to 1.1% of the global GDP (USD 818 billion) in 2015 and most of the care to be provided by family caregivers (WHO 2017). Having a high physical, psychological, social, and financial impact, not only on persons with dementia, but also on their families, caregivers and the society at large, dementia has been established as the leading cause of dependency and one of the ten leading causes of mortality, worldwide. The WHO and the EU have placed dementia as a growing challenge and one of the main priorities for the global public health in the next decades. Both have invested in dementia-friendly initiatives, increase of the social awareness and limitation of stigmatization, early diagnosis and the training of patients, families and carers, as well as healthcare professionals in order for PwD to preserve their competences and Quality of Life (QoL), while the disease progresses. In recent years, innovative approaches (Bamidis, Antoniou, & Sidiropoulos 2014) allow persons with dementia to preserve their life as full as possible outside the healthcare environment, introducing alternative residential care models and improving their Quality of Life (QoL), while the development of assistive technologies and ICT Tools enhance their daily life competencies (Bamidis et. al, 2015) in a secure environment with the relevant supports (Bamidis, Konstantinidis, Billis, & Siountas, 2017). AD-Autonomy is a European-funded project that aims to “increase the (ICT) competences (attitudes, skills, knowledge) of Persons with Initial/ Mild Alzheimer's Disease (PwD), Families and Caregivers, about how to improve Autonomy and subsequently Quality of Life (QoL) of PwD through an innovative training program”. Embracing the co-creation methodology


and the participatory design principles, the AD-Autonomy project partners set end-users in the spotlight, by actively engaging end- users (PwD, families & carers, healthcare professionals) from Spain, Greece, Slovenia, the UK and Turkey in two milestone phases of the training program development:(a) the cocreation of the methodological training guide, and (b) the co-validation of the training program, by testing the e-platform (eLearning environment) and evaluating the final training material.

2 Materials and Methods AD-Autonomy is launched having as its main objective to enhance the competences (attitudes, skills, knowledge) of Persons with Initial/Mild Alzheimer/Dementia (PwA/PwD), Families and Caregivers, in order to improve their Quality of Life of PwA and preserve/increase their Autonomy through an innovative training program. Through the development and implementation of this programme, AD-Autonomy aims to address the following specific objectives: ● To aware and motivate the targeted stakeholders on the importance of maintaining the autonomy of people, within a security and support environment, as an element of QoL for PwD and their families. ● To increase the Autonomy of PwD for decision-making and independent living, adopting a universal Well-being and QoL approach. ● To transfer good practices and recommendations to PwD, as well as to their formal and informal caregivers, to promote Autonomy through training in the execution of daily routines ● To transfer tools, including assistive technologies and ICTs, to support the processes of empowering PwD and increasing their Autonomy. ● To transfer good practices for the emotional management of impairment associated with dementia and related problems, based on the concept of mindfulness. ● To involve professionals and experts, coming from different specialties, regarding PwD health and care, based on the exchange of their knowledge and experiences and with end-users, their relatives and the research team. To achieve its objectives, launching a training programme that really meets the needs and requirements of end-users (PwD, families and caregivers), the ADAutonomy consortium followed a participatory approach, based on the co-design methodology and the mutual exchange of experience and know-how among all the involved stakeholders. Over the last decade, the co-design methodology has emerged as a significant bottom- up process - social in the means that it uses and in its ends - in which researchers, social innovators, creative communities, citizens, vulnerable groups and civil servants co- create solutions to tackle societal challenges from every aspect of the public life (social inclusion, health & well-being, employment, migration, climate change, etc.) and address the unmet needs of the society, based on the strong collaboration and involvement of all the different types of stakeholders (Moulaert, et. al. 2013; Terstriep, et. al. 2015).


The participatory approach has been implemented through all the preparation steps of the training programme, to collect requirements, to develop the training materials and specifications, to build the ePlatform, as well as during the testing phase, regarding the co-validation and evaluation of entire training programme (face-to-face sessions, experiential activities with end-users, use of the ePlatform). In order to maximize the impact of stakeholders’ involvement in the development of a coherent and effective training programme, the AD-Autonomy also asked them to co-define Autonomy and the aspects of daily life that are crucial for a person to live independent (Figure 1). To achieve this, a staged cocreation model was followed: i) expertise and know-how on related projects and initiatives were collected through structured questionnaires and interviews with experts from the dementia healthcare sector, in order for the research team to establish the “State-of-the-Art” of Autonomy and its importance for PwD, ii) open dialogue with older adults and their caregivers/relatives, regarding the co-creation of a universal definition of Autonomy and its impact on the main aspects of the everyday living.

Figure 1. The training co-creation methodology followed in the five pilot sites

3 Co-creation of the methodological training guide Two rounds of co-creation sessions with end-users (older adults with dementia/mild cognitive impairment, their relatives and health professionals) were conducted in late spring 2018 in the five pilot sites (three patient associations and two universities) of the AD-Autonomy involved countries: the AFA-Castellon Alzheimer Association (AFA), Turkish Alzheimer Association (TAD), Spomincica-Alzheimer Slovenia, the Thessaloniki Active and Health Ageing Living Lab from the Aristotle University of Thessaloniki (Thess-AHALL) and the Bournemouth University Dementia Institute (BUDI). The conclusions of these sessions led to the development of a co-created methodological guide for the training program, including, i) the definition of Autonomy and its main aspects in one person’s everyday life (dimensions of Autonomy), ii) the design -with and


for end-users- of the experiential training activities (good practices, strategies), supported by technological solutions (existing ICT tools and apps), simulating real life contexts and situations, and iii) the setting of the user requirements for the etraining platform for the program. Both sessions were organized as focus groups of two-hour duration. 3.1 Co-defining Autonomy and its main dimensions In Greece, the first “AD-Autonomy” co-creation session was organized and held by the Thessaloniki Active & Healthy Ageing Living Lab (Thess-AHALL), located and governed by the Medical Physics Laboratory of the Aristotle University of Thessaloniki (AUTH) in early May 2018.Five older adults (4 females, mean age 77.2 y.o.), suffering from Mild Cognitive Impairment (MCI) or having subjective memory complaints, participated. One of them had the dual role of the family caregivers, since they provided daily care at their sister, suffering from AD (Billis, Mantziari, Zilidou, & Bamidis 2018). In addition, three healthcare professionals participated in the session, sharing their experience and perspectives on the caring and autonomy of the PwD. Thess- AHALL trainers guided the discussion to specific issues, related to i) the perception of Autonomy and its main dimensions, ii) good practices, strategies and iii) ICT tools and apps that in everyday routine contexts could help PwD to preserve theirautonomy and improve their QoL and assist caregivers to promote patients’ independency. Greek PwD summarized Autonomy as: “to be able to serve yourself”, and remain “physically, mentally and emotionally healthy”. Regarding the Autonomy dimensions, experts provided the initial “AD-Autonomy” list of nine proposed main aspects of the everyday life, as presented in Table 1. Table 1. Initial AD-Autonomy list of dimensions Table 1:

Security & Safety Personal Hygiene Health & Recreation Food Preparation & Eating Housekeeping Outdoor Activities Finance, Administration Orientation Entertainment & Social Interaction

Participants commented on the proposed dimensions and promoted their own views and suggestions on the design of a second, more comprehensive and inclusive list. PwD highlighted the importance of “Security & Safety” dimension, arguing that it is one of the primary aspects for both patients and their caregivers. Moreover, they mentioned personal hygiene, money management, medication management, housekeeping and meal preparation, sleep routines, orientation and public transportation as core


dimensions for the everyday life and the preservation of the autonomy of people, suffering from memory problems. Greek participants also commented on the significance of socializing and the need for limiting the social cultural stigma towards people, facing cognitive function impairment. On their side, professionals and the caregiver emphasized on emotional control, referring to communication and effective personal relationships, expression of emotions and understanding of one’s feelings, the patience and the skills that relatives and caregivers should train. Regarding emotional skills and personal relationships, PwD agreed with patience and raised the issue of companionships and each other care for elderly couples, as well as sexual relationships and inappropriate behaviour of people, affected by Alzheimer’s. Older adults with dementia, their relatives and health professionals worked in a similar way in the rest four pilot sites, providing their personal beliefs on Autonomy and its dimensions. Accordingly, based on the results of the first cocreation sessions, and with respect to cultural differences, raised by participants in the five pilot sites (e.g. praying for Turkish older adults, sexual relationships for Greek PwD), the AD-Autonomy research team consolidated the final eight main Autonomy dimensions and an elementary dimension to introduce and assess the Autonomy, on which the training program activities have been designed (Table 2): Table 2. Final AD-Autonomy list of dimensions Table 2:

Autonomy & Autonomy Assessment (Dimension Zero) Security & Safety Personal Hygiene Health Management & Sleep Food & Housekeeping Meaningful Activities (Including Leisure, Social Activities, Recreation)

Finance Orientation & Navigation Emotional Skills

In a second round of co-creation sessions, end-users in every participating country were informed about the results and the suggestions of end-users in every pilot site, which led to the transformation of the initial dimensions’ list. In Greece, Thess-AHALL organised in early June 2018 its second session, as a focus group with the active involvement of four older adults (3 females, mean age 74.8 y.o.), with MCI or subjective memory complaints -one of which had the dual role of family carer- a family caregiver and three experienced health professionals. Greek participants agreed on the final list, also expressing their satisfaction for having common views on the Autonomy issue with participants in the rest of the project involved countries. Moreover, Greek older adults agreed


on the co-created term of Autonomy, as it stemmed from the contribution of all participants’ views, highlighting that “maintaining physical and psychological health” is the most complete approach, which encompasses all aspects of Autonomy in one person’s life. Providing additional comments on the Autonomy dimensions, Greek MCIs and professionals emphasized on the “Security & Safety”, “Health Management & Sleep”, “Orientation & Navigation” and “Meaningful Activities”, also confirming participants from other countries that they constitute the highest priorities for people with memory problems to preserve their competences in everyday life. Summarizing, the evaluation and determination of Autonomy, as part of the cocreated methodological guide sessions, motivated participants to express their personal views of the impact of dementia in persons’ daily and inclusive life. Within the same context, participants also provided valuable information about the impact of dementia in both the emotional and physical well-being and the possible barriers and supports needed for the implementation and preservation of Autonomy by PwD and their carers. 3.2 The co-design of the training program material A second objective of the AD-Autonomy co-creation sessions was the participatory design and determination, with and for end-users, of the training material -types of material, good practices, activities, ICT tools- as well as of the skills and key competences related to the implementation of the training program by PwD, their family caregivers and healthcare professionals. In order for the AD-Autonomy research team to develop the training program material and activities, based on the needs and experience of both PwD and caregivers, participants in the five pilot sites were asked to provide practical and empirical information about their everyday life activities, the possible barriers and how they find ways to cope with them, preserving the levels of their Autonomy and their QoL. In the framework of the first co-creation session, Greek MCIs highlighted the importance of developing and keeping habits and daily routines as key factors for maintaining their independence. On their point of view, professionals insisted on the prioritization of daily duties focusing on positive aspects, like what older adults can do without any support and which activities increase their self-confidence ("do easy tasks instead of spending a lot of time to something that you cannot figure how to do by oneself"). Both older adults and professionals recommended the adjustment of the everyday activities to the pace and extend that the PwD can effectively perform them, avoiding any comparisons to what they used to do in the past. Reminders, keeping notes, and to-do lists were promoted by all Greek participants as example strategies for scheduling and prioritizing daily duties, while also dealing with daily problemsolving. Professionals recommended that people with cognitive impairment should exercise their memory, following practical strategies, like trying to remember information as the shopping list, bringing into mind pictures of their indoor spaces, e.g their fridge, while older participants added the brain exercise by reading books and playing crosswords, puzzles. The feeling of anxiety, fear and discouragement by relatives were identified by PwD as the main reason of failure for developing the required Autonomy skills.


In addition to the proposed experiential strategies and good practices, participants in each pilot site were asked to provide their views on how existing apps and ICTs could act as assistive technological tools to support the levels of Autonomy of PwD, enhance their competences and QoL, as well as those of their caregivers. Greek participants appeared positive in such a possibility, stating that every kind of support for Autonomy is welcomed and admitting their basic knowledge and previous experience in using ICT tools and technologies -mainly smartphones and tablets- through their active participation in other Thess-AHALL’s research and co-creation. Moreover, participants characterized the use of ICT Tools/Assistive Technologies as valuable for the preservation of their Autonomy, the maintenance of their social interaction and communication levels and their daily life and health management. However, participants, older MCIs and professionals highlighted that training should be provided on an individual learning pace and on a continuous base, so as to sustain in the long term the benefits obtained by the use of technology. Older adult participants provided real-life context scenarios, where apps and ICTs could potentially act as complementary or substitute traditional methods and experiential strategies, used by PwD and their caregivers. Concerning the possible barriers and doubts on assistive technologies, except for the need for a long-lasting learning process, the Greek MCIs mentioned the complex design of existing apps and tools, as well as some ethical issues of using the Internet and ICTs, like misleading information and personal data use and protection. In the framework of the second round of co-creation sessions, Greek participants received feedback on the responses given by end-users in the rest pilot sites, concerning good practices and tools (e.g. alarms, smoke detectors, pill-boxes, key/object finders), thinking positively of integrating some of these additional tools to meet their own needs for maintaining the Autonomy levels. Besides the review of the Autonomy dimensions and the lists of tools and techniques proposed by the total number of end-users, the main objective of the second co-creation sessions was the participants from all the stakeholder groups (PwD/MCIs, relatives and healthcare professionals) to provide their views over the development of a training programme for Autonomy, as well as to express their requirements for both the training material and the e-Platform. As far as it concerns the Thess-AHALL participants, they found the idea of developing a training program innovative, very interesting and useful. Older participants positively commented on the co-creation of an e-Training Platform that would include and combine technological and non-technological tools and activities, which could help both PwD and caregivers to maintain the Autonomy levels and enhance the QoL. On their side, professionals and family caregivers appeared willing to attend the “AD- Autonomy� online training programme, while the older adult participants recognized their crucial role for the final design of the platform and the training material. All the Greek participants agreed that a complete training programme, based on the conclusions of the co-creation sessions, would be very helpful and an innovative approach for both patients and caregivers. Regarding the type and


content of the training material, both older participants and professionals recommended the training program be “everyday life” orientated and respond to patients needs and interests. MCIs insisted on the use of simple and practical activities, based on step-by-step guidelines, while they also highlighted the need for caregivers’ training on the platform, requesting for activities on their behaviour and patient towards persons with dementia or other memory impairments. Health professionals recommended the use of printed manuals; written instructions with screenshots of the presented tools and technologies and family caregivers suggested the use of multimedia, like informative videos, which combine visual and audio instructions on good practices and tools. Referring to the type of training tools and assistive technologies, older participants appeared willing to use easy- to-use programs and applications, which could help them manage daily routines, like online lists, alerts and medication reminders, electronic device control apps, GPS and tracking tools (like Google Maps and the public transportation timetable apps), social interaction apps (including Viber and Skype). 3.3 Setting up the e-Training Platform requirements The AD-Autonomy e-Training platform has the two following specific objectives: i) to operate as a repository of all the practical training materials and strategies required to train end-users on the importance of maintaining autonomy, within an environment of security and support and ii)to give an insight of how assistive technologies can help PwD and their carers to implement and increase Autonomy, providing tools, including existing apps and ICTs and online practical activities and self-evaluation tests. During the second co-creation sessions, participants were asked to express their opinion about the functional requirements and the kind of the information that the e- Platform should contain. In the Greek pilot site, Thess-AHALL all the Greek MCIs noted that it is an extra motivation for them to participate in the design of an e-Training platform, which could help not only their lives, but also the lives and the preservation of the Autonomy levels of many other people with memory complaints. MCI participants and their relatives sided for a flexible, in their own pace, online training program that would allow them to repeat the activities and use the tools as many times as they want. Family caregivers also appeared willing to use and show the platform to their relatives, suffering from memory problems. Both asked for clear instructions, as well as a user-friendly and simple online training environment, which will include as less as possible information, so as not to be “chaotic” and confusing for persons with dementia. Greek participants followed the common line for the information that should be included in the e-Training Platform, in relation to what the Training Program should contain. Specifically, they mentioned the need for simple written instructions, clear objectives of activities and tools, step-by-step videos and guidelines, repeated actions and not technologically advanced terms and training materials. Tools and activities should meet the need for lifelong learning and continuous training and become a daily life routine for participants, who will use them to enhance their or their patients’ Autonomy levels and competences. In


any case, Greek older adult participants and family caregivers asked for face-toface sessions, practical activities and repetition to be trained on how use the ePlatform.

4 Co-validation working sessions with end-users A new round of participatory co-validation sessions with end-users has started in each participating country since May 2019, a year after the co-creation sessions for the design of the training materials and the e-Training platform and is still ongoing, expected to finish by end of June 2019.In the meantime, the ADAutonomy project team developed the final material, considering the requirements, suggestions and limitations that all participants (n=135) provided for both the practical strategies and assistive technologies, as well as e-Training platform features and content. The five pilot sites invited the participants from the first two co-creation sessions, as well as new ones, persons with dementia or other memory and cognitive impairment issues, their relatives and healthcare professionals to attend three coworking sessions (Face-to-Face meetings, experiential workshops) and to have some online practice in the training platform, to test and assess the material and methodology of the training program. The initial goal was a minimum number of 40 participants (30 older adults with memory impairment and their family carers, as well as 10 professionals) to validate the training programme in each country, divided in four groups, with a balanced representation of all the different stakeholders. In Greece, a total number of 29 participants (3 male) -18 older adults with MCI or other memory complaints, 3 family caregivers 8 healthcare professionals, some of whom had the dual role of taking care PwD relatives-attended the co-validation sessions organised by Thess-AHALL. It is noted that the three out of the four groups with Greek end-users ran in the premises of Thess-AHALL, while one group ran with participants from the 11th Municipality of Thessaloniki “Open Care Centre for the Elderly� (KAPI). Participants of each group attended two Face-toFace sessions of three-hours duration each, breaks included. In the first Face-toFace session Thess-AHALL’s trainers presented the Autonomy dimension and the main objectives of the co-validation procedure, as well as asked for participants consent to be their first collaborators in testing the e-Training Platform. In addition to the introductory dimension, trainers presented one of the two main dimensions. The dimensions for each group were divided, as following (Figure 2):


Group A:

Group B:

Personal Hygiene

Security & Safety

Orientation & Navigation

Food & Housekeeping

Group C:

Group D:

Emotional Skills Finance

Figure 2. Final AD-Autonomy list of dimensions

The second Face-to-Face session included the presentation of the second main dimension and training to the use of the ePlatform, for end-users to be able to navigate and explore the material assigned to the dimensions they attended, as well as to access the rest at their own pace at home. These two sessions were organized as focus groups and participants were motivated to interact and discuss on their personal views, experiences or possible doubts and questions, so as the sessions not have the structure of a passive classroom with distinct roles (teacher- student). Therefore, all the meetings and workshops that held in Thess-AHALL’s premises were hosted in the Living Lab’s eHome (Bamidis, Konstantinidis, Billis, & Siountas, 2017), a specially designed room, similar to a living room, like a meeting with friends with coffee and treats in a cosy and welllit place (Figure 3).

Figure 3. Co-creation in the real-life setting of Thess-AHALL

The third meeting of each group had the form of an experiential workshop, during which end-users had the opportunity to try some ICTs and apps, related to their


assigned dimensions, as well as to have the experience of the practical implementation of acquired knowledge and strategies in real-context or simulated user-scenarios. In the closing of this third meeting all the participants completed a Likert-scale questionnaire, which differed from MCIs to family caregivers and professionals, assessing the materials, the online platform and the trainers of the program in terms of functionality, usability, content, type of proposed tools and strategies, concerning all the Autonomy dimensions. According to early, pre-compiled results, Greek participants appeared satisfied and appreciated the fact that their needs and opinions were not only heard, but also included in the development of the final training materials. Especially, the MCIs, who either attended the co-creation phase or have cooperated many times with Thess-AHALL in the framework of other research projects, expressed their appreciation of acting like partners and ambassadors of the AD-Autonomy project, representing the Greek end- users community. 4.1 Validating the co-created training materials Concerning the theoretical and experiential materials of the programme, the ADAutonomy research team implemented a similar approach for all the Autonomy dimensions. For each one out of the eight main and the one introductory dimension, the research team followed end-users’ request for written informative materials, regarding dementia and its impact on PwD and their caregivers’ everyday life aspects. A handbook, including all the necessary information, strategies, tools, and step-by-step instructions for their implementation, have been designed for each dimension. The handbooks mainly address to trainers, the people who would train the PwD, either professionals or family caregivers. A number of practical strategies for the enhancement of competences and skills of PwD and the support of caregivers and healthcare professionals have been developed, according to the good practices and personal experience that endusers shared during the co-creation sessions. The most common strategies, addressed in most of the Autonomy dimensions were use of lists and diaries for everyday duties (e.g. the shopping list, cooking recipes, health management, financial management), brain exercise activities (e.g. crosswords, reading, brain games),practical information for the adjustment of home as a secure place (e.g. furnishing, colours, signs and indoor lighting), social/communication skills and activities (e.g. scheduling of outdoor activities, reminders for calling family and friends on the phone, physical exercise, expression of emotions). The practical advice provided mainly by PwD and family caregivers has also constituted the base for the respective experiential activities of the dimensions. The user stories of participants from the five pilot sites, as well as their doubts and possible barriers on how to cope with everyday duties and needs were used as the raw material for building the real-context or simulation scenarios for the third round of the validation co-working sessions. Attending the experiential workshops, older adults with cognitive impairment acted similar to their everyday life (e.g. scheduling the weekly households in an individual plan/ organising medication in a pill-box), while family carers and professionals focused on the same issues, assisting the older participants with their activities (e.g. motivation


and encouragement to build up an effective household plan/ practice with older adults on how to use the pill-box). Non- technological tools and ICTs were demonstrated and used by participants, meeting the request of end-users for traditional methods (pill-box) and assistive technology alternatives (apps for medicine management). Explanatory videos, external links to additional information and sources, as well as repetition and combination of methods and tools for more than one dimensions were also adopted for the training materials. Regarding the multimedia sources and the apps which have been included in the programme, the AD-Autonomy research team focused on videos, for which the use of subtitles or narration were not necessary, and on apps, supporting many languages (e.g. the Google maps for navigation, viber/ whatsapp for communication) or apps, easy-to-use and supported by pictures and video examples for their use (e.g. sudoku and other brain games, money counting apps, emotion flash cards). 4.2 Validating the e-Training Platform The e-Training Platform ( (Figure 4) was validated in a two-step process by Greek participants, who attended a live demonstration and practice with the support of Thess-AHALL’s trainers, during the second Face-to-Face co-working session, as well as on their own pace online practice at home for those older adult participants, who had Internet connection and access to smartphones or tablets. Although the AD-Autonomy research team did not provide any written instructions for navigating the e-Platform, ThessAHALL’s approach on demonstrating the online training materials through a oneshot visit in the framework of the group meetings and the guidance for exploring the platform and the entire material at home has proved effective. Family caregivers, but mostly MCIs got familiar with the structure of the e-Platform, its main features and the type of the materials, expressing their satisfaction for unlimited use and self-paced training at home, either alone or with the help of their family. The platform included all the training materials presented and used for the Autonomy dimensions in the group meetings, like the informative material about dementia, the strategies/good practices, trainers’ handbooks, experiential activities and a full catalogue of the assistive technologies/ICT tools promoted for the maintenance of Autonomy. The training material has been organized according to the nine dimensions of the programme and split in small, but comprehensive pieces of information, using clear and simple language, not much text, pictures and videos, additional links and simple colours, as requested by the majority of end-users. The absence of login was helpful for older participants with memory problems, who perceived the training platform as a useful repository of all the necessary material and information for training themselves or other people on the Autonomy issue. At the same time, responding to end-user’s request for being part of this European initiative and be aware of how people from the rest participating countries deal with similar to their needs and problems, the e-Training Platform included a community forum.


Also, although this is for a European research project, there were no language issues, meeting the end-user’s expectations, for having the material and the platform translated in their native language. Moreover, in terms of navigation though dimensions, the e- Platform adopted both a linear and a non-linear approach, offering the opportunity to users for one-click access to the activities and materials of their interest, also providing the possibility of exclusively selection of either the non-technological strategies and experiential practices or the assistive technologies, ICTs and online tools.

Figure 4. The e-Training Platform home page

5 Conclusions The co-creation approach for the development of an innovative training programme for increasing and maintaining the Autonomy levels and QoL of PwD and their caregivers, led to a round of participatory sessions that provided useful information from different participants’ perspectives for both the importance of Autonomy and its main dimensions in everyday life, as well as for the design of the training material itself. The main contribution of this work is the involvement and active collaboration of Dementia Care multidisciplinary experts (both clinical and engineers) and endusers at the receiving end of care, such as older adults and their informal caregivers. Building on top of existing knowledge and experience, older adults and their caregivers attempted to juxtapose experts’ knowledge on their daily lives and routines. Self- reflection and real-life situations, allowed to work on examples and develop new strategies in a way that: are fit for the older adults’ real-life living environments, and maximize the adoption of assistive technologies at home settings. The AD-Autonomy research team took into consideration the majority of the needs, suggestions and expectations of end-users in the five involved in the project countries (Spain, Greece, Slovenia, Turkey, the UK) to design a complete


training programme and an online platform that include both experiential practical activities and assistive technologies for the maintenance and support of Autonomy of people, suffering from memory problems. At the same time, realizing their integral role, as members of a European-wide co- design initiative that could help other people with similar issues, Greek endusers, and especially older adults with MCI or memory complaints, appeared willing and felt comfortable to share their thoughts and personal experiences on everyday living with cognitive impairment and the preservation of Autonomy levels. Acting like a kind of ambassadors for the training programme, Greek participants also contributed to Thess- AHALL’s co-validation working sessions of both the training material and the e- Platform. They expressed their appreciation of consulting the initiative, as well as of their needs and requests having been counted for the final development of the programme.

Acknowledgements The work has been funded by the Erasmus+ Programme of the European Union under grant agreement no 2017-1-ES01-KA204-038608 as well as, it has been supported by members of the 11th Municipality of Thessaloniki “Open Care Centre for the Elderly” (KAPI) and members of the “Collaboration & Research Community for the Independent Living” of the Medical Physics Laboratory, Faculty of Health Sciences, School of Medicine, Aristotle University of Thessaloniki. For more details, please see and


References Bamidis, P. D., Antoniou, P., and Sidiropoulos, E. A., “Using simulations and experiential learning approaches to train careers of seniors,” Proc. - IEEE Symp. Comput. Med. Syst., pp. 119–124, 2014. Bamidis, P. D. , Fissler, P., Papageorgiou, S. G., Zilidou, V., Konstantinidis, E. I., Billis, A. S., Romanopoulou, E., Karagianni, M., Bearatis, I., Tsapanou, A., Tsilikopoulou, G., Grigoriadou, E., Ladas, A., Kyrillidou, A., Tsolaki, Α., Frantzidis, C., Sidiropoulos, E., Siountas, A., Matsi, S., Papatriantafyllou, J., Margioti, E., Nika, A., Schlee, W., Elbert, T., Tsolaki, M., Vivas, A. B. and Kolassa, I. T., “Gains in cognition through combined cognitive and physical training: The role of training dosage and severity of neurocognitive disorder,” Front. Aging Neurosci., vol. 7, no. JUL 2015. Bamidis, P. D. , Konstantinidis, E. I., Billis, A. S., and Siountas A., “From e-homes to living labs: founding and organising the Greek active and healthy ageing living lab (Thess-AHALL) and its networked services,” J. Hell. Soc. Nucl. Med., no. Supplement Proceedings of the 10th Panhellenic Interdisciplinary Conference on Alzheimer’s Disease and Related Disorders and 2nd Mediterranean Conference on Neurodegenerative Diseases, pp. 112–125, 2017. Billis A., Mantziari D., Zilidou V., Bamidis P.D., “Co-Creation of an Innovative Vocational Training Platform to Improve Autonomy in the Context of Alzheimer's Disease”, in Proceedings of the 16th International Conference on Informatics, Management and Technology in Healthcare (ICIMTH 2018), Athens, Greece, July 2018. Permalink: Moulaert, F., MacCallum, D., Mehmood, A. & Hamdouch, A. (2013). The international handbook on social innovation: collective action, social learning and transdisciplinary research. Cheltenham, UK: Edward Elgar Publishing. Terstriep, J., Kleverbeck, M., Deserti, A. & Rizzo, F. (2015). Comparative Report on Social Innovation across Europe. Deliverable D3.2 of the project «Boosting the Impact of SI in Europe through Economic Underpinnings» (SIMPACT), European Commission – 7th Framework Programme, Brussels: European Commission, DG Research & Innovation OECD/European Union (2018), "Health at A Glance: Europe 2018. State of Health In The EU Cycle" (Revised version: February 2019). Retrieved June 4, 2019, from nce_rep_en.pdf World Health Organization (2017) "Global action plan on the public health response to dementia 2017 - 2025"Retrieved June 5, 2019, from _2025/en/ World Health Organization (2019) "Adopting a healthy lifestyle helps reduce the risk of dementia". (n.d.). Retrieved June 5, 2019, from



World Health Organization (2019) "Dementia". (n.d.). Retrieved June 4, 2019, from



Responsible Design for an Older Future Gareth Priday1 and Sonja Pedell1

1 Future

Self and Design Living Lab, Swinburne University of technology, Australia Category: Full Research

Abstract In this research we focus on a Living Lab co-design process for the development of an ‘active ageing’ portal with a local council and the community as part of their WHO age-friendly city strategy plan. Narratives of ageing have been defined by models of deficit and decline where technology is meant to provide a solution; only recently notions of capabilities and individual interests as part of the technology provision are emerging and are coming to the foreground and getting more attention. We found that co- design acts as a protective process against generalised deficit ideas of ageing and to avoid more subtle ageist assumptions. We show that, on what may be considered a straightforward process and ICT product, co-design can play a significant role in preventing unintended ageism. The co-design process in our case provided a pathway to increased adoption and uncovered future opportunities for a Living Lab to play role in systemic change regarding the perception of active ageing that extends beyond the technology solution. Keywords: Active Ageing, Co-design, Ageism, Ageing World, Web design, Communities, Age-friendly cities


1 Promoting Active Ageing The world is getting older. The world health authority reports that the percentage of people over 60 will nearly rise from 15% in 2015 and 22% percent by 2050 (WHO, 2018). By the end of 2020 it is expected there will be more adults over 60 than children under 5 years old. Australia follows this trend with the dependency ratio (the number of dependants for every 100 workers) changes from 50% in 2013 to between 58-61% by 2033 (ABS, 2014). As a result, the pension age in many countries is changing; in Australia the pension age is moving from 65 years to 67 years by 2023. At the same time, we are extending people’s lives without necessarily improving the quality of life resulting in a long period of decline nor enable people being able to actively enjoy or contribute to society (Gore, Kingston, Johnson, Kirkwood and Jagger, 2018). Providing adequate health and social care for an aging population is a major economic change for governments at all levels. The greater number of older people economically dependent on fewer people of working age creates fiscal challenges for state and local governments (Productivity Commission, 2005). Rural local councils are at particular risk with higher rates to older people needing services that are more expensive in remote areas, while the rate paying base is diminishing owing to youth migration to the major cities. These changes drive different needs in community infrastructure and create the need to tackle multiple complex transitions at the same time often with competing agendas; for example, the transition to smart cities, resilient cities, age friendly cities and dementia friendly cities. As Lancet notes, "how to provide adequate health and social care to an ageing population is a major economic challenge for all countries, threatening to cripple already overstretched health and social care budgets in some" (Lancet, 2018 p.587). As a result of these drivers there has been a significant interest in promoting “successful ageing” and more recently, since WHO coined the term in the late 1990’s, “active ageing” (Bounty, 2012). Active Ageing has become a key policy driver (EU, 2012) with an associated Active Ageing Index (Zaidi, 2012). Active ageing and its predecessor “successful ageing” are part of a positive ageing narrative that attempts to counter negative attitudes towards ageing. Active ageing has become a significant area of interest for local councils (municipalities) in Australia as a mechanism to improve health of older adults and prevent unnecessary or premature decline. The objective is to improve quality of life through increased participation in physical, mental and social activities, consequently reducing the need for health care (especially the costs for chronic diseases) and increase the ability of older adults to participate actively in society. 1.1 Ageing and Design There is increasing research and criticism about the tendency in society and design to treat older adults as a homogenous group with similar needs that can be documented and designed with a one-fits-all approach (Lindley et al, 2008; Vines et al, 2015). There can be pressure for older adults to accept these norms and active hostility against older adults who openly disregard prescriptive norms


of behaviour. “Given the different types of age stereotypes and prejudices that older adults face, it is not surprising that age-based stigma can negatively affect them” (Chasteen & Cary, 2015, p.106). Self-adoption of negative stereotypes can also have negative impacts (González-Domínguez et al, 2018) which may result in non-use when concerned with inadequate technology. There is therefore a lost opportunity for these users to engage with technologies that could be in accordance with the way they like to see themselves and feel about ageing. Research shows that there are not only diverse pathways in ageing (Browning et al, 2018), but “Progresses in terms of longevity, healthy ageing and technological innovation have shaped today’s older people as a generation that actively contributes to both society and family” (Bordone et al, 2019 p.2). This should also be reflected in technology design. HCI designers are not immune to an old model of ageing either and were historically “concerned with the downside of aging; focusing on assistive technologies that can help compensate for people’s frailties and the assumed needs that arise when getting older” (Rogers et al, 2014 p. 3913). This often came with a view of older people as passive technology receivers, resulting in ICT that is produced with design that looks and feels as if it has been tailored to a disabled audience contributing to this suggestion and stereotyped elderly users as such (Renda & Kuys, 2013). This notion is now being challenged with the HCI field’s focus moving towards interests, cultural and leisure activities rather than decline; and design with older adults to mitigate against social ideas of ageing (Sayago, 2019). Durick et al (2013) suggest that “when designing for ‘older users’ we need, first and foremost, to regard them as ‘users’ who are defined by their specific and contextual needs, and not their age” and “to remain mindful of how embedded notions of what ageing means may limit our design thinking (p.473).” Mallenius (2007) similarly suggests design should not be focused on age at all, but functional capacity. 1.2 Living Labs and Wellbeing Living Lab approaches are well established for creating solutions for wellbeing particularly for older adults with about 30% of ENoLL members having a health and wellbeing interest and Australia having four living labs that are concerned with Ageing. The Future Self and Design Living Lab is an ENoLL and ALLIN (Australian Living Lab Innovation Network) member. The Lab has a focus on developing technology solutions using emotion led design to increase the quality of life with a focus on older adults. The Living Lab has success with the emotion led design as a key element of its co-creation approach to inform tailored technology development and increase adoption rates (Pedell et al, 2017).


1.3 Active Ageing Project Our project concerned the development and implementation of an ‘active ageing portal’ on a local council website. The Council is a foundation partner of the Future Self and Design Living Lab hosted by Swinburne University of Technology. The Council's main objective is to improve ageing outcomes for their constituents through greater involvement in physical, mental and social activities. The Council is approaching this aim as part of their age friendly city strategy in several ways: through the new website, peer to peer training, promotion with community activity providers (e.g. U3A, community houses) and health care providers. The success of the implementation for the Council will be measured by the number of people using the website and, over the longer-term, increased participation in activities by local providers. Most website design standards for ageing focus on adoptions for declining physical capabilities, such as eyesight changes resulting recommendations for increased contrast and font size (e.g. W3C, 2008). This is a vital part of website design to make the website accessible. However, these types of standards do not address the other elements of the design that make the website attractive to a wide range of people, with very different capabilities. The Council’s interest in local people adopting the website as a central portal for Active Ageing created an opportunity to investigate these critical elements for adoption in a more holistic manner. The investigation into the website design included language, visuals (aesthetics as well as photographs as representation of ageing), content, search interactions and hardware use as key elements of the investigation. Taking a Living Lab approach, we included other actors in the eco-system as part of the research this included council staff, doctors and aged care providers. This paper focuses on the results of our co- design research and the issues it uncovered.

2 Approach A prototype of the website portal had already been developed at the time that the Living Lab got engaged in the project. The core component of the site was a search function that returned activities available from local providers at no or low cost. Activity types included physical exercise (e.g. aqua aerobics), mental stimulation (e.g. language courses, creative classes) or more social activities (such as trips, social gatherings) and events (e.g. art exhibitions at the council hall). The Council has 17 centenarians within its borders The Council has a higher proportion of people post retirement age than other councils in the Melbourne Metropolitan Area. Three sessions including interview and prototype scenario-based walk through were conducted with each end-user participant and a focus group was run with other eco-system stakeholders (doctors and council staff). As is typical of this type of project the website development was ongoing. The Council was keen to have the early findings incorporated into the updated Website. An interim report was developed after the first 5 interviews of phase 1 and 2; consistent with literature where 5-6 users are sufficient uncover major needs and usability


problems. The prototype site supported wording and picture changes without developer intervention; updates were made to the site and included in the remaining phase 1 interviews. Updates of major findings during phase 2 were provided continuously to enable addressing these issues for final development (Figure 1.).

Figure 1. Overall participant consultation process

2.1 End -User Approach The research was based on the concept of co-designing a revised solution with volunteers sourced by The Council. The Council recruited volunteers for the enduser research from its networks. Participants input was acknowledged after the final session with a gift voucher for a local store. Interviews were scheduled for an hour each in their homes. The first two sessions focused on the initial prototype website and participants’ attitudes and understanding of active ageing walking through a set of use scenarios on the prototype as part of the session. These could have been performed as a single interview, but it was considered that an interview of two or more hours would be too long. An additional benefit was that participants often reported additional feedback on a prior section having had time to reflect on the first session between interviews. The first interview included questions about general health, technology use including hardware (iPad, mobile phone, computer), Internet usage and search engine type. It also covered questions about their understanding and attitudes towards active ageing. The remainder of the interview in the second interview protocol covered the major elements of the website including functionality, wording and visual elements mostly guided through typical use scenarios. Based on the initial findings a revised high-level prototype was developed. This included most of the critical and highly prioritised recommendations from the first two interviews. The third session focused on an updated Website version. Seven females and five males were taking part in the end-user study. Participants had an age range from 55 to 80 years. Two of the volunteers were acting on their own behalf and in the role of carer for their 90+ year old father who had vision impairment which was considered a common scenario for the Website use. Several of the participants were in semi-retirement. All the participants were still


driving, were living in their own home and were mobile, although one needed some support and could not walk far. An interview script (Figure 2) was developed and agreed upon with the Council. This included a standard introduction to the project, a review and signing the necessary consent forms. Consent procedures concerned the audio recording of the interview and an additional request for video recording for two of the volunteers. The short videos were made with two users of very different physical and technological capabilities for the development team to have a clear sense of the range of capabilities and the impact the design had on participants. The audio transcripts and notes were analysed using content analysis according to Patton (2002).

Figure 2. Session script extract

In all the sessions participants were encouraged to interact with the website in ways that were natural to them and explore the site as they saw fit. The guide with the scenarios was used as a checklist to ensure that all the functions were covered and for prompting questions about content, visuals and other aspects when they were not covered by the participants. As with many website developments there are more user requirements than budget and time to deliver them straight away. The final design was based on the high impact revisions; high being defined as based on number of users requesting the same or similar changes and considering them as important.

2.2 Focus Group Approach Older users’ data were complemented by a data collection with additional key stakeholders. A focus group discussion with council support staff, local age care providers and a doctor was conducted. The site was demonstrated to the users in the group session. A short guide was developed and agreed on with the council


and was used as a prompt for the group discussion. The discussion focused on the usefulness of the website for their organisations to support their clients. For these stakeholders the community members they supported were often more isolated and needed more health and social support from the council and community groups. The focus group also explored the key intervention points that the website needed to support as well as usability features.

3 Findings 3.1 End- User Interview Results 3.1.1 Headline and Written Content The site was originally headlined as “Active Ageing” (Figure 3). Although this might be a standard term in policy making and academic settings, it was unfamiliar to the participants. Most participants found active ageing to be a “marketing” term that they did not feel had relevance for them. This is illustrating that familiar policy language may still need to be introduced to the community. The alternative headline line “Add life to your years” was considered a cliché and a bit patronising by most of the participants. “Active ageing” was mostly associated with the idea of physical activity, nearly all participants added mental activity or learning or creative expression only thinking about it for longer. In the home page description participants found the “Find a range of activities to keep your body and brain active” clear and concise. Although, some participants noted that social elements were missing. Words like 'seniors', 'older adults', 'retirees' or similar were all firmly rejected. Not surprisingly and in accordance with literature outlined previously anything that was associated with the idea of being in some way infirm, incapable or needing support was rejected. The idea of "over 55s", the current name for the portal is problematic as it emphasises the age rather than capabilities. Several participants suggested that they would all qualify as “seniors” believing that they could apply for a senior’s government concession card in the states of Victoria, however, this only applies to people over 60 years of age. This highlights the difficulty of naming a group that the site is targeting without the intended audience, or parts of it, feeling disenfranchised. The longer description “Active ageing is about helping people realise their potential for physical, social, and mental well-being and participate in society, while providing them with adequate protection, security and care when they need it” was more generally disliked. Participants commented that it was marketing speak or overly academic” (Figure 3). Several felt that this description mixed up the concepts of active ageing and independent living. They questioned whether the idea to “...realise their potential...” as a retired person was important. They preferred clear, concise language with an emphasis on active and concise wording such as “meeting people”, “going to places” “fun”, and “enjoyment.”


Figure 3. Are they having fun? - Original home page

Much of the detailed written content participants liked in terms the brief descriptions of the events, news and other items. The feedback focused on highlighting key details like time, place, contact details. The events had more of the wording base on enjoyment lacking in the headline page for example “Ageless grace is a fun, seated, exercise program to uplifting music”. Participants were able to assess quickly if this was of interest to them. The wording of the activities showed similar results. A wheelchair-based exercise class, “Ageless grace” (fig. 4) was often listed first in the results in the initial prototype. One participant suggested that this was “more for people who might be in an aged care facility” and her interest was to find a new Zumba class. Although there were Zumba classes available, she indicated she'd already "think that the website wasn't for her" based on that first result. This was re-enforced as this same result appeared under physical, mindfulness and social categories. Other less able participants found this class appealing. This illustrated the difficulty of providing attractive offerings to such a wide range of ages and capabilities.


Figure 4. Search Results

3.1.2 Visual design and photographs The initial prototype website had very few visuals elements. Most participants made some reference to the overall aesthetic, as being “a bit vanilla”, “lacking colour and movement”, “boring” or noting that colour had been “banned”. All but two participants want more pictures as a short cut to understanding what activities were on offer and what they entailed. This was backed up by the participants comments about the pictures on the website which they used as a quick guide to the activities and provided the motivation to read the associated content or move on. They also indicated that the picture gave them cues as to whether they identified with the people who appeared in the picture. Participants noted that the pictures are part of the decision-making process giving a sense of the activity. It was important that any picture is aligned with the activity. One of the header pictures showed older adults at a farmer’s market (Figure 3). As one participant noted “I expect to see pictures that represent active-ageing activities” adding that pictures defined the term, and another said, “they all look really boring.” Like the language issues, participants were sensitive to any pictures that equated being an older adult with being in anyway infirm or incapable. The prototype site had links to the service provider sites. Participants noted that most of the other sites used pictures more usefully, but also pointed out where the picture and the activity did not align. For example, a cycle club where the bicycles and cyclists were not clearly shown on the picture, instead a Chinese dragon was shown to represent a place they had visited. Similarly, participants noted that the Seniorpreneurs group photo did not seem to include anyone over 55 (Figure 5).

Although all the participants were of European heritage, they were disparaging of photos that lacked diversity or showed clichés. For example, one early photo showed a group of older adults all Anglo- Saxon heritage, all a similar age range and in couples (Figure 3). Participants pointed to the lack of cultural diversity, the stereo type of male/female couples and sedate activity that had little relationship


to active ageing. Participants did not like having a male only and no female only listing in the main drop down. This was in part due to the obvious unfairness and that they wanted a principle of inclusion as they searched, the listing name e.g. “men’s shed” being enough in to determine gender specific events. Several participants noted that some in the LGTBIQ community may also feel discriminated against.

Figure 5. Picture and word mismatch

3.1.3 Search Interactions In the original prototype the search only had a dropdown list by category (e.g. physical, creative, etc) and then options to limit the search based on other criteria such as operating organisation type. There was no general search engine where key words could be used. While the intention was to make this an easier process, this was universally disliked. All participants had used Google search or an equivalent search engine before, so the limited search was not in keeping with their expectations. It created unintended consequences as noted above where items were listed alphabetically with the same results appearing in different categories. The only option was to page through several pages to view all the items. In the case of the participant interested in the Zumba class this would have taken some time which more generally is unsuitable for people who know what they are looking for. The revised prototype included a key word-based search engine that was universally preferred by the participants. From the second round of testing two types of search mechanisms emerged. Participants used the search function in directed and exploratory ways. Directed is a ‘Google style’ key word search that is used to quickly narrow down a search to a small set of items, e.g. “Cycling Club.” The second search was described by one participant as, “I don’t know what I’m looking for, but I’ll know it when I see it”. This style of searching was used to look for new things and find out what was available. One participant noted that this was especially important for people moving into a new area and who might be looking for ways to engage socially and need help to find out what is available in their community.


Other important aspects included that the advanced search also had an “apply” button to activate any selection. Owning to poor positioning of this button most participants waited for the results to apply automatically and quickly became frustrated. Similarly, lack of clear navigation to get back to the search start or home screen had the same result. 3.1.4 Content by Age group As noted above the detailed content generally was succinct, clear and the short descriptions were able to give participants enough information to decide whether to read the long descriptions or not. Participants did question the notion of segregation by age as one participant emphasised "I want to go to a philosophy lecture not a philosophy lecture talk for over 65’s”. Others questioned the idea of event curated for over 55's: what was the criteria for the event of news item being in or out of that grouping? Was it specifically only events that over 55s were eligible for, or events and news that someone thought appropriate for over 55s? The idea that it may only be items that over 55's were eligible for was very limiting. The idea that someone was deciding on appropriateness raised the question of on what basis these decisions were made. Some participants explained that they need to be aware of school holiday programs in their capacity as a grandparent, others so they knew when to avoid crowded libraries over the holidays. Many participants pointed out that capability across the spectrum of active ageing was more important than age itself. Pointing to 70+ year old people who were very fit and active compared to some younger people who were not. Another example included 60 to 90 years olds playing an Australian Stock Market game online as part of a Probus Club. Demonstrating technical know-how and the ability to make sophisticated decisions about stock picks, communicate online and follow stock related news while playing the trading game. 3.1.5 Technology Use Only one user had real trouble navigating websites and using the internet in general. Nearly all participants had PCs and most also had iPads. Nearly half of the participants conducted the interview session on an iPad or equivalent android tablet devices. Most had mobile phones but would not access the council's site using the mobile phone. Several of the participants had careers that included work in developing, testing or maintaining websites, or some other technology. Recruitment was on a volunteer basis and may have led to a technophile biased sample. However, it would seem to point to a transition between adults who did not use internet technologies in their working life and those that did. These participants were very comfortable using technology and able to point to very specific aspects of the design including, in one case, the misapplication of certain standards at a screen level where cursor movement using the tab key was incorrectly applied. This


being especially important for people who are visually impaired and are more likely to use tab keys than a mouse. 3.2 Focus Group Results The practitioners focus group consisted of eight participants from the Council, respite services, health and aged care providers and a retired GP. The session was about 50 minutes long and discussed the ways the portal and the Council’s active ageing agenda may intersect with their organisations and practices. This group tended to work with older people in the community who needed the most support. The idea of key intervention points emerged from a discussion of how the website could support their activities, shown below (Figure 6).

Figure 6. Key intervention points for older adult

At each of these points people’s social connections and the ability to maintain them changes. People would “reflect on their own wellbeing” and wonder “what they would do”. A second grouping of interventions was driven by their children (Figure 7), possibly triggered by the same crisis or by a perceived crisis. This expanded the potential audiences to children and the community groups represented who might be using the website and supporting resources that the Council provides. 3.3 Implementation A final report was drafted after the focus group and final round of interviews. The final high priority changes were made to the system which went live in August 2018. The websites visitor numbers doubled within the first month and the website was a finalist for the Municipalities Association of Victoria, Customer Achievement of the Year award.


Figure 7. Key intervention points from children

3.4 Recommendation Summary The final site was much more acceptable to the users and the overall process worked well and could from the basis of an activity package for similar developments (Figure 8). Given more time we would have tried to include more cultural variation to ensure the site was useful for an emerging demographic of users with different cultural backgrounds. Starting earlier in the project lifecycle we may have been able to incorporate more changes. While we considered the use of phones and iPads, we could have investigated changes in technology use in emerging demographics.

Figure 8. Suggested activity package for similar projects

4 Discussion 4.1 Emerging context


The average expected age of death in Australia is 84.6 years for males and 87.3 years for females. A website for over 55s would need to support active ageing for an average of 32 years. Similar statistics are observed globally. What is responsible design for an older future look like with such a range of ages, capabilities and the number of years where an individual is a target of the system? Lancet (2018) notes that there are increasing calls for ageing to be classified as a disease with WHO introducing a new category. Biologically there are no agreed markers or timing for transition to ageing (The Lancet Diabetes & Endocrinology, 2018). People subjective age can be very different to the actual age, often much lower and its increasingly common to see older adults performing feats that would have been unthinkable a generation ago. The image of a frail older adult who needs support and is technologically incapable is a strong one. By 2020 the internet will have been mainstreamed in developed countries for 22 years, smart phone available for 14 years and pads available for 7 years. Adults who will be turning 60 years of age would have spent a significant proportion of their working life with access to computers, smart phones and other technologies either in their workplace or at home. The assumption of technological incompetence for these types of technology will no longer be reasonable. This was certainly evident from our participants several of whom had worked in the computer industry for many years including coding, testing and content management. This target user group does not want to feel that the design is for an elderly person, rather the design is tailored to meet their interests and capabilities. Our results are consistent with the move to understanding older adults as a diverse grouping in both capabilities and interests. 4.2 Co-design against Ageism? The project illustrated the tensions between acknowledging the extra support that can be required as people age and the ease with which approaching this groups can be stigmatising and re-enforce ageist ideas. While we can understand the desire and potential benefits to reach out to such a community, even the act of doing so and defining a group by age in many ways is stigmatising already. It would seem odd to have a site targeting the 22 to 49-year olds for example. Demographic transitions and increasing life expectancies demand interventions that remove the notion of ageing as one point in time segregation or a cohort that can be managed based on age alone. We must move to a position of recognising that ageing does bring changing needs, but those changes are specific to the individual and highly contextual. There were some design elements such as the limited search engine, which suggested a deficit model in the early design process. However, with other elements such as the use of the term “Active Ageing”, it was not as immediately apparent that the participants would have a strong negative reaction. São José, Timonen, Amado, & Santos (2017) argue that seemingly new positive discourses such as ‘active ageing’ can pose problems. They suggest new discourses that ‘models’ ageing, becomes a model of old age; and that ‘active ageing’ suggests older adults have to behave in a certain way (being active) to become the “solution” to the perceived societal “problem” of ageing and longevity. As


previously noted, older adults who do not match this social idea of ageing may face a hostile reaction; we run the risk of replacing a deficit model of infirmity with an active ageing one. For most councils’ websites will remain the dominant technology they use to interact with their local populations. While a website re-design may seem at first glance a straightforward process, this project illustrates the impact co-design can still have to counter act bias towards normative ideas and assumptions about ageing that would have resulted in a poor adoption of the website when launched. It also enables other stakeholders around the older person to give input into the design process, picking up important elements of their use and transition points for clients. Design must focus on these contextual needs as a continuum, and as a result they must be addressed via processes of co-design within a framework that supports systemic innovation. Righi, Sayago, and Blat (2017, p26) propose a “turn to the community” that invites an adoption of “a situated lens on the social category of older people by looking at the communities in which they engage on the basis of their interests, skills, needs, goals, self-identities, and contingencies of daily life and life transitions.”. This model fits with our research findings. The co-design with community members reveals these changing patterns of understanding and context, to which designers must respond to. The focus group also highlights other pathways and moments in time that are important for the site to capture. This remains largely unexplored and would be an interesting area of future research. Transitions such as bereavement and retirement were noted in the focus group as key points to capture people as they make new decisions about their life. These are opportunities to provide pathways to activities and social connection are vital to enable people to create their own goals for ageing well that will have a long-term impact. Similarly, other potential users such as family members, council staff and other carers are important stakeholders who can encourage participation at these transition points.

5 Conclusion While we speak in this paper of older users, the defining characteristic that drives the design that was successful by the participants was the one that was appealing to a wide range of capabilities, interests and ages. The main purpose of the developed Website is to attract older adults and connect them to suitable activities, irrespective of their technological capabilities or age. Co-design enables designers and end-users to bypass stigmatisation by exploring the values, attitudes, life experiences and ambitions of older people and support these through design rather than focusing primarily on their physical or cognitive limitations (see also Edlin-White et al, 2012; Vines et al, 2012). Co-design then takes a positive stance, in the context of older adults, as being competent regarding their own needs, a value neutral stance on what those needs might be and a many pathway stance recognising the variety within a community. A Living Lab provides a larger mechanism to explore this variety and the systemic causes of ageism and exclusion in a community. These maybe


simple opportunities. However, one participant noted the car parking spaces near community houses were shorter in their time restrictions than the length of most of the classes offered. Similarly, making the link between the website, actual changes in activity uptake and long-term health and happiness are worthy of further research.

References Australian Social Trends, 2014, Australian Bureau of Statistics, 82014 Bordone, V., Arpino, B., & Rosina, A. (2019). Forever young? An analysis of the factors influencing perceptions of ageing. Ageing and Society, (May), 1– 25. Bounty, K. (2012). ‘Active ageing’: from empty rhetoric to effective policy tool. Ageing and Society, (July 2012), 1–22. Browning, C. J., Enticott, J. C., Thomas, S. A., & Kendig, H. (2018). Trajectories of ageing well among older Australians: A 16-year longitudinal study. Ageing and Society, 38(8), 1581–1602. Chasteen, A. L., & Cary, L. A. (2015). Connections to Research on Subjective Aging. Annual Review of Gerontology and Geriatrics, 35, 99–120. Durick, J., Robertson, T., Brereton, M., Vetere, F., & Nansen, B. (2013). Dispelling ageing myths in technology design. In OzCHI ’13 Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation (pp. 467–476). European Commission, 2012, “Guiding Principles for Active Ageing and Solidarity between Generations”, 12%20INIT González-Domínguez, S., Muñoz, M., Ausín, B., Castellanos, M. A., & PérezSantos, E. (2018). Age-related self-stigma of people over 65 years old: adaptation of the Internalized Stigma of Mental Illness scale (ISMI) for use in age-related self-stigma (IS65+) in a Spanish sample. Aging and Mental Health, 22(2), 250–256. Gore, P., Kingston, A., Johnson, G., Kirkwood, T. and Jagger, C. (2018). New horizons in the compression of functional decline. Age and Ageing, 47(6), pp.764-768. Lancet Diabetes and Endocrinology, 2018, Opening the door to treating ageing as a disease,


Lindley, Sian; Harper, R. S. A. (2008). Designing for elders: Exploring the complexity of relationships in later life. Proceedings of the 22nd Annual Conference of the British HCI Group (HCI 2008), 1, 77–86. Retrieved from Productivity Commission 2005, Economic Implications of an Ageing Australia, Research Report, Canberra. Patton, M.Q. (2002). Qualitative research and evaluation methods (3rd edition). Thousand Oaks, CA: Sage Publications. Pedell, S., Keirnan, A., Priday, G., Miller, T., Mendoza, A., Lopez-Lorca, A., & Sterling, L. (2017). Methods for Supporting Older Users in Communicating Their Emotions at Different Phases of a Living Lab Project. Technology Innovation Management Review, 7(2), 7–19. Righi, V., Sayago, S., & Blat, J. (2017). When we talk about older people in HCI, who are we talking about? Towards a ‘turn to community’ in the design of technologies for a growing ageing population. International Journal of Human Computer Studies, 108(June), 15–31. Rogers, Y, Paay, J, Brereton, M, Vaisutis, K, Marsden, G, Vetere, K. (2014). Never Too Old : Engaging Retired People Investing the Futues with MaKey Makey. In CHI 2014 (Vol. 1, p. 183). São José, J. M. de, Timonen, V., Amado, C. A. F., & Santos, S. P. (2017). A critique of the Active Ageing Index. Journal of Aging Studies, 40, 49–56. Sayago, S. (2019). Perspectives on Human-Computer Interaction Research with Older People,. Springer International Publishing,. The Lancet Diabetes & Endocrinology. (2018). Opening the door to treating ageing as a disease. The Lancet Diabetes and Endocrinology, 6(8), 587. Vines, J., Pritchard, G., Wright, P., Olivier, P., & Brittain, K. (2015). An Age-Old Problem : Examining the Discourses of Ageing in HCI and Strategies for Future Research. Tochi, 22(1), 1–27. W3C, 2008, Web Accessibility for Older Users : A literature review, World Health Organisation (WHO), 2018, Ageing and Health, Key Facts, Zaidi, A., Gasior, K., Hofmarcher, M.M., Lelkes, O., Marin, B., Rodrigues, R., Schmidt, A., Vanhuysse, P. and Zolyomi, E., 2013. Active Ageing Index 2012: Concept, Methodology and Final Results. EC/UNECE, Active Ageing Index Project, UNECE Grant ECE/GC/2012/003. European Centre for Social Welfare Policy and Research, Vienna.




A Case Study of a Living Lab through a Bus Improvement Committee in the Yeongjong are of Incheon City

Min-ho Suh*1, Junghyun Park1, Minki Kim1 and Won-Kyun Joo1

1 Korea

*Corresponding author Institute of Science and Technology Information (KISTI), South Korea

Category: Innovation Paper Abstract The concept of the living lab has been developed as a social problem-solving tool and recently it is being reexamined as a social problem-solving tool for disasters and urban problems based on the social contributions of science and technology. In terms of science and technology, the data age, represented by the fourth industrial revolution, is emphasized such that the living lab is now based on the support of data analysis and ICT technology. The purpose of this study is to examine the planning, operation, and meaning of ICT support by means of a database analysis of a living lab case of a bus improvement committee in the Yeongjong area of Incheon, Korea. Based on the data analysis, it was found that evidence-based rational discussions among citizen members as well as the civil servants enable the living lab to operate on a more groundbased basis and facilitate problem solving. By systematizing the data analysis technology, it is expected that the effects of the living lab will accelerate as city officials and civic organizations share important information. Keywords: Transportation, Living Lab, Bus, Data Analysis


1 Introduction Living Labs is a rather new research area and phenomena that introduces new ways of managing innovation processes (Bergvall-Kareborn et al., 2009). In this paper, living labs can be considered as a tool that can accelerate the innovation process (Bergvall-Kareborn and Stahlbrost, 2009) even though living labs have been interpreted differently such as an environment, approach, etc. in many cases (Wolfert et al., 2010; Schuurman et al., 2011; Ruijer and Meijer, 2019; Chroneer et al., 2019). In recent years, Korea has emphasized the use of living-lab tools in solving the problems associated with safe living of people using science and technology, and promoting smart cities. The Ministry of Science and ICT announced in May of 2018 that the second plan for the national public safety issue would illustrate the use of living labs as a demonstration tool for related developed technologies (National Science Advisory Council, 2018). Moreover, with regard to the smart city plan, the concept of a living lab is also considered to be important (Fourth Industrial Revolution Commission, 2018). Although the concept and promotion of living labs have long been in existence, the concept and application of a living lab is emphasized again in science and technology applied fields, which are closely related to current civic life. The reexamination of artificial intelligence technology (Google) following the fourth industrial revolution and the conversion to a data-based science and technology research paradigm must be global, and this is taking place not only in Korea but also around the world at present. It is expected that the data-driven technology that will be used as the digital lubricant of the fourth industrial revolution (Schwab, 2016) will be combined with artificial intelligence and that other applications will be the core of national competitiveness in the future. KISTI, the Korea Institute of Science and Technology Information, is a research institute concentrating on work in the fields of science and technology information and data and supercomputing as a government-funded research institute. KISTI has been providing services such as science and technology information services, super computing services, and science and technology information analysis to the Korean scientific and technological community and has been producing and spreading public technology mainly through projects entrusted by the central government. However, in order to respond to the recent social expectations of science and technology, public life is emphasized by the current government, as are the expectations of contributing to public science and technology via a disaster response system, diffusion predictions, earthquake predictions, fine dust predictions, and traffic problem solving efforts. In terms of beneficiaries, it is expanding the scope of its applications by providing support for direct problem solving using actual data from local governments KISTI has signed a MOU with Incheon City, the fourth largest city in Korea. Incheon City provides actual data from the city, and KISTI has developed a data analysis solution that can be used by Incheon City to develop data analysis solutions in the four fields of water immersion, earthquakes, traffic, and fine dust. Specifically, in the transportation problem solving sector, we are developing solutions with the


goal of optimizing data bus routes, providing bus demand analyses, simulationbased feasibility analyses, and route-optimization calculations. The Yeongjong area is an island that accounts for approximately 10% of bus traffic demand in Incheon. It is connected to the mainland, and a bridge to the Incheon International Airport is nearby, as are rural areas. There are many complaints about the bus routes, where there are various characteristics. Incheon City has decided to run a living lab with citizens, municipal councilors, police, city officers, and other interested parties to improve the bus route operation in the Yeongjong area. In this living lab, KISTI participated in the data analysis, contributing to the living lab and gaining experience through a pilot study of the Yeongjong area in developing an overall traffic solution for Incheon City. In this paper, by presenting data obtained from the execution of the transportation living lab project of Incheon City, the contribution of the data analysis by KISTI, the progress of lessons and learning and of future plans is detailed. As data-based living lab progress knowledge, the author makes this contribution.

2 Planning The planning meeting for the Incheon City Living Lab started with lectures on living labs aimed at sharing definitions and examples of living labs, and forming a consensus of city officials about the necessity of utilizing the living lab tool for Incheon City. In the city administration, considering citizen participation is a major burden for city government officials. Although citizens' complaints about city officials are important clues to identify and take action to solve administrative problems, and efforts to initialize civic responsibilities are substantial, direct contact with citizens requires some courage out of concerns for city officials. Nevertheless, the effect of civic participation is significant. The use of a living lab is not only a source of direct idea acquisition but also a global trend, and it has had a promotional effect on the interactivity of city administrations (as opposed to one direction of administration). Building a consensus on the need for city officials in a coherent direction is not an easy task, and strong initiative from top government officials is also important. The lecturer for the planning meeting is a specialist who has provided advice on various living labs. She focused on the success factors of a living lab, emphasizing that one important factor is social economic organization, which represents citizens and the participation of chairman-level members of representative civic groups. Hence, a living lab can be successful with the center of ordinary citizen volunteers, but should also be able to share conclusions with professional knowledge. The living lab should be operated with public-interestbased communication to achieve successful goals. In the end, the living lab requires participants with good qualities. On the other hand, the pursuit for real benefits is more important than pursuing an exhibition of city administration. Fortunately, the main decision-makers among the city officials ordered an efficient operating system that would make


the citizens not only gain publicity but also allow them to make substantial progress toward this goal. First, it should be emphasized that living labs find and solve complicated social problems such as those experienced by Incheon City through the direct use of existing expert networks, ICT technology and citizen ideas. This was interpreted as emphasizing the efficient operation of a living lab by strengthening the use of existing expert networks. Second, social problems should be divided into individual problems and meetings, and living labs should be operated intensively. In other words, specialists and civil joint committees that are specially designed for each problem would be effective. Third, it was advised to utilize big data analysis technology and to utilize the support of KISTI. Empathy and momentum regarding the direction of this top decision-making group have become the main directions of the promotion of the Incheon City living lab. Many other experts have provided advice on the successful start of the Incheon Living Lab, and below we present several noteworthy examples. There was an opinion that it is important to find problems inherent in Incheon City itself. In other words, the idea of solving the problem using citizens' creative solutions was presented. Recognition and discovery of problems unique to the region marked a successful start of the living lab. There was also an opinion that emphasized the role of the private enterprise sector. This is very meaningful advice because it emphasizes the participation and role of private enterprises from the beginning, as the final realization is carried out by a private enterprise, regardless of the hard work and solution development by the public domain. Finally, occasionally we will face situations in which the opinions of the municipal government and citizens conflict. When it is necessary to discuss the issue, emphasis is on the involvement of citizen representatives who have been mentioned before with regard to the public interest. It was the responsibility of responsible citizens to ensure that the living lab does not become a tool for individual citizens to pursue their own interests. These preliminary discussions have helped Incheon City to plan for the use of the concept of the initial living lab as a customized tool for Incheon City. Among the characteristics of Incheon City living labs are the will to utilize science and technology actively and the emphasis on citizen participation that considers public interests. There are several distinctive strategies related to these characteristics. First, they are actively seeking help from big data analysis solutions. Existing living labs were using technology as a means of realizing the improvement ideas of ordinary citizens. However, this case emphasizes the use of scientific technology, especially big data analysis techniques, in helping to find solutions and to discuss various ideas of citizens. Second, the plan is to utilize living labs for the purpose of demonstrating data analysis solutions and acquiring ideas for developing services that utilize them. This emphasizes the intention to service and systematize the knowledge gained when solving the problem. It considers the enhancement and diffusion of public technology developed through living labs. Finally, they will still utilize the existing expert network, which has already been emphasized in the directional orientation as previously discussed. Even if citizen ideas are emphasized, the use of existing expert networks will play a major role.


In this section, I explained how Incheon City planned the living lab and how the living lab was implemented with some directions and strategies. It was emphasized that this effort differs from existing citizen voluntary small living labs, and it is thought that this approach can be used as a reference when planning the promotion of living labs initiated by the public.

3 Methodology Different living labs are being organized for each problem in Incheon City, and the field will expand in the future. In particular, the transportation sector was considered first because traffic issues are very popular with the general public, and KISTI also considered an area where it wanted to develop data analysis solutions. In Incheon, traffic jams are an important issue in the Yeongjong area, and public officials considered an improvement committee that included citizens. This marked the beginning and most difficult mission of the living lab, i.e., the participation of the three main actors of civil servants, citizens, and experts, all of whom are important among the members of a living lab. With such preparations moderately ready, the project was launched in December of 2018. The Yeongjong Bus Improvement Committee Living Lab was constructed and operated due to the rapid increase in the resident population and the expansion of the Incheon International Airport (the opening of Terminal 2). New large-scale apartment complexes are being constructed and residents are continuously demanding improvements. It is planned to continue to operate during the reorganization of the bus routes in the Yeongjong area. A total of 25 members composed of city officials, experts, representatives of bus transportation companies, civic groups, resident representatives, police, and city councilors comprise this committee. The committee is held in principle once a quarter, though it is also held whenever necessary. The main roles of the committee are defined as follows. First, consultations about the coordination of local buses and public buses in the Yeongjong area are held. Second, they collect opinions from Incheon International Airport about linking and transit routes. Third, they prepare an efficient route adjustment plan for the Yeongjong area. The first meeting mainly focused on understanding members' professions and roles, sharing thoughts about the traffic situation in the Yeongjong area and devising improvement directions. City officials attempted to improve their understanding of participating citizens and to reach a citizen consensus on the situation of city financial support through an explanation of the Semi-Public Bus Operating System, which was introduced in Korea in large cities in the 2000s. In addition, there were plans to consider the dispatching interval of bus circulation routes, an issue of high demand from residents. Moreover, flexible residential times and the creation of a new bus parking lot in the Yeongjong area were presented. For new large residential apartment areas, bus route adjustments are being considered. In addition, there are a number of buses connecting the Seoul Subway and the Incheon Metropolitan Subway, and city officials announced the opening of a number of issues, including the activation of public buses and the discovery of several tourist routes. In the first gathering of the living lab members, it was very important to understand and share the issues with each other.


4 Results and Discussion A total of ten route reorganization plans were announced at the second Yeongjong Bus Improvement Committee held in March of 2019, two of which were already in operation. Through the committee, city officials explained in detail the advantages and disadvantages citizens would receive through the changes in the routes, and they listened to the opinions of citizens. Most of the cases involved routes that had started to move into the new apartment complex, with changing of existing routes. In such cases, the main concern of residents representing the excluded station areas was the possibility of the presence of alternative routes and the inconvenience caused by the changes. Most resident representatives were already aware that residents' route change demands with regard to new, large apartment complexes should be resolved through some adjustment of the given number of routes and vehicles. However, the resident representatives had to appeal to the committee as much as possible about the discomfort of their neighbors that would arise due to such changes. The traditional communication method between city officials and citizens when discussing transportation affairs was that the city official explained the changes to the residents without clearly explaining the benefits and losses related to these adjustments and attempted to persuade citizens with ambiguous explanations, such as the total cost of the city bus grant. In addition, it is difficult for citizens to grasp how much profit is obtained through such adjustments, and it is an outcome which cannot be obtained with only a slight sacrifice. This fact provides a good indication of the main performance implications of a data analysis. Gains and losses pertaining to the change plan are reported in real time or on the basis of the meeting, indicating that the meeting must be held while the city officials and the citizen representatives are sharing ideas. Below is an example of the feasibility study of the Yeongjong area bus route improvement plan by the city government officials on the basis of a data analysis. These results were not revealed to citizens in the second living lab directly, but this incurred a considerable amount of controversy, such as in the development of the forms, meaning that the public officials and KISTI should induce public debate such that there are no misunderstandings after much discussion. For example, as shown in Figure 1, the plan from the city official is to change the 203-bus line in the Yeongjong area to satisfy the bus usage demands of newly constructed large-scale apartment complexes.


Nubdi : 203,307, 598,304

: 203, 307


Nubdi : 203, 307,

: 203, 307


new Apt.

Figure 1: Route change plan example of line No. 203 in Yeongjong area

The important items about the 203-route change plan are the exclusion of the 'Unseo Elem. School' stop and the route to the new apartment complex stop. Therefore, considering the addition of a new apartment complex stop to such a change, it is necessary to re-evaluate it. How should one measure and judge the exclusion effect of the 'Unseo Elem. School'? For this feasibility analysis, the following analysis is required, and it can be done based on user data through traffic usage cards. As shown in Table 1, the Unseo Elementary School (35-270) is bus stop of No. 203, which is the fifth bus stop, and the ride share of No. 203 on this stop is 31.01%. Unseo Elementary School (35-270) does not have a large portion of the total number of departing passengers at No. 203, and there are many passengers departing from other lines (No. 598 and No. 304). Particularly, approximately 48% (453/944 people) of the passengers traveling to the Incheon International Airport from Unseo Elementary School (35-270) account for 90.96% of the O-D demand. The remaining part of the O-D demand is covered by No. 203 (17-minute intervals) and No. 307 (36-minute intervals). Therefore, an alternative means to the Incheon International Airport should be considered as important

Table 1: Status of Unseo Elem. School (35-270) Bus Stop


Line Ride Ranking of this stop on this line Taking off Ranking of this stop on this line Top 3 arriving stop from this stop O-D


Excluded Stop

5th (944)

Ride Sharing of this line on this stop (%)

22th (141)

Taking off Sharing of this line on this stop (%)

Unseo Elem. School (35-270, to airport) 31.01


Incheon Airport T2

Unseo Station

Lotte Mart




This line dependency of this O-D (%)

As presented in Table 2, at the Unseo Elementary School (35-282) stop, there are not many passengers of but No. 203, and there are many passengers who ride other routes (No. 598). Unseo Elementary School (35-282) is the fifth-ranked bus stop for No. 203, and the departure share of No. 203 at this stop in comparison to other lines is 36.44%. Particularly, about 32% (228/707) of the passengers who travel from Incheon Airport T2 to Unseo Elementary School (35282) account for 86.69% of the corresponding O-D demand. No. 307 (36-minute intervals) covers the remaining 13.31%. Hence, an alternative means of coming from the Incheon International Airport should be considered as important. In summary, passengers (average 113.5 persons/day) on the "Unseo Elementary School - Incheon Airport T2" O-D using the existing No. 203 were required to take a six-minute walk (400 meters) when using the alternative “Nub-di� station. However, the merits of enhancing the new apartment passenger demand should be compared with the lack of an Unseo Elem. School stop in the near future. KISTI reported the results of the above analysis pertaining to ten route change plans announced by the second Yeongjong Bus Improvement Committee Living Lab, and the city officials agreed with the results of the data analysis and hope to refer to it. However, it is emphasized that the final route decision should take into consideration not only the data analysis results but also numerous practical considerations. However, the results of the feasibility study based on the data analysis are important criteria because they can be the basis of one objective discussion, showing that data that can support a living lab and demonstrating a type of relevant ICT support Table 2: Status of Unseo Elem. School (35-282) Bus Stop




Ride Ranking of this stop on 24th(142) this line Taking off Ranking of this stop 5th(707) on this line Top 3 departing Incheon Ariport T2 stop to this stop O-D 228

Excluded Stop

Unseo Elem. School (35-282, To Skycity)

Ride Sharing of this line on this stop (%)


Taking off Sharing of this line on this stop (%)


Unseo Station

KT airport branch



This line dependency of this OD (%)

In this case study, KISTI supported the Incheon city government's data analysisbased route improvement review and then decided to develop a traffic living lab into a citizen-led idea discussion method. This study is ongoing, and it is likely to take a more developed direction through cooperation with citizens and city officials in the future. As a short-term goal, the third (scheduled to be held in June of 2019) committee plans to announce the improved route plan by Incheon City, with urging by KISTI to refer to the data analysis results actively. In addition, a conference meeting will be held during the second half of 2019 to encourage citizen-led improvement ideas to be discussed in the living lab. At the same time, ICT support will be provided for real-time analysis so that residents can confirm the effects of the route improvement during the meeting.

5 Conclusions This study deals with the contributions of science and technology in solving a national life safety problem in Korea, mainly the application of data-based technology. Particularly in the case of transportation living labs, the significance of data-based technologies, which can promote living labs, the process of persuading stakeholders, and the methodological directions of those data-based technologies, can facilitate the efforts of living lab operations to share knowledge. Discussions on improvements based on clear facts during social problem-solving discussions are very important, and in this case, it can be said that science technology, especially data analysis technology, supports these efforts. This is meaningful because it is a clear example of the possibility of leading a datafoundation- based administration and citizen debates based on data when attempting to solve social problems. In addition, it is important to reaffirm that the support of civic-oriented living labs is important, but the participation of social organizations, civil servants, the police, and municipal councillors is also crucial. The public use of public administration data and the development of analytical


technology that fills the gap between data and citizens can lead to the development of living labs in the data era. In this sense, in the future it can be expected that these discussions will be more active regarding the environment, where researchers are expected to use data actively to understand each other

Acknowledgements This research was supported by Ministry of Science and ICT, Republic of Korea (Project No. K-19- L05-C01-S01). Also, we would like to thank Incheon City for local data support and practical opinions.

References 4th Industrial Revolution Commission. (2018, January). Smart City Strategy for City Revolution and Future Creative Engine. Bergvall-Kareborn, B., Hoist, M., and Stahlbrost, A. (2009, January). Concept Design with a Living Lab Approach. In Proceedings of the 42nd Hawaii International Conference on System Sciences, IEEE. Bergvall-Kareborn, B. and Stahlbrost, A. (2009). Living lab: an open and citizencentric approach for innovation. International Journal of Innovation and Regional Development, Vol. 1, pp356-370, DiVA. Chroneer, D., Stahlbrost, A., and Habibipour, A. (2019, March). Urban living labs: Towards an Integrated Understanding of their Key Components. Technology Innovation Management Review, Vol. 9, pp50-62, Carleton University. Google., retrieved at April 1st, 2019. National Science Advisory Council. (2018. June). The second science and technology-based national life (society) comprehensive problem solving plan ('18 ~ '22). Ruijer, E. and Meijer, A. (2019, February). Open government data as an Innovation Process: Lessons from a living lab experiment. Public Performance & Management Review, published online: 15 Feb 2019, Taylor&Francis Online. Schuurman, D., Moor, K.D., Marez, L.D., and Evens, T. (2011). A living lab research approach for mobile TV. Telematics and Informatics, Vol. 28, pp271-282, Elsevier. Schwab, K. (2016, January). revolution-what-it-means-and-how-to-respond/, retrieved at April 1st, 2019 Wolfert, J., Verdouw C.N., Verloop, C.M., and Beulens, A.J.M. (2010, March). Organizing information integration in agri-food – A method based on a service-oriented architecture and living lab approach. Computers and Electronics in Agriculture, Vol. 70, pp389-405, Elsevier.



Adapting the Urban Living Lab approach to marginal contexts and urban regeneration: the case of Mapping San Siro Lab Francesca Cognetti1 and Elena Maranghi1

1 Department

of Architecture and Urban Studies – Politecnico di Milano, Italy Category: Innovation Paper

Abstract The aim of the following paper is to offer a reflection on the role and the characteristics that an Urban Living Lab could assume in processes of urban regeneration of marginal and deprived contexts of contemporary cities. The paper analyses Mapping San Siro case study: an on-going University-promoted Lab, opened in 2013 in one of the biggest Milanese public housing neighbourhoods (Milan, Italy). After briefly introducing the potential of Urban Living Labs in urban regeneration, the authors will contextualize the case study within the approach, highlighting the most significant points of contact. To conclude, they will open up the reflection on critical points to be considered when orienting an Urban Living Lab to the local development of a marginal context. Keywords: Living Lab, Transportation, ICT Tools, Big data, GIS, Decisionmaking

** Although the paper is the result of a collective work of the two authors, paragraphs 1, 2 and 5 should be attributed to Elena Maranghi; paragraphs 3 and 4 to Francesca Cognetti.


1 Introduction In the last decades, in the Italian context, the topic of urban regeneration and local development of marginal and peripheral areas has expanded its boundaries and meanings, becoming a more and more complex arena, characterized by the presence of very diverse actors. Increasing importance was especially gained by citizens and local organizations, considered able to interpret and bring out local resources and competencies, essential to develop successful interventions, especially in an era of scarce availability of funds. In Italy, the concept of participation – as applied in the Nineties to urban regeneration – was progressively overcome by the ones of social activation and social innovation: notions that are questioning the mere information and consultation to which participation risked to be confined in the widespread institutional approach to urban policies. As a result, nowadays, local communities of practice (Wenger, 1998) are assuming a more and more active and direct role in the transformation of their own environment and in the fulfilment of their social and cultural needs. But what happens to especially fragile communities or populations, not always able to promote or actively participate in local development processes? It appears urgent for engaged researchers and practitioners in urban regeneration to question themselves on the development of effective tools that could support the transformation of local citizens and organizations in truly empowered actors, able to promote and “control” changes, but also to claim for effective institutional support.

2 SoHoLab project: questioning Urban Living Lab methodologies in marginal contex Currently, it can be observed, indeed, how marginal territories suffer from a lack of chances in terms of bottom-up promoted regeneration. On the one hand, because of the difficulties for especially fragile populations to access to certain languages or tools (among other factors because of a substantial digital divide); on the other hand, because of a widespread sense of distrust, generated by the perceived “absence” of competent institutions, which seem to be no longer able to promote effective policies in such contexts. As a result, the process of exclusion of these territories is currently worsening. In the last years, applying the Urban Living Lab (ULL) approach to urban regeneration has been seen as promising to tackle this issue: existing literature has especially underlined their potentiality in terms of transformation and enlargement of urban governance and of empowerment of the different social actors (Concilio, 2016; Steen & van Bueren, 2017; Nesti, 2018; Naumann et al., 2018). ULLs are described, indeed, as “cross-boundary objects/arenas” and knowledge creative contexts (Concilio, ibid.), able to connect stakeholders and relevant actors at different levels (institutions and the so-called “users”: here, citizens, dwellers, communities); and, at the same time, capable of fostering social innovation (Naumann et al., ibid.) through the emphasis given to co-design and co-creation, as elements able to promote a really participated change.


Indeed, as several authors have pointed out so far (Hakkarainen & Hyysalo, 2013; Concilio & De Bonis, 2012; Franz, 2015; Ståhlbröst et al., 2018), even if considered to be valuable tools in developing smart and innovation strategies in the urban context, the academic debate on ULLs in urban studies still remains underdeveloped and unclear, especially when coming to a more socially-centred orientation of this approach (Franz, ibid.). Moreover, so far ULLs have rarely addressed deprived and marginalized contexts and superdiverse (Vertovec, 2007) communities, characterized by severe conditions of social, cultural and economic exclusion. Started in 2017, the SoHoLab Project “The Regeneration of Large-scale Social Housing Estates through Living Labs”7 has, indeed, the aim to establish and evaluate how ULLs could deal with the regeneration of social housing complexes, focusing on the role of University-promoted Labs8 and aiming «to develop an action research to effectively deal with the social-spatial exclusion of residents in underprivileged large-scale social housing estates in Europe through a Living Lab approach9».

3 The case study: Mapping San Siro Lab Moving from these general statements and assuming the framework of ULLs as developed so far, as Politecnico of Milan team we have started to question ourselves on if and how it was possible to “translate” (Franz, ibid.) the technologically-centred approach to ULLs to a more socially- oriented one especially addressed to marginalized contexts, starting from our own on-field experience. In fact, The SoHoLab project involves as a case-study an already existing and on- going experience in which we are involved in as researchers: Mapping San Siro (MSS) Lab. Supported by the Department of Architecture and Urban Studies and by Polisocial Program (Politecnico of Milan public engagement program), MSS was started in 2013 as a workshop activity, involving the participation of a group of students, researchers and teachers, interested in challenging the negative narratives associated to San Siro neighbourhood, one of the biggest public housing complexes of the city 10, and in producing shared and “usable” representations that could effectively trigger local change. Thanks 7

Financed through the JPI URBAN EUROPE CALL 2016 - ERA NET COFUND SMART URBAN FUTURE the SoHoLab Project (2017 – 2020) involves three Universities: Vrije Universiteit Brussel (international coordinator), DAStU – Politecnico of Milan and AHTTEP – AUSSER – École Nationale Supérieure d’Architecture Paris La Villette. The project also includes non-academic partners at national levels. More information available on the website: 8 The approach is developed, tested and refined on the basis of a retrospective evaluation of existing projects in Paris, of action research in an ongoing LivingLab experience in Milan and a new one in Brussels. 9 From project proposal 10 Located in the North-West part of the city, not far from the city centre, San Siro is composed of about 6.000 housing units and with a population of about 10.000 inhabitants, the neighbourhood is characterized by the presence of fragile populations and by strong socio-spatial inequalities and intercultural/intergenerational conflicts (around 50% of the population are immigrants, with about 85 nationalities represented). Despite being also characterized by the presence of diverse and strongly committed local actors (associations, cooperatives, groups of inhabitants), San Siro has always been heavily stigmatized in public discourses with the effect of worsening its exclusion from urban dynamics. More information on the website (developed by Master in Journalism of Università Cattolica of Milan together with Mapping San Siro) and on


to the involvement of local actors, the workshop was particularly successful and a group of researchers decided to continue to work in the neighbourhood. In 2014, MSS obtained from Aler – the Regional Agency for Public Housing of the Lombardy Region, which owns and manages the housing stock – the possibility to re-open a vacant shop in the neighbourhood, located on the street level, which became the headquarter of the group. It was the beginning of a new phase: inhabiting a space and becoming a locally rooted actor, on the one hand developing research and teaching activities on three main topics – home and dwelling conditions; courtyards and public spaces, non-residential vacant spaces – and, on the other hand, trying to tackle the urgent issue of promoting participated local change, in a neighbourhood characterized by abandonment, distrust and inertia. We have defined our presence on the field as situating (Cognetti & Castelnuovo, 2019): building significant relationships with the context and gaining an internal perspective to the neighbourhood through a long and slow rooting process. An aspect which profoundly shapes the other two dimensions of our practice: inquiry (embedded research) and acting (promoting participated change).

4 A local research-driven Urban Living Lab? Even if MSS was not intentionally started as a ULL, but as an action-research experience, several elements could be identified that it shares with the approach (among the others see Concilio, ibid.). In particular, elements that – even if already present in the broader approach – should be taken into particular consideration in order to steer the ULL approach when dealing with marginalized contexts. (1) The centrality of the co-research phase for co-learning and co-design (inquiry): as broadly known, LLs are based on a co-creation approach that directly involves the so-called users. To our experience, when coping with marginalized and fragile contexts, particular attention should be paid to make the phase of coresearch as inclusive as possible: identifying and highlight a shared and coconstructed vision of problems and resources, especially by significantly linking scientific knowledge with common knowledge (Dewey, 1938), produced by the so-called everyday-makers (Bang & Soresen, 1999) as to say dwellers and local organizations. Co-research is here intended as a mutual learning process (colearning) which constantly shapes the phase of co-design. It is particularly relevant because it allows local actors to mutually acknowledge themselves a competence and a voice on issues concerning the neighbourhood and its possible transformation, empowering their ability to act 11. As MSS, we have practiced this aspect in particular through the development and coordination of the local network of formal and informal organizations, called Sansheroes (see Maranghi, 2019), which became able to produce a shared vision on the

11 We refer here to the concepts of “right to research”, proposed by Appadurai in 2006, and of the one

of knowledge as a “condition for development”, Freire, 1970.


neighbourhood and a platform of planning, elements that reinforced their capacity to interact with institutions. (2) Co-creation as an incremental process to experiment on effective crossboundaries arenas (acting): in marginal context – where a relationship of mutual trust and acknowledgement of the different actors often needs to be re-build – we have experimented how the co-design of little and incremental pilot projects, which engages both institutions, local organizations and inhabitants, is a powerful tool to rebuild a learning-friendly context, fertile to acquire the ability of working together and generating new forms of governance. As Franz has already pointed out (ibid.), socially- oriented LL are usually concerned with process-based innovation rather than with product based one. To our experience when coping with marginal contexts, the ULL itself – as an arena of collective work – should be considered as a significant incremental outcome, capable of generating innovation in the way in which different actors interact. The University as a promoter of the ULL in this sense plays a key role since its “third position” allows it to effectively engage institutions, civil society, local actors and city level actors (Castelnuovo & Cognetti, ibid.).

5 Provisional conclusions To conclude, we would like to share some open issues that we consider to be able to influence the “success” of so-called Living Labs in marginalized contexts. Elements that challenge, on the one hand, the competencies that we should develop as researchers and practitioners involved in the process, on the other hand, the possibility to activate practices able to produce a durable and scalable change in terms of local development/regeneration. (1) Situating: spatiality and time matter. We have defined situating as the practice of conducting embedded research through the opening of a space in the field (specifying, somehow, the concept of geographical embeddedness of ULLs, Voytenko et al., 2016). The physical space could be considered the core of our activity since it helps to practice contingency (Karvonen & Van Heur, 2014): a constant process of co-learning – related to the “here and now” – grounded in facts, relationships and situations directly experienced and shared with the research field. We argue that when setting up a ULL in a marginal context opening a physical space matters: “being local” helps to build trust and reliability – both towards the institutional level and the local level – and it opens up access to different forms of knowledge (local one, institutional one, scientific one, etc.), key elements for urban regeneration. We recognize, however, that such a rooting process is profoundly demanding in terms of time and possibility to engage in a deep relationship with the context (Kondo, 2012). (2) Potentiality and limits of a local scale. To intend the ULL as an incremental object, questions how not to be “trapped” not only on the local scale but in very minute issues (referring to San Siro, for instance, the transformation of public space in a neighbourhood where housing is the most urgent matter). Besides, it more broadly questions the ULL approach, tackling the matter of how to create a durable and significant change, even if starting from a very local dimension


(Steen & van Bueren, ibid.). In this sense, we are exploring the possibility to intend a ULL in marginal contexts precisely as a device to engage different actors – especially the ones usually perceived as distant from the local level – through the development of pilot projects, constantly giving them the chance to experiment new configurations of urban regeneration governance (co-producing knowledge, working together, etc.) Both points, indeed, question whether do we have to intend local regeneration as an incremental process and, if so, ULLs as local permanent structures, always capable of reinventing themselves but at the same time to be stable and durable.

References Appadurai, A. (2006). The right to research. Globalisation, Societies and Education, 4(2), 167- 177. Bang H.P., E. Sørensen E. (1999). Everyday Maker: A New Challenge to Democratic Governance. Administrative Theory & Praxis, 21 (3), 325-341. Castelnuovo I., Cognetti F. (2019). Mapping San Siro Lab: experimenting grounded, interactive and mutual learning for inclusive cities. Transactions of the Associations of European Schools of Planning, 3, 37-54. Chronéer, D., Ståhlbröst, A., Habibipour, A. (2018). Towards a unified definition of Urban Living Labs. Presented at the ISPIM Innovation Conference – Innovation, The Name of The Game, Stockholm, Sweden on 17-20 June 2018, International Society for Professional Innovation Management (ISPIM). Cognetti F., Padovani L. (2017). New meanings for public housing through the co-production of knowledge. Policies for everyday life in marginal neighbourhoods. In: Bargelli E., Heitkamp T. (Eds.), New developments in Southern European housing. Pisa: Pisa University Press. Concilio, G., De Bonis, L. (2012). Smart Cities & planning in a Living Lab perspective. In: Campagna M., De Montis A., Isola. F., Lai S., Pira C., Zoppi C. (Eds.), Planning Support Tools: Policy Analysis, Implementation and Evaluation, Proceedings of the VII Int.l Conf. on Informatics and Urban and Regional Planning INPUT 2012. Milan: Franco Angeli. Concilio G. (2016). Urban Living Labs: opportunities in and for planning. In: Concilio G., Rizzo F. (Eds.), Human Smart Cities Rethinking the Interplay between Design and Planning. Berlin: Springer. Dewey, J. (1938). Experience and Education. New York: Macmillan. Franz, Y. (2015). Designing social living labs in urban research. info, 17 (4), 5366. Freire, P. (1970). Pedagogy of the Oppressed. NewYork: Herder & Herder.


Hakkarainen, L., & Hyysalo, S. 2013. How Do We Keep the Living Laboratory Alive? Learning and Conflicts in Living Lab Collaboration. Technology Innovation Management Review, 3(12): 16–22. Karvonen, A., & Heur, B. (2014). Introduction. International Journal of Urban & Regional Research, 38, 379-392. Kondo, M.C. (2012). An ethic of love for planning. In: Porter L., Sandercock L. , Umemoto K., Bates L. K., Zapata M.A. , Kondo M.C. , Zitcer A., Lake R.W., Fonza A., Sletto B. , Erfan A. & Sandercock L. (2012). What's love got to do with it? Illuminations on loving attachment in planning. Planning Theory & Practice, 13 (4), 593-627. Maranghi, E. (2019). Networking collective knowledge to foster change. The case of Sansheroes network (San Siro, Milan). Presented at the Annual Conference of the Associations of European Schools of Planning – Planning for transition, Venice, Italy on 9 -13 July 2019. Naumann, S., Davis, M., Moore, M., & McCormick, K. (2018). Utilizing Urban Living Laboratories for Social Innovation. In: Elmqvist R., Bai X., Frantzeskaki N., Griffith C., Maddox D., McPhearson T., et al. (Eds.), Urban Planet: Knowledge towards Sustainable Cities. Cambridge: Cambridge University Press. Nesti, G. (2018). Co-production for innovation: the urban living lab experience. Policy and Society, 37(3), 310-325. Steen, K., & van Bueren, E. (2017). The Defining Characteristics of Urban Living Labs. Technology Innovation Management Review, 7(7), 21-33. Vertovec, S. (2014). Super-diversity. London and New York: Routledge Wenger, E. (1998). Communities of practice: learning, meaning and identity. New York: Cambridge University Press. Voytenko, Y., McCormick, K., Evans, J., & Schwila, G. (2016). Urban Living Labs for Sustainability and Low Carbon Cities in Europe: Towards a Research Agenda. Journal of Cleaner Production, 123, 45–54.



Intelligent Living Lab: Supporting data-centric decision-making using ICT tools

Minki Kim*1, Junghyun Park1, Min-ho Suh1 and Won-Kyun Joo1

1 Korea

*Corresponding author Institute of Science and Technology Information (KISTI), South Korea Category: Innovation Paper

Abstract Many studies have reported a combination of information and communications technology (ICT) tools to support scientific analyses of living lab concepts which intend to solve various urban problems, such as urban traffic problems. Similarly, in this study, the results of a big data analysis focusing on traffic were visualized in a way that can be easily understood by citizens who lack specific domain knowledge. The goal was to improve a bus route, which is a typical public transport facility in a city. In this paper, we introduce the case study of an intelligent living lab that utilizes visualization ICT tools based on big data related to traffic. Keywords: Living Lab, Transportation, ICT Tools, Big data, GIS, Decisionmaking


1 Introduction The living lab is a research concept which operates based on public-private collaborations as a user- centred open-innovation ecosystem (Pallot, Trousse, Senach, & Scapin, 2010). A living lab is literally defined in various ways, such as a ‘living laboratory’ or ‘laboratory of daily life’, a ‘laboratory in our town’, and an ‘innovation space with user participation’ (Tang & Hämäläinen, 2012). A living lab, unlike a common ‘laboratory’ and existing ‘test bed’ businesses, is a system in which users voluntarily participate and study technological innovations. Moreover, recently, the meaning of a living lab has been expanded and now can refer to a concept that enhances governance and sustainability. In addition, living labs are being used as experimental spaces to shift to a ‘sustainable social and technological system’. This trend is particularly evident in ICT. In Korea, we see that ICT tools as a medium of user participation and participation contribute to lowering entry barriers and creating performance outcomes in various studies (Seong & Park, 2015). In overseas cases, much early experience was gained in the European Network of Living Labs (ENoLL), a platform established in 2006 for ICT-based innovations (ENoLL, 2014). In addition, research has reported the use of a living lab as a collaborative tool for integrated development with ICT tools to develop the agriculture industry (Wolfert, Verdouw, Verloop, & Beulens, 2010). As mentioned above, compared to overseas cases where living labs have been actively used, Korea has more recently become aware of the importance of citizen-led living labs. A living lab differs from a traditional ‘research laboratory’. We consider users as subjects of research innovation activities, not as subjects of research innovations, and emphasize testing and demonstrations in real-life situations away from a closed laboratory. Living labs are rapidly spreading, as evidenced by the growth of the ENoLL, and they have received much attention from policymakers, a trend that is expected to continue (Dell'Era & Landoni, 2014). In this paper, we introduce an example of the application of traffic data visualization ICT tools implemented by our research institute to a living lab called the ‘Bus Route Improvement Committee of the Yeongjong-do Area’ to improve an actual bus route in Incheon Metropolitan City. The purpose of this research on an intelligent living lab is to support data-centric decision-making using visualization ICT tools, which process and analyse traffic big data such as smart card data secured through the cooperation of local governments. In Section 2, we introduce living lab research cases linked with ICT tools. In Section 3, we introduce the actual living lab research that combines traffic big data-based ICT tools to improve a bus route, which is the focus of this research. In Section 4, we explain the traffic- related big data-based ICT tools in detail. At the end of the study, we present the lessons learned and the conclusion obtained through this research and introduce future research plans.

2 Related Work Living labs are attracting attention as a new type of approach for problems that require long-term, effective, sustainable innovations by involving users instead of restricting users and designing their surroundings. This is possible because living


labs are suitable for implementing new methods of user- centred research to promote sustainable lifestyles and product innovation (Liedtke, Jolanta Welfens, Rohn, & Nordmann, 2012). As a result, in order to ensure policy sustainability for various issues in the future, the utilization of living labs is expected to become more widespread. Recently, different municipalities have built smart cities by networking all infrastructure units in the city using advanced ICT and big data. A smart city is defined as a city where citizens can enjoy a convenient and comfortable life after traffic, environment, and housing problems, and facility inefficiencies associated with city life have been solved utilizing ICT technologies. Each municipality actively uses the smart city construction process together with a living lab to solve various problems in the city. Examples of such smart cities are ‘U-City’ in Songdo, Incheon, and smart-life cities based on the 3D Internet in Japan (Prendinger, et al., 2013). In addition, based on the development of information technology systems that enable measurements of odor emissions and the collection of citizen feedback, the living lab approach is being implemented to include citizens, public authorities, and industry and environmental non-governmental organizations (NGOs) (Reiter, Gronier, & Valoggia, 2014). There is also an interesting study on a new urban economic model that distributes information gathered and generated through smart city construction to the marketplace (Cosgrave, Arbuthnot, & Tryfonas, 2013). Thus, in the future, many studies of smart cities and living labs, as well as ICT tools to support data- centric scientific analysis, will be conducted at the basic methodology level in an attempt to solve various urban problems more efficiently.

3 Case Study: Intelligent Living Lab In this paper, we introduce the ICT tools implemented by the living lab entitled ‘Bus Route Improvement Committee of the Yeongjong-do Area’. As introduced in Section 2, Incheon city plans to make the entire city a smart city, starting with Songdo ‘U-City’. The living lab concept can also be used to solve various urban problems, such as fine dust, flooding, and earthquakes and can be used to improve public transportation. In particular, Incheon City and the Korea Institute of Science and Technology Information (KISTI) signed an MoU in July of 2018 to demonstrate KISTI’s competence as a data agency and to carry out various studies to solve urban problems in Incheon City. In Incheon city, which is expected to overhaul the bus route in 2020, we present here a case study of a living lab called ‘Bus Route Improvement Committee of the Yeongjong-do Area’, which is one of various efforts to improve public transportation routes, a popular topic at present. We are interested in improving public transportation in Incheon because this city is the third-largest city in Korea, with a mainly metropolitan population, and because Incheon City has experienced numerous fiscal deficits since the introduction of the semi-public bus system in 2009. In the Yeongjong-do area, Incheon International Airport, which opened a second passenger terminal on January 18, 2018, is located, and many new apartments have been built according to new city policies with titled such as ‘Yeongjong International City’. As a result, traffic levels in the Yeongjong-do area


are expected to increase steadily. Although Yeongjong-do is an island, users of public transportation can enter Yeongjong-do through only two major bridges (the Yeongjong Bridge and the Incheon Bridge) which connect the inland to Incheon City. Related to the various characteristics described above, in Yeongjong-do, public transportation-related complaints are relatively frequent relative to the rest of Incheon City. Meanwhile, Incheon City paid 59.5 billion won in 2016 and 106.2 billion won in 2018 after it started to use a semi- public bus system in 2009. As a result of the increase in subway users, the proportion of bus users was 49.3% in 2016 compared with 43.3% in 2018, and the share of buses out of all public transportation is steadily decreasing. In general, adjusting transit intervals and routes to increase citizens’ convenience increases the overall cost of transit operations. In this study, we utilize a living lab that integrates ICT tools to undertake bus route optimization in order to improve the convenience of public transportation and the efficiency of bus routes, considering a trade-off relationship. In addition, research is underway in the agency to carry out the bus route reorganization work and to understand the functions desired by city government officials and policymakers through interviews and then to implement them on the web. The web system currently being implemented includes all of the basic functions of public transportation research, such as an economic analysis according to changes in departure times, calculations of the bend radius of each bus route, as well as user-customized functions obtained through interviews with city government officials in Incheon City. In the future, we believe that this traffic data analysis based on the web will become the basis for constructing a real- time public transport control system (Amini, Gerostathopoulos, & Prehofer, 2017).

4 Use ICT Tools to Support the Living Lab In this study, Tableau (Tableau Software, 2003) was used as an ICT tool to support the living lab to improve bus routes. In addition, with regard to trafficrelated big data, which Tableau cannot easily handle, we used Python’s NumPy, Pandas, and other libraries for programming using the Python programming language with Jupyter Notebook. Additionally, the traffic network is constructed with traffic data such as bus routes and stops collected through a bus information system (BIS) and bus management system (BMS) for integrated visualization using ArcGIS (Esri, 1999). 4.1 Smart Card Data In this study, we investigated the level of demand for buses in order to implement the bus route improvement process. For this purpose, we obtained traffic card data from the Korea Smart Card Company and from Eb Card, which operate automatic fare collection (AFC) systems. According to earlier work (Song, Eom, Lee, Min, & Yang, 2015), Seoul, the capital city of Korea, has more than 10 million transits per day, according to the smart card system. As a result of checking the traffic card data of Incheon City in this manner, similar to the Seoul case, as shown in Table 1, the traffic card usage records of nearly 15 million instances per weekday are shown. The largest difference between the Korea smart card and


the Eb Card traffic card data is that Eb Card provides detailed information about each transfer. Table 1: Data from the Smart Card Automated Fare Collection Systems Date

Korea Smart Card Eb Card File size (GB) # of records File size (GB) # of records Sunday, July 1, 2018 1.06 6,932,275 1.64 5,152,984 Monday, July 2, 2018 2.16 14,084,184 3.22 10,025,347 Tuesday, July 3, 2018 2.3 15,011,778 3.44 10,697,155 Wednesday, July 4, 2018 2.35 15,352,585 3.52 10,949,652 Thursday, July 5, 2018 2.33 15,215,930 3.49 10,870,453 Friday, July 6, 2018 2.44 15,939,027 3.68 11,471,180 Saturday, July 7, 2018 1.8 11,802,249 2.78 8,718,348 Thursday, November 1, 2018 2.38 15,568,227 3.54 11,033,829 Friday, November 2, 2018 2.47 16,154,709 3.71 11,564,515 Saturday, November 3, 2018 1.89 12,447,079 2.94 9,224,146 Sunday, November 4, 2018 1.41 9,265,914 2.2 6,915,418 Monday, November 5, 2018 2.33 15,266,342 3.46 10,777,226 Tuesday, November 6, 2018 2.36 15,464,232 3.51 10,903,731 Wednesday, November 7, 2018 2.37 15,491,086 3.51 10,921,823 Sum 29.65 193,995,617 44.64 139,225,807

Therefore, as shown in Table 1, the file size is large despite the small number of records. Essentially, the contents to be introduced later in this paper are based on traffic card demand data as introduced in the current section. As a result of analysing the traffic card data obtained from Incheon City, it was found that 99% of users of Incheon Metropolitan City bus users use the transportation card as opposed to paying cash. In addition, the dropout rate of the endpoint information of traffic card data was 1~2% of the total data. This is in contrast to the results of studies in which most AFC systems only record boarding information (Li, Sun, Jing, & Yang, 2018). Incheon city's traffic card data is better than Seoul’s traffic card data because Seoul has a single fare system for bus use only, while Incheon Metropolitan City has a metropolitan unity fare system that uses distance-proportional rates, as shown in the list below: ● Higher-grade bus: Basic charge within 10km, additional 100 won at every 5km for 10~40km, additional 100 won charged after 40km ● M bus (wide-area express bus): Basic charge within 30km, additional 100 won at every 5km for 30~60km, additional 100 won after 60km ● Subway: Basic charge for 10km, additional 100 won at every 5 km for 10~50km, additional 100 won at every 8km after 50km 4.2 Origin to Destination Matrix (ODM) In this study, the origin to destination matrix (ODM) shown in Figure 1 was created by calculating the numbers of passengers at boarding stops and the numbers of passengers departing at stops according to the traffic card data introduced in Section 4.1. After evaluating the demand based on the generated ODM, with different colors for each grade, it was possible visually to grasp at a glance the usage demand according to each stop along each route. In order to express this,


first a set of a boarding stops and a set of departing stations are obtained; when the number of passengers for each boarding and departure stop pair is combined, the above-mentioned ODM is generated. Modeling and analysis with a traffic network based on a geographic information system (GIS), as described in Section 4.3 based on the ODM built here, can also be performed.







Figure 1: Origin to destination matrix (ODM)

Another important factor when attempting to measure the demand for public transportation is the level of peak-hour customer demand. It becomes possible to adjust the allocation of vehicles according to the level of demand for buses when dealing with different time zones and to grasp the current state of a route to judge the efficiency of bus route management. Based on this information, citizens’ convenience can be improved by reducing the number of buses on routes with relatively few passengers at peak times and by placing more buses on routes with a large number of passengers at such times. In order to make these analyses possible, in this research, we present the number of passengers according to the travel time for each route in Figure 2. In this visualization, the 23 routes passing through Yeongjong-do, Incheon City are expressed in different colors. As a result, it is possible to understand the peak hours of routes, such as when people go to work and when they return home, and the number


Figure 2: Number of passengers on each Figure 3: Comparison of usage rates for route by time frame each route

Figure 4: Tableau online dashboard

of passengers according to the time of each route. This also enables a proper efficiency analysis of the overall bus operation. Further, as shown in Figure 3, one can intuitively confirm the share of each route by confirming the utilization rate in each case, as expressed here by a pie chart. Hence, based on the information of stops along each route, we determine the number of passengers on the buses by time zone through passenger information, specifically who boarded and departed the buses at each stop. This visualization analysis to determine basic public transport levels represents how a living lab can improve bus routes in areas when civil servants and those living in the area work together.


This can be used to support data-centric scientific decision-making for citizens who participated in a living lab. Through these ICT tools, we were able to confirm the claims based on the experience of civil servants and citizens with actual data, which was used as a tool to verify the validity and usefulness to meet all concerned parties’ needs. We will deal with the results of using the materials created here in the living lab in Section 5. Finally, readers reading this article can also review the traffic demand data configured as shown in Figure 4 through a dashboard provided through links provided here. The dashboard presented here is identical to that used by the living lab discussed in this study. Figure 4 shows the tableau online dashboard as an integrated representation of Figures 1 to 3 described in the above sections. The graphs on the dashboard were designed with reference to the graphs of web pages provided by Incheon city to the citizens. In addition, it was customized to visually check the lack of bus data provided by Incheon city. This Tableau dashboard is designed to be used as reference material for numerical presentations based on current data on the opinions of living lab participants. For this purpose, it was designed to be as intuitive as possible for non-experts to utilize, and experts participated in the actual living labs to support the use of ICT tools and to explain current situation based on actual data. This dashboard was used to support the decision making through the confirmation of facts about the opinions and the status of the opinions after the experts had heard the opinions of the living lab participants. The dashboard, which can be checked via the web, supports faceted searches and interactions with each route and bus stop. These different ICT tools as described here were leveraged to help make data-centric decisions in the living lab to improve bus routes. 4.3 Traffic Network Modeling at Yeonjong-do 4.3.1 Yeongjong-do Bus Routes Incheon Metropolitan City has a web portal service for citizen based on the ArcGIS platform (Figure 5) (Incheon Metropolitan City, 2014). Most of the area of Incheon City is within a traffic network, but Yeongjong-do does not provide web portal service due to regional specificity. Figure 6 shows the integrated traffic and speed data management screen currently provided by Incheon, showing that the Yeongjong-do area is excluded (i.e., the island on the left). In order to analyze the optimal route of public transportation, the network of the target area (transportation network) is required. The transportation network is constructed using the navigation network and national standard data based on GIS. The design of the transportation network must be extendable to reflect the flows of various modes of transportation, such as taxis, buses, and trains. The traffic network of Incheon is divided into a road network and a public transport network considering the characteristics of both, and it is composed of nodes and links. In the road network, a node represents a major intersection point, and a link represents a unique road between nodes.


Figure 5: Smart-GIS of web portals at Incheon Metropolitan City

Figure 6: Visualization of traffic volume and speed data of Incheo

In the public transport network, a node represents transport between nodes that link the main station (bus stop or other stop or station). In order for network composed of nodes and links to represent real-world networks, links and nodes must have certain attributes necessary for this analysis. For a road network, for example, a node indicates an intersection or an end of a road. A link must also include information about the costs associated with the use of the road (Chae, 2011). Different types of cost information can exist, such as the travel time on the road and travel distances depending on the application.


Bus stop

(b) Link & node

(c) Bus traffic network Figure 7: Bus traffic network using Incheon regional GIS data

In this study, a new bus network based on the Incheon traffic network is devised to support GIS for the optimal reorganization of the bus routes in the Yeongjongdo area. The bus network was established for all bus routes entering Yeongjongdo using information about bus stops, intersections, and links in the shapefile(.shp) format provided by Incheon in 2018 (Incheon Metropolitan City, 2011). The bus network consisted of 5402 bus stops, 3093 nodes, and 7774 links, with a total of 24 bus routes (Figure 7). Bus routes include a range of information, including station information (station ID), route information (route ID), operation schedules (timetables), average running speeds (km/hour), and other data. Using correlations between data points and a network algorithm which determined shortest paths, optimal bus routes were created. Figure 8 shows the 23 bus


routes (Bus routes number: 1, 2, 2-1, 3, 4, 111, 117, 202, 203, 204, 222, 223, 302, 303, 304, 306, 307, 308, 320, 303-1, 310, 330, 598) entering Yeongjong-do for bus network analysis. An analysis of the transportation network shows that the bus route network to Yeongjong-do is reasonably operated, but problems exist on some routes.

Figure 8: 23 Bus routes to Yeongjong-do

In 2018, Yeongjong-do opened a second passenger terminal at Incheon International Airport, and additional bus routes were added. As a result, a bus route with a travel distance of 120 km and a travel time of 2 hours was created. Figure 9 shows the five longest routes out of the newly added bus lines. The five bus lines seem to have improved convenience on the user side, but the bus driver's job intensity has greatly increased. This is not only a decline in the quality of the bus service but also the safety of bus users. These bus routes can improve the problem by ensuring that the route is divided or that the bus operator takes a break. 4.3.2 The curvature of Bus Routes based on ODM Route curvature is one of the most important indicators of route design related to route optimization. The degree of curvature of a route has a value of 1.0 when connecting the starting and endpoints of a certain route at the shortest distance, with this value calculated according to the actual operation of the route. The curvature is not considered to have an adverse effect on the routing system unconditionally, but it is considered to be the most important factor when evaluating a bus route because this is recognized as necessary to improve the time, ensure a short travel time and to minimize the time cost.


Figure 9: Five longest bus routes to Incheon International Airport Terminal 2


Figure 10: OD cost analysis using a network analysis based on ArcGIS pro

As the OD information of actual bus users has released been recent, an OD matrix-based route bend analysis has been suggested as the most effective evaluation index (Park, Ha, Kwon, & Oh, 2019). A combination of the ODM constructed in Section 4.2 and the Yeongjong-do bus routes were used to analyze the curvature experienced on a per-user basis. In order to calculate the degree of curvature, a network analysis was conducted using the origin-destination cost matrix provided by ArcGIS (Figure 10). Figure 11 shows the curvature OD dissolution and curvature distribution calculated by the network analysis. The average degree of curvature was 1.17 for 14 days, and most users were not greatly inconvenienced when traveling to Yeongjong-do using the current bus route. However, on some routes, users may be inconvenienced with the curvature 2.0 or more. For these routes, it is necessary to reflect user-based policies such as the addition of new routes to reflect passenger numbers and changes of bus stop locations based on population densities.

6 Conclusions Here, we introduce the concept of the living lab, introduce a living lab which used ICT tools, present an overview of ICT tools based on traffic data, and describe a case study conducted by an intelligent living lab. While concluding the paper, we will also discuss the lessons learned through the actually implemented intelligent living lab and then introduce future research plans. First, while carrying out this research, we confirmed that it is a desirable policy direction to introduce the living lab concept for sustainable policy setting and troubleshooting. In particular, while investigating various research cases related to living labs, we were able to confirm that introducing this type of living lab concept to solve various social problems is a worldwide trend. Similarly, we identified various research cases that combine ICT tools to support data-centric decision-making for the general public when they participate in living labs. This shows that the living lab concept will evolve intelligently. For the bus route reorganization carried out here, the case of an actual living lab with ICT tools


based on traffic-related big data to support scientific decision-making was explained. Among the four major urban problems (traffic, flooding, earthquakes, and fine dust) for which our research team is currently constructing solutions, the most important issue is fine dust. However, in order to carry out research successfully on public transport, used by many citizens from the past to the present and the future, our research team put in a great effort. Furthermore, for Incheon City, the focal point of this research, cooperation between businesses and our research team is also succeeding, and we are planning to reorganization bus routes completely in 2020. Given this situation, the living lab is operated to determine the optimum points of improvement in the trade-off between convenience for users and the efficiency of bus operations. The coordination of existing bus routes has mostly been determined by complaints from citizens who use the buses and from urban planning events. As a result of rule-of-thumb management, Incheon City has been increasing the size of the budget, nearly in a snowball manner, since the introduction of the semi-public bus system. This is done to support the living lab to improve bus routes, to support traffic-related big data-based ICT tools, and to provide objectivity to assertions and immediately consider citizen claims stemming from the data gathered to the greatest extent possible. In addition, by helping citizens access the web anytime and using big data visualization ICT tools, this effort is helping citizens given the scientific analysis conducted to improve their bus routes. While carrying out this research, we were able to confirm the good aspects of using ICT tools, but we also found that some items cannot be confirmed using data alone. For example, in the case of Incheon City, some private buses are operated, but their routes show various utilization rates, and the rates and times of operation are paradoxical when attempting to reduce the deficit through route adjustments to reduce traffic and the number of operating buses. Because the analysis used only data from a fixed period provided periodically, the seasonality analysis was insufficient. Solving these problems is limited in that they require much time from the actual traffic data collection stage. Therefore, using data pipeline tools such as ‘OpenAPI’ to automate the processes of data collection, processing, and analysis we allow us to save time when addressing the above items. Based on previous research, the methodology presented in this paper will be utilized for the total reorganization of bus routes in Incheon in 2020 and will be expanded further to research on other forms of public transport, such as subway including buses.


Figure 11: OD flow maps (upper panel) and distribution of curvature (lower panel) to Yeongjong-do

As the problems to be solved grow, artificial intelligence for traffic route optimization (AI Solver), which we are continuing to study, together with the supercomputing resources of KISTI, can be used to cope with this. In addition, in conjunction with the other urban problems (flooding, earthquakes, and fine dust) dealt with by our current research team, we are also promoting research on bus route optimization through data fusion methods. Finally, various urban problems,


including the traffic problems discussed here, are problems that occur not only in Korea but also in the cities of other countries. Accordingly, the goal of the research that our research team ultimately pursues is for our research to be used in many different cities all around the world.

Acknowledgements This research was supported by Ministry of Science and ICT, Republic of Korea (Project No. K-19- L05-C01-S01). Research is being conducted to solve various urban problems such as traffic problems through cooperation between Incheon City and our institute. We would like to thank the local government for their help giving much assistance for the research on the data-centric solution of social problems.

References Amini, S., Gerostathopoulos, I., & Prehofer, C. (2017). Big data analytics architecture for real-time traffic control. 2017 5th IEEE International Conference on Models and Technologies for Intelligent Transportation Systems (MT-ITS) (pp. 710-715). IEEE. Chae, W. (2011). Analyzing the service effectiveness of the bicycle road plan by GIS network analysis method. Master's thesis. Cosgrave, E., Arbuthnot, K., & Tryfonas, T. (2013). Living labs, innovation districts and information marketplaces: A systems approach for smart cities. Procedia Computer Science, 16, 668-677. Dell'Era, C., & Landoni, P. (2014). Living Lab: A methodology between usercentred design and participatory design. Creativity and Innovation Management, 23(2), 137-154. ENoLL. (2014). Retrieved







Esri. (1999). ArcGIS Online | Interactive Maps Connecting People, Locations & Data. Retrieved from ArcGIS: Incheon Metropolitan City. (2011). Incheon Bus Information Management Systems. Retrieved from Incheon Metropolitan City. (2014). Smart-GIS Incheon. Retrieved from Li, T., Sun, D., Jing, P., & Yang, K. (2018). Smart card data mining of public transport destination: A literature review. Information, 9(1), 18. Liedtke, C., Jolanta Welfens, M., Rohn, H., & Nordmann, J. (2012). LIVING LAB: user-driven innovation for sustainability. International journal of sustainability in higher education, 2(13), 106-118. Pallot, M., Trousse, B., Senach, B., & Scapin, D. (2010). Living lab research landscape: From user centred design and user experience towards user cocreation. First European Summer School "Living Labs".


Park, M.-C., Ha, T.-J., Kwon, S.-D., & Oh, S.-J. (2019). Improvement of Bus Route System Considering Route Curvature. JOURNAL OF THE KOREAN SOCIETY OF CIVIL ENGINEERS (pp. 93-103). Korean Society of Civil Engineers. Prendinger, H., Gajananan, K., Zaki, A., Fares, A., Molenaar, R., Urbano, D., . . . Gomaa, W. (2013). Tokyo virtual living lab: Designing smart cities based on the 3d internet. IEEE Internet Computing, 30-38. Reiter, S., Gronier, G., & Valoggia, P. (2014). Citizen involvement in local environmental governance: A methodology combining human-centred design and living lab approaches. Electronic Journal of E-Government, 12(2), 108. Seong, J., & Park, I. (2015). Living Lab as Transition Arena : Case Analysis and Implication. Proceedings of the Korea Technology Innovation Society Conference, (pp. 907926). Song, J.-Y., Eom, J., Lee, K., Min, J., & Yang, K. (2015). Public transportation service evaluations utilizing seoul transportation card data. Procedia Computer Science. 52, pp. 178-185. Elsevier. Tableau Software. (2003). Business Intelligence and Analytics Software. Retrieved from Tableau: Tang, T., & Hämäläinen, M. (2012). Living lab methods and tools for fostering everyday life innovation. 2012 18th International ICE Conference on Engineering, Technology and Innovation (pp. 1- 8). IEEE. Wolfert, J., Verdouw, C., Verloop, C., & Beulens, A. (2010). Organizing information integration in agri-food—A method based on a serviceoriented architecture and living lab approach. Computers and electronics in agriculture, 70(2), 389-405.



Open Innovation Camp (Oic) – A Tool for Solving Complex Problems Rapidly

Teemu Santonen1*, Julia Nevmerzhitskaya1, Aletta Purola1, and Harri Haapaniemi1


* Corresponding author Laurea University of Applied Sciences, Finland Category: Innovation Paper

Abstract This paper proposes Open Innovation Camp (OIC) concept as a novel methodological solution for overcoming the constraints on upscaling Living Lab experiments. OIC is co-creation sprint type of multi-day event grounded on an open innovation 2.0 principles where a group of carefully selected stakeholders having diverse but complimentary expertise creates a common understanding of (a complex societal) challenge and work together to develop in a co-creative manner user centred concepts and solutions to pre-defined challenges in a set timeframe. Based on the experience from the OIC implementation it is suggested that OIC can be an effective tool for overcoming 1) lack of time and financial resources, 2) unbalanced stakeholder representation, 3) silo effect in co-creation activities problems. Based on the feedback analysis of 47 OIC participants, it is suggested that OIC provides the most value, when implemented in the very beginning of a project. How OIC can help to overcome multi-stakeholder engagement constrains in Living Lab setting is discussed. Keywords: innovation camp, open innovation 2.0, design sprint, service design, design thinking, complex problem, co-creation, quadruple helix


1 Introduction Solving complex problems (Murthy, 2000) – also known as ill structured problems (Simon, 1973) or wicked problems (Navarro et al. 2008) – requires sociotechnological environments that bring together people with different, complementary, and often controversial knowledge and skills. Diversity can be associated to any attribute to indicate that another thing, person, group, organization, network or ecosystem is different (adapted from Williams and O’Reilly, 1998). In an innovation process, a novel thinking outside the box can be boosted when diverse people having complimentary skills and knowledge follows open innovation principles (Chesbrough, 2006). However, the participant diversity can also reduce innovation performance due too high level of task conflicts (also known as cognitive conflicts), which can be defined as perceived disagreement among group members relating their opinions and ideas (Simons and Peterson, 2000). In worst case scenario task conflicts are causing relationship conflicts, (or emotional conflicts), between the group members and leading to negative impact on group satisfaction, commitment and decision quality. Thus, effective facilitation and management of the development and innovation efforts is among the key challenges of any open innovation activity which is grounded on a multi-stakeholder collaboration including Living Lab approach. Service design (SD) (Zomerdijk & Voss 2010) and design thinking (Brown, 2008) have become a central framework to co-create novel solutions. It is about planning, developing and innovating product, services and concepts through specific iterative development processes while utilizing various methods, techniques and tools. The main purpose of SD is to create a customer-centric experience that meets the needs and demands of the end-customers but also fulfils the business objectives. It is argued that through a SD approach, diverse teams can collaboratively identify needs, ideas, experiences, opportunities and generate fast prototypes to be tested by the real users and customers. SD helps to innovate (create new) or improve (existing) services to make them more useful, usable, desirable for customers and efficient as well as effective for the organization. 1.1 Objective and structure of this study The goal of this study is to introduce Open Innovation Camp (OIC) concept, which in relatively short-timeframe brings together in a controlled manner a diverse set of actors having complimentary skills to rapidly co-create visions and practical solutions for complex problems, which can further to be developed and tested e.g. in series of Living Lab activities. The structure of this paper follows a constructive action research paradigm (Cassel and Johnson, 2006) process which is a methodology to develop solutions to a practically relevant problem by applying theoretical knowledge and demonstrating the functioning and innovativeness of the suggested solution in real life (Jaatinen and Lavikka, 2008). The structure follows a process framework originally proposed by Kasanen et. al. (1993) and refined by Oyegoke (2011) as follows:


1) Justify the practical relevance of the proposed problem (i.e. the challenge of solving complex problems in section 1), 2) Present the theoretical connection (i.e. open innovation 2.0, management of participant diversity and design sprint type of approaches justified in section 2), 3) Construct the solution (i.e. OIC-concept description as presented in section 3), 4) Demonstrate that the suggested solution is working (i.e. real-life implementation of the OIC and collecting feedback from OIC participants) and 5) Present the research contribution including applicability of the solution (i.e. discussion and conclusions of this study as presented in the section 5).

2 Theoretical foundations of Open Innovation Camp (OIC) concept 2.1 OIC as an Open Innovation 2.0 approach Theoretical foundations of OIC are grounded on Open innovation approach (Chesbrough, 2006). Open innovation provides a generic framework for a distributed innovation process and knowledge flow management across organizational boundaries by using pecuniary and non-pecuniary mechanisms to support organization’s business model (Chesbrough and Bogers, 2014). However, OIC as a tool to solve complex problems needs to go beyond facilitating collaboration between individual organizations to achieve its’ ambitious goals. Therefore, OIC leans more on the Open Innovation 2.0 (OI2) framework, which emphasis ecosystem centric cross-organizational innovation grounded on the collaboration between quadruple helix actors (Curley & Salmelin 2013). Quadruple Helix model (Carayannis and Campbell, 2009) describes an innovation system where government, industry, academia and civil society work together to co-create the future and drive structural changes far beyond the scope of what any single organization or person could do alone. As a result, OIC provides possibilities to have sufficient capability to successfully integrate the information obtained from the external sources into internal processes which is crucial for open innovation to be effective (Nonaka, 2007). 2.2 Enhancing creativity by managing participant diversity Careful management and diverse inclusion of complementary stakeholders is an important part of OIC. This suggestion is in-line with stakeholder theory (Freeman, 1984) and stakeholder engagement in an open innovation processes as suggested by Gould (2012). Importantly, successful stakeholder engagement in OIC will go beyond the acquisition of specific information from external experts to “trust and relationship building” and “mutual understanding” among various innovation ecosystem actors. Participant diversity management in OIC is based on a model proposed by Santonen (2016), which includes cultural, organizational, user-driver, crossfunctional, disciplinarity and cross-industry diversity as presented in Figure 1.


Figure 1: Diversity management model (Santonen, 2016)

Cultural diversity is one of the major challenges in the 21st century, which have a direct impact on the innovation acceptance. In the suggested diversity management model, cultural diversity reference is two folded. Cultural diversity refers to a need to recruit participants from different countries in which the market conditions and code of conduct are differing. Organizational diversity is linked to Quadruple Helix model, which is the foundation of the OI2 approach as suggested by Curley & Salmelin (2013). User-driven diversity highlights the need to understand different kind of end-users, who are expected to use the co-created solutions. Personas derived from service design methodology (Zomerdijk & Voss 2010) are archetypes of actual users and can be successfully used to verify userdriven diversity. Cross-functional diversity is related to an idea of a crossfunctional team in which is a group of people with different functional expertise are working toward a common goal (Kahn 1996). Basically, this referrer to making sure that OIC includes persons having different job descriptions such as R&D, marketing or management. Finally, disciplinarity diversity and cross-industry diversity are referring to involving participants from different scientific disciplines or industries. By considering the above participant diversity dimensions and making consciously decisions who to recruit to OIC, the likelihood of co-creating more radical innovations should increase. Furthermore, the participant recruitment challenges within Living Lab projects have been identified as a major constrain to on upscaling Living Lab experiments (Dijk et al. 2019). Therefore, more systematic stakeholder management is essential for Living Lab success. 2.3 OIC as a design sprint


Methodologically OIC concept belongs to a family of several time-constrained agile development exercises such as hackathon, design sprint, service jam, innovation camp, solution camp and entrepreneurship camp. The definitions of the above terms vary, but typically these approaches can be characterized by following attributes (Halvari et. al 2019): 1) short time-bounded event, 2) intense collaboration, 3) competition, 4) collocation, 5) offline: people meet locally, 6) ideation, experimentation and creativity, 7) teams, 8) pitching/presenting and 9) recognition. Hackathons are team-based coding or product development marathons, which originates from the 1960s, but became popular soon after the turn of the millennium (Halvari et. al 2019). Scientific Hackathons can be used for deep data analysis as an agile interdisciplinary collaboration between organizations and researchers (Ghouila, 2018). Hackathons can also be competitive rather than collaborative events (Richterich, 2017). Design sprints are co-creation events focusing on testing prototypes. Probably the most popular Design sprint approach is the five-day model which is inherited from the Google (Knapp et. al. 2016) and includes idea, build, and launch and learn stages. Service jams are defined as a type of short-term innovation communities (Römer et al, 2011), which usually last 48 to 72 hours and bring together thousands of experts and interested people globally in order to openly work on predefined challenges, problems, or topics. Most known format today is the Global Service Jam (GSJ, 2018), which is a volunteer community of service design experts. Innovation Camps can take different forms. For example, the Aalto Camp for Societal Innovation (ACSI) is addressing societal challenges via open-ended challenges while relying on self-organising working approach (Rissola et al, 2018). Innovation camps (also known as solution camps or entrepreneurship camps) can also be used for educational purposes (Bager 2011). In this type of an event students with other stakeholders are co-creating solutions for innovation challenges while learning team building, creativity and innovation skills. As a result, it is argued that OIC type of events can take many forms depending on the thematic focus of the event. Based on the above theoretical foundation following definition for Open Innovation Camp (OIC) is proposed for purpose of this study: “Open Innovation Camp (OIC) is co-creation sprint type of multi-day event grounded on an open innovation 2.0 principles where a group of carefully selected stakeholders having diverse but complimentary expertise meet locally and creates a common understanding of (a complex societal) challenge and work together in teams to develop, present and review in a co-creative manner user centred concepts and solutions to pre-defined challenges in a set timeframe”.

3 The key characteristics of Open Innovation Camp concept 3.1 Roles, stakeholders and working groups of the OIC OIC is a co-creation event, which follows the definition of co-creation as an “act of collective creativity shared between two or more people” (Sanders et al, 2008). It means that planning and implementing OIC requires collective effort of different stakeholders with defined roles. The main roles of OIC are described in Table 1.


Table 2. OIC roles Role


OIC Orchestrator

The main organizer who is setting the frame of the OIC and acting as a mediator between other involved parties. Similar to Camp Convener role (See Rissola et al, 2017, p.57)

Group Owner

Representative of an organization to which the challenge is of a strategic importance, who sets up the scope of a challenge, and who is motivated, direct interest, and means for solving the challenge. Have substantial understanding of the given challenge. Similar to a challenge owner, case owner or product owner used in design sprints.

Main tasks during the OIC Overall content and outcomes planning, coordination and practical arrangements - Recruitment of challenge owners, facilitators and participants - Being a host and main point of contact during the OIC -


Create background materials for the challenge - Introduce the challenge to the team during OIC - Help the team to answer contentspecific questions during creative process - Experts in the challenge field. -


Person who facilitate people's expressions of creativity at all levels (Sanders et al, 2008). Experts in service design and cocreation.


People taking part in OIC. OIC participants are internationally recognized experts in their field and end-users who have been selected based on the participant diversity management framework presented in section 2.2

Ensure that creative process is implemented according to the plan - Ensure that the right set of service design tools is used to unleash the co-creation potential of a diverse group of experts - Bring people into the design process in the ways most suitable to their ability to participate -

To bring the expertise based on the role he/she is representing in the OIC To share their knowledge with other and co-create new ideas towards solving the challenge

The four defined roles can be further divided into subgroups. For example, the role of an orchestrator can be shared between different stakeholders, one being responsible for practical arrangements and OIC logistics, and another one for content and facilitation. Also, the challenge group owner role in case of a challenge related to a specific business model, or consumer understanding, can be performed by a researcher or a consultant, whereas group owners of sectoral/industry-specific challenges are usually represented by business decision-makers. During the camp, participants belong to a home group, which composition is defined by following the participant diversity guidelines. The home group composition remains the same during the whole OIC event. Two types of home group are identified: Industry and Thematic home group. Industry group participants represent various industry ecosystem expert roles in the given industry. They have profound knowledge on this specific industry field. Thematic


group member represents experts who have specialized skills in a specific thematic field area, which can be used in multiple industry setting. 3.2 The main OIC phases The preparation and execution of the OIC follows the three stages described below: the planning phase, the implementation phase, and the communication phase. PLANNING PHASE: In the planning phase the topics and scope to be addressed during the OIC are first collaboratively defined by the OIC Orchestrator and Group Owners. Once the agreement is achieved, the collection of the background information and preparation of the starting point materials and challenge descriptions can start. The Group Owners have the main responsibility in this process. Based on the OIC scope and availability of the background information, OIC Orchestrator will define the duration and structure of the camp as well as select suitable service design tools for the camp in collaboration with the facilitators. Defining, inviting and recruiting participants based on what complementary expertise is also an essential part of the planning phase. To ensure the optimal balance of participants, recruitment process includes invitation and application processes. An invitation is a process where orchestrator(s) and group owners sends a personal request to an expert to join the OIC. Only those experts who are considered as the “world’s leading experts” within their field and their presence in OIC will provide exceptional value for the camp, receive an invitation. All other participants is suggested to go through an application process in which applicants describe their motivation to participate; their relevant expertise, and argue what would be their potential contribution to the open innovation camp. To follow participant diversity management practices described in section 2.2, the desired overall composition of the OIC must be predefined before starting the recruitment process. However, the optimal overall composition is always depending on the given challenges of the OIC. Finally, practical event arrangements and budgeting are also carried out in planning phase. IMPLEMENTATION PHASE includes carrying out the OIC according to predefined plan and making adjustment to camp program and tools if needed during the camp. First, concentrating on creating the team spirit and enabling creative atmosphere. Next managing design thinking workflow, and collaborative decision making on selected concepts to be further developed e.g. in Living Labs. Finally collecting feedback from the participants as well as documenting and analysing the key contributions and impact of the OIC. COMMUNICATION PHASE is about sharing the final deliverables with the OIC participants and to a wider audience. The hopefully positive OIC experience aims to further engagement of the participants into follow-up activities such R&D, testing and demonstrations in Living Labs. Also maintaining established relationships with OIC participants by inviting them into external advisory group of the possible follow up project.


3.3 OIC content structure and daily program The OIC structure is designed taking into consideration the complexity and diversity of the challenges to be addressed by a high number of experts who are not familiar with each other and the context of the challenges. Regardless of the challenge, the daily programme follows the basic structure presented in Figure 2.

Figure 2. OIC daily structure

PHASE 1: Creating shared understanding and trust: The first day of the OIC is designated to getting to know each other and building trust among the participants, in order to generate a secure creative open innovation culture. These activities include a series of ice breakers and facilitated introductions among participants (Preziosi, 2006). Alongside getting to know each other, a shared understanding of the OIC goals, outcomes and vision is co-created which together are forming the foundation for the shared commitment. The shared understanding will be used as a benchmarking tool for reviewing the generated ideas during the final day of OIC as well as later on during follow up Living Lab activities. PHASE 2 (to max day 4): The day 2 to until the day 4 (depending on the duration of the OIC), are grounded on the series of short-term design thinking workshops in which various working groups are collectively and iteratively developing solutions for the defined challenges. During the workshop’s participants work in cross-group teams to co-create and refine the concepts and practical solutions which are expected to solve the defined challenges. The facilitated workshop are all about sharing the knowledge and learning from each other while also critically reviewing the suggested ideas against the shared commitment. The mixed teams are benefitting from complimentary expertise of diverse stakeholders. PHASE 3 (day 3 to day 5): The first part of the final day (day 3 to 5 depending on the duration of OIC) includes fine-tuning, documenting and presenting the outcome deliverables of the camp to all camp participants. In this process the original home group compositions can be changed, to dedicate resources where they are mostly needed. Based on the presentations, a collaborative reviewing of the developed solutions is conducted via crowd voting system. The group


assessment helps to selected and make decisions which concepts should be further developed after the OIC. Collective selection process also aims to lowers the resistance in the OIC follow up stages. A daily program example based on real-life OIC implementation is presented in Appendix 1. The example consolidates collaboration processes between the within home groups and between mixed groups and shows how the iterative cocreation process is evolving during the OIC.

4 Feedback on real-life OIC experiment As a part of European Commission’s H2020 circular economy programme funded project, OIC camp developing Circular Economy (CE) solutions for four different industries and including 80 participants was implemented in Autumn 2018. After the camp, online feedback survey was sent to all participants resulting 58.6 % response rate (47/80). The feedback received from the OIC participants supports our assumption that an OIC is an excellent tool for rapid stakeholder engagement when addressing complex societal challenges such as circular economy. Nearly ninety percent (89.4%) of all respondent had found new contacts initiated by the OIC. All respondents gained new insights and knowledge while over half of them gained to great or to very great extent (51.1%). Over third of the respondents (36.2%) could apply the new knowledge to great or to very great extent to their work. OIC participants were willing to recommend the camp for others. At the time of the data collection, eleven respondents (23.4%) had already done it and twenty (42.5%) would definitely do that. Fifteen respondents (31.9%) would probably recommend and only one participant said that he/she probably would not recommend. Astonishing 95.7 percent of respondents would probably or definitely attend OIC again.

5 Discussion Based on the experience from the OIC, we believe that it can solve a number of constrains related to multi-stakeholder engagement in innovation and development processes. For the discussion purpose we adopted the constrains to Living Labs identified in SmarterLabs project (Dijk et al, 2019) and clustered them in 4 major groups, each followed by a suggestion how OIC can address the constrain. Lack of time and financial resources problem: OIC helps to address the issue related to lack of time and financial resources of stakeholders and users to participate in open innovation process and Living Lab activities. This lack of resources often creates an issue of certain stakeholder group not participating in the Living Lab activities. OIC is a from 3 to 5 days event which gathers in one place diverse stakeholders, thus allowing participants to use the time efficiently to address complex problems which otherwise require substantial amount of time. Considered that the participants are located in the same hotel during the Camp and OIC includes also social activities during the evening time allowing un-formal discussion in relaxed atmosphere to discuss further bilateral cooperation


opportunities. Since the participation in the OIC is pro bono, but the travel and accommodation expenses are covered by the OIC organizers, such events offer a greater return on investment (time spent in the Camp) than traditional brokerage events or co-creation activities. Unbalanced stakeholder representation: This constrain cluster includes a number of issues. First, some of the relevant stakeholders do not share the urgency to discuss the issues at stake and take actions, and hence they are often left out from the Living Lab activities. Second, often happens so that some stakeholders, such as academic partners or Living Lab experts, are often given more weight than other participants in the co-creation activities. OIC can solve the issues by identifying relevant target groups based on the participant diversity management approach and make sure that as proper representation of stakeholders is achieved. Selection and recruitment of these stakeholders are crucial to the success of an OIC. Silo effect in co-creation activities: Stakeholders participating in co-creation activities, are often highly fragmented, which results in so-called silo effects and lack of cooperation between different types of stakeholders. Also, indirect stakeholders are excluded from co-creation activities, for example, representatives of different industry sectors. The matrix structure of an OIC daily program emphasises close interactions among diverse stakeholders enabling a seamless workflow between different subgroups, and making sure that stakeholders from different ecosystems interact with each other. Furthermore, the cross-group review and reflections are making different stakeholders’ needs and requirements visible. Finally, the professional facilitators and purposefully selected co-creations tools enhance the co-creation productivity. Co-creation is a process not an event: Often co-creation activities are organized as separate steps in service design process. Experiences and ideas from stakeholders who participated in earlier stages of co-creation are often overlooked. Using OIC in the very beginning of the Living Lab planning process allows for systematic engagement of diverse stakeholders also later in the Living Lab activities, as OIC creates a feeling of shared understanding and joint responsibility towards solving the challenges. Towards the end of Living Labs OIC can be used as a validation tool for developed solutions. It is important to keep in mind that an OIC is a part of a process, not a separate tool.

6 Conclusions and Recommendations To conclude, OIC is proposed as a tool to rapidly establish new collaboration relationships, discover new insights by sharing knowledge and co-creating novel solutions by diverse set of actors who can apply outcomes of to their work. Thus, OIC is recommended as a tool especially when there is a need to establish new innovation and knowledge sharing networks in a situation when actors do not know each other before hand. However, selecting suitable participants of OIC and defining fluent workflow across subgroups during the OIC days is demanding task, which requires careful planning but also flexibility to change plans based on the daily deliverables if needed. The following recommendations are offered for


replicating and upscaling the OIC concept to other types of complex societal challenges: OIC is a good tool for solving complex societal challenges rapidly while providing economic value, however, the tool alone is not efficient without supporting Living Lab processes. We recommend to use OIC in the very beginning of a project, to create a sense of shared responsibility among different stakeholders, and a common understanding of a challenge and possible solutions, and at the end of a project, as a validation tool for developed solutions. When using OIC at the very beginning of a living lab project, it has the greatest likelihood to provide economic value. The early phase of innovation process (Cooper, 1988) also known as a fuzzy front end (FFE) of innovation (Smith and Reinertsen, 1991) is important since quality, costs, and timings of the innovative solution are mostly defined during this stage (Herstatt and Verworn, 2004). According to Cambridge dictionary, economic value can be defined as the value of an asset calculated according to its ability to produce income in the future. Furthermore, Bowman and Ambrosini (2000) referred to multiple resource-based theory studies and argued that resources in general are assumed to be valuable and especially if they 1) enable customer needs to be better satisfied, or 2) satisfy customer needs at lower costs than competitors or 3) enable a firm to conceive of or implement strategies that improve its efficiency and effectiveness. As stated in the discussion section, OIC can address multiple the living lab constrains including 1) lack of time and financial resources problem by using pro bono as well as short and intensive time period approaches to engage world class experts with lower costs than operating e.g. via paid consulting contracts, 2) overcoming unbalanced stakeholder representation and silo effect by applying participant diversity approach, thus increasing the likelihood for better satisfying the customer needs by enriching the co-creation process with multiple viewpoints. OIC offers an efficient solution for engaging stakeholders who do not know each other. In this sense OIC concept is addressing initiation of new relationships in complex ecosystems. However, maintaining this relationship is a different challenge which can be addressed by systematic engagement of an OIC participants into Living Lab activities and validation process which takes place after the OIC. This is especially important since among network theorists of innovation (e.g. Snehota and Hakansson, 1995), organizations are rarely capable to innovate independently. Some even argue that networks are the main source of innovation (Von Hippel, 2007). Networks and knowledge as the key components of the knowledge and networked society are indisputably the core components of the any business success. Thus, the findings that most of the OIC respondent had found new contacts initiated by the OIC as well as participants were able to apply their new insights to their own work, provides strong tools for knowledge and network driven innovation processes. OIC is based on pre-defined structure and carefully planned group interaction between complimentary actors. The success of an OIC is dependent on creating the matrix structure which enables systematic co-creation process, when results of one subgroup interaction are reflected and further developed by the following


subgroups. Therefore, designing OIC structure that is easy to implement but also allows seamless interactions between all challenge groups is crucial.

Acknowledgements This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No [776503] for A circular economy approach for lifecycles of products and services – project (CIRC4Life). For more information see The authors gratefully acknowledge this support and present also our gratitude and appreciation to CIRC4Life project partners and innovation camp participants.

References Bager, T., 2011. The camp model for entrepreneurship teaching. International Entrepreneurship and Management Journal, 7(2), pp.279-296. Beaulieu, Luce & van Durne, Gabrielle & Arpin, Ml. (2016). Circular Economy: A Critical Literature Review of Concepts. Bowman, C. and Ambrosini, V., 2000. Value creation versus value capture: towards a coherent definition of value in strategy. British journal of management, 11(1), pp.1-15 British Design Council, 2015, Design methods for developing services, retrieved from n%20methods%20for%20developing%20services.pdf Brown, T. (2008). Design thinking. Harvard business review, 86(6), 84. Carayannis, Elias G., and David FJ Campbell, (2009) 'Mode 3'and'Quadruple Helix': toward a 21st century fractal innovation ecosystem. International Journal of Technology Management 46.3, 201-234. Cassell, C. & Johnson, P., (2006), Action research: Explaining the diversity, Human Relations; Vol. 59, 6; 783-814. Chesbrough, H., Bogers, M., (2014) Explicating open innovation: clarifying an emerg-ing paradigm for understanding innovation. In: Chesbrough, H., Vanhaverbeke,W., West, J. (Eds.), New Frontiers in Open Innovation. (Oxford University Press,Oxford, Chesbrough, H.W., 2006. Open innovation: The new imperative for creating and profiting from technology. Harvard Business Press. Chesbrough, Henry. (2012). Open Innovation: Where We’ve Been and Where We’re Going. Research-Technology Management. 55. Cooper, R.G. (1988), Predevelopment activities determine new product success, in: Industrial Marketing Management,Vol.17, No 2,pp. 237-248 Curley, Martin & Salmelin, Bror (2013) Open Innovation 2.O: A New Paradigm. Dijk, M., da Schio, N., Diethart, M., Höflehner, T., Wlasak, P., Castri, R., Cellina, F., Boussauw, K., Cassiers, T., Chemin, L., Cörvers, R., de Kraker, J.,


Kemp, R., van Heur, B. (2019). How to anticipate constraints on upscaling inclusive Living Lab experiments, SmarterLabs project 2016-2019, JPI Urban Europe European Commission, 2018, Open Innovation 2.0, retrieved


Freeman, R.E. (1984). Strategic Management: A Stakeholder Approach. Pitman, Boston. Ghouila, A., Siwo,G., Entfellner, G., Domelevo, J., Panji, S., Button-Simons, K., Davis, S., Fadlelmola, F., Ferdig, M., Mulder, N. (2018), Hackathons as a means of accelerating scientific discoveries and knowledge transfer, April 2018, Genome Research 28(5). Gould, R (2012) Open Innovation and Stakeholder Engagement, Journal of Technology Management and Innovation 7(3):1-11 GSJ,

2018, Global Service Jam: About,



Herstatt, C. and Verworn, B., 2004. The ‘fuzzy front end’ of innovation. In Bringing technology and innovation into the boardroom (pp. 347-372). Palgrave Macmillan, London. Halvari, S., Suominen, A.H., Jussila, J., Jonsson, V., Bäckman, J., (2019) Conceptualization of hackathon for innovation management, in the the proceedings of The XXX ISPIM INNOVATION CONFERENCE Celebrating Innovation - 500 Years Since Da Vinci - 16-19 June 2019 Florence, Italy, (EDS.) Bitran, I., Conn; S., Gernreich; C., Heber; M., Huizingh; K.R.E., Kokshagina; O., Torkkeli; M., Tynnhammar, M., Tutkimusraportit – Research Reports Jaatinen, M., Lavikka, R., (2008), Common Understanding as a Basis for Coordination, Corporate Communications: An International Journal, Vol. 13 No2., pp.147-167 Kahn, K.B. (1996). “Interdepartmental Integration: A Definition with Implications for Product Development Performance, Journal of Product Innovation Management, 13 (2), 137–51. Kalmykova, Y., Sadagopan, M., Rosado, L., (2018), Circular economy – From review of theories and practices to development of implementation tools. Resources, Conservation and Recycling, Volume 135, 2018, pp. 190-201 Kasanen, E., & Lukka, K. (1993). The constructive approach in management accounting research. Journal of management accounting research, (5), pp. 243-264. Kirchherr, J., Reike, D. and Hekkert, M., 2017. Conceptualizing the circular economy: An analysis of 114 definitions. Resources, Conservation and Recycling, 127, pp.221-232. Knapp, J., Zeratsky, J., & Kowitz, B. (2016). Sprint: How to solve big problems and test new ideas in just five days. Simon and Schuster.


Miaskiewicz, T., & Kozar, K. A. (2011). Personas and user-centered design: How can personas benefit product design processes?. Design studies, 32(5), 417-430. Murthy, P. N. (2000). Complex societal problem solving: A possible set of methodological criteria. Systems Research and Behavioral Science, 17(1), pp. 73-73. Navarro, J., Hayward, P., & Voros, J. (2008). How to solve a wicked problem? furniture foresight case study. Foresight: The Journal of Futures Studies, Strategic Thinking and Policy, 10(2), pp. 11-29. Nonaka, I. (2007). The knowledge creating company. Har-vard Business Review, 85(7/8), 162-171. Oyegoke, A., 2011. The constructive research approach in project management research. International Journal of Managing Projects in Business, 4(4), pp.573-595. Preziosi, R. C. (2006). Icebreakers. American Society for Training and Development. Richterich, A. (2017). Hacking events: Project development practices and technology use at hackathons. Convergence: Convergence: The International Journal of Research into New Media Technologies1–27. Rissola, Gabriel & Kune, Hank & Martinez, Paolo, 2017. "Innovation Camps Methodology Handbook: Realising the potential of the Entrepreneurial Discovery Process for Territorial Innovation and Development” Römer, M., Thallmaier, S., Hormeß M, Lawrence, A, Habicht, A, 2011, Jams as emerging practice of innovation communities: The case of the Global Service Jam 2011, Sanders, Elizabeth B.-N. & Stappers, Pieter Jan (2008) Co-creation and the new landscapes of design, CoDesign, 4:1, 5-18, DOI: 10.1080/15710880701875068 Santonen, T. (2016). Management of diversity in open innovation processes. In Open Innovation: A Multifaceted Perspective: Part II (pp. 631-658). Simon, H.A., (1973), The structure of ill structured problems. Artificial intelligence, Vol. 4. pp. 181-201. Simons, T.L. and Peterson, R.S., (2000) Task conflict and relationship conflict in top management teams: The pivotal role of intragroup trust. Journal of applied psychology, 85(1), p.102. Smith, P.G. & Reinertsen, D.G. (1991). Developing Products in Half the Time. NY: Van Nos-trand Reinhold Snehota, I., and Hakansson, H. (Eds.). Developing relationships in business networks. (Londres: Routledge, 1995). Williams, K. Y., and O'Reilly, C. A. (1998) Demography and diversity in organizations: A review of 40 years of research. Research in organizational behavior, 20, 77-140.


Von Hippel, E. (2007) The sources of innovation (111-120). Gabler. Zomerdijk, L.G. and Voss, C.A., 2010. Service design for experience-centric services. Journal of Service Research, 13(1), pp.67-82.


Appendix 1: OPEN INNOVATION CAMP (OIC) program example for Circular Economy


Sustainable Living Lab Processes, Business Models And Goals 242


Building a platform of social entrepreneurship and living together

Athanasios Priftis*1, Leonor Afonso2, Theo Bondolfi3 and Jean-Philippe Trabichet4 * Corresponding author of Applied Sciences in Switzerland (HES-SO) / Ecopol Living Lab, Switzerland 2, Switzerland 3 Ecopol Living Lab 4 University of Applied Sciences in Switzerland (HES-SO), Switzerland

1 University

Category: Research-in-progress

Abstract The goal of this paper is to present the initial steps of a web / mobile application of co-living and social entrepreneurship to be used in European ecovillages. The application is set to improve existing co-living conditions through more collaboration between its members, thus contributing to more stability and better long-term relations. The main hypothesis is that if we manage to decode and reintroduce co-living activities, already taking place in ecovillages and ecoquartiers, in a clear, open and collaborative way, then we can stimulate more entrepreneurship in between communities, as well as other actors. In order to test and implement the first version of this application and build on an open and collaborative approach, we participated in a Social Hackathon (2018) presenting our concept for establishing a prototype. The results of this effort are included in this paper. Finally, our initial deployment target public will be one network of ecovillages (Ecopol - Smala) based in Switzerland. Keywords: Living Lab as a service, Living Labs, Business model, Design-driven innovation


1 State of the art and research questions Co-living and sustainable communities is an issue of intense research interest. Ecovillages incorporate a variety of ways of living in community with others, providing new departures in personal, social, and ecological living (Bang, 2005). While technologies themselves cannot address the societal challenges (Bierens de Haan, 2006) in ecovillages, they consist of an inevitable playground, particularly when they are coupled with collaborative skills and social entrepreneurship opportunities. Reuse and improvement of existing process in a community can be a form of innovation and extend to include frameworks, processes, and policies (Waugaman, 2016). However, adaptation of communities to new technologies can be a quite difficult task. Especially, when it comes to using new application and tools that go beyond the existing habits there can be significant resistance. Nathan (2008) provides as with an interesting perspective and an example of a digital technology paradigm that has resisted adaptation. He states a situation of a community where all members have access to and check email at least once a week in order to receive critical information; business meeting agendas, proposals, and minutes. According to a key member of the steering committee, “email is much more efficient�. Yet during each meeting there are members who have not received the information because: (1) email was not addressed correctly, (2) attachments were missing, or (3) email was not read by recipient before meeting. This perspective demonstrates that specific applications have a clear role to play when it comes to organising every day or longer-term activities in communities of people that share similar living habits. Particularly, when it comes to multiple tasks with different context, participants and level of complexity. This is why this project adopts the methodological approach of collaborative action research (Somekh, 2006), which requires a feedback loop that links the processes of planning, acting, observing and reflecting throughout the project cycle. Methodologically this leads us to the development of a mixed methods research plan, whereby different data collection tools and the data resulting from them (web analytics, rich data resulting from interviews and focus groups with stakeholders) are used in a complementary manner during different phases of the project. Our evaluation will thus address the following broad themes: How do people perform certain tasks in a community? Can internet applications help them in their organisation and implementation? How do communities understand collaboration into their ongoing online and offline practices? What about privacy in and between communities? How should the application help people interact differently with it? What kind of objectives should it serve? What are the forms of socio-technical innovation produced during the use?

2 Objectives and methodology Following the work of Daly (2015) there is a need for visioning process of the process in an ecovillage providing a means of raising many practices and elements, particularly meanings, from the practical to the discursive consciousness of all members of the community. Bringing daily habits, or elements of practice, into a discursive consciousness is a crucial step in creating pro-environmental change, and one that should involve a social exploration of


new alternatives at a group or community level (Kitchen, 2009). Our application seeks to move between the social and the technological, proposing four (4) major objectives: a) Better organise existing activities in communities, b) create a platform based on (social) co-living tasks and results, c) identify and promote social entrepreneurship opportunities and d) improve and promote intentional communication and strengthen the bond between people involved in the ecovillages. The main socio-technical characteristics of the application as set in collaboration with the Ecopol – Smala community can be described as follows (Bondolfi, 2005): a) free and open source development and licensing, b) open participation with the invitation of various stakeholders including inhabitants, visitors, researchers and policy makers to review its deployment, c) respecting user privacy using cryptography for personal data and d) controlling published metadata with explicit user authorization and decentralised database architecture that secures its long term sustainability and community oriented approach. The application itself should be modular and transformable to a social entrepreneurship market where participants agree on the data aggregated and published at a central space. These data can be communicated and connected to other community platforms, matching supply and demand in regional, national and international level. More specifically, we contacted five (5) interviews with the coordinating team of the Ecovillage, including habitants and selected members of the co-working space. Our questions were structured around two areas leaving together and opportunities to work together. Following several working sessions during 2018 within the actors of the Smala – Ecopol ecovillage, we came up with specific proposed, initial, functions that are described below. Their listing serves as a way to explain to participants the potential use of our application for: 1. Preparation, animation, and follow-up of co-inhabitants/co-operators reunions 2. Attribution of responsibilities within workgroups, satisfaction feedback from the beneficiaries of the workgroup’s services 3. Management of the resources acquired through common budget (rooms, furniture, equipment, shared spaces etc) 4. Online buy and sell possibilities connected to local networks, coordinated by secretaries/facilitators/delivery people 5. Satisfaction indicators for services provided by the community members to the members (cleaning, garden, personal tidying, maintenance) 6. Online support and documentation of various checklists, online-based, request forms. Our main assumption is that the tasks and needs deriving from the above functions will be covered from the communities themselves, while creating opportunities for larger partnerships with other communities, such as SMEs, entrepreneurs and activists. Our initial deployment target public for the prototype version of the application is a specific network of ecovillages (Smala - Ecopol). The plan of action consists of: a) co-designing the application by assigning concrete activities, roles, logistics, evaluations, services, based on its early prototype described above, b) validating the functions and evaluate its results in


specific pilots, c) measuring the entrepreneurial potential within the selected communities and d) promoting, at a later stage, the results within european and international ecovillages. Testing and deployment of the prototype could lead to the appropriation of the application as a collective, co-working platform. Following Silvestro’s conclusions, the ecovillage is intended to create from scratch a micro-society where each member will be able to discuss and voluntarily integrate the proposed social contract (Silvestro 2005). Further, the platform should organise groups and data in a way that will facilitate exchanges and transactions between the members of every group (community) but also at intra community lever.

3 Co-designing and co-developing the application As already highlighted, our approach comes with a collaborative impetus focusing on creating a positive space and experience for this to happen. We look to boost collaboration within communities and promote entrepreneurship opportunities in order to make co-living possible. Our collaborative approach is twofold: on the one hand, co-design the application with members of the Ecopol - Smala Living Lab through focused interviews described above and, on the other, present these requirements to a wider - open community both for their review and development (Social Hackathon). With the first wave of requirements listed above, we decided to test our collaborative approach openly from the very start of the implementation of the application and participate to the third edition of the Social Hackathon Umbria (SHU, 2018). The Hackathon was focusing on unveiling how digital competence, sense of initiative and entrepreneurship represent some of the most required competences by the labour market and, therefore, their development should be strongly promoted by the European Union for citizens of all ages and origins. The - Smala team actually won the b-work challenge of the SHU 2018 and received the prize for the “Best Digital Innovator for Entrepreneurship”. During the 48 hours Hackathon, teams choose to produce a pitch, a prototype or a product to be presented in front of an international jury. participated on the B- WORK challenge. The Social Hackathon Umbria collaborative process can be described as follows. Each team was composed by a member of the organisation who submitted the idea (the team leader - Ecopol Smala Living Lab in our case), three (3) hackers coding specialists, and four (4) support team members. The roles in the team were easily defined, which helped the workflow. The “hackers” worked on different parts of the app: while one constructed the “brain” of the app, a graphic designer gave it a face, while the third one made it compatible with portable devices (android and IOS). The support team members had a crucial role in the brainstorm phases and in the development of the content for the final product, while the team leader was responsible for presentations, delivering the pitches, meeting with the audience and answering to all brand/product representations needs.


We started out this process with an open exchange of ideas with the team. The team leader took the time to go over the first draft of the idea, the specific and concrete needs the app is trying to answer, and the main characteristics of “living and working in an ecovillage�. Each team member had the chance to question and contribute with their own ideas. This was a very critical moment as it was the first time the product idea was submitted to a brainstorm group exercise. This moment allowed to redefine the product and at the same time it became more realistic and doable in the time we had available (48h). After few hours of brainstorming, the idea was presented to a panel of experienced members of the jury. A seven-minute pitch was prepared and the jury gave their first feedback. This initial feedback from the jury allowed the team to better understand the criteria under judgement and better adapt to the requests. Frequent team meetings took place in order to distribute workload and tasks, to find solutions for problems and to promote the flow of ideas and emotions between the members of the group. During the two days in which the app was developed, members of the jury, audience and other important stakeholders including local politicians and members of other international organisations were involved by giving targeted feedback. They were also encouraged to visit the work space of the teams and check the work in progress. This moment resulted to a rich feedback with concrete questions and specifications. The team extrapolated new needs and new solutions to those needs enriching the final product.

4 Initial results In a nutshell, we competed against one other international team to build a mobile application promoting better communication and entrepreneurship attitude in ecovillages all over the world. As an overall evaluation of the Hackathon process, we consider that the collaborative methodology with the clear time frame and moments of feedback/pitching are an extremely useful environment for development of creativity and problem solving of very real issues of our 21st century society. Below, we present a more detailed description of the results of the common work between our team, developers and activists in place.


Figure 1. My community

In this section, the user becomes member of her ecovillage (“Ecovillagers”) with access to the private part of the ecovillage discussions and tasks lists and forums. Members are also able to see public posts on their ecovillage and others ecovillages. Friends of the ecovillage are all the other stakeholders of the ecovillage that want to participate in the PUBLIC discussions of the ecovillage (meaning that “ecovillagers” can also be “friends of the ecovillage” if they are interested in following other ecovillages besides their own ecovillage).


Figure 2. My community, my village

In this section, we can navigate through the different categories of discussions. This is an organisation of the discussions more or less by theme (to facilitate the organisation of the discussions). Examples of other themes are the maintenance of the common infrastructure, common budgets, rules of cohabitation and discussion in the topics, parties and events, projects of the ecovillage and others. Posts come with a set of metadata (description of the post, body of the post) and number of tasks are identified in the post. Different answers from other members of the ecovillage are possible and new tasks are added to the list of things to prepare in the checklist.


Figure 3. The Checklist

In the checklist section in the example, the author of the task agrees to do the task but needs help from other people. She adds a description of the task (small text, plus costs, timeline to do the task) and check the people who said they could help with this task. The colour code selected includes: • Vivid Green: task was completed with success. • Bluish Green: Task is open. It means that it still needs people to volunteer. • Red: the task is now closed and it was not completed successfully. • Grey: task was cancelled or deleted. This colour scheme allows an easy and informal diagnosis of the ecovillage’s activities.


Figure 4. Events and Global

If needed a Post can change its status from PRIVATE to PUBLIC (and vice versa), in order to reach a larger audience and become an Event to promote an activity. In this case, the post will be visible in the “Global” section of the application. In image 4, we can see all the PUBLIC posts of the different ecovillages we follow as “Friend of the ecovillage”. We can easily see the number of new notifications (unopened new answers) in each Topic.

5 Conclusions and next steps Following these initial results, the team is spending time in 2019 to secure the budget for implementing the code and the process of the proposed application. Our deployment target public will be the network of ecovillages of Smala - Ecopol with a plan of action that consists of: • Continuously co-designing and testing the next version of the application by assigning concrete activities, roles, logistics, evaluations, services to the Smala - Ecopol participants. • Validating the functions and evaluate its results in specific workshops. • Measuring the entrepreneurial potential within the selected communities. • Finding collaboration and funding opportunities to improve, develop and diffuse the application.


We remain confident that our application will be in full production and used in the Ecopol Smala Living Lab during 2019 - 2020 giving us new insights for this work.

References Bang, J. M. (2005). Ecovillages. A Practical Guide to Sustainable Communities. Edinburgh, Floris Books. Bierens de Haan, C. (2006). Entre écovillages et projets d’architectes, les écoquartiers. Urbanisme, (348), 41-44. Bondolfi, T. (2016). Ecopol : Labels et services pour les écovillages. 324p. Smala & Collection eCulture. Daly, M (2015). Practicing Sustainability: Lessons from a Sustainable Cohousing Community. In State of Australian Cities Conference 2015: Refereed Proceedings, edited by Paul Burton and Heather Shearer. State of Australian Cities Research Network. Dawson J. (2008). How Ecovillages Can Grow Sustainable Local Economies. Communities n°133, 2006. pp. 56-61. Imre K. (2009). Ecovillages: In Vitro Sustainability. World Futures, vol. 65 (5-6), pp. 365-371. Kitchen, L. and Marsden, T. (2011). Constructing Sustainable Communities: a theoretical exploration of the bio-economy and eco-economy paradigms. Local Environment: The International Journal of Justice and Sustainability, 16(8), 753-769. Nathan L. (2008). Ecovillages, values, and interactive technology: balancing sustainability with daily life in 21st century america. CHI '08 Extended Abstracts on Human Factors in Computing Systems, pp. 3723-3728. Silvestro M. (2005). Les écovillages comme stratégie holiste de développement durable et d’économie sociale. Centre de recherche sur les innovations sociales, UQAM. Social Hackathon Umbria (2018). Grande successo per la terza edizione 2018. Retrieved from Somekh B. (2010). The Collaborative Action Research Network: 30 years of agency in developing educational action research. Educational Action Research, (18)1, 103-121. Waugaman A. (2016). From Principle to Practice : Implementing the Principles for Digital Development. Washington, DC: The Principles for Digital Development Working Group, 76p.


Business model review for Living Labs: Exploring business challenges and success factors of European Living Labs

Justus von Geibler1*, Julius Piwowar1 and Linda Weber1 *Corresponding Author Institute for Climate Environment and Energy, Division Sustainable Production and Consumption, Germany 1 Wuppertal

Category: Research-in-progress Abstract Living Labs offer an open-innovation infrastructure for co-creation and product testing and gained increasing attention with regard to their potential to support sustainable innovation. However, many Living Labs face the challenge of financing their services, especially, when the business models focus on solving wicked problems such as urban transition and the future of health. Based on a desktop research and four qualitative interviews with active Living Labs, this paper explored experiences of Living Labs and their business model challenges as well as main future success factors. The findings demonstrate the need and opportunities to transition from public towards private funding, e.g., to clarify the Living Lab-as-a-service beyond technology showroom and innovation workshops; to be distinct from traditional R&D tools and consultancies and to place services within established innovation funding schemes, e.g., start-up vouchers. Furthermore, the results indicate challenges according to balancing the innovation process of flexibility and standardisation. Although iterative and agile processes are core values of the Living Lab, the innovation process needs some standardisation to allow efficiency gains and to comply with public regulations, e.g., by structuring the process and to include descriptions where and how to use Living Lab methodology as well as to clarify linkages to other established design approaches. The paper concludes that future research needs to better understand the linkages between entrepreneurship and design research, e.g., towards designdriven innovation and user experience design. Keywords: Living Lab as a service, Living Labs, business model, design-driven innovation


1 Introduction 1.1. Living Labs for innovation development Living Labs have been developed as a new approach for innovation development and (Greve et al., 2016; Ogonowski et al., 2015, Bódi et al., 2015; Ståhlbröst, 2013) gained increasing attention with regard to their potential to support sustainable innovation (Keyson et al., 2016; Ley et al., 2015; Liedtke et al., 2015). According to Brown (2009), innovations must solve the three key dimensions of desirability (what makes sense to people and for people), feasibility (what is functionally possible within the foreseeable future), and viability (what is likely to become part of a sustainable business model). Living Labs are a promising approach to address these three key innovation dimensions by placing people and users at the centre of the innovation process; users are actively engaged in the development process e.g. seniors interact and test innovative health solutions in their own homes (Bamidis et al., 2017; Konstantinidis et al., 2016). This useroriented approach is intended to increase acceptance and market success in the event of fundamental innovations or high market and technology uncertainty (Clausen et al., 2011; Geibler, Piwowar, Greven, 2019). Hence, Living Labs can be described as user-centred, open innovation ecosystems based on a systematic user co-creation approach, integrating research and innovation processes in real life communities and settings (EnoLL, 2019). Thereby, Living Labs provide an extensive open-innovation infrastructure and thus can be an effective approach of wicked problem solving and to address sustainability challenges (Liedtke et al., 2015; Geibler et al., 2014; Thienen et al., 2014). Wicked problems (e.g. Rittel and Webber, 1973) are not defined by a clear process for solutions (e.g. as tame problems in mathematics) but information and systems are confusing involving many decision makers and clients with conflicting values (Churchman, 1967). However, the economic sustainability of Living Labs still presents a critical issue due to their lack of continuity once the initial public funding has depleted (Grezes et al., 2013; Burbridge et al., 2017). There are only few success cases of turning Living Lab research into usable new products and services and uncertainty remains on what Living Labs actually do and contribute (Katzy, 2012). Mulvenna et al. (2010) refer to this issue as the “Achilles heel of living labs” and affiliate it to the absence of explicit business models and profitable partnerships. Yet, if Living Labs focus on realising commercial benefit of R&D and user-centric processes, they have a higher potential to become self-sufficient (Mulvenna et al., 2013). Katzy (2012) states that Living Labs can generate revenue from investors such as venture capitalists or industrial firms by providing opportunities to create their sustainable business model. That way, not only flows of information and benefits, but also flows of money can be identified to capture the value of the societal engagement with Living Lab stakeholders (Grezes et al., 2013). Hence, Mulvenna et al (2010) propose the combination of seed capital models, such as Y-Combinator or HackFwd, with Living Lab concepts, which produces a hybrid, combined engagement model, comprising the four-stages of ideation, cocreation, seed capital, and evaluation. Rits et al. (2015) argue that business model research should be included in the "Living Lab as a service" concept. So


far, only few studies have focused on the self-sufficiency of Living Labs. Nevertheless, there is a substantial need to address this challenge (Grezes et al., 2013). Against this background, this paper reviews business models of European Living Labs based on qualitative interviews. The research question is: What business challenges do Living Labs face and which success factors are relevant for their (future) revenue streams?

2 Research Methodology In order to identify business challenges and experiences of established Living Labs, the research process involved two main steps combining desktop research and inter- views. Step 1: Desktop research: Screening and selecting Living Labs for interviews Based on the European Network of Living Labs (ENoLL, 2017) and the Building Tech- nology Accelerator - Living Lab Network (BTA, 2017) different Living Labs were iden- tified based on two main criteria: a) The Living Lab is active in wicked problem solving and sustainability, e.g., urban transition or future of health. b) The Living Lab provides a website with information on a service offer and general characteristics (see Table 1).

The selected Living Labs were then screened based on general characteristics (see Table 1) and evaluated based on their service offerings (see Geibler, Piwowar and Greven, 2018 or table 2). The evaluation criteria were: (++) services are very visible on the website; (+) visible on the website; (-) not visible on the website. In total, 18 Living Labs were selected. Table 1. Summary of characteristics for the screening of Living Labs in Europe

Characteristics Name / Location Activity Institutionalisation Living Lab Driver Innovation theme / area of expertise Technology infrastructure and user pool

Examples Smart Kalasatama Living Lab Helsinki, Finland yes / no project based / institutionalised Industry / Research/ University; Public Institution Office Lab, Health Lab, Mobility Lab, Retail Lab etc. 3d printer, physiological sensors, cameras; wearables etc.


Table 2. Living Lab service characteristics for the screening (based on Geibler, Piwowar and Greven, 2018)

Service segment



Support to collect user feedback for prototypes in showrooms.

User studies

Support to analyse target user groups with quantitative and qualitative research methods.

Business model development

Support to generate and develop business ideas (models), e.g. within a workshop setting (business model ideation).

Stakeholder net- working and brokerage

Support to connect and engage with relevant stakeholders.

Co-design Co-prototype

Support to co-design innovative product concepts with relevant stakeholders and potential users. Support to co-design prototypes with relevant stakeholders and

UX testing and evaluation

potential users. Support to design, test and evaluate the user experience of products and services.

Motivational design

Support to design innovative product concepts, which consider intended user behaviour and the motivation of users (e.g. by gamification principles).

Sustainability assessment and evaluation

Support to analyse sustainability potentials of innovative product concepts.

Step 2: Exploring different business models of Living Labs based on interviews In order to further explore the business models of Living Labs expert interviews were carried out. The selection of the interview partner was based on the results of desktop research and availability: a) Coverage of comprehensive Living Lab services; b) Coverage of different Living Lab drivers (industry, research/ university, other public institutions); c) Representing a geographical spread (North, South, East, West Europe); d) Availability of a representative for an interview at the Open Living Days 2017 in Krakow.

A total of four interviews were carried out in Krakow (August 2017) and the interviewees were experts and practitioners in Living Labs (see Table 3). The interviews were held and recorded in English with a duration of 45 to 60 minutes.


Table 3. Overview of Living Labs represented by interview partners12 Code

Living Lab Affiliation

City, Country



Smart Kalasatama Living Lab

Helsinki, Finland

Imec Living Lab

Ghent and Antwerp, Belgium



Krakow Technology Park Living Lab


Active and Healthy Ageing Living Lab

Krakow, Poland

Thessaloniki, Greece

The exploration of business models and experiences utilised a semi-structured interview approach with guiding questions to discuss in an open manner and to allow for a detailed exploration (Mayring, 2002). The development of the interview questionnaire (see Annex 2) is based on the approach of the Business Model Canvas (BMC; Oster- walder and Pigneur, 2010), which has become one de facto standard for business model development. It defines a business model as the rationale of how an organisation creates, delivers, captures value, and provides a practitioner's tool to operationalise this. The interviews were transcribed and analysed through a qualitative content analysis (Mayer, 2015) with five predefined categories based on the interview guide: • Background (goals and organisational structures) • Main customers • Service offerings • Revenue streams and costs (public / private funding) • Lessons learnt (elements of success, challenges) Structure of the paper The remaining part of the paper is structured in the following way: The results section provides the results of the Living lab screening (desktop research) and the interviews focussing on the Living Lab business model. The discussion section summarises on success factors for a Living Lab business model. The report ends with a conclusion on the Living Lab business model.

3 Results Screening of LL and service evaluation (Step 1) In total 18 Living Labs were identified and evaluated based on their general characteristics (see Annex 1) and their services (Table 4).

Table 4. Services of identified European Living Labs (Living Lab selected for interviews are displayed in bold letters)

12 Names of the interviewees are kept anonymous as their confidentiality has been agreed upon. The

codes are used in the remaining paper to refer to the related interview.






Estonian Smart City LaB, Estonia






DOLL, Denmark





Adaptive Governance Lab, Ireland



Dublin Living Lab, Ireland



Urban Management Fieldlabs, Amsterdam


imec.livinglabs, Belgium



Botnia Living Lab, Sweden


Basaksehir Living Lab, Turkey


Bristol Living Lab, UK



Creativity/ Motivational Design Sustainability service


UX Testing/ Evaluation

Co-Prototype and technology development

Stakeholder networking and brokerage


Co -Design

Business model development

De andere Markt, Belgium

Living Lab, city, country

User studies














































































NEST, Switzerland











Green Living Lab, Netherlands











Marconia, Netherlands











City of the Future Living Lab, Italy




















Smart Kalasatama Living Lab, Finnland











Krakow Technology Park, Poland











THESS, Greece











Note: To assess the services offered, the websites were screened for direct (e.g., “we provide user tests in simulated environments”/ “a network of…”) and indirect (e.g., “we help you to innovate” /” bring you together with your clients”) information. As a result, the services are very visible on the website (++), visible on the website (+), not visible on the website (-)

Experiences on business model (Step 2)


This section describes the results of the interview regarding the following topics: background (organisational setting); revenue streams; main customers (target group); service offerings and lessons learnt (elements of success, challenges). Organisational settings The Living Labs represented by the interviewees are embedded in different organisational settings. The Living Labs are run as part of an innovation company [1]; non-profit research institute [2]; a public-private-partnership of a business innovation centre and a city municipal office [3]; and a unit of a university [4] (see Table 5). Table 5. Overview of organisational settings Living Lab Name

Smart Kalasatama Living Lab, Helsinki, Finnland

imec.livinglabs Ghent and Antwerp, Belgium

Krakow Living Lab, Krakow, Poland

Healthy Ageing Living Lab (ThessAHALL), Thessaloniki, Greece

Organisational setting Smart Kalasatama is a district based Living Lab coordinated by the City of Helsinki’s innovation company Forum Virium Helsinki. Kalasatama is a model district for smart urban development and a pioneer regarding the climate goals of the City of Helsinki. In 20142018, Kalasatama was developed as part of the Six City Strategy, which develops open and smart city services in the areas of the six largest cities in Finland (Helsinki, Espoo, Vantaa, Tampere, Oulu and Turku). The project has attracted over a hundred companies to engage in collaboration with the City and each other while also raising major national and international interest. The project was granted follow-up funding for 2018–2020 from the City of Helsinki Innovation Fund. The Imec Living Lab is a business unit of the non-profit research institute Imec. It is a research and innovation hub in nano electronics and digital technologies. The Living Lab unit includes seven people (user research) and cooperates with experts from the business model unit to fit user research into the business model logic. The Kraków Living Lab was developed in 2013 as a joint project of the business innovation centre “Krakow Technology Park” (KTP) and the Municipal Office of Krakow. The development was supported by the strategic partner Forum Virium Helsinki and the advisor Jarmo Eskelinen. The initial touch point of KTP to Living Lab methodology were activities in smart city innovation. ThessAHALL was founded in 2014 in context of a EU funded project on smart homes for elderly people and independent living. It is governed by the University of Thessaloniki, Laboratory of Medical Physics.


Revenue streams Living Lab revenue models are coming from public and business funding (see Table 6). The interview results showed that public funding is predominant [1; 3; 4], but future strategies also aim for an increase of private funding options. Table 6. Different revenue models in Living Labs Revenue model

Description EU (e.g. FP7, H2020, Interreg), national, local institutions (e.g. subsidising innovation vouchers) provide funding for Living Lab activities based on Public funding public objectives, e.g., promoting sustainability, performing research, (EU, national, local) industrial competitiveness [1; 3]. Start-ups and SMEs contract the Living Lab for innovation services and thus Business funding receive revenues [1; 2; 3; 4]. A hosting partner can realise revenues by, e.g., setting up a showroom to Hosting partner demonstrate their (future) products or organising visitor tours [1]. Scientific partners take part in research activities in the Living Lab Scientific partner infrastructure. These partners can host a full, partial location or be involved in the research performed at other locations [4]. Self-funded projects


Revenues are based on self-funded projects e.g. LLMCare (2017), FitForAll (2017) [4]. Organisations can become a member e.g. with a memorandum of understanding. Anybody can become a member, as long as they are involved in the research project objectives and thus could participate in events and receive information [3]. Non of the interviewed LL charges a membership fee.

Private funding is partly generated by positioning Living Lab Services within a traditional innovation scheme (accelerator, incubator) and as the logic of start-up vouchers. Thereby the Living Lab services are integrated in the language of business thinking. “The approximate proportions of the revenues are coming from public (90 percent) and private sources (10 percent). The aim is to shift the models to 50/50 in the next 2- 3 years.” [4]. “In future there will be an increase of private money for Living Lab services, but public money will remain the most important revenue stream” [3]. “The vouchers are funded by regional authorities and directed to start-ups, which are demanding small scale RD services. Companies only pay between 10 to 40 percent of the actual value of the voucher”. [3]. “The start-ups vouchers cover business and innovation development services worth max. 10.000 Euro and are released by public and private organisations. Interested start-ups are able to buy the voucher but get 70% discount of the vouchers value” [2]. Funding challenges


Nevertheless, the financial sustainability of Living Lab services is still very challenging: “The share of private funding for Living Lab services will increase but the services will never be economical feasible”. [3] None of the interviewees utilise industry sponsors or make use of shares of startups. “We have a company partner but without membership fee. As long as we have public funding we can't take any money” [1]. “The Living Lab does not have any ambition to buy shares of start-ups because the value of the service is below the amount to take shares from and public funding schemes we use do not allow us to take shares e.g. as other accelerators, which involve equity seed funding for their programs” [3]. Target group The Living Labs’ main target groups are start-ups [1; 2; 3], a limited number of larger organisations [1; 2; 3] and the public sector [2]. Furthermore, customers are linked to the research community [4] and patient association, which can be both customers and participants. Start-ups as customers may differ in their motivation depending on their project background: “We are working with start-ups in internal and external projects. In internal projects the Imec incubation team hires us, the Living Lab staff, and in external projects start-ups contact us directly. Mostly the start-ups from external projects are more motivated, active, and co-creative.” [2]. Furthermore, start-ups were characterised as the main target group because of their lean corporate structure and innovation culture: “Start-ups have a quick business development cycle because there is mainly just one contact person. Furthermore, the Imec Living Lab experienced that start-ups are very open, critical, and demanding. They are asking a lot of questions and expect quick responses. These attitudes are beneficial for an innovation process” [2]. In contrast, public organisations such as hospitals were characterised as very challenging because of the decision-making processes: “Greek hospitals receive funding by the ministry of health; they are not allowed to decide on the money and a Living Lab collaboration - it’s a central strategy. It takes a lot of effort to convince policy makers and government for Living Lab services in hospitals. Therefore, strategic decisions are primary directed to the private sector, who easier understands the benefits of the Living Lab” [4]. Service offerings


The interviewees point out the following unique selling points (USP) of a Living Lab: • Matchmaking with contacts of the innovation field, e.g. access to an (international) partner network of research institutions and big organisations and city representatives, city data [1]. • Understanding the usage context and access to space for real-world experiments, e.g. in contrast to (virtual) simulation under strict lab conditions and forecasting methods. This includes user studies, e.g. to better understand user habits and user practices in daily life as well as user tests with prototypes in the real-life setting (e.g. in private households or in a factory) [2; 3]. • Direct contact to potential users and access to group dynamics of cocreation workshops [2], e.g. in contrast to homogenous focus groups in traditional market re- search. • Providing a neutral innovation ecosystem, e.g. in contrast to competing accelerator programmes of large private organisations [1]. • Accelerating innovation projects due to agile and experimental processes “to fail fast and to learn fast, e.g. in contrast to traditional structures of larger organisations [1]. • Providing collaborative consultancy activities and research insights to create a solution together and not isolated, e.g. in contrast to traditional consultancies where one asks for the problem and then provides the solution or advice [2]. The services are operationalised in different ways and service packages, e.g.: • Living Lab services as a start-up programme, e.g. Agile Piloting13. This service lasts for 1-6 months [3] or 1-12 months [1] and aims to accelerate innovation projects. The investment for this service is comparably small with max. 8.000 Euros [1] or 10.000 Euros to 40.000 Euros. It covers e.g. one •

to two iterations and three co- creation sessions [3].

Living Lab services fit to other innovation services and schemes, e.g. “innovation challenges” [1] or innovation accelerator programmes [2]. A start-up challenge in- volves innovation development support (e.g. over 12 months14) and the final prize (e.g. 100.000 Euros). The innovation development is partly supported by the Living Lab services (max. 10.000 Euros; six months). Living Lab services cover a certification system of a testing process, e.g. to offer infrastructure for clinical trials and thus providing certification for smart health devices, which cannot be provided by individual organisations [4].

Challenge: communication and market positioning

13 Forum Virium Helsinki. Agile Piloting Drives Innovation in Smart Kalasatama (2018, November 22) Retrieved from 14 Nordic Innovation. (2017, November 8). The Nordic Independent Living Challenge. Retrieved from


Interviewees point out that they are challenged to appropriately communicate their added value and market position, e.g., Living Lab services compete with traditional research and development schemes as well as tools from universities. “Discussion on the value of Living Lab services with the regional authority was not easy because it meant to agree on equivalent value of traditional, university RD service. The regional authorities believed that Living Lab services are quite soft compared to the university services” [3]. Furthermore, the understanding and communication of the lab approach can be quite a challenge: “It is easier to communicate lab services at a physical “high tech house" in the university, but it does not cover the unique value of the Living Lab” [4]. Challenge: political regulations The Living Lab services are dependent on highly flexible processes to, e.g., cocreate, to allow for iterative innovation tests [3] and to be open for the unexpected [2]. However, political structures and laws can be barriers for such experimental activities in the real-world. “Unfortunately, under the inflexible polish public procurement law it is impossible to iterate and to co-create in deep tech in energy or mobility” [3]. The innovator needs to specify its intentions at the beginning and in detail and is involved in a call for tenders”. [3] Challenge: innovation process and methodology The Living Lab process is flexible and open but also needs structure and some standardisation to allow efficiency gains, e.g. to allow for a shared understanding, direction and orientation in the team. “I would develop a process list on how the methodology of the Living Lab works and follow that from the beginning” [4]

4 Discussing success factors for a Living Lab business model Based on the results presented, we derive potential success factors for viable business models of Living Labs and discuss practical implications. 1. Communicating a clear added value (USP) to funding partners and target groups. Living Labs could communicate their services beyond the provision of a pure technology infrastructure (“high-tech house”) to make their added value clearer. For example, Living Labs could illustrate the relevance of a thorough analysis of the real-world context with cases of prototype tests in private households or in factories. Furthermore, the demonstration of co-creation sessions engaging non-homogeneous user groups (technology lovers, critical users etc.) could indicate the benefit of lively and creative debates between the innovator and users. 2. Better understanding of potential customers (and start-ups). The Living Lab founders could spend more time to understand the needs of the market. This


could involve interviews or meetings with potential customers, e.g. at conferences. Potential customers are for example start-ups since they have a quick business development cycle and often lack of own R&D facilities. In this way Living Lab services could be better adapted to specific start-up needs, e.g. affordability (max. 10.000 Euros / 6-12 months). 3. Linking to funding schemes of third parties, e.g. through start-up vouchers. To increase financial stability Living Labs could offer their services aligned to existing funding models of third parties. Examples might be innovation or start-up vouchers, offered in accelerator or start-up programmes of public or private funding organisations. In this way, Living Lab services could be more recognised and co-financed by third parties. Furthermore, Living Labs could get better access to public funding sources by clearly defining indicators on how their activities are supporting public policy objectives, e.g. supporting the development of sustainability business models. 4. Building a start-up success story. To become recognised, the impact of Living Lab activities should be visible and provide a substantial contribution to the success of an organisation, e.g., a Living Lab helps a start-up to gain their first users and to increase employees from two to forty within two years. Based on a good start-up success story, the Living Lab can attract other customer segments such as SMEs, larger organisations and the public sector. 5. Multidisciplinary team: Living Lab teams trust their Living Lab methodology with a hands-on mentality. The Living Lab experts (user researchers) are part of a multidisciplinary team including business model experts and technology experts, so that there are no technical or disciplinary silos but reasoning from and with a users’ perspective (see Figure 1). 6. Involving existing design and innovation approaches: Although the Living Lab approach builds on a creative and highly flexible process, there is the need for standardised processes to allow efficiency gains. Established design-driven approaches could be linked to the Living Lab approach to enable appropriate structures, e.g. human centred design (Norman, 2013, see Figure 2), or visions in product design (Dijk and Hekkert, 2014). Furthermore, the understanding of connections and relationships of design and innovation tools could further structure the analytical and creative processes, as for example suggested by Echternach et al. (2016). 7 Convincing regulation authorities for flexible innovation experiments in real life: Living Lab services are dependent on highly flexible and iterative processes, e.g. to co-create and experiment with innovations (“to fail fast and to learn fast�). Regulations, public procurement law and other political conditions could better allow for flexible processes and possibilities to rearrange parameters. These adaptive capacities facilitate creativity and a process of learning by doing; being open for the unexpected.


Figure 1. Living Lab competencies and way of reasoning based on design thinking innovation model (adopted from Brown, 2009).

Figure 2. Structure of the iterative process as well as user and expert involvement (based on Norman, 2013 and Meurer et al., 2015) Note: the involvement illustrated with the different boxes (light/ dark grey) is symbolic and does not represent results.


5 Conclusion and future research Based on a desktop research and four qualitative interviews with active Living Labs, this paper explored experiences of Living Labs and their business model challenges. Furthermore, potential success factors are suggested and discussed. Due to the limited number of interviews the results and suggested success factors should be interpreted with care. Still, a number of conclusions can be drawn. The findings demonstrate that the Living Labs represented in the interviews search for new funding opportunities coming from private sources. To support this transition, Living Labs should communicate the Living Lab-as-a-service more clearly. Thereby the Living Lab can be better positioned on the market and distinguish itself from traditional R&D tools or consultancies. Furthermore, Living Lab business models could benefit from offering services directed to start-ups because of their agile innovation culture and their lack of own R&D facilities. In addition, collaborations with start-ups could demonstrate the impact and substantial role of the Living Lab service, e.g. to acquire first users. Accordingly, success stories could make the Living Lab activities visible and meaningful, which also helps to attract other customer segments such as SMEs, larger organisations and the public sector. Finally, Living Labs are challenged with balancing flexibility and standardisation regarding the innovation process. Standardised processes could promote efficiency gains and compliance with regulations. The findings suggest following a structured Living Lab process with descriptions on where and how to use Living Lab methodology. In addition, this could involve linkages and methodologies coming from known, approved design approaches, e.g., human-centred design, design thinking and user experience design. In conclusion, future research could deepen the results based on a broader empirical basis, and test the hypotheses based on more interviews and more Living Labs analysed. Additionally, linkages between entrepreneurship and design research could be explored e.g. to support design-driven innovations and strengthen Europe’s competitiveness and promote collaborative learning.

References Bamidis PD, Konstantinidis EI, Billis AS, & Siountas A. (2017). From e-homes to living labs: founding and organising the Greek active and healthy ageing living lab (Thess-AHALL) and its networked services. Hell J Nucl Med Suppl, 20(2): 112-125. Binder, T., Brandt, E., Halse, J., Foverskov, M., Olander, S., & Yndigegn, S. (2011). Living the (co- design) Lab. Nordes, (4). BoĚ di, Z., Garatea, J., GarciĚ a Robles, A., & Schuurman, D. (Eds.) (2015). Living Lab Services for Business Support and Internationalisation. ENoLL. Brown, T. (2009). Change by design: How design thinking Transforms Organizations and Inspires Inno- vation. HarperBusiness.


BTA. (2017). EIT Climate-KIC: Building Technologies Accelerator. Retrieved from https://bta.climate- Burbridge, M., Morrison, M. G, van Rijn, M., Silvester, S.; Keyson, D., Virdee, L., Baedeker, C., Liedtke. C. (2017). Business Models for Sustainability in Living Labs. In: Keyson, D. V., Guerra-Santin, O., & Lockton, D. (Eds.). Living Labs: Design and Assessment of Sustainable Living. 391-403. Springer, Cham. Churchman, C. W. (1967). Wicked problems. Management Science, 14 (4), B141-B-142. Clausen, J., Fichter, K., & Winter, W. 2011. Theoretische Grundlagen für die Erklärung von Diffusions- verläufen von Nachhaltigkeitsinnovationen – Grundlagenstudie. Verbundvorhaben im Rahmen der BMBF Bekanntmachung „Innovationspolitische Handlungsfelder für die nachhaltige Ent- wicklung“ im Rahmen der Innovations- und Technikanalyse. Borderstep Institut für Innovation und Nachhaltigkeit gemeinnützige GmbH, Berlin. Echternacht, L., Geibler, J. v., Stadler, K., Behrend, J., & Meurer, J. (2016). Methoden im Living Lab: Unterstützung der Nutzerintegration in offenen Innovationsprozessen (Entwurf Methoden-hand- buch). Arbeitspapier im Arbeitspaket 2 (AS 2.2) des INNOLAB Projekts. Wuppertal Insti-tut für Klima, Umwelt, Energie, Wuppertal. ENoLL. (2019). What are Living Labs. Retrieved from EnOLL. (2017). Network Living Labs. Retrieved from European Commission (2015). Implementing an Action Plan for Design-Driven Innovation. Commission Staff Working Document. FitForAll (2017). Online at (available online at 2017). Forum Virium Helsinki. Agile Piloting Drives Innovation in Smart Kalasatama (2018, November 22) Retrieved from Geibler, J. v., Erdmann, L., Liedtke, C., Rohn, H., Stabe, M., Berner, S., ... & Kennedy, K. (2014). Ex- ploring the potential of a German Living Lab research infrastructure for the development of low resource products and services. Resources, 3(3), 575-598. Geibler, J. v., Erdmann, L., Dönitz, E., Stadler, K., & Zern, R. (2018). Roadmap Living Labs für eine Green Economy 2030. Kurzfassung. Broschüre zum Arbeitspaket 7 (AP 7.4) im INNOLAB Pro- jekt: „Living Labs in der Green Economy: Realweltliche Innovationsräume für Nutzerintegration und Nachhaltigkeit“. Wuppertal Institut für Klima, Umwelt, Energie, Wuppertal. Geibler, J. v., Piwowar, J., & Greven, A. 2019. The SDG-Check: Guiding Open Innovation towards Sus- tainable Development Goals. Technology Innovation Management Review, 9(3): 20–37.


Greve, K., Martinez, V., Jonas, J., Neely, A., & Möslein, K. (2016). Facilitating cocreation in living labs: The JOSEPHS study. 23rd EurOMA Conference. Trondheim, Norway. Grezes, V., Fulgencio, H., & Perruchoud, A. (2013). Embedding business model for sustainable collab- orative innovation in African Living Labs. 2013 ISTAfrica Conference & Exhibition (pp. 1-9). IEEE. Hassenzahl, M. (2010). Experience Design: Technology for All the Right Reasons. Morgan and Claypool Publishers. Hekkert, P., & Dijk, M. V. (2014). ViP (Vision in Design). IDEO. (2017, November). Our Approach: Design Thinking. Retrieved from Katzy, B. (2012). Designing Viable Business Models for Living Labs. Technology Innovation Management Review. 19-24. Keyson, D. V., Guerra-Santin, O., & Lockton, D. (Eds.). (2016). Living Labs: Design and Assessment of Sustainable Living. Springer. Konstantinidis, E. I., Billis, A., Bratsas, C., Siountas, A., & Bamidis, P. D. (2016). Thessaloniki Active and Healthy Ageing Living Lab: the roadmap from a specific project to a living lab towards open- ness. In Proceedings of the 9th ACM International Conference on PErvasive Technologies Re- lated to Assistive Environments (p. 73). ACM. Liedtke, C., Baedecker, C., Hasselkuß, M., Rohn, H., & Grinewitschus, V. (2015). User-integrated inno- vation in Sustainable LivingLabs: an experimental infrastructure for researching and de-veloping sustainable product service systems. Journal of Cleaner Production, 97: 106 – 116. DOI: 10.1016/j.jclepro.2014.04.070 LLMCare (2017) Online at (available online at 2017). Mayer, I. (2015). Qualitative research with a focus on qualitative data analysis. International Journal of Sales, Retailing & Marketing, 4(9), 53-67. Mulvenna, M., Bergvall-Kåreborn, B., Wallace, J., Galbraith, B., & Martin, S. (2010). Living labs as en- gagement models for innovation. In eChallenges e2010 Conference (pp. 1-11). IEEE. Mayring, Philipp (2002). Einführung in die qualitative Sozialforschung, Eine Anleitung zu qualitativem- Denken (5th ed.). Weinheim: Beltz. Norman, D. A. (2013). The Design of Everyday Things. Human Factors and Ergonomics in Manufactur- ing (Vol. 16). Norman, D. A., & Verganti, R. (2014). Incremental and radical innovation: Design research vs. technol- ogy and meaning change. Design Issues. 30 (1), 78-96. Nordic Innovation. (2017, November 8). The Nordic Independent Living Challenge. Retrieved from Ogonowski, C., Jakobi, T., Stevens, G., & Meurer, J. (2015). Living Lab As A Service: Das Living Lab als Dienstleistungsbaukasten zur Nutzer-zentrierten Entwicklung und Evaluation innovativer Smart Home Lösungen. In Mensch & Computer Workshopband (pp. 701-711).


Osterwalder, A., & Pigneur, Y. (2010). Business model generation: a handbook for visionaries, game changers, and challengers. John Wiley & Sons. Rits, O., Schuurman, D., & Ballon, P. (2015). Exploring the benefits of integrating business model re- search within living lab projects. Technology Innovation Management Review, 5(12), 19-27. Rittel, Horst W. J., & Webber, Melvin M. (1973). Dilemmas in a General Theory of Planning. Policy Sciences. 4: 155–169. StühlbrÜst, A. (2013). A living lab as a service: creating value for micro-enterprises through collaboration and innovation. Technology Innovation Management Review, 3(11). Thienen, J. P. A. von, Meinel, C. & Nicolai, C. (2014). How design thinking tools help to solve wicked problems. In H. Plattner, C. Meinel and L. Leifer (eds.), Design thinking research. Building inno- vation eco-systems (97-102). Berlin: Springer.


Annex 1: Selected leading Living Labs in Europe

Technical Infrastructure and user pool


De andere Markt, Genk, Belgium


Estonian Smart City Lab, Tartu, Estonia


Unemployment, future of work Smart city, energy, health, social welfare ICT

Mobile printing press


300 test users


DOLL, Glostrup, Denmark

Smart city, lighting

None/ city grid



Adaptive Governance Lab, Limerick, Ireland

Public spaces

information not available



Dublin Living Lab, Dublin, Ireland

Clean municipal tech (lighting, EV, buildings, waste, water)

information not available



Urban Management Fieldlabs, Amsterdam, Netherlands

Resilient communities

Field labs in different city districts



imec. livinglabs, Ghent, Antwerp, Bel- gium Innovative digital solutions

21.000 test users



Botnia Living Lab; LuleĂĽ, Sweden

Future ICT

information not available



Basaksehir Living Lab, Istanbul, Turkey

Smart living, smart home

3D printer, design table, solar, fiber optic internet



Bristol Living Lab, Bristol, UK

5500 households



NEST, DĂźbendorf, Switzerland

Smart living & office

Modular building for testing innovations built on-site



Green Living Lab, Amsterdam, Netherlands

Healthy urban living


Marconia, Rotterdam, Netherlands


Digital needs for local citizens








Biomeiler, ecological toilets, outdoor research lab


Architecture, sustainable water, energy, waste systems

information not available



City of the Future Living Lab, Milan, Italy

eHealth, smart living

information not available




EVOMOBILE, Paterna, Italy

Electric mobility

Several charging stations, electric vehicles



Smart Kalasatama Living Lab, Helsinki, Finland

Smart city innovation

Small Lab, Field lab in city district; engaged 800 users



KTP Living Lab, Krakow, Poland

Smart city, creative industry in- novation

Multimedia and field lab, + in city districts; in industries


AHA Living Lab Thessaloniki, Greece

Health innovation, Smart home for the elderly

Health lab and field labs in private households



Other Public Institutions

Area of expertise



Urban transition



LL Driver


X x



Annex 2: Interview questionnaire Thank you for taking the time to for this interview. The aim of this interview is to explore the different business models of living labs in Europe. The interview is conducted within a publicly funded project of the "Knowledge and Innovation Community on Climate" (Climate-KIC). I. PORTRAIT: LIVING LAB GOAL AND ORGANISATIONAL SETTING Please describe in a few sentences your Living Lab and your role. a) What are the specific goals and objectives of the Living Lab? Is sustainability an objective? What type of innovations do you mainly support (incremental, radical disruptive)? b) What is the legal entity and organizational structure? What are related benefits and challenges? Who was the founder of the Living Lab? c) What is the founding year, size (m²) and number of employees?

II. LIVING LAB VALUE: SERVICES AND REVENUES: Please describe your services and revenues. a) What are your main service and technological equipment offerings to your clients? Are there plans for future services? b) Who can make use of the Living Lab services? (Which pricing models do you have?) c) Is membership needed? Does the fee vary between members? d) What are the main sources of revenue / funding for the Living Lab? e) What approximate proportions of the budget arrive from public (national, subnational, international (e.g. EU) private and other sources? What are improvements to for this ratio?

III. COSTS: MAIN BARRIERS FOR SUCCESS Please describe your main barriers of the LL business model. a) What are your main three advices for LL founders? b) If you would open a LL again what three things you would do differently today? c) What are your main costs/ investments? What are improvements would you like to improve costs? Innovation pipeline not filled?


Facilitate innovation and collective intelligence through play Yves Zieba1 and Isis GouĂŠdard1

1 IsYnnov,


Category: Innovation Papers Abstract While companies, universities, citizen and governments become aware about the Sustainable Development Goals, they are confronted with challenges as well. How to set the goals, how to agree on priorities, how to convince everyone, how to mobilise employees or advocates? That is where our specifically designed methods and gamified tools help stakeholders turn SDGs intentions into action plans. This article relates our experience and gives a taste of our magic recipe and ingredients. Simple game rules and an inclusive climate of trust; openly oriented towards co-construction. These basic principles are the fundamentals of our innovative, inclusive and participatory approach based on the establishment of a permanent dialogue between populations and technical agents, on mutual respect and the principle of partnership, as well as on the recognition of local know-how. We use the concept of boundary objects as a foundation for the pursuit of a common goal and help to minimize or avoid conflicts. Our playful approach combined with our games made of natural materials allows us to highlight the multi-sensory dimension of the experience we offer. By stimulating all the senses, everything makes sense! Keywords: Boundary objects, Collective innovation, Collective intelligence, Digital fabrication revolution, Sustainable Development Goals, Inclusive and participatory approach, Living Lab tools, Natural Resources, Economic and social value creation, Mission Innovation, Gamified tools and technics, Inclusive mindset, Co-creation and co-construction, Quadruple helix, Multi-sensory experience


1 Introduction Our experience and knowledge are shared from Geneva, Switzerland. Our dream: to nurture collective intelligence in the service of the Sustainable Development Goals on a daily basis! 1.1 The SDGs (Sustainable Development Goals) have been gamified! We are entrepreneurs passionate about human interaction, agility (defined as groups of practices for managing and implementing projects with high stakeholder involvement and responsiveness), collective intelligence and innovation. We have partnered to embark on the adventure of facilitating co-creation, change and collective innovation in Geneva, Switzerland. Collective intelligence, according to Malone & al (2010), is built slowly and is experiential. As humble practitioners, we strive to create the conditions for others to let themselves be transformed. Applying Living Lab approaches and tools (DubÊ & al, 2014) combined with play and boundary objects, we help them to accept to open themselves to questions, their own and others'. New individual and collective benchmarks are emerging, creating a fertile ground for innovation. We apply our collective intelligence methods to innovative projects through an approach that allows us to bring out new, sustainable solutions that are better adapted to the complexity and constant changes of our world. We are co-constructing serious games aimed at facilitating communication, creativity and expression to help unleash the power of collective intelligence to co-create the future. They allow the serious dimension to become more attractive by providing interactivity, a set of rules and some playful objectives. That’s the reason why we oriented our action, our resources and our methods on games, on the pleasure of playful interaction that allows for the construction of shared visions, group animation, team facilitation, governance and deriving action plans. For further reading, Rieber & al (1998) gave an interesting insight on the value of serious play. Both deeply convinced by the necessity to act on the UN's Sustainable Development Goals (SDGs:, we apply our methods of facilitating collective innovation through games to opening up reflections on these themes. Our objectives range from raising awareness only to supporting the institution to identify which SDGs are most relevant in its context and deriving a roadmap that can be activated and supported by its management to take concrete actions. The first element on which our games are based on the SDGs is a 20-sided dice, an icosahedron, which we co-created (Payne & al, 2008) in partnership with SDG Solution Space and published under a Creative Commons license. 17 of the faces correspond to the SDGs and 3 faces are customizable according to the needs of the game / institution. In particular, we use it to launch the debate and raise awareness of the SDGs among stakeholders and it allows us to gently approach the serious playful world we want to bring to life.


Figure 1.

Then, we deploy our range of games and methodological know-how according to the context and can also co-construct the experience with our clients when it makes sense. Our success factors? Simple game rules and an inclusive climate of trust; openly oriented towards co-construction. In this context, using transposition exercises or games calling on the participants' personal experiences, we encourage the use of "I" aiming to focus on the identity and values of the stakeholders. These basic principles are the fundamentals of our innovative, inclusive and participatory approach defined as a dynamic process in the sense that it is evolving in time given the local specificities and conditions. It is based on people's knowledge and perception of their environment and the interaction of the various elements involved in the management of its specificities. The participatory approach is based on the establishment of a permanent dialogue between populations and technical agents, on mutual respect and the principle of partnership, as well as on the recognition of local know-how. We also use the concept of boundary objects which are defined as a tool for mutual understanding and cooperation that allows different heterogeneous social worlds to communicate/collaborate through reconciliation around the same concepts in different worlds. We use them as a foundation for the pursuit of a common goal and help to minimize or avoid conflicts. This term would have first appeared in a study by Suzan Leigh Star and James Griesemer (1989). Our usage of boundary objects in innovation games fosters connexions between participants and facilitate the emergence of collective intelligence. They are mostly made of natural materials, locally produced or even created by the participants themselves! We are sometimes asked why we have chosen to produce games with objects locally. It is true that in the digital age, it would have been convenient to create online games or smartphone applications. We prefer the use of objects because


they allow several people to play together to achieve the same objective: whether it is a pawn, a voting token or an origami made by the participants themselves, the time spent appropriating, or even designing, the object makes it possible to initiate dialogue and to propose a simultaneous approach at the physical and intellectual levels. To facilitate debates and unleash collective intelligence, to get everyone to express themselves and capture as many good ideas as possible from a group, there are various methods. But through boundary objects, stakeholders have the possibility to pass the object and therefore symbolically the focus / right to speak, which invites each and every one to participate and express themselves in a soft and efficient way. Just like the “talking stick� in holocracy (governance organization system based on the formalized implementation of collective intelligence), this allows us, as facilitators, to involve everyone, even the most shy or resistant who would tend to hide at the back of the room! What else does our approach bring? By stimulating all the senses, everything makes sense! Our playful approach combined with our games made of natural materials allows us to highlight the multi-sensory dimension of the experience we offer. Indeed, touch is one of the senses that we generally stimulate little, and the physical body allows us to deeply anchor experiences. A form of communication that requires both hemispheres is richer in codes and interpretations. Experiencing concepts through the body in addition to verbal activities refers to all the structures of the brain and makes communication more obvious.

Figure 2.

In the field of the transmission/assimilation of knowledge, we draw inspiration from the work of Maria Montessori ( 167406-maria-montessori-et-sa- vision-de-l-enfant) which, like so many other works, tends to prove that teachings are more easily and deeply assimilated when


the body is associated with reason. If we add the noise of the machines, the odours of smoke or chemical treatment (non-toxic!), several senses are also stimulated when passing through a Fablab and the experience remains in our memory longer. 1.2 Two levels of interaction for the most participatory governance possible Because we want to capture information from people from a wide variety of backgrounds, with different skills, going through the object allows us to create a link. With our games, we act on several levels. We first call on volunteers to give them the freedom to express themselves. This is our first level. Generally, many participants take this opportunity to express an idea, share a positive or negative experience. At the second level, we invite participants to vote or rank the ideas, opinions or opinions expressed; this helps to keep everyone's attention and detect common features between the votes, in order to provide a first interpretation of the results. We also offer fewer objects than there are participants to observe the social interactions that are taking place. From the feedback we have gathered so far, what comes back frequently is the satisfaction of having listened to their peers and of having been able to express themselves / to be listened to / to interact in community. The empathy thus facilitated makes it possible to forge a bond and initiate a dialogue of the type "and you, how did you cope with this situation? ". This is what we experienced during the Open Living Lab Days in August 2018 in Geneva. We proposed two games around the Sustainable Development Goals, in particular SDG #5: "gender equality" and SDG #11: "sustainable cities and communities". With the first, in the form of a traditional adapted "Snakes and Ladders", we opened the debate on opportunities and constraints related to gender equality. Despite the short time available and the diversity of the profiles present, the examples were profound and personal and we discovered, for example, that women and men diverged in their prioritization of the elements identified when we gave them a limited number of tokens to use to support an idea. Thanks to our approach, connections were forged, ideas unleashed and concrete conclusions drawn. The second, oriented as a change management exercise, aimed to open up reflection and sharing of experience on the mobility issues of the future. Participants from all over the world were able to share in a fun way while becoming aware of the difficulty of taking the necessary actions to achieve a goal when one has invested body and soul in a personal construction project, however simple it may be.


Figure 3.

The approach raised deep reflections associated with loud laughter!

1.1 Our contribution to the SDGs on a daily basis Finally, to have a real and positive impact in ecological, economic and societal terms (Allingham & al, 1975), we have chosen to make our games using local materials and natural resources. In this way, we also hope to inspire the communities with whom we interact in the context of collective intelligence. That's why we naturally turned to the nearest Fablabs : a contraction of the words Fabrication (Manufacturing) and Laboratory (Experiments), the Fablab (Gershenfeld, 2012) is an affordable place, open to the public, providing its users with the technical, technological and human resources (machines, tools, software, processes, know-how, mentors) necessary for the design, optimization and repair of all kinds of objects. A Fablab is also and above all a place of sharing, where members are in turn beneficiaries and contributors, where experiences are shared in order to optimize the global potential for innovation.

2 Conclusion It is with great joy that we have, in these few lines, shared some elements of our dream: to nurture collective intelligence in the service of the Sustainable Development Goals on a daily basis. Through our Living Lab approaches combined with specifically designed methods and gamified tools, we help stakeholders turn SDGs intentions into action plans. Simple game rules and an inclusive climate of trust; openly oriented towards coconstruction; are the fundamentals of our innovative, inclusive and participatory


approach. Moreover our games made of natural materials allows us to highlight the multi-sensory dimension of the experience we offer. We are starting locally with the ambition to scale globally!

References DubĂŠ, P., Sarrailh, J., Billebaud, C., Grillet, C., Zingraff V. Kostecki, I. (2014). Le livre blanc des Living Labs. Edition SAT Montreal. Suzan Leigh Star and James Griesemer (1989). Social Studies of Science, Vol. 19, No. 3 (Aug., 1989), pp. 387-420. Sage Publications. Neil Gershenfeld, (2012), How to Make Almost Anything, The Digital Fabrication Revolution, Vol. 91, No.6 (Nov., 2012), Foreign Affairs. Thomas W. Malone, Robert Laubacher and Chrysanthos Dellarocas (2010), The collective intelligence genome, Vol.51, No 3, MIT Sloan Management Review. Rieber, L. P., Smith, L., & Noah, D. (1998). The value of serious play. Educational Technology, 38(6), 29-37. Allingham, M.G. Zeitschr. f. NationalĂśkonomie (1975) 35: 293. Economic power and values of games Payne, A.F., Storbacka, K. & Frow, P. J. of the Acad. Mark. Sci. (2008) 36: 83. Managing the co- creation of value.


Launch Process of a Living Lab and Required Leadership for Practitioners Masataka Mori1 and Kyosuke Sakakura2 1

2 Tokyo

Miratuku, Japan City University, Japan

Category: Full Research Abstract This research aims to clarify the process to set up living labs and required leadership for its practitioners regardless of social condition each country or community has. Using Forum Virium Helsinki in Finland, High Tech Campus Eindhoven in the Netherlands and Living Labs Taiwan as a case study, data were collected through web surveys and interviews and analysed with Grounded Theory Approach. As a result, research shows three phases of launch process (launch, foundation and involvement) and eight components ( theme setting, ecosystem formation, co-creation approach, funding and framework, places and opportunities, media and transmission, citizen-based projects and leadership are essential to set up a living lab, and five principles with sixteen actions(cocreation, empowerment, exploration, open and fair, and reflection)are required for leaders, which we call “orchestratorship”. Keywords: process of launch, launching, components, leadership, orchestration


1 Introduction Historically, Japan has achieved economic growth and social formation through industrial development. As Japan does not have abundant natural resources, it has achieved economic development by importing the raw material, processing the products and exporting them to abroad in the textile products, steel and automotive industries. Traditionally, these manufacturing companies have been closed to avoid the leaks of ideas and technology. Along with the transition from the industrial society facing the limit of closed development, there has appeared the development and creation beyond the organisational frameworks. This phenomenon is the open innovation advocated by Henry Chesbrough (Chesbrough, 2003, xxiv). Companies focus on making the relationship with other stakeholders such as creating innovation hubs, hosting acceleration events or using co-working spaces as the satellite office. This dynamic movement is also being focused by the Ministry of Economy, Trade and Industry as a driver for national growth (Open Innovation White Paper, 2018). With rising expectations for open innovation, living labs have also attracted much attention from many industries. Since the appearance of first living lab in Japan a few years ago, there are approximately 44 living labs at present. In 2017, Future Centre Alliance Japan, as a flagship organisation to accelerate crossborder knowledge creation, and European Network of Living Lab formed a partnership, which helped to promote living lab activities in Japan (Future Centre Alliance Japan, 2019). However, there are still many challenges in launching a living lab in Japan such as the characteristics of Japan as an industrial country, the national character of the people and tolerance to open innovation. One major characteristic is that Europe and Japan have different social condition, historical background and cultural contexts. Although many of the researches has been done for living labs, there are many studies dealing with cases and social situations in Europe as research and practice are leading in Europe. Therefore, it is necessary to do the research that can be diverted into other regions without being too dependent on the European context. This research aims to identify the phases of process and components to launch a living lab and the leadership required for a leader to orchestrate the ecosystem. Focusing on Finland, the Netherlands and Taiwan as a case study, we have collected data by web-based survey and interview and analysed data by Grounded Theory Approach. One outcome is three phases of launch process and eight components to launch living labs. The other outcome is the five principles and sixteen actions required for a leader who orchestrates the ecosystem. Still, since this research has only started two years ago, we introduce the future avenue of this research to conclude this paper.


This research has just started. The purpose of this research is to conduct case studies mainly in Europe, and to create a common framework that does not depend too much on culture and social background. We believe that adding culture and social background is essential for launching a living lab in various places. Therefore, the next step is to apply a framework in the context of Japan.

2 Previous research Since ENoLL was founded in November 2006, many studies have been conducted to support the better and deeper understanding of the concept. ENoLL Projects Portfolio published by ENoLL in 2018 introduces the six common features of a living lab: active user involvement, orchestration, real-life setting, multi-stakeholder participation, multi-method approach and co-creation (PROJECT PORTFOLIO 2018 - 2019, 2019). Another research based on the article searching of 851 published documents illustrates the eight facets of living labs such as real-life environment, challenges or sustainability (Hossain, Leminen & Westerlund, 2018). The question that still remains is why it is not easy to launch living labs in Japan. We have done the previous research and reached to the hypothesis that many researches done so far mainly deal with the cases based on social situation, historical background, cultural context, and natural environment in Europe. It could be said that the development of living labs has progressed with such conditions of Europe as the standard since living labs were first born in Sweden. Furthermore, there is no research focusing on launch and development processes. This is probably due to the fact that information sharing and discussions on the launch and development of living labs are taking place at events held at Open Living Lab Days and elsewhere, and there is no need to conduct research in Europe. The development of living labs has reached a turning point that it has been spreading from Europe to other regions and scaling up from local to global. This innovative method of working with various stakeholders to develop products or services is expected to be more increasingly needed in many areas in the future. Therefore, it is a preferable situation that living labs can be launched without being too dependent on the social situation and culture background each community has. There may be may social characteristics or cultural features understood as common in Europe but not in different situation, which often happens when it comes to the launch of living lab.

3 Methodology 3.1 Research question In order to support the launch of living lab, two research questions have been set up. The first focused on the necessary components to launch a living lab in the chronological order. It would be possible to launch and grow a living lab more smoothly if the practitioners know the necessary components and efforts in line with the time axis. The second is the required leadership for practitioners.


Harmony with ecosystem stakeholders is one of the key features of living labs so that focus is put on the leadership required for orchestration. 3.2 Case selection It is essential to select countries with different backgrounds in order to understand the components needed for a launch that are not too dependent on social context or cultural context. Therefore, are chosen Finland from Nordic country as the origin of a living lab, the Netherlands from Europe as the next wave and Taiwan from Asia as the developing region for a living lab. Considering the exposure and external evaluation of ENoLL and other paper, each living lab is selected: Forum Virium Helsinki from Finland, High Tech Campus Eindhoven from the Netherlands and Living Labs Taiwan from Taiwan. Forum Virium Helsinki is the City of Helsinki innovation company with the mission of making Helsinki the most functional smart city in the world ("What does Forum Virium Helsinki mean to the city? - Forum Virium Helsinki", 2019). One of the remarkable projects is called Smart Kalasatama started in 2013 to re-create Kalasatama as the model district of smart city development. Citizens, companies and public sectors co-creates the agile projects to realise a smart city in Kalasatama utilising the living lab as methodology. High Tech Campus Eindhoven is the campus located in Eindhoven, the Netherlands where more than 185 companies and institutes, and 12.000 researchers, developers and entrepreneurs work on developing future technologies and products. Started as the core laboratory of Phillips for its national R&D activities, it was sold to Ramphastos Investments from Philips so that the Campus has been utilised to accelerate the open innovation beyond the borders of companies, research institutes, citizens or academia ("High Tech Campus Eindhoven: Campus History", 2019). Living Labs Taiwan is one of the projects carried out by the Institute for Information Industry (III) in Taiwan which was established in 1979 as a NonGovernmental Organisation (NGO) under the partnership of public and private sectors. The project is called “Integration of Wearable Devices and Personal Health Records�. It aims to enable senior citizens to manage their health condition more easily by themselves with ComCare platform, a service that combines wearable devices and IoT. This project was awarded by ENoLL for the second year in a row in 2006 and 2007 ("Taiwan III Living Lab", 2019). 3.3 Data collection Data collection was mainly conducted by web survey and interviews. First, we collected the basic information, activities and events of each living lab from websites, journals, papers or web articles. Gathered information is tagged considering the contents such as citizen participation or media exposure, and is put in spreadsheet in chronological order. Second, we have conducted a semistructured interview based on gathered information by web survey. In the interview, we asked about the parts that could not be obtained in the web survey and the parts that should be understood in detail. Questions for components and process for launching a living lab focus on actions and events that are impactful


and inevitable for each living lab such. For the questions of leadership, the focus was put the significant values or actions leaders should take such. Selection of interviewee was limited to the person who started up, or who has been involved since the start up. For the components and process, it is essential that interviewee can look back on history and process to share the most effective initiatives and actions that provided values. For the leadership, it is important to gain the values, experiences, and tacit knowledge that only practitioner who have launched or operated a living lab must have. We used GTA (Grounded Theory Approach) for the analysis of this research. GTA is one of the analytical methods such as qualitative survey data in social surveys and was invented by Glaser and Strauss in 1967. Data collected from web surveys and transcripts of each interview were divided into short sentences as elements, considering the contexts, and were labeled and grouped to represent the phenomena (Glaser & Strauss, 1967).

4 Process and Components The three key components of the Smart Kalasatama Project, run by Forum Virium Helsinki, are introduction of scheme, vision making with citizens and place making for stakeholders. In particular, innovators club and developer’s club as place making has very much contributed to the progress of this project and Smart Kalasatama. The Smart Kalasatama Innovators Club is held four times a year, where participants regularly share news and create their own project, which help build a community for passionate stakeholders. Kalasatama Developers Club aims to support further connection between different actors in the ecosystem. Small amount of budget from 1000 to 8000 euros can be provided for entrepreneurs to help their product development and agile testing. This community accelerates the passionate entrepreneurs, developers, designers or artists who can contribute to the area by creating product, which is interlinked with Smart Kalasatama Innovators Club ("Smart Kalasatama initiative enters a new phase - Forum Virium Helsinki", 2015). Having three components of place making, citizen participation and creating an ecosystem, High Tech Campus Eindhoven emphasised the significance of the last one the most. One remarkable contribution is the establishment of Holst Centre as an independent R&D centre in 2005 under the cooperation of imec, TNO, and local, regional and national governments (TNO, 2019). Since its establishment, Holst Centre has become the physical space for researchers and academia to conduct research and development, which enables them to be a part of an ecosystem. Moreover, this research facility is so supported by local, regional and national governments that it strengthened the relationship between the Campus and the public sector. Another one is an award winning of Intelligent Community of the Year 2011 for its remarkable development of the Campus as an open innovation platform despite the financial crisis ("Eindhoven", 2019). This global award has spread the name of the Campus not only to the Europe but also to other parts of the world, which has attracted more entrepreneurs, researchers, startups or medias from all over the world. What is more, this award


has created the bridge between the Chinese investment and the Dutch high-tech ecosystem through holding regular events in China ("HighTech Connect China", 2018). Living Labs Taiwan has three components for place making, co-creation and citizen-based project. Although it has the strong relationship with national government, it cherishes the community of citizens in Taipei, Taiwan, which has rooted its historical background. It used to be colonised by Japan for 50 years from 1895 to 1945 and then became a part of China. During this period, social system such as education and public transportation has organised rapidly, and the capital investment from Japan also made the economy grow significantly. However, in 1945 when World War II ended, Taiwan became a part of China, from which social friction has appeared such as the issue of independence of the nation and freedom of speech (Jara Pallana, 2016). Therefore, citizens have strong intention and motivation to create their own area by themselves so that citizen-based projects and co- creation involving citizens works very well for Living Labs Taiwan.

Figure 1.

5 Three phases of process: launch, foundation, involvement In the launch phase, it is essential to set the direction of living labs. There are many ways to begin with such as the introduction of urban planning schemes and the hearing of citizens, but it is important to prepare the condition in that different stakeholders are able to run in the same direction. For example, in the case of High-Tech Campus Eindhoven, the fact that the Campus has been open to other research institutes and startups is very important. It issued a message


that they have a will to achieve open innovation across the region beyond the boundaries of the organisation. The essential part of this foundation phase is to create a foundation that will play a central role in managing a living lab. Such central core of a living lab can be generated by creating an ecosystem in which stakeholders from various fields and sectors participate or a physical space where citizens can gather freely. Smart Kalasatama Innovators Club and Kalasatama Developers Club is the good example to show its impact, as mentioned above. These clubs have created the opportunities for stakeholders in an ecosystem to gather regularly for exchanging information or creation of project. It is inevitable to empower such passionate stakeholders including citizens to do what they can do as a part of the project. In the involvement phase, it aims to expand and accelerate the activities of living labs. Exposure to the media, agile project with citizens and creation of public facilities will create a greater impact by involving more people who have never had a chance or are uninterested to be part. Living Labs Taiwan, they distributed 300 tablets to elderly people for use as a test bed for a project to support elderly people's healthcare. Initially, it took half a year to distribute only 100, but then information was spread by user's word of mouth, and it was possible to distribute 300 at a stretch. This activity showed the existence of Living Labs Taiwan to the public and citizens started to have an understanding toward their projects. Three phases of launch, foundation and involvement circulate. To enter into involvement phase does not mean the end of going in but it come back to the launch phase to start new projects and small initiatives from there. By circulating and spiraling up, it can be expected to move forward more smoothly than in the first flow. 5.1 Theme setting Theme setting means the same to indicate the vision: what kind social issues the living lab aims to solve or what kind of future the living lab aims to create. Of course, stakeholders satisfy their own interests and earn benefits, but if they do not agree with the vision, it is expectable they will soon withdraw from participation. By creating a clear vision, relevant stakeholders can gain access to the core and an ecosystem of high purity is formed, which can increase sustainability. In the Smart Kalasatama Project, the vision is defined to be free up one hour of extra time for its inhabitants every day ("Smart Kalasatama attracts innovation tourists to Helsinki - Forum Virium Helsinki", 2017). It is a good example of presenting an attractive and easy-to-understand vision for each stakeholder, particularly citizens. For discovering the clear vision, it is also effective to find the similar approach or the living lab as a benchmark, and to name the project in order to enable stakeholders to have attachment. 5.2 Ecosystem formulation Ecosystem is the basic component of a living lab. Particularly as proposed by Quadruplex Helix Model, the four actors are indispensable as the basis of the ecosystem; academia, civil society, company and the government (Carayannis & Campbell, 2009). The idea of an ecosystem may be common and may not be


difficult to create in Europe, particularly in Nordic countries where living lab was first adapted However, in a society such as Japan, where there is still little interaction between industries, and a vertically closed organisational system is still majority, the concept of ecosystem becomes the key for building up the living lab. Diversity of citizens and stakeholders is the source of innovation in living labs. 5.3 Co-creation approach Co-creation is the very heart of living labs as well as proposed by ENoLL. Stakeholders’ participation is essential for the realisation of the vision through living labs. Based on the ideas and needs of citizens as end users who well understand the area or community, living labs shows its value by co-creating with universities, companies, and governments that can facilitate the realisation of research and innovation at the same time. There are also many studies in the field of co-creation such as The Handbook for Co-creation published by city of Espoo illustrating well-organised method of co- creation ("Handbook for Cocreation", 2019). 5.4 Funding and Framework One of the major differences between Europe and the rest of the world is scheme and framework. As long as the concept of open innovation and a living lab is still minority, there are very limited grants or open call to projects to support a living lab from the government or foundation. The concept has not yet been so generalised that the role of external evaluation or mass media that support the project is very large. Furthermore, the need to build frameworks and systems that encourage participation is growing, as citizens and other stakeholders are not familiar with it. 5.4 Places and Opportunities It has a great meaning in creating places and opportunities for stakeholders to meet regularly. In particular, through the active try in the provided places and opportunities, citizens and stakeholders can become the active participant from a mere a mere audience to the living lab. MindLab, which was created with the aim of co-creating policy with the public in Denmark, also regularly held events and meetups for anyone wish to participate. One of the goals was to involve more people and enrich the ecosystem, but it focused more on a point that each citizen would be the active creator in the lab ("MindLab event", 2016). 5.6 Media and Transmission In order to promote the circulation of the ecosystem, it is essential to advertise the mission and activities of a living lab to a wider audience. Media exposure and transmission play a major role in achieving such purpose. Open Living Lab Days is one of the good examples. By participating in the Open Living Lab Days held by ENoLL every year, participants from a living lab can widely tell the vision and activities of the living lab not only by communicating with other participants but also by holding the workshop or displaying the poster ("Call for Workshops", 2019). Having the global connection with researchers and practitioners as well as domestic ones can enhance the impact of each living lab.


5.7 Citizen-based projects Citizens who are active in a living lab activity are the source of innovation. It is crucial to support those who have the passion for doing the project for the city or community. This is based on the idea of lead-user innovation that users know the most what they really need, advocated by Eric von Hippel (Hippel, 1986). Forum Virium Helsinki provide the open call for the enthusiastic people to submit their proposal of the project under the theme and provide a certain amount of grand to carry on the project if it is accepted ("Open call arkistot - Forum Virium Helsinki", 2019). 5.8 Leadership Leadership is one of the essential elements of living labs, but in Europe it has a different way of understanding. It is the image of general leadership that goes to the vision and purpose, and lead the stakeholders, but in Europe, the emphasis is on the harmonisation of stakeholders with different historical backgrounds and values that exist in the ecosystem. Listen carefully to the ideas and ideas possessed by local residents and citizens, cherish the diversity that is said to be the source of innovation, and work toward your vision.

6 Orchestratorship Another focus is put on the leadership required for practitioners of a living lab. This section begins with the brief summary of each interview and then describes 5 principles and 16 actions that are identified by GTA analysis. In honor of Kaisa’s expression in her interview, we call the required leadership for practitioners of the living lab as “orchestratorship”. 5 principles and 16 actions that compose the orchestrator ship are described below. High Tech Campus Eindhoven At High Tech Campus Eindhoven, Mr. Cees Admiraal, Business Director, cooperated with the interview. Having joined Phillips as a researcher, he was involved in launching High Tech Campus Eindhoven in particular for creating an ecosystem with professors or making a vision with stakeholders. He mentioned the significance of knowing what you do not know through listening to the professionals for becoming a better leader. On the other hand, it was added that it was also important to hear the voice of the younger generation who had new values. Open your mind by exploring these new opportunities is significant action for leaders. Forum Virium Helsinki From Forum Virium Helsinki, Kaisa Spilling, Development Manager of Smart Kalasatama joined our interview session. In the interview she had repeated the word “orchestration” over and over. It is necessary for leaders to harmonise stakeholders in the ecosystem who have the different values and ideas. Thinking about what their value is and providing value in order to get them involved in the project. Building relationships of trust, delivering value to each stakeholder and orchestration for greater impact are the actions that living lab leaders need. Living Labs Taiwan


From Living Labs Taiwan, Ms. Belinda Chen has answered our interview in Taipei, Taiwan. She is the director of Living Labs Taiwan and also is the deputy director of Institute for Information Industry (III) which manage Living Labs Taiwan. She is always thinking about what a living lab can offer for the future in five to ten years when conducting a testbed with citizens as end-users based on joint research with a private company, which is one of her duties. What she cherishes as a leader was to understand that it is a matter of course that many failures occur in the living lab as a field of demonstration experiments. It is more important to learn what you can learn from the mistakes, she added.

Figure 2.


6.1 Co-creation The first principle is to promote co-creation, which is one of the essential principles for living lab practitioners, as highlighted by the eight main components and by ENoLL. There are two action that are proposed for promoting co-creation. The first is to work with multiple stakeholder people with different background in ways of thinking, in nationality, in history and in future. By respecting the opinions of stakeholders who have different values, ideas, and ideas for the future that exist in the ecosystem, it is possible to create unexpected innovations from living labs. It is essential for leaders to be able to respect for differences and diversity in an era of accelerating globalization and increasing mobility of human resources. The second is to have experience of cooperation at all levels from policy making to working in the field. In order to carry out demonstration experiments involving the region and the subsequent social implementation, not only cooperation with citizens and end users at the field level, but also higherlevel experiences such as policy making are required. Belinda from Living Labs Taiwan focuses on not losing the opportunity to actually interact with citizens and end users even while working with the government or public sectors. 6.2 Empowerment The second principle is empowerment. Enabling citizens and other stakeholders to be able to actively participate in the project instead of being involved will increase the sustainability of living lab with the higher motivation of them and leads to a greater impact. Leaders needs to empower stakeholders in the ecosystem by sharing their own energy and vision to support their proactive efforts. To be more specific, there are four actions in this principle. The most representative action is to have the social and emotional skills. It is very important to read social change and think about the next trend in order to produce a larger impact. On that basis, understanding what an individual needs and what kind of emotion it has can empower the individual in a way that is good for both the individual and society. To Influence the young generation with motivation and capability to want to challenge is also important. Influencing young people with passion and capability also leads to better ecosystem metabolism. As the pace of change in society is increasing at an accelerating pace, the younger generation is the driving force for the creation of new values. In High Tech Campus Eindhoven, Cees Admiraal is consciously making acceleration events or pitch contests for young researchers or entrepreneurs to challenge. 6.3 Exploration The third principle is exploration. It is important for the leaders themselves to be inquisitive and to respect new values, ideas and whatever they have not been encountered before, which fosters a culture that respects new things as a whole. For this principle, three action are provided. The most representative action is to listen to and learn from the professionals and the young. It always comes from young generations who live in the new culture or professionals who know the cutting edge. The leaders are required to listen to them to encounter whatever you may not know. Cees Admiraal from High Tech Campus Eindhoven also make it a practice to talk to professionals on the Campus or to younger generations in the organisation when he faces the challenges.


6.4 Open and fair The fourth principle is open and fair. In order to let more stakeholders to be a part of the project, it should be open enough that anyone can participate in projects and discussion, and reasonable enough that participants can understand the progress and decision process. In Smart Kalasatama Project, only the elected members can attend Smart Kalasatama Innovators Club but many of the opportunities are open to anyone who has the interests and motivation to be part of it. There are four actions for this principle. The most representative action is to show project process, structure and progress. Even if people can get into part of the discussion or project but are not provided the mechanics of decision- making, they are not enabled to participate but exploited. Mads Bonde Clausen of MindLab explains this situation that exploitation of citizens’ participation and motivation does happen for the participatory design project or co-creation process unless leaders pay attention. It requires certain costs to share the process, structure and progress but by sharing these helps to foster the independence of the participants. 6.5 Reflection The last principle is reflection. By looking back on your own efforts, you will be able to see if you are approaching the future you want to go both as an individual and as an organization. Three actions are required to do for this principle. One is that the stakeholders are diverse, the value provided is also diverse. If you emphasize only one actor, it may happen that you cannot provide value to other actors, so it is necessary to reflect on the value provided. The second is that although the ecosystem is the foundation of the living lab, it is also important that the leader in charge of the center review the behavior is. The last is Have the long-term view based on new innovative behavior. While living lab efforts can have greater impact, they cannot always deliver immediate value. Based on the new principles of behaviour, it is essential to think things in the long run.

7 Conclusion Living labs are spreading mainly in Europe, but social background and cultural context are different from other part of the world such as Japan, with the history of industrial development. Previous research shows there are many researches introducing the components and features for living labs, but these usually deal with case study of European living lab, which cannot be directly diverted into other countries with different conditions. Research turned out to show that there are three main phases of launch, foundation and involvement and eight main components of theme setting, ecosystem formation, co-creation approach, funding and framework, places and opportunities, media and transmission, citizen-based projects and leadership. There are also actions to be taken in each phase so that potential users can use it as guidelines understanding in which phase they are in. Another outcome is the leadership required of living lab practitioners. We conducted interview with the practitioners in Finland, the Netherlands and Taiwan and found five principles and 16 principles of action, which we call orchestratorship.


The first future avenue is to find a blind spot. This time, it is assumed that there are important components that have not been found yet and actions that are required of the leader, because research results were generated from interviews with the limited number of practitioners. The next development is to increase the number of interviewees and explore the blind spots. The second is to increase the resolution of research. Although we could have identified the framework through this research, the purpose of this research is to support practitioners so that we aim to bring it into a state that is easy for practitioners to use. The last is to validate the research. By using research output together with researchers and practitioners, we aim to produce more valuable research by making improvements. In particular, in this research, although the framework was produced based on the successful cases in Europe, verification of its usefulness is still insufficient. In Japan, there are many close approaches that are not called living labs. For example, we feel great potential in living a lab on the theme of utilization of forests and mountains, which the administration of Shiojiri City, Nagano Prefecture is working on. As the next phase, we would like to apply the framework and principles created from this research to the context of Japan and create a mechanism that this framework can be applied to each region. Since this research has just started two years ago, we would like to work on these future avenues and co-create this research with researchers and practitioners around the world.

References Call for Workshops. (2019). Retrieved from Carayannis, E., & Campbell, D. (2009). 'Mode 3' and 'Quadruple Helix': toward a 21st century fractal innovation ecosystem. International Journal Of Technology Management, 46(3/4), 201. doi: 10.1504/ijtm.2009.023374. Chesbrough, H. (2006). Open innovation (1st ed., p. xxiv). Boston: Harvard Business Review Press. Economic and Industrial Survey Committee of Japan. (2018). Open Innovation White Paper. Tokyo. Eindhoven. (2019). Retrieved from European Network of Living Labs. (2019). PROJECT PORTFOLIO 2018 - 2019 [Ebook] (1st ed., p. 5). Brussels. Retrieved from Forum Virium Helsinki - European Commission. (2019). Retrieved from uusimaa/forum-virium-helsinki. Future Center Alliance Japan. (2019). Notice of partnership agreement with ENoLL. Retrieved from Glaser, B., & Strauss, A. (1967). The discovery of grounded theory. Chicago: Aldine Pub. Co.


Handbook for Co-creation. (2019). Retrieved from cocreation/html5/index.html?page=1&noflash. High Tech Campus Eindhoven: Campus History. (2019). Retrieved from HighTech Connect China. (2018). Retrieved china/en.


Hippel, E. (1986). Lead users: a source of novel product concepts. [S.l.]: [s.n.]. Hossain, M., Leminen, S., & Westerlund, M. (2018). A Systematic Review of Living Lab Literature. SSRN Electronic Journal. doi: 10.2139/ssrn.3307055. Intelligent Community of the Year. (2011).



Jara Pallana, M. (2016). Taiwan after 70 years: How Japan's colonialism still have an impact in modern day Taiwan. Academia Edu. Retrieved from ans_colonialism_still_have_ an_impact_in_modern_day_Taiwan?auto=download. MindLab event. (2016). Retrieved from newsdisplaypage/?newsid=48ad5889-b3d9-4c8f-84cb- c2d2448fd08f Open

call arkistot - Forum Virium Helsinki.

(2019). Retrieved


Pentland, W. (2013). World's 15 Most Inventive Cities. Forbes. Retrieved from cities/#414b6e34ec5d. Smart Kalasatama attracts innovation tourists to Helsinki - Forum Virium Helsinki. (2017). Retrieved from tourists-helsinki/ Smart Kalasatama initiative enters a new phase - Forum Virium Helsinki. (2015). Retrieved from Taiwan III Living Lab. (2019). Retrieved from en/index.php. TNO,

H. (2019). Holst Centre in a nutshell. Retrieved


What does Forum Virium Helsinki mean to the city? - Forum Virium Helsinki. (2019). Retrieved from



Living Labs need sustainable revenue models: The Funding Mix Framework to bridge the gap between theory and practice Edoardo Gualandi*1 and Flavia Fini2


*Corresponding author University of Technology, The Netherlands 2 Alma Mater, Italy Category: Innovation Papers

Abstract Living Lab (LL) represents an emerging innovation methodology which has the potential to bring together different actors in a collaborative process to develop solutions to diffuse social problems. Nevertheless, a substantial number of Living Labs struggle to translate the value created into a sustainable revenue model and, thus, they often present an unintended temporary nature. Research about Living Labs is primarily focused on theoretical and methodological aspects, while good practices, especially for what concerns funding, revenue and business modelling, are still under-researched. In this paper, we analyze good practices and critical problems of six LLs from across Europe. Then we apply the previously developed Funding Mix Framework to understand if it can be considered valuable support for LLs to develop a more sustainable revenue model, ensure long term viability and scale up their operations Keywords: Living Lab, financial sustainability, long-term viability, social value, social challenges, business model, revenue model


1 Introduction and objectives Deeply rooted in real-life environments, Living Lab (LL) is either part of, or constitutes an Innovation Network of people, private firms and public institutions. LL is a methodology based on knowledge and observation, and guided by a practice-driven approach; these elements combined, in the form of innovation projects, concur in the realization and implementation of innovative solutions that are user- or community-driven, co-created by the customers, and tested and validated in real-life settings (Gualandi, 2018). Indeed, LLs have a great potential to serve their community and develop innovations which can solve diffused social problems and improve the life of the citizens. In order to concretely have an impact on society, LL’s operations must be ensured for an adequate time: financial sustainability is fundamental for a LL to be viable in the long-term and eventually scale up its operations. Although that, a substantial number of LLs struggle to translate the value created into a sustainable revenue model and, thus, LLs often present an unintended temporary nature since they stop their activities when the funding ends (Katzy, 2012). Despite that, most studies focus on theoretical and methodological aspects of LLs, while good funding practices are still under-researched and there is a lack of concrete research on viable revenue models for LLs. Therefore, in this paper, we first analyze the value creation process for LLs through a systematic literature review, with a prevailing focus on the social value which makes LLs a promising methodology to address the social and environmental challenges of our time. Then, we explain the problem of financial sustainability, which often precludes to the LLs the possibility to concretely better society. Then, we present the Funding Mix Framework (FMF), a tool developed in previous studies, which can be considered a practical support to design a sustainable funding model. Then, taking the FMF as a reference model, we analyze six LLs from Italy, Spain, Serbia, Slovenia and the Netherlands. The results of the case study provide concrete insights over good financing practices and common problems. In the conclusions, on the one hand, we have some confirmation over the suitability of the FMF in developing a self-sustainable funding model and, on the other hand, we reflect on the implications of the findings for LL’s long-term viability and scalability.

2 Literature review The last decades marked a radical shift from the traditional conceptions of organization and market. Many firms collaborate with customers to improve the effectiveness of product development which often results in considerable benefits (Nijssen et al., 2012). Furthermore, customers are valuable sources of product and service innovation (von Hippel, 2005). Moreover, the need for always new and complex products and services paved the way for many organizations to draw from a broad set of external sources of knowledge: users, customers, suppliers, partners and competing enterprises, universities, research centres and governmental institutions can concur to a shared objective. It is based on these premises that Chesbrough’s (2006) developed the concept of Open Innovation. Living Lab is an emergent methodology that has the potential to fulfil the role of


bridging the gap between User-Innovation and Open-Innovation (Schuurman, 2015). 2.1 Living Lab and value creation According to ENoLL( LLs are “user-centred, open innovation ecosystems based on a systematic user co-creation approach integrating research and innovation processes in real-life communities and settings”. The value created by the LL varies in terms of nature and affected actors. Hence, to better understand and direct the activities of a LL, can be helpful to adopt the classification between economic (transactional), business (company) and public (social) value (Gualandi & Leonardi, 2018; Gualandi, 2018). Transactional value covers aspects that are highly tangible from different stakeholders’ perspectives such as the growth of the companies that collaborate, an increased competitive advantage or the extent to which new business are generated and survive (Baccarne et al., 2014). LL’s innovation process delivers transactional to the whole value chain in the form of better products and services. (Ståhlbröst & Holst, 2012). Company value is an extension of economic value and includes other forms of value such as employee value, customer value, supplier value, managerial value, and societal value. (Bergvall-Kåreborn et al., 2009). Finally, social value is one peculiar characteristic that differentiates LL from other innovation methodologies and makes it a relevant innovation methodology for the future due to its ability to support Sustainable Development Goals. The LL serves as a connection between research, citizens and the actual living environment (Franz, 2014) and works as an innovation intermediary and aggregators of external inputs that are then translated into requirements and design parameters for valuable social innovation (Mention & Torkkeli, 2015; Almirall & Wareham, 2011); researchers, businesses, and users explore novel manners of contextualizing knowledge and technologies in real-life to result in a concrete impact on society (Pierson & Lievens, 2005); by the inclusion of the user at the centre of the development process, it is possible to meet the specific needs and aspirations of local contexts with the proposed solutions (ENoLL, 2016). Moreover, per Ståhlbröst (2012) the innovation processes supported by LL must address environmental and social issues while considering the economic impact and LL also takes responsibility of environmental, social, and economic effect (Bergvall-Kåreborn et al., 2009) and pursue societal improvements (Leminen et al., 2012). LL offers a mechanism to support stakeholders’ collaboration and generation of innovation outcomes in social environments, ranging from improving everyday living conditions to systematic citizens innovation (Leminen & Westerlund, 2015). Indeed, LLs dedicated to social value can effectively increase cohesion in society (Schuurman et al., 2016) or improve users’ behaviour with the respect of issues like environmental awareness (Ståhlbröst & Holst, 2016).

2.2 Financing strategies: the struggle for a revenue model Social value should be the core of any LL projects but it is also the hardest to obtain: long-term viability is necessary to have a concrete impact on society since


LLs require continuous funding and a sustainable financing model to support and scale up their innovation (Guzman et al., 2013; Evans et al., 2015). Nevertheless, LLs are often not financially sustainable and struggle to translate the value created into a sustainable business model (Brankaert et al., 2014, Katzy, 2012). Hence, a significant number of LLs presents an unintended temporary nature (Leminen et al., 2012) and financial sustainability seems to be the key condition for LLs to become permanent and remain operational in the long-term (Veeckman et al., 2013). Moreover, the ability to implement solutions with a concrete social impact must be supported and ensured during the entire life of a LL. Several researchers found out that most of LLs that are adequately funded primarily rely on public grants and subsidies (i.e. Brankaert, den Ouden, & Grotenhuis, 2014; Wu, 2012). Even if this is a feasible financing option in the short-term, it does not ensure the viability in the long-term since many of these LLs stop their activities when the funding ends. A potential cause of this struggle can be linked to the fact that many LLs do not make systematic use of business modelling techniques for themselves (Mastelic et al., 2015). In addition, often common business modelling tools such as the Value Proposition Builder (Barnes et al., 2009), the Value Proposition Canvas (Osterwalder et al., 2015), and the People Value Canvas (Wildevuur et al., 2013) are not adequate to consider certain peculiar characteristics of the LL (Äyväri & Jyrämä, 2017; Schuurman et al., 2019). Finally, LL theory lacks solutions and concrete financing strategies: there is only a limited amount of literature available that combines LLs with business models (Hossaina et al., 2019). Being the challenge of financial sustainability of great importance in the success of LL projects and a key condition for the generation of social value, the lack of concrete instruments to define a viable revenue model is a critical deficiency in LL research (Gualandi, 2018). In line with that, in a previous study (Gualandi & Romme, 2019), we proposed the Funding Mix Framework as a concrete tool to better identify potential revenues and sources of funding which can be potentially exploited by any LL in order to reach financial self-sustainability, long-term viability and thus to scale up their operations. The FMF represents a practitioner-oriented tool that can help to bridge the gap between theory and practice, and that can improve the chances of a LL to successfully deliver social value to its stakeholders and to the citizens 2.3 Funding Mix Framework: a novel revenue model In line with that, in a previous study (Gualandi & Romme, 2019), we proposed the FMF as a concrete tool to better identify potential revenues and sources of funding which can be potentially exploited by any LL in order to reach financial self-sustainability, long-term viability and thus to scale up their operations. The FMF represents a practitioner-oriented tool that can help to bridge the gap between theory and practice, and that can improve the chances of any LL to successfully deliver social value to its stakeholders and to the citizens. According to the FMF, LLs can draw on different funding options, which we classify as pay per service (PPS), subsidies (SUB), out of network funds (ONF)


and cross-financing (CRF). In the next paragraphs, we explain more in detail the four categories.

Figure 1. Funding Mix Framework

2.3.1 Pay per service Pay per service is the most immediate monetary recognition of the services offered by the LL. For this reason, it can be considered the financial return for the transactional value generated. Hence, transactional value is primarily delivered to business partners which turn to the LL for the development or improvement of commercial products and services. Indeed, the source of PPS is mostly private. In rare occasions, PPS can be related to company and social value. In these cases, the source can shift partially toward the public sector. The stakeholders that contribute with PPS belong to the LL’s the network. Finally, PPS is a funding option at a project level: indeed, PPS relates to the services provided by the LL in the context of a specific project. 2.3.2 Subsidies Subsidies are the most frequent funding option connected to social and company value and are ensured by the strategic partners. In fact, social and company value is mostly recognized by actors committed to a long-lasting relationship, in which the interest is not limited to projects but aims at the development of shared goals and objectives. Social value is delivered to the citizens and to stakeholders from public sector and education, which compensate the LL in the form of subsidies. In addition, public sector, education and businesses are the main recipients of company value, if involved in long-lasting relationships and compensate the LL also with subsidies. Hence, SUB is a funding option that mainly relies on public sources. Finally, we noticed that SUB is a funding option linked with the entire


innovation process and operations of the LL and is provided by stakeholders and actors belonging to the network. 2.3.3 Out of network funds LL’s mission is generally in line with the Sustainable Development Goals. For this reason, LLs have the possibility to gather important funds by systematically applying for European Community, national and regional calls. LL’s projects are often compatible with public policies, and open calls are good options to finance the creation of public value. The funds are made available primarily by public bodies and, thus, are mostly coming from public sources. These calls can also be thrown by private entities, like banks and cooperatives, but is a less common situation. The organizations issuing the funds are not directly involved in the LL network, but they are only responsible to grant funds based on predefined criteria. Finally, the funds are a support for the mission of the LL and, thus, relates mostly to the strategic level. 2.3.4 Cross-financing Differently from PPS, SUB and ONF, this financing option is not linked to the activities of the LL, neither contribute to the network. In fact, cross-financing is rather an alternative way to profit from the LL’s assets, such as the physical location (i.e. the LL can sublet permanently part of its space to a bar or to a coworking office, or temporarily to events, conferences, meetings) or the complementary equipment (i.e. the LL can lease printers, software etc.). The source of CRF is almost exclusively private and completely external to LL activities.

3 Research questions The objective of this paper is to better interpret the FMF developed in the previous studies to have a deeper understanding of how it can concretely support any LL in exploiting the full potential of its network and activities. Therefore, the empirical research will be guided by the following research questions: • How do LLs exploit the different funding options proposed by the Funding Mix Framework and how can a balanced approach help Living Labs to become financially self-sustainable? • How can the Funding Mix Framework Support Living Labs’ innovation projects in order to have a greater impact on society and aim at scalability? The hypothesis is that any LL can have greater chances of success in generating social value, initiating virtuous processes that can be scaled up or replicated if they correctly approach their business and revenue model. In particular, we hypothesize that LLs that address the different funding options presented in the FMF in a balanced way. Only in this way, LLs can succeed in having a substantial impact over society.


4 Methodology 4.1 Case study design To answer the research question, we adopt a multiperspectival methodological approach both in regards to the case selection process (ยง 4.2) and the data collection and analysis (ยง 4.3). This method is deemed the most appropriate given the diverse case studies intended to be analysed, and the most effective in giving to the results a more compelling nature and thus ensure higher robustness, which is a fundamental measure of the quality of the research design. This has been done in accordance with Yin (2003) which states that the choice of the number has a strong relationship with the purpose of the investigation and thus it is to be defined by the practitioner coherently with the adopted replication logic. In fact, the cases can be selected in such a way that they either predict similar results or provide contrasting results but for predictable reasons. Therefore, in an exploratory study like the current one, we decided to have to have multiple perspectives and examples, but, at the same time, we still wanted to include not an extremely large sample so that we could still explore them in detail in a qualitative way. Accordingly, we designed the case study to involve between 5 and 10 cases. 4.2 Case selection In order to select the cases, around 50 Living Labs from all over Europe were analyzed based on different sources of information (websites, newspaper articles, academical studies) to preliminarily understand their positioning about orientation and source of funding. Then, 24 Living Labs that were considered consistent with the research purposes were contacted. Only a limited number of them replied, and among the respondents several were not available for research purposes or were no longer active. Three cases, Stratumseind Living Lab (SLL) in Eindhoven (the Netherlands), Amsterdam Fieldlabs (AFL) in Amsterdam (the Netherlands) and Textile&Clothing Living Lab (TECLA) in Palermo (Italy) were already involved in previous studies thus we iterated the analysis in the light of an expanded theoretical framework and through new perspectives. To further expand the previous studies, we included three additional cases: ORbITaLA in Maribor (Slovenia), PA4ALL in Novi Sad (Serbia) and BIRD Living Lab in Bilbao (Spain) are completely new additions to this research. The six cases are very different from each other under many points of view: nevertheless, we do not consider this aspect a limitation for the current research but rather an opportunity: in fact, the previous studies suggested that the FMF can be suitable for heterogeneous LL. Indeed, we selected the six cases in a fashion that could ensure the broader view as possible over different casistics of LLs. Therefore, in this case, we present the insights about LLs active in different fields: from textile to open data, from social policies to natural environments and farming, from urban contexts to rural areas. At the same time, among the potential candidates we selected LLs in different phases of their life - from the ones in their start-up phase to ones that are already considered well-established institutions in the context they operate in. 4.3 Data collection and analysis In regard to data collection and its analytical process, we employ a wide array of data. Interviews are the primary source of data collection. Interviews are often


used since they allow the researcher to retrieve experiences, behaviors, opinions, values, feelings, factual knowledge, and personal background (Esterberg, 2002). When possible, we opted for semi-structured interviews, which means that we began with few preset questions to then follow the respondent’s tangent of thoughts. That seems an adequate choice in an exploratory study in which the research direction is not completely defined upfront and new unexpected information can raise. It also seems the most appropriate methodological tool in order to balance the potential disparity between the researchers’ positionality and expectations in regards to the outcome of the data collection and our interlocutors’ interpretation and personal analysis of the data which, in this way, is not constrained and orientated by a strict set of questions. Furthermore, we employed also a heterogeneous set of complementary sources of information and methods. The three Living Labs provided us with additional study material, ranging from handbooks to the official applications to ENoLL, from videointerviews of important partners to official PowerPoint presentations. In certain cases, in situ research was not possible and thus we involved the manager in a detailed survey and digital interviews covering several meaningful aspects of their LLs still respecting the semi-structured and open-ended question approach. In order to analyze the collected information, we combined ex-ante and ex-post coding, and content analysis to process the entire set of collected data. The second approach to coding is particularly adequate in these cases where the interviews’ questions are mainly open-ended, and thus it is harder to predict the data. Finally, we employed thematic content analysis since it is an adequate method to represent sentences and topics referring to the questions of the research. 4.4 Quality of research We can state that the research and the resulting models present a good degree of quality of research thanks to the employment of heterogeneous sources of information and by systematic triangulation of the results. The nature of the research, based on qualitative methods and on an exploratory approach, is a limit to the external validity of the results. At the same time in situ researches and cross-data analysis increase the internal validity of the approach. This is in line with the broader objective of this study - to develop a coherent framework gathering several aspects which are not investigated in a systematic manner in current literature and favour the path to future researchers about promising novel approaches rather than developing robust models.

5 Observations and results of the case study Orbitala is a LL founded by the Maribor Development Agency with the objective to guarantee better connectivity between research and SMEs and perform research for industry or public administration around meaningful local challenges. The LL has a strong connection with the public administration and the business sector while the education sector is almost absent. ORbITaLA’s financiers, both from inside and outside its network, are mainly involved on a project level and provide funding only for specific projects. For this reason, the Living Lab is not


able to perform its activities on a regular basis and is often forced to operate below the expected standards. The various innovation projects often depend on a single source and the generation of a concrete impact over society is dependent on the objective of each project, which is often determined by the main partner. The approach to funding is often improvised and the LL is far from a concrete financial self-sustainability. Despite that, also thanks to EU funds ORbITaLA ran several successful projects which realized transactional, company and public value for the main partners involved and which had a concrete impact on society. The municipality of Eindhoven developed various LL initiatives. SLL is one of the most successful examples. The mission of SLL is to support companies and institutions in the development of innovative products, services and policies which can foster the economic environment of the region and improve the quality of life in the neighbourhood. SLL has a very strong connection with public bodies, with the many tech companies in the area and with several educational institutes. The LL generates consistent company value and transactional value especially for the businesses and for the public administration, while the social value is rather marginal. The LL developed very tight relationships within the network and counts on important funding both on project and strategic level. Therefore, SLL is able to perform its activities regularly, respecting very high standards. The approach to funding is highly structured, systematic and relies on multiple sources. Nevertheless, the local government and other public organizations are the stakeholders that provide the most substantial part of the funding. In conclusion, SLL has a sustainable revenue model which ensured long-term viability to the LL. The municipality of Amsterdam, together with the Amsterdam University of Applied Sciences (AUAS) formalized a knowledge alliance and initiated three LL initiatives, the “Fieldlabs�, in areas at the edge of the city centre to address diffused social issues. In 2017, AFL was able to upscale its operation to a metropolitan perspective. The public administration and AUAS are the two main strategic partners and, in many projects, they are the only financers. At the moment AFL regularly performs its activities respecting very high standards but its long-term viability is at stake: in fact, it is strongly dependent on two main sources and thus cannot guarantee viability if the funding ends. In addition, AFL mostly generates social value while transactional value is neglectable. Moreover, AFL does not have a structured approach to the revenue model and the generation of revenues from different sources is not contemplated by the board. Nevertheless, AFL developed a trustful relationship with its main partners and the social value that generates is highly recognized also by the public opinion and thus represent a project legitimation. PA4ALL focuses on precision agriculture and was founded in 2012 by the BioSense institute in Novi Sad as a meeting place for heterogeneous stakeholders in the agrifood sector. PA4ALL has a strong connection with the business sector which is often involved on a strategic level and is the main receiver of value. Despite that, the funding from this sector is rather scarce. Also, the education sector is often involved as a strategic partner and the amount of funding is more substantial. The public administration is mostly involved on a project level and the funding is marginal. Indeed, international calls represent the


primary source of financing. PA4ALL has a structured and systematic approach to funding which ensures a good degree of financial self-sustainability and longterm viability. Nevertheless, PA4ALL does not contemplate cross-financing. In conclusion, PA4ALL, which was the first precision agriculture lab in Europe, has a strong revenue model which allowed it to have a concrete impact on society with the creation of new businesses, the establishment of synergies and raising the public awareness around relevant topics BIRD is situated in the Biosphere Reserve of Urdaibai in the Basque Country, an environmentally protected area with high industrial development constraints. The LL is the result of an initiative started in 2009 and is now owned by the Cluster GAIA the association of technology firms of the region. The LL connects technology and smart economy with relevant social issues especially in the field of sustainable ecology and biodiversity. BIRD involves partners across the quadruple helix, especially from the business sector which are also the main receivers of value together with the education sector. The LL has a structured and systematic approach to business modelling but is mainly able to ensure funding at a project level and thus only certain projects are adequately funded. Also for BIRD, national and EU calls are fundamental. In addition, the LL exploits several alternative ways of funding: workshops, colocation and events are just some of the opportunities that the lab exploits outside its core activities. TECLA is a physical space in Palermo, that encourages to discuss ideas and projects, meet partners, develop cooperation methodologies where textile and clothing manufacture meets technologies and advanced multimedia tools. TECLA is hosted by Consorzio ARCA, a university business incubator. Local authorities are not deeply involved, and the public sector is mostly represented by the European institutions involved in the Horizon2020. Some education institutes and small businesses are involved but only marginally and in specific projects. Despite the efforts of the LL to structurally and systematically approach the revenue model, the amount of funding from the network of the LL is not substantial and thus TECLA is not always able to perform its activities according to the planned standard and some projects cannot be executed. The main source of financing is EU calls which allows TECLA to take part in European initiatives. In addition, TECLA shares its space with other realities such as a coworking and a restaurant. Nevertheless, the amount of funding is not enough to perform on a constant basis, hence it is not always possible to have the desired social impact.

6 Discussion The case study shows that financial self-sustainability is still perceived as a challenge by the LLs which address it in different ways: while in some cases the approach is systematic and well-structured, in others is rather improvised and circumstantial. Moreover, the employed funding options are various and the six cases have different approaches which suggest that they do not refer to a common paradigm. Nevertheless, the empirical evidence suggests that the Funding Mix Framework adequately represents the full range of financing options available for the analyzed LLs and thus it can be interesting to analyze their funding strategy accordingly.


Four LLs can be considered financially self-sustainable, even though not every case employs all four options. These four LLs are able to perform their activities on a regular basis, to respect the expected standard and almost all their projects are ran until the end with successful results: (1) BIRD is the only LL to make use of PPS, SUB, ONF and CRF and in fact it evolved over the past ten years, it was scaled up, and was framed in different initiatives. (2) Despite SLL only counts on PPS and SUB, it established very strong relationships with strategic partners from public and education sectors and with many businesses from the vivid high-tech field. (2) PA4ALL does not employ CRF but thanks to a balanced mix of the other three options, together with the broad national and international network, ran many projects with success. Nevertheless, the LL’s activities are performed regularly but not on a daily base: thus, has a different approach: it mostly counts on SUB from two main partners and does not have a structured approach to financing. This strategy has been successful in the last years and AFL was able to scale up its operations. At the same time, it is a factor of risk: long-term viability might be at stake if one of the two main stakeholders retire. On the contrary, BIRD, PA4ALL and SLL can count on a broad and heterogeneous network which makes them not only self-sustainable in their routine activities but also eventually viable in the long-term. The remaining two cases present some similarities: they are not able to perform their activities on a regular basis but they only function occasionally. The staff of the two LLs is mainly constituted by the two founders which are employed by the institution owning the LLs. Moreover, the prevailing funding option is represented by ONF: the LLs are activated in the context of European projects and their budget is mostly provided by EU calls. These are still interesting cases: in fact, they are far from a financial self-sustainability which can allow routine operations and activities. Nevertheless, thanks to their ability to systematically apply to international calls they have the potential to be viable in the long-term. This is primarily suggested by ORbITaLA which has been active already for seven years. In contrast, TECLA is still in a start-up phase and is still fine-tuning its business model. Another interesting aspect emerging from the case study is related to the geographical location of the six LLs: there are hints which suggest the existence of very different approaches between economically depressed areas, mostly in the South of Europe, and the wealthier countries in the North. Indeed, the cases from the Netherlands are the only ones that do not need to apply to EU calls to finance their activities: in these cases, they are capable to obtain substantial financial support by the public administrations and by education institutes, while in the other cases the involvement of the institutions is rather marginal when not completely absent. Furthermore, it is interesting to notice the importance of having a diversified set of financers and sources of funding: even if it is possible to be financially selfsustainable for several years only counting on one or few main partners while pursuing social value (i.e. AFL), we also noticed that when a LL rely on too few partners it tends to become strongly dependent on him, not only economically,


but also in the direction that the LL’s projects take. In addition, the viability of such LLs is at stake in the moment that an important partner resigns or stop providing funds. Therefore, having a broad and heterogeneous set of financers is not fully necessary but it might be an important advantage in achieving a stable long-term viability. Finally, we could have some interesting insights over the social impact of the six LLs. AFL’s stakeholders are primarily interested in the social value and thus the social impact, which is the main purpose of the LL, is an adequate project legitimation. Also, for BIRD, social value is a central aspect and thanks to the EU funds the LL is able to spread the environmental awareness across the entire network and direct innovation towards shared sustainable goals. PA4ALL, also thanks to EU funds, delivers social value in its community through sustainable rural development and through the creation of new businesses fostering employment. A similar approach is adopted by TECLA in the textile field but, since it is not able to perform regularly, the achieved social impact is far from the objectives. This is a problem that also ORbITaLA is experiencing: in fact, being forced to operate occasionally only in specific projects the generation of social value is strongly dependent on the financers: some project has a concrete impact on society, but in certain cases the financer is rather interested in transactional and company value and social value remains marginal. Finally, SLL is strongly business oriented and only rarely involves actively the citizens in its processes. Therefore, social impact in the short term is mostly limited to an increase in safety and security in the nightlife of the neighbourhood. Nevertheless, the LL is actively involved in the definition of state-of-the-art open data policies whose social value will eventually materialize in a later time. In conclusion, participation in EU projects and a tied relationship with institutions and education centres favour the realization of social value. On the contrary, a too marked business orientation or occasional project orientation are impediments for adequate attention to social value.

7 Conclusions We believe that this paper represents an important contribution to the state-ofthe-art of LL both from a theoretical and practical perspective. We put in the spotlight several deficiencies in the current literature which represent an obstacle for the definition of virtuous practices financial self-sustainability. In line with previous studies (Brankaert et al., 2014; Leminen et al. 2012; Ståhlbröst, 2012; Schuurman, 2015; Gualandi & Romme, 2019), the empirical research showed that a structured approach to business and revenue modelling can increase the possibilities for a LL to become financially self-sustainable and viable in the longterm. The research shows two main tendencies in the definition of a sustainable financing model: (1) some LLs confirmed the idea of Niitamo et al. (2006) and strongly rely on their network and secure long-term funding establishing strong relationships with important stakeholders, while it emerged that (2) other cases mainly count on EU calls and operates in the context of European projects. Both directions come with consequences:


In the first case, LLs are capable to finance regularly their activities but they risk that the interest of the partners prevails over the mission of the LL which becomes a problem when the network has a strong business orientation which may prevent the LL to pursue actively social value. Moreover, when a LL counts on a limited number of financers, also long-term viability is at stake: if one of the main partners stop financing the LL, financial self-sustainability cannot be ensured. In the second case, LLs are more independent in pursuing their goals, and if they systematically apply to EU funds they can be viable in the long term. Nevertheless, LLs that rely on European calls often cannot give continuity to their activities and operate occasionally with a prevailing project orientation. In this way, it is harder to have a concrete impact on society. The research suggests that an optimal strategy for a LL is to place at the middle between these two diverging approaches: indeed, the FMF seems to be a valuable instrument to be employed by LLs in defining the most suitable funding mix. In fact, all the funding options employed by the LLs participating in the case study can be framed in the FMF. Nevertheless, we suggest that there is not a financing model which is optimal for any LLs, but it depends on several conditions of the LL such as the mission, the field of application, the local context, the maturity of the institutions, the innovativeness of the network. Moreover, the empirical research confirmed that when they are not financially self-sustainable, LLs struggle in generating the expected social impact. At the same time, we discovered that most LLs do not fully exploit all the possible sources of financing identified by the FMF. Therefore, LLs could use the FMF to identify new funding options which can eventually increase their impact on society. Finally, being the scalability of a LL connected with its self-sustainability and long-term viability, the FMF can also help LLs to scale up their operations. This aspect is under-researched in current LL theory. Indeed, this point is truly based on in-field observation and on reasoning: indeed, it seems clear from the analysed cases that financial self-sustainability does not necessarily imply the possibility to scale up the LL, but if a LL is not financially self-sustainable then it does not represent a promising virtuous model to be replicated or scaled up.

8 Limitations and future work Even though the Funding Mix Framework seems to be a promising tool to reduce the gap between theory and practice and valuable support to fine-tune an adequate business and revenue model including a full range of complementary sources of funding, this framework needs further validation with more quantitative studies and with continuous observation of LL cases. Moreover, the paper puts in the spotlight the deficiencies in current LL research: the topic of long-term viability needs to be analysed and explored broadly. For further research in this direction, it seems also necessary to look at the specifics of LLs’ activities from a geographical point of view, examining whether the presence of a LL in a certain territory can impact its practice and its ability to achieve a long-term and selfsustainable revenue model. Therefore, we advocate for substantial and


systematic research around this topic to develop a shared understanding and a reference theory.

References Almirall, E., & Wareham, J. (2011). Living labs: Arbiters of mid-and ground-level innovation. Technology Analysis & Strategic Management, 23(1), 87-102. Äyväri, A., & Jyrämä, A. 2017. Rethinking Value Proposition Tools for Living Labs. Journal of Service Theory and Practice, 27(5): 1024–1039. Baccarne, B., Mechant, P., & Schuurman, D. (2014). Empowered cities? An analysis of the structure and generated value of the smart city Ghent. In Smart City (pp. 157-182). Springer, Cham. Barnes, C., Blake, H. and Pinder, D. (2009). Creating & delivering your value proposition. London: Kogan Page. Bergvall-Kåreborn, B., Eriksson, C. I., Ståhlbröst, A., & Svensson, J. (2009). A milieu for innovation: defining living labs. In Proceedings of the 2nd ISPIM innovation symposium: Simulating recovery - the Role of innovation management, New York City, USA. Brankaert, R., den Ouden, E., & Grotenhuis, F. (2014). Identifying differences between living labs - Learning from practice. In ISPIM Conference Proceedings (p. 1). Dublin, Ireland. Chesbrough, H. W. (2006). Open innovation: The new imperative for creating and profiting from technology. Boston, Massachusetts: Harvard Business Press. ENoLL (2016). Introducing ENoLL and its Living Lab Community. Retrieved from Evans, J., Jones, R., Karvonen, A., Millard, L., & Wendler, J. (2015). Living labs and co-production: university campuses as platforms for sustainability science. In Current Opinion in Environmental Sustainability, 16, 1-6. Franz, Y. (2014, September). Chances and challenges for social urban living labs in urban research. In Conference Proceedings of Open Living Lab Days (pp. 105-114). Amsterdam, the Netherlands. Gualandi, E. (2018) Models for Living Lab Financial Sustainability: a Dutch-Italian Case Study. Master Dissertation. Eindhoven University of Technology, the Netherlands. Gualandi, E. & Leonardi, L. (2018) Models for Living Lab’s sustainability: Evidences from Italy and the Netherlands. Proceedings of Open Living Lab Days (pp. 134-149). Geneva, Switzerland.


Gualandi, E., & Romme, A. G. L. (2019). How to make living labs more financially sustainable? Case studies in Italy and the Netherlands. Engineering Management Research, 8(1), 11-19. Guzmán, J. G., del Carpio, A. F., Colomo-Palacios, R., & de Diego, M. V. (2013). Living labs for user-driven innovation: A process reference model. Research-Technology Management, 56(3), 29-39. Hossain, M., Leminen, S., & Westerlund, M. (2019). A systematic review of living lab literature. Journal of Cleaner Production, 213, 976-988. Katzy, B. (2012) Designing Viable Business Models for Living Labs. Journal of Organizational Virtualness. 2(9), 19-24. Leminen, S., Westerlund, M., & Kortelainen, M.J. (2012). A recipe for innovation through living lab networks. Proceedings of ISPIM Conference, Barcelona, Spain. Leminen, S., & Westerlund, M. (2015). Cities as labs: Towards collaborative innovation in cities. In P. Lappalainen, M. Markkula, & H. Kune, Orchestrating Regional Innovation Ecosystems–Espoo Innovation Garden, Aalto, Finland: Otavan Kirjapaino Oy. Mastelic, R., Sahakian, M. & Bonazzi, R. (2015), How to keep a living lab alive? In P. Ballon, & D. Schuuman. Living

Labs: Concepts, Tools and




Mention, A.-L., Torkkeli, M. (2015) Open Innovation: A Multifaceted Perspective. Singapore, World Scientific Publishing Company. Nijssen, E., Hillebrand, B., De Jong, J.P.J., & Kemp, R. (2012) Strategic value assessment and explorative learning opportunities with customers. Journal of Product Innovation Management, 29, 91-102. Niitamo, V. P., Kulkki, S., Eriksson, M., & Hribernik, K.A. (2006). State-of-the-art and good practice in the field of living labs. Proceedings of the 12th International Conference on Concurrent Enterprising: Innovative Products and Services through Collaborative Networks (pp. 349-357). Milan, Italy. Osterwalder, A., Pigneur, Y., Bernarda, G., & Smith, A. (2015). Value Proposition Design: How to Create Products and Services Customers Want. Chichester, UK: John Wiley & Sons. Pierson, J., & Lievens, B. (2005) Configuring living labs for a ‘thick’ understanding of innovation. Proceedings of Epic 2005 (Ethnographic Praxis in Industry Conference). Redmond, Washington. Schuurman, D. (2015) Bridging the Gap Between Open and User Innovation? Exploring the Value of Living Labs as a Means to Structure User


Contribution and Manage Distributed Innovation. Doctoral Dissertation, Ghent University / Vrije Universiteit Brussel, Belgium. Schuurman, D., Ballon, P., Baccarne, B., De Marez, L., & Veeckman, C. (2016) Living Labs as Open Innovation Systems for Knowledge Exchange: Solutions for Sustainable Innovation Development. International Journal of Business Innovation and Research, 10(2-3), 322-340. Schuurman, D., Herregodts, A.-L., Georges, A., & Rits, O. 2019. Innovation Management in Living Lab Projects: The Innovatrix Framework. Technology Innovation Management Review, 9(3): 63–73. Ståhlbröst, A. (2012) A Set of Key-Principles to Assess the Impact of Living Labs. International Journal of Product Development, 17(1-2), 60-75. Ståhlbröst, A., & Holst, M. (2012). The Living Lab Methodology Handbook. Luleå, Sweden: Luleå University of Technology Press. Ståhlbröst, A. & Holst, M. (2016) Living lab: Stimulating adoption of smart city innovations. Proceedings of Open Living Lab Days (pp. 145-162). Montreal, Canada. Veeckman, K., Schuurman, D., Leminen, S., Lievens, B., & Westerlund, M. (2013) Linking characteristics and their outcomes in living labs: A FlemishFinnish case study. In Proceedings of the XXIV ISPIM Conference – Innovating in Global Markets: Challenges for Sustainable Growth. Helsinki, Finland. Von Hippel, E. (2005) Democratizing Innovation. Cambridge, Massachusetts: MIT Press. Wildevuur, S., van Dijk, D., Hammer-Jacobsen, T., Bjerre, M., Äyväri, A., & Lund, J. 2013. Connect – Design for an Empathic Society. Amsterdam: BIS Publishers. Wu, H.Y. (2012). An empirical study of UK living labs. In Proceedings of International Association for Management of Technology IAMOT, pp. 121. Yin, R. K. (2003). Case study research design and methods. Applied social research methods, Second edition, Sage, Newbury Park, California.



Sustainable Person-centred Living Lab for regional management as extension of Japanese dementia care activities Atsunobu Kimura*1, Mizue Hayashi*1, Fumiya Akasaka1, Masayuki Ihara1


*Corresponding authors Service Evolution Laboratories, Japan Category: Research-in-progress

Abstract One of the difficulties of Living Labs (LLs) is ensuring their sustainability. Our research focuses on creative person-centred care activities on dementia for 19 years in Omuta city, Japan. We analysed their sustainable co-creation activities and extracted 3 key functions (pursuing regional philosophy, sharing the philosophy with neighbours and co-creating activities with the neighbours). This paper proposes sustainable LL as regional management method to utilize those 3 key functions. To societally implement the sustainable LL based on Omuta’s philosophy of person-centred (person-centred LL), Omuta future co-creation centre was established in collaboration with Omuta local municipality. The centre and Omuta local municipality tackle comprehensive regional management through sustainable LL activities. Keywords: Philosophy of person-centred, Living Lab, regional management, cocreation, sustainability


1 Introduction Digital technologies like AI, IoT and social robots are rapidly maturing and they are expected to achieve innovation (D'Emidio, 2015). About society, quality of life or well-being topics are receiving attention within Sustainable Development Goals (SDGs) (United Nations, 2015). Against this backdrop, we have to rethink the Socio-technical system (STS) (Trist, 1951) which refers to the tight fusion of technical systems and social systems where it becomes impossible to design and operate separately. For reestablishing a clear relationship between technology and society, LLs (ENoLL, 2016, Leminen, 2015) offer the possibility of establishing a partnership of equality between citizens, companies, and local municipality throughout the entire participatory design process. A systematic review (Hossain, 2019) of 114 academic papers on LLs indicates that one of the big issues is “sustainability”. It says that despite advancements in collaborative innovation for smart cities, few studies address sustainable development. Our research focuses on creative dementia care activities on dementia in Omuta city, Japan. Because Omuta has a public-private organization whose member created cutting-edge activities collaborating with citizens, the local municipality, and companies in 19 years over normal care activities. That’s why Omuta is called holly cite of dementia in Japan. This paper systematically analyzes their activities and extracted 3 key functions which contribute sustainable cutting-edge activities.

2 Analysis of Welfare Activities in Omuta Japanese ministry’s report (Japanese ministry, 2018) says problems of sustainable operation of reginal management organization are following; lack of members, lack of activities for regional problem, less relationship with local municipality and less budget. With reference to this report, this chapter introduces key viewpoints for sustainable co-creation activities. Reginal philosophy is added on those topics, because all activities and plans are based on Omuta’s philosophy of personcentred case and members regard the philosophy is the most important attitude and mind-set. 2.1 Pursuing Reginal Philosophy Problem: If there is no core philosophy, the actions of people in various situations become unfocused so that of activity sustainability is lost. That’s why most organizations and companies create their own mantras called vision, mission, or question. Moreover, some of them change the vison frequently to encourage independence of individuals and/or organizations. Function: To pursue issues for the age to come, it is important that the philosophy provides a way for citizens and concerned individuals to establish their thoughts or culture based on regional history and to deepen and broaden them.


In the case of Omuta, they have, over the last 19 years of person-centred care (Kitwood, 1997), established the philosophy of person-centred activities. The philosophy of Omuta welfare started in 1997 based on the methodology of person-centred care (Nakajima, 2009). Over 20 years, Omuta welfare practices were refined and deepened which has clarified the philosophy; “person-centred”. Person-centred is a concept that captures the attributes of human existence such as "inconsequential", "irrational" and “limitless capability (Sen, 1992)”. Based on the assumption that a human's life is built upon social connections developed through knowing, meeting, linking etc., the good life consists of a limitless variety of capabilities elicited by psychological environment (problem consciousness, curiosity or etc.) and social connection. (Figure 1)

Figure 1. Conceptual diagram of person-centred

Example from Omuta: Here is a story of two women. One senior woman with dementia and the other who had mental illness, both entered hospital at the same time. Both had no family to support them, so the Japanese welfare system would normally send both to “closed” care facilities after medical treatment. But social workers in Omuta felt this would not be the best life. To change this pattern, social workers tried to know and follow each person’s personality, narrative and humanity by talking with them, their family and their neighbors and also watching their home and neighborhood. Common wishes are "wanna go home" and "wanna meet my child". This is, however, often impossible because their home is too old and dangerous. Sometimes they fail to recognize their children. To explore better care solutions, the social worker developed the new approach of house sharing by the two of them (Figure 2). This example illustrates the person-centred philosophy in action.


Figure 2. Omuta practice of house sharing

2.2 Sharing the Philosophy with Neighbors Problem: It is impossible to maintain activities sustainably with few members. To achieve long term success, it is necessary to get new members shared personcentred care philosophy. Function: Practical Leaning provides opportunities for welfare professionals, citizen, local municipality staff and company members to learn and deepen their understanding of life based on the practical person-centred philosophy. Example from Omuta: In Omuta, the above-mentioned public-private council provides dementia coordinator training program. The program is designed to create experts that understand "person-centred care" deeply, support the dignity of dementia suffers in daily life and promote community renovation focusing on the suffers and their families. 2.3 Co-creating Activities with the Neighbors Problem: Before yielding activities for regional problem, it is important to foster close relationships which push forward members to co-create something. If there is no such relationship, co-creation will fail or fake co-creation will take place. Another problem is the cost of fostering such relationships. If a company tries to form deep relationships; it is emphasizing activities that are not directly related to earning money. The company finds it easy to stop the volunteer or CSR (corporate social responsibility) activities unless their commercial activities are strongly connected to developing the relationship. Function: To help close relationship, social workers (SWs) coordinate citizen, companies and local municipality. In daily task, SWs have opportunities to talk with people experiencing troubles and to network with citizen, companies and local municipality to support the people who have troubles. SWs’ activities are already co-creation and also new partnerships are yield through their activities.


This can be said co-creation in daily life which is one of the ideal styles of LL to provide a win-win opportunity for citizens and local municipalities / companies. In this approach, partnership means a relationship to co-create better life and social environment of citizen, not just flat relationship based on workshops or interviews. In some cases, user’s role is simply to live as is and to discuss with company staff about usual activities or reminiscences. Through this communication, company staffs refocus on way of life based on the person-centred philosophy. In other cases, user might talk about his/her inner thoughts only to special people such as family, friends or favored welfare professionals. Collaboration with such people is important in deepening person-centred partnerships. Example from Omuta: Local municipalities are required to establish a total welfare inquiry counter named the local integrated support center. At the support center, welfare professionals hear citizen’s daily life problems and support them or connect them to the resources required. Based on this relationship, they co-create new values with citizens and other stakeholders. One example is the collaboration between a car dealer and a daycare facility. One day-care facility was attempting to help a senior user but he lacked motivation to undertake rehabilitation, he did, however, love cars. He also knew that a local car dealer was suffering a staff shortage. Though discussion among the parties, the senior user rehabilitated himself by washing cars at the car dealer with a helper from the day-care facility (Figure 3).

Figure 3. Omuta practice of car washing for rehabilitation on article from a newspaper


2.4 Relationship with local municipality Example from Omuta: In 2000, a public-private organization named Long-term care service provider council was jointly established by Omuta city and private long-term care service providers in Omuta. Activities collaborating with citizens, the welfare department of Omuta city and those private providers were promoted. Such formation was key point to pursuit activities based on philosophy. 2.5 Financing Example from Omuta: The private-public organization is financed by membership fees paid by private service providers. Omuta city pays outsourcing expenses to a private-public organization as operation fees of activities. The dementia coordinator training is available at cost to other city’s care professionals. Omuta city and this organization also secure some funds from the MHLW for national model projects that explore future care systems and the social impact bond schemes.

3 Model for Sustainable Activities Omuta’s cutting-edge co-creation activities have been continued for over 19 years and we found they have a good mechanism for ensuring sustainability. By analyzing the previous chapter, we develop a model for sustainable activities with 3 key functions, see Figure 4. The three key functions for sustainable activities are interlinked. Pursuing regional philosophies yields educational concepts for value creation. Sharing philosophies increase the number of neighbors who will join co-creation activities. Then co-creation activities provide opportunities to educate the neighbors in practical matters. The educational opportunities ensure that neighbors can rethink the attitudes induced by uncomfortable feelings. Moreover, relationship with local municipality and financing will be considered when concreate organizations are established.

Figure 4. Model for sustainable activities


This model was made from analysis of care activities but is useful for operation of co-creation activities like LLs. We expect this model will be very effective inservice design, for example making products, service development, policy making and community design, and those activities can restructure society based on each philosophy and resolve future social issues.

4 Sustainable Person-Centred Living Lab To collaborate in not only welfare field but also industry field, like healthcare, education, transportation or housing services, among local municipal entities, citizen communities, and private companies, Omuta future co-creation center was established as a private company. The center and Omuta city aim to regional management utilizing the model described above based on philosophy of personcentred. We call it the person-centred LL.

Figure 5. Formation of person-centred Living Lab

Formation of the person-centred LL is shown in Figure 5. One of the key players is intermediator who is a challenge owner from regional point of view bridging company and region and a facilitator. The welfare professional is also key player as he/she has close and long-term relationship with citizens and knows the citizens’ life histories or motivation in life. 4.1 Person-centred cultural meeting at 1st layer The first function is to pursuit the philosophy that underlays all activities. In the VUCA world (Stiehm and Townsend, 2002), the design process starts with questions. We have to think about what kind of society we want to make or what kind of life we hope for. Omuta’s philosophy; person-centred has been discussed in the welfare field, and it is hoped that it will be deepened through discussion with various types of professions; humanities, sociology, engineering and so on. Person-centred design training program at 2nd layer The second function is to share the person-centred philosophy with neighbors and members, like local children and adults, municipality staffs and staffs of


companies. Omuta was the first to start dementia coordinator training in the welfare field, and last year we created a training program for the industry field as a LL education program under the Ministry of Economy, Trade and Industry (METI). Figure 6 shows the mind-set and process of person-centred design. These activities aim to create new services based on the person-centred philosophy.

Figure 6. Mind-set and process of person-centred design

4.2 Co-creation project at 3rd layer The 3rd function is to lay out situations for empowerment of activities with the neighbors. Co-creation opportunities should not be temporary and exclusive like workshop events restricted to certain groups of people. To generate fundamental partnership, in some cases official formation helps people to co-create and in other cases unofficial relationship helps people to co-create. Some citizens like dementia suffers notice their problems in daily life and share them with welfare professionals who have close relationships. We think this is a kind of co-creation. Through such daily dementia care, the person-centred LL accumulates the problems of citizens and regional social problems. The facilitator uses the close relationship with citizens and accumulated problems to coordinate with companies depending on their purpose and resources. This last activity is similar to the traditional LL. 4.3 Relationship with local municipality The Center has partnership with local municipality. For examples, both of them collaborate for SDGs and were selected as SDGs future city in Japanese Cabinet office in 2019. And also they tackle clearance gap social issues, like habitation, transportation or education, which are beyond the jurisdiction of each department of the local municipality. The center member and staffs of local municipality build a team and iterate try and error processes. 4.4 Financing About financing, the person-centred LL elucidates and informs new values based on the person-centred approach to outside stakeholders, and also obtains financial and human support from outside. For example, companies will pay Omuta future co-creation center to assist in the creation of new services in person-centred LL. Municipality staff will pay for training in the Omuta future cocreation center to learn mind-set and process of the person-centred approach. Inside of the region, local municipalities can cut expenditures needed to resolve


social issues like social impact bond (Nikki, 2014). In the case of Omuta city, they trying to not only cut the cost of caring but also to maintain the quality of caring by empowering regional actors who provide informal care services.

5 Summary This paper presented a model for sustainable activities analyzed of Omuta welfare activities and a person-centred LL applied by the model to LL. Omuta future co-creation center was launched to operate activities of person-centred LL and to realize open innovation with light foot-work over sectionalism. The center has already started LL project with citizens, companies and the municipality and will develop new services series based on person-centred philosophy. In future those person-centred services, policies and communities are created in Omuta city, our ideal concept of person-centred City will be realized in which people live as he/she is, to elicit of their capabilities within their social connections.

References Bergvall-Kareborn, B., et al. (2009). Concept design with a living lab approach, Hawaii International Conference on System Sciences. D'Emidio, T., et al. (2015). Service innovation in a digital world. McKinsey Quarterly. European Network of Living Labs, (2016). Introducing ENoLL and its Living Lab community. Folstad, A. (2008). Living Labs for Innovation and Development of Information and Communication Technology, The Electronic J. Virtual Org. and Networks, 10. Hossain, M. et al. (2019). A Systematic Review of Living Lab Literature. Journal of Cleaner Production. Kitwood, T., (1997). Dementia Reconsidered, Open Univ. Press. Lasher, D. R, et al. (1991). USAA-IBM partnerships in Information technology, MIS Quarterly, 15(4). Leminen, S. (2015). What are living labs?, Tech. Innov. Mgmt. Rev., 5(9). Nakajima, T.,(2009) Challenge of Omuta city: Toward the city in which people with dementia live the way he/she is, National community life support center. Nikki, B., (2014), Social Impact Bonds in Emerging and Developing Countries, AV Akademikerverla. Trist, E. L. et al. (1951) Some social and psychological consequences of the Longwall Method of coal-getting. Human Relations, 4(1). United Nations, (2015). Transforming our world: the 2030 Agenda for Sustainable Development, General Assembly. Sen, A., (1992). Inequality Reexamined, Harvard Univ. Press.


Stiehm, J and Townsend, N (2002). The U.S. Army War College: Military Education in a Democracy. Temple Univ. Press. Ministry of Internal Affairs and Communications (2018). Investigation research report on formation of Regional operation organization and sustainable operation (in Japanese).


The value of participatory approaches in developing energy services Joelle Mastelic*1 and StĂŠphane Genoud1

1 Switzerland

*Corresponding author University of Applied Sciences Western Switzerland, Switzerland Category: Innovation Paper

Abstract How can stakeholders be involved in the development of energy services to increase energy efficiency? What is the optimal process for engagement? This is what has been tested in the EnergyLiving Lab, which focuses on energy efficiency and the development of renewable energy. This innovation paper is based on several applied research projects. Its objective is to disseminate research results of a PhD thesis (Mastelic, 2019). The advantages of the Living Lab method for developing energy services are highlighted. The main steps of the Living Lab Integrative Process are summarised in a checklist for professionals and includes: (1) Selection of a practice, (2) Identification of barriers, (3) Integration of stakeholders, (4) Development of a pilot, (5) Measurement of results, Communication and dissemination. In conclusion, this vulgarisation article facilitates the transition from the local to the global scale by encouraging the development of Living Lab mode initiatives in the energy sector. Keywords: Living Labs, Energy Services, Social Marketing


1 The context: Energy services and stakeholders Although the Living Lab method has been in use for a decade, it has not been widely explored in the field of energy. Some papers describe the use of the method for energy savings, such as in Krogstie et al. (2013). The Energy Living Lab presented in this innovation paper has been developing projects since 2014 specifically in the field of energy efficiency and the promotion of renewable energies. The key question is to know what value co-design methods bring to a field that is still very much driven by technological innovations. This article proposes approaches for integrating stakeholders in the development of energy services using the Living Lab method. 1.1 What is an energy service? A 2017 study by Michael James Fell indicates that only 0.5% of the 185 scientific articles analyzed by two major energy journals mention “energy service,” and only 10% of these studies define what an energy service is. Clarification has, therefore, been required, and Fell proposes the following definition: “Energy services are those functions performed with energy that are means of obtaining or facilitating the desired services or end states.” Lighting, for example, is an energy service that can be produced using different primary energy sources and leads to a desired end state: illumination in the home or office. How does this service create value? According to marketing researchers Vargo and Lusch, value creation occurs when the service is consumed by the customer (2004). A watt lost during transport does not create any value. In marketing, we are considering the consumer as a co-creator of value. It is, therefore, always a value perceived by the consumer. It is sometimes thought that consumers are not rational, for example, when they leave windows open in winter. From their point of view, their actions are rational because otherwise they would not behave in this way. For a specialist, it is often difficult to put oneself in the consumer’s shoes and understand these types of practices that are detrimental to energy efficiency (EE) and the environment. 1.2 The intangibility of energy services This perceived value causes problems for energy service providers. Indeed, the value of services is often not perceived. Users expect to benefit from services such as heating, lighting, and ventilation. They realize the value of such services only when they experience poor quality or an interruption in service. This is what was measured in our previous project in a sustainable neighborhood (Mastelic et al., 2016). When the heating system fails in the middle of winter, the value of the energy service is realized by its absence. How, then, can we raise awareness among users of the value of energy services when everything is working well? It is a challenge to make energy tangible, to make it visible. Today, we are witnessing a decoupling between primary energy and “the desired end state”. During our grandparents’ time, people were well aware of the primary energy needed because they had to feed the fire to warm their homes. They had to cut wood, carry it, dry it, and manage the supply to the furnace or woodstove so as not to let the temperature drop. There was a direct and tangible link between primary energy and the desired end state. And what our grandparents


experienced 100 years ago, our African neighbors still experience every day. In Europe today, it is mostly an automatic system that does the work for us, and as a result, most of us have lost this link with primary energy. 1.3 Raising user awareness Energy planners then consider solutions to make the population aware of energy savings. Appealing to users’ common sense, they think they can solve the problem by carrying out educational actions. Unfortunately, for the past two decades, social marketing scholars haveindicated that, although necessary, awareness and education are not enough to get users to take the desired actions (McKenzie-Mohr, 2000). It is also difficult to produce lasting effects over time, and this type of intervention must be repeated. Admittedly, mentalities have changed, and today, a larger segment of the population declares that it wishes to act in favor of the environment. Attention must be paid to the potential gap between attitudes, intentions, and actions (Kollmuss and Agyeman, 2002). In a sustainable district that we studied and according to the results of the tree of correlations between the satisfaction of living in the district and the various services provided, energy was not among the priorities; they were looking first for satisfactory social relations, an attractive place to live, and a location (Mastelic et al., 2016). 1.4 Limited perceived control The power to act personally is often underestimated by stakeholders. Most people think there are other, more competent people who will act to increase EE. The problem is not being addressed, as demonstrated by our study on the development of an energy management system (Mastelic et al., 2017). If no EE targets are set by management, employees will not take the ownership of the challenge and leave it to the management team. If employees (and especially custodians) are not measured annually on their energy performance, then why should they act? In addition, stakeholders often do not have much knowledge about energy; they have low “energy literacy.” The notion of kW/h is highly abstract for them, and they don’t know how much it can represent. Most of the people interviewed in past studies do not know how much they spend on energy, and even if they do, the percentage is a relatively low one based on their total budget (about the price of a cup of coffee a day for electricity in a household) and, thus, provides little incentive for EE actions. Subsidies can, to some extent, provide an incentive for action. 1.5 Automation of energy services EE specialists, therefore, imagine increasingly complex technical systems that can solve problems. These include automatic lighting with presence sensors, regulation of temperature control in buildings according to theoretical occupancy data, and the use of windows that prevent opening. They use technical and economic models to implement them and sometimes forget that these systems will interact with users. Indeed, as proposed by Geels, they are socio- technical systems composed of rules, technical artefacts, and human actors (2004). User are sometimes forgotten or relegated to the end of the process to test the final service, thought of in a patriarchal way as: “We know what is good for you”. Users often do not oppose active resistance in this phase as they have a low technical knowledge and tend to delegate ownership of the challenge to experts.


The problem is that, if the technical solution and/or the rules put into place (system regulation, laws...) do not meet their requirements, users will find inventive ways to try to circumvent the system (bypass use). When specialists encounter blockages, researchers in social sciences are often asked for interventions that promote “social acceptance,” a kind of magic wand that would be used to accept technical solutions which do not work optimally. Unfortunately, it is often too late to change the technical artefacts, and only small adjustments can be proposed. It is, indeed, at the start that action should be taken.

2 Ways to promote stakeholder engagement How to engage stakeholders in a co-design process to co-develop energy services? This is what has been tested in the phd thesis (Mastelic, 2019). The following section gives recommendations for professionals based on four years of research. 2.1 Transformative research and quasi-experimentation When we want to act on energy services, we act on a complex system. In such a system, the relationships between variables are not linear, and the preferred way to test whether or not an intervention has an effect is through experimentation (Kurtz and Snowden, 2003). This is referred to as quasi-experimentation because we do not have a random sample of the population. In addition, we work in the field, in situ, and therefore, we cannot control all the variables in the system. The effects of certain variables, such as the influence of weather on energy consumption, must therefore be considered. We are attempting to move from one energy system to another, so we want to change the reality we observe. This is not considered acceptable in all scientific disciplines. We are in a constructivist epistemological paradigm: the researcher is not neutral; we conduct action research that is called “transformative” because it transforms the observed reality (Schneidewind, 2016). The author tested the Living Lab approach to co-design energy services with stakeholders. 2.1 What is a Living Lab? The Living Lab (LL) approach is relatively new. It has been in place for about ten years at the European level, but few experiments exist in the field of energy services. Research on the “Living Lab” phenomenon is, therefore, in its infancy. Definitions abound, but it remains difficult to capture the complexity of the phenomenon. We propose our own definition in the thesis: “A Living Lab is an innovation intermediary, which orchestrates an ecosystem of actors in a specific region. Its goal is to co-design products and services, on an iterative way, with key stakeholders in a public private people partnership and in a real-life setting. One of the outcomes of this codesign process is the co-creation of social value (benefit). To achieve its objectives, the Living Lab mobilises existing innovation tools and methods or develop new ones.” (Mastelic, 2019).


What differentiates the Living Lab from other participatory methods is the combination of several factors, listed in the definition and detailed below: 1) An ecosystem of stakeholders: This laboratory emulates a partnership between public authorities, companies, citizens, and academics. The Living Lab manager acts as a catalyst to build a common vision, provide methods, coordinate experiments, and measure results. To select the key stakeholders, we use the power/interest matrix, detailed in another article (Mastelic, 2017). 2) Co-designing solutions: The prefix co- means “with.” We do not develop solutions for users but rather with users. 3) An in situ environment: Research does not take place in a laboratory but in the place where energy is consumed or produced; it adapts to different contexts. 4) A societal-improvement objective: A strategy for individual well-being is not developed but rather societal well-being is the aim. 2.3 Living Lab for energy services The Energy Living Lab was created in 2014 to support the development of field interventions and to help achieve the objectives of the Swiss Federal Council’s Strategy 2050. It operates in two main sectors: increasing energy efficiency (particularly in the built environment) and the diffusion of renewable energies. After an initial test in the Chablais region, the Living Lab increased its interventions in a portfolio of projects supported by public and private funds (work on the energy performance gap, the dissemination of renewable energies in a region, the dissemination of photovoltaics in French-speaking Switzerland, the deployment of district heating systems, etc.). Some of these interventions will illustrate the purpose and feedback below. 2.4 The process developed by the Living Lab The process is based on “Community-Based Social Marketing” as proposed by McKenzie- Mohr (2000) and integrates knowledge of social marketing and social psychology. Marketing gives particular importance to the choice of targets for interventions. You can’t talk to “everyone” because it’s the most effective way to avoid addressing anyone. Social marketing, then, focuses on barriers to proenvironmental or pro-social action. Understanding barriers is key to understanding why EE interventions fail. Measuring results is also central in marketing because, in order to achieve objectives, it is necessary to know how to determine performance indicators and how to measure them. As part of the research, we combined social marketing and the Living Lab approach; this involves analysing and integrating key stakeholders from the beginning of the value chain. Non-specialist actors work alongside experts to co-design a solution (product/service/action plan). It is also important to go into the field quickly to test the proposed solution under development and then return to development iteratively (agile methods). More information on social marketing can be found in the thesis (Mastelic, 2019): 1) Selecting a practice. An analysis of existing data is conducted to determine which practice(s) we want to act on. In the sustainable district studied, for example, the focus was on heating and mobility. We attempt to take a step back and choose according to the context and data rather


than choosing a field of use a priori. The PESTEL model (political, economic, social, technological, ecological, legal) can also be used to understand the complexity of the usage context. 2) Integrating stakeholders. Stakeholders are listed, and then a matrix is used to classify them, in this case, the power/interest matrix (Eden & Ackermann 1998, in Bryson, 2004). We look at who has the power and who has the interest in changing the service, for example, in the case of heating. If citizens are not the ones with the power to make an impact, why focus on them? (Mastelic et al, 2017). We place our research hypotheses a priori in this matrix because we do not know a priori who has the power and interest. Efforts will also be made to integrate four types of stakeholders, the Quadruple Helix model (Carayannis & Campbell, 2012): academics, public authorities, companies, and citizens/users of the energy service. 3) Identifying the Barriers. Qualitative interviews will then be conducted with each of the key stakeholder groups based on the matrix, in order to better understand their perceptions regarding the level of power and interest, their motivations, and the barriers. In one of the fields studied, even the director of the school thought he did not have the power to change the situation. There was no energy-saving objective. Who does have the power if the director doesn’t think he holds it? (Mastelic et al., 2017). There is often a lack of ownership of the challenge. In LLs, the first step is to encourage stakeholders to take ownership of the challenge. This is achieved, for example, by asking the director in front of the stakeholder assembly: “What is your goal for energy efficiency?” A public and voluntary commitment is necessary. This allows better ownership of the project and future results (Cialdini, 2001). Once we have completed this stakeholders’ analysis, we compare our research hypotheses with what stakeholders have mentioned. We redraft a power/interest matrix in relation to their perspectives and see if there are any differences. We also analyse all the other barriers to action such as, for example, lack of cash flow, low motivation, and technical ignorance. 4) Co-designing the solution. We then collaborate with stakeholders to codesign a common vision. We have had cases where companies have seen municipalities as actors who are there only to place obstacles in their way. Conversely, sometimes municipalities see companies as interested only in their profit margins. If we do not offer this space for dialogue between actors to correct certain biases, they cannot co-develop a common vision. Qualitative interviews are always conducted first to gain a clear understanding of individual barriers to change and then to moderate this type of multi-stakeholder process that involves building trust between stakeholders, we have developed a model to build trust among stakeholders, detailed in anther article (Dupont et al., 2018). Simply bringing people to the table and providing an environment that fosters trust and the development of a common vocabulary often helps to move the


process forward. Many tools exist to co-design energy services and are presented in other works by the same authors. 5) Piloting an experiment. Primary importance is given to testing codesigned solutions in a real-world situation with authentic users. Agile methods facilitate regular trips back and forth between the experimental field and theory. Today, we see the emergence of laboratories in which researchers live and test new products and services. Although these experiences are similar to a real-world environment, they do not convey the actual and authentic conditions of use, including users’ knowledge. Thus, barriers may be overlooked related to technology adoption, culture, lack of time, and a range of social factors that will not be reflected in the test environment. 6) Evaluating performance. A measurement and verification plan is proposed before the pilot. EE standards can be referred to, for example IPMVP14. A large volume of data is now available, such as energy consumption from smart meters. To interpret the results, it is often useful to collect sociodemographic data using surveys and to cross-reference them with consumption data to provide analyses by consumers’ clusters (Cimmino et al, 2016).

3 Impacts of Living Labs on energy services The Living Lab approach, tested in several research projects mandates, is beneficial from many perspectives, as presented below.


3.1 Reducing barriers to change The approach provides a better understanding of barriers to action from different perspectives. This focus on barriers is similar to the social marketing methods. A new and “naive” look at the energy challenge is taken. Indeed, barriers are not always found where we expect to find them. An example is the development of district heating system (DHS) in an article from Previdoli and her colleagues (2015). The contracting authority thought that prospects could raise economic barriers, as the installation of the network has high initial costs. However, after stakeholder analyses, a different and unexpected barrier emerged: resistance to change from the environmental service. For years, the service had required the switch from oil to gas. With the arrival of the DHS, a new argument had to be developed to convince employees in contact with prospective users and then the prospects themselves. Without involving all key stakeholders and focusing only on users, this important barrier to deployment could have been missed. 3.2 Development of a common vision An ecosystem of actors is integrated into the reflection from the very beginning of the projects. The first step is to network the partners and define a common vision. Interactions and mediation promote trust-building and knowledge-sharing 14 International Performance Measurement and Verification Protocol


(Dupont et al, 2018). The LL method makes it possible to build bridges between actors and between disciplines. An example is the municipality of Saint-Martin. The president of the village tested the LL method to develop renewable energies in the village. School children, aged 12 years old, teamed up with engineering and economics students from the university to propose a renewable energy development plan for the municipality. The best project was selected by experts and presented by the young people to the primary assembly. The latter accepted the budget for the preliminary studies for the deployment of the plan in the village by local companies. 3.3. Increased perceived control by change agents Stakeholders sometimes feel that they cannot act. The example of the hummingbird, cited by Pierre Rabhi, comes to mind. The little bird is the only animal who attempts to extinguish a raging forest fire by filling his beak with water and spraying it. The other animals tell him not to bother as he won’t make a difference, and the tiny bird answers, “Maybe so, but I’m going to try.” In the energy field, stakeholders also need examples, support, and reassurance to give them confidence that they can act and have an impact. Our project in LL mode tested the TupperWatt evenings, workshops that bring neighbours together around the theme of EE. An expert attend and presents technical solutions. The participants can freely express the obstacles they foresee, and the exchange of experiences is rich in learning. EE materials are then offered for sale, and participants are given a “good practice” guide and a small gift. In this way, they become aware that they can act at their own level. 3.4 Changing the role of users Users become active co-designers of the energy services and even sometimes prosumers. Users have often lost their link with primary energy; they have become passive consumers of automated services. As part of the LL’s activities, we are trying to give them back an active role as co-designers alongside specialists. For example, we have developed a serious game: the poker design (initially proposed by Cité du Design from St-Etienne). After conducting qualitative interviews with stakeholders, personas, a kind of stereotype of the system’s actors, were created. For example, there is the energy distributor, the elderly person in subsidized housing, the employee of the real estate agency, etc. We then developed three types of cards: (1) persona, (2) actions, (3) uses. Stakeholders were able to combine the cards to develop an EE plan for their neighbourhood. For example, the “energy distributor” card is chosen, which “encourages” the “inhabitants” to “install a HYDRAO shower head to save water.” The persona allows users to take a step back from their own practices and to avoid guilt. The game stimulates discussions and the co-design of solutions to be implemented in the neighbourhood. 3.5 Increasing the social acceptance of systems When stakeholders come up with ideas to improve energy services, they express their own needs. Co-designed solutions are closer to these needs. They are all the more easily adopted by the actors, even if they did not participate in the codesign process. In the example of Saint- Martin cited above, the whole village is


behind the projects of mini hydraulic turbines in drinking water systems, joint tendering for solar panels on public and private roofs, and DHS in the village. The assembly unanimously voted the budgets to implement the plan developed by the children and students, and local companies are working on it. 3.6 Reconnection to primary energy When working with participatory methods, actors re-examine questions that they had omitted, such as the source of primary energy (“Where does the energy I put in my car or in my heating system come from, and who benefits from the money I spend to purchase it?”). Several illustrations of this phenomenon can be cited. During the qualitative interviews conducted to understand the motivations and barriers of the actors to the deployment of a DHS, many understood the need to valorize the energy from waste rather than importing fuel oil (article by the same author). The mental images communicated are often more meaningful than the words: “Imagine the distance that fuel oil must travel in tankers and then by road to get here.” The argument of local energy speaks to many and favors the circular economy: the forester who feeds the DHS with wood waste from the local forest reinvests his salary locally. 3.7 Piloting the co-designed service The quasi-experiments used in the LL method allow the solutions to be tested in situ to see the results of the system. This allows quick round-trips between the field and R&D to adapt the energy service. For example, one idea that emerged several times in the co-design workshops was to develop a large red button for households to turn off all sources of unnecessary electricity consumption when leaving the house. This is a simple idea based on a common need and could be installed by default in senior citizens’ housing, for example. Its implementation is more complex, however: Do we really want to cut everything off? This requires prior testing in a real-world setting, which is the case in the studied sustainable neighbourhood.

4 Conclusions How can we achieve the energy transition while engaging key stakeholders? The share of household energy consumption is very high, accounting for about half of the total consumption. It is, therefore, illusory to imagine an energy transition without citizen. It is also completely unrealistic to imagine that, by giving them information only, this could be enough to get household users to drastically change their consumption practices. Although they represent about half of the energy consumption, these consumptions are highly diffuse, with very varied uses. It is, therefore, difficult to establish a cost-effective economic process to help them reduce their consumption because, apart from communication solutions, which have demonstrated their limits, the time required to help them achieve an energy transition within the expected time frame generates significant consulting and support costs. This transition will have to be rapid and will be very difficult to achieve without effective participatory approaches. LL’s approaches are perfectly aligned with this spirit, as co-designed solutions are more easily accepted. Many indicators demonstrate the awareness of the population—and young people, in particular—of the need to reduce our impacts on the


environment. This is very encouraging and adds one more reason to help redefine market rules. We must accelerate the energy transition, for example, in the field of construction where, with a rate of 1% per year of renovation of existing buildings, it would take us 100 years to refurbish our real estate assets. We obviously don’t have that time available. Today, a large part of the activities conducted by economic actors i will have to be refocused on actions around the energy transition. This is also true for universities and the institutions that finance them. They should contribute with a managerial impact to the reduction of CO2 emissions and the production of renewable energies, not only through theoretical contributions. There are currently many solutions on the market that are energy efficient, both technically and financially. The deployment of these solutions should be supported with as many resources as fundamental research on technological solutions. Clearly, technology must continue to evolve, but in today’s society we need much more action around stakeholders’ participation, who will ultimately decide whether or not to join the collective effort. This is certainly the main benefit of “action research” to promote the energy transition that really contributes, here and now, to increasing the production of renewable energies and reducing CO2 emissions.

5 Checklist for getting started with the “Living Lab Integrative Process” The Living Lab Integrative Process is explained step by step in the following checklist. The aim is to transmit the standardised method to professionals and researchers wanting to experiment it. Figure 1. Living Lab Integrative Process

1) Selecting a practice Study the available data on your energy service. What are the practices of the actors that have a strong impact (positive or negative) on the efficiency of your energy service? Select between 1 and 3 practices (e.g., taking a bath instead of a shower, leaving windows open, changing the temperature set point, allocating charges to the residential surface...). Try defining the “roots” of the problem and not only the symptoms. 2) Integrating stakeholders Make a list of stakeholders who have influence over your energy service. Try to place them on the power/interest matrix (your own assumptions): (e.g., the commune’s energy delegate, the building janitor, the end users, the financiers of the solution, the energy distributor...). (Eden & Ackermann 1998, in Bryson, 2004).



Context Setters






Level of Interest


Level of Power

Figure 2. Power-Interest Matrix, adapted from Eden & Ackermann in Bryson 2004.

3) Identifying the barriers Interview the key players individually (box: Keep satisfied, manage closely, and Keep informed). Are your assumptions true? What are the barriers and levers of action of these actors toward efficiency? 4) Co-designing the solution Then bring together the key players (e.g.: workshops, world cafĂŠs, BarCamps). Ensure that you invite the four types of actors: companies, public authorities, citizens/users, academics. Work toward developing a common vision and shared objectives for your energy service. Co-develop the solution WITH users, not FOR users (using design service, design thinking, crowdsourcing, etc.). Adapt your vocabulary to an audience with a low level of energy knowledge.


5) Piloting an experiment Test the co-designed solution in the field and not in the offices! Collect feedback to improve your energy service (interviews, ethnographies). Perform as many iterations as necessary without waiting for a final prototype (agile methods). 6) Evaluating performance Establish the measurement and verification plan before the pilot (e.g., IPMVP) and evaluate the results regularly. Triangulate the data to verify your conclusions (qualitative/quantitative, simulation/real consumption data etc.). 7) Communicating results and replication Communicate the results of your project to all stakeholders and celebrate success with them (media communication, end-of-project event, etc.). Share your success to allow others to replicate it (open innovation, open science).

References Bryson, J. M. (2004). What to do when Stakeholders matter. Public Management Review, 6(1), 21-53. Carayannis, E. G., Campbell, D. F. J. (2012), Mode 3 Knowledge Production in Quadruple Helix Innovation Systems. New York, NY: Springer New York. doi: 10.1007/978-1- 4614-2062-0_1. Cialdini, R. (2001), Harnessing the Science of Persuasion, Harvard Business Review, 79(9), 72-81. Cimmino, F., Mastelic, J., & Genoud, S. (2016, September). Multi-method approach to compare the socio-demographic typology of residents and clusters of electricity load curves in a Swiss sustainable neighbourhood. In 2016 ENTRENOVA Conference Proceedings. Dupont, L., Mastelic, J., Nyffeler, N., Latrille, S., Seuillet, E. (2018), Living Lab as a Support to Trust for Co-creation of Value: Application to the Consumer Energy Market, Journal of Innovation Economics and Management, DeBoeck Superior. Fell, M. J. (2017). Energy services: A conceptual review. Energy Research & Social Science, 27, 129–140. Geels, F.-W. (2004). From Sectoral Systems of Innovation to Socio-Technical Systems: Insights about Dynamics and Change from Sociology and Institutional Theory. Research Policy, 33(6), 897–920. Kollmuss, A., & Agyeman, J. (2002). Mind the gap: why do people act environmentally and what are the barriers to pro-environmental behavior? Environmental education research, 8(3), 239-260. Krogstie, J., Stålbrøst, A., Holst, M., Gudmundsdottir, A., Olesen, A., Braskus, L., ... & Kulseng, L. (2013). Using a Living Lab methodology for developing energy savings solutions.


Mastelic, J., (2019), Stakeholders’ engagement in the co-design of energy conservation interventions: The case of the Energy Living Lab, doctoral thesis, University of Lausanne. Mastelic, J., Genoud, S., Cimmino, F.M., Previdoli, D., Fragnière, E., (2016), Perceived value of energy efficiency technologies in a sustainable neighborhood: an empirical enquiry from the Energy Living Lab, Conference Proceedings, Open Living Lab Days 2016, Montreal. Mastelic, J., Emery, L., Previdoli, D., Papilloud, L., Cimmino, F., & Genoud, S. (2017). Energy management in a public building: A case study codesigning the building energy management system. In 2017 International Conference on Engineering, Technology and Innovation (ICE/ITMC), pp. 1517-1523, IEEE, DOI 10.1109/ICE.2017.8280062Rabhi, P. (2006), La part du colibri. L’espèce humaine face à son devenir, Ed. MiKros.



Theoretical & Methodological Living Lab Challenges 336


Blockchain, a promising way for scaling up cocreation of innovation from local to global Eric Seulliet1

1 La

Fabrique du Futur & Co., France Category: Innovation Paper

Abstract It is difficult to mobilize co-creating users over time: lack of motivation to contribute, difficulty in capitalizing on contributions. The outcomes of traditional co-creation processes are therefore often limited in scope. The use of blockchain is one way to overcome these limitations and scale up. By allowing contributors to be recognized or even remunerated, the blockchain produces a "nudge" effect thanks to the climate of trust it creates. Thanks to the traceability and capitalization of contributions, the blockchain also makes it possible to make the most of them by promoting their recombinations. Finally, the blockchain allows new approaches to intellectual property to be unleashed and new types of organizations (DAO) based on sharing to emerge. Keywords: Blockchain, Nudge, Trust, DAO, Collective Intelligence, Motivation, Engagement


1 Introduction The interest of co-creation is increasingly recognized in innovation approaches. Co-creation makes it possible to optimize and enrich these approaches at all stages of the innovation process. At the ideation stage, co-creation makes it possible to produce more ideas and, above all, it generates a biodiversity of creative ideas, these coming from a plurality of actors. The observation of various situations and practices in real life contexts, which are the very basis of living labs approaches, also makes it possible to detect more emerging uses. Then, when it comes to experimenting with creative approaches, the multiplicity of actors allows faster iterations and generates more relevant results. In the development phase of desirable and viable solutions, co-creation makes it possible to build a constructive climate through the mobilization of the collective intelligence of stakeholders. Thus, co-creation is the intrinsic value of the Living Lab movement and is its fundamental characteristic. However, co-creation processes encounter a number of limitations that prevent them from having a greater impact. To sum up, one could say that the challenges of co-creation are both of a quantitative and qualitative nature. This double glass ceiling that co-creation faces must absolutely be crossed if we want the actions of living labs to be more recognized and that living labs move from a sympathetic consideration to a real recognition. It is at this price that we can hope to ensure that the results of co-creation have a real impact, much broader than those often too limited in traditional communities of co-creators. Our paper proposes to share, modestly, our insights on the issue based on our own research, our experience, and our exchanges with professionals. Thus, our conviction is that the blockchain can be a formidable lever to boost co-creation and open innovation and boost it from an artisanal stage to a larger scale. Our paper presents concrete examples and we also present a project in the process of being launched as well as our thoughts on future directions for blockchain in a context of co- creation.

2 The limits of co-creation Co-creation, as mentioned above, faces two kinds of limitations: Quantitative limitations Co-creation approaches encounter a quantitative limitation as it is often materially difficult to gather a large number of contributors.


The scope of the innovations produced also seems limited, since contributors are generally recruited from a limited circle of users, who are often endogenous in nature. If we really wanted to scale up co-creation, it is necessary to mobilize many more co-creators. Qualitative limitations A major problem in a co-creation process is to be able to sufficiently and sustainably mobilize the participants in the process. Several factors can explain these difficulties in mobilizing and involving co-creators. There are above all psychological difficulties: These difficulties are above all problems of envy, jealousy and competition between individuals, which tend to lead to a natural tendency for everyone to overvalue their contributions and to minimize those of others. Distrust, jealousy, denigration that can combine with this lack of discernment lead to withholding of information and reluctance to cooperate. The difficulty in measuring, and therefore valuing, the various contributions and their impacts is another obstacle. Another important factor to be considered is the issue of intellectual property. Current legal solutions (patents, NDAs, etc.) do not offer a satisfactory answer, particularly in the case of multiple contributors from different organizations, particularly for large-scale projects or projects that address societal issues. Participants in a co-creation process may be frustrated by having to abandon any claims in this area and feel that they are deprived of the opportunity to value their contributions. Therefore, this is undeniably a barrier to their motivation to get involved. These situations can generate inertia and losses leading in the first place to a lack of overall efficiency. Human beings also need a minimum of personal recognition. They may be willing to engage in a collective process, for example in an ideation sequence based on brainstorming to collectively produce creative ideas, but their full engagement will quickly be limited if they feel that their contributions are not being credited to them. The saying "let's repay to Caesar what belongs to Caesar" is most valid in this case. Organizational and logistical difficulties: How can co-creation approaches be optimized to make the most of them? How can we best capitalize on each other's contributions in such a way as not to "reinvent the wheel" but on the contrary to ensure that we build real avenues for innovation? How to effectively sort contributions and prioritize them? How to evaluate them? How to organize productive group meetings? What are the most effective methods and means? What are the most appropriate environments and contexts? These are all questions that do not easily find answers in the usual co-creation processes.


Nor does classical co-creation seem suitable for producing radical innovations because it is based above all on everyday life experiences. Moreover, it is not easy to prioritize and structure the insights collected because they are delivered in a disorderly manner. In practice, it is also complicated to recruit co-creators, it is difficult to motivate them over time..... In addition, the whole process is rather slow because of the necessary iterations that it requires. The process must also be organized and animated, which requires human resources and tools. Living labs, for which the issue of user involvement is particularly important, seem to be increasingly aware of the need to find ways to take the co-creation of innovation to a more advanced stage. It seems that the solutions can be combined around two main axes: The nudge axis: In concrete terms, nudge consists in indirectly encouraging individuals (citizens, clients, patients, etc.) to adopt new behaviors considered responsible and beneficial for themselves and the community, by modifying the context of their decision-making, without forcing them. This concept of nudge was popularized by economist Richard H. Thaler, who received the Nobel Prize for Economics in 2017. This concept of nudge obviously applies perfectly to the issue of the involvement and motivation of co-creators. Two examples can be mentioned in this regard from the latest OpenLivingLab Days in Geneva in 2018: • A workshop entitled "Involving end-users: intrinsic or extrinsic rewarding" clearly questioned what motivates participants in a co-creation group. The outcome stated: “the discussion was interactive and very vivid. The biggest motivation for end-users to participate is intrinsic (acknowledgment, contribution, ownership, social contact), but extrinsic rewarding must be taken into consideration because it helps to get a good mixture of participants. A monetary incentive (voucher, cinematicket, freelunch) can be an important trigger for some target groups”. •

Another workshop entitled "Technology, Goal setting and behavioral nudges: 1000 ways to save energy" focused on the question of the incentives needed to motivate residents and building users to save energy.

The trust axis It all boils down to the question of trust: to co-create and co-innovate with complete serenity, it is essential to establish a climate of trust in the first place. The workshop on the search for energy savings mentioned above was extended by a research work that gave rise to an academic paper entitled “Living lab as a support to trust for co-creation of value: application to the consumer energy market15” which highlighted this question of trust: “The notions of trust and contract are strong issues in Living Labs. Indeed, Living Labs as innovation 15


networks require the establishment of strong links of cooperation and collaboration.../... In addition, the mobilization of actors is generally based on mutual trust, which must be built and nurtured”. This research work went as far as developing a tool to assess trust in a living lab process (Co-coon Matrix). Trust is a prerequisite for individuals to engage together and combat the natural tendencies of human beings to put forward their own point of view. This need of trust plays out on several levels: • Trust in contractors and other stakeholders • Trust between group members, especially in the case of larger groups, with concentric circles of participants • Trust in external partners and contributors • Trust in the facilitation methods and teams Trust in the tools used in co-creation processes: a certain reluctance to use open innovation tools due to the lack of trust that contributors may feel towards current co- creation and open innovation platforms, particularly with regard to the protection of their personal data.

3 The blockchain to boost co-creation In this second part of our paper we will share our deep conviction that the blockchain is an extremely promising path for co-creation. We are convinced that the blockchain is an extraordinary opportunity for co-creation both qualitatively (better results) and quantitatively (possibility of extending the reach, finally allowing it to scale up from a confidential to an extended stage). We will support our point by giving examples and presenting a concrete project. We will also indicate what implementation precautions have to be taken and we will present our vision for future developments that can be anticipated. What is blockchain? The blockchain is a transparent and secure information storage and transmission technology, operating without a central control body. By extension, a blockchain (sometimes called DLT "Distributed Ledger Technology ") is a database that contains the history of all the exchanges made between its users since its creation. This database is secure and distributed: it is shared by its various users, without intermediaries, which allows everyone to check the validity of the chain. While its first applications were limited to the field of fintechs. Bitcoin is the most well-known use case of blockchain. It was created in 2008 by an unknown person whose pseudonym is Satoshi Nakamoto. It refers to both a secure and anonymous payment protocol and a cryptocurrency. Blockchain is now finding new uses in many sectors: energy, health, logistics, intellectual property, etc. As far as intangible capital is concerned, one area that seems particularly promising for the blockchain is that of co-creation and open innovation. Blockchain and co-creation The blockchain was initially linked to monetary transactions in the field of cryptocurrencies. But what is interesting is that by extrapolation it can be applied to all


kinds of transactions. But what is an innovation process if not transactions of ideas, suggestions, creative and inventive contributions? By broadening the concept of innovation, it can also be described as transactions in intangible assets. -


Therefore, all the advantages of the blockchain can be applied to cocreation: Traceability of the co-creator’s contributions: this makes it possible to identify precisely who was at the origin of what and also to capitalize them. Transparency allows for the recognition of the authorship of ideas and contributions Authenticity of transactions The capitalization of contributions has an essential effect: this capitalization makes it possible to build new paths through the remixing of innovative ideas and tracks. It is a powerful lever for creativity and inventiveness. The interest of the blockchain is that it makes it possible to go beyond the debate mentioned above about co-creation (quantitative improvements vs. qualitative improvements): by making it possible to capitalize on ideas and contributions, the blockchain makes it possible to avoid "reinventing the wheel", by constantly remixing them, it constantly enriches them, thus arousing the desire to push them even further. At the same time, by allowing everyone to express themselves, the blockchain maximizes contributions. Quality of contributions x quantity of contributions results in a virtuous spiral of creativity and inventiveness!

The Nudge and Trust axes have been mentioned above. It is striking to note that the blockchain allows precisely major contributions in these two areas: Trust: Blockchain is a technology that has intrinsic virtues of transparency and equity. It generates a spirit of collective sharing and intelligence as well as a sense of belonging to a community. It thus promotes the reunion of individuals who share common values and are oriented towards the same goal. All these qualities make the blockchain an ideal instrument for building trust. Moreover, by protecting intangible assets, the blockchain makes it possible to secure innovators, thus increasing their trust in the system, which in turn encourages their involvement.


Nudge: By ensuring the traceability of contributions, the blockchain enables to know who was at the origin of the value creation in a co-creation process. And even if this is a collective process, it ensures that each contributor is truly recognized. And when we know that individual recognition is a source of motivation, we see all the potential that blockchain can bring to boost co-creation. But of course, blockchain can allow this nudge effect to go far beyond simply distinguishing individuals for their personal contributions. The blockchain may, for example, include a scoring and voting mechanism to assess individuals for their contributions by their peers. The blockchain can also go further in encouraging by setting up a system of remuneration for contributors through tokens. These incentives make it possible to create a sound emulation between contributors in a spirit of cooperation. Therefore, with these trust and nudge characteristics, it is not surprising that blockchain can be considered as the technology with the most potential to advance the co-creation of innovation. The blockchain can considerably increase the business value for co-creation approaches through its intrinsic added values: more inventiveness and creativity are generated while there are fewer losses thanks to the traceability of these ingredients, possibility to involve more participants. Blockchain, co-creation and Living Labs Several initiatives have taken place in recent years under the impetus of a living lab around the role of the blockchain in the field of co-creation and open innovation, evidencing that the subject of co-creation and blockchain is a brandnew research domain. Among these initiatives, we can mention: • The creation in July 2016 of a LinkedIn group entitled "Blockchain, open innovation and co- creation"16. To date, it has more than 1140 members and generates extensive exchanges, which shows the interest of the subject. • Conferences such as the three editions of the Blockchain Agora17 event held in Paris in December 2016, December 2017 and November 2018, which brought together more than 120 people each time. Each event had a particular angle related to the general theme of blockchain and cocreation: the 2016 edition focused on coopetition, the 2017 edition on the empowerment of individuals through blockchain, the 2018 edition on the positive and societal impact of blockchain. • Workshops such as those held during the last two OpenLivingLab Days (first one in Krakow in 2017 which was an introduction to the blockchain, the second one in 2018 in Geneva, in a more elaborate and concrete form which generated great interest), or during The Journée Innovation in May

16 17


2018 in Paris on the theme of innovation ecosystems, which brought together several living labs. Publication of articles dedicated to this theme, notably for Harvard Business Review France.

These initiatives were diverse but had the common objective of arousing interest in the new field of blockchain and co-creation, identifying experiments, identifying actors, creating partnerships in this field, evaluating the potential of blockchain for co-creation. They also generated a lot of reactions and testimonials which have been quite useful for digging the subject of co-creation and blockchain. Several projects have been identified in this specific field. These projects range from the simplest (identification of contributors' contributions) to more ambitious projects (real co- creation approaches). A benchmark that we conducted at the beginning of 2019 showed that many open innovation and co-creation platforms based on blockchain have recently emerged, such as Connecty18, Tribute 19, Ideation20, Crowdholding21, ISH22, Kakushin23, ValYooTrust, ... It is not the purpose of this paper to detail these projects. However, it should be noted that they each have their own specificities in terms of angles (some are more oriented towards research, others towards innovation), fields of activity, organization, functioning, etc. The ValYooTrust project Among projects mentioned above, the ValYooTrust project is probably the most advanced. Built around a blockchain, ValYooTrust was born from the meeting at the end of 2017 of three actors: a living lab, a major academic institution (Institut Mines Télécom24), and an entity of the French Ministry of Defense. As its name may imply, ValYooTrust places trust at the heart of its project, with the objective that everyone can enhance their own innovative contributions. ValYooTrust is distinguished by several advantages: • Its scale due to its international dimension • Its polymorphic side (in addition to being a platform for co-creating innovative projects, ValYooTrust is also a market place for intangible assets and a "phygital" incubator (physical and digital) • Its multi-stakeholder and interdisciplinary side because ValYooTrust brings together diversified communities of innovation stakeholders: startups, SMEs, large groups, public institutions, but also any individual wishing

18 20 21 22 23 24 19


to develop their own intangible assets (researchers, creators, inventors, authors, knowledge producers, etc.). Its multidisciplinary dimension: ValYooTrust is interested in various domains like business innovation but also R&D, education, health... for example, in the field of health ValYooTrust brings together communities of patients on the assumption that they can be experts in their own pathologies and therefore able to indicate avenues for innovation. Its disruptive side thanks to several innovations: a sophisticated system of tokens (ValYooCoins) allowing to vote for the best projects and to value them, new mechanisms of mobilization of communities gathered by ValYooTrust, the use of artificial intelligence which allows to propose an advanced "mix and match" engine to put in contact co-creators Its mixed human/virtual operation: alongside automated mechanisms (based on blockchain, smart data, AI,..) ValYooTrust leaves a place to human interventions. These interventions consist in complementary evaluations of contributions by experts and mentors. The human side is also brought by a JuryGreen™ mechanism, based on assessment by legal experts giving a legal basis to transactions. Another major interest of this JuryGreen™ blockchain is to generate a reduced carbon footprint, reducing the blockchain's energy consumption problem. Its innovative and viable business model: registration on the platform is free for individual entrepreneurs, startups, citizens, so a critical mass of members can be quickly reached; financial resources come from the major players thanks to a triple source of income: the availability of the platform in white label, memberships for registration on the platform, transactionbased and results-based fees. Its universal ambition: ValYooTrust has goals that go far beyond simple business innovation. The project has objectives to promote initiatives in the cultural, artistic, societal, philanthropic and other fields that are likely to attract as many people as possible. Beyond a co-innovation platform, ValYooTrust is a crowd innovation platform.

ValYooTrust will be officially launched in the fall of 2019. The ValYooTrust project delivers a perfect example of added value for all stakeholders thanks to the blockchain: • Living labs and similar organizations engaged in collaborative innovation achieve more results (see paragraph on business value) and thus will gain more credibility and global recognition • It is also easier for them to manage the full process thanks to the traceability of contributions • Participants are more motivated and can get gratification and even monetary rewards • Other stakeholders can be reassured on the living labs they supervise or partner with as thanks to the blockchain they have tools to measure and assess their effectiveness. • On a broader scale, the whole civil society will of course benefit from added creativity and innovation


4 The great future prospects provided by the Blockchain The blockchain will open up new horizons in many areas. We can even talk about paradigmatic revolutions. IP Renewal As individuals are encouraged to become more innovative and creative, it is clear that regulation as we know it today in the field of intellectual property is no longer necessary and will change profoundly. The current patent system, while it has been an engine of innovation and growth in the post-war period, is now reaching its limits: unbearable costs and delays in filing and approval for small structures, the "tragedy of anti-communities" that prevents the use of basic research when it comes from a holding entity, too many patents and annuity research... While it is currently difficult to envisage immediate use of the blockchain in this field (because it concerns a legal framework that can be changed very slowly), its adoption will pave the way for a less costly, faster, automated (proof-ofinvention) patenting process, the arbitration of which will depend on an algorithm and no longer on a judge. On the other hand, it can be argued that tokenization via blockchain has been described as a new form of patent system, which would increase trust among competing firms, stimulate cooperation and eventually, further support open innovation. In that respect a solution comes from Nalebuff and Stiglitz (1983), who suggested that prizes should be used to reward great ideas. According to the Global Intellectual Property Center (2009), prizes are better at proving a concept than bringing concrete, useful technologies into existence. Accordingly, tokenization could be used to assign prizes in a web-based idea competition. Empowerment of individuals A major revolution brought by the blockchain is certainly to empower everyone. The blockchain is thus a formidable lever of inclusiveness as well as a social elevator: everyone can express himself, put forward his ideas and obtain rewards (not only moral but potentially in sound and stumbling cash). You don't need to be well-born, nor to have prestigious diplomas or privileged access to technological tools to be an innovator. This extension of opportunities opens up huge opportunities for an extensive deployment of innovation and thus for a much broader creation of value that can benefit everyone.

Scaling Up The blockchain allows to create virtual communities. These can be very large and extend ubiquitously to the four corners of the world. In addition, the acceleration of the production of intangible assets and knowledge in the world is generating an exponential need for "matching" between supply and demand. The blockchain can accompany and multiply this growth in innovation by taking it to a much larger scale.


The great strength of the blockchain is that it reproduces a natural mechanism and develops organically. In biology, stigmergy is an indirect coordination mechanism between agents. The principle is that the trace left in the environment by the initial action stimulates a subsequent action by the same or a different agent. In this way, successive actions tend to be reinforced and thus lead to the spontaneous emergence of coherent, apparently systematic activity. Thanks to the stigmergical processes it induces, the blockchain makes it possible to massively mobilize collective intelligence, to introduce a spirit of sharing, to mutualize the contributions of community members, to generate a broader cooperation that is more natural, fairer and more motivating. By managing the individual reputation of community members, the blockchain also creates a sound emulation between co-creators. Finally, by combining the principle of cooperation with the co-creation of innovation, it creates a new paradigm, deploying more efficient, productive and ethical innovation. New forms of organization and social impact Beyond allowing this individual empowerment, the spirit of the blockchain is fundamentally based on collective values: those of sharing, cooperation, and the creation of common goods. In the words of French entrepreneur Gilles Babinet, currently the digital champion of France at the European Commission, the blockchain can help to "horizontalize" the world. With the enormous prospects that it brings the blockchain is much more than a technology that will boost the co-creation of innovation. The blockchain paves the way for a new economy that some call crypto-economy, based in particular on peer-to-peer transactions without confiscating the value created through intermediaries. In this approach, the value is fairly distributed with minimum resources being allocated to everyone and the added value being shared between those who created it. But it is clear that beyond these economic challenges, these new schemes will make it possible to bring about new decentralized, more democratic and more ethical social organizations such as DAOs (Decentralized Autonomous Organizations): these are organizations whose governance rules are automated and immutable and transparent in a blockchain. Aragon25 is a good example of a prototype platform based on the DAO principle. The DNA of these new types of decentralized organizations naturally tends to be oriented towards the common good. Organizations like Positive Blockchain 26 are already working in this direction by identifying blockchain initiatives with a real social impact. Required developments But let's face it, as it stands, the blockchain is not free of shortcomings, of a technological, practical and societal nature.

25 26


In particular, the issue of the enormous computing power required, which generates costs, low yields and slow transactions, will have to be resolved. One approach initiated by the ValYooTrust project with its blockchain JuryGreen™ consists in reintroducing a human factor into certain mechanisms of the blockchain, thus reducing its energy consumption. An economic challenge is to find practical applications: the blockchain is now entering a paradoxical phase. It represents a great dream, it makes everything seem possible and its fields of applications are potentially unlimited. But on the other hand, it is now time to sell something more than dreams. Companies expect concrete, down to earth applications, they want to see and use the imagined business cases which can be based on PoCs (Proofs of concepts) that really work and not only on paper. And even, beyond these experiments it is necessary to go further by transforming the pilot tests into a profitable business model. This business model must lead, for the most ambitious projects, to the digital transformation of companies which have seized blockchain’s disruptive opportunities and apply it to their strategic business areas. Technologically, the challenge for the blockchain is to be able to grasp ideas and co-creations and to transcribe them correctly. As much as standardized data from bitcoin transactions are easy to enter in chained blocks, it becomes more complicated when it comes to multidimensional content with strong qualitative and therefore subjective components. There is also the challenge to bring together the innovators best able to collaborate. This is where matching solutions come into play, such as artificial intelligence and smart data. When it comes to developing POCs and mock ups, virtual reality is a solution that can bring great added value to a co-creation platform. Another issue is the security and reliability of the system. Recent events have shown that the blockchain is not free of loopholes. Should we not fear that groups of individuals will set up a coalition to influence or divert the added value of "blockchained" contributions to their benefit? In the end, provided that individuals are not chained to a technology, however promising it may be, and that care is taken to give primacy to human beings and ethics, the paths of the blockchain certainly deserve to be explored by those who believe in the virtues of co-creation. European perspectives We are well aware that these reflections are only the beginning of a vast field of research and experimentation in which it would be essential for ENoLL to take its part in evaluating how the living labs could have a new impetus thanks to the blockchain. Of course, many questions are raised by the introduction of blockchain in a cocreation process: • How to secure the technical feasibility and security of the whole process • If participants are individually recognized and even financially rewarded, won’t living labs lose a sense of collective spirit?


• •

How to train people to use blockchain methods and tools? Won’t it constitute a technical obstacle and thus introduce a barrier to the participation of people without sufficient technical background? How to cope with the footprint issue of the blockchain?

We can see that this field with its enormous stakes is attracting increasing attention from Europe. Several recent or current initiatives can be mentioned: the projects #Blockchain4EU "Blockchain for industrial transformation"27, the prize Blockchains for Social Good28, The European Union Blockchain Observatory and Forum29 or the #DLT4Good "Co-creating a European Ecosystem of Distributed Ledger Technologies (DLTs) for Social and Public Good"30. As stated on the website of the Commission for this latter project "Distributed Ledger Technologies (DLTs), the most well-known being Blockchain, are one of the emerging technologies foreseen to have a deep and broad impact over the next ten years. Looking in particular into public or third sectors and other civil society organizations, DLTs are expected to transform how such sectors and organizations operate or connect with citizens, businesses and other stakeholders". The sustainability of the concept of co-creation via the blockchain is directly derived from the business value, as well as the stakeholder and social impact brought by the blockchain. In addition, it must be noted that governments and administrations in many countries are eager to support blockchain-based projects. This is also notably the case at the European level, as described in the above paragraph. Of course, this public support will benefit initiatives like cocreation and blockchain and strongly contribute to its sustainability.

5 Conclusion We wanted to show in this paper that the blockchain could be a lever for the future of the co- creation of innovation thanks to two major ingredients: the nudge effect and confidence building. Thanks to these, the blockchain makes it possible to overcome the shortcomings of co-creation. In doing so, it opens up exciting opportunities for living labs. It extends their influence far beyond the local to the global and universal. It can help them evolve into more transparent, horizontal and ethical organizations. It allows individuals to be re-invigorated and, by allowing them to truly flourish, it creates the conditions to put the human being, rather than the user, at the heart of innovation processes. And this is for the betterment of both the individual condition and the collective good and progress.

References Dupont L., Mastelic J., Nyffeler N., Latrille S., Seulliet E., (2019, January) Journal of Innovation Economics & Management (pages 53 – 78), Living lab as a 27 29 30 28


support to trust for co-creation of value: application to the consumer energy market Seulliet E. (2016, July 17) Open innovation, co-creation: why blockchain is a small revolution, retrieved from Shavit D., Seulliet E., (2017, December 2) The empowerment of people thanks to the Blockchain in 7 points, retrieved from Duvaut P., Seulliet E., Shavit D. (2018, February 16) Reinventing co-creation thanks to the blockchain, retrieved from Duvaut P., Seulliet E., (2018, November 29) Blockchain, a technology that also protects and promotes your intangible assets, retrieved from Duvaut P., Joly L., Seulliet E., Solani S. (2019, July 16) Libérer la propriété intellectuelle grâce à la blockchain, retrieved from


Cross-cultural Differences in Living Lab Research Nele A.J. De Witte*1, Ingrid Adriaensen*1, Leen Broeckx1, Vicky Van Der Auwera1 and Tom Van Daele*2

*Corresponding authors More University of Applied Sciences, Belgium 2 Expertise Unit Psychology, Thomas More University of Applied Sciences, Belgium 1 LiCalab/Thomas

Category: Full Research Abstract Cross-border collaboration is an important part of living lab research, as circumstances and requirements for services and products can vary greatly depending on the region in which they are introduced. While cross-cultural differences can be of interest for these studies, they can also be confounding factors for data collection and analysis. Dissimilarities in the recruitment and in the participation of end users in different regions could influence the outcomes of international studies with multiple implementation sites across countries. Therefore, the current survey study investigates awareness of such crosscultural differences. The sample consists of 36 living labs from 20 countries. Results show that regional differences are reported in terms of participants’ motivation for participation and the impact of gender, age, professional status, and socio-economic status on participants’ contribution. Additionally, regional differences influence whether a moderator should take the role of a facilitator or a more dominant guide of the process during group sessions. Implementing wellchosen strategies for recruitment, for grouping, and for supporting equal contribution in sessions could improve the quality of international living lab research, while still maintaining sufficient standardisation. Keywords: Cross-border research, Group dynamics, Moderator, Recruitment, Living Labs


1 Introduction Living lab organisations are flourishing across Europe and there is a noticeable rise of these labs in the rest of the world as well (Ballon, Van Hoed, & Schuurman, 2018). Cross-border collaboration is an important part of living lab research, as circumstances and requirements for services and products can vary depending on the region in which they are introduced. Obtaining relevant information on local preferences and customs in respect to a service or design can be of great importance for successful dissemination across borders. However, dissimilarities in the recruitment and in the participation of end users in different regions could influence the outcomes of international studies (e.g., Im, Page, Lin, Tsai, & Cheng, 2004). Therefore, it is important to investigate the influence of such crosscultural differences and adjust study protocols accordingly. There is an increase in organisations that describe themselves as living labs. Governmental and international bodies (such as the European Union) are promoting living lab methodology for innovation (Ballon, Van Hoed, & Schuurman, 2018). While different interpretations exist, living labs can generally be defined as open innovation systems where end users and other stakeholders are involved in the exploration, co-creation and evaluation of solutions in realistic circumstances (Ballon et al., 2018). Living labs can improve the understanding of factors contributing to the success of innovations in different social, environmental and cultural contexts (Mulder & Stappers, 2009). Developers making use of living labs have been shown to benefit of such research in terms of gaining new insights, being able to test product-market fit and achieving positive economic effects (Ballon et al., 2018). There is a growing need for international living lab research since the cultural, professional and legal context of specific regions can have important implications for the products and services of interest. International cooperation between living labs provides benefits not only for the innovations that are being developed. Living labs benefit equally, since this provides them with opportunities to learn from one another, exchange best practices and harmonize ideas (Mulder & Stappers, 2009). While cultural differences are of interest for the further development and implementation of innovation, they could however also influence data collection and be confounding variables for study outcomes. Characteristics that can be associated with culture (e.g. socioeconomic or educational status) could threaten the validity and reliability of cross-cultural studies (Im, Page, Lin, Tsai, & Cheng, 2004). Moreover, methodological approaches that work well in one country or context might not necessarily be appropriate in a different context. Group sessions (e.g., co-creation/co-designing sessions, focus groups) are commonly used in living lab research since they are considered to be a culturally sensitive methodology in which group interaction is key to gaining insight into diverging and converging opinions (Greenwood, Ellmers, & Holley, 2014; Liamputtong, 2011). However, group dynamics could also hinder data collection in some conditions. Not all participants might actively contribute to the discussion, which could be due to the social context and to the characteristics of the peers that are present (Liamputtong, 2011). Individuals with certain personality traits or


social standing can (unintendedly) intimidate group members and hereby influence the outcomes of the session. Guidelines recommend that participants of group sessions are homogenous in personal characteristics and background, but studies investigating the effect of heterogeneity on outcomes in group sessions are sparse (Greenwood et al., 2014). Additionally, it is not always feasible to cluster individuals based on all potentially relevant characteristics, such as gender, age, ethnicity and socioeconomic status (SES), the latter of which is often measured as a combination of education, income and occupation. An important benefit of cross-border living lab collaboration as opposed to other, more traditional, cross-cultural research is that each living lab executes the study in its own region. Experienced moderators and researchers speak the local language and are aware of preferred communication styles. However, the use of standardized protocols for recruitment and study execution might not always be optimal for their specific context. Greenwood et al. (2014) investigated ethnical diversity in focus groups and observed that more heterogeneous groups experienced more difficulty than homogenous groups when discussing ethnic and cultural differences. However, no effects were found on other topics (i.e., being a caregiver). Xie et al. (2012) reported that combining individuals from different age groups (children and older adults) in a co-design study had implications for the preferred methodology (e.g., using sticky notes or making drawings) and the organisation of the sessions (e.g. group size, timing and activities during breaks). Older individuals could also be considered more important figures, especially in non-Western cultures (Halcomb, Gholizadeh, Digiacomo, Phillips, & Davidson, 2007). Therefore, younger individuals might not consider it to be appropriate to voice differing opinions, which is problematic for unbiased data collection. Gender could have a similar effect in some contexts (Halcomb et al., 2007). As different international contexts might require adjusted methodological approaches, such flexibility should be included in international research protocols. However, in order to be able to do so, we first need to be aware of which differences exist and might be of importance. Living labs use a broad repertoire of methodological approaches (e.g., Ballon et al., 2018) and mapping all of the specific approaches or designs is beyond the scope of the current study. The current study aims to investigate which regional differences can influence the participation in living lab research in different regions across the world. The aim is rather to investigate the perceived impact of such differences, both in group contexts (e.g., focus groups or group co-designing sessions) and in individual settings (e.g., interviews, human factor studies), across different countries. The current study intents to map to what extent local contexts are considered when organizing and moderating living lab research and whether the living labs expect potential differences to have an impact on study outcome.

2 Methods 2.1 Recruitment Organisations performing living lab research were invited to complete an online survey, through personally addressed e-mails, social media, and through


networks, including the European Network of Living Labs (ENoLL). Although we strived for equal representation of different regions across the world, living labs are less common and more difficult to reach outside of Europe. Data was collected between December 2018 and April 2019. All participants provided informed consent. 2.2 Survey A survey assessing cross-cultural differences in the participation of end users in living lab research was designed. Firstly, the survey inquired about the recruitment process in the specific region of the participating living lab. Secondly, local differences in study participation depending on gender, age, professional background and SES were assessed. Finally, the participants reported on the use of inclusion strategies and the role of the moderator, and reflected on potential cross-cultural differences. The survey consisted of multiple choice and open-ended questions. Completing the survey took about 35 minutes. 2.3 Analyses Frequency analyses were used to compare the responses of different regions. Additionally, thematic qualitative analyses were used to gain more in-depth insight into cross-cultural differences.

3 Results 3.1 Description of living labs In total, 36 living labs of 20 countries participated in the survey (Table 1). The majority of these living labs are situated in European countries (n=28) and are specialized in the area of Health & Care and/or the Silver Economy. No living labs from Africa and Eastern Europe could be included. All living labs, apart from one Danish living lab, completed the entire survey. One living lab identified as being both French and Spanish. Its inputs are, therefore, included in both the Western and Southern European regions. Due to a small sample size, findings in countries outside of Europe should be interpreted with care. All living labs perform group sessions (e.g., co-creation sessions or focus groups). On average, they organize 26 of such sessions annually (M = 25.5 SD = 44.94), however this ranges from 1 session to about 225 sessions per living lab per year. The living labs perform about 38 individual sessions (e.g., interviews, human factors studies) annually (M = 38.45, SD = 73.77). Again, differences between labs are large with two living labs performing no individual sessions at all and others performing up to 300 sessions per year. Additionally, living labs perform other activities, such as stakeholder meetings (M = 17.09, SD = 16.47 annually), survey studies (M = 7.59, SD = 8.78 annually), and other activities such as innovation labs, community building, business model development, live sessions, and boot camps. There is a large range in the number of participants that are being included on a yearly basis. Most living labs include between 10 and 500 participants (M = 156.76, SD = 126.14, n = 27). Six living labs include more than 500 participants each year, specifically in Colombia (n = 1000 participants), UK (n = 1300 participants and n = 6892 participants), Finland (n =


2000 participants), Belgium (n = 5000 participants), and Canada (n = 15000 participants). Table 1. Participating countries organised according to the United Nations Geoscheme of geographical regions of the world.





Western Europe

Austria Belgium France Germany Switzerland Netherlands Denmark Finland Ireland United Kingdom Italy Spain China Taiwan India Turkey MĂŠxico Canada Colombia Australia

Northern Europe

Southern Europe Asia



Eastern Asia Southern Asia Western Asia Central America Northern America Southern America Australia & New Zealand

Participating living labs 1 3 5a 1 1 4 1 2 1 5 2 3a 1 1 1 1 1 1 1 1

Notes. a One living lab indicated that it was operational in both Spain and France.

3.2 Recruitment of participants Participating living labs were asked to report which communication channels they use for recruitment and which mode of recruitment was most effective in their region (Figure 1). Living labs most commonly recruit through other organisations, such as patient organisations, professional associations, or collaborating partners and developers. It is the most effective mode of recruitment according to 13 living labs, mostly because these other organisations have an existing relationship with - as well as existing knowledge of - the target group. Additionally, recruitment through the work-related or through the personal environment of the living lab (e.g. previous contacts, staff & patients of the hospital) is regularly used and eight living labs find it an effective strategy. This approach has the benefit that an relationship with end users already exists, which contributes to availability and motivation. While a mailing list and panel are common ways of recruitment, they are rarely the preferred means. Few regional differences exist with regard to


the communication channels that are being used or preferred. However, online advertisement (e.g. on fora or social media) is solely practised in Northern Europe (5/9 living labs) and in Western Europe (by 5/14 living labs). Through other organisations Personal environment Newsletter/mailing list of living lab Maintaining an active panel Website of living lab Advertisement online Printed advertisement Recruitment agency Direct & face to face contact Word of mouth Marketing agency Multiple channels (non-specific) 0 5 10 Organisations that use this channel






Organisations that report the channel being most effective in their region

Figure 1. Frequencies of the communication channels used to recruit participants and the communication channels that were reported to be most effective in a region. Living labs were able to report multiple communication channels.

On average, attrition rate is around 15% in the participating living labs (M = 14.88, SD = 10.31). Estimates were somewhat lower in Northern and Central America (8% and 2% respectively) and Australia (5%), as compared to European (16%) or Asian countries (16%). Categories for motivation for participation were defined by thematic analysis. The values presented in Table 2 indicate the number of living labs reporting this motivational principle to be present. Multiple driving principles could be reported per living lab. In general, participation in living lab research seems to be most strongly driven by an intrinsic motivation for personal reward, such as gaining appreciation and knowledge. Other motivating factors are gaining material rewards, contributing to helping others or the society, being interested in the specific product or application, or having social contact. Results suggest that personal reward is predominantly important in Europe and Australia, while external (monetary) reward is more important in Central America and Eastern Asia (Table 2). Turkey (Western Asia) appears to be situated more closely to the European values in the field of motivating factors. Differences between specific countries within one region are limited.


Table 2. Motivation for participation. The five middle columns give an overview of the number of living labs reporting different principles, which motivate participants in their research studies. The final column on the right represents the extent to which an external reward is needed to motivate participants on a scale from 0 (not needed at all) to 10 (participation solely for reward). Region

Reported motivating principles

Persona l reward

Central America Northern Americaa Southern America Eastern Asia Southern Asia Western Asia Australia Northern Europe Southern Europe Western Europe All living labs

External reward


External reward needed Interest in innovation

Social contact


Mean (SD)

8.00 5.00

1 2

1 1 6

1 1



1 1

8.00 (1.41) 5.00

1 6


1 1 3





1.00 3.00 3.67 (1.94) 2.00 (2.00)






4.73 (2.69)






3.83b (2.40)

Note. a The participating living lab (from Canada) did not report specific motivating factors. b While the input of the French-Spanish living lab was included in the results of both southern and Western Europe, it was only counted once in the entire sample.

Living labs report that differences between target populations and study designs relate to whether external rewards are necessary for participation. External rewards appear more important for professionals (with limited time available) or for individuals recruited by recruitment agencies. Participants with an intellectual interest in the topic and patients who can gain from the innovation have a lower need for external motivators. While a Dutch living lab experienced that older people are more intrinsically motivated (which they suggested was possibly because they have more time for participation than their younger counterparts do), a living lab from the United Kingdom (UK) reported that reimbursement of costs was relevant for the elderly. Apart from cross-cultural differences, external factors can also have implications for remunerations. Studies that require more investment from participants (in time or due to experiencing some discomfort) are more likely to need external rewards. Additionally, rewards can be influenced by project budgets and whether the innovation is for public design as opposed to a private enterprise or academic research. 3.3 Group selection


Many living lab activities consist of group sessions (e.g., focus group, co-creation sessions). Group sizes can vary depending on the innovation of interest, target population, and study design. However, living labs indicate that they generally consist of about 10 participants (M = 10.54, SD = 5.68). These groups can contain individuals differing in gender, age, professional status, and SES. Figure 2 provides an overview to what extent living labs indicated that these factors would influence contribution in living lab research. Contribution was defined as actively collaborating in the process, providing input, and formulating ideas in the session. More than half of living labs (n = 20) report that they do not consider gender to be of influence to study contribution, while about a quarter does believe it could have an impact (n = 8). Gender is mainly reported to have an impact in Austria, India, Mexico, and the UK. However, only India and one Italian living lab organize separate sessions for male and female participants. Living labs were asked to estimate the contribution of male and female participants in three conditions: mixed gender group session, separate group sessions per gender, and individual sessions. Although living labs find it difficult to make this assessment, the majority of labs believes that both genders contribute evenly in all three conditions (mixed: n = 19; separate: n = 13; individual: n = 22). The remaining living labs mostly report that female participants contribute somewhat to a lot more actively (mixed: n = 5; separate: n = 7; individual: n = 5). Three living labs (from Austria, Germany, and the UK) report that male participants would contribute more actively in mixed group sessions, but that organizing separate group sessions would results in increased contribution from female participants (n = 2) or equal contribution of both genders (n = 1). While age is not considered to have an impact on contributions to living lab research according to the majority of living labs in France and Australia, most living labs from the other countries do believe that age could be of influence on living lab research. Again, assessing contribution depending on age is considered difficult (do not know: n = 12 to n = 13 in the three conditions). The majority of labs believe that contribution is not influenced by age in all three conditions (mixed: n = 12; separate: n = 18; individual: n = 19). About half of the remaining living labs report that older participants contribute somewhat to a lot more actively (mixed: n = 6; separate: n = 3; individual: n = 3) while the other half beliefs that younger participants contribute somewhat to a lot more actively (mixed: n = 5; separate: n = 3; individual: n= 2). In Turkey (Western Asia), older participants are a lot more active in mixed group sessions, while older and younger participants contribute equally when separated by age in group or individual sessions. Different age groups are consequently always included in separate sessions in this country. Seven other living labs (from Austria, China, India, Italy, Mexico, Spain/France, and the UK) also try to organise separate sessions for different age groups when possible. The majority of European labs include different age groups in 1 session. Some labs are also specifically focussed on research in certain age groups (e.g., elderly) and therefore already have a homogeneous sample in relation to age.


(A) Gender

(B) Age


number of livng labs

Number of living labs

16 12 10 8 6 4 2

16 14 12 10 8 6 4 2 0 Of Might be Not of influence of influence influence

0 Of Might be of Not of influence influence influence




Do not know

The Americas


(C) Professional status Number of living labs

Number of living labs

14 12 10 8 6 4 0



The Americas

16 14 12 10 8 6 4 2 0 Of Might be Not of Do not influence of influence know influence




(D) Socio-economic status


Of Might be of Not of influence influence influence


Do not know

Do not know




The Americas

The Americas

Figure 2. These graphs represent to what extent living labs believe personal characteristics, specifically gender (panel A), age (panel B), professional status (panel C), and socioeconomic status (panel D), would influence active contribution in living lab research.


The vast majority of living labs report that professional status, which refers to whether participants are end users/patients or professionals (e.g., hospital staff), definitely (n = 22) or potentially (n = 5) influences study contribution. Nevertheless, only three living labs (all European) try to organise separate sessions for participants with a different background, while 22 living labs include them in the same session. Overall, most living labs consider professionals and end users to contribute evenly in mixed group sessions (n = 14), separate group sessions (n = 14), and individual sessions (n = 13). However, there were also many living labs that were not sure how professional status influenced the contribution in these different sessions (n = 9, n = 13, and n = 11 of the sample, respectively). Of the living labs that did observe a difference in contribution, it is mostly the case that professionals contribute somewhat to a lot more actively than end users do (n = 10, n = 6, and n = 8 of the sample, respectively). Finally, estimating the influence of SES can be more difficult as compared to gender and age, since it is harder to observe and not always documented or even disclosed. Participants with a high SES have a good income and perform at a high educational/professional level. Participants with high and low SES are equally likely to participate in research according to 14 living labs. The majority of Belgian and Dutch living labs and about half of living labs from the UK believe individuals with high SES are more likely to participate as compared to low SES individuals. On the other hand, the Turkish, Colombian and one other UK living lab indicate that individuals with low SES are more likely to participate. The remaining living labs were not sure about the representation of SES in their samples. Opinions were also divided regarding the influence of SES on study contribution, with most participants reporting that it could potentially have an impact (Figure 2). SES is estimated to have an impact on contribution by seven living labs, representing the majority of living labs from the UK, Taiwan, and Turkey as well as half of the Dutch living labs. Seven other living labs, representing Mexico, Colombia, Finland, Germany, Ireland, Italy, and the UK, do not generally believe SES to have an impact on contribution in the sessions. Participants with different SES are often represented in one group, with only three living labs (in Belgium, China, and the Netherlands) trying to organize separate sessions if possible. Participants with differing SES are often included in one group due to the fact that SES is often not known (beforehand) or not of interest for the study. The majority of participants were not sure how SES influences the contribution in groups with mixed SES (n = 19), separate groups (n = 23) or individual sessions (n = 21). All remaining living labs report equal contribution or more contribution of high SES participants (representing living labs from Belgium, China, Turkey, the Netherlands, and the UK). Living labs were able to report other individual characteristics that could influence participation or contribution to living lab research. Five labs indicated that geographical living situation is of importance. Not living nearby the living lab and/or lacking mobility impedes participation and environmental characteristics (e.g., living in a town vs. rural living) could also influence participation. Secondly, the match between the product and participant (in terms of interest, experience, but also having the feeling that they can have an impact) was deemed important by five living labs. Health can influence participation in two ways. On the one


hand, health problems and disability could lower participation rate (through lack of access or lack of ability to interact with innovations; n = 3). On the other hand, the health situation could increase the relevance and lead to stronger contribution (n = 1). The economic situation could also have an impact (n = 3), both in relation to availability for research (higher workload means lower availability) as well as when it comes to attitude (e.g., more negativity in unemployed participants). Next, ethnicity appears to be important in some regions. In the UK, an underrepresentation was observed of individuals of black and minority ethnicity (n = 2). Having a migratory background is reported to have an impact in Turkey. Relatedly, language (e.g., not being a native speaker) can influence contribution in the sessions (n = 2). Other factors influencing participation in research or contribution to the sessions are personality characteristics such as technological competence (n = 3), extraversion (n = 2), position in a group (in studies with employees of a company; n = 2), previous living lab experiences (n = 1), and early adopters being overrepresented in research (n = 1). 3.4 Use of strategies When living labs experience that they are not reaching certain participant groups in the recruitment process or experience differences in the contribution of individuals in sessions, they can use different strategies to increase equal representation in contribution. In the recruitment process, the majority of living labs from all regions sometimes (n = 22) or always (n = 5) apply strategies to reach populations that will otherwise be less likely to be represented in the sample. The most common way to reach these populations is through intermediary organisations or individuals (n = 9, mostly used in Belgium, the Netherlands and the UK), by contacting individuals the labs have previously worked with or are in their database (n = 3, used in France and Turkey), or by carefully selecting one or more communication channels to reach the targeted sample (n = 4). Other strategies consist of following up on recruitment and adjusting strategies to target underrepresented groups (n = 2), being embedded in a disadvantaged community (n = 1), or optimizing the practical circumstances, i.e., easy-to-reach venue, time outside or rush hour, ability to bring carer (n = 1). During group sessions, almost all living labs sometimes (n = 16) or always (n = 16) use specific strategies to make sure that all participants are equally involved and their opinions are equally heard. Table 3 gives an overview of different strategies. The living labs often combine multiple strategies. The most commonly used strategies consist of directly addressing individuals, using different modes of communication within sessions (e.g., writing individually on sticky notes combined with oral group discussions), splitting up into smaller groups, and having a skilled facilitator to conduct the sessions.


Table 3. Strategies to promote different individuals (or groups) to be equally represented in the output of a session. Strategies were based on thematic analyses.

Number of living labs Strategy applying strategy Directly and personally addressing individual participants 10 Combining different modes of communication: writing, verbal communication, constructing models, etc. 6 Implementation of good facilitation/moderation methods 5 Temporarily splitting the group up into smaller groups or duos 5 Doing a round of the table 4 Increasing participants’ comfort: selection of room, planning breaks, using icebreakers & games, etc. 4 Giving clear instructions that everybody needs to be heard 2 Techniques of service design 2 Appropriate language & communication skills 2 Careful preparation of sessions according to needs 2 Using tools to construct a non-hierarchical environment 1 Promoting trust (by including a psychologist) 1 3.5 Moderator selection & involvement As mentioned above, a moderator (also called facilitator) has a large impact on the course of a session. However, there could be cross-cultural differences in how directive a moderator should be for optimal outcome. There is a large consensus that the role of a moderator should be somewhat flexible, depending on the characteristics of the product, study, and sample. A moderator should generally be able to create an environment that puts participants at ease, but also ensure that all research questions are answered. That being said, there are some local differences. Predominantly in the UK, Netherlands, Germany, Colombia, Switzerland, and Mexico, the moderator is seen as a facilitator and participants should take the lead. The Indian living lab goes even further by calling the moderator a “co-participant�. However, living labs from Taiwan, Italy, Turkey, and Ireland believe the moderator should be a director of the session and guard the process, a role that is somewhat more authoritative. More than half of living labs regularly (n = 15) or sometimes (n = 4) use diverse moderators in different studies or sessions. Moderator selection occurs mainly based on the type of session or research, which might require specific skills or knowledge. Additionally, it could vary based on group characteristics (such as age) or practical reasons (such as availability). The majority of living labs (n = 21) report that the sample does not influence which moderator is being used. Four living labs (from Belgium, China, Italy, and India) try to match the moderator with the sample (e.g. a young moderator with a young sample), while two living labs (from Colombia and the UK) try to use a complementary moderator (e.g., an older moderator with a young sample). Twelve living labs consistently use the same moderator and report the following reasons: practical constraints (e.g. not having


other moderators available), the assumption that a moderator should have sufficient skills to moderate all kinds of sessions, and using the same moderator to build a relationship of trust with the participants. 3.6 Other international differences When living labs were asked whether participants of their own region differed from those in other regions of the world, eight living labs reported differences in culture that were not further specified (Mexico, Colombia, Ireland, the UK, Italy (n = 2), Spain (n = 2), Switzerland, and Spain/France). Additionally, some other specific participant characteristics were reported. Two Dutch living labs reported that their participants are often more direct in their input in the sessions. One of these labs also reported that Dutch people are more likely to adopt new innovations. The two Finnish living labs report greater commitment to research in their country, possibly due to greater equality between citizens. A difference between Australia and the largest part of Europe is that there is more rural living (and the associated problems) in Australia. In Austria, the urban/rural structure is reported to be different from other countries (without further specification). In the UK, greater privacy concerns have been reported (n = 1), as well as differences in the use of technology in age groups (n = 1; not further specified). Apart from these differences between countries in participant characteristics, the location and approach of the specific living lab could influence how individuals behave in sessions. A Belgian living lab reported that their panel approach resulted in the development of a personal bond with participants. A German living lab reported that due to their location in a university city, they had a strong representation of individuals high in SES and educational level. Participating living labs also reported on other cultural differences that were not necessarily important in their own region, but could be relevant in other parts of the world. Firstly, policies and business models could influence the input in living lab research. This specifically concerns differences in the social welfare system and how secure individuals consequently feel, differences in corporate structure (hierarchy), and the (political/historical) openness of society and related openness towards innovation. Other differences could be based on culture and religion, migration or the geographical location of individuals (within countries). Secondly, potentially important cross-cultural differences consist of health, personal and professional beliefs and the existence of trust between participants and researchers. Anecdotal evidence of a Dutch living lab also hints to some underlying cross-cultural differences. They experienced that, as compared to the direct, open and critical stance of participants in the Netherlands, participants in other countries are more polite and hold back harsh criticisms (e.g., in the UK) or appear to be more sensitive to social desirability (e.g., in Taiwan). However, these differences were not reported by the other (respective) countries.

4Conclusion Cross-border research can deliver highly relevant information for developers and allow living labs to exchange best practices and ideas. Although cross-cultural differences could be of interest to developers, it could also be a confounding factor for data analysis. Differing social norms and personal status related to


individual characteristics could imply that the group composition needed for open communication is dissimilar between countries. The extent to which regions are aware of differences in individual characteristics varies greatly and awareness does not always lead to changes in study designs. A first relevant individual characteristic is gender. Several countries are aware of differences in this area, which are mostly reflected in female participants contributing more in the sessions. Secondly, a vast majority of living labs reported age to have an effect on living lab contribution. Although European living labs mostly combine different age groups in one session, constructing separate groups depending on age does appear recommended in multiple countries in- and outside of Europe, especially in Turkey. Professional status and SES can also influence contribution according to the vast majority of participating living labs. Results suggest that professionals might contribute more than patients, but nevertheless participants with a different professional background are mostly included in the same session. The findings suggest that organizing separate group sessions and using specific strategies to support equal representation in study outcomes is warranted. Participating living labs provided several useful strategies to promote participation of individuals with diverse background characteristics. Regularly implemented recruitment strategies consist of using intermediary organisations, using known participants, and combining multiple communication channels. Strategies to promote equal contribution in sessions include directly addressing individuals, combining different communication modalities, and having a skilled moderator. However, the definition of a “skilled moderator� can differ. Participating living labs generally agree that a moderator should be able to create an open and welcoming environment, but also ensure that the procedures are being followed. Nevertheless, some countries report the need for a more facilitating stance (e.g., India, Netherlands) while a more authoritative stance might be required in others (e.g., Taiwan, Italy). The current self-report study shows that awareness of cross-cultural differences differs between regions. Further cross-border research is needed to confirm the effect of such differences. However, the current study indicates that cross-cultural differences should be a topic of discussion in cross-border living lab collaborations. Without the necessary methodological considerations, the results of a study might represent the whole population of a region or might not even reflect the views of all individuals present in a session. Being aware of crosscultural differences and consciously implementing strategies to promote equal representations as much as possible can improve living lab research. Study protocols could suggest good practice strategies for promoting equal representation independently of the included regions. Additionally, when including certain regions or sensitive topics, it might be advisable to separate individuals in different groups based on individual characteristics (while maintaining sufficient standardisation for cross-border comparisons).



Ballon, P., Van Hoed, M., & Schuurman, D. (2018). The effectiveness of involving users in digital innovation: Measuring the impact of living labs. Telematics and Informatics, 35, 1201-1214. Greenwood, N., Ellmers, T., & Holley, J. (2014). The influence of ethnic group composition on focus group discussions. BMC Medical Research Methodology, 14(1), 1–13. Halcomb, E. J., Gholizadeh, L., Digiacomo, M., Phillips, J., & Davidson, P. M. (2007). Literature review : considerations in undertaking focus group research with culturally and linguistically diverse groups. Journal of Clinical Nursing, 16, 1000-1011. Im, E., Page, R., Lin, L., Tsai, H., & Cheng, C. (2004). Rigor in cross-cultural nursing research. International Journal of Nursing Studies, 41, 891– 899 Mulder, I., & Stappers, P. J. (2009). Co-creating in Practice: Results and Challenges. In Proceedings of the 15th International Conference of Concurrent Enterprising: ICE 2009 (pp. 1–8). Leiden, The Netherlands: Centre for Concurrent Enterprise: Nottingham, UK.



Developing a quasi-experimental research design framework using analogue observation to evaluate the performance of a Living Lab output Benjamin Nanchen*1, Emmanuel Fragnière1, Patrick Kuonen1, Joelle Mastelic1, Randolf Ramseyer1 and Henk Verloo1

1 Living

*Corresponding author Lab Handicap, University of Applied Science Western Switzerland, Sierre, Switzerland Category: Research-in-progress

Abstract Since its introduction in the nineties, the concept of Living Lab has evolved from a space where technological innovations is tested directly by users for an innovation eco-system. The creation of the European Network of Living Labs (ENoLL) in 2006 characterizes this evolution. Although Living Lab are well disseminated around the world, there is a lack of consensus on how a Living Lab should be organized (macro level), which types of projects are considered as Living Lab projects (meso-level) and which methods should be used (micro level) (Schuurman, et al., 2015). Furthermore, Living Lab need tools to evaluate the performance of its output (Schuurman, et al., 2015). Therefore, we developed here a quasi-experimental research design framework using analogue observation to evaluate the performance of a Living Lab output. In this paper, we illustrate how to operationalize this research design framework in a case study that aims at introducing Autonomous Vehicles (AVs) in a city in Switzerland. Keywords: Living Lab, Quasi-Experiment, Analogue observation, R&D for Services, co- design, empowerment


1 Introduction The concept of Living Lab was introduced in the nineties at MIT in the US. Originally, a Living Lab was a space where technological innovations used to be tested directly by users (Dvarioniene, et al., 2015). Thanks to these infrastructures, researchers had the opportunities to collect data about the use of their novel technologies as well as users’ perceptions. Since this first experience, the concept of Living Lab has evolved to an innovation eco-system and is now an essential trend in innovation development. This evolution is characterized by the creation of the European Network of Living Labs (ENoLL) in 2006. Today, ENoLL counts more than 150 active Living Labs worldwide 31. As Schuurmann (2015) defined it, three levels characterized a Living Lab: 1) macro level, the ecosystem, 2) meso-level, the project and 3) micro level, the tools. Another characteristic of a Living Lab is the involvement of stakeholders in all categories, as shown in the quadruple helix: users, companies, academics and public authorities 32. Finally, the user involvement during the whole co-design process is the most important characteristic of a Living Lab (Schuurman, et al., 2015). Although the Living Lab concept is well disseminated around the world, there is a lack of consensus on how a Living Lab should be organized (macro level), which types of projects are considered as Living Lab projects (meso-level) and which tools are used (micro level) (Schuurman, et al., 2015). Furthermore, the Living Lab initiative need tools to evaluate/assess the performance of its output (Schuurman, et al., 2015). A Living Lab approach is also described as “an Open Innovation ecosystem” were users are involved in the R&D process (Pallot, et al., 2010). R&D projects are conducted in a laboratory (in vitro). For their part, Living Lab projects are conducted in natural environments (in situ). Therefore, we propose to structure Living Lab projects (meso-level) as a Research and Development (R&D) function. In addition, given that one of the six perspectives typifying a living lab is service creation (Mulder, et al., 2008), we will use the concept of R&D for services. As R&D projects use experiment to support, validate or refute a hypothesis, we propose a quasi-experimental research design framework using analogue observation (micro-level) to assess causal impacts related to an intervention by using empirical testing protocols. This tool enables to evaluate the performance of a Living Lab output. This paper is organized as follows. In Section 2, we present a literature review related to the evolution of Living Lab (macro level), R&D for services (meso level) and evaluation methods (micro level). In Section 3, we describe our methodological development of a quasi- experimental research design framework using analogue observation to evaluate the performance of a Living 31

32 ENoLL Application Guidelines - 13th Wave on



Lab output. In Section 4, we illustrate how to operationalize this quasiexperimental research design framework. The presented case study is based on a project who aims at introducing Autonomous Vehicles (AVs) in a city in Switzerland. As an example, we will test the hypothesis that the use of Autonomous Vehicles (AVs) improve the mobility of people with disabilities. Finally, in section 5, we discuss our proposition and provide directions for further development.

2 Literature review 2.1 The evolution of Living Lab and the need for methodology to assess performance Since the Helsinki Manifesto (2006), the European Living Labs start to structure their innovation efforts and methodologies (Mulder, et al., 2008). Mulder and her colleagues first try to harmonize the methods and tools used in Living Labs. As in other methodologies for innovation like Design Thinking and Service Design, there is not one way to foster innovation (Brown & Katz, 2011; Schneider & Stickdorn, 2011; Fragnière, et al.,2012), but an agreement on several principles like, the end-users participation in each phase of the co-design process, the stakeholder involvement (Mastelic, 2019) and the goal to foster a better society. However, there is a need to find appropriate methods to assess the performance of a Living Lab, especially of its output (Schuurman, et al., 2015). The Living Lab harmonization cube does not propose explicitly methods and tools to assess the performance of a Living Lab output (Mulder, et al., 2008). 2.2 R&D for services – A way to structure and measure innovation In the Oslo Manual, innovation is defined as “a new or improved product or process (or combination thereof) that differs significantly from the unit’s previous products or processes and that has been made available to potential users (product) or brought into use by the unit (process)” (OECD, 2018, p. 20). A key tenet of the Oslo Manual is that innovation can and should be measured. Therefore, it “provides guidelines for collecting and interpreting data on innovation. […]” (OECD, 2018, pp. 19, 20). Pallot, et al. (2010) described the Living Lab approach as “an Open Innovation ecosystem” were users are involved in the R&D process. A Living Lab can be seen as a “Living Laboratory”, at the level of a region, in which users participate in the development of innovative goods and services (co-design). Its main goal is to explore the insight, the salient features valued by a specific population and to co-create value together. It is also a test environment, open and benefiting from technological and methodological tools. It is, therefore, an ecosystem allowing a participatory process, using appropriate tools and methodologies (Liedtke et al., 2012). The Frascati Manual (2015, p. 28) defined R&D as “creative and systematic work undertaken in order to increase the stock of knowledge – including knowledge of humankind, culture and society – and to devise new applications of available knowledge.” The new edition of the Frascati Manual put a greater emphasis on the R&D in the social sciences, humanities and the arts (2015, p. 44). However,


the R&D for services process is not formally organized as in other industries (Miles 2007, Sundbo 1997). Sawatani and Fujigaki (2015, p. 166-168), propose a Service R&D Model based on the Service-Dominant logic and extended Moeller’s model where “service processes are divided into facilities, transformation and usage with three spheres, such as R&D, value co-creation and site”. “The classic process of manufacturing innovation follows three steps in a linear manner: research, R&D and finally manufacturing.” This process cannot be strictly followed for service innovation because service “raw material” is knowledge (explicit and implicit) and customer is also co-producer. In addition, “R&D for production is based on the “make-to-stock” principle. The goods are produced in batches, then stored and finally sold in the markets.” (Fragnière, et al. 2018). This is not possible for services, because services are perishable, instantaneous (Lovelock & Gummesson. 2004) and co-produced, e.g. the learning and practice of music instrument is not possible without the participation of the professor and the student. The iterative Living Lab process – 1. co-creation, 2. exploration, 3. experimentation and 4. evaluation - (Task, et al., 2017) differs from the R&D process only in the co-creation component, “which breaks the linearity of the process and involves every stakeholder taking part in the innovation” (Fragnière, et al. 2018). “Exploration, experimentation and evaluation have the same similar roles and purposes as the identification of needs, prototyping and testing in the classical R&D manufacturing process”. Fragnière, et al. (2018) propose a model that combines the Living Lab process with the classical R&D process (see Figure 1). To structure Living Lab projects (meso-level) as a R&D function could contribute to a better comprehension and acceptance by all stakeholders. Indeed, security, safety, compliance or social acceptance are attributes that this rigorous and systematic (i.e. scientific) process can bring to this innovation ecosystem. Furthermore, this scientific framework facilitates the use of scientific procedures like quasi-experiment. Given that, one of the six perspectives typifying a living lab is service creation (Mulder, et al., 2008), we propose to use the concept of R&D for services.


Figure 1. Integrating Living Lab methods in the more formal R&D process



2.3 Evaluation Methods for Living Lab outputs Impact evaluation relates to the analysis of the causal effect of an intervention. It is linked to the notion of counterfactual analysis. The outcome has to be compared with a sample without intervention (Ballon, et al., 2018) and the developed intervention must create value for its intended users (Ståhlbröst, 2012). In this subsection, we will list some evaluation methods previously used for Living Lab outputs. In the field of Information & Communication Technologies and smart home technologies, many research projects run experiments in Living Lab environments. In these physical spaces (room, building or district) artefacts (devices, technologies or services) are tested by users with or without their direct intervention (Bendavid, et. al, 2012; Budweg, et al., 2012; Buhl, et al, 2017; Flammini, et al., 2018; Perentis, et al., 2017; Schuurman, et al, 2011;). With the emergence of user-centered design in the late 1980, “Usability evaluation with real users became a key part of product development” (Nielsen, 1994). Qualitative and quantitative data collection techniques are used to assess the effectiveness of technologies (Bassoppo-Moyo, T. C.,2010). To assess the users’ acceptance of a Zero Emission Building in the Trondheim Living Lab, Korsnes, et al. (2017) used a quasi-experiment design, qualified by the authors as a variant of a qualitative experiment. The authors collected data through a mix-method approach in which qualitative data is complemented by quantitative data like the measurement of energy consumption or the outside and inside temperature.


3 Methodological Development 3.1 Quasi-experiment An experiment is, according to William Ralph Inge, “a test under controlled conditions that is made to demonstrate a known truth, examine the validity of a hypothesis, or determine the efficacy of something previously untried” (Shadish, et al., 2002). Schuurman et al. (2013) propose a quasi-experimental approach for Living Lab projects. To this end, they include a pre-measurement, an intervention (a reallife experiment) and a post- measurement. For them, evaluation phase can be seen as “the assessment of the impact of the experiment with regards to the current state in order to iterate the future state” (Task, et al., 2017). It enables to generate a “post-measurement of the intervention and compare it to the ‘premeasurement’ benchmark, illustrating the potential impact and added value created by the innovation” (Task, et al., 2017). 3.2 Naturalistic and analogue observation “Naturalistic inquiry differs from conventional science in minimizing constraints on antecedent conditions (controls) and on output (dependent variables). Naturalistic inquiry is phenomenological rather than positivists” (Guba, 1978). When the experimental approach is implausible, it offers an alternative for researchers. In a naturalistic experiment, data is collected under natural conditions, i.e. not in the lab (Guba, 1978). The naturalistic observation relates here mainly to the field of human ethology, especially for the case of experiments (Eibl-Eiblsfeldt, 1989). Actually, the ethological experiment falls into the category of quasi-experiment. It means that we are not able to control all the variables present in the experiment, as it is the case with pure laboratory experiments. Our goal is not to generalize findings but rather to discover new behavioural patterns. There is also no standardized approach. In a Living Lab setting, Fragnière et al. (2017) used naturalistic observation and ethogram to test their hypothesis that queue structuring can have a positive impact on wait time perception. In a Living Lab context, analogue observation could be more appropriated (Norton & Hope, 2001). 3.3 A quasi-experimental research design framework using naturalistic to evaluate the performance of a Living Lab output As the Living Lab output is no more than a set of hypotheses addressing the ways in which users will interact with the service or the object, it can be evaluated or tested through a quasi- experiment with a control group and an experimental group. In a quasi-experiment, participants are not randomly assigned to a case (Shadish et al., 2002), after which the results of the two groups are compared (Campbell & Stanley, 2015). Knemeyer and Naylor (2011) have identified the necessary conditions for quasi-experiments to establish the causality of two tested variables. First, with all other things being equal, solely the independent variable is changed, and second, the independent variable might or not affect the participants, and thus the dependent variable might also change.


We can, however, describe it as a process containing the four following main steps: • Hypothesis formulation. We start by formulating a hypothesis related to a given human behaviour. This hypothesis is the answer to the stated research question. In a Living Lab process, the hypotheses are formulated during the co-creation and the exploration phase. • Hypothesis “operationalization”. We then “operationalize” the hypothesis through a simulation of the ecosystem under study to obtain a prediction of it. This correspond to the experimentation phase of the Living Lab process. • Data collection and analysis. We collect and analyse the data of the experiment (e.g. a simulation of passengers waiting at airport security gates). This is the evaluation phase. • Conclusion. We compare the statistical results to the prediction and also to other findings in the literature to assess the validity of the hypothesis.

4 The operationalization of the quasi-experimental research design framework - A case study This case study illustrates how to operationalize the quasi-experimental research design framework. Based on a project who aims at introducing Autonomous Vehicles (AVs) in a city in Switzerland, we illustrate how quasi-experiment could be designed according to our R&D service approach to be able to test hypothesis related to use of Autonomous Vehicles (AVs) for improving the mobility of people with disabilities and reduced mobility. This case study shows how the quasiexperimental research design framework we develop would structure such a problematic. We focus here solely on the aspect of the framework and not on the statistical tests of the quasi-experiment. 4.1 Context of the case study 4.1.1 The challenge of autonomy of people with disabilities and reduced mobility One of the numerous challenges faced by people with disabilities and reduced mobility is the accessibility of public space. Improving the mobility of people with disabilities give them autonomy and reduce the risk of social isolation (Simplican et al., 2015). The mobility of people with disabilities is much more than a technological challenge. Indeed, regarding our ageing society, it is a necessary and crucial democratic debate and a process of empowerment (Lord & Hutchison, 1993). “In the mobility sector, a large number of new technologies such as autonomous vehicles (AVs) and services are emerging. AVs involve not only passengers, but also authorities, manufacturers, public transportation companies, law enforcement officials, drivers, pedestrians and merchants” (Ramseyer, et al., 2019). For people with disabilities, the introduction of autonomous vehicles (AVs) as public transportation represents a challenge. The objective is to deliver a much better quality of life by identifying and providing opportunities for better social inclusion. Furthermore, it will be necessary to better define the city's general traffic plan and the appropriate layout of pedestrian streets tacking into account the need of people with disabilities while integrating these new means


of transportation. Therefore, it is important to design transportation modes that consider the needs of people with disabilities. For a successful implementation, the service provided by a new technology must be a relevant problem solving that changes the users’ life. As such, new services must be pre-tested and validated by users not only on performance measures but also on perceptions. Therefore, it is essential to consider the factors that will affect the adoption and user acceptance of these novel systems and to integrate as well the felt needs of people with disabilities (McCreadie & Tinker, 2005). 4.1.2 A Swiss Living Lab dedicated to handicap In Switzerland, about 1’300’000 people find themselves with a handicap33. Switzerland as adopted in 2014 the Convention on the Rights of Persons with Disabilities34. This convention is an international agreement that in principle guarantees people with disabilities the equality of experience in all areas of citizenship (Darcy, 2012). Furthermore, the laws of the Swiss Confederation dictate equal opportunities for all. Particularly, the Federal Act of 13 December 2002 on the elimination of discrimination affecting persons with disabilities (Disabled Persons Equality Act - LHand) provides remedies and rights of legal action to make it easier for individuals with disabilities to assert their rights. In 2018, the HES-SO Valais-Wallis and the Innovation Centre for Assistive Technologies (IATLab) founded the Living Lab Handicap (LLH) with the collaboration of ASA Handicap Mental and the Foundation for Research in Favour of People with Disabilities (FRH). This initiative connects people with disabilities, their families and caregivers, with scientists, companies, public authorities and all the other stakeholders (care institutions, charitable associations, etc.) interested in collaborating in the field of disability and the codesign of innovative solutions. “Nothing About Us Without Us”35. This sentence perfectly illustrates the vision of this Swiss Living Lab. Being at the center of the co-design process and the vision of empowerment, participants will elaborate with all the stakeholders detailed scripts in order to co-design adequate and useful products. 4.1.3 Autonomous vehicles on public road in Switzerland In the city of Sion in Switzerland, AVs are in function since 2016 on public road (Eden, 2017). The pilot project of AVs begins in the summer of 2016 in the old city of Sion. In 2019, two routes are in function. The first conducts passenger from the rail station to the old town. The second drives in the old town. The AVs are in function from Wednesday to Sunday from 7 to 10 am and from 1 to 6 pm. The goal of this pilot project is to understand if AVs could offer new services and forms of mobility in regions currently deprived of public transport and to test if the introduction of autonomous shuttles in the public space is technically and operationally feasible, while offering added-value to customers.

33 34 35


4.2 Quasi experiment framework implementation 4.2.1 Hypothesis formulation People suffering from reduced mobility primarily use wheelchairs. According to the World Health Organization (WHO), the prevalence of disability ranges between 12 - 18% of the total population and the majority of people with mobility issues use wheelchairs 36. Our quasi- experiment tests the hypothesis that the use of Autonomous Vehicles (AVs) improves the mobility of people with disabilities. 4.2.2 Hypothesis “operationalization” To operationalize this hypothesis – the use of Autonomous Vehicles (AVs) improves the mobility of people with disabilities – we recruited three users with reduced mobility who used wheelchair. The three users had to use the AV in the Sionold town route. 4.2.3 Data collection and analysis In the AV, only one user in wheelchair can embark at the same time. The experiment was run three times. Each time, two same observers collecting data during the whole experiment accompanied the three different users. The collected data are pictures, movies, observation notes and phenomenological quotes. Then, this data are analysed with the help of software like RQDA and NVIVO. 4.2.4 Conclusion Even if this case study is solely used to illustrate how to operationalize the quasiexperimental research design framework, the collected data are interesting. Indeed, without the help of a third person, it is impossible for a people with disabilities to use the AV. The ramp to embark and disembark has to be installed manually (this feature will be automatized in the next generation of AVs) and is too steep. Furthermore, in many buses stops it was impossible for the user to embark or disembark due to the lack of space between the bus and an obstacle (e.g. building). Consequently, here based on these results, we are not able to confirm the hypothesis that the use of Autonomous Vehicles (AVs) improves the mobility of people with disabilities. However, our goal here was just to show how our R&D for services framework can be operationalized in a handicapped Living Lab to ultimately test the validity and performance of new research hypotheses.

5 Discussion Even if Living Lab is now an essential trend in innovation development, the initiative needs tools to evaluate the performance of its output (Schuurman, et al., 2015). In this paper, we proposed to structure Living Lab projects (mesolevel) as an R&D for services. This rigorous and systematic framework permits the scientific validation of operationalized hypotheses through quasi-experiment protocols. In particular, we developed a quasi-experimental research design

36 WHO. (2017b). World report on disability.


framework using analogue observation (micro-level) for the evaluation of a Living Lab output performance. In the case study, we illustrated how to operationalize the quasi-experimental research design framework. Our quasi-experiment was designed to test hypothesis that the use of Autonomous Vehicles (AVs) improves the mobility of people with disabilities and reduce mobility. Even if this case study is solely used to illustrate how the quasi-experimental research design framework we develop would structure such a problematic, results here could not confirm our starting hypothesis. For a Living Lab manager, this quasi-experimental research design framework using analogue observation do not require much efforts and specific competences (e.g. statistics). However, it is efficient and answers an important question: do the living lab output solve a problem for the users and improve their situation? Furthermore, the observation data collected during the quasiexperiment also represents insights for a new co-creation iteration. Even if these results based on qualitative data are interesting, the use of quantitative data will be complementary. In future work, we will use quantitative data collection techniques to reinforce our quasi-experimental research design framework in order to enhance our results through statistical testing.

References Ballon, P., Van Hoed, M., & Schuurman, D. (2018). The effectiveness of involving users in digital innovation: Measuring the impact of Living Labs. Telematics and Informatics, 35(5), 1201–1214. Bendavid, Y., & Cassivi, L. (2012). A ‘Living Laboratory’environment for exploring innovative RFID-enabled supply chain management models. International Journal of Product Development, 17(1-2), 94-118. Brown, T., & Katz, B. (2011). Change by design. Journal of product innovation management, 28(3), 381-383. Budweg, S., Lewkowicz, M., Müller, C., & Schering, S. (2012). Fostering Social Interaction in AAL: Methodological reflections on the coupling of real household Living Lab and SmartHome approaches. I-Com Zeitschrift Für Interaktive Und Kooperative Medien, 11(3), 30–35. Buhl, J., Hasselkuß, M., Suski, P., & Berg, H. (2017). Automating behavior? : An experimental Living Lab study on the effect of smart home systems and traffic light feedback on heating energy consumption. Retrieved from Campbell, D. T., & Stanley, J. C. (2015). Experimental and quasi-experimental designs for research. Ravenio Books.


Darcy, S., 2012. Disability, Access, and Inclusion in the Event Industry: A Call for Inclusive Event Research. Event Management 16, 259–265. doi:10.3727/152599512X13461660017475 Dvarioniene, J., Gurauskiene, I., Gecevicius, G., Trummer, D. R., Selada, C., Marques, I., & Cosmi, C. (2015). Stakeholders involvement for energy conscious communities: The Energy Labs experience in 10 European communities. Renewable Energy, 75, 512–518. Eden, G., Nanchen, B., Ramseyer, R., & Evéquoz, F. (2017). On the Road with an Autonomous Passenger Shuttle: Integration in Public Spaces. Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, 1569–1576. Eibl-Eiblsfeldt I. (1989). Human ethology. Aldine de Gruyter, NY. Flammini, A., Pasetti, M., Rinaldi, S., Bellagente, P., Ciribini, A. C., Tagliabue, L. C., … Pedrazzi, G. (2018). A Living Lab and Testing Infrastructure for the Development of Innovative Smart Energy Solutions: the eLUX Laboratory of the University of Brescia. 2018 AEIT International Annual Conference, 1–6. Fragnière, E., Nanchen, B., & Sitten, M. (2012). Performing service design experiments using ethnomethodology and theatre-based reenactment: a Swiss ski resort case study. Service Science, 4(2), 89-100. Fragnière, E., Cimmino, F., Emery, L., Héritier, M., Kambly, M., Nanchen, B., & Ramseyer, R. (2017). An ethological experiment to improve airport security gate process reliability: Understanding time perception and personal awareness of older adult travelers. 2017 2nd International Conference on System Reliability and Safety (ICSRS), 68–73. Fragnière, E., Nanchen, B., Héritier, M., Ramseyer, R., Dubosson, M., & Mastelic, J. (2018). Developing a mutualized R&D for network organizations based on Living Lab methods: the transformation of Sion military airport into a commercial airport. Zenodo. Guba, E. G. (1978). Toward a Methodology of Naturalistic Inquiry in Educational Evaluation. CSE Monograph Series in Evaluation, 8. Center for the Study of Evaluation, Graduate School of Education, University of California, Los Angeles, California 90024. Knemeyer, A. M., & Naylor, R. W. (2011). Using behavioral experiments to expand our horizons and deepen our understanding of logistics and supply chain decision making. Journal of Business Logistics, 32, 296-302. Liedtke, C., Welfens, M.J., Rohn, H., Nordmann, J. (2012). Living Lab: UserDriven Innovation for Sustainability. International Journal of Sustainability in Higher Education, 132, 106-118.


Lord, J., & Hutchison, P. (1993). The Process of Empowerment: Implications for Theory and Practice. Canadian Journal of Community Mental Health, 12(1), 5–22. Lovelock, C., & Gummesson, E. (2004). Whither Services Marketing?: In Search of a New Paradigm and Fresh Perspectives. Journal of Service Research, 7(1), 20–41. Mastelic, J. (2019), Stakeholders’ engagement in the co-design of energy conservation interventions: The case of the Energy Living Lab, Doctoral Thesis, University of Lausanne. McCreadie, C., & Tinker, A. (2005). The acceptability of assistive technology to older people. Ageing & Society, 25(1), 91–110. Miles I., (2007). Research and development (R&D) beyond manufacturing: the strange case of services R&D. R&D Manag, 37, 249–268. Mulder, I., Velthausz, D., & Kriens, M. (2008). The Living Labs harmonization cube: Communicating Living Lab’s essentials. The Electronic Journal for Virtual Organizations and Networks, 10, 1-14. Nielsen, J. (1994). Usability engineering. Elsevier. Norton, P. J., & Hope, D. A. (2001). Analogue observational methods in the assessment of social functioning in adults. Psychological Assessment, 13(1), 59–72. OECD. (2015). Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental Development, The Measurement of Scientific, Technological and Innovation Activities, OECD Publishing, Paris. DOI: en OECD. (2018). Oslo Manual 2018. Guidelines for collecting, reporting and using data on innovation. Pallot, M., Trousse, B., Senach, B., & Scapin, D. (2010). Living Lab Research Landscape: From User Centred Design and User Experience towards User Co-creation. Presented at the First European Summer School « Living Labs », Paris. Perentis, C., Vescovi, M., Leonardi, C., Moiso, C., Musolesi, M., Pianesi, F., & Lepri, B. (2017). Anonymous or Not? Understanding the Factors Affecting Personal Mobile Data Disclosure. ACM Trans. Internet Technol., 17(2), 13:1–13:19. Ramseyer, R., Cimmino, F., Emery, L., Grèzes, S., Grèzes, V., Nanchen, B., ... & Fragnière. E. (2019, April). Using Phenomenology to Assess Risk Perception of a New Technology in Public Transportation the Case of the Autonomous Vehicles as Mobility as a Service (MaaS) in Switzerland. In 2018 3rd International Conference on System Reliability and Safety (ICSRS) (pp. 289-293). IEEE. Shadish, W.R. Cook, T. D., & Campbell, D. T. (2002). Experimental and quasiexperimental designs for generalized causal inference. Wadsworth Cengage learning, Boston.


Sawatani Y., Fujigaki Y. (2015). Service R&D Program Design Aiming at Service Innovation, in : Service Systems Science, Translational Systems Sciences, 2, 163-174. Schneider, J., & Stickdorn, M. (2011). This is service design thinking: basics, tools, cases. Wiley. Schuurman, D., De Moor, K., De Marez, L., & Evens, T. (2011). A Living Lab research approach for mobile TV. Telematics and Informatics, 28(4), 271– 282. Schuurman, D., Baccarne, B., Kawsar, F., Seys, C., Veeckman, C., De Marez, L., & Ballon, P. (2013). Living Labs as quasi-experiments: results from the Flemish LeYLab. In XXIV ISPIM Conference: Innovating in Global Markets: Challenges for Sustainable Growth. Schuurman, D., & De Marez, L. (2015). Living Labs: a structured approach for implementing open and user innovation. In 13th Annual Open and User Innovation Conference. Schuurman, D., De Marez, L., & Ballon, P. (2015). Living Labs: a systematic literature review. Open Living Lab Days 2015, Proceedings. Presented at the Open Living Lab Days 2015. Retrieved from Simplican, S.C., Leader, G., Kosciulek, J., Leahy, M., 2015. Defining social inclusion of people with intellectual and developmental disabilities: An ecological model of social networks and community participation. Research in Developmental Disabilities 38, 18–29. doi:10.1016/j.ridd.2014.10.008. Ståhlbröst, A. (2012). A set of key-principles to asses the impact of Living Labs. International Journal of Product Development, 17(1–2), 60–75. Sundbo J. (1997). Management of innovation in services. Serv Indust J, 17(3), 432–455. Task, T., Malmberg, K., Vaittinen, I., Ståhlbröst, A., LTU, A. S., Breuer, J., & Carter, D. (2017). D2. 2: Living Labs Methdology Handbook. Wacker, J. G. (1998). A definition of theory: Research guidelines for different theory building.



Identifying Living Lab orchestrators’ individual‐ level skills Anne Äyväri*1, Tuija Hirvikoski1 and Heidi Uitto1

1 Laurea

* Corresponding author University of Applied Sciences, Finland Category: Research-in-progress

Abstract This paper sheds light on the individual skills needed to orchestrate open Living Labs networks and activities. Since orchestrators (also called mediators) are people working on the interface of the macro, meso and micro levels of Living Labs, and in between various stakeholders such as universities, organizations, NGOs and citizens, specific skill sets are needed in order to enhance inclusiveness, balance, and communication among the different parties and to improve the sustainability of the Living Labs’ projects according to the responsible research and innovation principles. Based on the literature, the skills are classified in three partially overlapping bundles: first, skills in building relationships, networks and ecosystems; second, skills in maintaining them; and finally, skills in executing multistakeholder innovation processes. As a summary of the literature review, a preliminary framework of orchestrator skills is presented. Keywords: Living Lab, multistakeholder co‐creation, orchestrator, skills


1 Introduction Since 2011, the European research and innovation policy has emphasized Responsible Research and Innovation (RRI) (von Schomberg, 2013; von Schomberg, 2019) as a core criterion for public funding for research, development and innovation (RDI) activities. RRI emphasizes the inclusion of all societal stakeholders, including citizens, throughout the innovation process. This perception is coherent with the open Living Labs approach, in which companies, public players, universities, the civic society, and citizens interact and co‐learn by co‐creating knowledge and adding value in order to unleash innovation given the ability to laterally communicate with all the stakeholders. According to the European Network of Living Labs (2019), Living Labs (LLs) are defined as user‐ centered, open innovation ecosystems that are based on a systematic user co‐ creation approach, integrating research, and innovation processes in real life communities and settings. Living Labs have been widely scrutinized as open innovation networks. Consequently, a concept of multistakeholder Living Lab collaboration has evolved over time. However, research has also pointed out challenges and paradoxes that occur while dealing with open innovation networks. According to Klerkx and Aarts (2013), the following three challenges and paradoxes require a network orchestration of the participating actors. First, balancing new relationships and existing relationships i.e. exploiting weak ties and fostering strong ties; second, determining the most appropriate way of interacting with other organizations, and third, balancing informal and formal relationships. We maintain that determining and enhancing the right skill set of the orchestrator is critical in order to solve these grievances pointed out by Klerkx and Aarts (2013). Living Labs are seen as intermediary organizations (Almirall & Wareham, 2011) and the Living Lab methodology emphasizes competences to facilitate co‐ creation processes within those organizations (Dell'Era & Landoni, 2014). Likewise, the demand for skilled co‐creation facilitators, such as T‐shaped innovators, has been widely recognized for their ability to enhance innovation within industries (Barile, Saviano & Simone, 2015; Demirkan & Spohrer, 2015). However, the results of recent systematic reviews on Living Lab literature and studies (Habibipour, 2018; McLoughlin, 2018; Westerlund et al, 2018) imply that extant literature on individual and organizational level competences needed in orchestrating the Living Lab networks and implementing activities is very scarce. Thus, to address the knowledge gap, the aim of this paper is to identify individual‐ level skills needed to orchestrate Living Lab networks and ecosystems and to execute Living Lab projects. In other words, this paper focuses on the individuals’ skills related to open Living Labs activities, where the ability to motivate and include various actors in the network and co‐creation activities are vital. The collaboration among various stakeholders is seen as a key success factor in joint problem solving and the innovation process based on RRI (Gray & Purdy, 2018, p.5; Lusch, Vargo & Tanniru, 2010; Von Schomberg, 2013; Von Schomberg, 2019). We acknowledge the importance of organizational capabilities (see more e.g. Kazadi, Lievens & Mahr, 2016; Schuurman, 2015, p.314; Vontas &


Protogeros, 2009); even though that is not the focus of this paper (see the Research design section below). Due to the knowledge gap in Living Lab literature, we widened our literature search to cover the following themes: innovation ecosystem & orchestration skills, multistakeholder co‐creation skills, networking skills, mediating skills (in the context of a multidisciplinary project), facilitating skills, and T‐shaped innovator skills. Following the three‐layered Living Lab model by Schuurman (2015, p.316), we aimed at identifying individual‐level skills at macro (Living Lab constellation), meso (Living Lab innovation project), and micro level (Living Lab methodology). In the next section, we discuss the skills identified in the above‐mentioned streams of literature. The section concludes with a preliminary framework of the Living Lab orchestrators’ individual‐level skills. Thereafter, we present the research design of the larger study; this paper being the first step in the study. The article concludes with a short summary and discussion on potential contributions of the preliminary framework.

2 Literature review This paper presents the Living Lab orchestrators’ skills in three, partially overlapping bundles (Figure 1): first, skills in building relationships, networks, and ecosystems; second, skills in maintaining them; and finally, skills in executing multistakeholder innovation processes. It is emphasized that the skills presented in one bundle, e.g. maintaining relationships, networks, and ecosystems, are not exclusively needed in the particular task area, but they might also support successful execution of Living Lab projects.

Skills in

Figure 1. Living Lab orchestrators’ skills categorized according to the three task areas

2.1 Skills in building relationships, networks, and eco‐systems


Individuals’ skills in building relationships, networks, and eco‐systems in Living Lab constellations are categorized in two bundles of skills: visioning skills and networking skills. Visioning skills are needed to outline challenges and opportunities for multistakeholder innovation processes; the vision guides decisions on which opportunities should be seized and who should be contacted and selected for partnership (Gray & Purdy, 2018; Jyrämä & Äyväri, 2005; Spekman, Isabella & MacAvoy, 2000; Äyväri & Jyrämä, 2007). In addition, visioning skills are important in fostering a willingness to see opportunities or competitive advantages and interdependences in multistakeholder partnerships (Gray & Purdy, 2018; Ritala, Armila & Blomqvist, 2009). Entrepreneurial skills e.g. sensing and discovering new opportunities, are similar to visioning skills (Ritala, Armila & Blomqvist, 2009). Wishful thinking and open‐minded gifts are horizontal capabilities characterizing T‐shaped innovators. The former refers to “a thought that emerges from a wish and creates a new opportunity… and is capable of triggering new future scenarios” (Barile, Saviano & Simone, 2015, p.1188). An ability to be open‐minded refers to exploring unknown phenomena, seeking new experiences, and looking at the same phenomena through a different light and with different perspectives (Barilo, Saviano & Simone, 2015). Individual level networking skills have been identified in several studies (e.g. Kazadi, Lievens & Mahr, 2016; Ritter & Gemünden, 2003; Äyväri & Jyrämä, 2007) and been acknowledged for consisting of complex routines and individual skills (Kazadi, Lievens & Mahr, 2016). Contact seeking abilities (Äyväri & Jyrämä 2007) as well as an ability to use one’s own contacts (Kazadi, Lievens & Mahr, 2016; Äyväri & Jyrämä, 2007) and the partners’ contacts (Äyväri & Jyrämä 2007) to identify potential new partners have been identified in previous studies. The Living Labs as innovation networks combine dispersed resources, knowledge, and capabilities (Kazaki, Lievens & Mahr, 2016) in public‐private‐ people partnerships (Westerlund & Leminen, 2011). In multistakeholder co‐ creation ecosystems (as Living Labs have been described by Westerlund & Leminen, 2011), all actors are resource‐integrators that collectively co‐create the shared value (Pera, Occhiocupo & Clarke, 2016). Hence, the following skills are needed when building relationships: stakeholder competence mapping skills (Kazadi, Lievens & Mahr, 2016), an ability to identify the needs of one’s own organization and then inform other actors of those needs (Äyväri & Jyrämä, 2007), and skills in diverse recognition and ensuring inclusiveness (Azadegan & Kolfschoten, 2014). Furthermore, an ability to understand certain stakeholders’ interests and aims and take them into consideration when communicating them to other potential collaborators has been identified by Jyrämä and Äyväri (2015). Personal influencing and motivating skills have also been emphasized when inviting others to join multistakeholder innovation processes (Ritala, Armila & Blomqvist, 2009). Another aspect in networking is brokering or bridging (Äyväri, Jyrämä & Hirvikoski, 2018), i.e. supporting and enhancing relationship building among different actors within one’s own network by sharing contact information and introducing actors to each other.


2.2 Skills in maintaining relationships, networks, and eco‐systems Next, we discuss the individual‐level skills of Living Lab orchestrators in maintaining relationships, networks, and eco‐systems. However, we wish to emphasize that the following skills also support relationship building and executing Living Lab projects. Communication and negotiation skills are prerequisites for fruitful interaction and collaboration. Besides information sharing, communication skills (Azadegan & Kolfschoten, 2014; Jyrämä & Äyväri, 2015; McFadzean, 2002; Ritala, Armila & Blomqvist, 2009) are needed in building shared interpretations and in the framing and reframing of meaning making (Gray & Purdy, 2018, p.197‐198; Purdy, Ansari & Gray, 2017; see also Pearce, 2008). Negotiation skills refer to the orchestrator’s capabilities of taking the partners’ interests into consideration (Barile, Saviono & Simone, 2015; Äyväri & Jyrämä, 2007) and to learn from tension between stakeholder interests in order to mediate those differences (Pera, Occhiocupo & Clarke, 2016). In addition, they include balancing skills (Ritala, Armila and Blomqvist, 2009) to maintain an equal distribution of power among different actors (Gray & Purdy, 2018, p.183). In multistakeholder innovation processes, conflicts can seldom be avoided, hence, conflict management skills (Azadegan & Kolfschoten, 2014; Gray & Purdy, 2018, p. 87; IAF, 2019) are vital for maintaining relationships in ecosystems. Social skills (Ritala, Armila & Blomqvist, 2009; Ritter & Gemünden, 2003; Äyväri & Jyrämä, 2007) are linked to communication and negotiation skills. In the context of Living Labs, social skills are especially related to creating an empathic and trusting atmosphere, they include an ability to sense other actors’ feelings, an ability to show empathy, and listening skills (Barile, Saviano & Simone, 2015; Jyrämä & Äyväri, 2015). Social skills refer to social flexibility (Pera, Occhiocupo & Clarke, 2016; Äyväri & Jyrämä, 2007) supporting adaption to diverse backgrounds (Pera, Occhiocupo & Clarke, 2016) and building an appreciation among cross sector partners (Gray & Purdy, 2018, p.87). Social skills help to maintain shared ownership and consensus building in collaboration, thus increasing commitment in partnerships (Gray & Purdy, 2018, p.183). Coordination skills are needed when coordinating the activities of the actors in innovation networks and eco‐systems and when systematizing the routines linked to coordination (Äyväri & Jyrämä, 2007). When orchestrating and coordinating the multistakeholder collaboration, skills in lateral thinking involving re‐ combining and re‐setting (Barile, Saviano & Simone, 2015) might become useful. Creativity and innovation capabilities to create and sustain a participatory environment for the ecosystem (Jyrämä & Äyväri, 2015) and to find original solutions (Pera, Occhiocupo, Clarke, 2016) that are effective, are needed to ensure the actors’ willingness to continue with collaborative actions. When considering the Living‐Lab‐as‐service approach, client‐centric service modification skills might be needed to ensure relationship continuity (cf. Äyväri & Jyrämä, 2007). Finally, we propose that Living Lab orchestrators need to have an ability to manage time (Äyväri & Jyrämä, 2007) in order to reserve enough


time for nurturing the relationships, thus supporting building trust and commitment among network actors. 2.3 Skills in executing living lab projects When executing multistakeholder innovation processes, Living Lab orchestrators need project management skills consisting of e.g. planning skills, human resource management skills, skills related to funding and financial issues (see literature review on project management skills in Hwang & Ng, 2013). Extant studies on project management skills indicate that they cover a wide array of both management and leadership skills, and some of them are context specific. In Living Lab projects, we wish to emphasize the importance of the LL context specific skills (to be discussed next), hence they are not included in the bundle of project management skills. Collaborative activities and co‐creation in group settings are the core of Living Labs. Therefore, facilitating skills are necessary in managing group processes. Azadegan and Kolfschoten (2014) have developed an assessment framework for practicing facilitators. One of the starting points of their study is the list of core facilitator competencies provided by International Association of Facilitators (2019). A competent facilitator plans appropriate group processes, evokes group creativity, facilitates group self‐awareness of the task, and guides the group to consensus and desired outcomes (Azadegan & Kolfschoten 2014, International Association of Facilitators 2019). It has been suggested that in addition to skills in group dynamics, skills in problem‐solving and decision‐making processes are part of the facilitators’ competency (McFadzean 2002). In the Living Lab literature, learning, knowledge‐creation and knowledge transfer have been described as Living Lab activities (e.g. Hakkarainen & Hyysalo, 2016, Voytenko et al., 2016). A Living Lab project manager supports all the other actors’ efforts in learning, knowledge co‐creation and transfer with his or her pedagogical skills to build learning spaces for creating shared vocabulary and shared meanings (Jyrämä & Äyväri, 2015). Furthermore, a skilful orchestrator possesses an ability to share one’s own knowledge, is willing to learn from others (Äyväri & Jyrämä, 2007), and open to new ideas originating from other actors (Pera, Occhiocupo & Clarke, 2016). Learning and reflection skills (Jyrämä & Äyväri, 2015), knowledge‐ seeking capabilities (Barile, Saviano & Simone, 2015), and employee level absorptive capacity (Kazadi, Lievens & Mahr, 2016) refer to competencies in personal growth and development (McFadzean, 2002, see also IAF, 2019). According to Schuurman’s three‐layered model of Living Labs (2015, p.317), different research steps are taken to generate user input and contribution on the micro level. User involvement methods and tools are provided by user innovation research (Schuurman & de Marez, 2015); hence the skills in this area are needed. In addition, research skills are needed in making conclusions based on the context and the opportunities available (Jyrämä & Äyväri, 2015). Besides research skills, a good command of design methods and tools is a valuable asset when executing Living Lab projects with multiple stakeholders, including users.


Evaluation is just one of the tasks for the Living Lab project managers. For example, McCormick (2016) maintains that “evaluation of the actions and impacts of Urban Living Labs is important to feedback the results, and revisit and refine the goals and visions over time. Evaluation underpins the ability of ULLs to facilitate formalized learning amongst the participants”. Thus, learning and evaluation skills are intertwined. Depending on the context of the innovation co‐created, tested, or validated during the execution of a Living Lab project, skills related to business model development and commercialization or skills related to upscaling or mainstreaming (in the context of social innovation) might also be needed. Figure 2 summarizes the skills described above as a preliminary framework of the Living Lab orchestrators’ individual‐level skills.

Building relationships, networks, and ecosystems (macro and meso level) Visioning skills Networking skills

Maintaining relationships, networks and ecosystems (macro and meso level) Communication skills Negotiation skills Social skills Coordination skills Innovation and lateral thinking skills Client-centric service modification skills Ability to manage time

Executing Living Lab projects (meso and micro level)

Project management skills Facilitation skills Learning and knowledge co-creation and transfer skills Research skills Design skills (tools, methods) Evaluation skills Commercialization and upscaling skills

Figure 2. A preliminary framework of the Living Lab orchestrators’ individual‐level skills

3 Research design The literature review presented in this paper is the first step in an upcoming larger study on individual‐level skills and organizational capabilities needed in orchestrating Living Lab networks and ecosystems. The study is part of a project called Co‐creation Orchestration (CCO), an evidence‐based governance model enhancing the ecosystem’s value in co‐creation, knowledge transfer, and business development funded by the Finnish Ministry of Education and Culture and Laurea University of Applied Sciences. The project will run until the end of 2021.


We will proceed to gather insights on individual‐level skills through thematic interviews and interactive workshops with experienced Living Lab orchestrators or mediators. We will use abductive reasoning (Shank, 2002, p.119) to refine the preliminary framework (Figure 2). After finalizing the qualitative phase of the study, we will conduct a survey aiming for results that are more generalizable. A similar kind of multi‐staged research process will be conducted in order to identify the organizational capabilities needed to orchestrate multistakeholder innovation ecosystems.

4 Discussion Addressing the challenges that have emerged from the existing Living Labs experiences, this paper has classified and described skills that are vital for orchestrating Living Lab networks and activities, with a focus on the orchestrator’s role. The paper takes into consideration the sustainability of the ecosystems and also addresses the skills needed in navigating the networks in the long run. The individual‐level skills are classified into three partially overlapping bundles: first, skills in building relationships, networks and ecosystems; second, skills in maintaining them; and finally, skills in executing multistakeholder innovation processes. The findings of this paper are beneficial for RDI and HR strategy building within all types of organizations and multistakeholder projects.

Acknowledgements The authors are thankful for the financial support from The Ministry of Education and Culture (Finland) and Laurea University of Applied Sciences in conducting the study.


References Almirall, E. & Wareham, J. (2011). Living labs: arbiters of mid‐ and ground‐level innovation. Technology Analyses & Strategy Management, 23(1), 87-102. Azadegan, A. and Kolfschoten, G. (2014). An assessment framework for practicing fasilitator. Group Decision and Negotiation, 23(5), 1013-1045. Barile, S., Saviano, M. &Simone, C. (2015). Service economy, knowledge, and the need for T‐shaped innovators. World Wide Web, 18(4), 1177-1197. Dell'Era, C. & Landoni, P. (2014). Living Lab: A Methodology between User‐ Centred Design and Participatory Design. Creativity and Innovation Management, 23(2), 137‐154. Demirkan, H. & Spohrer, J. (2015). T‐shaped innovators: Identifying the right talent to support service innovation. Research‐Technology Management, 58(5), 12 15. European Network of Living‐us/







Habibipour, A. (2018). Living Lab research: A state‐of‐the‐art review and steps towards a research agenda. In Research and Innovation Conference Proceedings 2018. Open Living Lab Days. Brussels: European Network of Living Labs. 432‒447. Hakkarainen, L. & Hyysalo, S. (2016). The Evolution of Intermediary Activities: Broadening the Concept of Facilitation in Living Labs. Technology Innovation Management Review, 6(1), 45-58. Hwang, B‐G. & Ng, W. J. (2013). Project management knowledge and skills for green construction: Overcoming challenges. International Journal of Project Management, 31(2), 272-284. IAF,

International Association of Facilitators. (2015). Core Facilitator Competences. https://www.iaf‐ (Retrieved 23.4.2019).

Jyrämä, A. & Äyväri, A. (2015). Art encountering society; identifying the skills. The paper was presented at AIMAC 2015, Aix‐en‐Provence & Marseille, France, June 28th – July 1st 2015, organized by The International Association of Arts and Cultural Management, Aix‐Marseille University and Kedge Business School. Kazadi, K., Lievens, A. & Mahr, D. (2016). Stakeholder co‐creation during the innovation process: Identifying capabilities for knowledge creation among multiple stakeholders. Journal of Business Research, 69(2), 525-540. Klerkx, L. & Aarts, N. (2013). The interaction of multiple champions in orchestrating innovation networks: Conflicts and complementarities. Technovation, 33(6), 193-210.


Lusch, R. F., Vargo, S. L., & Tanniru, M. (2010). Service, value networks and learning. Journal of the Academy of Marketing Science, 38(1), 19–31. McCormick, K. (2016). GUST Policy Brief: Urban Living Labs. GUST Project. McFadzean, E.(2002). Developing and supporting creative problem solving teams: part 2 facilitator competences. Management Decision, 40(6), 537 551. McLoughlin, S., Maccani, G., Prendergast, D. & Donnellan, B. (2018). Living labs: A bibliometric analyses. Paper presented at the 51st Hawaii International Conference on System Sciences (HICSS), January 2 6, 2018, Hawaii, USA. Pera, R., Occhiocupo, N. & Clarke, J. (2016). Motives and resources for value co‐ creation in a multi‐ stakeholder ecosystem: A managerial perspective. Journal of Business Research, 69(10), 4033-4041. Pearce, W. B. (2008). Toward a new repertoire of communication skills for leaders and managers. The Quality Management Forum, 34(4), 4-7. Purdy, J., Ansari, S., & Gray, B. (2017). Are Logics Enough? Framing as an Alternative Tool for Understanding Institutional Meaning Making. Journal of Management Inquiry. Purdy, J., & Gray, B. (2018). Collaborating for Our Future. Multistakeholder Partnership for Solving Complex Problems. New York: Oxford University Press. Ritala, P., Armila, L. & Blomqvist, K. (2009). Innovation orchestration capability defining the organizational and individual level determinants. International Journal of Innovation Management, 13(4), 569 591. Ritter, T. & Gemünden, H. G. (2003). Network Competence: Its impact on innovation success and its antecedents. Journal of Business Research, 56 (9), 747 755. Schuurman, D. (2015). Bridging the gap between Open and User Innovation? Exploring the value of Living Labs as a means to structure user contribution and manage distributed innovation. Doctoral dissertation. Ghent University: Ghent, Belgium. Schuurman, D. & De Marez, L. (2015). Living labs: a structured approach for implementing open and user innovation. 13th Annual Open and User Innovation Conference, Proceedings. Presented at the 13th Annual Open and User Innovation Conference.‐6888241 Shank, G. D. (2002). Qualitative Research. A Personal Skills Approach. Upper Saddler River, New Jersey, USA: Pearson Education Inc. Spekman, R. E., Isabella, L. A. and MacAvoy, T. C. (2000). Alliance competence: maximizing the value of your partnership. Upper Saddle River, New Jersey, USA: Pearson Education Inc.


Strauss, D.A. (1999). Designing a consensus building process using a graphic road map. In L. Susskind, S. McKearnan, and J. Thomas‐Larmer (eds.). Consensus Building Handbook, 137 168. Thousand Oaks, CA: Sage. Thorton, P., H., Ocasio, W., & Lounsbury, M. (2012). The Institutional Logics Perspective: A New Approach to Culture, Stucture and Process. New York: Oxford University Press. von Schomberg, R. (2013). A vision of responsible innovation. In Owen, R., Heintz, M. & Bessant, J. (eds.). Responsible innovation. London, UK: John Wiley. von Schomberg, R. (2019). Why Responsible Innovation. In von Schomberg, R & J. Hankins (eds.). The International Handbook on Responsible Innovation. A Global Resource. Cheltenham: Edward Elgar Publishing. Forthcoming. Vontas, A. & Protogenos, N. (2009). Evaluating Living Labs core competences and assets. Paper presented in 3rd IEEE International Conference on Digital Ecosystems and Technologies. Voytenko, Y., McCormick, K., Evans, J. & Schwila, G. (2016). Urban living labs for sustainability and low carbon cities in Europe: Towards a research agenda. Journal of Cleaner Production, 123 (1), 45-54. Westerlund, M. & Leminen, S. (2011). Managing the challenges of becoming an open innovation company: Experiences from living labs. Technology Innovation Management Review, 1(1), 19-25. Westerlund, M., Leminen, S. & Rajahonka, M. (2018). A topic modelling analysis of living labs research. Technology Innovation Management





Äyväri, A. & Jyrämä, A. (2007). The networking abilities of craft entrepreneurs. In P. Naudé, J. Zolkiewski and S. Henneberg (eds). 23rd IMP Conference. Exploiting the b2b Knowledge Network: New Perspectives and Core Concepts. Manchester Business School, 30.8−1.9.2007. Manchester Business School, Manchester, UK. Äyväri, A., Jyrämä, A. & Hirvikoski, T. (2018). The Circle of Mediators: Towards a governance model for tackling sustainability challenges in a city. In Research and Innovation Conference Proceedings 2018. Open Living Lab Days. Brussels: European Network of Living Labs. 287‒305.



Improving Quality in Higher Education by using Living Lab Methods

Karin Axelsson*1, Yvonne Eriksson1 and Anders Berglund1

1 School

*Corresponding author of Innovation, Design and Engineering MDH Living Lab@ IPR Mälardalen University, Eskilstuna, Sweden. Category: Full Research

Abstract This conceptual paper presents a Living Lab model of how university, society and organisation may strengthen the co-creation capacity on regional and international level to improve quality in higher education. Our conclusions are that successful co-creation between universities and society/organizations is built on mutual contribution, knowledge sharing as well as engagement from all involved. Here, living lab methods can help improve quality. Further, from a university perspective, a challenge is to keep the engagement in the project from all involved, and to assure an equal status between stakeholders. This calls for a skill in how to conduct project in co-creation with several partners. A skill that has to be taught to students as well as to teachers and researchers. Keywords: Lifelong learning, Co-creation, Living Lab, Higher education, User involvement


1 Introduction Digitalization is expected to affect all parts of society including the role of universities; it is not only young people that are expected in higher education. The requirements in relation to the changes in organizations (industry, services, public sector) leads to a need of higher education for people already establish on labour market. The universities need to adapt to the requirement of lifelong learning. One such platform constitutes of Living Labs that has risen from the urgency not only to collaborate for short time cycle improvements but also to sustain longterm societal impact. This paper presents a Living Lab model of how university, society and organisation may strengthen the co-creation capacity on regional and international level. The paper is foremost of conceptual nature, using theories to describe how co-creation efforts can improve quality. However, it further builds on many years of practical co-creation experience from workshops, activities, projects and research between the Mälardalen University research environment IPR (Innovation and Product Realisation), which hosts the Living Lab environment, and external stakeholders. 1.1 Background The pressure on companies to enhance their growth through innovation and digitalization is extremely high. One key element of today’s intensified rivalry between companies is found in the competencies and skills utilization that exists in each employee. The access to new knowledge is no longer important it is fundamental. The admittance to high-qualitative knowledge environments and different forms of collaboration have become more important than ever before. In this context, Higher Education Institutions (HEIs) have been very successful in different triple-helix collaboration platforms that has proven fruitful over time. One such platform is constituted by Living Labs that has risen from the urgency not only to collaborate for short time cycle improvements but to sustain long-term societal impact. In Sweden, this collaborative, or co-creative, role for HEIs is enshrined in the Higher Education Act (Högskolelag, 1992:1434), and after an additional amendment regulation from 2009 the wording now being both to collaborate with its surrounding society, inform of the HEIs operations and promote that research results developed within the organisation become useful, for instance in solving societal challenges. On a national level, the recognition of collaboration is further strengthened by a new Governmental initiative rewarding HEIs competencies and results from co-creation. Even if public outreach can take various forms in practice, most Swedish HEIs, express the goal as to co-create knowledge and economic and social growth in partnership. Co-creation is performed with surrounding industry, public authorities and non-profit organisations. At Mälardalen University being active in collaboration lies in the overall strategy and vision of the University, further claiming a market position in the Swedish academic landscape based on its cocreation excellence. Here the University’s Living Lab plays a vital role to both keep the position and continuously progress as regard to co-creation. Further, in relation to the University’s students, there is an explicit goal that all students examined from MDH should have both practical and theoretical knowledge of collaboration and co-production. This since working in co-creating contexts can help students develop and exercise important soft skills, besides the facts and


knowledge received in education. Hereby preparing the students for expectations and requirements from employers in their future working life. The work of the Lab included this study - the MDH Living Lab@IPR - position itself as a milieu for co-creation, where the focus is on developing innovative and collaboration methods and models. Thus, in relation to Nesti’s (2017) research, the MDH Living Lab@IPR is both a tool for or co-creation and co-producing knowledge and innovation between academy, business and public administrations, at the same time is co-creating tools, methods and models for improving co-creation as such. However, as pointed out by Hughes (2014), creating an effective and collaborative research culture is highly challenging. Even though the collaborative assignment has been both stimulated and imposed on Universities all over Europe, and work through for instance Triple Helix constellations have enhanced, there is still room for improvement. Amongst other things, developing better working procedures and ways of understanding each other in order to evolve more effective and innovative co-creative processes. Thus, Hughes (2014) suggests the different parties need to become better on addressing issues related to cooperation during the process and reach a better consensus, which in the end leads to a better result. Furthermore, according to Nesti (2017) and Fuglsang and Vorre Hansen (2018), there are some apparent challenges and issues concerning co-creation, posing a gap within the Living Lab literature. They propose it is unclear how co-creation and user participation is really organized. Further, users seem not to have strong decision power.

2 Literature Review There are many definitions of Livings labs flourishing. Bergvall-Kåreborn et al. (2009, p.1) propose that ‘a Living Lab is a user-centric innovation milieu built on every-day practice and research, with an approach that facilitates user influence in open and distributed innovation processes engaging all relevant partners in real-life contexts, aiming to create sustainable values. Ballon et al. (2005 p.3) rather suggest ‘an experiment environment in which technology is given shape in real life contexts and in which (end) users are considered “co- producers”’. Fuglsang and Vorre Hansen (2018) side with Nesti (2017) suggesting viewing Living Labs in a broader context of co-production and co-creation, in relation to this, Hagy et al. (2016) describe Living Labs as places for open innovation where co-creation is a theoretical methodology, which is transdisciplinary, generating knowledge by addressing real-life issues. Living Labs are according to Almirall et al. (2012) motivated by primarily two ideas; to involve users as coequal co-creators and to experiment in real-world settings. The user involvement can occur in different ways. Some well-known approaches are provided by von Hippel (1986) positioning the user as the innovator, or main creator, directly involved in the product creation process, or by Chesbrough’s (2012) thoughts on open innovation where firms can and should use external ideas alongside the internal to deliver better innovations. Howe (2009), approach the end-user through crowdsourcing allowing the power of the


crowd to contribute to development, and Brown (2008) position the users as participants and co- creators in a design thinking process. In many Living Lab contexts users’ needs, knowledge and experience provide external ideas as a resource for innovation (Furestein et al., 2008; Fuglsang & Vorre Hansen, 2018; Ståhlbröst, 2008), with which different stakeholders such as researchers, companies, users and public organisations, develop, test or evaluate new products and services, working together in an innovation process in real-life settings (Bergvall-Kåreborn et al. ,2009). Thus, many different organisations and individuals can participate in Living Lab activities. Leminen et al. (2017) claim typical stakeholders are institutions, organisations, research units at universities as well as suppliers, customer and users. As pointed out by Hughes (2014), co- creation is generally perceived as a process where two or more partners collaborate to create value for themselves and/or others. Thus, the assumption is that these parties, or others, will benefit from the collaboration. However, to jointly decide what benefits to focus on and, if conflicting, reach a consensus, is not necessarily an easy task. In relation to this, Fuglsang and Vorre Hansen (2018) claim there is a lack of clarification concerning the Living Labs role and for whom they are to create value. This since, as shown by Nesti (2017), even if the Labs often are financed through public funds these organisations are not necessary the primary recipient of the value. Fuglsang and Vorre Hansen (2018) mean it is not clear if the labs are expected to create knowledge and benefit for the public good or profit for the participating actors. So, what prerequisites are important for cooperation? Huxham and Vangen (2005) propose explicit leadership. Andersson et al. (2011) agree with this adding communication, commitment as well as rules and regulations. Inherently, there are risks involved in cooperative activities such as not knowing how the other parts will act or respond. Therefore, Tillmar and Lindkvist (2007) suggest also trust between the partners is necessary. The organisation needs to have cooperative skills on all levels in an organisation, such as at the strategic, top-management and operative level (Sullivan & Skelcher, 2002; Barnes & Sullivan, 2002). An organisation’s operative level is formed around structures and processes that bring about collaborative activities in practice. Since co-creation and collaboration are dominated by social activities between people, it is becoming increasingly important to strengthen cocreating skills among co-creating participants independent of level within the organisation (Berglund & Bernhard, 2015). 2.1 Higher education – learnings from living lab methods Research methods used in living lab contexts require a mutual understanding among the involved in the project regarding e.g. how to conduct the project, and what role the participants have. Thus, this is important to follow up during the entire project (Dell’Era & Landiano, 2014). Living lab methods derive from the life lab theories that not only emphasises user perspective and involvement, but also includes situated action. Users are affected by the context and the situation and situated action is defined as the interaction


between the situation and socio-cultural aspects. These are dynamic and could change over time (Vera & Simon, 1993). Like situated action, phenomenology takes the experience of situations into consideration. According to phenomenology the context emerges in interaction between individuals, groups, organization and the situation. The context is therefore not something that surround a user, it is occur in interaction between individuals, his/her socio-cultural background, position and the situation. There are three levels of situated action. Firstly, the sociocultural level that could be described as a result of the interaction between structures or cultural norms that are expected from individuals and groups. Secondly, the situation that emerge in a context. How this context is understood/interpreted is dependent on the individuals’ aims and goals. This is also dependent on the sociocultural level. Thirdly, in order to act the individuals, use accessible tools and artefacts. The third level’s prototyping concerns in particular an intersection in expressions of internal and external thinking (Berglund & Leifer, 2013). These levels and actions are also affected by earlier situations and experience as well as the sociocultural level (Vera & Simon, 1993). The focus on real-life situation challenges, inclusion of different stakeholders, cooperation and co-creation and a multidisciplinary approach are important features of the Living Lab. In addition, the notions and goals of the Living Lab are similar and applicable to the entrepreneurial learning format (Axelsson et al., 2018; Jones & Iredale, 2010; Leffler, 2014). Therefore, the Living Lab milieu can provide a fruitful environment for students practicing and developing entrepreneurial, or more suitable in the Living Lab context, co-creation skills. In so doing, the Living Lab activities can improve the quality of education and help students be better prepared for their future working life and professionals their life-long learning.

3 The case of Premium - an example of adaption for lifelong learning This conceptual paper builds on results from many years of co-creation activities and research in relation to the MDH Living Lab@IPR. This section describes experience from a competence development program for professionals, planned and executed in co-creation within this Living Lab context. Sweden’s prosperity is built on innovative and successful export companies repeatedly managing to renew and reorganize production and products to keep pace with changing markets. The need to strengthen companies' competitiveness concerns transforming knowledge intensive activities, and how to utilize the opportunities once more connected and automated, internally and externally. The importance of knowledge intensive product realization, regionally, internationally is confirmed by OECD which has identified a need for increased knowledge content in the products that are manufactured, as well as in the processes used to produce them.


Co-creation, or in this context more suitable term ‘co-production’, also incorporate that regional industrial partners has granted a significant multimillion co-creation project named PREMIUM. Funded by the Swedish Knowledge Foundation (KKS), PREMIUM is the sustainable competence development program that put cutting edge technologies and learning in the forefront for the impact follows the Industry 4.0 expansion. It is a Manufacturing Innovation education program that target professionals to specifically co- produced courses in-between industrial stakeholders and the local university. Led by Mälardalen University (MDH) the project also involves Jönköping University (JU) as academic co-producing partner together with two strong industrial clusters. It is the interaction and knowledge sharing between these nodes that has initiated the spark that now has begun to glow. Based on the urgency and needs uncovered, co-production to transform modern manufacturing stand to set off true magic in the time to come. The purpose is equally straightforward as it is complex and far from easy to solve: To strengthen Sweden’s competitiveness and increase industry 4.0 production transformation and efficiencies. The use of shorter course packages increases flexibility and modularity for external stakeholders such as professionals, which help to benefit from education while working in parallel. Another approach on the same issue is that professionals need to be able to participate in more comprehensive courses/course packages in order to really focus on skills development. This development focuses on how MDH's education offering is structured and the balance between programs, independent courses and possible completely new forms of education. Therefore, the PREMIUM project will play an important role in MDH's development to position itself as an institution that supports specialist’s knowledge while supporting lifelong learning through a new target group mostly overlooked in previous initiatives. The stakes are high and so is the extensive offer that in total presents 21 completely new courses provided in with flexible learning format to reassure best utilization of different digital tools. The changing labour market increases the need for skills development in working life. The shifting need from industry also places new demands on the flexibility of education and results in a more diversified student group involving both traditional program students and working students. The basis for establishing a need-based approach to the exploration of new forms of education, to a large extent independent of time and space, put pressure on new flexible course designs. Important success factors in the co-creation phases of the PREMIUM program is for instance to start the co-creation in an early phase, to get all perspectives, needs and demands on the table. A clear value for each stakeholder is discussed and visualised. This is an ongoing process, with multiple meetings providing a foundation for the future cooperation. Further, driven by the focus on impact following the Industry 4.0 expansion, the design is important to both fit the acquirements of the academy as well as the practical realities for the industrial partners. During the implementation and execution of the programme mutual trust, openness to innovation, continuous change and an experimental approach has been vital. In addition, a discussion on how to, and when, to disseminate knowledge and potential results from the co-creation is important, since the


different stakeholders have different organizations, law, regulations and steering to consider.

4 A Living Lab co-creation model for Higher Education Based on the result of many years of co-production between university and external, MDH Living Lab@IPR see the benefits of starting form a holistic perspective. Challenges are defined jointly among stakeholders, and then these are explored, exploited and evaluated together in an iterative process. It consists of a stakeholder-triangle including the university, external organizations and the society. For MDH Living Lab@IPR users are mostly employees in companies. In order to improve quality in higher education by a strengthened co-creation through using Living Lab methods there is a need for developing, testing and evaluating new models and methods for co-creating knowledge which are developed in a real-life context. In an attempt to do so, building on research and insights and experiences from previous co- creation projects and activities at the MDH Living Lab @IPR, especially the above described PREMIUM programme, a tentative Living Lab cocreation model for higher education is proposed. This consists of a proposed cyclic approach of the co-creation process, as well as an additional 5-phase analysis approach. To briefly explain the different parts in the model: Exploration is an early phase need-finding approach of involving a cascade of multiple perspectives and stakeholders. Conceive is the process of defining purpose and needs. Design is the creation and implementation plan of what is to be. Implement is the realisation of plans, including manufacturing, testing and validation. Operate is about delivering value over time on the basis of current status. Disseminate is the cocreation efforts to increase level of influence and involvement, new partnerships and the communication of visions, goals, activities and foremost achievements. The inner circle is modified after the Deming cycle (e.g. Moen & Norman, 2006; Langley et al, 2009) and from our view applied from the operational level in contrast to the overarching strategic level that is concerns explore yet from another level. For operations plan concerns activities, do means conducting what has been planned. Improve is the development that is done through iterations that leads to refinement and learning. This quality improvement model has been successfully adopted in other university and industry co-creating initiatives in recent time (Borys et al, 2012).


Figure 1. Proposed cyclic approach of the co-creation process by MDH Living Lab@IPR.

In addition, in order to achieve a deeper learning from and future improvement of the work, in relation to the model we suggest a five-phase analysis model. It is not to be perceived as sequential, depending on such as the co-creation aim, goal and outline some phases and questions are more relevant than others. Addressing the different questions, help revealing initial values and assuring the co-creation aspects are considered. 4.1 5-phases for analysis approach Context for the project planning: What is the design of the co-creation project? Which partners are involved? Who initiated the project? How was the planning of the project conducted? From what criteria was the research method(s) selected (including the analyses of current state, for collection of data, for the design/innovation process, and for analyses of data/process)? How are the cocreation stakeholders involved in the planning phase, ensuring e.g. equal engagement, decision power, necessary resources and goal setting? Analysis of the process: What methods have been used? How have the methods been used? What is the impact of the choice of methods on the result(s)? Had it been possible to use other methods? What new knowledge regarding methodology is gained from the project? Form an academic perspective and from the perspective of the external organization. Follow up phase: What parts of the project is based on previous experience from the external organization and/or previous research? What new academic and practical knowledge is gained from the project?


Impact: What is the impact of the project on the organizations involved and the society - initially, during the project, and when it was completed? Has the project generated new policy or new working routines? What is the academic impact, new knowledge, research and/or improved co-creation methods? How do the students perceive their involvement regarding co-creation and exercising/developing soft skills? Continuous meta-analysis: To what extent has the project created impact on the research group/students/teachers and external partners during the project? On the individual level? On the organizational level?

5 Discussion In this conceptual paper, the aim is to strengthen how university, society and organisation can enhance the co-creation capacity on regional and international level, focusing on improving quality by co-creation in higher education using Living Lab methods. To meet the requirement from the society i.e. graduated students that are prepared to meet future challenges in society, who have the capability to work in agile processes, and handle flexibility, MDH Living Lab@IPR has developed courses and parts of courses that contain co- creation with external partners. The aim is to let the student face real problems and to learn to collaborate with various stakeholder. In addition, we have experienced the benefit of integrating people with working experience in courses, by mixing young undergraduate students with those who are on supplementary training creates value in the courses both for teachers and for the young students. The presented model (see Figure 1) support the development in the early phases of courses, however in order to secure the involvement of all partners in a project or course we suggest a combination of the model and the five phases analysis. Based on the perception of Living Labs in a broader perspective as co-creation or co- producing environments, as do Nesti (2017), and Fuglesang, and Vorre Hansen (2018) the focus is on stakeholders as equal contributors and equal beneficiaries or the results. Thus, in line with Almiral et al. (2012), the Living Lab in focus for this paper, as most Living Labs, is motivated by involving users as co-creators and to experiment in real-life settings. Many researchers highlight stakeholders as research units, companies or organisations (e.g. BergvallKĂĽreborn et al., 2009). However, embedding Living Lab methods in higher education also mean highlighting the students as stakeholders. Therefore, these are in this context considered as ´users ‘of the education, however at the same time co-creators of the content, process and results. Here one need to pay attention to possible new challenges rising on for instance equality and balance of power between often young and novice contributors and more experienced researchers or company stakeholders. Thus, there is a need for supporting students with skills in how to conduct project in co-creation with several partners, for usage both during education and afterwards in their future working life. In addition, the teachers need to develop skills on how to teach these skills and how


to work with co-creation and Living lab methods in their teaching, as well as to relate the work to learning outcomes. Thus, as pointed out by for instance Hughes et al. (2014), there are challenges in developing a high effective collaborative research culture, and there is room for improvement. Therefore, all stakeholders in such a co-creation environment need models and guidance in order to improve quality. In this paper we contribute to this development by proposing a cyclic approach to the co-creation process, which by adding the five phases of analysis, assure that all stakeholders and their perspectives are taken into consideration. From initial attempts to implement the steps in some of the Information Design courses at Mälardalen University we have learned that an explicit discussion related to co-creation, and the meaning of it, create an awareness among students according to the benefit of university-society/industry collaboration in cocreation (Gottlieb & Eriksson, 2019). Next action is to implement the model in an underground course together with students, teachers and external partners. An extensive training in co-creation is needed to need the aim to strengthen the quality in higher education by implement co-creation in courses and integrate it as learning objectives.

6 Conclusion We are convinced that successful co-creation between universities – society/organizations is built on mutual contribution and knowledge sharing and co-creation requires engagement from all involved, like living lab methods. The challenges from a university perspective when it comes to living lab projects in co-creation university – society/company is to keep the engagement in the project from all involved. This needs a skill in how to conduct project in co-creation with several partners. A skill that has to be teach to students as well as to teachers and researchers. In addition, the external partner like employees in companies also need some training in how to engage and contribute to knowledge in a project. To meet the future needs for lifelong learning we suggest that living lab methods for co-creation that have a clear process for how to develop and follow up the different stages and steps during the project or course is necessary. By that, it is possible to continually evaluate and improve the process.


References Andersson, J., Åhgren, B., Axelsson, S., Eriksson, A., Axelsson, R., & Ahgren, B. (2011). Organizational approaches to collaboration in vocational rehabilitation-an international literature review. International Journal of Integrated Care, Vol. 11. Almirall, E., Lee, M., & Wareham, J. (2012). Mapping living labs in the landscape of innovation methodologies, Technology Innovation Management Review, pp. 12–18. Axelsson, K, Höglund, L. & Mårtensson, M. (2018). Is what’s good for business good for society? Entrepreneurship in a school setting. In U. Hytti, R. Blackburn & S. Tegtmeier (Eds.) The Dynamics of Entrepreneurial Contexts Frontiers in European Entrepreneurship Research, (pp 54-75). Cheltenham: Edward Elgar Publishing Ltd. Ballon, P., Pierson, J., & Delaere, S. (2005). Test and experimentation platforms for broadband innovation: Examining European practice. In Studies on Media, Information and Telecommunication (SMIT)—Interdisciplinary Institute for BroadBand Technology (IBBT). Belgium, Brussels: Vrije Universiteit Brussel. Barnes, M. & Sullivan, H. (2002). Building capacity for collaboration in English health action zones. In C. Glendinning, M. Powell & K. Rummery (Eds.) Partnerships, New Labour and the Governance of Wealfare, pp 81-96. Bristol, UK: Policy Press. Berglund, A., & Bernhard, J. (2015). Co-creation beyond the expected: LAB environments as mean to enhance learning. In 43rd SEFI Annual Conference 2015, SEFI 2015, June 29th- July 2nd, 2015. European Society for Engineering Education (SEFI). Berglund, A., & Leifer, L. (2013). Why we prototype! An international comparison of the linkage between embedded knowledge and objective learning. Engineering Education, 8(1), 2-15. Bergvall-Kåreborn, B., Eriksson, C., Ståhlbröst, A., & Svensson, J. (2009). A milieu for innovation: defining living labs. Proceedings Of The 2nd Ispim Innovation Symposium. Borys, M., Milosz, M. & Plechawska-Wojcik, M. (2012). Using Deming cycle for strengthening cooperation between industry and university in IT engineering education program, 15th International Conference on Interactive Collaborative Learning (ICL), Villach, 1-4. doi: 10.1109/ICL.2012.6402164 Brown, T. (2008). Design Thinking. Harward Business Review. 86(6), 1-10. Bölling, M. & Eriksson, Y. (2016). Collaboration with society: The future role of universities? Identifying challenges for evaluation, Research Evaluation, 25(2), 209–218 doi: 10.1093/reseval/rvv043 Chesbrough (2012). Open Innovation Where We’ve Been and Where We’re Going, Research- Technology Management, 55(4), 20-27, DOI: 10.5437/08956308X5504085


Dell’Era, C. & Landiano, P. (2014). Living Lab: A Methodology between UserCentred Design and Participatory Design, Creativity and Innovation Management, 23(2), 137-154. Fuglsang, L. & Vorre Hansen,A. (2018). A research note on living labs – exploring the link to public value and service coproduction, presented at the RESER Conference, 22-22nd September, 2018, Gotheburg, Sweden. Feuerstein, K., Hesmer, A., Hribernik, K. A., Thoben, K. D., & Schumacher, J. (2008). Living labs: A new development strategy. In J. Schumacher & V. P. Niitamo (Eds.), European living labs (pp. 1–14). Berlin: Wissenschaftlicher Verlag. Hagy, S., Morrison, G., & Elfstrand, P. (2016). Co-creation in living labs. In Living Labs: Design and Assessment of Sustainable Living (pp. 169–178). Springer International Publishing. Howe, J. (2009). Crowdsourcing : why the power of the crowd is driving the future of business . New York: Three Rivers Press. Hughes, T. (2014). Co-creation: moving towards a framework for creating innovation in the Triple Helix, Prometheus, 32:4, 337-350, DOI: 10.1080/08109028.2014.971613 Huxham, C. & Vangen, S. (2005). Managing to Collaborate: The theory and practice of collaborative advantage. New York: Routledge. Högskolelag (1992:1434). Retrieved from Sveriges Riksdag website: 19921434_sfs-1992-1434 Jones,B. & Iredale, N. (2010). Enterprise Entrepreneurship + Education. 52(1), 7-19.




Langley, G., Moen, R., Nolan, K., Nolan, T., Norman, C., & Provost, L.. (2009). The Improvement Guide, 2nd Ed. Jossey-Bass: San Francisco. Leffler, E. (2014) Enterprise Learning and School Subjects – A Subject Didactic Issue?. Journal of Education and Training, 1(2): 15-30 Leminen, S., Rajahonka, M. & Westerlund, M. (2017).Towards Third-Generation Living Lab Networks in Cities. Technology Innovation Management Review, 7(11), 21-35. Moen, R., & Norman, C. (2006). Evolution of the PDCA cycle. Nesti, G. (2017). Living Labs: A New Tool for Co-production? In Bisello. A, Vettorat, D, Stephens, R., & Elsei,P (Eds.) Smart and sustainable planning for cities and regions. Cham:Springer. Ståhlbröst, A. (2008). Forming future IT – The living lab way of user involvement. Doctoral Thesis Luleå: Luleå University of Technology. Sullivan, H. & Skelcher, C. (2002). Working Across Boundaries: Collaboration in Public Services. New York: Palgrave Macmillian.


Tillmar, M. & Lindkvist, L. (2007). Cooperation Against All Odds Finding Reasons for Trust where Formal Institutions Fail. International Sociology, 22(3): 343–366. DOI: 10.1177/0268580907076575. Vera, A. H & Simon, H.A (1993) Situated Action: A Symbolic Interpretation, Cognitive Science 17, 7-48. von Hippel, E. (1986). Lead Users: A Source of Novel Product Concepts, Management Science 32(7), 791-805.



LivingLab 65+ - Co-creation with retirement and nursing homes Veronika Hämmerle*1, Stephanie Lehmann1, Cora Pauli1 and Sabina Misoch1

1 University

*Corresponding author of Applied Sciences, Interdisciplinary Competence Centre for Ageing IKOA-FHS, Switzerland Category: Research-in-progress

Abstract In Switzerland many seniors live in retirement and nursing homes. At the same time on the global level the healthcare sector is suffering from a shortage of nursing staff. Although technology use could be a promising strategy, its potential seems not yet fully exploited. To shed light on the specific needs and requirements of retirement and nursing homes and their inhabitants regarding technology use and technology implementation, we have extended our Living Lab approach and cooperate with retirement and nursing homes throughout Switzerland. This paper discusses challenges of this test environment and strategies to overcome them. Furthermore, the Living Lab approach itself shall be evaluated as a method to facilitate technology implementation in care institutions. Keywords: Living Lab, retirement and nursing homes, technology testing, nursing staff, seniors


1 Background In 2017, 149 000 seniors lived in retirement and nursing homes in Switzerland (Federal Statistical Office, 2017). At the same time, the health care sector is confronted with a global shortage of nursing staff (WHO, 2017). Although technology could help to overcome these demographic challenges the use of Active Assisted Living technology in retirement and nursing homes is not widespread (Fehling, & Dassen, 2017a). Possible reasons for the absence of technology in retirement and nursing homes may be that technologies are not adapted to the needs of the elderly and caregivers and that creating interoperability within the complex infrastructures of retirement and nursing homes is difficult. The defensive attitudes of employees towards technology in care (Fehling, & Dassen, 2017b), a lacking overview of technological options and sparse information on systems available on the market and their quality may also play a role. Since assistive technology is evolving rapidly as demographic structures change, it is indispensable for institutions to deal with the potential of these new technologies (RĂźegger, Roulet Schwab, & Eggert 2016).

2 Living Lab 65+ To close this gap and to promote dialogue between researchers, developers and end-users, we set up Living Labs (LL) in Switzerland. In LL seniors test innovations during their everyday life and in their "natural" living environment over a longer period of time (up to 6 months). The aim of the participatory tests in the LL65+ is to understand the needs of seniors regarding assistive technologies and to test technology with target groups in their living environment. Our LL provides a neutral hub that mediates between developers and consumers. The conducted age research is based on the guidelines of the Swiss Ethics Committee. Whereas previous research in the LL65+ mainly focused on tests in private households, we extended our approach to include retirement and nursing homes with the following considerations in mind: 2.1 Access to vulnerable target-groups Retirement and nursing homes provide access to a hard to reach target group. Broadening the LL65+ to include retirement and nursing homes, will allow us to gain information on the needs and technology use of very old, fragile and technology-averse participants, their health status being constantly monitored by professionals. By enlisting the help of nursing staff, who can assess the health status and motivation of participants, we can save time in recruitment and reduce dropout rates. 2.2 Increased interoperability Doing research in institutions means technologies can be tested while embedded in a holistic system. A device should meet the needs and demands of older users, caregivers and the involved institution. This implies interoperability between different infrastructures, institute-specific routines and practices, various types of


devices and different professionals. Based on test results, technologies can then be adapted to optimally fit the requirements of these environments. 2.3 Improved matching Caregivers are an important application group for technologies. The feedback on their needs, expectations and user-experience is indispensable for the development of technology for their patients. The use of technology in healthcare institutions depends on technology acceptance and openness of professionals and decision-makers (Kowitlawakul 2011; Aloulou, Mokhtari, Tiberghien, Biswas, Phua, Lin & Yap, 2013). The cooperation of developers, researchers, medical and care experts and the elderly allows to shed light on the development of technology in a multiperspectival manner and ensure that all target-users are reached. 2.4 Caregivers taken seriously Institutions carefully consider their investments and purchasing technology is usually associated with high costs. By making technical innovations available free of charge during tests, institutions can get accustomed to new products with the support of researchers. This might lead to a higher sensitization of institutions towards technology use and could help overcome access barriers. The possibility of job losses is an ethical concern of caregivers and their rejection or skepticism is often greater than that of the elderly (Evans, Hielscher, & Voss, 2018; Shire & Leimeister, 2012). Caregivers are often sensitive towards privacy issues of seniors, especially when it comes to technology that automatically gathers data. Testing technology in retirement and nursing homes allows nursing staff to familiarize themselves with innovations and reduces feelings of anxiety (Liedtke, Welfens, Rohn, & Nordmann, 2012). By testing technologies with no strings attached, caregivers can evaluate concerns and discuss them with researchers and developers freely. 2.5 Facilitated market entry Through cooperation with institutions, developers can reach many potential users. This can improve innovation development and market opportunities and lead to a considerable market advantage, as the institutions represent a potential field of application. If institutions are satisfied with the performance of a device during the tests, they are more likely to invest in this technology.

3 Method Our approach consists of 4-5 phases. First, the needs and expectations of the end-users are assessed. Second, the end-users test the product over several months in their everyday life. The end-users are then asked about their experiences during installation, commissioning and use at different points in time. The data from phase 1 and 2 provide the basis for the report that is forwarded to the developers. Phase 4 involves the implementation of requests by the developer. The adapted product is ideally tested again in phase 5.


We have performed two LL tests in retirement and nursing homes. One product tested was a fall detector which we tested in 3 different retirement and nursing homes. The participants aged 65 and more (N=14) were at a high risk of falling and very frail. Only one could be interviewed personally. As the study was focused on the interoperability of the detector with the local infrastructure and the routines of the institutions, we refrained from involving more seniors in interviews or observations. The nursing staff and management representatives were interviewed in detail (N=6). The focus was on questions of needs, acceptance and safety from the point of view of the resident and questions of userfriendliness, safety and integration in routines from the point of view of the nursing staff and institution. The technical functionality and compatibility of the sensor with the locally used systems were evaluated with tests lasting 3-4 months. The semi-structured interviews were conducted before (P1), while and after the testing (P2). The participants’ feedback was analyzed and presented to the developer in a report (P3). This led to a new version of the device: push messages on the mobile phone were integrated, the fastening was improved with a click system and the SIM card was integrated directly into the device (P4). In our current study, a robotic seal developed for dementia care (PARO) is tested with cognitively healthy but lonely residents. We aim to find out whether the robot has an impact on loneliness and well-being of the elderly. Tests with 3 participants revealed that it is difficult to gain the trust of lonely people. Due to the frailty and vulnerability of the participants, we decided neither to conduct interviews with them nor to observe or record them. Only in a protected and private environment, it is possible to gain access to this group. We were able to create such a test setting by involving the participants’ activation therapists. The professionals (N=4) were interviewed regularly and gave feedback on the state of the seniors. Additionally, activation therapists were asked to evaluate usability and describe their own user experience in semistructured interviews. We prolonged the project phase to involve more participants and refine our methods and recruiting strategy. A similar procedure as in the fall detection study can be expected after completion.


More tests in retirement and nursing homes are planned in the upcoming years. Additionally, we plan follow-up research on the impact of the LL tests on the technology use and implementation of participating retirement and nursing homes. Six months after completing the LL tests, we aim to investigate whether and how the test contributed to technology acceptance in retirement and nursing homes, whether it had an impact on the attitude towards technology and whether the experience gained during the tests boosted the technological self-efficacy of nursing staff. Furthermore, the number of integrated technologies since the LL tests will be evaluated.

4 Challenges and strategies to overcome 4.1 Dropouts Rapidly changing circumstances and high mortality rates can lead to high dropout rates in research with elderly people. Therefore, it is indispensable to recruit participants with care (Ogonowski, Ley, Hess, Wan & Wulf, 2013). Institutions can be beneficial partners in the task. Attending staff can estimate which persons are able to participate according to their health status and who would benefit from the innovation. With the help and the motivating nature of nurses, dropout levels are low and seniors are willing to take part in the test. 4.2 Permanent contact person As vulnerable participants’ health status must not be endangered by tests, the involvement of a caregiver is recommended. Caregivers perceive the concerns of the participants and ensure that participants don’t feel negatively affected by the testing. A specific researcher should be defined as a contact person (Georges, Schuurman, Baccarne, & Coorevits, 2015; Hess & Ogonowski, 2010; Ogonowski et al., 2013) and during the tests the nurse involved must receive full backing and support of the institution and researchers. 4.3 Appropriate timing Another challenge is the limited time and opportunities for testing because of the high nursing workload and frailty of the elderly participants. The use of carefully selected and developed interview instruments for seniors and caregivers are essential. Adequate instruments should take the limited time resources of nurses into consideration. We noticed for example that it is easier for nurses to report their experiences in an oral interview rather than a written questionnaire. The use of other media like phone or e-mail to conduct interviews keeps the data collection more flexible and reduces the effort for participants. The quality of research can be endangered by wrong timing. Institutions follow a strict daily routine, that ensures a smooth workflow for nurses and structures the elderly’s day (Zysberg Young, Schepp &, 2007). Research should not interrupt or disturb these routines. Consulting nurses in advance can prevent interviews being conducted at inappropriate times, e.g. during rest periods or immediately after treatments. 4.4 Limited infrastructure


Clarifying infrastructure requirements with the developer and the institution is crucial, as institutions may not have the necessary technical resources and infrastructure to test certain technologies. Many retirement and nursing homes, for example, do not provide WIFI in private rooms, a requirement for the use of many devices. Clarifying requirements with the developer and the participating institution before the testing is therefore important. If necessary, temporary alternatives can be found for the testing period, e.g. portable WIFI extenders. 4.5 Adapted instruments Vulnerable seniors might not find it easy to protect their interests and rights (Moser-Sigmeth & Hofer, 2013). High sensitivity, ethical conduct and the development or adaption of instruments that are operable with the special circumstances of very old and fragile persons are needed. Our tests show that seniors often prefer qualitative interviews over quantitative questionnaires. Because of possible health problems like hearing deficits, visual impairments, cognitive slowing, etc., questionnaires should be adapted accordingly (Brandt et al., 2018; Lang, 2014; Resch & Aumayr, 2011), and should be short and easy to understand (Mody, Miller, McGloin, Freeman, Marcantonio, Magaziner & Studenski, 2008).

5 Conclusion Currently, we are expanding our LL network and evaluating first results. We gained knowledge about specific requirements and advantages of LL tests in retirement and nursing homes and contribute to the LL movement by providing this information. We learned that successful research must take the frailty and impairments of seniors, as well as the limited time resources and the fears of caregivers into account and that it is important to embed the tests into the daily routines and given infrastructures. Moreover, it is important that decision-makers are open to the tests. Institutions can discover new technologies, they can help make technologies better suitable for users, and they can