Extended Interaction Space

Extended Interaction Space

Felix Schwamkrug

Research questions: ¬ Why do we need to change the current way of interacting with the digital world, in the particular case of working from home and which situations would benefit the most from a more refined interaction? ¬ Is the incorporation of gamification elements capable of improving the adaptation or the overall experience of a service? ¬ How can a remote workers’ home be transformed into an interactive space where rooms can be altered to fit new purposes as well as perceiving the presence and actions of coworkers, all using a universal solution?

Extended Interaction Space Examining working models to find a solution to achieve a more sociable and effective workday Felix Schwamkrug Superviced by: Prof. Carola Zwick Prof. Judith Glaser Prof. Dr. Patricia Ribault Master of Arts in Product Design Kunsthochschule Berlin Weißensee April 2022

Research & Theory

Concept Development

3 6 8 10 14 16 18 20 22 28 30 32 34 36 40 44 48 50 52 56 62 64 66 68

Research questions.

Contents.

Preface.

The status quo.

Introduction.

Preliminary problems.

Work.

Working from home.

COVID–19.

Working together.

Trends for basic needs.

Communication.

Motivation.

Gamification.

Flow.

Psychology.

Smartphones.

Screens.

Interacting with tech.

Sensors.

Wearables.

Computing and intelligence.

APIs. Timeline of Smartphone Milestones.

70

Connecting the dots.

77 78 80 82 83 85 87 91 93 95 100 103 104 105 106 108 109 111 112 114 118 120 128 132 136

Concept definition.

The decision-making process laid out.

Expert interviews.

A Day In A Life.

The Takeaway.

The detailed chart of daily smartphone use.

Comparing the methods of digital interaction.

Inputs & Outputs.

Daily situations sorted by working models.

The conditions that emerged.

The interaction mechanics.

List of hardware requirements.

Moodboard.

Design notes.

Design Process.

1:1 scale model.

Working model.

Information on screen.

Interaction in HOMELY OFFICE broken down.

Gamification in HOMELY OFFICE. The three main modes.

A day of work at home with HOMELY OFFICE. Dock explained.

Interface.

Discussion.

137

Research questions revisited.

8

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

people do things the way they do in recent years, combined with education that enabled me to use this tacit knowledge (4) in a reproducible and comprehensible process of ideation, product development, and solutions. Having to make a difficult decision with numerous ties to other aspects would be impossible or extremely slow without this implicit knowing emerging from the subconscious library of general deductive knowledge, past experiences (inductive knowledge), and biases (abductive inference). This means that if a paragraph contains information that is not common knowledge, it will be marked with a number and a footnote linking to the source – or it is based on my differentiated experiences with the subject.

In my research on how to use technology to ease the transition from work to retirement, I’ve spent a lot of time looking at different generations and their use of technology. It was the research into the interface between man and its digital world that finally led me to focus on the smartphone, as it precisely represents the state of the art for that purpose from a technical point of view. In face, it is so high-tech, some of it is capabilities, sensors and data gathering possibilities never get utilized in a lifetime of such a device — more on that later.

Preface.

Identifying problems I’ve always been someone who either finds technological solutions incredibly inspiring or sees so many problems and opportunities for improvement that I cannot stop myself from thinking up new ones. Most of the time, however, I am both of those people at the same time, trying to figure out what I find so brilliant about these things and applying them to other areas and issues.

I grew up in a designers environment, surrounded by timeless and practical things. School did not help me with creative expression or critical thinking, my studies at the University of Wuppertal, internships, workshops, and sponsored projects were all the more rewarding as a result. I eventually followed in the footsteps of an industrial designer, using my own drive and enthusiasm to improve the world of products and services.

From then on, resolving each problems by applying logical processes, I found myself having to take a step back again by getting new impressions, meeting people and starting new studies - to get away from problem solving and questioning established practices and situations and understanding shifts in society, technology and resources.

This interest in many technological and engineering principles, trend-setting technological developments, sociology, and why

I learned a lot in the past months about interpersonal and interaction possibilities, artificial intelligence possibilities; about hardware currently accessible to us and technologies changing our future; about the various hybrid and non-hybrid work models, practices and sociological changes that are taking place partly slowly in the background, partly right before our eyes. Then I formulated preliminary questions about work, technology, and communications, compared contemporary and common work models, their benefits and drawbacks, and began to carry out further research into each topic that emerged on the way to my solution to some of the findings.

2 Antonio Damasio. Descartes’ Error: Emotion, Reason, and the Human Brain, 1994 3 ericssoners.wordpress.com, GS88 ‘Penelope’: Smartphone Concept by Ericsson, received 17.02.22 4 Michael Polanyi. Personal Knowledge: Towards a Post-Critical Philosophy, 1958

I will begin with a prologue of scenarios from the various work models currently in use, and then transition into the research with the preliminary questions adding to the main research questions.

Subsequently of the research, there will be a conclusion where all dots are connected to create a profound structure for creating a concept resultant of that analysis.

9

10

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

activity, like getting water from the dispenser, or getting coffee from the machine.

At the workplace, in order to be able to focus more effectively, one can wear headphones with supportive music, let colleagues know that one does not want to be disturbed, and possibly even redirect phone calls to a colleague. People in the vicinity, on the other hand, may interfere with the effectiveness of work – it is hard to get in the zone with people running in your peripheral vision.

Some modern offices offer separate rooms or areas where you can work all on your own.

Lunch breaks are scheduled every day and are spent with colleagues in the cafeteria or nearby lunch restaurants. On these occasions, you get to know each other and meet new people.

The status quo.

The modern workplace – the proven reference model Working at your own workstation in a room that accommodates people of different disciplines.

But first, you have to get there. For some, the workplace is located in the neighborhood, while for others it is a long daily trip.

Employees are provided with good conditions and all the necessary tools for effective work.

The work schedule is regulated; most colleagues start and finish their daily work at a similar time.

The rest of the lunch break, if you are lucky and depending on the office layout, can be spent alone in the recreation area of the office.

Napping is rarely an option.

The end of work is time-bound for most of the regular employees, and through a chat, overlapping hobbies are discovered and lead to a joint activity after work.

The end of the workday usually marks the start of personal life, with only the most urgent concerns being addressed through private communication channels.

There is always the possibility to exchange work-related issues with colleagues. This can be done by phone, chat, or face-to-face conversation with a colleague that starts with a simple hand gesture or by going to the colleague’s workplace. Pinning thoughts on a collaborative white board adds to the spirit, and seeing others work passionately is encouraging.

People meet in the hallway, at the water dispenser, at the coffee machine on the balcony, at the front door, or in the parking lot.

These are different settings with different retention times and thus bring diverse contexts to the conversation. These encounters come along the way; the actual focus at the locations is on the main

11

12

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

During downtime, you can recharge without being disturbed.

Whether you are alone during this break or other people elsewhere are having loud conversations, you would not notice it anyway. You can take a short nap, engage in a hobby, or do errands at home.

While working, there is very little opportunity to indicate to colleagues that you are busy. A project has to be completed in only a little time, and you do not want to be disturbed in any other way for a specified time. Setting the status to “busy” in the instant messaging platform and replying to calls with a “Can not talk right now” quick response is the usual approach.

In addition to colleagues, you often get in your own way. At home, various entertainment sources are in close proximity, as are other distractions irrelevant to work, which disrupt the flow.

The status quo.

The remote home worker – standard in recent years Being self-disciplined is the first item on the agenda as soon as you get up. Keeping the daily starting point of the work the same is possibly not checked or needed, especially when self-employed.

The end of the workday can be as ceremonial as closing the laptop and leaving the apartment, or less than ideal, like closing the work document on the computer and opening something else. In between business hours, personal and work-related matters are often discussed on a variety of channels, which can make it difficult to finish the day.

Depending on the situation, one may wake up in the office and start the day in the in-house cafeteria – the kitchen.

A short walk in the neighborhood provides you with fresh impressions – or at least, fresh air.

The workplace is in the living room, and for some, it is in a room of their own. It is flexible and can vary, but it needs to be set up for high efficiency. A day at home is usually filled with video conferences, phone calls, and exchanges of files and screenshots. In collaborative work, part of the effort always goes into preparing and sending out work that has been done in the form of presentations or annotated files.

Breaks are taken at times that fit the agenda. Informal communication is possible via phone and chat. If you have already met and exchanged contact information, you can set up a time to talk.

However, the time spent on break tends to be used for private conversations with people outside of work.

Comparing two parallel working model status quos.

When comparing these different work models, it shows that both being in the office at a scheduled time and being at home building their own workflows have their own set of characteristics. Differences are apparent, especially in the collaborative and social aspects. While working from home provides a more flexible and calm environment, it requires self-discipline and strong communication skills to achieve the efficiency of a dedicated workplace. Meeting new people and socializing remotely is much harder and less spontaneous than at the office, too.

13

14

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

remote working (noun) the practice of an employee working at their home, or in some other place that is not an organization’s usual place of business.

(Cambridge Dictionary)

Introduction.

It is time to redefine work and life The way we work has changed. The lines between work and life are blurring. The standard of 9-to-5 is also melting away.

Societal, technological, and administrative shifts will need to take place to make these transitions happen. The groundwork for the new normal had already been laid, albeit involuntarily hastened by a worldwide pandemic. Now there are different types of alternative models, spanning from hybrid office-centric presence requirements with a negotiated Friday working from home to full-time remote employment without ever getting to know your co-workers.

There is not one model that fits all and it probably never will be one, but flexible work hours and locations are here to stay, and that is a good thing. Later on, I will get into the ups and downs of that new lifestyle, if imposed or earned, in detail.

How can a remote workers’ home be transformed into an interactive space where rooms can be altered to fit new purposes as well as perceiving the presence and actions of coworkers, all using a universal solution – like the smartphone.

Aside from the changes in work structures, the smartphone as used today has created the greatest shift in technology since the Internet did in the 90s. With this shift came a lot of new possibilities. Both the personal and commercial landscapes have changed dramatically, with things like apps, voice and video calls, and powerful computational power outrunning personal computers. Next to that, there are built-in cameras, which are also more capable than compact cameras, and sensors like microphones,

“In a few years remote work will just be work.”

(Prithwiraj Choudhury, Bloomberg Business 2022)

geolocation, and several wireless connection transceivers. The latter is sometimes used to make calls. The “phone” part of the smartphone’s name is almost becoming misleading, as it allegedly has its origins in a 1997 Ericsson phone concept that could do some of the things mentioned above. (3) On my journey, I was more interested in the “smart”-part of the smartphone, and you will find the answer later in the thesis. The pseudo-anglicistic German term “handy” is quite fitting and timeless, though it may be a little too general when used in an English context.

Many things became more efficient, pleasant, cheaper and easier with the smartphone. Others got more annoying, more invasive, more stressful more addictive. With every technical upgrade, simplification of high-tech and new applications this device became more of a part in our life.

In the following pages the topics leading up to the final research questions are being outlined and concluded. Complicated information framing this thesis is untangled and the general base of knowledge is being laid out.

Aside from the main questions of why we need to change the current way of interacting with the digital world and which situations would benefit the most from a more refined interaction I am elaborating on other questions too.

What technology helps amplify the benefits of working from home and minimize the downsides?

Is the implementation of elements of gamification able to enhance the adaptation or the entire experience of this service?

15

16

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

Three main subjects Subsequently these are the three major areas I will focus on over the course of the work and the development of the concept.

Preliminary problems and opportunities regarding the status quo to working remotely: ¬ Spending too much time in front of the screen ¬ Excessive amount of distraction ¬ Time-consuming daily planning ¬ Insufficient communication ¬ Solitude when working from home ¬ Lack of separation of work from personal life ¬ Untapped smartphone’s whole potential ¬ Making use of various spaces and behavioral patterns

17

masterthesis

2021 / 2022

18

extended interaction space

theory and research

felix schwamkrug

presence (noun) “The office during the day has become the last place people want to be when they really want to get work done.”

(Jason Fried and David Heinemeier Hansson 2013)

the fact that someone or something is in a place or the feeling that someone is still in a place although they are not.

(Cambridge Dictionary)

Work.

At the office or somewhere else?

Heavy automation is responsible (and is being continued to be blamed) for loss of so many jobs in various industries that were highly dependent upon humans, just a decade ago. In the future as well, automation will continue to rise, and continue to make lives easier for the people, and harder for those looking for jobs that can be automated.

(4)

Remotely.

Few people will say “the office” when asked where they go when they really need to get work done. If they say “office”, they will add a qualifier like “super-early in the morning before anyone gets in”, “I stay late at night after everyone’s gone”, or “I work on the weekends.” (1)

meeting, or some other institutionalized unnecessary interruption fills each segment.

When your workday has been cut into smaller task, it is incredibly difficult to get meaningful work done. (1) Work that is meaningful, creative, or important requires long periods of uninterrupted time. Long stretches of time are hard to come by in today’s workplace.

close to the office to make commuting easier.

Fully remote employees, on the other hand, can work from anywhere in the world. (1) People are booking working vacations, moving their remote offices to be near parents they have not seen for more than a year or to tropical locales, and thinking about what the ideal work-life balance should look like. (5)

One of the major benefits of working remotely is the ability to be alone with your thoughts. When you work alone, away from the busy office, you might find your own productive zone.

Working from home, a cafe, or a co-working space is what remote work entails. (2) Remote workers use digital tools to manage activities, complete projects, and communicate They are trying to tell you that they are unable to with their team, rather of coming into the office complete tasks at work. When it comes to getting and meeting with coworkers face to face. If you work done, the office has become the last place have a laptop and a reliable internet connection, people want to be during the day, states Jason you can work remotely from anywhere.

Fried in his book dealing with the topic. (1) In fact, Employees might work fully remote or partly offices have turned into a factory of interruptions. remote. Partially fully remote employees may be A crowded office works like a food processor, obliged to visit the office at least a few times per chopping up your day into tiny pieces. 15 minutes month, whereas fully remote employees may not here, 10 minutes there, 20 minutes here, 5 minbe required to visit the office at all. utes there. A conference call, a meeting, another Partially remote employees are required to stay

1 Jason Fried & David Heinemeier Hansson. Remote: Office Not Required, 2013 2 kissflow.com, Remote Work vs Work From Home vs Flex Work vs Telecommuting: What’s The Difference?, received 21.02.22 3 William Gant. Remote Work, 2021 4 Mehek Kapoor. uxdesign.cc, The future of UX design is not on a screen, received 12.04.22

19

20

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

Tips for working from home:

Working from home.

Why should people consider

Opportunity Cost.

Working from home does not have to be limited to the fully remote working employees. It can also be for people who want to take a few days off from While there is a hidden, insidious cost to wasting work to focus on a few important tasks. Working time and money driving to and from work, comfrom your living room, bedroom, or even a dedimuting to work can also limit your future success cated home office is an ideal solution. You could in a variety of ways. even set up your laptop on the kitchen table while preparing lunch.

Given approximately half a dozen hours a week of Working from home, in general, represents a sig- wasted time in a commute, one would be wise to nificant shift in your normal work pace, which for consider what else they could be doing with just many people can be beneficial. It is not your typi- the drive time and money spent on commuting. cal way of working, but it is also very efficient. (2) If you do not have to go to work on a regular basis, For those who enjoy working from home, it may be you may be able to relocate to a less expensive, a good fit for their long-term goals. Others benefit more relaxed area. Housing expenses account from it by saving money, time, and stress, which for a significant amount of practically everyone’s improves their quality of life. Others, on the other living expenses, so lowering them is immensely hand, find that it allows them to have a job at all. useful. (3) Many companies are now enabling employees to work remotely one or two days per week, which is a good balance for both the employer and the employee.

Remote work, on the other hand, has its critics.

For some of those people, especially those who tend to be more social, remote work sounds like a terrible level of isolation. Working remotely sounds like a recipe for job loss to those who lack discipline or are concerned about how management will perceive remote workers.

Anyone can easily surpass coworkers who are not able to work remotely if they work an extra five hours a week in a more pleasant setting. If handled properly, this can have a significant impact on the individual.

Of course, during the COVID-19 pandemic, when people were suddenly forced to work from home, workers and management optimized infrastructure for this workflow, significant progress was made. More on that in the next chapter.

¬ Communicate expectations with members of your home. ¬ Make a dedicated (clutter-free) workspace. ¬ Interact with other humans face-to-face. ¬ Take time for “water cooler” chats. ¬ Prepare meals the night before. ¬ Be easy and kind on yourself. ¬ Pick a definitive finish time. ¬ Set and stick to a routine. ¬ Take clear breaks. ¬ Eat and sleep.

(3), (4), (5), (6)

2 kissflow.com, Remote Work vs Work From Home vs Flex Work vs Telecommuting: What’s The Difference?, received 21.02.22 3 William Gant. Remote Work, 2021 4 Erik Devaney. hubspot.com, How to Work From Home, received 12.04.22 5 inc.com, Essential Tips for Working Remotely, received 21.02.22 6 nhs.uk, simple tips to tackle working from home, received 21.02.22 7 net.nz, simple wellbeing tips when working from home, received 21.02.22

21

22

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

COVID–19. accelerated digitalization …

How it changed how we work, …

Since the beginning of the COVID-19 pandemic in March 2020 in Germany, many companies allow their employees to work from home. (1) In addition to social contact restrictions, working predominantly from home also reduces social contacts among colleagues. Thus, with increasing duration and intensity of home working, employees may be affected by adverse factors such as the increasing risk of social isolation. (2) Currently, no robust occupational psychology evidence is available in this regard from the COVID-19 pandemic period. Therefore, for the design of a work organization that is as health-promoting as possible and counteracts social isolation of this target group, conclusions can only be drawn from previous research findings on virtual teamwork and teleworking.

However, these do not fully represent the current complex situation.

Social isolation in general can be associated with various negative mental health consequences, such as depression and anxiety disorders, for both older and younger people. (3) However, working in a home office, in addition to some health-promoting aspects such as flexible working hours or increased freedom of action, which can lead to higher job satisfaction, motivation and performance (4), (5), also has disadvantageous factors, such as the increased risk of

social isolation.

Close contact with the manager, information exchange, feedback and trust play a decisive role. (6) The latter forms the basis for successful virtual collaboration and has a greater impact on performance in location-independent teams compared with traditional on-site collaboration.

Digitization refers to the technical process of converting analog or traditional paper-based tasks or processes to digital form so that computers can help in accessing, storing, and transmitting information. (7) By contrast, digitalization refers to “the sociotechnical process of leveraging digitized products or systems to develop new organizational procedures, business models, or commercial offerings”. (8)

1 Deutscher Gewerkschaftsbund. Corona und Homeoffice: Wann darf ich, wann muss ich zu Hause bleiben und von dort arbeiten? Fragen und Antworten zum Arbeiten von zu Hause, 2020

The Covid-19 crisis has opened senior leaders’ minds to the idea of adopting work-from-anywhere for all or part of their workforces.

We got our answer during the pandemic lockdowns. We learned that a great many of us do not in fact need to be co-located with colleagues on-site to do our jobs. Individuals, teams, entire workforces, can perform well while being entirely distributed — and they have.

However, concerns persist regarding how workfrom-anywhere affects communication, including brainstorming and problem-solving; knowledge sharing; socialization, camaraderie, and mentoring; performance evaluation and compensation; and data security and regulation. (9)

2 Tammy D. Allen, Timothy D. Golden & Kristen M. Shockley.

Psychological Science in the Public Interest Journal, How effective is telecommuting? Assessing the status of our scientific findings, 2015 3 Susanne Röhr, Felix Müller, Franziska Jung, Christian Apfelbacher, Andreas Seidler & Steffi G. Riedel-Heller. Psychiatrische Praxis, Georg Thieme Verlag KG, Psychosocial impact of quarantine measures during serious Coronavirus outbreaks: a rapid review, 2020 4 Ravi S. Gajendran & David A. Harrison. Journal of Applied Psychology, The good, the bad, and the unknown about telecommuting: meta-analysis of psychological mediators and individual consequences, 2007 5 Sumita Raghuram & Batia Wiesenfeld. Human Resource Management, Work-nonwork conflict and job stress among virtual workers, Summer/Fall 2004 6 T.A. Bentley, S.T.T. Teo, L. McLeod, F. Tan, R. Bosua & M. Gloet. Applied Ergonomics, The role of organisational support in teleworker wellbeing: a socio-technical systems approach, 2016

Prithwiraj Choudhury from Harvard business school states that synchronous communication gets increasingly challenging when workers are

separated. Zoom, Skype, Microsoft Teams, and Google Hangouts can benefit individuals who work in the same or comparable time zones, but not those who operate in other time zones.

Work-from-anywhere organizations must become accustomed to asynchronous communication, whether through a Slack channel, a customized intracompany portal, or even a shared Google document in which geographically dispersed team members write their questions and comments, trusting that others in different time zones will respond as soon as possible.

7 Jason Bloomberg. Forbes, Digitization, digitalization, and digital transformation: confuse them at your peril, 2018 8 Ted Saarikko, Ulrika H. Westergren, Tomas Blomquist.

Digital transformation, Five recommendations for the digitally conscious firm Business Horizons, 2020 9 Prithwiraj Choudhury. Harvard Business Review 98, Our Work-from-Anywhere Future, (No.6) 2020

23

24

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

and left many people unprepared in the process.

Many people were unprepared for the new work situation because COVID–19 swept the globe without warning, leaving them overwhelmed and inefficient.

“Colleagues are less likely to become close friends because their face-to-face interactions are less frequent.”

(Cecily D. Cooper and Nancy B. Kurland 2002)

Working hours.

Work-home interference.

Procrastination.

Monitoring.

In our society today, characterized as it is by economic crisis and downsizing, the workload of the employees who have managed to keep their jobs tends to increase. In addition, technological advances have resulted in an increase in non-standard work schedules, including evening, night, and weekend work. (7)

Most participants in a study on working at home mentioned that they were struggling with hometo-work interference and work-to-home interference. First, working at home means more interruptions from family, which may negatively influence work effectiveness. Notably, schools had been shut down during the COVID-19 outbreak, working parents, therefore, faced a bigger challenge in balancing work and family roles. (1), (13) In addition, individuals’ work invaded their life domains during the period of working from home.

These interferences from work domains could make people feel exhausted. A project manager shared her experience of being “always online”: “I’m basically always online…my supervisors and colleagues may come to me whenever they need something from me, and you have to give immediate response.” The need to be always online affected this project manager’s ability to meet her family obligations. (1)

Procrastination, defined as the irrational delay of behavior (2), is one of the biggest productivity killers at work. Procrastination is common in the office-based workplace (3) and it can become even worse when people work from home. Most workers are committed to working productively as usual, they are sometimes struggling with self-regulation failure. The studies indicated procrastination as a challenge whilst working from home. (1) They found that participants delayed working on their core tasks via spending time on non-work-related activities during working hours, such as using social media and having long breaks.

A study done in China of 39 full-time employees who were obliged by their employers to work from home until further notice was done. The participants worked in a variety of sectors and jobs. Ten participants reported that they experienced different forms of monitoring from their supervisors, including daily reports, clocking in/ out via applications such as DingTalk, and being required to have a camera on whilst working.

They also suggest that work-related social support can aid in the reduction of procrastination.

Procrastination can be used to relieve stress. (4) In this unique situation, social support is especially crucial since it can operate as a “negativity buffer” (5), assisting workers in coping with stress and focusing on duties. Furthermore, prior research has demonstrated that social support can contribute to increased organizational commitment. (6) As a result, employees who receive greater social support prefer to reciprocate their employers by focusing more on their work.

25

26

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

Ineffective communication. Loneliness.

Remote workers rely heavily on information and communications technology (ICTs) to communicate and collaborate with colleagues, supervisors, and clients. Especially during the pandemic, ICT-mediated communications almost become the only option because workers were not able to engage in face-to-face meetings. Twenty-one participants identified that they suffered from low productivity caused by poor communications during this period. Workers experienced lower efficiency in ICT-mediated communications as opposed to face-to-face communications in the office. Online communication supposedly has a time cost.

Remote working means fewer face-to-face interactions with colleagues and supervisors. Given the restrictions on non-essential social gatherings during the pandemic, people also lost social opportunities to meet their friends or colleagues, which inevitably contributed to the feeling of loneliness. Workers indicated loneliness as a challenge. Although individuals can connect with colleagues via ICTs, conversations with colleagues were more task-focused, which cannot meet her psychological needs for belongingness or relatedness. (1)

1 B. Wang, Y. Liu, J. Qian & S.K. Parker. Applied Psychology, Achieving effective remote working during the COVID-19 pandemic: A work design perspective, 2021 2 P. Steel. Psychological Bulletin, The nature of procrastination: A meta-analytic and theoretical review of quintessential self-regulatory failure, 2007 3 J. Kühnel, R. Bledow & N. Feuerhahn. Journal of Organizational Behavior, When do you procrastinate? Sleep quality and social sleep lag jointly predict self-regulatory failure at work, 2016 4 Jennifer A. A. Lavoie & Timothy A. Pychyl. Social Science Computer Review, Cyberslacking and the procrastination super-highway, 2001 5 Yuen Lam Bavik, Jason D. Shaw & Xiao-Hua (Frank) Wang.

Academy of Management Annals, Social support: Multidisciplinary review, synthesis, and future agenda, 2020 6 V. Rousseau & C. Aubé. Journal of Social Psychology, Social support at work and affective commitment to the organization: The moderating effect of job resource adequacy and ambient conditions, 2010 7 M. Härmä. Scandinavian Journal of Work, Environment & Health, Workhours in relation to work stress, recovery and health, 2006

27

28

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

Working together.

Human collaboration.

… when being apart People clearly require socialization, particulartopics of conversation. This is important in any ly in the monotonous environment of working social setting to help frame conversations - for from home. This deficiency is clearly visible in both listeners and speakers. A dditionally this today’s organizational structures, according to kind of low-threshold connection of diverse indistudies (1). Some technology is already available, viduals of life while making the talks relevant was and the COVID-19 pandemic has made great very creative for that time. strides in addressing this problem. Many business meetings are now held using video conThere are also a few startups working on soluferencing services like Zoom, Microsoft Teams, tions to bring social office life home, such Google Meet, and others. Some of them even as “Kosy Office,” “Twine for Zoom” and host digital breakout sessions and even Friday “SpacialChat”. Virtual interfaces, digital spaces, sundowner meetings. customizable avatars and proximity chat are all part of the solution. This means that you move around in a digital space as a digital character that is personalized to the user’s cosmetic preferences, meet other coworkers, and can communiCommunication encompasses not only verbal cate with them when you are in virtual proximity. communication, such as spoken words, but also That sounds like something out of a video game. nonverbal communication, such as facial expresThat is because all of these elements come from sions, contexts, and body language, all of which popular games that have enabled communicaare difficult to convey via digital means. Later in tion and socialization among players for years research, I will go into deeper levels of this top– particularly during the pandemic, when more ic. I’ve already discussed the “Clubhouse” platpeople got into casual gaming and missed out on form, which provides people with a stage, shares socialization. this communication space with listeners and interlocutors, and thus provides context to the

Working on a particular subject is also greatly benefited by having all participants in the same room. What is it about collaborative work that makes it so powerful and enjoyable that it is so difficult to recreate when teammates are working in different locations? It is all about the human element. This human element is what is lacking in remote cooperation and what brings us together when we work in a physical place. It is a type of energy felt when people are working together passionately, and it is highlighted in studies and results from workshops.

Digital substitutes.

1 B. Wang, Y. Liu, J. Qian & S.K. Parker. Applied Psychology, Achieving effective remote working during the COVID-19 pandemic: A work design perspective, 2021 2 E. R. dos Santos. ncbi.nlm.nih.gov, Association between COVID-19 and anxiety during social isolation: A systematic review, 2021, received 12.04.22

Other people can, of course, be disruptive, particularly in the workplace. People are sometimes under pressure, feel insecure, and certain structures even encourage competitive behavior amongst members. Working from home helps you push this unpleasant aspect to the background. Furthermore, people lose some of their ability to engage with others, or they develop communication concerns and even fears. (2) However, I will focus more on the positive elements of working together, since this should be the norm and the ideal environment.

29

30

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

Trends for basic needs.

New tricks for old matters

Naps.

People no longer need to leave their homes to This app delivers a welcomed break from monotcommunicate with friends or meet new people. ony, boredom and screen time with the power of Traditional motivation methods that rely on face- voice. (1) to-face meetings are being impacted by this type of social connection. Weight Watchers’ success, The app creates a sense of intimacy that is similar for example, can be attributed in part to their to sitting next to someone and having a chat - it online meetings. Getting support from a group of is the human touch that social media has been people who are going through the same change is missing. Members can listen in real time, particinexpensive and simple thanks to social network- ipate in a chat, and register to speak in front of ing sites like Facebook. (3) potentially thousands of people. However, for the time being, the hype has declined.

Podcasts & Clubhouse.

A podcast is the streaming age’s answer to radio.

You can listen in on your preferred people’s conversations whenever you want.

Since both forms are audio-only, Clubhouse has been compared to a live podcast. However, there is one key distinction between the app and podcasts: spontaneity. Podcasts need preparation and planning. Clubhouse is a spontaneous gathering. Members can go from room to room, and those who want to stay in one club or room for an extended period of time are nevertheless exposed to dynamic change as subjects alter and listeners, participants, and presenters arrive and depart. Because events are not recorded, you either heard it or did not.

Sleep deprivation affects perspicacity, patience, and creativity.

Younger workers are more likely to take a midday nap. Millennials are 2.5 times more likely than Baby Boomers to nap, while Generation Z is 3.4 times more likely. It is also worth noting that Better Sleep’s poll is relatively new. (4) These naps are not mischievous attempts to shirk their duties — they are pre-planned blocks of time intended to find some restoration in the middle of the day. The NeuroLeadership Institute contends we can only produce six hours of legtimitate “work focus” a week. (5)

1 Valerie Lalonde. deptagency.com, received 15.03.22 2 mayoclinic.org, Napping: Do’s and don’ts for healthy adults, 2020, received 28.02.22 3 William Gant. Remote Work, 2021 4 bettersleep.org, Sleeping on the job, August 2021, received 20.03.22 5 insidehook.com, The Pandemic-Induced Rise of the Secret Workday Power Nap, September 2021, received 20.03.22 6 sleepadvisor.org, How Long Should It Take To Fall Asleep? – Tips and Advices, 2020, received 20.03.22 7 Junhong Zhou, et al. nlm.nih.gov, Pink noise: effect on complexity synchronization of brain activity and sleep consolidation, 2012 8 Nelly A. Papalambros, et al. frontiersin.org Acoustic Enhancement of Sleep Slow Oscillations and Concomitant Memory Improvement in Older Adults, 2001

People typically take upwards of seven minutes to fall asleep, and this is in their own beds, in the dark, at the end of the day. (6) What if you’ve convinced yourself that you need a power nap to finish an afternoon assignment, but you cannot sleep?

Surprisingly, a good nap necessitates a less-thanideal sleeping arrangement because the ideal power nap, according to studies, is between 20 and 30 minutes, and sleeping on the desk or the couch prevents deep sleep.

The longer the nap, the more likely it is that individuals will wake up tired and lethargic. Young

adults, on the other hand, may be able to tolerate longer naps. Naps should be taken early in the afternoon. Napping after 3 p.m. can disrupt nighttime sleep. Individual factors such as sleep need, sleeping schedule, age, and medication use all influence the best time of day to nap.

The atmosphere should be relaxing. It is preferable to nap in a quiet, dark place with a comfortable room temperature and few distractions. (2) Many people enjoy facilitating “sleeping aid” noises. White noise and pink noise have been shown to have positive effects. Researchers discovered that constant pink noise reduces brain waves and that there is a positive link between pink noise and deep sleep, which enhances stable sleep in a 2012 study published in the Journal of Theoretical Biology and a 2017 study published in Frontiers in Human Neuroscience. Rain, a constant sound of a highway, or the sound of the ocean are also popular noises to sleep to. (7) (8)

31

32

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

gesture (noun) a movement that you make with your hands, your head or your face to show a particular meaning.

(Oxford Dictionary)

Communication.

Eye contact.

Understanding the basics Any given situation may involve many interacting contexts, including the retrospective context and the emergent context. The retrospective context is everything that comes before a particular behavior that might help understand and interpret that behavior, while the emergent context refers to relevant events that come after the behavior.

(1) The meaning of nonverbal cues appear to persist across cultures. (2) That is true for inter-human communication. With artificial assistents and data collection on the rise, we nowadays expect devices to know and understand the context where conversation and interaction take place and interact accordingly.

Communication can therefore divided in verbal, nonverbal and context. I will also ignore other forms of communication for this matter like written or other sensory communication.

Verbal.

Verbal communication, often known as communication, is a type of oral communication in which the message is conveyed using spoken words. I will not go into extensive detail, but in general, the success of verbal communication is determined not just by the speaker’s capability, but also by

haptic communication. Each of these sends nonverbal clues about the intentions and emotions of the person who is being touched. They also trigger emotions in the receiver, both pleasant and negative. (7)

the receiver’s ability to listen. The effectiveness of communication is determined by how well a human or a device listens to the subject’s matter.

(5)

Non-verbal.

Nonverbal behavior, such as facial expressions and body movement, appears to reveal more information about what one is thinking or feeling than words, according to research. (1)

Vocalics: A person’s speech and voice patterns can provide a lot of information. Nonverbal cues might include tone, pitch, volume, length, intonation, and pace, voice quality, speaking style and speech intelligibility, and accent.

Touch: The study of touching as a kind of nonverbal communication is known as haptics.

Handshakes, holding hands, kissing, back slaps, high-fives, shoulder pats, arm brushing, and other motions like that can be classified as

When two individuals gaze into one other’s eyes at the same moment, this is called eye contact.

It is a nonverbal type of communication that has a significant impact on social behavior. (7) Sustained eye contact, without the need to look at other people’s eyes, indicates that a person’s attention and interest is focused on the subject in front of them.

universal, it is argued that people derive meaning from facial expressions by relating them to the context in which they occur, including both verbal and nonverbal behaviors. (3) Recurring criticism is that the terms “verbal” and “nonverbal” are themselves outdated misleading and obsolete and no longer useful. It makes no sense to speak of verbal communication and ‘nonverbal communication because there is only communication. (4)

Body language.

Body language is a broad term that refers to forms of communication that use body movements or gestures in place of or in addition to other forms of communication.

Proxemics.

Is a type of nonverbal communication. Personal space norms differ significantly from those of other nonverbal communication channels such as facial expressions. The distance between two people conversing can reveal a lot about their relationship or the context in which they are having a conversation. (6)

Context.

Meanings of certain facial expressions are

1 Mark L. Knapp & John A. Daly. Handbook of interpersonal communication, 2002 2 Elfenbein and Ambady, On the universality and cultural specificity of emotion recognition, 2002 3 S. E. Jones & C. Lebaron. Journal of Communication, Research on the Relationship Between Verbal and Nonverbal Communication: Emerging Integrations, 2002 4 A. Kendon. American Journal of Psychology, Review of the book Kinesics and context by Ray Birdwhistell, 1972 5 businessjargons.com, Definition of Verbal Communication, 2019, received 26.03.22 6 umn.edu, Nonverbal Communication in Context, received 26.03.22 7 psychologistanywhereany- time.com, Non-Verbal Communication, 2005, reiceived 26.03.22

33

34

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

“Motivation arises from the interaction of three elements: Autonomy, Mastery and Purpose.”

(Brian Burke, Research Vice President at Gartner, 2014)

Machine autonomy.

Quick excourse: Machine autonomy can be understood as “the ability of a computer to follow a complex algorithm in response to environmental inputs, independently of real-time human input”.

(7) More advanced forms of machine autonomy typically depend upon the use of AI.

Motivation.

Where can you buy it?

Motivation arises from the interaction of three elements: Autonomy, Mastery and Purpose.

Autonomy.

Autonomy is linked to the independence of practical reason and the freedom that reason grants persons to govern themselves independently of obedience to others, including the state. (5) Immanuel Kant’s concept of personal autonomy as rational self-government has, in turn, become increasingly expanded beyond adherence to universal law to include governing yourself according to your “own” authentic desires and impulses.

(1)

of control over how our choices are realized, and not being subject to excessive oversight or control in their pursuit, is also important for our sense of autonomy. (3), (4) We can increase a person’s autonomy by helping them to make more authentic choices, both in the sense of more choices that are authentic and choices that are more authentic.

Job autonomy.

In a study done in China during COVID–19, thirteen participants suggested the role of job autonomy to be important in remote working. Job autonomy means employees can decide when and how to accomplish their tasks. Individuals’ „In effective gamified solutions, players opt in to participate, and once they do, they make choices performance and well-being benefit from job autonomy, as those with higher job autonomy about how they will proceed through the chalcan balance work and rest and choose the most lenges to achieve their goals. Players are given the opportunity to discover and learn using differ- productive ways to do their work. Job autonomy was also identified as beneficial for work-family ent paths through the solution.“ (2) balance. For example, participant 14 stated: “I can control the rhythms of work and rest. If it is Autonomy can also be diminished and empownot during the meeting, I can have a short break, ered through the quality of the choices availaround ten to thirty minutes, and then continue able to us. When we are given more control over to work. That also means more time to spend important aspects of our lives and access to a diverse range of meaningful choices, then our au- time with my family.” (6) tonomy is increased. Further, having some degree

Mastery. „We all have a deep-seated need to improve in aspects of our lives, but often we lack the motivation to take the first step.” Brian Burke states ins his book “Gamify: How Gamification Motivates People to Do Extraordinary Things”. (2) Gamification offers positive feedback and simple onboarding, which can inspire users to improve their performance within that area. However, mastery is a journey, not a destination. There are several progress markers along the journey, but there is never a finish line. There is always another level in almost all of life’s endeavors, whether it is running, painting, or learning a language. (2)

Purpose.

A purpose is the reason why work is done or why a specific goal is intended to be reached.

Gamified solutions, by definition, differ from typical games in terms of their goal. Gamification is used to achieve one or more of three goals: changing habits, increasing skills, or stimulating innovation. Gamification must begin and end with a goal in mind: to help players achieve meaningful objectives. (8)

1 Onora O’Neill. A Question of Trust: The Bbc Reith Lectures, 2002 2 B. Burke. Gamify: How Gamification Motivates People to Do Extra-ordinary Things, 2014 3 Richard M. Ryan & Edward L. Deci. Self-Regulation and the Problem of Human Autonomy: Does Psychology Need Choice, Self-Determination, and Will?, 2006 4 R. M. Ryan & E. L. Deci. Self-determination theory: Basic Psychological Needs in Motivation Development and Wellness, 2017 5 J. B. Schneewind. The Invention of Autonomy: A History of Modern Moral Philosophy, 1998 6 (B. Wang, Y. Liu, J. Qian & S.K. Parker. Applied Psychology, Achieving effective remote working during the COVID-19 pandemic: A work design perspective, 2021) 7 AI assisted ethics Etzioni & Etzioni, 2016 8 Melinda Jacobs. Presentation at Gamification Europe Gamification & UX: The Big Picture, 2019

35

36

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

trigger (noun) an event or situation, etc. that causes something to start.

(Cambridge Dictionary)

Gamification.

Triggers peoples emotions Gamification is not a gimmick — it works because it triggers powerful emotions. By using game mechanics in a non-game context, gamification makes tasks more fun and engaging.

Gamification is not about having fun, despite what it may appear. Many people assume that by adding points and badges to activities, we can make them more fun, just like in a game. (2)

The sole purpose of games is to entertain the players. To accomplish this, video games employ elaborate storylines, graphics, and animation to provide players with realistic experiences. The goal is to immerse players in the game world and the role they play within it.

Customers with higher value and repeat transactions are rewarded through rewards programs, while employees are rewarded for meeting goals.

Airline, hotel, and retailer loyalty programs are the most common, but there are also employee incentive programs and other types.

Gamification has three primary goals: it changes behaviors, it develops skills, and it drives innovation. People need to be reminded to change their behavior by a trigger until it becomes part of their routine. Breaking down the change into small, manageable steps is the key to achieving larger goals.

Over time, complexity must be built. When something is done repeatedly, it develops into a new habit.

By providing a clear transformation route with simple milestones and encouragement along At the moment, there are dozens of gamified ap- the way, gamification can play an essential role plications available to change personal behaviors, in implementing change. Goal-setting, triggers, but they are often simple reminder systems with and baby steps are all used in gamification to some points and badges slapped on. In order to assist people change their behaviour. With social reinforce motivation with social acknowledgment, sharing, players can meet like-minded people more successful systems integrate with social and enlist the help of their friends. Gamification networks. Gamified apps to help people lose encourages people to repeat activities until they weight, quit smoking, improve fitness, correct become second nature, to keep the process inposture, manage personal money, take medicateresting, and to evolve through time. tion, and boost memory are just a few examples.

Researchers distinguish intrinsic and extrinsic motivation. With intrinsic motivation, we do things because we find them inherently satisfying and interesting. With extrinsic motivation, we do things to achieve rewards, and that is the problem with rewards. They massively undermine intrinsic motivation. Numerous studies showing that extrinsic rewardare not sufficient to sustain engagement, and sometimes have the opposite effect. Extrinsic rewards can deliver a short-term boost — just as a jolt of caffeine can keep you cranking a few more hours. But the effect wears off — and worse, can reduce a person’s long-term motivation. (2)

characters exist and the real-world is not affected. Games offer an opportunity to alter elements in this world in a playful way. Games can also help to simplify complex situations.

Gamification is about engaging people on an emotional level and motivating them to achieve their goals. “Pain Squad” is an iPhone app designed to collect daily information about children’s pain levels. The app includes a progression structure, so that when the children complete their pain report three days in a row, they progress from rookie to sergeant and through ranks until they become a chief. (2)

Gamification does not work.

When gamification appears to work, it is because the underlying experience is already a game. (2) So why design a product or service like a game?

Games are extraordinarily successful at creating engagement, triggering states of flow and a decision to design experiences based on a narrative, much like we would design a game.

The magic of games creates a different world where other rules, goals, stories, mechanics, and

Educational presentation: Gamification Europe - Gamification & UX: The Big Picture, 2019 2 B. Burke. Gamify: How Gamification Motivates People to Do Extra-ordinary Things, 2014

37

38

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

Elements of Gamification and Game Design.

Game design principles have been translated into other application areas, such as products and services. These gamification elements have a lot in common with game design, but they’ve been adjusted and tailored to these environments.

Robust controls and consistent rules.

Immediate, frequent and clear feedback.

Players do not have to think about what buttons to press or how to carry out a particular action when good game design mechanics are in place.

This clearly links actions to consequences, which makes learning easier and faster.

The element of uncertainty.

Make the next action obvious.

Knowing what to do next keeps you in the present moment and in a state of Flow.

This is the neurological gold mine: we keep going because we want to find out what comes next.

Experience bars.

Windows of enhanced attention.

These measure our incremental progress and help us visualize how far we’ve come and how far we have to go.

Gameplay helps boost memory and confidence. Game players are more willing to take risks and pursue difficult tasks.

Multiple long- and short-term goals.

Other people.

These are our main quests and side quests, which are broken down into smaller tasks.

Gameplay gives us a chance to connect with multiple players online.

(1)

Rewards for effort.

Every bit of effort exerted in a game is rewarded through credit, such as gold, loot and experience.

Working hard in the kitchen made fun with good game design seen in “Overcooked” Image source: Ghost Town Games

1 Melinda Jacobs. Presentation at Gamification Europe Gamification & UX: The Big Picture, 2019

39

2021 / 2022

40

masterthesis

extended interaction space

theory and research

felix schwamkrug

to be in the zone (phrase) Being completely unaware of what’s going on around you as you are so extremely into what’s going on right in front of your face.

Flow.

(Urban Dictionary)

Getting work done Total concentration and almost obsessive dedication to one subject. It is even possible that one does not notice changes in the environment or other sensual stimuli while in that state. This describes being “in the zone”; also known as “Flow state”.

better Focus music by ensuring that distracting moments do not appear. (2) ai

After testing the effectiveness of Brain.fm, it reminds me of the Shepard tone. Instead of creating the auditory illusion of a tone that seems to constantly ascend or descend in pitch but never Because this is such an unusual and counterintu- gets any higher or lower, it appears to be playing itive state, it is a very fragile mental and physical an ever-evolving music track that never changes state. The less exciting or dangerous the subject, and (almost never) ends. It worked remarkably the more important factors such as the appropri- well for me. ate amount of background noise, a comfortable temperature, adequate lighting, fulfilled basic needs, stress, and comfort. Humans have not evolved to be engrossed in a task without noThe Pomodoro Technique is a widely known time ticing a saber-toothed tiger approaching from management method that encourages people behind. One exception would be if their lives to work with rather than against the clock. You depended on them concentrating on the task at divide your workday into 25-minute pieces interhand, which would not be running away from the rupted by five-minute intervals using this stratetiger. gy. Pomodoros are the name for these intervals.

You take a longer rest of about 15 to 20 minutes To many people the experience of extreme focus after approximately four Pomodoros. (3) is not a familiar feeling. For some this manifests The strategy works by instilling a sense of urgenas tingling sensations, or pleasant light headedcy through the use of a timer. Instead of feeling ness, or a feeling of being dissociated with your like you have endless time in the workday to get environment. (1) things done and then waste those crucial work hours on distractions, you know you only have 25 Sounds can grab our attention, even when we do minutes to make as much progress on a task as not want them to. If we can understand and prepossible. (4) dict this effect (‘auditory salience’), we can make

high

anxiety

l

nne

cha w o fl

challenge

boredom

Pomodoro Technique.

low low

skill

high

Flow is when you are fully immersed and everything feels effortless. A flow state is the sweet spot between being challenged and being competent.

1 Brain.fm, How to use Brain.fm for Focus 2 Brain.fm, Theory & Process (whitepaper) 3 francescocirillo.com, The Pomodoro Technique, received 24.03.22 4 Kat Boogaard. themuse.com, Take It From Someone Who Hates Productivity Hacks - the Pomodoro Technique Actually Works, received 06.04.22

41

42

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

A selection of apps and functions that can help you work in a flow state.

Characteristics of a flow state: ¬ Intense and focuses concentration on the present ¬ Merging of action and awareness ¬ Loss of reflective self-consciousness ¬ Sense of personal control ¬ Distortion f temporal experience

Awork combines time tracking and invoice creation with a calendar and task manager for remote workers.

Image source: Awork

Planting virtual trees when not using the phone, Forest App

Conditions for flow:

Image source: Abby Jewett, The app that helped me write a book, 2019

¬ Knowing what to do next ¬ Knowing how to do it ¬ Freedom from distractions ¬ Clear and immediate feedback ¬ High perceived challenges and high perceived skills Many of the previously discussed items must be fulfilled in order to achieve this “flow start” and to be “in the zone”. Elements or conditions for a good flow state coincide with elements from the field of gaming design defining how to make games entertaining and immersive. The balance between skill and challenge is essential so that neither boredom nor overwhelming demands arise. The ultimate killer of the flow state is distraction. As a result, these must be avoided at all costs. Noises, distractions, as well as other parallel responsibilities obstruct the flow.

Apple’s “Do Not Disturb”-mode syncs across devices

Blocking distracting apps and websites with Freedom, Freedom’s Blocked Page

Image source: Apple

Image source: Own screenshot

43

44

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

“Tasty food, sex, exercising and successful social connections can release dopamine.”

(Krach, Paulus, Bodden, & Kircher 2001)

Denying access to beloved apps will irritate your limbic system but satisfy your prefrontal cortex. (freedom.to, rephrased)

Psychology.

Excessive use of technology We are becoming true junkies thanks to cellphones and the social media platforms that they promote says Trevor Haynes in an Science in the news article. While it is easy to dismiss this assertion as hyperbole, social media networks like as Facebook, Snapchat, and Instagram exploit the same neurological circuits that slot machines and cocaine use to keep people using their services as much as possible.

Smartphones have given us access to an almost limitless source of positive and negative social stimuli. Whether it is a text message, an Instagram “like,” or a Facebook notification, every notification has the potential to be a positive social stimulus and dopamine flood.

While cellphones themselves are not inherently addictive, the hyper-social surroundings they create are the main drivers of our addictions to them. Smartphones allow us to carry vast social environments in our pockets through every waking moment of our life, thanks to apps like Facebook, Snapchat, Instagram, and others.

Dopamine is a neurotransmitter produced by our brains that plays a key function in behavior motivation. It is released when we take a bite of tasty food, have sex, exercise, and, most significantly, have successful social connections. It rewards us for favorable behavior and pushes us to repeat them in an evolutionary environment. (1)

If you’ve been a Facebook user for a few years, you’ve definitely noticed that the site’s notification criteria has been expanding. When you first join Facebook, your notification center is centered on the first set of connections you create, establishing the critical link between notification and social reward. That notification center will grow more active as you use Facebook more and interact with various groups, events, and artists.

After a while, you will be able to launch the app whenever you want and expect to get rewarded.

When you combine that with the low cost of looking at your phone, you’ve got a pretty good reason to check in as often as you can. (2)

When a reaction to a stimulus results in a reward, a process known as long-term potentiation strengthens these linkages. By raising the intensity with which neurons respond to specific

stimuli, this process builds commonly used connections between brain cells called neurons.

Smartphones and social media apps are here to stay, therefore it is up to us, the users, to decide how much of our time we want to devote to them.

(3) Companies like Facebook will continue to do

everything they can to keep your eyes riveted to the screen as much as possible unless the adbased profit model changes. They stack the deck — and our brains — against us by utilizing algorithms to exploit our dopamine-driven reward system.

Trevor Haynes continues end the interview by saying that there are a variety of techniques if you want to spend less time on your phone.”

Disabling social networking app notifications and keeping your phone’s display in black and white will diminish your phone’s capacity to capture and maintain your attention. Above all, the best tool you have is to use technology with care. So think about it the next time you pick up your phone to check Facebook: “Is this really worth my time?”

1 S. Krach, F.M. Paulus, M. Bodden & T. Kircher. Frontiers, The Rewarding Nature Of Social Interactions, 2001 2 Trevor Haynes, Science in the news, 2018 3 linkedin.com, Dopamine and Social Media - Unrealised Truth in this Era, 2021, received 06.04.22

45

46

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

“If you’re not paying for the product, you are the product.”

(Krach, Paulus, Bodden, & Kircher 2001)

This widely known statement becoming popuplatforms — are only as valuable as their ability lar in Netflix documentary “The Social Dilemma” to predict the user’s next move and their ability from 2020, is not new thinking. Richard Serra and of presenting advertiser’s content in an elegant Carlota Fay Schoolman, artists, broadcasted a manner. short video titled “Television Delivers People” as We were always able to adapt to these new things early as 1973, wherein they say, “It is the conin our lives that craved our attention before, with sumer who is consumed.” and “You are a product print, television, the internet, and smartphones of television.” (1) In today’s social media envi— what is different now is that we cannot keep up ronments, advertisements have evolved from a with technology that transcends our intelligence moving billboard on which you can learn about and ability to differentiate the content. a product to something more aggressive and inconspicuous.

There are now apps that block other apps in orThis is more true today than ever before. “It’s the der to increase productivity. One of the most popgraual imperceptible change in your behaviour ular, while also being cross-platform, is Freedom. that is the product.” (2), according to Jaron It is an app that limits Internet access for up to Lanier, a pioneering VR computer scientist, artist, eight hours at a time. The way to get back online and author. As a result, advertisements became is inconvenient enough to act as a deterrent. Fred more embedded in entertainment content as well Stutzman, a Graduate student in information and as the products themselves. Millions of dollars library science, created the application after his have been spent on research into how to get usown failsafe self-binding approach was to write in ers’ attention and change their behavior, which a café without an Internet connection until that is then sold to advertisers. Today, they’ve gotten location set up free Wi-Fi. people to enjoy watching ads by making them extremely targeted, entertaining, or bizarre, to the 1 Richard Serra “Television Delivers People” (1973), Authors of point where they do not even realize they are dovideo: Richard Serra and Carlota Fay Schoolman, uploaded Feb. 2011 ing so. The companies that collect personal data about you — social networks and entertainment 2 Jaron Lanier. Netflix, The Social Dilemma, Director: Jeff Orlowski, 2020

47

48

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

“An average person taps their phone 2617 times a day, unlocks it 150 times per day, and spends up to 3.7 hours on it.”

smart device (term) Is a context-aware electronic device capable of performing autonomous computing and connecting to other devices […] for data exchange.

(Source INC., Research by Dscout 2016)

(What Is a Smart Device? (3))

Smartphones.

The bicycle of our minds (4)

Smart.

As already mentioned in the introduction, the modern smartphone is much more than just a phone. In just 10 years, the usage of smartphones So what is so smart about smartphones? and mobile devices has grown to an incredible “Smart” is used as a prefix or as a standalone magnitude. Every month, new and better devices term to claim a functional, classificatory techare released, making it difficult to keep up as a nical property. It is also used as a buzzword, or consumer as well as a digital designer. Screens as a supposedly upvaluing feature. The conclucan be found almost everywhere, and they help sion of extensive research done by an engineerus live more productive lives with more opporing team in 2018 states that “A smart device is tunities. Screens, on the other hand, are not the a context-aware electronic device capable of final chapter on this technology-fueled story. performing autonomous computing and connectHumans have evolved to crave efficiency and ing to other devices wire or wirelessly for data convenience more than ever before. We expect exchange”. (1) things to be delivered immediately, and our patience for communication, commerce, and enter- According to this definition, a smart device has tainment has dwindled to almost nothing. three main characteristics: context awareness, autonomous computing, and connectivity. The Team’s Silverio explains an article. This definition conformes to the IoT’s main concept, which is that any “thing” can be a part of it. If we add a sensor, a small amount of computing power, and network connectivity to a chair, it can become a smart chair. What about the user’s participation?

A smart device can interact with humans directly or indirectly. A weather probe, for example, could collect weather data and send it to the Internet of Things. Of course, humans will make use of the data, but the weather probe did not require any direct human interaction. (2)

Smart Devices.

The requirement to say a wake-up command that cannot be changed before every command makes using voice assistants unintuitive.

Interpersonal communication is almost never structured in this way. Surprisingly, it reminds me of a relatively new way of interacting with people who spend their days wearing Bluetooth headphones in their ears. If you want to talk to them, you will have to do some sort of wake-up command or gesture to get them to put their headphones out. We could say that instead of changing voice assistants to be more natural companions, we changed our behavior to be more like robots. It is a fine line between how long voice assistants wait after you stop speaking — too short, and the rest of the question is not relayed; too long, and you have to wait unnecessarily for the answer.

1 M. Silverio-Fernández, S. Renukappa & S. Suresh. What Is a Smart Device? - a Conceptualisation Within the Paradigm Of the Internet Of Things - Visualization In Engineering, 2018 2 M. Silverio-Fernández. towardsdatascience.com, What Is a Smart Device? — The Key Concept Of the Internet Of Things, 2021, received 12.04.22 3 M. Silverio-Fernández, S. Renukappa & S. Suresh. What Is a Smart Device? - a Conceptualisation Within the Paradigm Of the Internet Of Things - Visualization In Engineering, 2018 4 based on: Steve Jobs. interview with WGBH, 1990

49

50

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

pattern (noun)

“If people don’t understand the context for the work they’re doing, the work loses some meaning.”

the regular way in which something happens or is done.

(Oxford Dictionary)

(Lawrence Coburn, Co-founder and CEO of Twine)

Screens.

From new to nuisance The screens of our mobile devices eventually MacBook product lines at the moment. Some have become another roadblock in our nevconfigurations are so similar that the main interer-ending quest for efficiency. (1) When you are action methods for manipulating content within always in a hurry, taking the extra step of tapping the screens are the most distinguishing features. an icon on your phone and then continuing with iPads have touchscreens, whereas Macbooks multiple processes to get to the end result feels have keyboards and trackpads. daunting. The days of having to wait for letters to come in the mail, for your crush to call you on your corded home phone, and for dial-up Internet to manifest are gone. We want it right now, and new The term Zero UI refers to a paradigm in which technology can make it happen. our movements, voices, glances, and even When it comes to creating digital experiences, thoughts can all cause systems to respond to us context is crucial. We investigate and investigate through our surroundings. At its most extreme, the role of technology in the user’s daily life, as Zero UI refers to a screen-free, invisible user inwell as the impact of the user’s environment on terface in which natural gestures initiate interacour digital designs. We must now think beyond tions, as if the user were conversing with another the digital. A broad environmental context is reperson. (4), (5) quired for modern strategies. (2)

Zero UI.

There are several trends in the technological However, screens have a place today. Laptops, world moving away from screens as the main tablets, monitors, TVs, and specialized devicform of interaction all together. Zero UI removes es such as digital whiteboards are becoming the traditional graphic user interface (GUI) from more popular — as well as larger in size. The the equation and uses nature interactions of hapscreen quality is the most important aspect of tic feedback, context aware, ambient, gestures the hardware setup if you want to display a lot of and voice recognition. information and media such as video, photography, graphics, and print. It makes little difference What is displayed on the screen is still importwhether it is a touchscreen or not for this purpose. ant, but designing for the stages leading up to Apple is blurring the lines between its iPad and

or outside of a screen is just as important. A task can be started with a facial expression, a wave of the hand, or a bodily gesture. Saying “Alexa”, “Hey Siri” or “OK Google” activates a fresh set of instructions. Telemetry sensors are used in smart homes to turn on lights when you come into a room and turn them off when you depart. New automobiles will open the door and start the car while your keys are out of sight, and then drive down the road without your hands on the steering wheel. (2) The idea of controlling electronic devices by pure thought might sound more futuristic than it is actually feasible today. Since we live in a time where enough data is collected about us in the form of preferences, likes, behavioral patterns, retention times, and histories while consuming content, the back-end systems responsible for serving content are already capable of knowing us well enough, even if not our exact thoughts.

Further, why press a button at all when you can simply call out the name of the channel you want to watch? Or, instead of having to speak the channel you want, maybe your TV automatically flips to the Bears game because you watch them play every Sunday.

1 Creative Honey. Post-Screen Design: The Future of Designers in Tech 2 Jim Combs. How UX-Designers can prepare for a world with no screens, 2017 4 Andy Goodman’s definition, fjordnet.com, 2015 5 t-3.com, Thinking how UX can prepare for a world with no screens, received 02.04.22

51

52

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

Zero UI (term) Refers to a paradigm where our movements, voice, glances, and even thoughts can all cause systems to respond to us through our environment.

“Stop trying to make air gestures happen.”

(UX Alpaca, published on Medium, 2019)

(Andy Goodman from Fjordnet) (4)

Interacting with tech

Apples AirPods. Remove one AirPod, and music playback switches to mono. Remove both, and playback stops and resumes when the user puts them back in.

… and tech interacting with us.

Many experts and scientists in the fields of interaction design and science differentiate between different kinds of interactions when it comes to controlling and receiving information by technology apart from screens.

Experiments and studies of interacting with screens in public spaces has been done and came to the conclusion that despite the fact that users had trouble understanding how the gestures worked, the results show a strong preference for gestures over voice commands. (1) Another study co-pared. (2)

By Voice.

Since the early 1970s, speech recognition has been a cornerstone of science fiction, and it has now made its way into the mainstream with the inclusion of voice recognition and search in both Context-aware devices and apps that eliminate the need for further interactions improve digital iOS and Android phones. Users will emphasize and physical experiences by anticipating user syllables, words, and phrases in the hopes that desires. Personalization enables Zero UI devices the system would recognize their questions and and apps to be proactive and anticipatory. (3) requests. Additional user research is required when designing the conversational UI to deterDomino’s Zero Click App is a working example mine how users will phrase and construct their of such an interaction in the consumer market. It inquiries or remarks. The necessity to adapt the works on the premise that if you launch the app, algorithm to regional dialects and slang is one of you want pizza delivered. It requires some setup the more difficult aspects of voice recognition.

When all the many ordering options are examined, and learning about the user’s preferences and dislikes, but by doing so, it is very quick and effisomething as easy as ordering a pizza can becient and without any extra processes. (4) come a difficult design problem. Understanding the user’s desires and motivation, or motive, aids Another example involving smart hardware are in phrasing. (3)

Nest Thermostats, rely on collecting data on usage and interactions to adapt to the user’s anticipated needs. We can build contextual experiences that become implicit rather than explicit interactions by exploiting sensors within a device or positional data. What is the user’s current circumstance or context? What is the device’s application context? It is necessary to develop and construct backdrop and temporal interactions.

Contextual Awareness.

Gestures.

Gestures must be simple to learn and repeat. One of the challenges with gestures is getting systems to detect the physicality of the motions in three dimensions (wave, hover, and swipe). The system’s responsiveness to gesture inputs can cause issues if users use various gesture sets in an attempt to get the system to respond. There must be a balance between teaching the gestures and delivering feedback to the user when actions and tasks are completed correctly.

In the article on medium “stop trying to make air

1 J. Hakulinen, T. Heimonen, M. Turunen, T. Keskinen & T. Miettinen. Gesture and speech-based public display for cultural event exploration, in Proceedings of the Tilburg Gesture Research Meeting, 2013 2 A. E. Sarabadani Tafreshi, A. Soro, & G. Tröster. frontiersin. org, Automatic, Gestural, Voice, Positional, or Cross-Device Interaction? Comparing Interaction Methods To Indicate Topics Of Interest To Public Displays, 2018 3 R. Dharmik. intuz.com, Zero UI: Get Ready For The Upcoming Screen-Less Future, 2021, received 05.03.22 4 fastcompany.com, What is zero UI and why is it crucial to the future of design?, received 02.04.22

53

54

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

“The biggest constraint of the next 100 years of computing is the idea of metaphors.”

(Evan Spiegel, CEO of Snapchat)

gestures happen” “UX Alpaca” talks about those unintuitive hand gestures you see in movies and automobile media control systems.

However, air gestures are not completely useless.

She sees a market for it, and as with any technology, they can be useful beyond playing with it once when you set up your new phone. Just like it happened with augmented reality (AR), there are apps where it adds value to the users (like fitting IKEA furniture in your home, or showing the constellations when you point your camera at the sky). Simply because it is technically possible, new technology does not have to be applied to everything.

Google’s Project Soli makes “your hands the only interface you need” by using radar to detect fine movements. (It is a more refined approach to conventional ‘rough’ air gestures because it senses movement very precisely. UI layers and interactions should not overextend the user’s motion away from the body as this can be fatiguing. Make gestures small and natural. Currently, it finds its place in Google Nest products like the Thermostat and Hub. Built in, it senses a human approach and hand gestures. I will concentrate more on Google Soli later on.

increased by keeping haptics to a minimum and avoiding overuse.

Glanceability & Ambiance.

With no need to open apps or read notifications, ambient devices operate on the principle of glanceability. Like a wall clock or a single-day calendar, a single glance should provide the user with all of the necessary information or context.

The “Nabaztag Rabbit” was a companion style device that combined colored lights and changing positions of the rabbit’s ears to provide glanceable experiences. (3) The “Chumby”, a smart screen from 2006, used widgets to provide a snapshot of data. Ambient devices are designed to create a seamless transition between physical and digital spaces. These are interconnected, browserless interactions. (3) Both of these devices were short-lived due to their inability to adapt to changing technologies and user information requirements though. (3) New iterations would be the Google Hub or Amazon Echo Show.

When it comes to the tools we use every day, people will always choose the path of least resistance, prioritizing utility and pragmatism over all else — this is something to keep in mind when creating Haptics, or kinesthetic communication, is a form gestures and other novel forms of interaction. of motion or vibration-based feedback the user can feel. In our current technological environment, vibrations are the most widely used haptic feedback. From game controllers to phone notifications and haptic feedback on touch interfaces, there are a variety of ways to experience haptic feedback. The value of that feedback for users is

Haptics.

The Moto-Cross video from Sega included haptic feedback via the controller to enhance the experience of colliding with other players on screen.

With the “Rumble Pak” accessory, Nintendo brought haptics to home consoles. When we interact with small touch screens today, we frequently experience haptics. Fitness trackers and smartwatches currently use haptics to provide notifications to the wearer. Haptic feedback is being applied to clothing, and virtual reality (VR) and Microsoft Kinect-style gaming (6) will benefit from ultrasound or ultrahaptics.

of Snapchat, Evan Spiegel, said that it is hard for us to elevate technology and how we interact with it because we are thinking too much about how we have started to technologize it. He had been thinking about digitally replicating the ways we talk in real life, and materialized many of those thoughts in the app.

Besides of the well-known self-destructing messages — he changed the way users call their friends. Instead of allowing you to ring friends for a video chat, as with FaceTime or Skype, Snapchat forces both users to be present inside a chat window before video can begin. The “Hey, want to chat?” text replaces the ring entirely. (7)

The touch of a touchscreen can also represent a kind of haptic input method. If the touch itself, which the touch surface can register, is recognized as an action or reaction to the system, it can even transmit information in this way.

Object manipulation.

Sensors enable technology to detect the movement of non-technological objects. Toniebox, for example, is a user-friendly, wireless, portable audio player that can import audio stories, music, and other content from Tonie audio characters with NFC capabilities. Choose an audio character, or Tonie, and place it on the Toniebox to begin your audio adventure. (4) It takes a fresh approach to storytelling and allows users to interact with the content. (5)

2 Project Soli. google.com, received 06.04.22 3 R. Dharmik. intuz.com, Zero UI: Get Ready For The Upcoming Screen-Less Future, 2021, received 05.03.22 4 whichplm.com, Press release tonies® (Boxine), FlexPLM, 2021, received 30.03.22 5 red-dot.org, Toniebox, 2016, received 30.03.22 6 3D camera or LiDar assisted movement detection setups

In an interview with The Verge, CEO and founder

7 Evan Spiegel. theverge.com, Real talk: the new Snapchat brilliantly mixes video and texting, 2014, received 03.04.22

55

56

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

Sensors.

What they are and why they exist

Touchscreen.

Bluetooth Low Energy.

The main screen naturally comes first when it comes to glanceability, or the delivery of information that can be grasped quickly. There are also a few models with secondary screens for quick access to information. Some models also have a status LED that flashes and changes colors to indicate the presence of notifications.

Bluetooth Low Energy (BLE) beacons are mounted on objects, walls, ceilings, and other surfaces transmit radio signals at certain intervals, allowing for indoor navigation. After determining that two objects are close to each other, Bluetooth location services can estimate the distance between them using the Received Signal Strength Indicator (RSSI). In particular, the new direction-finding capability in Bluetooth 5.1, enhances precision to the centimeter level. The antennas detect the angle at which the signal is received to determine the direction when locating the BLE-Tag. (1)

A phone’s screen not only detects when it is touched, but it also knows where it has been touched — and, in recent years, it has been able to tell how many fingers are touching and how strongly.

Vibration.

The smartphone has a vibration mechanism, which can be either a motor or an electromagnet, in order to provide haptic feedback on interactions.

Simulating different buttons with different amplitudes and frequencies of vibrations, as well as reacting to pressing a virtual two-element on the screen with different strengths, has made progress in the last years.

Wireless networks.

A cellular network or mobile network is a type of wireless connection facilitated by cellular towers.

Bluetooth is a wireless technology that allows devices to exchange information over short distances. (4) Wi-Fi is similar to Bluetooth in that it also uses radio waves for high-speed data transfer over short distances without the need for a wire connection. Bluetooth and Wi-Fi differ in terms of their purpose, capabilities, and other factors. Bluetooth allows for short-range data transfer between devices. As an example, it is commonly employed in headsets for mobile phones, enabling hands-free phone use. Wi-Fi, on the other hand, allows devices to connect to the Internet. (5)

RFID & NFC. RFID (radio frequency identification) technology has been around for decades, and NFC (Near Field Communication) is an evolution of it. NFC is now found on nearly all smartphones and wearables. The main difference between RFID and NFC is their transmission ranges; the former is frequently used over greater distances. Some regions, for example, use RFID to collect road tolls automatically. Toll tags are usually attached to the windshields of vehicles, and all you have to do is drive through the toll booth.

The convenience of NFC extends to devices without a screen. It is used by a lot of wireless speakers and headphones to exchange pairing information with your phone.

NFC technology is currently used in data transfer, mobile payments, quick pairing, public transport access, gaming, home automation and other more niche areas.

UWB. Similar to BLE, UWB (ultra wideband) technology has the ability to transmit info at low energy usage — but across a wide radio bandwidth, from 500 MHz (megahertz) to several gigahertz. That gives it a short range, compared to Wi-Fi. It even tried to compete with Wi-Fi back in the days, but it main purpose today is locating and measuring things. (7) It has security extensions built in, allowing it to be used for sensitive data transmission and other high-security applications.

57

58

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

A real-world possibility for UWB includes using it in smart locks for passive, key-less entry, which means the tech can tell not only if you are approaching, but also if you are inside or outside the building or car.

Apples’ UWB tag compared to a coin, photograph and x-ray.

Image source: iFixit

While RFID, NFC, BLE, and UWB may appear to be similar, there are significant differences in technology and capabilities, and each has its own set of applications in the real-world.

Prositive thing is that UWB can deliver a much more precice location because it uses more frequency bands, the negative is that you must broadcast a signal, so UWB tags need energy compared to passive UHF RFID tags.

GPS. Global Positioning System (GPS) is locating on a different scale than the technologies mentioned above. It is a sensor with antennas to help navigation. It receives continuous signals from satellites that help calculate the distance traveled and the location of your phone.

When a signal is received, the GPS sensor records a location. Depending on the time difference between any two signals, the distance is calculated. Navigation apps use both GPS and magnetometer to identify location and direction. (6) As the name suggests, GPS is used globally and focuses on the outdoors, while Bluetooth is more useful indoors because of its directional capabilities and higher accuracy.

Cameras.

Cameras consist of a lens and an image sensor. An image sensor is a device that captures light and converts it into an image; when combined with an AI-enabled processor, the camera can classify objects without the need for human intervention. Certain Objects, signs and shapes could deliver information or trigger

predetermined events and settings. Hand and face gestures can also be identified in this way to control interactive elements and initiate processes.

LiDar.

Starting with the iPhone 12 Pro, Apple introduced a LiDAR scanner to its list of sensors. LiDAR, or Light Detection And Ranging, is similar to the ToF (time-of-flight) sensor, found in other devices, mainly to help with getting in-focus pictures. Instead of firing one laser, it fires multiple lasers for a more accurate 3D map of the environment. (3)

Radar.

Radar is a detection system that uses radio waves to determine object distance, angle, and velocity. (8) The 60 GHz (gigahertz) radar chip from Infineon is one of the most advanced and capable radar chips available. This sensor was first shown at the Google Developer Conference (Google I/O) in 2016, and it is now included in Google’s smartphones as “Soli”.

Soli, like other radar sensors, detects objects that reflect some of the energy back to the radar antenna by emitting electromagnetic waves in a broad beam. The data collected and analyzed includes, among other things, information about the object’s size, shape, and movement. This radar sensor can track sub-millimeter motion with extreme speed and precision.

Google Soli project uses radar expertise to enable new types of touchless human-machine interactions, such as those in which the human hand serves as a natural, intuitive interface for new devices and services. (3)

59

60

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

Proximity sensor.

Microphone.

Proximity sensors detect how close a certain object is to your phone. A quick example of this is your phone’s display turning off when you pick up and answer a call.

Microphones detect the sound waves created by the expansion and rarefication of the air and turn this into a sound we can hear through a speaker. (2)

Light.

Ports.

Ambient light sensors measure the intensity of light around the device. These sensors detect the changes in brightness of the surroundings and record its intensity.

The smartphone can be controlled via cable and control other devices or expand its functionalities via USB, jack, or special connectors such as Apple’s smart connector.

Larger screens and additional sensors and output devices are common examples.

Magnetometer.

Measures Earth’s magnetic field, typically in three directions, often as a compass.

Combined with GPS, and some smart interpretation algorithms, measuring the magnetic field can help you automatically know where you are heading West on the freeway and not East. (2)

Accelerometer.

An accelerometer is designed to measure non-gravitational acceleration.

The sensor could detect movement in different rooms. If you are wearing a wearable, your movements around the house could trigger lights or other devices, especially if they are connected via Bluetooth.

Gyroscope.

A gyroscope measures three dimensional rotation. Used in todays smartphones as a very sophisticated camera-shake correction: A gyroscope can detect your shaky hand and remove that exact motion to create a better picture.

In certain situations, the gyroscope can also be a better indicator for direction than a magnetic compass. (2)

Shock, Step sensor.

A sensor, similar to the accelerator, specific to detect step movements.

…and others.

In some phones there are — or were even more sensors. The remaining sensors are either obsolete and have been removed from current devices’ feature sets, or they are unrelated to the research topic and conceptualization though. (hygrometer (humidity sensor), barometer (air pressure sensor), thermometer, RGB light sensor, AM/FM radio, heart rate monitor)

1 Amos Kingatua. medium.com, Bluetooth Indoor Positioning and Tracking Solutions, 2020, received 16.03.22 2 Golden Krishna. The Best Interface Is No Interface/Designers Toolkit, 2021 3 smartphonedomain.com, Smartphone Sensors Explained, 2021, received 19.03.22 4 signalboosters.com, The Differences Ethernet, WiFi and Cellular, received 18.03.22 5 britannica.com, What’s the Difference Between Bluetooth and Wi-Fi?, received 19.03.22 6 techmende.com, What Is A Smart There So Many Of Them?, received 19.03.22 7 Eric Griffith. pcmag.com, Ultra-Wideband Explained, 2021, received 19.03.22 8 Nature Magazine, Radio detection and ranging, 1943 9 infineon.com, received 20.03.22

Enough to work with.

Following this list, it should be clear that modern smartphones, as well as the ecosystem in which they exist, are extremely capable devices that provide an ideal platform for experimenting with, testing, and establishing novel interaction methods. Of course, these sensors are interconnected by a processor that is capable of connecting data from multiple sensors and interpreting it in a way that is appropriate for the current application. If technologies like artificial intelligence and especially machine learning are used, the smartphone can even work with contradictory and changing values to arrive at the same conclusion.

61

62

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

Wearables. … and other cyborg modules “Wearables” refers to computer systems that are worn on the body and are designed to assist the wearer in their daily activities. They can be connected to the internet or wirelessly to a smartphone or tablet. (1)

are worn. Wearables, for example, can be woven into textiles and used as smart apparel. Google Glass, a wearable worn as visual data addition on the face was announced in 2012 with a lot of media attention and interesting features. (3)

Unlike other mobile computing systems, wearables’ primary functions include tracking via sensors, applications, hardware, and software, as well as mobile data processing. The data gathered by tracking is gathered from the user’s environment, behavior, and physiological condition, and is usually only displayed in broad strokes to the user. For a more complete presentation and review of the data, you will need a smartphone, tablet, or PC.

In last years the concept has spread to other sectors of application, which has been made possible by the devices’ increasing miniaturization and communication capabilities. (4)

Wearables have one thing in common: despite being worn consistently or frequently, they are usually hardly apparent and should provide the wearer with useful added value.

Smartwatches, data glasses, headsets, smart earbuds, technology tattoos, fitness trackers, running watches, smart jewelry, and other devices are classified based on their use and how they

Besides being able to collect data and acting as haptic feedback, microphone can help with communicating actions of users because of their nature being placed at a specific location on the body and therefore acting as specific interface for input or output of communication. For example can the pickup of the Bluetooth In-Ear headphones act as a trigger for music to start playing.

These triggered actions just have to be programmed and do not necessarily need additional hardware because the action of turning something on or getting into some wireless range itself carries the information of the intent and purpose.

The steps of opeening a smart doorlock of a screen based flow.

The steps with a sensor based flow – either using the smartphone or wearables.

1 Prof. Dr. Oliver Bendel. wirtschaftslexikon.gabler.de, Gabler Wirtschaftslexikon, Wearables, received 10.04.22 3 foerderland.de, Förderland: Business Magazin für Entscheider, Was sind Wearables?, received 10.04.22 4 Manfred Klein. healthcare-computing.de, Healthcare Computing, Was sind Wearables?, 2019, 10.04.22

63

64

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

Computing and intelligence.

Design and AI.

Artificial Intelligence (AI) AI Is the theory and development of systems that are able to perform tasks requiring intelligence. (5) Artificial intelligence, then, refers to the output of a computer. (3)

We are forgetful, fragile, and busy. Computers should do the things we do not want to do, that we do not know we should do, and that we are not able to do.

When to use AI in a service design context?

Machine Learning Is a form of AI that mimics how humans learn. (1), (2) The system learns to solve problems in ways that are not explicitly programmed. In machine learning, the learning process is supervised, and the programmer has to be specific when telling the computer what types of things it should be looking for. (3)

¬ People do not know how to do something ¬ People cannot do something ¬ The task is boring, repetitive or dangerous

When not to use automatic AI?

Deep Learning Is a brain-like logic structure for machine learning algorithms. (1), (2), (3) Each algorithm in the hierarchy performs a nonlinear transformation on its input before generating a statistical model as an output. Iterations continue until the output is accurate enough to be useful. The label “deep” was inspired by the number of processing layers that data must pass through. To achieve an acceptable level of accuracy, deep learning programs require access to immense amounts of training data and processing power, neither of which were easily available to programmers until the era of big data and cloud computing. (3)

1 levity.ai 2 ibm.com, AI vs. Machine Learning vs. Deep Learning vs.

Neural Networks

¬ People enjoy or feel responsible for the task ¬ High stakes situation ¬ Complicated personal preferences

(1)

At the moment, many machine learning services require you to send your data to the cloud in order to perform the actual intelligent analysis. Advanced AI necessitates computers performing a large number of small calculations quickly. Regular CPUs (central processing unit in a computer) are unsuitable for the demands of machine learning. As a result, GPUs (graphics processing units) are used — they are designed to perform small simultaneous calculations. (4) Other architectural changes must be made to increase the number of simultaneous tasks your chip can perform, so manufacturers have already begun to install dedicated AI chips, essentially specialized GPUs, by default on new devices. As a result, the above concerns are a thing of the past, and data-heavy and data-sensitive AI applications will be available on any new smartphone in the coming years.

3 techtarget.com, deep learning (definition), 2021, received 21.03.22 4 James Vincent. theverge.com, A brief guide to mobile AI chips, 2017, received 21.03.22 5 oxfordreference.com, artificial intelligence

65

2021 / 2022

66

masterthesis

extended interaction space

theory and research

felix schwamkrug

IoT (abbreviation) Internet of Things Objects with computing devices in them that are able to connect to each other and exchange data using the internet.

(Cambridge Dictionary)

APIs. You probably use them everyday An API (Application Programming Interface) is an “interface that is defined in terms of a set of functions and procedures, and enables a program to gain access to facilities within an application.” (3) Simply put, an API is an interface that allows software to interact with other software. This is part of its name and is core to its functionality.

APIs are held allowing an existing or new business capability available from across a network as an on-demand service for usage by other applications and software. (4) APIs helped transforming “the web from a page-centric collection of documents” to an “architecture of participation” in which users create and share data with the help of a network of Web applications and communication with each other through APIs.” Therefore, they deem APIs as the true building blocks of digital transformation. (4)

Weather Report snippets.

In today’s economy, they play a bigger role than ever before. They enable us to share important information and expose valuable functionality among devices, applications, and individuals.

APIs are everywhere, powering our lives from behind the scenes, even if we do not notice them.

Paying via PayPal.

Some of the most relatable examples of API usage in our everyday lives.

For example, if you search for your local weather conditions on Google, you will see a dedicated box at the top of the search results with a forecast. Google is not in the business of providing weather data, so they source this information from a third party.

Log–In using Facebook, Google or others.

Instead of actually logging-in to users’ social media accounts, applications with this functionality leverage these platforms’ APIs to authenticate the user with each login.

When the user clicks the “Pay with PayPal” button, the application sends an “order” request to the PayPal API, specifying the amount owed and other important details. Then, a pop-up authenticates the user and confirms their purchase.

Finally, if everything goes to plan, the API sends confirmation of payment back to the application.

Twine for Zoom.

Twine is an example of an API that facilitates communication in the area of remote work and can be integrated into the existing infrastructures of the Zoom video communication service.

Twine for Zoom is a current and relevant example of implementing functions through the use of APIs. It is one of a few new breakout room-focused apps that will be added to the Zoom App Marketplace in the coming weeks. Zoom recently granted API access to a select few companies, allowing them to build platforms on top of its breakout room technology.

Twine for Zoom will give businesses the tools they need to create social breakouts strategically at all-hands meetings, company socials, onboarding events, and other virtual gatherings.

Virtual tables with specific conversation topics can be created by company leaders. The tables are visually represented as a sidebar within Zoom, and employees can see who is seated at each table and what the topic is before selecting one to join and changing at any time. (6)

Online travel agencies to compare flight prices.

APIs are great for travel services since they make it easy for machines to quickly and autonomously exchange both data and requests — in this case, trip availabilities and reservation requests. (5)

Joining the “Breakout Room” in Twine for Zoom Image source: Twine for Zoom

3 Dictionary of Computer Science – Oxford Quick Reference, 2016 4 Jukka Huhtamäki, Rahul C. Basole, Kaisa Still, Martha G. Russell & Marko Seppänen. Visualizing the geography of platform boundary resources: the case of the global API ecosystem, Proceedings of the 50th HI International Conference on System Sciences, 2017 5 nordica-pis.com, Five examples of apis we use in our everyday lives, 2019, received 02.04.22 6 inc.com, Zoom’s Breakout Room Feature Is About to Get a Lot Smarter Article, received 22.02.22

67

masterthesis

2021 / 2022

68

extended interaction space

felix schwamkrug

theory and research

MagSafe

Apps

Apple iPhone 12

Apples first iPhone

PowerSharing

Multi-Touch interface

Samsung S10

Apples first iPhone

Soli

Proximity & ambient light Sensor

Google Pixel 4

Apples first iPhone

Folding screens

NFC

Samsung Galaxy Fold

Nokia 6131

UWB

Accelerometer

Apple iPhone 11

Samsung SCH-S310

3D Face detection sensor

Fingerprint Sensor

Apple iPhone X

Pantech GI100

Qi Wireless Charing

Mobile Internet (3G)

Nokia Lumia 920

Motorola A820

Siri

Camera

Apple iPhone 4

Sharp J-Phone

Gyroscope

GPS

Apple iPhone 4

Benefon ESC

First “Smartphone”

22 20

21 20

20 20

19 20

18 20

17 20

16 20

15 20

14 20

13 20

12 20

11 20

10 20

09 20

08 20

07 20

06 20

05 20

04 20

03 20

02 20

01 20

00

99 19

20

98 19

97 19

96 19

95 19

94 19

19

93

Ericsson GS88

Timeline of Smartphone Milestones.

Enableing new ideas and possibilities for developers and designers.

The development in recent years toward equipping smartphones with ultra wide band (UWB) sensors to precisely locate devices inside, magnetic accessories, and other technologies that could support using the smartphone when working from home is inspiring to me and my concept.

69

70

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

Connecting the dots.

Introduction.

Remote work will soon be the norm.

Organizations will have to change structures and we have to see what it is that is important to be happy and efficient in this situation.

are used as a replacement for those social needs nowadays. It is obvious that this is a big issue in this generation and it is on a rising trend.

Physically working together in a room has also proven to be essential for effective and creative results.

Prelimanary Questions

Communication.

Keeping up boundaries, meeting new people, collaborative teamwork suffers when working remotely. Besides offering a more flexible and calm environment, it requires self-discipline and strong communication skills to be effective. According to many, the office represents a concatenation of interruptions. This is especially true when comparing the experience to working remotely.

That is partly because communication is so much more than words and partly because energy and motivation are transferred better in person.

Solutions that can transfer these human qualities will always prevail.

Work.

Additionally, remote work also cuts commute time and living expenses. So all in all, remote working is here to stay but needs to be optimized for the human factor. Working from home parttime will play a significant role in the future of office jobs, as will services that optimize a life lived in this way, because work brings essential purpose to everyone’s life.

COVID–19.

The pandemic eased changes made in digitalization and normalization of working from home and is offering now opportunities for new thoughts, workflows and products.

Trends for basic needs.

This decline in real social interaction has already created and brought up new products and services that try to bring people together. Podcasts

Motivation.

This kind of motivation can also come from within if the worker is capable of the task, can learn to add their own ideas and work in an individual approach to a purposeful goal.

Gamification.

A way to trigger people’s emotions and get them engaged in their work is through the means of game design, which is implemented in products, services, or tedious tasks with a meaningful purpose. This modification is called “gamification”.

A mix of feedback loops, alternating rewards and efforts packaged with entertaining elements can strongly trigger those emotions. Implementing those elements leads to a greater acceptance of annoyances and repetitive tasks, as well as ideas for how to break down complex processes in a playful way.

Flow.

Keeping in mind those elements and techniques can help with work to keep motivated. Analyzing

and creating recaps of the workday can show progress and break down large projects.

Awarding oneself with a pause after a block of work and a supportive environment helps stay in focus and in a state of flow.

Important for this to work are thinking about precautions and a good level of self-skill and challenge in work. The separation of actual work and its reflection also keeps you in the present.

Psychology.

Today’s devices and platforms often keep us from being in the present as they are increasingly made to be more engaging and distracting. The huge amounts of money being made in advertisements and user data has changed many digital products to play with people’s emotions so much that they have even become addictive.

Smartphones.

The smartphone has become the main transporter of almost all digital content. That made it indispensable for both users and companies. Since the device handles so much in our lives already and adds a lot of smart to areas of our lives, we should start using this hugely capable device for work.

Screens.

The screen acts as the main interface of smartphones, being a deliverer of entertainment, information, and advertisement content. People are unlocking their screens 150 times a day to use their phone. Since this is far from being perfect and intuitive in many situations, there are trends toward concentration of automation of processes and sensor utilization.

Interacting with tech.

Computers like smartphones are equipped with an unprecedented number of sensors, unknown and underestimated by many, and underused by services and apps. They can interpret several human communication methods. Users can interact by voice, gestures, and by manipulating the surroundings that engage with the phone.

The phone, on the other hand, can interchange information through its various outputs and by inferring.

Wearables.

Wearables are the most elegant solution when it comes to intuitively exchanging information with the user. Their proximity to the body helps feed gestures and data directly to the smartphone or a corresponding service for it to react or collect.

Computing and intelligence.

The inferences made by artificial intelligence in this case are to be used a limited amount.

Because the user and remote worker want to stay in control and because of privacy matters, the automation is to stay on the primary task of action.

APIs. One way services work together and collect data from each other is via APIs. These digital interfaces are working mostly in the background, raising the value of all the connected components. An interface could therefore solely work via APIs by interacting with colleagues on a collaboration platform like Zoom, Slack, or Teams by gesticulating near the smartphone. Of course, smartphones play a huge role in APIs too.

71

72

2021 / 2022

masterthesis

extended interaction space

theory and research

felix schwamkrug

We interact based on sensory impressions and communicate through our physical virtuosities.

As outlined in the beginning, my focus has been on remote work environments, sensor-based interaction and utilization, and collaboration and communication. My research on these topics has brought out a lot of new ideas and issues to work on for my concept. Considering remote work is the whole model of work outside of a dedicated office, I will focus on working at home. As for the aspect of collaboration, I will focus on the low-threshold, in-parts recreational conversations and efficient communication happening throughout the day in a good work setting. The sensor implementation is thought of as being used as the control and connection between those two areas. For my concept, I considered radar technologies the most useful for determining the location and setting of the user as well as inferring their reactions.

73