Page 1

F

I M

N A

A

S

T

PROJECT

L E

R

M22

R E P O R T //

E X A M I N E R \\ Loe Feijs

COACH Bart Hengeveld

\\ S T U D E N T Sebastiaan Krijnen s114143 M22

15-06-2017

s.j.krijnen@student.tue.nl

1


\HELLO\

T

hank you for taking your time to read my FMP report. Here, I will guide you through the design of Tuun. Tuun is a modular, wearable interface that makes learning social skills fun and fit for everyone.

It was designed from the perspective of young, autistic children in music therapy. Vibes Muziek served as a client and provided a focus group throughout the project. Later in the process, the goal was to find an application in a wider context and target group. From halfway GGzE served as an internship company to provide experts to achieve this. With this structure, I have tried to validate Tuun in the widest and deepest way.

Please mind that this report is not built up chronologically. Separate chapters will be given to separate subjects. Most evaluative and reflective statements will be given in the Validation chapter. Though, through earlier parts of the report some of these statements can be seen too, boxed in light grey rectangles. For now, please enjoy reading.

2


/BACKGROUND / T

E

C

H

4/CONCEPT

SCENARIOS N

O

L

O

G

28

/

Y

4

/ VA L I DAT I O N B

U

S

I

N

12

52/ E

S

S

6

4

/

REFERENCES 68 / APPENDIX 70

3


Background POS ITIVE

E F F E C TS

Music is something magical. Only listening to music can already arouse feelings of euphoria (Salimpoor, 2011). This positive effects has been applied in our current world, surrounded by technology. Music has become more ubiquitous and accessible than ever before (Weinberg, 2003). Actively making music is even more magical. It can increase the pain threshold (Dunbar et al., 2012), vitality (Rolvsjord, 2010) and intellectual development (Weinberger, 1998). Outside the individual atmosphere, music has

4


d a big social component too. This holds even more for people with

/M U S I C

TH E R A PY

6

special needs, take children with autism. For them, music has the power to increase self-esteem, relationships, participation and

/G OA L

&

CO NTE X T

7

personal identities (Rolvsjord, 2010; MacDonald & Miell, 2002), to release emotional blockages and tension and to explore positive

/R E L ATE D

WO R K

10

emotions (Andersson & Cappelen, 2013; Durnbar et al., 2012) and it can decrease the feeling of isolation and increase the feeling of self-accomplishment (Magee et al., 2011). And the list goes on. From this, we could conclude that the music’s striking social component fits the weaknesses of autistic children well.

5


MU SI C

THER A PY

Music therapy focuses on the positive social effects of making

By giving these children feedback on their social skills through

music. An example of such a therapy is MTBandcoaching at Vibes

the music, it does not feel as therapeutic. This principle has been

Muziek, the client of this Final Master Project. This relatively new

used in a variety of therapies such as art therapy, drama therapy,

methodology shows an improvement of social and communicative

dance/movement therapy and play therapy (McNiff, 2005).

skills in its target group (Kempen, 2013). Examples of target groups are children with autism, ADHD, a depression, anxiety disorder, learning disability, aggressive behaviour, etc. Music can

6

offer them a fun, low-threshold activity to cope with their issues.


‡‡

‡‡

I

WA NTED ‡‡

‡‡

TO

AC H I E VE

I want to enable autistic children to easily learn social and communicative skills in a musical context. By a using a lowthreshold musical interface, skills from social situations can be learned while making music together. Instead of dry or too therapeutic activities, I want to offer a fun design they feel comfortable with. It is in contrast to many existing therapy acitivities, which are often verbal. I see that as a strange things, because autistic childrens are not verbally strong. For this, I want to create an embodied interaction (Dourish, 2004) instead. The meaning of my design should be created in this specific social setting, making the users understand them by means of their experiences, accomplishments and activities.

7


User group & context

a milder and probably more common category in the spectrum

During the span of the project, Vibes Muziek provided a group

(Matson & Boisjoli, 2007).

of four to five youngsters who functioned as a focus group. On a weekly or fortnightly basis, there was a short contact between

As the project has known different foci, the target group and

the designer and the focus group on location at CKE or Popei

context needed to respecified a few times. The beginning of

in Eindhoven. The time was filled with doing observations, semi-

the project was characterised by a converging line. In the FMP

structured interviews, experimental setups, acting out sessions,

proposal, the aim was to empower children, novices to music,

expert consults and a tests on the design. The focus group was

by enabling them to make music in an easy way. This needed to

considered as representative to the target group of the project.

be respecified on recommendation of the client. It was because

This target group was specified as children from 14 to 18 years old

all the four children already had prior experience with making,

with ASD (Autism Spectrum Disorder). Where this spectrum itself

writing and performing music. The new focus was now on raising

ranges from nonverbal children who require constant care to very

their already existing musical skills to a higher level (see figure

verbal children with above average IQ’s, they have some aspects

2). To reach this higher level, the design could teach them social

in common (see figure 1). These are social and communicative

and communicative skills. The context was still limited to the

skills such as making eye contact, starting social interaction and

rehearsal room. For this quite specific user group and context,

creating verbal and nonverbal social cues (Weiss & Harris, 2001).

a first design cycle of Tuun was done. The second half of the

The focus group consisted of four to five members: a drummer,

project was characterised by a diverging line. With the existing

guitarist, one or two singers and pianist. Among the children,

Tuun concept, a broader user group and context was explored and

there were diagnoses of both classical autism and PDD-NOS,

evaluated (see page 62).

high (group) music skills some music skills novice to music FMP brief F I G U R E 1 / visual of the autism spectrum (Source: www.pcsb.org)

8

revised FMP brief

F I G U R E 2 / the shift of focus that happened just after the FMP approval. Instead of novices, the new focus is on children with prior experience with music.


F I G U R E 3 / playing a card game can teach skills such as turn-taking, eye contact or tuning in to each other (Source: cf.ltkcdn.net)

F I G U R E 4 / exercising together as an autism therapy activity (Source: ragsdaleymca.org)

Specifying social situations

Approach

In order to give the design a striking result, it was decided to

The two pillars “tuning in” and “lack of confidence” were the

design for specific social situations. Instead of trying to design for

starting point for further design steps. It was approached by first

all the aspects that come with ASD, only the most common ones

arranging brainstorms and further ideation on how to design

in the rehearsal room were tackled. The two main social patterns

for these pillars. After selecting the best suitable interactions

that came to the surface with the expert meeting were “tuning

and forms, a draft concept needed to be created. Meanwhile,

in” and “lack of confidence”. The former means that the musicians

the interactions with the two pillars could be explored as input

have difficulties in taking or giving each other the space, no turn-

for the final concept. A final concept and physical mockup was

taking, no communication, resulting in imbalance, e.g. where the

then created and proposed for evaluation. After the evaluation,

drummer has the tendency to play loud, he does not notice how

the concept was refined with the help of more expertise. The

that affects the others. The latter means that they don’t always

two pillars were divided into two separate scenarios, evaluated

feel confident in their play. This is possibly interconnected with

separately. This vertical evaluation was done in an early phase

tuning in, e.g. a shy pianist could easily be overwhelmed by the

with the help of lo-fi prototypes and acting out. Horizontal

other musicians who are unconscious of the pianist. To avoid this,

evaluation was done with the help of more experts and a business

simplicity in interaction is key for this user group to make them

model. Ultimately, an extracted version of the final prototype was

feel good about themselves (Keay-Bright & Howarth, 2012).

evaluated with the focus group.

9


\ R E L AT E D WO R K \ \CRDL Via the modality of sound and touch, CRDL (CRDL, 2017) aims to simplify making social contact for people with special needs, e.g. autism, dementia or a an intellectual impairment. It responds when the two user touch each other by means of different sounds. With this, it tries to convert social skills from being confronting or uneasy to something playful and fun. Tuun has this same approach, concerning its low threshold and playful character. F I G U R E 5 \ a user touching his fellow-user to make sound with the CRDL design (Source: crdlt.com/crdl)

\Skoog Skoog (Skoogmusic, 2016), is a low-threshold instrument, especially designed to suit a wide range of physical or cognitive abilities. It allows for a variety of playing options using different body parts, without having to learn traditional music theory. What we could learn from it, is that Skoog remains simple while still offering many possibilities. Another interesting point is empowering effect it can have while playing your instrument or singing. To keep the design interesting and challending, new sounds can be mapped to the buttons. For the same reason, Tuun is an adaptable tool too. F I G U R E 6 \ the Skoog instrument in use by a disabled girl (Source: www.westminstertech.com)

\BendableSound BendableSound (Cibrian et al., 2016) is an example of a design for autistic children that stimulates the development of motor skills. By moving and touching a fabric with a visual display behind it, children can play music during their music therapy sessions. The design demands an active attitude and was evaluated as easy to use and useful for the target group.

10

F I G U R E 7 \ a child and garegiver using the BendableSound interface (Source: newsnet.conacytprensa.mx)


/Orfi Orfi (Cappelen & Andersson, 2012) is a set of 26 mobile soft triangular shaped and tangible cushions with different sensors and actuators, complemented by a projection of graphics. Intended for children with disabilities and their parents, the design uses their voices as input to the system. By touching bending and throwing, recorded sounds of voice can be manipulated. This design fits this FMP project well for two reasons. First of all, it makes the players use their own voice as a way to evoke positive emotions and empowerment. The design makes something playful from something intimate and personal as the children’s own sounds, similar to the intentions of this FMP. Secondly, it is interesting that the interactions are likely to be used for a longer time. That could satisfy the need for an interaction that is not ephemeral, but longer and more engaged. With Tuun, this the same tried to achieve by offering customisation and possibility for expansions. F I G U R E 8 / the Orfi objects (Source: www.musicalfieldsforever.com)

learn social skills to individuals with ASD. The skills are learned through different social scenarios by means of multiple choice questions. Different choices give different outcomes. What is so valuable for this design project, is that the children are taught how to handle in specific situations. They learn what are the keys to successful social interactions. On a longer term, it stimulates them to redo completed scenarios to find what the best outcomes and how to act in (unexpected) social situations in the future.

Sovagame, a yet unreleased game in collaboration with GGzE uses F I G U R E 9 / decision-making in social situations with uChoose (Source: a5.mzstatic.com)

/uChoose + Sovagame

a similar approach. By playing different scenarios in a 3D world, you learn how to act in certain social situations. Offering more freedom and a more realistic touch, it could be more striking than

Being a video game instead of a musical instrument, uChoose

an abstract cartoon such as uChoose. Tuun tries to take it a step

(Morgenlander et al., 2015) offers a fun and engaging platform to

further; it is embodied into the real social situations.

11


Concep \I D E ATI O N

14

\TU U N

16

ITE R ATI O N 1

\E VA LUATI O N

20

\TU U N

21

ITE R ATI O N 2

\CO L L E C TI O N

12

22


pt 13


\ I D E AT I O N \ As visible in the images on the right page, brainstorming was used as a method to generate ideas for interactions and forms. These were generated, separately for each of the two pillars “lack of confidence” and “tuning in”.

For “lack of confidence” is was chosen to use gestures and motion

For “tuning in” it was chosen to use the spotlight. In other words,

to control the size of the musician’s sound. Big movements make

one light shows which musician should be in the spotlight. By

big sound, e.g. amplifying the instrument sound intensity, adding

using this explicit visual cue, it is clear for all the musicians who

reverberation effects or adding delay effects. A first reason for this

they should give the space. Imagine that the guitarist normally

is that gestures and movements are inherent to making music.

plays quite soft, while the drummer constantly plays very loud.

Nodding along with the music, dancing or just tapping your feet

When the guitar solo comes, the guitarist starts glowing. The

just go automatically, as there is a clear connection between

drummer now knows that he should play some quieter until

music and moving to it (Sievers et al., 2013). Another reason is

the solo has ended. The main reason for this interaction style is

that the musicians should not be overloaded with interactions

that the musicians are forced to look at each other. Normally, the

different from playing their instruments. Making it them too

musicians have the tendency to look down on their instruments

hard could result in errors, frustrations and thus the opposite

and play until the end of the song. This results in a passive, closed

effect. Creating simplicity in interaction is important for this user

attitude, where music is about being active, open and aware of

group to make them feel good about themselves (Keay-Bright &

others. Making them look at each other is a first step at making

Howarth, 2012).

them listen more to each other. This should ultimately result in tuning in to each other more. Another reason is that the light effect can contribute to the magical character of the design, probably very suitable for live performances.

14


F I G U R E S 1 0 - 1 2 / explorative drawings that were made after a group brainstorm on the social situations

15


Tackling “lack of confidence” and “tuning in”. Read more at page 9

To stay as comfotable as possible, the interface is an “add-on”.

Tuun

INTRODUCING

(Tune = a medoldy; Tuning in = adjusting to each other)

Tackling the “lack of confidence”. Read more at page 9

(Keay-Bright & Howarth, 2012). Read more at page 9

16


/

TUUN

V1

/

All musicians have a wearable snake or scarf-shaped object, 200 x 15 x 3 cm. The object’s shape allows the musicians to wrap it around their bodies or instruments the way they want and the way it suits their instrument. It is a wearable to precisely track the musician’s movements or gestures. Wearing it on the body close to eye level, nonverbal communication and eye contact between the musicians is encouraged. Different types of movement need to be supported: the singer moving her arms subtly, the pianist and guitarist looking up and down and the drummer moving on the beat. If a musician doesn’t want to wear it, it is also possible to put it next to him.

The object is an add-on to their usual instruments. The musicians can remain playing their own instruments; that is their strength and what they feel confident with, which is very important with autism. Their instrument’s audio jack can be put into one end of the object, as an input; at the other end, an audio jack can be used for the audio output. Basically, this makes the object a wearable effect pedal.

Sonically, the object modifies the incoming sound when it is changed in shape. Depending on the way the musicians move, their sound will become bigger or smaller. The instrument’s sounds can be made bigger/smaller or more/less refined. Normally, a musician would only look down at his instrument, focused in his own world. Now, he is encouraged to move and to look up, in order to make his sound bigger.

Visually, the object displays the musician’s sound. Just like the sound that gets bigger or more refined, the lights glow brighter and with more sparkle. The musician that moves the most, gets this glow.

The interaction with the object is very open; there is no right or wrong way of playing. A very open interaction creates simplicity in use, which is important to make them feel good about themselves.

17


Without Tuun

F I G U R E 1 3 \ The first Tuun concept communicated by a storyboard

18


& with Tuun

Movie with sound

19


E VA LUATIO N A slight contradiction appears in the design choice for movement

Imagine that they could give and pass the spotlight themselves

as a modality for controlling the sound. Not all musicians move

while playing. The glowing light could appear at the musician

so much to music, even though many do. Would that force them to

who gets the most encouragement or votes from the others. This

move and by that, step out of their comfort zone? If they usually

suggestion comes back in the Encouragement scenario (see page

have a lack of confidence to get to the foreground, this would not

36).

help in all situations then. Imagine the drummer moving heavily to the music, taking the lead during the entire song. A shy guitarist

An interesting addition to the concept would be to make it

should then move even more heavily than the drummer to get his

adaptable or adaptive. The desired effect of this is to make it more

Tuun glowing. Probably that is too much to ask. It could be fixed

engaging and surprising on a longer term. It could become very

in two ways. Firstly, the Tuun object could glow on the person

important, since the characterising weakness of the user group

that moves the most, relatively to his usual movement. So if the

is their (short) attention span. So it could be made adaptable,

guitarist usually stands still, only small gestures would already be

just like Skoog (Skoogmusic, 2016). New modes for sound effects

enough to get his Tuun object glowing. A second option is that it

could be uploaded to the Tuun object, for the sake of variation or

would not be automated whose Tuun object glows, but manually.

dedicated settings for specific songs. It could also be adaptive. This

Through interaction with the object, the musicians could decide

would mean that the product grows with the user, accordingly to

themselves whose Tuun should glow. In the next paragraph, a

the user’s behaviour. An algorithm could keep track of the user’s

suggestion for this will be given.

regular interactions, inviting him to try other ways of playing too. Ultimately, it was chosen to go for adaptability. Adaptiveness

20

The automated “spotlight effect” of the Tuun objects should be

would probably be too unpredictable and it could cause a loss

elaborated more. Constantly shining a light on the musician

of feeling of control. Both these predictions are in clear contrast

who needs the most attention probably does not always work.

to the characteristics of autism. Using adaptability for the sake

At some moments during a song, there might not be a musical

of personalisation and control was approved by expertise in the

leader. Should the system then still choose the one with the most

GGzE internship company. It was told that mostly youngsters have

movement, while nobody really moves? Is it the best choice anyway

issues with showing interest in things they see as dull or stupid.

to make the system decide who gets the spotlight? It could be

A test around this was conducted with the focus group (see page

unreliable or even unpredictable. It could get substantially more

60 for the Collection test). When at least offering them a choice to

natural when done manually by the musicians or a supervisor.

personalise their design, this problem can be tackled.


/ TUUN V2 / Tuun is a customisable design. Tuun is modular and customisable, to make it fit for many people and situations. It fits different people, as you can choose yourself how to wear and decorate your design, e.g. you can choose to wear a jacket or a scarf, sunglasses or a cap, you can decorate it with long or circular strips, etc. It fits different situations, as different configurations of modular elements create different social scenarios.

Tuun is a wearable. It measures data close to the body, e.g. if people point or look at each other. Furthermore it adds to the feeling of control and personalisation of the users.

With Tuun, social interactions are visualised with lights. Tuun gives feedback during a therapy activity in a concrete way. Often, therapist use verbal feedback in their therapies, which does not always suit the target group the best. With autism, mostly very clear cues should be given for getting the best results, which was confirmed by Vibes Muziek therapist Estella Kempen.

Different scenarios can be played with Tuun to learn different social and communicative skills. Therapists can try out and tweak different scenarios to fit their situation. One scenario can have different stages of difficulty (read the evaluation of the Taking turns scenario at page 30). The basic kit can also be expanded with more wearables and elements to explore new scenarios.

When a coloured Tuun sender aims at another Tuun receiver, the receiver will copy the received colour and cast it to a light element. It is a very basic interaction, but because you can customise the design, it still enables you to play many games.

In Tuun’s base jacket, the modular items can be attached to the general electrical circuit with snap fasteners and to the textile with velcro.. The different modular elements should be able to be placed on different locations of the wearable. The electrical snap connectors are placed on a few spots of the jacket, close to as many velcro strips. Connectors on the elements can only be clicked into the connectors on the jacket in one way. The elements can still reach different velcro strips, as their wires are flexible. They can easily be connected and disconnected to the velcro strips.

21


BASIC

COL L EC TI ON

Enough elements to play a basic vesion of the Taking turns game.

brains The part of the technology that makes all calculations is hidden in a pocket. It can be detached to use Tuun standalone on your own clothes.

jacket

remote

The base wearable you can attach the

The Tuun remote is included in the

Tuun receiver to and decorate with

basic package to play the Taking turns

Tuun lights.

scenario.

TA K I N G TU R N S E X PA N S I O N Additional elements to play an advanced version of the Taking turns game.

Tuun receivers in a different shape are included too. With this shape,

glassy For making the Tuun lights invisible for its wearer, four pairs of glasses are included. The Tuun lights, sender and receivers can be attached on the sides.

22

the receiver can be attached to the accompanying head elements.


receiver

light

The Tuun receivers have

To customise the light effect, users

corresponding colours to the buttons

can decorate the Tuun jacket with a

of the Tuun remote.

selection of three sizes Tuun lights.

E NCOU R AGEM ENT

E X PA N S IO N

Additional elements to play the Encouragement game.

sender With the Tuun senders, users can send their colour to others. The senders could be attached to the end of a Tuun jacket’s sleeve or at the sides of the Tuun glasses.

23


AD D ITI ONA L

WE A R AB LE S

In case the users want to wear different wearables, alternatives to the regular base and expansion wearables are available too.

handy

cappy An alternative to the Tuun glasses.

Alternative to the ends of a Tuun jacket’s sleeve, where you can place the Tuun sender.

bandy Another alternative to the Tuun glasses.

AD D ITI ONA L

bracy An alternative to the Tuun jacket too.

D ECO R ATIO NS

Extra pieces to further customise your Tuun are offered too.

tag Tags can be used to make the colours of the users clearer or to further personalise the design.

light More shapes of lights, possibly more feminine, are available too.

24


1 2 3

A Tuun receiver can be hooked up to a Tuun light element. (This can be achieved by connecting both to the Tuun brains, if desired with a wearable. For illustration purposes, the Tuun brains and wearables are left out.)

If a Tuun sender aims in the direction of a Tuun receiver, the receiver will cast the correct colour on the lights.

If more Tuun senders are received, all colours will be distributed over the lights.

25


F I G U R E S 1 4 - 2 5 \ photographs of the final Tuun prototype

26


27


Sce 28 28


enarios /TA K I N G

TU R N S

30

/E N CO U R AG E M E NT

36

/A E STH ETI C S

41

29 29


\ T

30

TA K I N G U

R

N

S


To make the user learn the principle behind tuning in and taking

a place where the wearer himself cannot see it. In other words,

turns, i.e. waiting for your turn and giving it away to others, a

a user would need the help of the others to know if it is his turn.

simple game setup was designed. It requires a base wearable,

A first way would be to look at the others to find out if it is his

worn on the head, LED strips and a coloured receiver per person

turn, resulting in more contact. Even if this contact is purely

and a single Tuun remote. Similar to an earlier version of the Tuun

functional and not social, it can still break their “safe zone� by

concept (see page 17), this game has the rule that only one of

demanding them to look up to each other instead of down on

the Tuun instances at a time is able to glow, controlled by the

their instrument. A second way would be that the group together

remote. The user wearing this Tuun is encouraged to take his turn.

communicates, probably non-verbally whose turn it is. So even if a

In different contexts, this could mean different things, e.g. talking

user is purely focused on his instrument, it is likely that he would

instead of being quiet, taking the musical lead by soloing, playing

notice the others playing at a lower volume. Similarly, the users

with toys, etc. Since this setup on its own does not particularly

are encouraged to listen and look more to each other. It is quite

have to succeed in creating social behaviour (a user could still

comparable to the famous game Who Am I?, in which someone

just stare at his own light without making contact to the others),

has to guess a name, written on his forehead with help of the

a slightly more advanced version is made too (from test results at

others. A visual scenario explaining this principle is proposed

page 57). For this version, the light elements should be worn on

next.

31


1. Choose your base wearable

2. Add coloured Tuun receiver

3. Decorate with LEDs

(LEDs should not be visible for the one who wears it)

4. Give the Tuun remote to a supervisor

5. Play the taking turns game Only one participant can light up at a time. The one who lights up, should take the turn.

32


The musicians have all chosen a wearable head element:

The therapist holds her own Tuun remote.

two caps, a headband an glasses. Every musician has a Tuun receiver with a unique colour and LED strips strapped to the

wearable.

All colours of the musicians are displayed on the remote.

The drummer counts off.

All musicians start playing. They are focused on their own

The therapist decides that the singer has had enough attention...

instrument, just playing their own part. It is starting to sound a little dull.

33


... so she pushes the yellow colour.

The guitarist wearing the yellow Tuun receiver glows up in a white colour. This is not visible for him, as the headband is worn

on his forehead.

But, the other musicians are able to see his light on his forehead.

Since the musicians regularly check whose turn it is by looking at each other, the singer notices the guitarist lighting up.

This means the singer should maybe give the guitarist some

Only looking down on their own istrument, the pianist and

space. He decides to stop singing for a while.

drummer didn’t really notice that the guitarist was glowing. But now, they hear that singer has stopped singing.

34


And yes, now the pianist and drummer see the guitarist glowing.

Finally, the guitarist notices the change.

Immediately, they lower their volume to give the guitarist some

space.

When the guitarist looks around, he sees nobody glowing. He

Respectively, the guitarist stands up from his chair and plays a

concludes that he is the one who is glowing.

solo.

F I G U R E S 2 6 / storyboard of the Taking turns scenario in the final Tuun concept

35


\

EN COU -

R A G E M E N T

36


To make the users learn how to be confident enough and to help

worn too. Then, dependent on a user’s situation, he can add a

others with this, another game setup was designed. It is inspired

Tuun aimer to point with his hands or to point with his head. For

by the evaluation of an earlier version of Tuun (see page 20). The

the focus group this was an essential addition, as the pianist or

game has an approach similar to the Taking turns scenario, as

guitarist claimed that they didn’t have an arm to move freely from

the users are encouraged too to play when they light up. Though,

their instruments. When the game starts, they can point their Tuun

this time this is not determined by a supervisor but by the users

senders towards the others in order to encourage them to play. A

themselves. A specific configuration of modular elements is

visual scenario explaining this principle can be seen next.

required: every user wears a base wearable that can be decorated with LED elements, visible for him. A Tuun receiver should be

37


1. Choose your base wearable A I M

W I T H

H E A D

A I M

W I T H

H A N D S

2. Add coloured Tuun aimer and receiver A I M E R

3. Decorate with LEDs

4. Play the encouragement game Everyone can aim at each other. The one who lights up the most, should take the lead.

38

R E C E I V E R


The musicians have all chosen a wearable jacket element, complemented by a Tuun receiver and aimer with a unique

The drummer counts off.

colour. The singer and drummer place their aimer on their sleeve, as they have one hand free while playing. The others don’t, so they place them on a cap and glasses.

All musicians play their part. Yet, everyone except the singer is

After a while, the singer is so overwhelmed by the others, that

playing on a high volume, quite intimidating.

he stops singing.

This is directly noticed by the pianist.

The pianist really likes the vocals in this song, so he likes to encourage the singer to continue singing. For this, he looks in the direction of the singer, pointing his Tuun aimer element to him.

39


As a result, the singer lights up in the blue colour of the pianist.

When the guitarist sees this happening next to him, he looks in the direction of the singer too. This makes the singer light up in both yellow and blue.

With all that encouragement from the others, he feels confident

After the chorus, the singer points his red aimer at the drummer,

enough to continue singing.

making him light up in red.

The others imitate this move, making the drummer light up in

The others imitate this move, making the drummer light up in

all colours. A cool drum solo follows.

all colours. A cool drum solo follows.

F I G U R E S 2 7 \ storyboard of the Encouragement scenario in the final Tuun concept

40


/Aesthetics of interaction

TA K I N G

TU R NS

With the scenarios sketched out to illustrate the desired

The strength of the The Taking turns scenario’s is in the social

interactions of Tuun, an evaluation will be done on the interactions’

interactions it addresses. Regarding the user-product interaction

aesthetics. The Frogger framework (Wensveen et al., 2004) will be

only, the interaction with the remote is very basic. Mind that the

used to evaluate the extent of the interaction’s intuitiveness. With

design of the remote was intentionally left simple in order to

an intuitive interaction as an ultimate goal, this can be assessed

make it fit within the resources of the project. As a reaction to this,

by the couplings between information and action, i.e. the

the evaluation will include suggestions for an improved design.

unification aspects of time, location, direction, dynamics, modality and expression (see figure 29). More and stronger couplings mean

First, let’s evaluate the feedback. As can be seen in the scenario,

a more intuitive and pleasant interaction.

the therapist holds the remote, pushes a coloured button and sees the corresponding musician light up. The functional

Within the resources of the project, I chose not to create a well-designed

feedback happens when one musician’s Tuun light up and the

remote. The reason for this was that the tests I had with my focus group did not require such a design. I was in charge of that remote anyway. What I wanted to know from these tests was probably not disturbed by it either. But,

other fades out. Though, this feedback is weakly connected to the action of pushing the button. Since the IR LED casts invisible

if Tuun is ever used by a therapist without the supervision of a designer, it will be necessary to create a more intuitive design as proposed in this paragraph.

light, it is impossible to know for the user if his actions were successful. The feeling of pushing and clicking the button is inherent feedback though, but it is not coupled on any of the unification aspects to the functional feedback. This gap could be filled by providing augmented feedback, coupled in time, location, direction, modality and possibly dynamics. For example, as soon as a colour is selected, the tip of the remote could light up in the corresponding colour. To couple the dynamics, it could even have a small light effect. When a colour is selected, you could see a stream of light towards the tip of the remote and the musician.

F I G U R E 2 8 / design proposal to improve the feedback on the Tuun remote

How the direction could be coupled, will be evaluated next.

F I G U R E 2 9 / a graphical overview of the Frogger framework (Source: https-//public-media.interaction-design.org)

41


F I G U R E S 3 2 \ suggestion for improving the feedback in the Encouragement scenario

Now, let’s evaluate the feedforward. Through the functional feedforward, the user is told that he can cast light. The inherent feedforward reveals the action possibilities, in this case holding the remote in the hands, aiming and pushing the buttons. Though, a function of this Tuun scenario is that only one musician can light up at a time. This is not in any way visible in the remote; you could still push multiple buttons at the same time. To give inherent feedforward for this function, a mechanical structure can be used. Inspired by a multi-coloured pen (see image 30), the remote could show four movable, coloured pins. By only offering one spot,

F I G U R E 3 0 \ a pen with multiple tips that inspired the improvement concerning the feedforward of Tuun

the user it directly told that only one musician can light up at a time (see sketches in figure 31). As suggested by the framework, such interactions draw upon our perceptual-motor skills, rather than our cognitive (Overbeeke et al., 1999). Since we originally

E N CO U R AG E M E NT

relate to the real world in this way, it has a higher probability

The Encouragement scenario used a slightly more advanced

of becoming an intuitive and pleasant interaction (Frens, 2006).

interaction. In this evaluation, only the interaction during the

As a bonus, this interaction couples the direction too. Instead of

social scenario will be discussed. The individual user-product

pushing a button down in the direction of the ground, a pin is

interaction while building up the modular wearable will be left

pushed forward in the direction of the musician.

out of consideration, since that is not the focus of the project.

F I G U R E S 3 1 \ suggestion for improving the feedforward of the Tuun remote in the Taking turns scenario

42


Let’s begin with the feedforward. For the Tuun send and receive

LED (see image 32). Logically, the red receiver is painted in the

elements, the functional feedforward is mostly achieved by

other way around. These visual cues draw upon the cognitive

showing the IR LED and receiver. These visible electronics provide

skills of the user. Together with the suggested shape, it can create

information to the user about the functions of the elements. The

a more powerful feedforward. As an addition, an extra LED could

IR LED shows that it probably can emit light, plus it sticks out a

be added next to the IR LEDs as an augmented feedforward that

little from the block. The receiver does not stick out of the block as

the sender element is actively broadcasting IR signals.

much as the LED. Shape-wise, it could be clearer that the objects have a relationship (see image 33). For instance, the LED could be

Now let’s discuss the feedback part of the framework. In the current

integrated in a little circular blob of the sender. The receiver could

design, the functional feedback is given at another musician. You

then have a cavity in the same shape, suggesting that the two

can see that your action was successful if your own colour is

elements fit into each other.. Look-wise, the current Tuun design

starting to light up at the other musician’s wearable. There is only

uses visual augmented feedforward. Painted in colour, the sender

a coupling of time and direction. The coupling of time can even

and receiver show lines that give a clue about the function. For

be stronger, as there can occur a little delay. Through augmented

instance, a red sender shows a diverging red line on top. It is

feedback, more couplings can be made. This feedback could be a

broader at the beginning, as if it picks up the red colour from

small addition to the behaviour of the Tuun light effects: imagine

the connector. Then, it becomes more narrow towards the ending,

that a user has a green instance of Tuun. Green could be initially

ultimately flowing into the side where the colour surrounds the

shining on his own wearable. Then, when he aims at someone else, this colour can be animated out. To couple that in dynamics and expression, an accelerometer could measure the motion and create a suitable light effect. So, if he quickly points at someone, his green colour will quickly fade out of his jacket. Preferably, the light effect should also look like the lights are transferred to and sucked up by the sender. If he carefully and gradually points at someone else, the ideal light effect would be that the lights will slowly and gradually appear at the other’s wearable.

F I G U R E S 3 3 \ suggestion for improving the feedfoward in the Encouragement scenario

43


Technol \IR

TE C H N O LOGY

\TU U N

ITE M S

\M O D U L A R IT Y

44

46

48

50


logy 45


\ Infrared technology

PROTOCO LS & CO D E S

The major challenge of Tuun’s technology was to enable it to

So for IR, (abbreviation for infrared), an emitter and detector

detect contact between different users. In its basic principle, it

device is needed. Blinking at a speed of 38 kHz, encoded

needed to distinguish pointing at one person from pointing

commands can be transmitted and decoded. Using specific IR

at someone else, standing a few meters from the other. A few

protocols, e.g. the NEC, Sony or NECx, detectors are only activated

technologies were considered, including GPS tracking and

when they receive IR messages known in these protocols. For

motion detection using an accelerometer and gyroscope. The

this reason, television remotes can sometimes be used to control

biggest pitfall with technologies like these, was their stability

different televisions too. Or, a simple Arduino setup can record

and accuracy. Reasoning from the Frogger framework (Wensveen

and playback IR signals to control every IR application. In other

et al., 2004), the technology needed to react witzh the least delay

words, IR is a generally used and thus open communication

in time and with the closest match of location. In response to

tool. Though, a downside of this openness is that “unwanted

these important requirements, the infrared communication

visitors” could disturb this communication. For Tuun’s context in

technology was chosen. Since this technology makes use of (non-

a rehearsal room or therapy room, that is not an issue. At larger

perceivable) light, it is fast, accurate, reliable and even low-cost.

venues during a live performance, it could be an issue. Assuming

Mind its great popularity in applications like remote controls for

this will not happen, the four different colours of the Tuun send

televisions, night vision, thermography, etc. At an early stage, a

elements are represented by four different hexadecimal NEC

simple test was done to check if the technology suited for my

messages. Made-up messages were chosen, not coinciding with

intentions. For this, two Arduino setups were created, responding

any other NEC command. The Arduino commands for sending

on an ordinary television remote. The volume up button was

them could be achieved with this code (see Appendix B for the

continuously pressed and the Arduino setups measured if they

integrated code):

detected any incoming message of a volume up command. If so, its corresponding LED would light up (see image 34-35). In

green: mySender.send(NEC,0xabcdefab, 0);

this test, it appeared that the technology fitted the intentions

yellow: mySender.send(NEC,0xbadcfeba, 0);

perfectly; it responded instantly and only when the remote was aiming in the Arduino’s specific direction.

blue: mySender.send(NEC,0xbafedcba,0); red: mySender.send(NEC,0xabefcdab,0);

A Tuun receiver could then detect if one of the other colours was aiming in his direction (see Appendix A for the integrated code):

if (code == “abcdefab”) { Serial.println(“Green is looking at this direction”); } if (code == “badcfeba”) { Serial.println(“Yellow is looking at this F I G U R E S 3 4 - 3 5 \ testing the IR technology by making a simple Arduino sketch with an IR receiver, red LED and an ordinary television remote

46

direction”);


} if (code == “bafedcba”) { Serial.println(“Blue is looking at this direction”);

detectors could not make any sense of the codes any more (see image 39). In the earlier test, this problem was non-existent yet, as the Arduino sketch sent the IR signals successively instead of simultaneously (see image 36-38).

} if (code == “abefcdab”) { Serial.println(“Red is looking at this direction”); } From this base code, the final Arduino program was created. Detecting the specific colours could then be translated into a simple visualisation in the LED strips.

F I G U R E 3 9 \ illustrating why the IR receiver could not make any sense of a constant stream of parallel IR codes

To fix this problem, a reset command from an external remote was brought to life. Then, a button can be pushed to start the session, e.g. by a therapist, band coach, making all Tuun instances reset a periodic sending pattern, all with a different starting delay. All IR signals now have a dedicated point in time when they can be selectively listened to.

F I G U R E 4 0 \ illustrating that a well-timed, periodic pulse can prevent the problem from figure 39 F I G U R E S 3 6 - 3 8 \ IR receiver decoding three IR codes with colours

With IR, I think the project uses the most suitable technology that is available. Though, I have encountered some issues, mostly regarding its stability. To

I NTER F ER EN C E As put forward earlier, interference can be an issue when receiving IR codes. An unforeseen problem arose when all four autonomous Tuun instances were emitting their colours at the same time: the detectors were not able to decode any correct code any more. This new malfunction turned out to exist, since the IR codes were now fired continuously from different sources. Now, the detectors only received a mixture of all IR light codes. For this reason, the

illustrate, there is a chance that an IR signal is sent, yet not received. Some devices don’t send their message once, but three times in order to prevent this. But even then, any object - which is likely in a furnitured or equipped room can block the IR light signals. I am using a similar approach, by sending the same message multiple times. Some issues have appeared in the other way around too, when an IR signal should be blocked, yet it is received. A reflection on a window, mirror or any reflective material can bend an unwanted signal to the detector. So, I think an IR beam should be bundled well enough, in order not to make it bounce in all directions of the room. By wrapping a small piece of firm material around the IR LED, this can be easily achieved. Still then, it can happen that the beam will reach an unwanted receiver. Though, I think that the many advantages of the technology compensate this small disadvantage.

47


\Tuun items As described in the Collection section (see page 22), a variety of modular items were designed. Here, you will find an overview of these items, accompanied by a short explanation on how and why they were constructed like this

.

TU U N

R EC EIVER

Now the infrared technology was validated to be successful, the components needed to be integrated into the modular wearables. For the Tuun receiver, the IR decoders were simply put into laser cut boxes, hiding the soldered connections between the three legs and electrical wires. F I G U R E 4 5 \ hiding the IR receivers in a lasercut box

TU U N

SEN D ER

The Tuun senders needed a closer look though. Where the initial plan was to only hide a simple circuit, including an IR , transistor and resistor, this did not suffice. The reason for this was that one Arduino board cannot transmit and decode IR signals at the same time. An alternative was chosen to make the Arduino constantly alternate between sending and receiving, yet it turned out to be too heavy for the Arduino. It may have been possible

F I G U R E 4 6 \ initially hiding the IR LEDs in a lasercut box

to make it work with some advanced programming, but it was chosen to integrate separate Arduino boards into the Tuun sender elements. The intended circuit with the emitter, transistor and resistor was then fully transferred into the sender blocks. The two electrical wires that were initially intended for the IR LED, could now be used to power the Arduino. As described in the previous paragraph, a reset delay was built too. In the code, the Arduino reacts to a specific reset command to turn on the electricity of the Tuun sender with a transistor.

48

F I G U R E S 4 7 \ ultimately adding more technology to the Tuun send casing to both send and receive IR codes at the same time


TU U N

L I G HTS

For the visual output of Tuun, a variety of LED strips were cut up in different shapes and hidden in between layers of textile. A layer of soft foam was added to make the lights more pleasant to wear and to diffuse the lights of the LEDs. This was sewn together by some bias binding and a velcro strip (see image 51). As the LEDs in the strips needed to be individually addressed,

F I G U R E 5 0 \ sketching how to make a combination of materials to create the Tuun light elements

digital WS2812B LED strips were used. Only requiring a 5V, GND and data pin and the IRLib2 Arduino, it was easly connected to the IR communication. A boolean array state[] stored if there were Tuun sender aiming in its direction. If so, the corresponding colours were displayed in evenly spaced sections in the strips. If a sender’s signal was lost, it waited to be 3 seconds without signal, followed by the colour disappearing. Since the signals were not always stable, this ensured that the LED strips wouldn’t be flashing when losing and regaining connection. F I G U R E 5 1 \ using a sewing machine to put the Tuun light elements together

Although the LED strips were not flashing, the light effect still looked too “digital” for its purpose. Imagine a user slowly turning towards the direction of another user. At a certain point, the other user would go from dark to fully lit at once. Because of the way I used the IR technology, there is whether no connection or a connection. While this technology can be binary, a social interaction could be more linear. Evaluating from the Frogger framework (Wensveen et al., 2004), there is not enough unity in dynamics and expression. Regarding the location, lights could be starting to fade in when someone else is slowly turning to him (read more on page 43). Though, the way I used IR would not suffice any more. As a first solution, camera detection or a gyroscope could be added to create the effect. A more simple solution would be to fade the lights in and out. It would then look like a linear effect, while just being a simulation.

F I G U R E S 4 8 - 4 9 \ showing three breadboards with Arduino setup, sending an IR pulse well-timed, successfully received by the fourth Arduino

49


\Modularity Tuun was designed to be both modular and a wearable. For this, the challenge was to create a design that kept a balance between offering and constraining possibilities to the user. For instance, the Tuun sender element only makes sense when placed near the hands (for pointing) or on the head (for looking in a direction). Consequently, the user can only place the sender there. But the LED elements should support different positions on the wearable; the user has to be able to choose the desired spots. For all these possible locations of the elements, connectors needed to be F I G U R E 5 2 \ the three unique connectors, not yet soldered

designed to connect the elements to the circuit. Mind that not every velcro strip has his own connector; only four locations were chosen strategically, to offer the most flexibility with the least amount of connectors. Three unique connectors were designed (see image 52) for the sender, receiver and light elements. It was necessary, as the user should not be able to click an element to a wrong connector. They were designed in such a way, that the elements could only fit in one possible way to its corresponding connector. This was necessary too, as the pins from the sensors and actuators can only fit one way. The connectors consisted of

F I G U R E 5 3 \ a finished connector, plugged onto the corresponding connector on the Tuun jacket

a few materials: The sewable snaps were perfect for this project for many reasons: they were conductive, easy attach and detach, small, low-cost and suitable for the charac-

Firstly, it consisted of metal snaps to ensure a secure connection between the electrical circuit. Header pins were considered too,

ter of a wearable. Though, I have learned that it not easy to solder the snaps to an electrical wire. Both sides of the snaps (but mostly the “male� part) need to have a low amount of soldering tin. If it forms a little bulge of tin or electrical

but could possibly disconnect too easily. JST connectors (or any other type of commonly used connector) were considered as well,

wire around the points where the snaps meet, it is hard or even impossible to click the snaps together. A first advice is to use as little soldering tin as needed. A second advice is to solder an electrical wire at the outer part of the snaps,

though a user would not be able to distinguish a 3-pin receiver element from a 3-pin LED strip element. Plus, snaps generally

instead of inside. If possible, the electrical wires should be replaced with conductive yarn. Then a firm connection is made and no soldering tin is required.

don’t have a technological association, since they are widely used in everyday clothes. Other connectors could be distracting, overwhelming or confusing.

Secondly, the connectors consisted on the side of the jackets of a synthetic leather, to which the snaps were sewn. It was a practical solution for holding the snaps into place while sewing. Another important argument is that the relatively firm leather does not bend as easily as the textile from the jacket. It was an essential

50

F I G U R E S 5 4 \ sewing the conductive yarn through the leather towards the flatcable in the jacket


consideration for preventing the 5V and GND snaps to make connection and cause short circuit. To even prevent this more, a small extension of the leather fabric was added to be flipped over and attached to the other side with velcro. It can protect the snaps when, for instance, the user accidentally leans the snaps to a metal surface, creating short circuit.

Thirdly, the connectors consisted of conductive yarn, which was used to sew the snaps to the leather. Behind the snaps, through the leather, a connection with the wires from the general circuit

F I G U R E 5 5 \ preparing the connectors by first gluing and then sewing the snaps to the leather with conductive yarn

was made. The entirety of the general circuit in the jacket was covered with a bias binding and at many spots, a velcro strips. While an initial design of the circuit would have used a five channel line of sewn conductive yarn, flatcable was ultimately chosen. Flatcable has less chance of breaking, making contact between channels and it is lower in cost. The only advantage of conductive yarn would be its possibility of making it stretchable, using a specific sewing patterns. Though, inspired by the Adidas stripes and military velcro patch jackets, it was chosen to go for the integrated flatcable.

F I G U R E 5 6 \ trying to make a strong connection between conductive yarn and the eletrical wires from the flatcable

Conductive yarn seemed to be perfect to fix conductive elements to the fabric and to sew an electrical circuit. Though, it was impossible to solder to the electrical wires. For the final prototype, the conductive yarn was tied in knots around the holes of a piece of protoboard. The holes were then filled with solder and connected to the electrical wires. To secure the connection, a dot of glue was added. Though this seemed to work for the prototype, it is not a stable way of connecting a thread with cable. In future situations, it is better to use crimp beads for the connections.

F I G U R E 5 7 \ the flatcable, partially pulled out to solder it to a connector

As a last addition to the modularity, it was decided that the brains (see image 58), covered in a laser cut box, should be able to be disconnected too. From a customisation perspective, it could then be possible to apply the Tuun technology to different types of wearables. Think of the Tuun cap, for which the brains could be placed into the hood of the jacket. The receiver, sender and strips (with slightly longer wires) can then be attached by joining the velcro and putting the wires behind the ears, via the neck, to the hood. With more possibilities, more game scenarios can then be

F I G U R E 5 8 \ the Tuun brains in constructing, still put together with header pins instead of snaps

built.

51


Va 52


alidation /TA K I N G

TU R N S

TE ST 1

54

/TA K I N G

TU R N S

TE ST 2

55

/TA K I N G

TU R N S

TE ST 3

/E N CO U R AG E M E NT

/CO L L E C TI O N

/E X PE RT

( F I NA L)

TE ST

TE ST

LU N C H

M E ETI N G

56

58

60

62

53


\Taking turns test 1 A first experimental setup was conducted in context to explore the effect of tuning in with a setup similar to the intended design direction. The musicians were asked to gather around in a circle with approximately two meter space in between. Then,

E VA LUATIO N

four ordinary table lamps were positioned right behind each

Although it seemed that the musicians were sensitised for the

individual. Every lamp was in such a position, that the “owner” was

subject of tuning in, the results were not successful in what it

the only one who couldn’t see it. The musicians were instructed

needed to achieve. The main reason for this was that the electrical

to play a song that they mastered and enjoyed playing, so that

wires of the lamps were not long enough. An effect was that plugs

they felt comfortable in looking around. Next, each individual

needed to be plugged in or out to switch the lamps off, in plain

was asked how he could tune in to others with his instrument,

sight of the musicians. It was too obvious and predictable when a

e.g. playing softer, less notes, higher notes or heavier chords, etc.

lamp would switch off and on, completely eliminating their need

Then, only one lamp at a time was turned on, indicating that the

to watch each other’s lamps regularly. A second reason was that

corresponding musician had the musical lead from that moment.

the lamps were not placed optimally behind them, sometimes

With this, the others needed to be more on the background. The

even within their own field of view. One musician could easily

interesting thing was that they couldn’t see their own lamps. They

see when his turn was in the corner of the eye, while the others

had to non-verbally and musically communicate whose turn it

couldn’t. A next, improved version was created and tested next.

was. Explicit cues of tuning in were stimulated with this little test.

F I G U R E 5 9 \ redesigning the setup for experiment 2

54


every individual completely impossible to see his own light. The LEDs were attached to a long wire, ending in an Arduino that was

/Taking turns test 2 As a response to the points of reflection from the first exploration,

controlled during the test. An Arduino sketch could determine which LED to turn on, without the musician noticing it. With this, two little tests were done:

a new test was conducted. For this, four wearable headbands with LEDs on it were created as a wizard of oz prototype. A headband

1.

The musicians were instructed to do the same as in exploration

was chosen for its characteristic location just above the eyes.

1.This was taking or giving the musical lead, as a reaction to the

This placed the LED directly on their forehead, making it for

LED shining on one person and reappearing somewhere else.

2.

The musicians were instructed to play a song, looping a certain piece of music for two minutes. Instead of following the LED, the LED followed them now. So the musician who was considered as the lead while playing (as assessed and controlled by the researcher) received the shining LED. The only rule was that everyone should have had his light turned on once before the end of the song. The reason for this test was to make them familiar with guiding this tuning in process themselves. Instead of only waiting for their turn, they can learn too how to take turns and how to be solidary

F I G U R E 6 0 / the LED headbands from experiment 2

with each other.

E VA LUATI ON

to assess who had the musical lead. It was mostly based guided

The tests worked significantly better than in the first exploration,

by who produced the most intense sounds. Yet, that does not fully

regarding the points of improvement. In general, the results of

suffice when assessing something like this. There could be other

the test were positive: the musicians seemed to easily adapt

important aspects too, such as movements or gestures. Reflecting,

to the first test mostly. It was clear what was expected, the

it could be dubious to make this system automated in a final

musicians regularly checked each other’s LEDs and reacted

design, where a computer should assess this.

instantly. Yet, there appeared a difference between the musicians. One musician was more sensible for the light due to its simplicity.

In general, these tests were done to get a first feeling of designing

The found it very easy and thus comfortable to pay attention to it,

for interactions within this context and user group. Plus, the user

explained by the expert as a result to his light mental disability.

group became sensitised for the subject too, which is important

Another musician wasn’t paying a lot of attention to the lights

for when the final concept will be proposed. For confidentiality

due to his short attention span, explained by the expert. The

reasons, none of the tests were recorded.

second part was slightly more unstructured, as it was quite hard

55


\Taking turns test 3 (final) The Taking turns test was tested again in a later stage of the process. Now, the Tuun prototype could be used.

SETU P

R E S U LT S

Three musicians were placed in a triangular formation, so they could easily see and hear each other. First, they were asked to fill

Stages of difficulty

in a questionnaire about their former experience in the band, e.g.

“I couldn’t see if it was my turn, because I couldn’t see the light.”

“There is a nice atmosphere while playing”, “I listen closely to the

“It is extra clear.”

other musicians” or “Every musician gets enough attention [while playing]”. Next, they were instructed with the Tuun prototype.

“It is nice to have a light, but it is good as well if you listen closely. That is a good exercise too.”

Three instances with a Tuun brains, receiver and light element were divided among the musicians. The elements were taped to

“If I can’t see it [the lamp], I get confused, that’s my autism.”

locations on their own clothes or instruments, preferably where they could not see their own light and where the receiver was

“[if you would first use the lamp and then eliminate] Yes, that would

easy to reach with the Tuun remote. Next, they were asked to play

be nice. That’s a nice way to build it up, then you learn something

a song they felt comfortable with. While playing, only one light

from it.”

of the musicians was triggered by the researcher. Just like in the original scenario, the corresponding musician was asked to take his turn. The musicians needed to make contact to communicate

Making contact “If you leave out the lamp, you can hear if it’s your turn or not.” “Hardly any contact was made. So if you make contact, you actually

whose turn it was. After finishing, the musicians received a similar questionnaire. The same questions from the first questionnaire were now complemented with a selection of ratings of the UEQ

don’t need to see the lamp.” “If we nodded at you when your light shines, it would already be fixed … We didn’t make an agreement about that.”

scale (UEQ-Online, 2016). This scale was specifically used to evaluate their user experience with the Tuun design. Afterwards,

“I did make some gestures at Bas, but that didn’t do anything.” “[he]

a semi-structured interview was done with the entire group.

didn’t see it.”

With the qualitative input, important quotes were extracted an processed in an affinity diagram, clustering them in the most important themes.

56

“I did look at him with a severe face, like, don’t play so loud … No [I don’t usually do such things], but I was really bothered with it now.”


F IN D ING S “[when things go wrong] we usually stop or our therapist says that afterwards … but while playing, no.”

& D ISC U S S IO N In general, the musicians seemed to like playing with Tuun. There was a technical error that created some confusion though:

Focus

sometimes two lights were shining instead of one. By all means,

“It is a good exercise, regarding communication and also to listen to

this should not happen, as it can disturb the effect and confuse the

each other. Even without looking, you can make contact by listening.”

users. Other doubts they had were mostly about the visibility of the lights, both expressed in the interview and the questionnaire.

“… [if another musician controlled the remote for her] then that person should know what kind of song [she wrote] it is.”

“I wouldn’t think it [controlling the remote myself while singing] is practical, because I am focused on singing and then I wouldn’t be able to concentrate on that any more.”

Not all of the musicians understood that the principle of not seeing your own light forced them to make contact. It seemed that they needed to get used to this situation and make agreements about it first. As a first suggestion, therapists should probably first discuss with the clients how to make non-verbal contact, before interacting with Tuun. Another second is to build the game up in

“If I look around, I’m afraid that I play wrong notes. That the fear of it.”

different stages of difficulty. The group could begin with a setup where they can see their own lights. Later, the lights could be

For the quantitative data, it was impossible to show any

placed around the head to raise the difficulty and to encourage

significance. Generally, the qualitative and quantitative data

more social interaction. This difficulty probably plays a bigger

seemed to correspond when looking at the individuals. Plus, the

role for users like musicians, who have to remain their focus on

questionnaire served as a conversation starter for the interviews.

playing their instruments too. For users like children in a group

For these reasons, only the qualitative data is taking into account.

conversation, it could be easier.

F I G U R E 6 1 \ a standard remote that was used during the user test

F I G U R E 6 2 \ typical rehearsal room at Popei, where the test was conducted (Source: https-//images0.persgroep.net)

57


\Encouragement test (final)

After this, the acting out began. Behind each of the musicians, another participant was placed, the so-called observers (see image 63). They could observe the situation and decide to point

GOA L

in the direction of a musician. If one of the observers noted that

To test the encouragement in an early stage of the process, an

he was pointed at by two other observers, he needed to give the

acting out session was conducted with the focus group. For the

researcher a sign. On that sign, a lamp in the middle of the circle

scenario would demand a fully working, advanced prototype, this

could be moved towards the corresponding musician. With this

method was chosen as an alternative. To act out the pointing,

“spotlight”, that musician was encouraged by the others to play

the musicians just needed to point their finger or nod into the

a little louder, play more notes or whatever made him stand out

direction of the aimed person. The one with the most “votes”,

from the res. Again, it was followed by a questionnaire with

would light up, very similar to the original interaction. As a

similar statements as the first one. The approach with the two

replacement of the glowing Tuun jackets, a single movable lamp

questionnaires was taken for two reasons. Firstly, it functioned as

was used. So where in the original scenario one user would light

a comparative quantitative tool for later analysis. Secondly, the

up, in this test the lamp would be moved to that musician. In

questionnaires were used as starting point for a semi-structured

general, the goal of the test was to validate the hypothesised

group interview after the acting out.

effect of giving the musicians more confidence.

SETU P Three musicians were placed in a triangular formation, so they could easily see and hear each other. First, the musicians were asked to play a song they felt comfortable with. After they finished, a first questionnaire was given. It included statements that needed to be rated, e.g. “I felt self-confident while I was playing”, “I listened well to the others” or “I made contact with the others by making eye contact”. F I G U R E 6 3 \ an overview of the participants of the test, seen in top view

58


R E S U LT S

D ISC U S S IO N The main point that can be questioned about this session is its

“I looked like I felt more confident in the second situation [with Tuun].”

representativeness for the intended interaction. One significant

“If that lamp turns on, it is a very clear signal for me.”

difference was that in this test, the musicians did not have to point themselves. It is still arguable if that makes any difference

“[when the light switches on] you notice that you really have to place.

for the feeling of encouragement. Why would it feel any more

That is different than without a lamp.” (or less) encouraging if one of the musicians would point at you “The (musical) dynamics were different. The social interactions

instead of an observer? The pointing musicians was tested too in a pilot test, but they claimed that it was too hard. This could

weren’t different.”

probably mean that the multitasking in the Encouragement “I tried to make more contact. But I noticed that the others were

scenario is quite difficult to master in a musical band context.

focused on themselves.”

At contexts like a business meeting or group therapy, this would be less of an issue. Another difference was that the acting out

“They were only looking at the lamp to see if it was on.”

did not make use of any coloured light, in contrast to the Tuun concept. It is likely that this can even have a bigger effect, as the users can see from whom the encouragement came, e.g. “Hey, I

Just like in the previous test, only the qualitative data will be

see my jacket lighting up in green, so now the singer must be

discussed, as it functioned as a conversation starter for the

pointing at me!”. Summing this up, the Encouragement scenario

interview too.

is still left with some assumptions. Yet by triangulation, i.e. the combination of input from quantitative data, qualitative data and expert feedback, these assumptions are minimised.

FI N DI N G S In comparison to the Taking turns scenario, the Encouragement scenario can still be played without making much contact. Though, as the users themselves have to pass around the turn, they can feel more empowered by the others, resulting in more confidence.

59


\Collection test

doubted, dialectical laddering (Neimeyer, 1993) was used to find an optimal combination of the wearables’ characteristics.

GOA L Earlier, it was assumed from the recommendation of a GGzE’s expert that youngster need to be able to customise their design.

R E S U LT S

This resulted in expanding the single Tuun object to a family of

From the conversations, the preferences were abstracted into

Tuun products (see page 24 for the full collection). Before further

“yes” or “no” symbols, accompanied by illustrative quotes. See the

designing this family into detail, this assumption needed to be

image on the right page.

validated with the focus group. So the goal of this test was to confirm the musicians do have a varying preference in wearables. By starting this conversation anyway, the actual preferences

CO NC LU S IO N

themselves were aimed at too. After all, this information could be

As an answer to the first question, it could be obviously stated

used to decide what part needed to be made as a minimum viable

that the different musicians had different preferences. While one

product for the final prototype.

musician seemed to like everything, the other almost nothing. Still, the users agreed to the minimum of one item for every cluster. The first slide with the wearable head elements showed

SETU P

popularity in the LED sunglasses (item D in picture 64). This

A small Keynote presentation was made, built up by three

was except for one musician, who said it could be “annoying, not

slides containing three little clusters of images. These three

particularly because it’s in front of your eyes, but because you can see

clusters were small moodboards, representing three parts of

it.” It was quite similar to another musician, who did agree, but

the collection: wearables worn on the head (for the Taking turns

admitted that “if the lights are behind [not in front near the eyes, but

scenario), wearables worn around the arms or hands to aim

more behind on the legs of the sunglasses, around the ears], it would

(for the Encouragement scenario) and more general wearables

be good… it is better if it is not too close to my eyes.” For this, the

worn around the body. The idea behind these clusters was that

sunglasses were chosen for the final prototype.

every user needed to be comfortable with the minimum of one

60

per cluster. A laptop with the presentation was installed just

In the second and third slide, the bracelet (item E in picture 65)

outside the rehearsal room, out of reach of the musicians. One by

and LED jacket (item F in picture 66) turned out to be popular

one, the musicians were brought to the laptop for an individual

and agreed upon by all. But, the bracelet could only work “if it

conversation. The reason for this was that, pointed out earlier by

would not stand in the way [while playing]” and “if it would be as

GGzE expert Estella Kempen, the attention span of the musicians

tight as a watch”. From these quotes, it can be concluded that the

could be significantly lower in a busy context. Plus, the musicians

musicians think it is important that the Tuun wearable should not

could affect or overwhelm each other, mostly when it comes to

be annoying, flying and hanging loosely. For the final prototype,

intimate subjects such as clothing preference. Per slide, each

it was decided not to make a separate bracelet, but to integrate

musician was asked through a semi-structured interview which

it into the stretchy ends of the sleeves of the jacket. There, it is

of the wearables he would definitely not like to wear and which

still at at comparable location and fixed to the arm well enough

one(s) he would. If a musician did not like any of them or if he

with the elastics.


F I G U R E S 6 4 - 6 6 \ results from the collection test

61


\Expert lunch meeting GOA L The previously discussed test were a part of the vertical validation in the project, i.e. evaluating in-depth on a narrow area of the project. Next to this, horizontal validation was done too, F I G U R E 6 7 \ talking to the experts during the lunchmeeting

i.e. evaluating a broader application of the project. For this, an expert panel was assembled, including seven GGzE therapists and co-workers, of which Jeanine van der Meijden, Menno Hartog

R E S U LT S

and Joep Kolijn provided feedback from years of experience in the field. A lunch meeting was arranged with five of them, inviting them over to the eLab. The goal of this meeting was to validate

Making abstract concepts concrete

(and maybe refine) the Tuun concept and to find a broader context

“You eliminate the (verbal) language. That’s where most

of application.

communication problems occur. That is a very promising part (of the concept). By using LEDs, essential parts of communication are enforced.”

SETU P The guests were welcomed in a medium sized meeting room, with a long meeting table in the middle, pointing towards a beamer

“The strong point is that you abstract the concepts of communication (from the social situation). This would fit by many other target group, if you focus on the methodology.”

image cast on the wall. On the table, there were basic creative tools, such as post-its, ordinary white and coloured paper, pens

“Social skills are very complex when you think about it. Sensing and

and markers. On a low table in the corner of the room, the Tuun

predicting (a social situation) is very hard and intangible and with

base wearables were presented, including the (not yet processed)

this concept, it would become concrete.”

black jackets, caps, glasses and sunglasses. First, an introduction on the subject and design was presented. Next, an introduction

“Maybe more abstract social concepts could be translated into something physical, such as the length of speaking (during a group

round was done among the guests to get acquainted with each

conversation).”

other and each other’s background. To work towards an evaluation of Tuun, the two scenarios of Taking turns and Encouragement

“The abstract information is not present in people with autism. With

were presented. A few questions as a base for a semi-structured

this, you make people still aware of it.”

interview were asked next, e.g. “Do you like the concept? Would it work?”, “What would other social and communicative skills or games

“People seem to know things that I don’t know. Recognising emotions

be?” “Disregarding autism, how could it be applied? Would any other

would be very useful for this too.”

Tuun items be needed?”. From the qualitative imput some main quotes were abstracted, put into an affinity diagram to cluster

Customisation “The first concept (Taking turns) fits better with therapy, because you

them into main themes.

62


need a therapist. The second one could be given (to use it at home).” social concepts can be created (see figure). This understanding is “It is beautiful that on a certain moment, the control can be passed

more valuable than just a set of gimmicks you can perform.

on.” Some further applications of the concept were found interesting “The nice thing is that you can make the user themselves determine at what moment which methodology can be used. It really depends on the (therapy’s) goal.”

too. Some social concepts are so difficult to explain, that no technology can be used to automatise them. Think of the concept of timing in a conversation, brought in by one of the experts.

Further application

Knowing at what moments to speak and when to be silent can

“Department meetings can be hard. Some people are quiet and others

be complex. No computer can or should do that for you. It can

talk a lot.”

be taken over by a supervisor though who controls the situation. For less difficult social concepts, automatisation could be used.

“In communication trainings, we are still talking a lot.” (paradox)

Think of the concept of balancing a group conversation. A new item, the Tuun timer element could be designed. I would react on

(about taking turns) “The group can give the turn or you can take the turn yourself, by making others help you.”

“If the goal is to make someone take the turn more often … maybe a light could slowly fade when that person hasn’t spoken for a while.”

the time passed and a microphone. When the microphone has no sound input, the connected lights could slowly fade in. In a group conversation, this could visualise someone has not spoken for a while. He can then eliminate the lights by speaking. If someone is constantly talking in a group, you can see that too. This way, Tuun

“On a department meeting, this could be implemented well. Someone

could concretely visualise the balance of a group conversation.

would join to control the turns. Probably even without this control.”

FI N DI N G S

Two more experts were consulted in expert meetings, both active

The core value proposition of Tuun is to make abstract social

in OPSY, an institute for (young) adults with a learning disabilities

concept concrete. It is a promising approach, especially for

or a mental weaknesses. Firstly, case manager and game therapist

people with autism. There is a paradox in many therapies, where

Anke van Ettinger confirmed that the Tuun concept would work

social skills are explained by means of words. Communication

well. Similar to the experts in the lunch meeting, she noted its

and feedback is often verbal, which is exactly the weakness of

strength of giving insights of who you are and how you behave.

participants.

The awareness it creates would work well. She summed up a list of activities that were part of therapies and pointed out that Tuun

Though stated differently earlier in the process, Tuun should not

would blend in perfectly in such activities. Also in a context with

focus its effectivity on application outside of the walls of the

both a child, parents and therapists it could simplify conversations.

therapy room too. Social skills should not be just a gimmick you

Secondly, Gijs Cuppen had similar thoughts. He noted that it could

learn and then copy later. It is about awareness. With autism, the

be used to improve social interactions in conflicts at work in a

knowledge of social concepts is not always present, where people

family with a child being raised. To conclude, many experts from

without autism it often is. Making this knowledge into something

different directions seem to see strength in the current Tuun

experienceable in practice, visual and concrete, awareness of the

product and give suggestions for further applications.

63


Bu 64


usiness When the Tuun concept became gradually clearer, a business

main reason for this was that the application of a Tuun design in

perspective was added to the project too. As with validated with

business would still have some riskiness; it is simply impossible

experts, Tuun fitted for a broader application beyond musicianship,

to tell how things will work out eventually. For this, the Lean

youngsters and even autism. The Lean Canvas (Maurya, 2012)

Canvas offers a tool to make such a canvas “as actionable as

was used was filled out and evaluated with internal GGzE expert

possible while staying entrepreneur-focused� (Maurya, 2012).

and internship coach Koert Mulders. He has participated in a

Here, the sections of problem, solution, key metrics and unfair

few Lean Canvas workshop and has experience with making the

advantage are added. The core rationale of the canvas will be

Lean Canvas for similar GGzE student projects. This canvas was

highlighted next.

chosen instead of the widely known Business Model Canvas. The

65


66

F I G U R E 6 8 \ the filled in Lean Canas


67

effect, they can always decide to buy extra expansion packs. Plus,

tion to purchase a Tuun kit. When the therapists notice a positive

pansion packs. This way, it can be more accessible for an institu-

The Tuun basic kit (see page 22) has a different price than the ex-

there is a distinction between the basic kit and expansion packs.

anymore when you decide how to do it”. In the revenue streams,

perspective of the clients could be that “Therapy won’t be boring

one’s comfort, needs or situation (fit). An alternative take from the

embodied interaction (fun) and it can be customised to every-

for everyone”. In other words, it uses a low-threshold, fun and

marised in one sentence: “(Tuun) makes social skills fun and fit

points). In general, the value proposition that is offered, is sum-

by the solution next to it (see the relationship between the three

De Grote Beek. The main problems that Tuun addresses are solved

in social and communicative disorders in GGzE in Eindhoven, at

segment, the early adopters would be the therapists specialised

quire different channels. So with the GGZ therapists as a customer

children. They are not excluded for using Tuun, but that would re-

a larger budget than private persons, e.g. parents with autistic

the best in the context of a therapy. Such institutions often have

education were chosen. The therapeutic activities of Tuun match

As a customer segment, GGZ therapists and teachers in special

\The Lean Canvas

could mean that Tuun is only suitable for places similar to GGzE.

to negotiate about a reasonable price for Tuun. If the price would be too low, it

gions of Holland. To fix this, a variety of these institutions could be contacted

not all directly financially connected, it could be a different story in other re-

budget for innovation, seen the existence of eLab. As the GGZ institutions are

differ significantly. I think larger institutions as GGzE have more space and

by more parties. Thinking about funding, budgets among institutions might

Mostly the financial part of the model is in need of more extensive verification

making a financial balance.

users. The revenue stream was tuned in to the costs structure,

both keeping existing customers interested and attracting new

expansion pack could for instance be released every six months,

it allows for extension of the Tuun collection over time. A new


\ R E F E R E N C E S \ Andersson, A. P., & Cappelen, B. (2013). Designing Empowering Vocal and Tangible Interaction. In The International conference on new interfaces for musical expression (pp. 406-412). Seoul National University. Bevilacqua, F., Fels, S., Jensenius, A. R., Lyons, M. J., Schnell, N., & Tanaka, A. (2013, April). Sig nime: music, technology, and human-computer interaction. In CHI’13 Extended Abstracts on Human Factors in Computing Systems (pp. 2529-2532). ACM. Buskermolen, D. O., & Terken, J. (2012, August). Co-constructing stories: a participatory design technique to elicit in-depth user feedback and suggestions about design concepts. In Proceedings of the 12th Participatory Design Conference: Exploratory Papers, Workshop Descriptions, Industry Cases-Volume 2 (pp. 33-36). ACM. Cappelen, B., & Andersson, A. P. (2012, July). Musicking tangibles for empowerment. In International Conference on Computers for Handicapped Persons (pp. 254-261). Springer Berlin Heidelberg. Cibrian, F. L., Pena, O., Vazquez, V., Cardenas, C., & Tentori, M. (2016, September). Designing a deformable musical surface for children with autism. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct (pp. 977-982). ACM. CRDL. (2017). CRDL. Retrieved from https://crdlt.com/crdl Dobrian, C., & Koppelman, D. (2006, June). The’E’in NIME: musical expression with new computer interfaces. In Proceedings of the 2006 conference on New interfaces for musical expression (pp. 277-282). IRCAM—Centre Pompidou. Dourish, P. (2004). Where the action is: the foundations of embodied interaction. MIT press. Dunbar, R. I. M., Kaskatis, K., MacDonald, I., & Barra, V. (2012). Performance of music elevates pain threshold and positive affect: implications for the evolutionary function of music. Evolutionary Psychology, 10(4), 147470491201000403. Frens, J. J. (2006). Designing for rich interaction: Integrating form, interaction, and function (Doctoral dissertation, Technische Universiteit Eindhoven). Keay-Bright, W., & Howarth, I. (2012). Is simplicity the key to engagement for children on the autism spectrum?. Personal and Ubiquitous Computing, 16(2), 129-141. Kempen, E. (2013). Muziek Therapeutische Bandcoaching (MTB) voor jeugdigen met een Autismespectrumstoornis (ASS). Retrieved from http:// www.lkhnederland.nl/wp-content/uploads/2013/07/MASTERTHESIS-MTBandcoaching-2013.pdf MacDonald, R. A., & Miell, D. (2002). Music for individuals with special needs: a catalyst for developments in identity, communication and musical ability. Magee, W. L., Bertolami, M., Kubicek, L., LaJoie, M., Martino, L., Sankowski, A., ... & Zigo, J. B. (2011). Using music technology in music therapy with populations across the life span in medical and educational programs. Music and Medicine, 3(3), 146-153. Mäkelä, S., Bednarik, R., & Tukiainen, M. (2013, April). Evaluating user experience of autistic children through video observation. In CHI’13 Extended Abstracts on Human Factors in Computing Systems (pp. 463-468). ACM. Matson, J. L., & Boisjoli, J. A. (2007). Differential diagnosis of PDDNOS in children. Research in Autism Spectrum Disorders, 1(1), 75-84.

68


Maurya, A. (2012). Running lean: iterate from plan A to a plan that works. “ O’Reilly Media, Inc.”. Maurya, A. (2012). Why lean canvas vs business model canvas. Available in: http://practicetrumpstheory. com/why-lean-canvas. McNiff, S. (2005). Expressive therapies: history, theory and practice. Expressive Therapies. Miranda, E. R., & Wanderley, M. M. (2006). New digital musical instruments: control and interaction beyond the keyboard (Vol. 21). AR Editions, Inc. Morgenlander, M., D’Eugenio, A., & Witan, D. (2015, June). uChoose by InteractAble: learning social skills via game play. In Proceedings of the 14th International Conference on Interaction Design and Children (pp. 403-405). ACM. Morreale, F., De Angeli, A., & O’Modhrain, S. (2014). Musical Interface Design: An Experience-oriented Framework. In NIME (pp. 467-472). Neimeyer, R. A. (1993) ‘Constructivist approaches to the measurement of meaning’. In G. J. Neimeyer (Ed) Constructivist Assessment: A Casebook London: Sage Publications Osterwalder, A., & Pigneur, Y. (2010). Business model generation: a handbook for visionaries, game changers, and challengers. John Wiley & Sons. Overbeeke, C. J., Djajadiningrat, J. P., Wensveen, S. A. G., & Hummels, C. C. M. (1999, September). Experiential and respectful. In Proceedings of the international conference Useful and Critical: the position of research and design (pp. 9-11). Paradiso, J. A. (1999). The brain opera technology: New instruments and gestural sensors for musical interaction and performance. Journal of New Music Research, 28(2), 130-149. Rolvsjord, R. (2010). Resource-oriented music therapy in mental health care. Gilsum, NH: Barcelona Publishers. Salimpoor, V. N., Benovoy, M., Larcher, K., Dagher, A., & Zatorre, R. J. (2011). Anatomically distinct dopamine release during anticipation and experience of peak emotion to music. Nature neuroscience, 14(2), 257-262. Schnell, N., & Battier, M. (2002, May). Introducing composed instruments, technical and musicological implications. In Proceedings of the 2002 conference on New interfaces for musical expression (pp. 1-5). National University of Singapore. Sievers, B., Polansky, L., Casey, M., & Wheatley, T. (2013). Music and movement share a dynamic structure that supports universal expressions of emotion. Proceedings of the National Academy of Sciences, 110(1), 70-75. Skoogmusic: Assistive Music Technology. Retrieved October 11, 2016, from http://skoogmusic.com/ Small, C. (1999). Musicking—the meanings of performing and listening. A lecture. Music education research, 1(1), 9-22. UEQ-Online. (2016). UEQ. Retrieved from http://www.ueq-online.org Wanderley, M. M., & Depalle, P. (2004). Gestural control of sound synthesis. Proceedings of the IEEE, 92(4), 632-644. Weinberg, G. (2003). Interconnected musical networks–bringing expression and thoughtfulness to collaborative music making. In Massachusetts Institute of Technology Media Laboratory. Weinberger, N. M. (1998). The music in our minds. Educational Leadership, 56(3), 36-40. Weiss, M. J., & Harris, S. L. (2001). Teaching social skills to people with autism. Behavior modification, 25(5), 785-802. Wensveen, S. A., Djajadiningrat, J. P., & Overbeeke, C. J. (2004, August). Interaction frogger: a design framework to couple action and function through feedback and feedforward. In Proceedings of the 5th conference on Designing interactive systems: processes, practices, methods, and techniques (pp. 177-184). ACM.

69


\ A P P E N D I X \

A

#include <Adafruit_NeoPixel.h> #define PIN

6

#define NUMPIXELS

16

Adafruit_NeoPixel pixels = Adafruit_NeoPixel(NUMPIXELS, PIN, NEO_GRB + NEO_KHZ800);

#include <IRLibDecodeBase.h> // First include the decode base //#include <IRLibSendBase.h>

// First include the send base

#include <IRLib_P01_NEC.h> #include <IRLib_P02_Sony.h> #include <IRLibCombo.h>

//IRsend mySender;

int amountColours = 0;

unsigned long previousMillis = 0; const long interval = 100;

//int greenState, redState = LOW; boolean state[] = {false, false, false, false}; // green, yellow, blue, red long lastPressTime[] = {0, 0, 0, 0}; int stripColour[] = {0, 0, 0};

//#include <IRLibDecodeBase.h> // First include the decode base //#include <IRLib_P02_Sony.h> //#include <IRLibCombo.h>

// to actually use. The lowest numbered // After all protocols, include this

IRdecode myDecoder;

//uint8_t codeProtocol; uint32_t codeValue;

// The data bits if type is not raw

uint8_t codeBits;

// The length of the code in bits

String code = “”; String oldCode = “”;

70

// The type of code


// Include a receiver either this or IRLibRecvPCI or IRLibRecvLoop #include <IRLibRecv.h> IRrecv myReceiver(2);

//pin number for the receiver

//Create a buffer that we will use for decoding one stream while //the receiver is using the default buffer “recvGlobal.recvBuffer” //uint16_t myBuffer[RECV_BUF_LENGTH];

void setup() { Serial.begin(9600); pixels.setBrightness(64); #if defined (__AVR_ATtiny85__) if (F_CPU == 16000000) clock_prescale_set(clock_div_1); #endif

pixels.begin(); // This initializes the NeoPixel library.

//delay(2000); while (!Serial); //delay for Leonardo //Enable auto resume and pass it the address of your extra buffer //myReceiver.enableAutoResume(myBuffer); myReceiver.enableIRIn(); // Start the receiver Serial.println(F(“Ready to receive IR signals”)); pinMode(9, OUTPUT); pinMode(10, OUTPUT); //digitalWrite(9, HIGH); digitalWrite(9, LOW); digitalWrite(10, HIGH); }

void loop() { unsigned long currentMillis = millis();

if (currentMillis - previousMillis >= interval) { previousMillis = currentMillis; //sendSignal(); //Serial.println(“Nu gaan we ontvangen”); myReceiver.enableIRIn();

//Restart receiver

//Continue looping until you get a complete signal received if (myReceiver.getResults()) myDecoder.decode();

//Decode it

71


//myDecoder.dumpResults(false);

//Now print results. Use false for less detail

codeValue = myDecoder.value; oldCode = code; code = String(codeValue, HEX); //convert from code to string Serial.println(code); receiveCodes(); setLedstrip(); //Serial.println(amountColours); } //Serial.println(code); }

int r, g, b; #define MAX_TIME 3000 // max ms between codes //long greenLastPressTime, redLastPressTime = 0;

void receiveCodes() {

if (code == “abcdefab”) { Serial.println(“GROOEEEN”); if (state[0] == false) { state[0] = true; fadeColours(); } lastPressTime[0] = millis(); //myReceiver.enableIRIn(); }

if (code == “badcfeba”) { Serial.println(“GEEEEEEL”); if (state[1] == false) { state[1] = true; fadeColours(); } lastPressTime[1] = millis(); //myReceiver.enableIRIn(); }

if (code == “bafedcba”) { Serial.println(“BLAAAUUWW”);

72


if (state[2] == false) { state[2] = true; fadeColours(); } lastPressTime[2] = millis(); //myReceiver.enableIRIn(); }

if (code == “abefcdab”) { Serial.println(“ROOOOOOD”); if (state[3] == false) { state[3] = true; Serial.println(“HEEE er is ROOOD”); fadeColours(); } lastPressTime[3] = millis(); //myReceiver.enableIRIn(); }

if (code == “ff38c7”) { if (oldCode != code) { Serial.println(“JAAAAA WE ZIJN AAN DE BEURT”);

for (int i = 0; i < 255; i++) { //Serial.println(i); for (int p = 0; p < NUMPIXELS; p++) { pixels.setPixelColour(p, pixels.Colour(i, i, i)); } pixels.show(); } } }

if ((code == “ff4ab5” || code == “ff9867” || code == “ff18e7”) && oldCode == “ff38c7”) { fadeOut(); }

if (code == “ff629d”) {

// the CH button is used to reset

reset(); }

73


}

void setLedstrip() { if (state[0] == true && millis() - lastPressTime[0] > MAX_TIME) { code = “”; state[0] = false; fadeColours(); // IF WE WANT TO FADE STUFF, PLACE FUNCTION HERE }

if (state[1] == true && millis() - lastPressTime[1] > MAX_TIME) { code = “”; state[1] = false; fadeColours(); }

if (state[2] == true && millis() - lastPressTime[2] > MAX_TIME) { code = “”; state[2] = false; fadeColours(); }

if (state[3] == true && millis() - lastPressTime[3] > MAX_TIME) { code = “”; state[3] = false; fadeColours(); }

}

void fadeColours() { amountColours = state[0] + state[1] + state[2] + state[3]; int breakLength = round(NUMPIXELS / amountColours); //Serial.println(amountColours);

if (amountColours > 0) {

// just to be sure it will never divide by 0

int r = 0; for (int i = 0; i < 4; i++) { // walk through all 4 possible colours

if (state[i] == true) { // check if the colour is being received

74


getColour(i); int beginPixel = r * breakLength; int endPixel = beginPixel + breakLength;

r++; for (int p = beginPixel; p < endPixel; p++) { pixels.setPixelColour(p, pixels.Colour(stripColour[0], stripColour[1], stripColour[2])); /*Serial.print(beginPixel); Serial.print(“ - “); Serial.println(endPixel); */ } }

}

pixels.show(); } else { for (int i = 0; i < NUMPIXELS; i++) { pixels.setPixelColour(i, pixels.Colour(0, 0, 0)); } pixels.show(); } }

void getColour(int i) { switch (i) { case 0: //rgb = “0,255,0”; stripColour[0] = 0; stripColour[1] = 255; stripColour[2] = 0; break; case 1: //rgb = “255,255,0”; stripColour[0] = 255; stripColour[1] = 255; stripColour[2] = 0; break; case 2:

75


//rgb = “0,0,255”; stripColour[0] = 0; stripColour[1] = 0; stripColour[2] = 255; break; case 3: //rgb = “255,0,0”; stripColour[0] = 255; stripColour[1] = 0; stripColour[2] = 0; break; } }

void reset() { digitalWrite(9, LOW); digitalWrite(9, HIGH);

}

void fadeOut(){ Serial.println(“NEEEE WE ZIJN NIET MEER AAN DE BEURT”); for (int i = 255; i >= 0; i--) { for (int p = 0; p < NUMPIXELS; p++) { pixels.setPixelColour(p, pixels.Colour(i, i, i)); } pixels.show(); } }

76


#include <IRLibSendBase.h> #include <IRLib_P01_NEC.h> #include <IRLib_P02_Sony.h> #include <IRLibCombo.h>

B

IRsend mySender; int ledPin = 10; int RXLED = 17;

void setup() { Serial.begin(9600); pinMode(ledPin, OUTPUT); }

void loop() { for (int i = 0; i < 3; i++) { digitalWrite(ledPin, HIGH); digitalWrite(RXLED, HIGH); TXLED1; mySender.send(NEC,0xbafedcba,0);

Serial.println(“Rood verzonden”); digitalWrite(ledPin, LOW); digitalWrite(RXLED, LOW); TXLED0; delay(25); }

delay(1925); }

77

Profile for Sebastiaan

M22 Report  

M22 Report  

Advertisement

Recommendations could not be loaded

Recommendations could not be loaded

Recommendations could not be loaded

Recommendations could not be loaded