Research Methods by Atif Nagi

REHABILITATION WITH BRAIN-COMPUTER INTERFACE (BCI) SYSTEMS.

3LBS NEURO COMMUNICATION SYSTEM

Professor David Edwin Meyers IXDS6813 Final Project, Spring-Full 2016 Assistant Professor Chris Quinn Master of Arts Degree in Interactive Design Lindsey Wilson College, Columbia, Kentucky

A final project thesis submitted to the faculty of the Department of Interactive Design in partial fulfillment of the requirements for the degree of Master of Arts – Interactive Design Lindsey Wilson College.

By Atif Nagi Lindsey Wilson College, Columbia, KY May 2016

TABLE OF CONTENTS 1.0

INTRODUCTION

4.0

EXECUTIVE SUMMARY 3 1.2 CONCEPT BRIEF 4 1.3 SOLUTION 5 1.4 HYPOTHESIS 6 1.5 PROJECT OVERVIEW 7 1.1

2.0

BACKGROUND

MEDICAL RESEARCH 2.2 TECHNOLOGY RESEARCH 2.1

3.0

REQUIREMENT SPECIFICATION 23 4.2 PROJECT TIMELINE 24 4.3 MOOD BOARD 25 4.4 SKETCHES 26 4.1

5.0

SOLUTION DESIGN

DESIGN PRINCIPLES 40 5.2 USER FLOW 42 5.3 WIREFRAMES 43 5.4 INTERACTIVE PROTOTYPE 53 5.1

9 11

RESEARCH

DEMOGRAPHIC & BEHAVIOURS 3.2 COMPETITIVE ANALYSIS 3.3 SUBJECT MATTER EXPERT INTERVIEWS 3.1

EXPLORATION

17 19 20

6.0

CONCLUSION

USER CENTERED DESIGN PROCESS 64 6.2 RECOMMENDATION 65 RESOURCES 68 6.1

22

1.1

EXECUTIVE SUMMARY

Brain-computer interface research and development generates tremendous excitement in scientists, engineers, clinicians, and the general public. This enthusiasm reflects the rich promise of BCIs. They may eventually be used routinely to replace or restore useful function for people severely disabled by neuromuscular disorders; they might also improve rehabilitation for people with strokes, head trauma, and other disorders.

I am proposing to leverage brain-computer interfaces as an alternative to verbal or physical communication that would help otherwise incapacitated patients communicate basic responses with medical personnel.

33

1.2

CONCEPT BRIEF

We have a reason to believe through initial research that there is not a single real-world mass production product available for incapacitated patients to communicate with medical personnel. It is a niche market with the potential to grow the disabled healthcare industry. I see an opportunity to give back to the community and solve a real-world healthcare problem. It will be a ray of hope for the patients who are trapped inside their own bodies.

4

1.3

SOLUTION

There are a variety of medical situations in which patients become incapacitated and lose the ability to communicate even while maintaining consciousness. In most cases, it is very critical for healthcare professionals to be able to elicit responses from these patients. Physicians often need simple “yes” and “no” replies to questions related to pain, abilities, senses, etc.

Situations such as paralysis, stroke, shock and certain types of comas can completely disable a patient’s ability to communicate with first responders, medical personnel and physicians.

A headset using noninvasive brain-computer interfaces that leverage brainwave mapping and related technologies, can be used to provide simple mental feedback from patients who could not otherwise verbally or physically respond.

5

1.4

HYPOTHESIS

Leveraging a brain-computer interface (BCI) system that acquires brain signals, analyzes them, and translates them into commands that are relayed to an output device to carry out elicit responses from the people severely disabled by a neuromuscular disorder.

6

1.5

PROJECT OVERVIEW

My goal was to investigate and research the hypothesis and then propose a solution by applying user-centered design process, e.g. initial research, competitive analysis, stakeholder’s interviews, persona development, technology investigation, concept sketches, solution design, user flow, wireframes, prototype, usability tests and followup stakeholder interviews.

Based on research findings, mentioned in the document later, I am proposing 3LBS, a neuro communication system, it’s features and a digital prototype in the first phase and possible product marketing material, e.g., a website, social media presence, that would increase the inquiries about the product and lead to fundraising for the development of an actual product in 2nd phase.

7

2

BACKGROUND

8

BACKGROUND RESEARCH

BACKGROUND: MEDICAL RESEARCH 2.1

The following background research will help to understand the medical terminologies and history of the persona, which is explained later in this document 2.1.1 Locked-in syndrome Locked-in syndrome (LIS) is a condition in which a patient is aware but cannot move or communicate verbally due to complete paralysis of nearly all voluntary muscles in the body except for the eyes. Total locked-in syndrome is a version of locked-in syndrome wherein the eyes are paralyzed as well. Fred Plum and Jerome Posner coined the term for this disorder in 1966. Locked-in syndrome is also known as cerebromedullospinal disconnection, de-efferented state,

pseudocoma, and ventral pontine syndrome.

The GCS is a reliable and objective way of

(1)

recording the initial and subsequent level of consciousness in a person after a brain

2.1.2 Glasgow Coma Scale

injury. It is used by trained staff at the site of an injury like a car crash or sports injury, for

The Glasgow Coma Scale (GCS) is a

example, and in the emergency department

neurological scale that aims to give a

and intensive care units.

reliable and objective way of recording the conscious state of a person for initial as

The GCS measures the following functions:

well as subsequent assessment. A patient is assessed against the criteria of the scale,

Eye Opening (E)

and the resulting points give a patient score

4 = spontaneous

between 3 (indicating deep unconsciousness)

3 = to voice

and either 14 (original scale) or 15 (the more

2 = to pain

widely used modified or revised scale).

1 = none

GCS was initially used to assess level of

Verbal Response (V)

consciousness after a head injury, and the

5 = normal conversation

scale is now used by first aid, EMS, nurses

4 = disoriented conversation

and doctors as being applicable to all acute

3 = words, but not coherent

medical and trauma patients. In hospitals it

2 = no words, only sounds

is also used in monitoring chronic patients in

1 = none

intensive care. (2)

Motor Response (M) 6 = normal 9

BACKGROUND RESEARCH

5 = localized to pain

damage. Moderate and severe brain injuries

of the body, such as the face or the

4 = withdraws to pain

often result in long-term impairments in

hand. Generalized paralysis is a group of

3 = decorticate posture

cognition (thinking skills), physical skills, and/

conditions that affect multiple body parts.

2 = decerebrate

or emotional/behavioral functioning.

Paralysis may be temporary or permanent.

1 = none

(3)

Bell’s palsy commonly causes temporary paralysis of the face. Strokes may

2.1.3 Paralysis

temporarily paralyze one side of the body.

and the best motor response an individual

Paralysis is a loss of muscle function in part

What causes paralysis?

makes. The final GCS score or grade is the

of the body. It can be partial or complete,

sum of these numbers. Every brain injury

temporary or permanent, and localized or

Some patients are born paralyzed. Others

is different, but generally, brain injury is

generalized. Most patients feel no pain in the

develop paralysis after an accident or

classified as:

paralyzed areas of their bodies. They also

medical condition. Common causes of

have no control over how those muscles

paralysis include:

Clinicians use this scale to rate the best eye opening response, the best verbal response,

Severe: GCS 3-8 (You cannot score lower

move. Paralysis can affect any part of the

than a 3.)

body at any time in a person’s life.

Moderate: GCS 9-12 Mild: GCS 13-15

• Stroke • Spinal cord injury

Types of paralysis

• Multiple sclerosis (MS) • Cerebral palsy

Mild brain injuries can result in temporary

There are many paralysis classifications.

• Post-polio syndrome

or permanent neurological symptoms and

They describe where paralysis is located,

• Traumatic brain injury

a neuro-imaging tests such as CT scan or

how severe it is, and how long it will last.

• Neurofibromatosis

MRI may or may not show evidence of any

Localized paralysis affects only one part

• Birth defects 10

BACKGROUND RESEARCH

difficult to identify, doctors may use X-rays, A study published in 2004 showed that

CT scans, MRI scans, and other procedures.

29 percent of paralysis patients become paralyzed from a stroke. The study also

Treating paralysis

showed that 23 percent of patients become paralyzed from spinal cord injuries

Treatments for paralysis depend on the

(Christopher and Dana Reeve Foundation).

underlying cause. Examples of treatments include:

What are the symptoms of paralysis?

• Mobility aids, such as wheelchairs, braces, mobile scooters, and other devices

The symptoms of paralysis are usually

• Surgery and amputation

easy to identify. A patient will lose feeling

• Physical therapy

in a specific or widespread area. Paralysis

• Medications, such as Botox or muscle

will make a muscle difficult or impossible

relaxers for patients with spastic paralysis

to control. Sometimes tingling or numbing

(4)

sensations occur before total paralysis of a limb or muscle.

Diagnosing paralysis Diagnosing paralysis is often easy because the loss of muscle function is obvious. For internal body parts where paralysis is more

BACKGROUND: TECHNOLOGY RESEARCH 2.2

The following background research will help to understand the scope of current technology available and choices I made to find the best possible solution to design 3LBS Neuro Communication System.

2.2.1 Biofeedback Biofeedback is the process of gaining greater awareness of many physiological functions primarily using instruments that provide information on the activity of those same systems, with a goal of being able to manipulate them at will. Some of the processes that can be controlled include 11

BACKGROUND RESEARCH

brainwaves, muscle tone, skin conductance,

in conjunction with changes in thinking,

human brain activity by means of EEG.

heart rate and pain perception.

emotions, and behavior — supports desired

Berger was able to identify oscillatory

physiological changes. Over time, these

activity, such as Berger’s wave or the alpha

The definition of biofeedback was defined

changes can endure without continued use

wave (8–13 Hz), by analyzing EEG traces. (6)

through consensus by three professional

of an instrument” (5)

organizations, the Association for Applied Psychophysiology and Biofeedback (AAPB),

Betts Peters, M.A., CCC-SLP and Melanie 2.2.2 Brain–computer interface

Biofeedback Certification International

Fried-Oken, Ph.D., CCC-SLP answered most frequently asked questions (FAQ)

Alliance (BCIA), and the International

A brain–computer interface (BCI), sometimes

about BCI in the ALS association article.

Society for Neurofeedback and Research

called a mind-machine interface (MMI), direct

I am referring few points from this paper’s

(ISNR), arrived at a consensus definition of

neural interface (DNI), or brain–machine

perspective, helping to understand, current

biofeedback in 2008:

interface (BMI), is a direct communication

BCI technology strengths and limitations,

pathway between an enhanced or wired

and the choices I made to design

Biofeedback “is a process that enables

brain and an external device. BCIs are often

information architecture and user interface

an individual to learn how to change

directed at researching, mapping, assisting,

of 3LBS Neuro Communication system.

physiological activity for the purposes

augmenting, or repairing human cognitive or

Here are few of the points:

of improving health and performance.

sensory-motor functions.

What is BCI used for?

Precise instruments measure physiological activity such as brainwaves, heart function,

The history of brain–computer interfaces

breathing, muscle activity, and skin

(BCIs) starts with Hans Berger’s discovery of

BCIs can be used for communication,

temperature. These instruments rapidly and

the electrical activity of the human brain and

computer access, or control of devices

accurately ‘feedback’ information to the user.

the development of electroencephalography

such as a wheelchair or prosthetic arm,

The presentation of this information — often

(EEG). In 1924 Berger was the first to record

among other applications. Virtually anything 12

BACKGROUND RESEARCH

that can be controlled by a computer could,

the size of an external hard drive). The brain

A and count each time it flashed, or think

potentially, be controlled by a BCI. BCI is

signals that are picked up by the electrodes

“Yes!� when you saw it appear on the

being examined as a rehabilitation device to

are sent to the computer, which uses

screen. Recognizing the A would trigger a

help people re-gain motor skills that are lost

sophisticated software to translate the brain

spike in your brain signals, which would be

from stroke, as well as a prosthetic device to

signals into computer commands.

detected by the BCI system. Usually, each

replace or compensate for motor skills that will never return.

letter must be selected multiple times, so How does BCI work?

typing with a BCI is quite slow. Systems designed to control a computer cursor

What does BCI look like?

Many people imagine that BCI will allow

often rely on movement imagery. You would

them to simply think of a word or phrase

imagine squeezing your right hand to move

There are basically two types of BCI systems:

and have it appear on the screen, or control

the cursor to the right, and your left hand to

invasive and noninvasive. Invasive systems

a wheelchair by thinking about where they

move the cursor to the left.

require surgery to implant electrodes on or

want to go. Unfortunately, this is not the

near the surface of the brain. Noninvasive

case with current BCI technology. There

Can BCI technology read or control

systems cause little or no discomfort, since

are a variety of types of BCI systems, and

minds?

most use electrodes placed on the scalp,

each one works a little differently. Most BCI

usually held in place in a cap that looks

spelling systems display a series of letters,

All BCI systems require conscious effort

like a fabric swimming cap. require the use

either one at a time or by highlighting letters

from the user to type a message or move

of conductive gel, which must be wiped

in a grid. When the letter you want lights up,

a computer cursor. BCI technology will not

or washed out of the hair after use. The

your brain wave changes. The computer

read minds or share private thoughts with

electrodes, whether invasive or noninvasive,

looks for that change and interprets it as a

others. In addition, BCI cannot be used for

are connected to a computer (usually through

keystroke. For example, if you wanted to

mind control. The electrodes are simply

an additional hardware component about

type the letter A, you would focus on the

sensors that detect brain activity, and 13

BACKGROUND RESEARCH

2.2.3 EEG (Electroencephalogram)

cannot send thoughts or commands into the

BCI technology is constantly changing. The

brain. The user controls the computer, not

future will bring faster, more reliable BCI

the other way around.

systems that work for more potential users.

As mentioned in previous section that the

Hardware will improve with wireless electrode

history of brain–computer interfaces (BCIs)

connections, and with dry electrodes that

starts with Hans Berger’s discovery of the

eliminate the need for conductive gel to be

electrical activity of the human brain and

Although most people are capable of using

applied to the user’s head. Software will

the development of electroencephalography

BCI technology, it is not for everyone.

provide new options for typing and speaking

(EEG) in 1924. The following section will

Current BCI systems are challenging to use,

messages; using the internet, email, and

discuss electroencephalography (EEG)

and require expensive equipment and time-

social media; creating artwork; safely driving

in depth and affordable commercial EEG

consuming setup. People who already have a

a wheelchair; controlling door locks, light

headsets available in the market.

reliable method of controlling a computer or

switches, entertainment systems, and other

communication device (e.g., using hand, foot,

features of the home; and much more. It

An electroencephalogram (EEG) is a test

head, or eye movement) are likely to find

is hoped that the cost of BCI will decline

used to evaluate the electrical activity in

that BCI is slower and more complicated,

as it becomes more widely used, and that

the brain. Brain cells communicate with

and simply not worth the hassle. BCIs will

insurance carriers will eventually cover

each other through electrical impulses. An

be most beneficial for people who have little

it as an assistive technology. Since this

EEG can be used to help detect potential

or no reliable muscle movement, including

technology is being developed for a much

problems associated with this activity.

some people with advanced ALS.

larger market than those with neurological

Who can use BCI?

What does the future hold for BCI technology?

disease, in the future we will see BCI devices

The test tracks and records brain wave

available for many computer functions. (7)

patterns. Small, flat metal discs called electrodes are attached to the scalp with wires. The electrodes analyze the electrical 14

BACKGROUND RESEARCH

impulses in the brain and send signals to a

• Sleep disorders

these games would not offer much in terms

computer, where the results are recorded.

• Strokes

of game-play. What, after all, can you do

The electrical impulses in an EEG recording

• Dementia

with just one button. And yet, in spite of

look like wavy lines with peaks and valleys.

this severe limitation, one-button games

These lines allow doctors to quickly assess

There are no risks associated with an EEG.

can have surprisingly complex game-play.

whether there are abnormal patterns. Any

The test is painless and safe. When someone

Not only that, some one-button games, e.g.

irregularities may be a sign of seizures or

has epilepsy or another seizure disorder, the

Poto and Cabenga, can offer a very stiff

other brain disorders.

stimuli presented during the test (such as a

challenge to gamers.

flashing light) may cause a seizure. However, An EEG is used to detect problems in the

the technician performing the EEG is trained

This ability to unleash complex and

electrical activity of the brain that may be

to safely manage the situation should this

challenging game-play using the simplest of

associated with certain brain disorders. The

occur. (8)

controls is what makes one-button games

measurements given by an EEG are used to confirm or rule out various conditions,

so magical and fun to play. The top five 2.2.4 One–button game interface

including:

online games are G-Switch, Arctic Blue, One Button Bob, Poto and Cabenga, Flabby

One button games have incredibly simple

Physics. (10)

• Seizure disorders (such as epilepsy)

gameplay, like a puzzle game that can be

• A head injuries

played entirely using a one-button. It is

One-button is the limitation on

• Encephalitis (an inflammation of the brain)

amazing how much fun you can have with

interaction. It involves low-level

• A brain tumors

games that use just one button.

mechanics of basic actions. I explored

• Encephalopathy (a disease that causes

There is something almost magical about

this limitation of interaction in the basic

brain dysfunction)

one-button games. You would have thought,

architecture of design solution.

• Memory problems

given the simplicity of their controls, that 15

3

RESEARCH

16

RESEARCH

3.1

DEMOGRAPHIC & BEHAVIOURS

After initial research of incapacitated patients’ case studies, medical conditions, available technology’s strength, and limitation I started to focus on demographics and behaviors, persona development, competitive analysis, and subject matter expert interviews 3.1.1 Primary target audience

to be able to elicit responses from these

3.1.3 Stakeholders

patients. Physicians often need simple “yes” and “no” replies to questions related to pain,

Stakeholders are healthcare personnel,

abilities, senses, etc. These patients can be

e.g., physicians, nurse, EMS, and caregiver

trained to use the headset and communicate

staff, who are going use 3LBS Neuro

with healthcare personals.

Communication System along with the patients.

3.1.2 Secondary target audience The secondary primary target audiences are male and female short-term incapacitated patients with the following background: Short-term incapacitated patients have lost the ability to communicate somewhat due

The primary target audiences for this product

to a shock of a medical condition. EMS

are male and female long-term incapacitated

personnel often need simple “yes” and “no”

patients with the following background:

replies to questions related to pain, abilities, senses, etc. Due to emergency situations,

Long-term incapacitated patients, who

these patients cannot be trained to use the

lose the ability to communicate even while

headset and communicate with healthcare

maintaining consciousness. In most cases,

personals.

it is very critical for healthcare professionals 17

RESEARCH

3.2 Primary persona Name: Anton Bruce Age: 39 Occupation: Database developer Neurological disorder diagnosis: Total locked-in syndrome Location: Cleveland, OH

Technology IT and Internet Software Mobile Apps Social Networks

Anton is a 39 years old Database Administrator (DBA) of a multinational retail chain in Cleveland. Recently he has been diagnosis with Total Locked-in Syndrome by the panel of neurologists from a leading Neurological Institute in Cleveland, which offers the full range of neurological and neurosurgical services.

Anton is a victim of a severe stroke. Recently he came out of a four-week deep coma, unaware of what happened to him, what was going on around him, and where he was. He is in a state of total paralysis, no control over his body, eyes wide open, incapable of any verbal communication, in what is known in the medical community as “Total locked-in syndrome.” The syndrome is typically caused by a lesion in the pons, effectively the part of the brainstem that acts as a bridge between brain and body.

The doctors have confirmed that Anton’s cognitive, auditory and visual capabilities are intact with his brain by applying auditory, and visual evoke potential (VEP), electroencephalography (EEG), electromyography (EMG), and clinical tests.

It’s like living his worst nightmare. Anton only has his thoughts for company and nothing to punctuate his day, which feels endless. There is the indignity

Motivations • To communicate with the healthcare professional and family.

• Fight back and recover from his disability.

Frustrations Anton’s brain is aware, alert and functional. He can listen to physicians, family and friends but he cannot tell anyone this. He is locked-in his body, struggling and willing to recover but doesn’t know how to communicate.

of being unable to do anything. He can feel pain and touch, but he cannot communicate or express himself. It is like paralysis with feeling.

It is very critical for his recovery and survival to somehow communicate with physicians, even simple “yes” and “no” replies to questions related to pain, abilities, senses, etc.

18

RESEARCH

3.3

COMPETITIVE ANALYSIS

Qualitative research “interviews” and online searches were used to uncover strengths and weaknesses as well opportunities and

3.3.2 Tier 2 competitors

3.3.4 SWOT

There are multiple tier 2 researchers.

A quick SWAT analysis of the 3LBS Neuro

Researchers are using brain-computer

Communication System.

interface technology to help patients, who are unable to perform basic activities such

Strengths

as walking, talking, typing or writing due to

Simple and double product development.

severe brain disorders caused by paralysis,

threats.

stroke, shock and certain types of coma, e.g.

Weaknesses

robotics, wheelchair, communication, and

Patient’s learning curve.

3.3.1 Tier 1 competitor

hearing aid devices.

Opportunities

After online search and interviews with

3.3.3 Key dimensions

Arshad Ali, MD, a neurosurgeon, It was

It’s a niche market and an opportunity to enter and grow quickly.

determined that there is no direct rival BCI

Competitors were analyzed based on based

product of 3LBS Neuro Communication

on the following dimensions, which will be

Threats:

System. There is some evidence of early

critical factors to the start-up of the program:

1. To find the funding to research and

research and technology projects, but there

develop the product.

is not a single commercial product available

• Sector

2. To enter the healthcare market.

in the healthcare industry to communicate

• Technology

3. Government compliance.

with total locked-in syndrome patients.

• Status (Research vs. Real-world product) • Availability • Cost 19

RESEARCH

SUBJECT MATTER EXPERT INTERVIEWS

Dr. Ali is a Neurosurgeon and expert in

and AML diseases.

Neurovascular diseases. Currently, he is

• Non-clinical test:

a consultant and Head of Neurosurgery,

- Speech (Sensation, receive, process,

Sibu General Hospital (Ministry of Health-

execution)

Malaysia), Sibu Sarawak, East Malaysia. He

- Hearing (Auditory evoked potential)

3.4.1 Dr. Arshad Ali, Neurosurgeon and Neuro-interventionist, Head of Department, The Ministry of Health Malaysia.

is gazetted as Specialist Neurosurgeon with

- Vision (Visual evoked potential)

core credentials to operate as Neurospinal

- Cognitive tests

Neurosurgeon including all in-patient

• Clinical tests: EEG/MRI

and out-patient privileges. As per latest

• Glasgow Coma (A neurological scale)

hospital statistics about Neurosurgery,

• Brain conscious levels

3.4

total admissions are 1209, outpatient 985, emergency surgeries 302 & Elective 91.

I spoke to Dr. Ali multiple times. He helped me understand the basic concepts of the

The above items are described in background research section in detail.

brain, common diseases, and diagnosis.

3.4.2 Duane Cash, Inventor / Innovator of Mind-Controlled Tech.

• Brain structure

Duane Cash is a developer, innovator and

• Vegetative state (Unresponsive wakefulness

inventor. He obtained a Master of Fine

syndrome)

Arts Degree in Interactive Design and

• Locked-in syndrome

Game Development from the Savannah

• Total locked-in syndrome

College of Art and Design in 2013. His

• Pathology of common neurovascular

thesis was on “Mind-controlled Systems for

diseases: Globalpuasiya, Myopathies, Troma, 20

RESEARCH

economical available headset Neurosky

way.

Mindwave and experimenting with the rest of devices at a later stage. Here is the summary

I hope some of the items I mentioned help

of our communication:

you in your decision. In my experience, I started with the MindWave Mobile, then

“Although I am recommending the Neurosky

bought the others later, and eventually

headset for this project, I do also recommend

returned to the MindWave Mobile because it

that if your budget allows later, you should

provided what I needed. You might eventually

also eventually pick up the Emotiv Insight,

find that the Emotiv Insight is a good option.

the Melon headset, and Interaxon’s Muse

If you are wishing to show a demo with

headset. These headsets will also give you

more impact, the Emotiv Insight has an

better insight into how brainwave patterns

elegant look that will win the attention of

propagate and might help you better

the audience. This is the headset that I do

understand the interaction between thoughts,

not yet have; I have the EPOC, but still the

Deeper Cognitive Connections with Virtual

speech and actions. Every thought can be

Neurosky is doing the job fine. The key is

Assistants.” He is an interactive designer

mapped to a verbal equivalent. Some EEG

really not in the headset chosen, but in the

who enjoys creating innovative applications

experts will try to dismiss the idea that one

software and how it is written. After all, your

for technology-driven companies that make a

can find anything in a single RAW EEG and

project is looking for a “yes” and “no” and

difference in the lives of others.

they will even say that having the Delta,

not for multiple commands or phrases like

Alpha, Beta, Gamma, and other patterns is

the ThynkWare Speech. The Neurosky can

I discussed the scope of EEG headset

still not sufficient to translate into a verbal

do it.”

products to proof the concept for the final

intent. They just need to look at the patterns

project. Duane advised starting with most

deeper and think about them in a different

Duane Cash, March 1, 2016.

21

4

EXPLORATION

22

EXPLORATION

4.1

REQUIREMENT SPECIFICATION

4.1.1 Requirement specification I planned to design a software application, 3LBS Neuro Communication System, utilizing a brainwave mapping headset and onebutton game philosophy.

Phase 1 - Final Project, Spring-Full 2016

• Dashboard/Home

Phase 3 (Expansion)

• Emergency • Talk to caregiver (Yes, no, and not sure)

Once the original functionality of the software

• Room comfort (Lights, thermostat, bed,

is nailed down for the primary persona,

and TV)

start work on other personas to add more functionality, e.g. power user, patients with

Phase 2 - Product prototype and

some motor control to expand the market.

marketing material

Here are a few ideas for future expansion:

Once the concept and digital prototype

• Emergency: Pinpoint pain area e.g.

are finished, start working on building the

highlight different body parts.

physical prototype in the second phase after

• Talk to caregiver: Highlight a help topic e.g.

graduation.

pain, food, bath, love, etc.

• Room comfort (Add more functionality

I applied user centered design process in Phase 1, Final Project, Spring-Full 2016 i.e. initial research, information architecture, conceptual sketches, user flow, medium fidelity wireframes, usability research, high fidelity wireframes, and digital prototype.

• Technology introduction • Configuration • Practice

• Develop prototype software and hardware.

e.g. room appliances, books, TV channels,

• Primary and secondary persona usability

movies, music, etc.)

research.

• Instant messaging (IM): Talk to a friend or

• Improve UI and functionality.

family.

• Fundraising for product development.

• Introduce more than one cognitive

• Traction channels: Bulls’ eye framework

command for faster navigation.

• Social media presence e.g. Facebook,

• Introduce muscle controls commands for

Pinterest, Linkedin

patients with some motor control e.g. blink

• Open conferences and seminars

and eyebrow movement.

• Apply for patent and explore legal aspects.

23

EXPLORATION

4.2

PROJECT TIMELINE

Flow and architecture | W6 Site maps | W6 User journey | W6 Sketches | W7 Written draft 2 | W7

Thesis statement | W2 Abstract | W2 Written outline | W3 Task timeline | W3 Visual timeline | W3

WEEK 2

WEEK 3

Recommendation | W14 Written paper 2 | W14

DESIGN EXPLORATION

PLAN WEEK 1

PROJECT TIMELINE

WEEK 4

WEEK 5

WEEK 6

RESEARCH Initial research | W4 Competitive analysis | W4 Persona | W4 Stakeholder interviews | W5 Mood board | W5 Recommendation | W5 Written draft 1 | W5

WEEK 7

WEEK 8

WEEK 9

CONCLUSION WEEK 10

WEEK 11

WEEK 12

DESIGN SOLUTION Wireframes | W8 Prototype | W9 - W12 Usability testing | W13 Written paper 1 | W13

WEEK 13

WEEK 14

WEEK 15

WEEK 16

DOCUMENT Design document | W15 Presentation | W16

24

EXPLORATION

4.3

MOOD BOARD

VIEW ONLINE

25

EXPLORATION

4.4

SKETCHES

left, right, forward and backward) plus one

important that each user masters the use of

additional action that exists only in the realm

the Mental Commands detection one action

of the user’s imagination: disappear.

at a time, only increasing the number of concurrent actions after he has first gained

4.4.1 Initial concepts Mental Commands allows the user to choose

confidence and accuracy with a lower

After researching the medical history

up to 4 actions that can be recognized at any

number of actions. (13)

of persona, available technology, user

given time. The detection reports a single

interfaces, BCI, EEG headsets and output

action or neutral (i.e. no action) at a time,

devices. I sketched initial concepts of BCI,

along with an action power which represents

how raw EEG signals will go to processing

the detection’s certainty that the user

application and output to devices for

has entered the Mental Commands state

meaningful results.

associated with that action.

The training application suite is based Emotiv Increasing the number of concurrent actions Xavieron SDK. The Mental Commands

increases the difficulty in maintaining

detection suite evaluates a user’s real time

conscious control over the Mental Command

brainwave activity to discern the user’s

detection results. Almost all new users

conscious intent to perform distinct physical

readily gain control over a single action quite

actions on a real or virtual object. The

quickly. Learning to control multiple actions

detection is designed to work with up to 13

typically requires practice and becomes

different actions: 6 directional movements

progressively harder as additional actions

(push, pull, left, right, up and down) and 6

are added. Although Emotiv Xavier allows a

rotations (clockwise, counter-clockwise,

user to select up to 4 actions at a time, it is 26

EXPLORATION

4.4

SKETCHES

27

EXPLORATION

4.4

SKETCHES

28

EXPLORATION

4.4

SKETCHES

29

EXPLORATION

4.4

SKETCHES

4.4.2 Second concept iteration During this phase, I finally named the product 3LBS Neuro Communication System. I started exploring training suite.

Introduce multiple users i.e. novice, skilled and power user. Novice users have to learn only one cognitive action, two-three for skilled, and 13 for power users.

30

EXPLORATION

4.4

SKETCHES

31

EXPLORATION

4.4

SKETCHES

32

EXPLORATION

4.4

SKETCHES

33

EXPLORATION

4.4

SKETCHES

34

EXPLORATION

4.4

SKETCHES

4.4.3 Third concept iteration I applied 20-80 rule to cut down the unnecessary options and focused on persona needs. The 3rd iteration is based on one-button interface. One button is the limitation on interaction. It involves lowlevel mechanics of basic actions. I explored this limitation of interaction in the basic architecture of design solution.

Almost all new users readily gain control over a single action quite quickly. Users can find important information through only one cognitive command, intuitive navigation affordance, which gives consistency and predictability to the users. *15 min learning curve as per Classification of EEG Signals in a Brain- Computer Interface System thesis research by Erik Andreas Larsen, Master of Science in Computer Science, Norwegian University of Science and Technology Department of Computer and Information Science.

35

EXPLORATION

4.4

SKETCHES

36

EXPLORATION

4.4

SKETCHES

37

EXPLORATION

4.4

SKETCHES

38

5

SOLUTION DESIGN

39

SOLUTION DESIGN

5.1

DESIGN PRINCIPLES

I applied multiple design principles during solution design. Here are some of the leading principles:

CLEAR

AFFORDANCE

CONSISTENT

The information of the system is straightforward & understandable with clear one-button navigation, simple language (labels), and icons.

I used explicit affordance signaled action buttons by language, negative affordance in the menu and contextual menu, metaphor affordance by using real-world menu icons as a metaphor.

The consistent use of one-button navigation, labels, language, colors, icons and user interface patterns.

FOCUS

ACCESSIBILITY

WAYFINDING

The user interface is singular and focused. It is intuitive because it focuses on core tasks of persona e.g. emergency, talk to a caregiver, and room comfort.

Appliedd W3C standards, e.g., high contrast, B/W design, Font size readability and visual hierarchy.

Users can find their way by landmark icons, well-structured paths, and fewer choices. They know where they are, what they can do, where they are going and where they’ve been through navigation.

40 40

SOLUTION DESIGN

5.1

DESIGN PRINCIPLES

Visibility of system status

10 Usability heuristics for user interface Aesthetic & minimalist design

design I applied Jakob Nielsen’s 10 heuristic

Match real world

principles for interaction design during design and evaluation of user interface.

Help and documentation

Recover from errors

HEURISTICS EVALUATION

Flexibility and efficiency of use

Recognition rather than recall

User control and freedom

Consistency and standards

Error prevention

41 41

SOLUTION DESIGN

5.2

USER FLOW

VIEW ONLINE

The user flow is based on Nielsen’s heuristics e.g. user control and freedom, consistency and standards, Error prevention, Flexibility and efficiency of use, Help users recognize, diagnose, and recover from errors. This user flow considers user’s happy path and also when things go wrong for the user, how to prevent and recover from errors.

42

SOLUTION DESIGN

5.3

WIREFRAMES

the duration of the training period (currently 8

by supplying consistent training data (i.e. a

seconds).

consistent Mental Commands visualization on the part of the user) across several training sessions for each enabled action.

I started converting low-fidelity sketch

Initially, the cube on the screen will not move, as

concepts to medium-fidelity wireframes

the system has not yet acquired the training data

by adding more screens and details based

necessary to construct a personalized signature

A training session is automatically discarded if

on user flow. I am showcasing high-level

for the current set of actions. After Neutral

the wireless signal strength or EEG signal quality

wireframes in this document. To view every

and each enabled action have been trained at

is poor for a significant portion of the training

interaction screen of wireframe, please click

least once, the Mental Commands detection

period. A notification will be displayed to the user

on the prototype button in the next section.

is activated, and the cube will respond to the

if this has occurred. (13)

Mental Commands detection, and your Mental The Mental Commands training process

Commands control, in real time.

Xavieron SDK cognitive suite.

enables the system to analyze brainwaves and develop a personalized signature which

*The training application suite is based Emotiv

Some users will find it easier to maintain the

corresponds to each particular action, as well necessary Mental Commands focus if the cube is automatically animated to perform the intended as the background state, or “neutral�. As the system learns and refines the signatures

action as a visualization aid during training. It is

for each of the actions, as well as neutral,

also important that the system gives feedback to

detections become more precise and easier

the user for visibility of system status.

to perform.

Finally, users are prompted to accept or reject During the training process, it is very

the training recording. Ideal Mental Commands

important to maintain cognitive focus for

detection performance is typically achieved 43

SOLUTION DESIGN

5.3

WIREFRAMES

Intro + configuration mode

44

SOLUTION DESIGN

5.3

WIREFRAMES

Configuration mode

45

SOLUTION DESIGN

5.3

WIREFRAMES

Practice mode

46

SOLUTION DESIGN

5.3

WIREFRAMES

Practice mode + emergency command

47

SOLUTION DESIGN

5.3

WIREFRAMES

Help + talk command

48

SOLUTION DESIGN

5.3

WIREFRAMES

Comfort + light Command

49

SOLUTION DESIGN

5.3

WIREFRAMES

Thermostat command

50

SOLUTION DESIGN

5.3

WIREFRAMES

Bed Command

51

SOLUTION DESIGN

5.3

WIREFRAMES

TV + back to home command

52

SOLUTION DESIGN

INTERACTIVE PROTOTYPE 5.4

LAUNCH PROTOTYPE

The Interactive prototype was made in AxurePR. I added missing screens and interactions to get the interactive prototype ready for usability testing with subject matter expert evaluators. *Click on the image to view in a browser.

53

SOLUTION DESIGN VALIDATION RESEARCH

5.5

USABILITY TESTING

Dr. Arshad Ali

Dr. Fariha Jamshed

Dr. Aisha Malik

Duane Cash

Khalil Laghari

MD (Neurosurgeon and Neuro-interventionist)

MBBS, DMRD (Radiologist)

MD (Neurologist)

Innovator of MindControlled Tech.

Doctor of Philosophy (Ph.D.), HCI

100% 3LBS Neuro Communication System is a practical and doable project. I would encouraged you to take this project to next level.

89%

79%

100%

89%

I like the simplicity and learnability of the interface.

3LBS a promising device for physically challenged patients.

I like the simplicity and The interface is selfexplanatory and intuitive.

There was an intial learning curve but at the end, it totally worked.

This project is absolutely doable and has practical value. Carousel navigation and one push command make a less cognitive load on a human brain.

Ease of use, simplicity, the minimalism of interface and cognitive functionality are the strongest features of 3LBS Neuro Communication System.

Usability test results

54

SOLUTION DESIGN VALIDATION RESEARCH

5.5

USABILITY TESTING

spinal cord, and the peripheral nerves and/

5. Add more details about each section

or have experience in brain-computer

to set users expectation.

interaction (BCI).

6. Fix back to home screen.

*A detailed usability procedure is described in

5.5.4 Future research

5.5.1 Usability research goals To determine if there were any significant usability issues. Research should identify usability pain points.

final project reserach paper.

5.5.3 Outcomes

Further usability research is required in future, after product prototype is built in phase 2, with primary persona (Total

5.5.2 Research methods and usability procedures The research method was remote usability

I summarized evaluators’ feedback and

locked-in syndrome long-term patient)

recommend to apply following improvements

and secondary (Situational incapacitated

to visual design phase:

patients).

interviews by hybrid task and observational

1. Create a storyboard for the demo video.

research method.

It should explain the product features through animation and personal demo video

The evaluator selection criteria were proxy

recording.

subject matter experts i.e. Physicians, neuro-

2. Add supporting animation in hi-fidelity

interventionist, psychiatrist, and BCI experts.

wireframes.

3. Add feedback progress bar in practice

Subject matter expert evaluators must have

mode.

previous experience with diseases and

4. Add restart button on restart feedback

conditions affecting the nervous system,

screen to empower the user to restart

which includes the brain, the spine, and

training instead of system dependent.

55

SOLUTION DESIGN

5.6

HI-FI WIREFRAMES

LAUNCH PROTOTYPE

I applied usability research recommendation to visual design high-fidelity wireframes.

Introduced colors and animation treatment.

56

SOLUTION DESIGN

5.6

HI-FI WIREFRAMES

Intro screens

57

SOLUTION DESIGN

5.6

HI-FI WIREFRAMES

Configuration screens

58

SOLUTION DESIGN

5.6

HI-FI WIREFRAMES

Practice screens

59

SOLUTION DESIGN

5.6

HI-FI WIREFRAMES

Practice and emergency screens

60

SOLUTION DESIGN

5.6

HI-FI WIREFRAMES

Talk to caregiver and talk screens

61

SOLUTION DESIGN

5.6

HI-FI WIREFRAMES

Comfort screens

62

6

CONCLUSION

63

CONCLUSION

USER CENTERED DESIGN PROCESS 6.1

A full user centered design process was explored in the 3LBS Neuro Communication System, Brain-computer Interface project from research, ideation, exploration, define, 1. Research & ideation

design, prototype and usability validation.

2. Persona development

The three-cycle interactive process with paper and interactive prototype usability validation

refined

the

product.

After

multiple iterations, I achieved effectiveness, efficiency, and satisfaction through user flow interactions, intuitive UI, labels, and icons of the user interface, which was supported and

3.3

COMPETITIVE ANALYSIS

Qualitative research “interviews” and online searches were used to uncover strengths and weaknesses as well opportunities and threats.

validated by subject matter experts’ usability

3.3.1 Tier 1 competitor

evaluations.

After online search and interviews with Arshad Ali, MD, a neurosurgeon, It was determined that there is no direct rival BCI

3. Competitive research and technology projects, but there analysis product of 3LBS Neuro Communication

System. There is some evidence of early

is not a single commercial product available in the healthcare industry to communicate with total locked-in syndrome patients.

4. Reserach methods application 64

CONCLUSION

5. User flow

6. Concept evolution

7. Design evolution

Multiple iterations

Multiple iterations

6.2

RECOMMENDATION

Based on the research data found for the proposed hypothesis through professional insight, in

experience, competitive analysis, and industry expert interviews, I recommend building this pro

All subject matter experts including a neurosurgeon, neurologist, brain-computer interaction

experts has consensus that “3LBS Neuro Communication System is a practical and doable pro They encouraged me to take this project to next level.

Seize this project to next level and finally into the global market, I will need to build on the res

with a formal business analysis to address financial considerations to build the product prot polish final product, legal aspects, and marketing efforts.

8. Concept & design validation

9. Hi-Fi design evolution

Usability testing

Multiple iterations

10. Research recommendations

The research data exhibits that there is a market need for 3LB Neuro Communication System

healthcare industry and It will be a ray of hope for the patients who are trapped inside their own

65

CONCLUSION

6.2

RECOMMENDATION

Based on the research data found for the proposed hypothesis through professional insight, industry experience, competitive analysis, and industry expert interviews, I recommend building this product.

All subject matter experts including a neurosurgeon, neurologist, brain-computer interaction (BCI) experts has consensus that “3LBS Neuro Communication System is a practical and doable project�.

They encouraged me to take this project to next level.

Seize this project to next level and finally into the global market, I will need to build on the research with a formal business analysis to address financial considerations to build the product prototype, polish final product, legal aspects, and marketing efforts.

The research data exhibits that there is a market need for 3LB Neuro Communication System in the healthcare industry and It will be a ray of hope for the patients who are trapped inside their own body.

66

7

RESOURCES

67

RESOURCES

1. “Locked-in Syndrome.” Wikipedia. Wikimedia Foundation. Web. 07

7. “Brain-Computer Interface (BCI).” ALSA.org. Web. 14 Feb 2016.

Feb. 2016. <https://en.wikipedia.org/wiki/Locked-in_syndrome>.

<http://www.alsa.org/als-care/resources/publications-videos/ factsheets/brain-computer-interface.html>. Betts Peters, M.A., CCC-

2. “Glasgow Coma Scale.” Wikipedia. Wikimedia Foundation. Web.

SLP and Melanie Fried-Oken, Ph.D., CCC-SLP

07 Feb. 2016. <https://en.wikipedia.org/wiki/Glasgow_Coma_Scale>.

8. “EEG (Electroencephalogram).” Healthline. Web. 14 Feb. 2016.

3. “What Is the Glasgow Coma Scale?” Brainline. Web. 08 Feb. 2016.

<http://www.healthline.com/health/eeg>.

<http://www.brainline.org/content/2010/10/what-is-the-glasgowcoma-scale.html>.

9. “8 Mind-blowing Gadgets You Can Control Just With Your Brain.”

Hongkiatcom. Web. 20 Feb. 2016. <http://www.hongkiat.com/blog/

4. “Paralysis.” Healthline. Web. 08 Feb. 2016. <http://www.healthline.

brain-controlled-gadgets/>.

com/health/paralysis>.

10. “Best One Button Games.” Best One Button Games. Web. 07 5. “Biofeedback.” Wikipedia. Wikimedia Foundation. Web. 14 Feb.

Mar. 2016.

2016. <https://en.wikipedia.org/wiki/Biofeedback>.

11. “10 Heuristics for User Interface Design: Article by Jakob 6. “ Brain–computer interface.” Wikipedia. Wikimedia

Nielsen.” 10 Heuristics for User Interface Design: Article by Jakob

Foundation. Web. 14 Feb. 2016. <https://en.wikipedia.org/wiki/

Nielsen. Web. 20 Mar. 2016.

Brain%E2%80%93computer_interface>.

68

RESOURCES

12. Krug, Steve. Rocket Surgery Made Easy: The Do-it-yourself Guide to Finding and Fixing Usability Problems. Print.

13. “Understanding the Mental Commands Detection Suite” Emotiv. Web. 20 Feb. 2016. <https://emotiv.zendesk.com/hc/enus/articles/201483705-Understanding-the-Mental-CommandsDetection-Suite>.

14. “Classification of EEG Signals in a Brain-Computer Interface System.” Classification of EEG Signals in a Brain-Computer Interface System. Web. 10 Apr. 2016.

15. Infofree8. “Emotiv.mp4.” YouTube. YouTube, 2010. Web. 25 Mar. 2016. <https://www.youtube.com/watch?v=r2ip_84XRjA>.

16. Gsmarena07. “Emotiv EPOC Heads-on.” YouTube. YouTube, 2012. Web. 25 Mar. 2016. <https://www.youtube.com/ watch?v=LZrat-VG4Ms>.

17. DigitalOmni. “4.5 Year Old Boy Plays with EPOC Emotive.”

YouTube. YouTube, 2010. Web. 26 Mar. 2016.

69