Mark Hancock, University of Waterloo

leveraging physical actions to interact with digital surfaces mark hancock university of waterloo

downhill

back-country (telemark)

downhill bindings attached at rear

telemark bindings detatched at rear

falling over is a good thing!

providing rich multi-touch manipulation together with the appropriate visual feedback can enable virtual objects to be used in a meaningful way, as tools we can use these virtual tools to leverage physical actions to interact with digital surfaces

part i

part ii

part iii

multi-touch manipulation + visual feedback = virtual tools embodied interaction

part iv

part i: multi-touch manipulation

how do I interact with these objects? pick them up? turn them over?

how do I use them like real objects?

technique one: one-finger interaction

one finger = two DOF input dedicated areas

can use on any one-touch display

technique two: two-finger interaction

two fingers = four DOF input dedicated translate area

separate and simultaneous actions

technique three: three-finger interaction

three fingers = six DOF input dedicated fingers

comparative study

twelve participants six male, six female

three tasks passing, docking, puzzle

results

people are faster with more touches

Task Completion Time (s)

25 passing

20

docking

15 10 5 0 one-finger

two-finger

three-finger

people prefer more touches

Agree

7 6 5 4

Disagree

3 2 1 I found the technique easy to use

one-finger

the object reacted as I expected it to

two-finger

I could easily move an object to where I wanted

three-finger

Agree

7 6 5 4

Disagree

3 2 1 I found the technique difficult to control

one-finger

I found it difficult I found it difficult to to turn objects in roll objects over the plane

two-finger

three-finger

people perform many simultaneous actions

Time Spent Touching (s)

12 10 8 6 4 2 0 one-finger

two-finger

three-finger

Time Spent Touching (s)

12 10 8 6 4 2 0 one-finger

two-finger

three-finger

Time Spent Touching (s)

12 10 8

passing docking

6 4 2 0 one-finger

two-finger

three-finger

Time Spent Touching (s)

12 10 8

translations planar rotations spatial rotations

6 4 2 0 one-finger

two-finger

three-finger

people are faster with more touches people prefer more touches

people perform many simultaneous actions

observation: “why is the object turning when I just move it?�

part ii: perception

perspective projection rays

center of projection (COP)

picture plane

point of view

center of projection

perception study

factor: POV-COP discrepancy

no discrepancy

medium discrepancy

large discrepancy

factor: projection type perspective

parallel

factor: motion parallax

factor: motion parallax

no parallax

parallax

task

results

80

parallel

70

perspective

Mean Error (°)

60 50 40 30 20 10

0

motion parallax discrepancy

 none

 med

large

 none

 med

large

mean error up to 60째

part iii: virtual tools

the dawn of digital tables

application: sandtray therapy

cooperative design face-to-face meetings iterative design via distance collaboration face-to-face feedback session interviews & mock therapy

design considerations narrative potential associative medium digital extensions facilitate interpretation

feedback construction storytelling actions arrangement

part iv: embodied interaction research in the Touchlab (and other labs)

Urp (Underkoffler & Ishii, 1998)

measuring embodiment stephanie mikulecky (ucalgary) miguel nacenta (st. andrews, scotland) sheelagh carpendale (ucalgary)

cloth interaction kimberly mikulecky (ucalgary) john brosz (ucalgary) sheelagh carpendale (ucalgary)

collaborative multi-touch navigation joseph shum (uwaterloo)

lego star wars

call of duty black ops

Shear Shear

Ripple

above the surface dmitry pyryeskin (uwaterloo) jesse hoey (uwaterloo)

understanding automation betty chang (uwaterloo) stacey scott (uwaterloo)

playing with data adam bradley (uwaterloo) sheelagh carpendale (ucalgary)

downhill

back-country (telemark)

with a better understanding of human perception together with interaction that uses our hands and bodies, we can leverage physical actions to better interact with digital surfaces

students:

adam bradley betty chang arezoo irannejad rebecca langer dmitry pyryeskin joseph shum kimberly mikulecky (ucalgary)

stephanie mikulecky (ucalgary)

acknowledgements: NSERC

Games Institute SurfNet GRAND NCE

thank you mark.hancock@uwaterloo.ca