Page 1

General Theological Seminary ​I'm not sure if any of you were here for the original VR World Congress when it was actually called Southwest VR in this very room so we return to our roots a speaker took to the stage and showed an amazing technology which then caused that immediate Stampede his booth afterwards which almost throughout the conference it's an amazing technology but I guess now I won't blow what it is I let him do that last time we saw it it was a working example should we say and it's become increasingly tangible so from ultra haptics I just want to welcome ya how if you if you don't know what all caps because is you are in for a you're in for a tree and if you do know you should be excited for the return of I picked you up enough now Tom Tom Carter Fran also altra haptics okay good thanks and hello everybody so yeah i'm tom from ultra optics we have a technology that lets you feel things in the air without touching anything so we did cause a little bit of a stampede last time they've pushed us back and given us a bit more space for the demos this year that i'll talk to you about a bit later but what I really want to talked about today is the importance of the sense of touch in VR what you get from the sense of touch and why you're going to need it when creating and really cool immersive VR experiences so I sat down to think about creating this presentation how do I say it how do I pitch it and the starting point has to be well surely the sense of touch increases the immersion of VR makes you feel more like it you're really there makes it feel more real and it turns out that's actually pretty easy to prove even without some of the amazing visual effects that we just saw all you need is a very quick youtube search and you can prove that that's different oh oh oh what is wrong with you I spent a long time watching these videos so that you can even find these all over youtubes just search for jump-scare videos of people envy are doing horror games and then somebody grabs them when the outside the best when they really climb it well so something jumps out at you in the game and you actually get grabbed and that moment really takes it from you sort of raised heart rate being scared in a game but still in the back of your mind knowing it's not real - all of a sudden losing that safety net that feeling that it's not that it really isn't really just said in your living room and it makes it real and that that that sort of breaks that that safety net that you have so it's kind of interesting but it's really a little bit of a cheap shot and and kind of actually undervalues the real importance of touch and what you get from it so to delve into that I've got to go back a number of years to when I was growing up and this is the first mobile phone that I got nokia 3310 and so I got one of these and it completely revolutionized my life I was now able to text and call my friends I could play snake and this was was sort of completely changed my world but when I got it I took it out of the box it then actually comes with a real a lot of learning of just how to use it as an interface so this is actually a screenshot from I believe it's page 134 of the instruction manual for the Nokia 3310 that shipped in the box I found that I got it out I read it again it's really good surprise it's not become a classic and so to lock the phone you press the bottom of the blue line which is here is the Menu button and then the star button really quickly but then to turn the lock off you press the line with a blue the bottom of the blue line which is now called the unlock button not the Menu button and then you press the star button very quickly again if you want to get to your speed dial settings you press the Menu then you press 6 and then you press 1 there was a lot of this sort of effort to learn what the shortcuts were on how to use the phone because of the interface it's a substantive array of buttons and we've learnt that for smart phones and we've progressed it to touchscreen so this into basic actually change and help you learn and understand what what it is you're supposed to be doing but we're looking at a relatively similar challenge that's facing as in virtual reality VR is way more complicated than a Nokia 3310 and so the challenge of how do we interact in VR and how do we how do we operate in there is huge in something we really need to put a lot of focus onto and of course VR is not just about controls or pressing buttons or moving dials it's a real free space that allows you to interact and move around and and do all sorts of things so we can't have these systems we're actually sort of tethers you to one place and the feedback that you get it's got to be really fast and responsive but also VR is able to create very powerful very human experiences and and you can get to the stage where you can actually believe that the persons that opposite you is really there and some of the graphics you just saw if you're still here in the previous talk showing the difference in graphics between the movie and the rendered version now is unbelievable stunning and so you're getting to this place

of having photorealistic people and when you get to these scenarios you're going to want to reach out and touch and interact with your hands whether that's objects in the environment whether that's people and if you have a person they're generally a lot of people you will reach out and touch and interact if you've ever sat in the arrivals lounge in an airport and watch people getting off their planes to meet somebody every time they meet they actually touch in some way shake their hands Pat them on the back give them a hug if it's a family member so if we don't have these kind of this interaction on the center touch then effectively we're going to become that awkward guy at the airport who meet somebody just going to go hey and then doesn't really do the handshake and we can see this is true because I'm sure pretty much everybody here has seen this experience when somebody's in VR reaching out and trying to touch the virtual stuff and you can see this happens so often there's something there they're amazed by it the brain sort of believes it and they go like this they know they can't actually touch it and feel it but people still try and really that got me thinking why do people do that why when you know you've got a headset on your why do you still go like this and try and touch it and even if it was a real physical object why would you reach your hand out and try and grab it and the answer is actually that what touch is really important just for exploring and understanding the world around us so we've got a challenge for you here's a spot the difference game there are two pictures see if you can see the difference between the two pictures so the answer is that the image on the right hand side they're the guys holding the glass and the image on the left hand side he's not actually touching it at all so this hand is completely not touching the glass and that one is actually grasping it and can lift it up and the reason you can't tell the difference is because as humans our eyes are stuck to the front of our head about this far apart and so when you reach your hand out to pick something up perspective kicks in and you can't tell the difference between small movements forwards or backwards or side-toside and you'd need eyes on stalks to come and look from different angles and get a wider aperture so just for the basic interactions of picking things up and moving them around we're entirely reliant on our sense of touch to know how they contacted it should I close my hands will it come with me if I pick it up and in the real world this is the difference between having a glass of water and knocking it all over the table but it's not just to compensate for your loss of visual accuracy at distance also your brain actually processes touch one point seven times faster than it processes visuals so if you show somebody or give somebody a piece of information through visuals it's not far off twice as difficult and in time and also sort of mental load for them to process and digest that information and work out what it is so we actually rely on touch just for this really quick and information feedback to us while we're just moving around in the space and then also the touch is really important for human communication and building trust and collaboration so there's a there's a study that's been conducted on the NBA so basketball league in America and what they found was really interesting the teams at the beginning of the season who actually physically touched contacted more ended up at the top of the league table at the end of the season and the team's you contacted les ended up near the bottom of the league there's a pretty strong correlation between their so high fives backpacks and this kind of stuff and it follows the trend of research that shows that human contact helps build trust and collaboration and so they can play together better as a team so you have exploring the world and actually being able to physically interact and move objects around and then also there's this real human element of building up these these essential aspects that we take for granted when we're interacting with people and in day to day life so that's why I think that such is really really important for virtual reality and for life in general and I think that in the future we will look at immersive experiences as being standard that they have some form of sense of touch some kind of haptic feedback mechanism for them and we saw back with my my nakiye 3310 that ended up getting killed by the touchscreen because of these interaction problems and I think that haptic sand and the sense of touch is going to be as important for these immersive virtual reality experiences as the touch screen was for smartphones breaking out of this early stage and becoming what they are today so that's what really drives us in our mission and what we're doing and to create this this sensation of touch so that you can feel what you interact with in these virtual spaces so we actually have a technology that uses ultrasound so we project these sound waves through the air and onto your hand and use them to vibrate your skin in order to give you feeling so we can create different textures and shapes and objects and actually it gets really great reactions when everybody's playing it out playing around with it and trying it we've used it to create buttons and sliders we've used it to create feedback for gestures we've created 2d shapes interacting with 3d objects and even sensations that don't exist in the real world so these guys are sticking their hands through a force field that they can't can't see as is actually downstairs in the in the museum we set that up as an exhibit and those kid nearly smashed my screen and but it creates these magical experiences where you can actually reach out and contact the digital content so we've spent the last three years mostly since the last time we were here working on integrating touchless controls into and cars home appliances consumer electronics products and the idea being that you can reach out and interact with the products without touching them but actually feel what you're doing and it's going very well we have a huge number

of products moving through one of which we can talk about this was shown off at CES in Las Vegas in January this is a concept car made by Bosch one of the big automotive Tier one suppliers I really like this is kind of like a blade runner-esque futuristic car with these screens everywhere and then you can control the center console using this touchless panel in the middle which means that you don't take your eyes off the road so it improves Driver alertness and keeps you more awake and attentive so this technology's going to roll out and a lot of products over the coming years but there's also a lot of ndas so we can't talk about too many of them so actually more interesting is to go and talk about some of the work that other people are doing using our technology we've got a large number now of partners in academic institutions and sort of interesting groups doing doing cool stuff and I want to talk about a couple of them the first one is dr. Mariana Obrist she's a researcher at the University of Sussex and she's been conducting research on actually conveying emotion through midair haptic feedback so she was using our technology to actually convey emotion and kind of crazy actually works so depending on how you project this feedback to the user you can actually make them feel happy or sad not just completely manipulate their mental state but they know that that is a happy feeling or a sad feeling you can push the feeling of emotion through these tactile sensations just in the same way that you could do it with say voice if I screamed or shouted versus talk softly you can do the same kind of things with with haptics and that's really quite fascinating for me because it's a huge power when you're creating these experiences to have another channel available to you as well as the content designers to feed this kind of emotional information through to the user the other one I wanted to talk about was an installation that a group of artists in London did and installed in the Tate Britain in London so this was a new way of experiencing art and adding extra layers of sensory information it was called the tapes and soaring and the concept was that you'd stand in front of a painting and rather than just bland the sort of staring at the painting and only observing it with your eyes they added extra sensors so in this one it's a painting called full stop this painting is about eight foot tall it is enormous and you have this huge black dot in the middle and you stand there looking at it you have a headphones on and you get binaural audio you have the lights kind of din in and out and fade it and fade up and down and you have your hand in this sort of glowing green pod and you receive tactile sensations on your hands that have synchronized with the audio and the light so you can almost when the audio sort of goes in and out you can almond and the sensation on your hand does a similar effect you can almost get the sense of perspective warping and this dot growing bigger and smaller on the wall as a really unbelievable experience and it was very successful there as well they sold out every day and actually ended up extending the run so three times longer than it was supposed to go on and so very excited about that and bringing new different ways to experience this kind of art and and attractions that you can go and see so then pulling this fully across to virtual reality where I see this going in the not-too-distant future is for a first step creating immersive experiences so these kind of attractions and installations that are starting to roll out in arcades theme parks cinemas and where you can actually enter a virtual world and an experience part of a story as one of the characters and the stories amazingly powerful experiences and then these even more so believable if you can reach out and actually contact and interact with the digital content if you can have the characters reach out and touch you if you can have a legion of spiders or creepy crawlies run across your hands or if you can cast magic spells or fill fireballs flying out of your hands it's going to make some some phenomenal experiences similarly in in enterprise in the workplace there's a lot of potential and indeed attraction now for VR and AR bringing three-dimensional content out of the display out of your twodimensional display you can actually see it in three dimensions while you're working on it I was actually in California pretty recently speaking to their team at Meza one of the leading augmented reality headset companies they were telling a story about how they're going to get rid of every single monitor from the desk of every single employee and their entire company by July of this year and and that will all be done in an augmented reality and first I thought wow that seems kind of crazy ly optimistic but they're sticking to it they do seem to be rolling that out and the reasons are pretty clear especially as human beings we're pretty terrible at spatial projection it's one of the things our brains finds difficult to do and challenging seeing a 2d image and working out what it would be like in 3d and how you could rotate it so if you can pull that content that a lot of people work on every day be it CAD models designs for new movies or games or and or even just datasets that will be better visualized in 3d you can pull them out of the out of the display and actually see them then you'll work better on them you'll get better results or work faster and the next step to that is actually being able to directly touch them feel them manipulate them and interact with them so that's what we're working on at the moment creating these new workspaces of the future where you'll be able to directly touch feel and manipulate your content finally as I've already mentioned this potential of human communication over distance in VR AR where you can actually talk to family or friends or business colleagues and rather than having to look at them on a small screen they can be there right with you so if you are traveling on a business trip rather than just having to sort of wave at your daughter through a little screen you could actually reach out and touch a ham this is going to be

a big change in how I think a lot of communication and interaction happens remotely and between humans so we've got a lot of work to do but we've come so far already and there's so many exciting things that are going to be revealed in the coming months and but we brought a lot of what we've created with us here today so there's a a booth in the demo tent which you should definitely come and check out and we have a number of demos and you can try the force field from the video but you can also have a go at training to become a wizard and cast magic spells and feel lightning bolts and various other spells flying out of your hands and as you do so is really good funding to definitely come and check it out so finally what I'd like to leave everybody with today is just a thought on how much you actually get from the sense of touch just in your daily life as you as you move around and interact so just think for the rest of the day about what you actually get from a handshake when you meet somebody or how you actually get something out of your pocket or even just how you go about tending the light switch off before you go to bed because we're working very hard on creating this technology and making it available for everybody but we know that the best experiences and the best ideas and the best creations are ultimately going to come from you guys and everybody out in the community creating these kind of experiences [Applause] The New School for Social Research.