Стенограмма первой лекции из цикла Томские популярные лекции «Технологии будущего - 2012» НОВЫЕ ИНТЕРФЕЙСЫ: ГАПТИЧЕСКИЕ ГОЛОГРАФИЧЕСКИЕ ДИСПЛЕИ Presenter: These lectures are devoted to the cutting age of technology. The partners of today's our lecture are Ritsumeikan University from Japan and Tomsk State University of Control System and Radioelectronics and Open University Skolkovo. Today's lecturer is Hiroyuki Shinoda. He is professor of Tokyo University. As you know Tokyo University is one of the top 20 universities in the world. Professor you are welcome and you can start. Hiroyuki Shinoda : Thank you very much for the very kind introduction. Hitoshi Ogawa from University of Tokyo and I would like to express a special thank to russian students and professor Krisanov and Abanov and all the committee members of IICST. It is a great pleasure for me to be here and to talk about my laboratory issues. Today's topic is Super Haptics. What is Super Haptics? I will explain it's meaning at the end of this presentation. Let's think about one thing - information technology supports the human by stimulating the human's sensitive organs as the interface. The screen of TV or computer stimulates our visual organs as the information from the screen to the human. And the speaker on the phone speculates the ear of human. And all the audio information from the device to the human. So what is the tactile or haptic stimulators? We have it in the our daily life. Of course we can give an example. For example - massage-machine. It is some kind of an official kind of tactic stimulator and the sense phone is also a kind of haptic stimulator. But the contribution to the human intellectual activity is apparently poorer than audio or visual stimulators. So what is the reason? Haptic focal point is not important for the human activity probably how many years are referred to this question. But I believe - no. Probably we cannot think over it without the information about haptic, haptic information. When You get up and walk, and open the door and brush the teeth, eat something, and pick up the phone, use computer - in all these activities we use haptic sensation , the haptic information; official information, to express our will and to do what we want to do. Therefore, if we can stimulate the haptic organ appropriate - 3, probably we can support human activity. Actually even when we talk about technology now if the tactile stimulation is ..distant - is boring. I think it's the main reason . So, because the steam has a large area. The eye and the ear concentrate but stimulating largely over the whole body of the human and haptic stimulation make an eye contact to device, for some device. That is the problem. 05:40 So, this is the typical haptic tactile spread device. It is called FANTOM – a very popular optic device and it has some specific application which is used in basic usage and laboratory. 06.20 But the mechanical system is a submotor, which generates force according to the motion and it's appropriate to force on the skin. But this device was not constructed for the human activity. We can't work using this device. We can't go out with this device. So, this is the wearing type device, that device is worn but if we want to produce very fastful stimulations this device becomes very barky, like this. We cannot imagine the practical use of this stimulator in our activity.
And here we began to think that making this type of device is more simple and likely. And this is the pen type device. The pen deforms according to the position of the pen. Another slide: this guy is watching a 3D object in 3D glasses and he is manipulating the objects. In this scenario large constant force never happens. This kind of floating device cannot be generated by large force or cannot generate constant force as it is caused by our touchings. And in this scenario the object rides away from the floating object. And it's obvious that only 4 poses happen at the moment of touch. And in this situation this simple device can generate very real thing. 9.00 The good news are that we have this kind of device to think over. It has a good function that stimulates human tactile organ and supports human activity. Use an active function with this device. Therefore recently many European companies were thinking about this kind of thing. But it's still a concern. If we have to have some device for enjoining the tactile support, how do we forget to habit? You have to have some device which condition is a very weak concerning. So probably if we can stimulate the skin we can do anything with this device. Probably that application and tactile haptic support this is the fil of this stock. So if it is really possible to stimulate human skin with the non-contact way? If possible in what kind of way? Thank you. Actually we found a good way to demote stimulation. I mean the using of autosound. The autosound is only the sound. The sound pressure is very weak. It's average is zero. And there is no persisting force - weak sound. We can easily concentrate into very small sort of thing and at the focal point we can create a very high energy density auto sound. And in that case there happens an interesting thing: the average of the sound don't have an offscript proportional to the energy scale of that point. Therefore we can push the object into ..?. of auto sound. Using this force and this is called radiation pressure. Radiation pressure is already well known phenomena in physics. Our contribution is to use it for the tactile innovation. Okay, I will show you a video. And in this video there is an autofile introducer, in which 100 sound transfuser are used. And mainly it's a major distance for sending an information and you can make it easy. Everyone can make it easy. At the combining this is the focal point of autosound. And at this point the large autosound is generated. Radiation force is pushing up this piece of paper. Actually it generates about 1 gram force. The autosound frequency is higher than human's tactile sensing. So we can control the focal point by controlling the fade of the each sound transfuser. And it also possible to moderate the amplitude easily. Because as you can see time constant of human tactile organ is about one ? and it higher than that, so we can create a various pressure pattern and control it on the scheme. And this is the two dimensional plot of additional pressure and the focal point. And the yellow part means high pressure and damped air. Every sound is wet(wept/wep) and as a nature of a wet(wept/wep) it can be combined to the comparable size to the wear rings and in this case we use a prototype of sound and a wet(wept/wep) to 8 meter and we can concentrate it about 1 centimeter or like that. Other important thing is that we can make such a concentration to the long distance. If the focal point is comparable or less than a part of the sound area we can create very small focal point comparable to the wear rings. Therefore, in principal we can stimulate the human's suffice by the autosound area founded on the ceiling of the room. Probably people think of other natural non-contact stimulation. Probably if we think of using the air jet it is also possible to use this jet to some stimulation in order to get some stimulation on the human skin. It's possible. But the sound can't be temporary controlled and it can't make a very small spot apart from the device. When there is the jet air - it is more. Then we have a subject device of radiation pressure. What happens with the haptic human haptic air? I will explain. Let's combine this radiation to the 3D visual stimulation. Okay, this is the 3D object and the epoch of the object tenders to touch it. What happen if we stimulate it on the skin?
This is the 3D apparatus that creates a 3D visual image. And this is not our work, this project is from USA. And it's a schedule of challenge to make a good 3D project and 3D visual creature. And the problem of the visual image is not ?? But if we have a sensor like this and if we major we can interact with visual image. And it's not new and it's not our work. 20:00 Our work is a typing stimulation to it. And we are making a focal point here. Focal point it can be controlled like this. Haptic stimulation which we find here and the focal point it can be controlled like this. Haptic stimulation is single way to visual images, you feel touching real effect. If the stimulation is perfect you cannot distinguish the real thing from the image. But of course this stimulation is not perfect. Easy application to the technology is, for example, when make a touch without any contact. The touch panel is a panel in the public place, especially in the hospital should be no-contact. If we have a floating image as we get a tactile simulation we can make no-contact touch. It's completely safe at the point of view of sanitary program. 22:02 But let's think about this consideration further. We are using 2 dimensional touchpads. But someone had made a 3D touch pad. So, this is very inspiring work by Chriss Harisson in 2011. I will show you this video. At first he explains the use of conventional interface of the panel, switch and keyboard which has 3 fixed dimensional sheep. The problem of this device is that the sheep is not programmable. We cannot change the sheep and location over it. The touch panel is good in a sense because it controllable, we can control it at the location. But the drawback of the front corner locks 3 dimensional sheep and it's not good in usability. This condition makes the 3D interface not quite programmable. It's position is terrifying. 24:29 The individual project has a hit - the touchpad was amplified with 3 dimensional interfaces. They tried many variants of interfaces. This can change the 3D sheep. And the author says that 2 shapes can be displayed. This is brown sheep and ?. sheep. But still the problems exist - only 2 kind of sheep can be displayed in this case. Let's think again- the keyboard has a 3D sheep and it is programmable. Touchscreen is programmable but 2 dimensional. 25:45 Then, is it possible to make a touchable interface -3D dimensional and free programmable? Is it possible to make it? One possible way to make it - is to invent the device that can change the shape. But another way is more practical when it is combined with the 3D image and noncontact . I will explain this. This is a touchscreen device and this is the dimensional screen. 26: 34 Please, imagine - you are hunting there and we can create a partial sensation of the object. A possible way to express this kind of thing is to combine the visual 2-dimensional subjects. But now we can create freely a 3 dimensional design. 27:19 of course this kind of botanic is impossible. And this kind of interface you can touch like in a reality and you can play with this kind of creatures. We have a big freedom of the design. But this is the application of the example of 3 interface. And of course this technology can be used for attracting people's attention in the town or in the museum and so on. 28:32 In this example we stimulate human's skin directly, but it's also possible directly to influence human's skin. This is the example of the stimulation of the object. In this example, the guy has a paper. if we think about sound vibration to the paper. We can change the tactic feeling of paper which is reach by sound stimulation. It's just a part of the story. It's very important to notice that there are 2 kind of touch - inhaptic (passive) and haptic (active) touch. And the examples I’m showing you now are all active touch applications. And the passive touch is being detached by environment in many cases without expectation. And other application, an important application of haptics may be also in passive touch mainly. We talk about passive touch today’s evening? This photo is a good example of passive touch. He is a university student but he is playing a shooting game and wearing a vest having a lot of actuators on it. And he is enjoying the shooting game passively feeling of being shot. Of course he doesn’t know the feeling of being shot. This kind of stimulation may increase the (31)??? of the game and the reality of the game. And this is passive touch. And this is also a very good example of a passive touch application. Probably some people, some students playing arcade experienced this kind of attraction. This is the 4D Theatre in Vancouver aquarium. And it is a very good example of that in a good scenario a simple haptic stimulation is very effective. In this museum, in this
aquarium every seat(32) has haptic display. The haptic display is a very simple display, just pushing the back, that’s a kind of actuator on the seat. So if without any visual information, stimulation is nothing, nothing to surprise, but if you acquire a view seeing 3D image of a sea creature like ??? or a sea snake when that sea creature is approaching the audience and then as they push and the actuator pushes them back, people feel the creature hicking there so everyone is ???(33) enjoying and this is a very good example that even a simple haptic stimulation can be very effective in a good scenario. And that’s a nocontact haptic actuator but in this case it serves to initiate the only thing - one way of stimulation, such a no-contact actuator can be a variation of this kind of passive stimulation. So this is also, this is the third good example of importance of passive stimulation. Ok, this is a car from the top view. And please imagine you’re driving this car and suppose some careless car is suddenly approaching your car. So what is the best way to give you the risk of this situation? What is the best way to get the risk? The fast choice is using voice: “be careful, a car is approaching”. Actually it is a effective way to notify that some risk is approaching but this count which direction is risk is very different: “from the left, from the right a car is approaching”, you worry, you a certain time to understand the situation, you a logical thinking to understand the situation around you. So, probably, in that case a tactile stimulation is the best way. The seat has a tactile stimulator surrounding you, your body, it pushes your body along the direction the car is approaching then you can understand direction the car is approaching from and without thinking anyway. So this is a very good application of passive haptic stimulation. And this is a very interesting research. I like this research very much. This is a walking navigation put in the ear(36.02) developed by Kasimotu laboratory in the part of electric communications. So do you understand this? This is an actuator put in the ear, then you can be navigated without thinking, you only follow the haptic touch like this ??(36.30). Important thing is that you can be navigated without thinking anything. And I think it’s a very cool invention. But of course the haptic is very cool but I don’t think many people want to wear that kind of device. But one of the challenges of no-contact tactile stimulation is to realize that kind of a function without wearing anything like this. And also the challenge is that in a public space, a public ??? application of haptic stimulation can give a ??? (37.27) of information and notify people about various things and give the direction of the action. So probably it is a very important research area of no-contact tactile stimulation. Ok, here I will show you that different (37.52) researches of our laboratory are based on passive sensation. It is very similar to the device I’ve just shown. This is ultrasound transmitter on the ceiling and the device I showed has 3D display, instead of it this system has a projector, usual projector, and it’s projecting an image on the hand and actually it is an easier structure of the system. And you can project a visual image and simultaneously get a tactile information of it, then you can get very realistic things on your skin, then what kind of thing becomes possible do you think? Ok, I will show the video. In this video we use laser carbon disk (39.25) about 6cm by 6cm and you can cover the disk, one side of disk. And we will project a thing of some device. And I like it. Ok. This is a very cool application. Actually this idea was inspired by Dr. Andreo Ianky. And his research was “haptic transmission of security information”. Haptic is a potential of security information transmission. So in this case – a code is projected, as the code ??? (40.36) is projected on the hand and haptic stimulation of the actuator is leaving pushing on the hand, on one of the sides, the user can know what character is pushed and they are inviting to push a character here. So the point is that we transcend secret information very safely in front of people. I think it is one of good applications of passive perception. And this is the example of training the attention of people and I think it would also challenge communication application using this tactile stimulation. But I think I explained only a good aspect of our device but we have to clarify that in case of ??? . And the most important thing is that ??? this ??? and ???. So that’d mean that ultrasound in air is absorbed and high frequency ultrasound ??? operated. So and the attenuation length is ???(43) is proposed you can ??? of the frequency. Therefore, if we use 40 kHz that is absorbed, radiation pressure decreases by 20% to 1 meter and we can use 1 meter or 2 meter distance using ultrasound. But 40kHZ ultrasound ??? (43.30) 8mm, it doesn’t mean much, I want to make it smaller, but if we use 100 kHz, for example, the attenuation in the air becomes like this. So if you use 1 meter apart from your device it’s only 20% that decreased by 80% or less (of loss) and only 20% remains, so the actual workspace is only
30cm or 50cm. So we can not increase the frequency of the ultrasound from 40kHz. Ok, this chart summarizes the application of this device. First, display of radiation pressure is smaller than 50 ground force per square cm, it is 50 ??? (44.43) So this 50 standard, this is much more power compared to 50 standard ground force. 45:00 Another function of this device is theoretical thing. If you use 3.4 watt to generate 1grand force, but tactile consumptoin cannot content (confront?) just at the moment of the touch or notifying something it is used, then this another consumption doesn’t take matter, might not matter. So under visual resolution it is about 1 sm and only ???? force is spread. That means that generated force is only ???? to the surface. It’s also inevitable. And 3 millisecond ???? half of 1 mm ??? that means 3 millisecond is not very large but not very short, so that is from the sound ferocity in the air, and it’s inevitable either. And weak against occlusion. So if something intercept the sound the people cannot receive any stimulation behind the obstacle. This is one point of this device, but it’s completely non-constraint. And my authority is to make this device a necessary item of the light in the room and every room has light like this, and I want to make it necessary item like this. And I think it is not impossible by the technological advancement and I think printing technology can be available and low cost ??? of the sound. Finally I will show some other aspect of super haptics using non-contact remote haptics. This is final topic I will talk today. This chart summarize the desired ???? haptic medium. In what meaning haptic stimulation is acquired? First of all in the competition for visual ???? This is a very conventional motivation for haptic stimulation. And enhancing humanity and computer interfere forcing this stress and teaching motion skill and allow guidance, I explain. And entertainment of the communication. Communication with touch is important item. And finally transmitting them truth. This means that only fortify curiosity of the people, who want to know the truth. So if you see TV you want to know unknown thing visually. And probably we won’t know haptic reaction, haptic feeling of unknown object. So finally it’s confident, it’s possible to make haptic broadcasting shifting. Haptic broadcasting shifting is ???? and this is an unknown object, million of people want to touch it and know tactile feeling, and is it possible to transmit tactile feeling to million of people? This is a problem of this topic. And if the user is only one, only one person wants to know the unknown touch, tactile feeling of an unknown object, we hold, we have already technology hold haptic teleresistance. In every system there are 3 manipulators like this. And three manipulators apart from the user grasp the object and transmit this grasping force to the user. Then user can remotely touch the unknown object and knows the interfering of the object. If the transmission communication is apart. This is a three resistant system using one to one commutication. But it’s obviously impossible. Using this method million people can feel the haptic feeling simultaneously. Then partially to real life million people can touch the object simultaneously. If we use the teleresistance system you have to prepare a million of them have to have the object simultaneously. This is impossible. So the other way is making 3D, making the model of the object of the 3D image the proof of the tactile information. And if we can make such a 3D image of the tactile information and send them model of the object, then we can freely touch the object, million people can touch it free. So, of course there are many features tactile features. But a very important factor is compliance, elasticity of the object, the form of the surface. But of course there are some frictions or some stitches or some microscopic features, that are important haptic features. But in this case we only concentrate, we only pay attention to the compliance of the surface, and probably it’s possible to make 3D model with compliance, with surface compliance. We can assume that the reality of the compliance. Actually no contact hardly fission, for example no contact hardly fission, we cannot find many examples of the touch and probably this is the only touch found in the capture. And they are using air jet and pushing that surface with air jet and measure the surface deformation optically. So if we use the auto????? We can quickly detect the compliance distribution of the surface by scanning the ???? spot. In this research the remote haptic is used before zincing of the object and pushing the object further by large pressure and measure the surface expressment option. This is the structure of a method. Then I will summarize the system. A vertical object. And we also have a 3D camera that can measure 3D shape of the object. And after that we also
measure the surface compliance by scanning this on force unit. And we get the 3D shape with hardness as a model in the computer and send that to million other people. This is a system we published. Actually the problem of a current system is that it takes long time to take a data of. This is the result of hardness tension of the surface. And this is a surface is completely rut, the hardness of the distribution like this and we measure the hardness of the distribution like this. And we get this 10 cm by 10 cm using the 20 ??? or one DET (data). And this is a result of surface hardness of ???? this kind of thing can be measured. Also this is a fiche and it is interesting that hardness distribution omit. And using this system we can take that picture of the object. Taking a picture we get 3D model and 3D hardness of the surface. And if I get the object data now we can touch it freely. This is a hardness distribution of object and we have a 3d shape model and we can create deformation from the surface hardness and shorter deformation. This a hardness distribution and this is a model shape and reaction of the surface is created from the hardness and the shape. This is all of my talk and I will explain finally ???? super haptics. But I wanted to say by this word, after this “superhaptics” is only a word made for this conference. I think superhaptics is haptic human support free from the problems and constraint ??? to the mechanical contact of the human haptic stimulators. So if we remove the problem, caused by mechanical contact we can create a radio application of human support by haptic stimulation. So that is what I wanted to say in this speech, that’s all, thank you very much. Presenter: So, any questions? Question: Is it possible to transmit - to feel something wet or slippery? Hiroyuki Shinoda : One of the important things is that the haptic feeling can be created easier as we imagine. It's very interesting question. In this grade I chose steady additional pressure. So if you create an additional pressure and move smoothly and slippery, you see exolastic material, so physically you can feel. Haptic touch cannot be distinguished by cluster holes. Therefore, I think that most of tactic feeling can be reached by this machine. I'd like to challenge the problem. Question: Did you ever consider the using of haptic displays for blind people? Hiroyuki Shinoda : Thank you for the question in English. Actually I think haptik interaction will increase the number of healthy people. I think about this kind of technology. But I have to an enter fee for making such a thing. We should have the archives of developments. We need to develop the industry of such technology. And also we have contribute it to the healthy people. Question: Can you imagine any alternatives to the ultrasound? You have told us previously that ultrasound have disadvantages. Hiroyuki Shinoda : Actually, I never thought about other alternatives. Although I tried a laser light. A laser light changed the temperature of the skin. During our experiments some sheets of paper were burnt by the laser and we felt some kind of risk to continue the experiments. But I haven't found the best alternative to the ultrasound. Question: What about another kinds of ways to pass the pressure? Hiroyuki Shinoda : I want to use other frequency. If the frequency becomes high the ? becomes very stick, we can't get a large work space. 40 kh is practically maximum magnitude. Question: Mr. Hiroyuki Shinoda thank you very much for the speech you have delivered. I have a question. You spoke a lot about advantages which technology can give. Can you think of any negative impacts? Is it environmentally and human friendly? Hiroyuki Shinoda: I feel 2 kind of risk. The first risk - is a physical risk. 50 is a focal point to the ear and it’s ok for the body. But I'm not sure how it will influence on the ear. We attack the ear at the focal point and I have experienced it already. And it's okay for me.
The second question is - the mental health. It’s a more serious question. I think I want to use haptic technology for the covering the bad point of information of technology. Here an eye and the ear are used. I think it will have a bad effect on young people. Haptic study Is very important for the mental study. I want youth to have this effect in a good direction. I'm very interested in communication with haptic stimulation and also in the habit. There exist a culture communication. We are the nation. Sometimes we hug. But we don't hug with unknown people - that is the reason of safety. But do we have? We have a faith device. Question: Вы показывали видео приложение, где циферблат выкладывается на ладони и человек пишет тактильный код. Его воздействие, это какое-то температурное давление? Hiroyuki Shinoda: Oh, that's a good question. I don't have enough knowledge to answer. I feel that the temperature we have is a local ability and very poor for the temperature sensation. I think it is impossible to detect the temperature correctly by temperature simulation. We have to be very careful, we feel that we clearly can distinguish the mechanical and the temperature steamers. ? but we are not sure. For example: if you touch the metal phases you feel a quick temperature change on the finger. but I don't believe that such kind of temperature never changes so quickly.We feel a very cool sensation. I think combining both haptic sensation. And we feel, we imagine temperature change very quick. Пропущено. I think it's difficult to answer your question quickly but it's a good question.