Robert Babuska
Desirable behaviour in robots On 8 January 2016, Professor Robert Babuska will give the Foundation Day Lecture during the TU Delft 174th anniversary celebrations. Babuska is professor of Intelligent Control and Robotics in the faculty of Mechanical Engineering, Mechanics and Maritime Technology (3mE). He is also scientific director of the TU Delft Robotics Institute. ‘It’s not so hard to make a robot happy’, says Babuska.
From a very early age I had a fascination for electricity. As a child I was always going around with a piece of cable and sticking it into the wall socket. My parents weren’t very happy about that. I went on to study electro-technology in the Czech Republic, at TU Prague, specialising in technical cybernetics, which is control technology with a dash of Artificial Intelligence and robotics thrown in. So robotics was always there in the background, but I also worked on countless other applications for control technology, for example drinking water purification processes and optimisation of dredging processes. Control technology is in everything, even if you can’t see if from the outside: your mobile phone, your car, an aeroplane. I research generic systems for control technology, mainly those that are used in robotics. I love the everyday nature of it. Once you discover one method, you can apply it to all kinds of systems. I’m mainly interested in learning control systems: how you can make a system improve itself. Imagine that a robot has to carry out a difficult task, such as walking. If a human demonstrates it once, a robot will only get it right partially. You want the robot to improve of its own
9
accord and ultimately to achieve more than a human. It is possible; it’s been proven. Anything set up by people can always be automatically optimised. A production process or climate management in the home. Yet these technologies are not often used. Where they are in use, however, is in ships. There are algorithms that are able to adapt to changing conditions at sea. The industry is incredibly careful however, as something has to behave as designed, without any surprises. So there is very little leeway for experimentation, or else you have to accept that a learning system will make mistakes from time to time, just like people. Our challenge is to ensure that these technologies don’t make any major mistakes while they are learning. This research is extremely worthwhile, as you can get systems to perform far better, saving materials and energy, for example. Reinforcement learning is one of the techniques that we use. It is based on the way in which people and animals learn, using punishment and rewards. You can’t explain things to a dog or a young child, but you can reward desirable behaviour. You can pretty much apply
the same principle to a computer. The reward is not a cookie, but a number. A learning algorithm tries to collect as many of these rewards as possible. Using this sort of technique, computers can start out at nothing and end up making huge achievements. There are countless examples of this in the literature and on the internet. Google DeepMind, for example, showed a computer some screens from Atari games, without explaining anything about the rules or tactics. After a couple of hours the computer was able to play well and after half a day, even better than a human. It’s not world-changing application, but it does show what’s possible. We do the same with walking robots. We walk smoothly without thinking about it, but that’s very difficult for a robot: it’s unstable on two legs and can easily fall over. That’s why it walks very carefully and looks so mechanical. You can code dynamic walking – like a human – as being the desired behaviour. So if a robot takes a good step and achieves a certain speed while not using too much energy, then you offer a reward. If it falls over, you can give punishment. In this way the algorithm can learn what the optimal operation is for the various motors, so