Optimizing a Humanoid Robot Skill

robótica

10

Luís Rei1, Luís Paulo Reis1,2 and Nuno Lau3,4 1 DEI/FEUP - Informatics Engineering Department, Faculty of Engineering of the University of Porto, Portugal 2 Artificial Intelligence and Computer Science Lab. Porto, Portugal 3 University of Aveiro Campus Universitário de Santiago, Portugal 4 IEETA -Institute of Electronics and Telematics Engineering of Aveiro, Portugal

ARTIGO CIENTÍFICO

Optimizing a Humanoid Robot Skill ABSTRACT Controlling a humanoid robot with a large number of joints and thus, degrees of freedom, presents a complex problem that requires knowledge in multiple fields, including biology, mechanics, physics, electronics and computer engineering. The challenge of making a humanoid robot play soccer adopted by the RoboCup initiative, a task that was created specifically for humans, is ideal for testing the performance of the robot’s motor skills. This work aims to improve the performance of these skills by applying to them an automated optimization process. The robot is the simulated RoboCup 3D agent, a simulated NAO robot, implemented by the FCPortugal3D team. Several different optimization algorithms, in particular Hill Climbing, Simulated Annealing, Tabu Search and Genetic Algorithms are adapted to this problem, used and compared in the optimization of a particular skill of the FCPortugal agent. Furthermore, the skill optimized, which allows the robot to get up after falling on its front, is also compared to the original, unoptimized, skill as well as to those of other teams participating in the RoboCup simulated 3D league. The achieved results are good, providing skill that performs considerably better than the original skill.

I. INTRODUCTION The RoboCup international competition uses soccer as a standard problem to foster research in the fields of artificial intelligence and robotics [1]. FC Portugal team project was conceived as an effort to create intelligent players, capable of thinking like real soccer players and behave like a real soccer team [2] competing in the RoboCup simulation leagues. The 3D humanoid soccer simulation league was created in order to promote research in the necessary techniques in order to make humanoid robots play soccer. The topics of research include physics, biology, control theory and machine learning with the aim of developing stable biped skills such as walk, turn, get up and kick.

The result of this kind of research may

be extended to other domains, such as the use on real humanoid robots, which may be able to perform social tasks such as helping a blind to cross a street or elderly people to perform tasks that became impossible to do alone. The use of simulated environments is popular since it allows the developers to make arbitrary or complex tests in the simulator without using the real robot thus avoiding expensive material to get damaged [3]. The teams have consisted of only a few agents, research in coordination has not been very important in the 3D league. Developing efficient low-level skills has been the main decisive factor in the 3D league [4].

Despite this fact, FC Portugal main focus has been on the high-level, multi-agent coordination [5], [6], [7]. This work instead focus on the low-level performance of a robot and aims to show how the robot’s low-level skills can be improved with the aid of automatic optimization methods and provides a comparison between different optimization algorithms adapted to this problem, namely, Hill Climbing, Simulated Annealing, Tabu Search and a Genetic Algorithm. The particular skill being optimized provides the robot with the means to get up after falling on its front and attempts to minimize the time it takes to perform it while maximizing the stability of the robot while, and immediately after, performing this skill.

Section II provides information about the problem, specifically the simulation system, the agent, skills in the FC Portugal agent in general and the skill to be optimized in particular. Section III gives

Figure 1. Simspark Monitor screenshot.

an overview of the optimizer developed and the algorithms used. Section IV describes the experiment conducted and its results. Finally, section V analyses the results of the experiment and extracts the conclusions.

II. PROBLEM FORMULATION A computer simulation, also known as a computer model, is a computer program that attempts to simulate an abstract model of a system. A system is a set of structured interacting or interdependent components that forms an integrated whole. Computer simulations have become a useful part of mathematical modeling of many natural systems in physics (computational physics), astrophysics, chemistry and biology, human systems in economics, psychology, social science, and engineering.

The official Robocup 3D simulation server is Simspark [8] a generic physical multi-agent simulator system for agents in three-dimensional environments. It simulates the laws of physics in the real world in the context of a soccer game (i.e. the soccer field, ball and rules of the game) as well as the robots (physical dimensions, look, sensors and joints).

The robotic platforms (either the most simple articulated arms or the most complex humanoid robots) are usually very expensive. The use of simulation environments for research, development and test in robotics provides many ad-

Table III. results for the optimization of getupfront -various algorithms compared.

Vienna, Austria: Itech Education and Publishing, December 2007, pp. 167–192.

Algorithm

Execution Time

Original

2.00 (s)

Hill Climbing

1.06 (s)

Simulated Annealing

1.37 (s)

Tabu Search

1.19 (s)

Genetic Algorithm

2.00 (s)

[6]

L. Mota, L. P. Reis, and N. Lau, “Multi-robot coordination using setplays in the middle-size and simulation leagues,” Mechatronics, Elsevier, 2010 (in press).

[7]

L. Rei and L. P. Reis, “Brain: A deliberative cibermouse agent,” in Proceedings of the 10th Conference on Mobile Robots and Competitions in con-

Table IV. Time for the agent to get up after falling on its front.

junction with the 10th Portuguese Robotics Open -ROBOTICA 2010, Leiria, Portugal, March 2010,

Team

Skill Execution Time

FC Portugal (original)

2.00 (s)

FC Portugal (optimized)

1.06 (s)

Apollo3D

2.00 (s)

Nao Team Humboldt

2.00 (s)

pp. 35–40.

[8]

J. Boedecker, K. Dorer, M. Rollmann, Y. Xu, and F. Xue, SimSpark User’s Manual, 1st ed., 2008.

[9]

J. Nicolas, “Artificial evolution of controllers based on non-linear oscillators for bipedal locomotion,” Master’s thesis, Ecole Polytechnique ´F´ederale De Lausanne, 2004.

Tom´e, “Automatic generation of biped walk behavior using genetic algorithms,” in 10th

Spain: Springer, June 2008, pp. 805–812.

[11] O. Obst, M. Rollmann, and M. Rollmann, “Spark

simulations,” in Computer Systems Science and

VI. ACKNOWLEDGMENTS This work was supported by the Fundação para a Ciência e Tecnologia through the project ”INTELLWHEELS -Intelligent Wheelchair with Flexible Multimodal Interface” (grant ref. FCT/ RIPD/ADA/109636/2009) and DEI/FEUP Department of Informatics Engineering of the Faculty of Engineering of the University of Porto.

Engineering, 2004.

[12] J. Boedecker and M. Asada, “Simspark -concepts and application in the robocup 3d soccer simulation league,” in SIMPAR-2008 Workshop on The Universe of RoboCup Simulators, vol. CD-ROM, 2008.

[13] J. Lima, J. Gonc¸alves, P. Costa, and A. Moreira, “Humanoid robot simulation with a joint trajectory optimized controller,” in Emerging Technologies and Factory Automation, ser. ETFA. IEEE, 2008, pp. 986–993.

[14] S. Behnke, “Online trajectory generation for om-

REFERENCES

nidirectional biped walking,” in IEEE International

[1]

H. Kitano, M. Asada, Y. Kuniyoshi, I. Noda, E. Osa-

Conference on Robotics and Automation (ICRA),

wai, and H. Matsubara, “RoboCup: a challenge

Orlando, FL, USA, May 2006, pp. 1597–1603.

problem for AI and robotics,” in RoboCup-97:

[15] O. the Mapping Problem, “Shahid bokhari,” IEEE

Robot Soccer World Cup I, 1998, pp. 1–19.

[2]

[3]

L. P. Reis and N. Lau, “FC portugal team descrip-

[4]

207–214, 1981.

tion: RoboCup 2000 simulation league champion,” ROBOCUP-2000: ROBOT SOCCER WORLD

nakakis, “How easy is local search?” Journal of

CUP IV, pp. 29—40, 2001.

Computer and System Sciences, vol. 37, no. 1, pp.

H. H. Lund and O. Miglino, “From simulated to

79–100, 1988.

[17] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Op-

tionary Computation, 1996., pp. 362–365.

timization by simulated annealing,” Science, vol.

N. Shafii, L. P. Reis, and N. Lau, “Biped walking

220, no. 4598, pp. 671–680, May 1983.

using coronal and sagittal movements based

[18] F. Glover, “Future paths for integer programming

on truncated fourier series,” in RoboCup2010:

and links to artificial intelligence,” Computer and

Robot Soccer World Cup XIII, ser. LNAI, no. 6556.

Operations Research, vol. 13, no. 5, pp. 533–549,

appear).

[5]

Transactions on Computers, vol. C-30 (3), pp.

[16] D. S. Johnson, C. H. Papadimtriou, and M. Yan-

Heidelberg: Springer, 2010, pp. 324–335 (to

ARTIGO CIENTÍFICO

IWANN 2009, ser. LNCS, vol. 5517. Salamanca,

-a generic simulator for physical multi-agent

real robots,” in International Conference on Evolu-

The two best algorithms, in terms of end results, were Hill Climbing which found the best solution and Tabu Search which found a solution very close to that found

by HC. TS was, however, much faster than HC at the finding solution, as evidenced by figure 5, and even though the solution it found was not as good, it was only marginally worse. The implemented Genetic

[10] H. Picado, M. Gestal, N. Lau, L. P. Reis, and A. M.

15

Algorithm failed to find better results for the GetUpFront skill. Its global search strategy does not seem adequate to optimize skills which, in spite of a very large search space, have an initial state very close to the optimum.

1986.

[19] J. Holland, Adaptation in Natural and Artificial

N. Lau and L. P. Reis, FC Portugal -High-level

Systems. The University of Michigan Press,

Coordination Methodologies in Soccer Robotics.

1975.

robótica

an automated process of reducing this difference by improving existing skills with the aid of automatic optimization methods. The initial work was made to create an optimization framework for the skills of the FC Portugal 3D humanoid agent which inhabits the simulated world of the RoboCup 3D simulation server. This required both modifications to the agent, the simulation server and the creation of the optimizer itself. The configuration of the optimizer allowed for easy distributed optimization using multiple agents and multiple simulations executing concurrently in multiple computers over a network. Subsequently, different optimization algorithms were implemented, namely Hill Climbing, Simulated Annealing, Tabu Search and a Genetic Algorithm. These algorithms were then used to optimize the GetUpFront skill, which uses Sine Interpolation for automatic trajectory planing The results were definitively good. The skill improved its execution time by 50% from around 2s to around 1s. The resulting skill is faster than the original skill as well as the equivalent skill implemented by other teams participating in the RoboCup 3D simulation league.