European land robotic trial 2008 (elrob) - a realistic benchmark for outdoor robotics

ARTIGO TÉCNICO Bernd Brueggemann, Timo Röhling, Hans-Ludwig Wolf and Frank E. Schneider B. Brueggemann, T. R¨ohling, H.-L. Wolf and F.E. Schneider are with the Research Group “Unmanned Systems”, Research Institute for Communication, Information Processing and Ergonomics Research Establishment for Applied Science (FGAN), 53343 Wachtberg, Germany {brueggemann,roehling,wolf,schneider}@fgan.de

EUROPEAN LAND ROBOTIC TRIAL 2008 (ELROB) – A REALISTIC BENCHMARK FOR OUTDOOR ROBOTICS ABSTRACT The European Land Robotic Trial (ELROB), which was held for the third time in 2008, is designed to compare unmanned ground vehicles in realistic outdoor tasks. It addresses the need to create a benchmark that can reproducibly compare and evaluate different robot systems.

While robot trials like the DARPA Grand Challenge or the RoboCup have proven to be adequate benchmarks to compare robots systems in specific scenarios, the ELROB provides benchmarking in a wide range of tasks which are oriented at prospective use-cases in both military and civil applications.

In this paper we describe the ELROB 2008, the rationale behind the scenario design and how the trial has been implemented. Further, we present the results which illustrate the remaining gap between requirements and abilities.

[10]

I. INTRODUCTION

A. Related Work

The European Land Robotic Trial (ELROB) is designed to demonstrate and compare the capabilities of unmanned systems in realistic scenarios and terrains. It was invented by the European Robotics Community and organised by FGAN [9]. The trial is held annually, alternating between a military and civilian focus. The first military ELROB (M-ELROB 2006) was performed at the military school in Hammelburg, Germany. In 2007 a civilian ELROB (CELROB) took place in Monte Ceneri, Switzerland. The latest ELROB, the M-ELROB 2008, was held again at the military school in Hammelburg. The aim of each ELROB is to get a deep insight into the field of ground robotics by testing existing solutions in practical trials. These trials • are conducted with a focus on short-term realisable robot systems, • are explicitly designed to assess current technology to solve real world problems at hand, and • are an opportunity to bring together users, researchers and industry to build a community.

The ELROB scenarios do not limit themselves to the abilities of today’s robots, but focus on realistic missions designed by prospective users in a demanding environment. The challenges in 2008 have become much harder than in previous ELROBs, reflecting the new requirements on ground robotics defined by the German Army. Thus, it was expected from the beginning that not all participants would be able to complete the scenarios.

This paper should give an insight both into the designing of the different trials as into the performance of the participants. The remainder of this paper is organized as follows: We present some reasoning why a trial is an adequate benchmark for robots, and why benchmarks are important to the robotics community. In section II the tracks and tasks for the participants are described. Section III briefly presents the participants and their robots.

In section IV the results of the ELROB are discussed. The paper closes with our conclusions and future work.

Generally it is a difficult task to compare different published approaches in the field of robotics[1]. Thus robot competitions are recognized as valuable benchmarks for real robot systems [2]. Several different competitions were held in the last years. Two of the largest and bestknown competitions are the RoboCup [4] and the DARPA Grand Challenge [3], which are also recognized outside the robotics community.

While the RoboCup is currently targeted at indoor robots, the the DARPA Grand Challenge aims to test and compare driverless cars. It started in 2004 with the rather simple task of following a 241 km long path, defined by several thousand GPS waypoints. Due to the difficult terrain and some teething problems, no participant was able to solve this task. In 2005, the task remained basically unchanged, and four participants successfully completed the race. In 2007, the DARPA Grand Challenge modified its goals from driving autonomously on difficult terrain to interacting with other vehicles in an urban scenario. Again this challenge could be solved by three teams.

The ELROB is somehow comparable to the DARPA Grand Challenge in its attempt to gauge the functionality of outdoor robots. However, the ELROB presents a variety of tasks instead of a single scenario, and generally puts less emphasis on competitive rankings. Thus, the participating teams are encouraged to explore the limits of their systems.

robótica

II. TRACKS AND TRIALS The chosen area for ELROB 2008 lies within the training facility of the military school in Hammelburg. Its size is of about 9 square kilometres. The accessible roads have different qualities, ranging from well paved to heavy dirt roads.

The environment is predominantly woody (see Fig 1).

The different tracks on site were chosen to test specific aspects of robot deployment. Some challenges were common to all tracks, others were specific to certain scnearios. In preparation for the trials, every track was tested with respect to

V. CONCLUSIONS AND FUTURE WORKS A. Conclusions The purpose of ELROB was not to get an overview over technological possibilities but to test outdoor ground robots in real world scenarios without regard to limitations of the robots. Thus, the scenarios were to show the gap between desired and possible applications for today’s robots. As could be expected, not every participant could cope with the designed missions.

So the results were not unexpected and definitely not disappointing. In retrospect, two main problems could be singled out: • reliable Hardware, including reliable communication • innovative autonomous software controller It was noticeable that while the industry generally had hardware in excellent quality available, they lacked the innovative autonomous control algorithms developed by the university teams. On the other hand, the University teams had most of their problems due to their restrained hardware budget and the required trade-off between functionality and cost. The combination from the robots used by industry and the control algorithms of university might achieve much better results.

Table V . Results: Mule Transportmission

B. Future Works F. Security Mission This mission was not about actively securing a military camp. In such a situation an approach with stationary sensors, dogs, and security personal could be much more appropriate. The intent of this scenario is to secure mobile facilities or close security gaps within large military areas (like airports).

Important is, beside the abilities a mobile sensor platform can offer, that the UGV can follow and identify persons (and vehicles).

To keep the runs of the robots comparable, each participant faced the same number of intruders. One intruder had a passport to legitimate himself.

All other intruders could not identify themselves. When an intruder was encountered by a robot, the person was cooperative and did not try to escape. Important for evaluation was the number of detected intruders and their corresponding positions. Other criteria included the time until the first intruder was detected and the level of autonomy. The environment was too cluttered to be reliably secured by a single robot. The combination of an UAV to overlook the site from above and an UGV to follow and stop intruders was an obvious solution and was employed by two participants.

In total, nine teams participated (for results, see table VI):

From the 15th to the 18th of June 2009 the second civilian ELROB will take place in Oulu, Finnland. It is titled with “Robotics in security domains, fire brigades, civil protection, and disaster control”. So the missions will be designed having typical scenarios of those fields of applications in mind.

And again the trials will be designed to present scenarios as close to real world applications as possible.

REFERENCES [1] Angel P. Del Pobil, “Why do We Need Benchmarks in Robotics Research?”,In Proceedings IROS-2006 Workshop on Benchmarks in Robotics Research, Beijing, October 2006 [2] S. Behnke, “Robot Competitions - Ideal Benchmarks for Robotics Research”, In Proceedings IROS-2006 Workshop on Benchmarks in Robotics Research, Beijing, October 2006.

[3] S. Thrun, M. Montemerlo, H. Dahlkamp, D. Stavens, A. Aron, J. Diebel, P. Fong, J. Gale, M. Halpenny, G. Hoffmann, K. Lau, C. Oakley, M. Palatucci, V. Pratt, P. Stang, S. Strohband, C. Dupont, L. Jendrossek, C. Koelen, C. Markey, C. Rummel, J. van Niekerk, E. Jensen, P. Alessandrini, G. Bradski, B. Davies, S. Ettinger, A. Kaehler, A. Nefian, and P. Mahoney, “Stanley: The Robot that Won the DARPA Grand Challenge” , Journal of Field Robotics, Vol. 23, No. 9, June, 2006, pp. 661-692.

[4] Hiroaki Kitano, Minoru Asada, Yasuo Kuniyoshi, Itsuki Noda, Eiichi Osawa, Hitoshi Matsubara, “RoboCup: A Challenge Problem for AI”, AI Magazine 1997 [5] W. Seemann and K.-D. Kuhnert, “Design and realisation of the highly modular and robust autonomous mobile outdoor robot amor”, In The 13th IASTED International Conference on Robotics and Applications, Wrzburg, Germany, August 29-31, 2007.

[6] M. Proetzsch and K. Berns and T. Schuele and K. Schneider, “Formal

Table VI . Results: Security Mission

Verification of Safety Behaviours of the Outdoor Robot RAVON”,Fourth International Conference on Informatics in Control, Automation and Robotics

The University of Bremen used a “Mikado” for this mission. All UAVs had big trouble with the choppy wind. Diehl could only run one robot out of two. The French team wanted to start with their UGV and an UAV. But due to the weather conditions the use of the “SpyArrow”, a very fragile Styrofoam glider, was not possible. The UAV from Siralab started outside the camp and flew about 8 minutes on a pre-programmed path. Telerob used two robots: the well known tele-operated Telemax and another one equipped with a complex environment recognition provided by the University of Freiburg. The Jakobs University of Bremen used two UGV, called RugBots. The Universities of Oulu and Siegen, and TNO used one remoteoperated robot each. TNO lost communication during the run. The University of Siegen used an additional UAV.

[14]

robótica

(ICINCO), Angers, France , INSTICC Press, 2007 pp. 157-164 [7] Kaustubh Pathak, Andreas Birk, Soeren Schwertfeger, Ivan Delchef, and Stefan Markov “Fully Autonomous Operations of a Jacobs Rugbot in the RoboCup Rescue Robot League 2006”, International Workshop on Safety, Security, and Rescue Robotics (SSRR),IEEE Press, 2007.

[8] van der Mark, W., van den Heuvel, J.C.,Groen, “Stereo based Obstacle Detection with Uncertainty in Rough Terrain”, Intelligent Vehicles Symposium, 2007 IEEE, Istanbul, 2007, pp. 1005-1012.

[9] Research Institute for Applied Sciences http://www.fgan.de/fgan/fgan en.html [10] Civilian and Military ELROB website http://www.elrob.org/

EUROPEAN LAND ROBOTIC TRIAL 2008 (ELROB) – A REALISTIC BENCHMARK FOR OUTDOOR ROBOTICS

because their robot already showed a very impressive autonomous driving during the first part of the mission. The other participants did the first part of the mission remote-operated (BASE10, Diehl and Oulu) or by following a leading person (University of Siegen), which was permitted by the rules.

Like the team Kaiserslautern, the University of Hannover chose to let their robot act almost completely autonomously. That is the reason for their second place. Additionally they achieved the second longest distance (1.8 km), with the robot being in close range of the turning point when time was up. The University of Siegen earned the third place because they travelled the longest distance (see table V).