JAMRIS 2008 Vol 2 No 4 by JAMRIS

JOURNAL of AUTOMATION, MOBILE ROBOTICS & INTELLIGENT SYSTEMS

Editor-in-Chief Janusz Kacprzyk

Executive Editor: Anna Ładan aladan@piap.pl

(Systems Research Institute, Polish Academy of Sciences; PIAP, Poland)

Associate Editors: Mariusz Andrzejczak (PIAP, Poland) Katarzyna Rzeplińska-Rykała (PIAP, Poland)

Co-Editors: Dimitar Filev (Research & Advanced Engineering, Ford Motor Company, USA)

Kaoru Hirota

Webmaster: Tomasz Kobyliński tkobylinski@piap.pl

(Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology, Japan)

Witold Pedrycz

Marketing office: Małgorzata Korbecka-Pachuta mkorbecka@piap.pl

(ECERF, University of Alberta, Canada)

Roman Szewczyk Proofreading: Urszula Wiączek

(PIAP, Warsaw University of Technology, Poland)

Editorial Office: Industrial Research Institute for Automation and Measurements PIAP Al. Jerozolimskie 202, 02-486 Warsaw, POLAND Tel. +48-22-8740109, office@jamris.org

Editorial Board: Chairman: Janusz Kacprzyk (Polish Academy of Sciences; PIAP, Poland) Plamen Angelov (Lancaster University, UK) Zenn Bien (Korea Advanced Institute of Science and Technology, Korea) Adam Borkowski (Polish Academy of Sciences, Poland) Wolfgang Borutzky (Fachhochschule Bonn-Rhein-Sieg, Germany) Oscar Castillo (Tijuana Institute of Technology, Mexico) Chin Chen Chang (Feng Chia University, Taiwan) Jorge Manuel Miranda Dias (University of Coimbra, Portugal) Bogdan Gabryś (Bournemouth University, UK) Jan Jabłkowski (PIAP, Poland) Stanisław Kaczanowski (PIAP, Poland) Tadeusz Kaczorek (Warsaw University of Technology, Poland) Marian P. Kaźmierkowski (Warsaw University of Technology, Poland) Józef Korbicz (University of Zielona Góra, Poland) Krzysztof Kozłowski (Poznań University of Technology, Poland) Eckart Kramer (Fachhochschule Eberswalde, Germany) Andrew Kusiak (University of Iowa, USA) Mark Last (Ben–Gurion University of the Negev, Israel) Anthony Maciejewski (Colorado State University, USA) Krzysztof Malinowski (Warsaw University of Technology, Poland)

Andrzej Masłowski (PIAP, Poland) Tadeusz Missala (PIAP, Poland) Fazel Naghdy (University of Wollongong, Australia) Zbigniew Nahorski (Polish Academy of Science, Poland) Antoni Niederliński (Silesian University of Technology, Poland) Witold Pedrycz (University of Alberta, Canada) Duc Truong Pham (Cardiff University, UK) Lech Polkowski (Polish-Japanese Institute of Information Technology, Poland) Alain Pruski (University of Metz, France) Leszek Rutkowski (Częstochowa University of Technology, Poland) Klaus Schilling (Julius-Maximilians-University Würzburg, Germany) Ryszard Tadeusiewicz (AGH University of Science and Technology in Kraków, Poland)

Stanisław Tarasiewicz (University of Laval, Canada) Piotr Tatjewski (Warsaw University of Technology, Poland) Władysław Torbicz (Polish Academy of Sciences, Poland) Leszek Trybus (Rzeszów University of Technology, Poland) René Wamkeue (University of Québec, Canada) Janusz Zalewski (Florida Gulf Coast University, USA) Marek Zaremba (University of Québec, Canada) Teresa Zielińska (Warsaw University of Technology, Poland)

Publisher: Industrial Research Institute for Automation and Measurements PIAP

If in doubt about the proper edition of contributions, please contact the Executive Editor. Articles are reviewed, excluding advertisements and descriptions of products. The Editor does not take the responsibility for contents of advertisements, inserts etc. The Editor reserves the right to make relevant revisions, abbreviations and adjustments to the articles.

JOURNAL of AUTOMATION, MOBILE ROBOTICS & INTELLIGENT SYSTEMS VOLUME 2, N° 4, 2008

CONTENTS ARTICLES 7

59 Methods for classification of tactile patterns in general position by Helmholtz's equation

Vision – based mobile robot navigation S. A. Berrabah, E. Colon

14 Dense 3D structure and motion estimation as an aid for robot navigation G. De Cubber

64 A hybrid system control approach to biped robot control Y. Yin, S. Hosoe

19 A behaviour-based control and software architecture for the visually guided robudem outdoor mobile robot D. Doroftei, E. Colon, G. De Cubber

DEPARTMENTS 3

Design and motion synthesis of modular walking robot Mero I. Ion, A. Marin, A. Curaj, L. Vladareanu

EDITORIAL Professor Andrzej Masłowski

IN THE SPOTLIGHT Safe reaction of a robot arm with torque sensing ability on the external disturbance and impact: implementation of a new variable impedance control D. Tsetserukou, H. Kajimoto, N. Kawakami, S. Tachi

38 A new 3D sensor system by using virtual camera model and stereo vision for mobile robots H. Lee, M. Y. Kim, H. Cho

45 Development of a stabilized wideband pendulum-type gravity direction sensor utilizing an electromagnetic clutch and brake for anti-swing T. Okada, A. Tsuchida, T. Mukaiyachi, T. Shimizu

52 Parallel robot motion applied to flight simulator control law synthesis C. J. Szczepański

EVENTS

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 2,

N째 4

2008

Editorial Oscar Castillo*, Patricia Melin This Special Issue contains a selection of papers from the IMEKO TC-17 International Symposium on Measurements and Control in Robotics ISMCR'2007, extended and significantly improved over conference proceeding's version of the papers. There are ten papers in this Special Issue, containing a range of topics of the Measurements and Control in Robotics area. The first paper titled Vision - Based Mobile Robot Navigation written by Sid Ahmed Berrabah and Eric Colon presents a vision-based navigation system for mobile robots. It enables the robot to build a map of its environment, localize efficiently itself without any artificial markers or other modification, and navigate without colliding with obstacles. The Simultaneous Localization And Mapping (SLAM) procedure builds a global representation of the environment based on several size limited local maps built using the approach introduced by Davison et al. Two methods for global map are presented; the first method consists in transforming each local map into a global frame before to start building a new local map. While in the second method, the global map consists only in a set of robot positions where new local maps are started (i.e. the base references of the local maps). Based on the estimated map and its global position, the robot can fined a path and navigate without colliding with obstacles to reach a user-defined goal. The moving objects in the scene are detected and their motion is estimated using a combination of Gaussian Mixture Model (GMM) background subtraction approach and a Maximum a Posteriori Probability Markov Random Field (MAP-MRF) framework. Experimental results in real scenes are presented to illustrate the effectiveness of the proposed method. The second paper titled Dense 3D Structure and Motion Estimation as an aid for Robot Navigation by Geert De Cubber presents three-dimensional scene reconstruction as an important tool in many applications varying from computer graphics to mobile robot navigation. In this paper author focuses on the robotics application, where the goal is to estimate the 3D rigid motion of a mobile robot and to reconstruct a dense three-dimensional scene representation. The reconstruction problem can be subdivided into a number of sub problems. First, the egomotion has to be estimated. For this, the camera (or robot) motion parameters are iteratively estimated by reconstruction of the epipolar geometry. Secondly, a dense depth map is calculated by fusing sparse depth information from point features and dense motion information from the optical flow in a variational framework. This depth map corresponds to a point cloud in 3D space, which can then be converted into a model to extract information for the robot navigation algorithm. Here, an integrated approach for the structure and egomotion estimation problem has been presented. The third paper titled A Behaviour - based Control and Software Architecture for the Visually Guided Robudem outdoor mobile robot written by Daniela Doroftei, Eric Colon and Geert De Cubber presents the design of outdoor autonomous robots, requiring the careful consideration and integration of multiple aspects: sensors and sensor data fusion, design of a control and software architecture, design of a path planning algorithm and robot control. This paper describes partial aspects of this research work, which is aimed at developing a semi-autonomous outdoor robot for risky interventions. The authors focuses mainly on three main aspects of the design process: visual sensing using stereo - vision and image motion analysis, design of a behaviour -based control architecture and implementation of a modular software architecture. I. Ion, A. Marin, A. Curaj and L. Vladareanu contributed a paper titled Design and Motion Synthesis of Modular Walking Robot Mero. This paper describes the walking robots, which are built to displace the loads on the not-aligned terrain. The mechatronic walking system protect much better the environment when its contact with the soil is discrete, a fact that limits the area that is crushed appreciatively. This walking robot has three two-legged modules. Every leg has three freedom degrees and a tactile sensor to measure the contact, which consists of the lower and upper levels. The body of the walking robot carries a gyroscopic bearing sensor to measure the pitch and roll angles of the platform. The legs are powered by hydraulic drives and are equipped with potentiometric sensors, which are used to control the walking robot in the adaptability to a natural ground.

Editorial

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 2,

N° 4

Dzmitry Tsetserukou, Hiroyuki Kajimoto, Naoki Kawakami and Susumu Tachi contributed a paper titled Safe Reaction of a Robot Arm with Torque Sensing Ability on the External Disturbance and Impact: Implementation of a New Variable Impedance Control. The paper focuses on control of a new anthropomorphic robot arm enabling the torque measurement in each joint to ensure safety while performing tasks of physical interaction with human and environment. A novel variable control strategy was elaborated to increase the robot functionality and to achieve human-like dynamics of interaction. The algorithm of impact control imparting reflex action ability to the robot arm was proposed. The experimental results showed successful recognition and realization of three different types of interaction: service task, co-operative task, and impact state. The sixth paper, by Hyunki Lee, Min Young Kim and Hyungsuck Cho, describes A New 3D Sensor System by Using Virtual Camera Model and Stereo Vision for Mobile Robots. Intelligent autonomous mobile robots must be able to sense and recognize 3D indoor space where they live or work. So many researches have been conducted to develop 3D sensing method for mobile robots. Among them, the optical triangulation, a wellknown method for 3D shape measurement, is also based on active vision sensing principle for mobile robot sensor system, so that the measurement result is robust to illumination noises from environments. Due to this advantage it has been popularly used. However, to obtain the 3D information of environment it needs a special scanning process and scanning actuators need. To omit this scanning process multi-line projection methods have been widely researched. However, they suffer from an inherent limitation: The results of multi-line projection method commonly have measurement errors because of 2p-ambiguity caused by regularly repeated multi-line laser pattern. In this paper, to overcome 2p-ambiguity effectively, the authors introduce a novel sensing method for a 3D sensing system using multi-line projection and stereo cameras, based on the virtual camera model and stereovision algorithm. To verify the efficiency and accuracy of the proposed method, a series of experimental tests were performed. Another example of sensing methods is presented in the paper tilted Development of a Stabilized Wideband Pendulum-type Gravity Direction Sensor Utilizing an Electromagnetic Clutch and Brake for Anti-Swing authored by Tokuji Okada, Akihiro Tsuchida, Tomofumi Mukaiyachi and Toshime Shimizu. This paper proposes the usefulness of combining an electromagnetic clutch and brake with a pendulum for anti-swing for developing a stabilized wideband pendulum-type gravity direction sensor. The pendulum-type sensor is high in response to direction change but liable to swing even when the change stops, in general. Therefore, antiswing control without degrading sensitivity of direction measurement is important. Continuous damping of the pendulum motion causes an error in a stationary condition unless the pendulum is free from damping repetitively in an appropriate term. To develop a quick response direction sensor without error, the authors use a magnetic device. Based on motion analysis of a double pendulum, they simulate behaviours of the pendulum-type sensor by actuating the magnetic device by using different signals in form, frequency and magnitude. In the experiment it has been demonstrated that the device is beneficial for suppressing pendulum vibration over a wideband for measuring resultant direction of the acceleration compound with motion and gravity on such a platform moving randomly. Next paper titled Parallel Robot Motion Applied To Flight Simulator Control Law Synthesis written by Cezary J. Szczepański describes a method of synthesis of the parallel robot applied as flight simulator motion system control laws came from author's 20 year experience in developing, integrating and testing the control laws and their software dedicated to different simulators. The goal of the simulator motion control law synthesis at the proposed method is not minimizing a cost function taken a priori at the beginning of that synthesis process but achieving positive assessment by the operator (e.g. pilot) team on the basis of simulator motion perception. Procedures adopted for those simulators FAT (Final Acceptance Tests) within proposed method were based on standard military equipment testing methods. Performing the final and the most important tests by the real device operators (pilots) was the new element here. The other important modification of the classical method was introducing the simulated object acceleration derivative into the filters controlling the simulator motion system. It appeared to be particularly effective in the cases of highly maneuverable airplane simulators. The ninth paper tilted Methods for Classification of Tactile Patterns in General Position by Helmholtz's Equation co-authored by Jaromír Volf, Martin Dvoøák and Josef Vlèek describes new and original methods of tactile pattern recognition in general position applying the solution of Helmholtz's equation for a tactile transducer. Three groups of methods have been formed, based on: calculation of the A matrix eigen value, with the matrix being formed either from the whole pattern, or from the limit points; the scalar characteristic distribution of the components of the pattern's A matrix; and the geometrical properties of the A matrix of the pattern.

Editorial

2008

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

NÂ° 4

2008

Finally, the paper tilted A Hybrid System Control Approach to Biped Robot Control submitted by Yingjie Yin and Shigeyuki Hosoe review the state of the art of biped motion control in the field of hybrid system control. The main motivation is to fix the understanding of the research field and clarify strong and weak points of the available approaches. The presentation is illustrated by a typical example of biped machines.

I am grateful for the efforts and willing collaboration of authors who helped to improve the quality of papers and the value of this special issue.

Professor Andrzej MasĹ&#x201A;owski Member of IMEKO Technical Committee 17 Executive Chairman of the ISMCR' 2007 October of 2008

Editorial

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N째 4

2008

VISION-BASED MOBILE ROBOT NAVIGATION

Sid Ahmed Berrabah, Eric Colon

Abstract: This paper presents a vision-based navigation system for mobile robots. It enables the robot to build a map of its environment, localize efficiently itself without use of any artificial markers or other modifications, and navigate without colliding with obstacles. The Simultaneous Localization And Mapping (SLAM) procedure builds a global representation of the environment based on several size limited local maps built using the approach introduced by Davison et al. [1]. Two methods for global map are presented; the first method consists in transforming each local map into a global frame before to start building a new local map. While in the second method, the global map consists only in a set of robot positions where new local maps are started (i.e. the base references of the local maps). In both methods, the base frame for the global map is the robot position at instant t0. Based on the estimated map and its global position, the robot can find a path and navigate without colliding with obstacles to reach a goal defined the user. The moving objects in the scene are detected and their motion is estimated using a combination of Gaussian Mixture Model (GMM) background subtraction approach and a Maximum a Posteriori Probability Markov Random Field (MAP-MRF) framework [2]. Experimental results in real scenes are presented to illustrate the effectiveness of the proposed method. Keywords: robot navigation, vision-based SLAM, local and global mapping, adaptive fuzzy control.

1. Introduction In recent years, an increasing amount of robotic research has focused on the problem of map building, navigation and executing task motion autonomously, i.e. without human guidance. Such ability is essential for robotic systems in tile emerging field of service robotics, which includes security guarding, waste management, cleaning, and others. To reach a reasonable degree of autonomy, two basic requirements are sensing and reasoning. Sensing is provided by an on board sensory system that gathers information about the robot itself and the surrounding environment. Reasoning is accomplished by developing algorithms that exploit this information in order to generate appropriate commands for the robot. The reasoning system is the central unit in an autonomous robot. According to the environment state, it must allow the robot to localize itself in the environment and seek for free paths. To accomplish these two

tasks, it bases its reasoning on a model or a description of the environment. The environment model is not always available and hence the robot should have the means to build such a model over time as it explores its environments. This latest problem is known as mapping problem. Mapping and localization are interlinked problems: If the robot's pose (spatial position and orientation) is known all long, building a map would be quite simple. Conversely, if a map of the environment exists already, it will be very easy to determine accurately the robot's pose at any time. In combination, however, the problem is much harder. The literature refers to the mapping problem often in conjunction with the localization problem named as Simultaneous Localization and Mapping (SLAM) [3, 4] or Concurrent Mapping and Localization (CML) [5, 6, 7]. In this study, we focus on monocular vision-based SLAM where a single camera is moving through the scene. This is interesting because it offers a low-cost and realtime approach to SLAM in unprepared environments. The camera identifies natural features in the scene and uses these as landmarks in its map. The proposed approach is an extension to a real-time SLAM approach, proposed by Davison et al. [1], which recovers the 3D trajectory of a monocular camera, moving through previously unknown room-size indoor domains. The role of the map in [1] is primarily to permit real-time localization rather than serve as a complete scene description of the scene. This method is however not suitable in large environments [1], and particularly in case of perceptual aliasing, which refers to situations where several places are perceptually similar enough to be confused by the robot. There-fore, currently observable features alone may not be sufficient to uniquely identify the robot's true location. To overcome these problems, we propose the use of a 'history memory', which accumulates sensory evidence over time to identify places with a stochastic model of the correlation between map features. This also will allow obtaining a complete description of the scene organized as global map composed of several small local maps. An other problem for SLAM methods is change over time of the environment. Some changes may be relatively slow, such as the change of appearance of a tree across different seasons, or the structural changes that most office buildings are subjected to over time. Others are faster, such as the change of door status or the location of furniture items, such as chairs. Even faster may be the change of location of other agents in the environment, such as cars or people. To deal with this problem of dynamic scenes (with moving objects) we use an algorithm for motion segmentation [2] to remove the outliers features Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

which are associated with moving objects. Based on the estimated global map, the robot can fined a path to reach a user defined goal and it uses a procedure based on fuzzy logic control to navigate and avoid unplanned obstacles in the SLAM procedure and the detected moving obstacles. To predict the position of the detected moving objects, Kalman filter tracking procedure is applied to the output object mask of the motion detection process. The fuzzy logic controller for obstacle avoidance is equipped with reinforcement learning algorithm, which consists of a scalar reinforcement signal as a performance feedback from the environment enabling the navigator to tune itself. The rest of the paper is organized as follows: Section 2 describes the different parts of the proposed approach. Experimental results are shown in section 3, followed by conclusions in section 4.

2. Method overview 2.1.

Vision-based simultaneous localization and mapping The SLAM problem is tackled as a stochastic problem using an Extended Kalman Filtering to maintain the state vector, , consisting of the robot state, , and map feature states, . It also maintains a covariance matrix , which includes the uncertainties in the various states as well as correlations between the states. In [1], feature states are the 3D position vectors of the locations of point features and the (robot) camera state vector comprises a metric 3D position vector , orientation quaternion , velocity vector , and angular velocity vector relative to a fixed world frame and 'robot' frame carried by the camera (13 parameters):

(1)

Features are image patches with a size of 11 Ă&#x2014; 11 pixels. The patches are supposed locally as planar surfaces perpendicular to the vector from the feature to the camera at initialization. When making measurements of a feature from new camera positions, each patch can be projected from 3D to the image plane to produce a template for matching with the real image. This template will be a warped version of the original square template captured when the feature was first detected. To avoid using outlier features, the moving object mask detected by the motion segmentation procedure intro-duced in [2] is used. Subsequently, during map building, the detected features on the moving parts are excluded. The coordinate frame is chosen at the starting position and the system is helped at the beginning with an amount of prior information about the scene. A shape of a known target is placed in front of the camera, which provides several features with known positions and known appearance. This will help to know the scaling of the estimated map and an initialization of the map as with only a single camera, the features cannot be initia8

Articles

NÂ° 4

2008

lized on the map based only on one measurement because of their unknown depth. The system starts with zero uncertainty. A constant velocity model is considered and the robot (camera) state update is produced by:

(2)

where is the transition function and is a random vector describing the unmodelled aspects of the vehicle. and are respectively the impulse of velocity and the angular velocity caused by unknown acceleration and angular acceleration processes of zero mean and Gaussian distribution. denotes the quaternion trivially defined by the angle-axis rotation vector . Feature matching is carried out using straightforward normalization cross-correlation search for the template patch projected into the current camera estimate. The image is scanned at each location until a match is found. This searching for match is computationally expensive. Considering a perspective projection, the position at which the feature would be expected to be found in the image is found using the standard pinhole model: (3)

where and tion parameters

are the standard camera calibra-

is the 3D position of the feature relative to the camera. The EKF consists in prediction and update steps. At each time step of the filter we obtain the predicted state and covariance using the state transition function. The Jacobian , which appears as a result of the linearization process of the transition function , is used to transfer the map covariance, with the addition of the process noise covariance . (4) (5) The covariance of the state, , is therefore in block form, containing the vehicle covariance , the feature covariances, , the cross-covariances between features, , and the cross-covariances between the features and the vehicle, . The state transition function does not alter the feature states. A result of this is that its Jacobian is very simple:

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

(6)

and the process noise (7) The update to include a new measurement incorporates the innovation , which is the difference between the measurement and its prediction. (8) (9) Where (10)

(11) (12) and are block-diagonal matrices obtained empirically defining the error covariance matrices characterizing the noise in the model and the observations respectively. A measurement of feature is not related to the measurement of any other feature so (13) where is the measurement model for the 'th feature and then

(14)

2.2. Global mapping for large environment The SLAM algorithm presented in the previous sec-

2008

tions has two important limitations when mapping large environments. First, the computational cost of updating the map grows with ( = number_of_features + size_of_robot_stat). Second, as the map grows, the estimates obtained by the EKF equations quickly become inconsistent due to linearization errors [8]. To overcome these limitations, we use the previous algorithm to build small independent local maps of limited size and join them in global one. For joining the local maps we propose two approaches, the first method consists in transforming each local map into a global frame before to start building a new local map. While in the second method, the global map consists only in a set of robot positions where new local maps are started (i.e. the base references of the local maps). In both methods, the base frame for the global map is the robot position at instant t0, . Each local map can be built as follows: at a given instant , a new map is initialized using the current vehicle location, as base reference is the local map order. Then, the vehicle performs a limited motion acquiring sensor information about the neighbouring environment features . The EKF-based technique presented in the previous section is used to obtain a local map: (15) where is the coordinate vector in the base reference of the detected features and is their covariance matrix estimated in . The robot decides to start a new local map when the number of mapped features reaches a pre-defined threshold. 2.2.1. First method for global map building In this method the first local map is used as global map. Each finalized local map is transferred to the global map before starting a new one, by computing the state vectors and the covariance matrix. The goal of map joining is to obtain one full stochastic map: (16)

which depends only on the measurements of the feature and the vehicle state. After identification of the first measurement a new feature is to initialize a 3D line into the map along which the feature must lie. This is a semi-infinite line, starting at the estimated camera position and heading to infinity along the feature viewing direction. All possible 3D locations of the feature point lie somewhere along this line. To estimate the correct location (its depth), a set of discrete depth hypotheses is uniformly distributed along this line, which can be thought of as a one-dimensional probability density over depth represented by a 1D particle distribution or histogram. The observations of the new feature in the next few time-steps will provide information about its depth coordinate.

N째 4

where is a concatenation of all features from local maps 0, 1, 2... (17) The location of feature from the local map in the frame as follows:

is given

(18)

where

is the feature location Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

in the local map

and

N째 4

2008

cides to build a new local map. and

is the

frame reference in the global frame

The same way, the location of feature from local map is given in the frame as follows:

(19)

If at instant , the robot decides to build a new local map and its location in the base frame of the local map , is

then: (23)

where is the feature location in the local map

and In this case, for feature matching at instant , the robot uses the local map with closest base frame to its current location:

is the

frame reference in the global frame

: (24) (20)

The covariance of the joint map is obtained from the linearization of the state transition function . As the local maps are independent, the Jacobian (from linearization) is then applied separately to the local map covariance:

(21)

2.2.2. Second method for global map building In this method, the global map is limited to the set of the coordinates of the local maps frame origins. (22) where 10

Articles

are the instants when the robot de-

2.3. Path planning A path planning system is used then by the robot to navigate in its environment and avoid unplanned obstacles by the SLAM procedure and the detected moving obstacles. The position of the detected moving objects is predicted by a Kalman filter tracking procedure applied to the output object mask from the motion detection process [2] and the used sensors for static obstacles detection are infrared and ultrasound sensors. The local path planning procedure is based on two fuzzy logic controllers: a goal seeking controller and an obstacle avoidance controller. The goal seeking controller tries to find the optimal path to the intermediate goals (defined by the global path planning), while the obstacle avoidance controller has the mission to avoid obstacles. A command fusion scheme based on a conditioned activation for each controller arbitrates between the two behaviours. Simple fuzzy control for the obstacle avoidance behaviour is suitable to design autonomous mobile robots. However, it is difficult to maintain the correctness, consistency, and completeness of the fuzzy rule base for obstacle avoidance constructed and tuned by a human expert. Therefore, a fuzzy system able to evolve and automatically improve its performance is recommended. Several learning algorithms have been proposed to construct fuzzy systems automatically, such as back-propagation algorithm [10], table-lookup scheme [10], evolutionary algorithm [13], and reinforcement learning [11, 12]. In our application we used reinforcement learning as it is an unsupervised method which does not need training data. This learning method requires only a scalar reinforcement signal as a performance feedback from the environment enabling the navigator to tune itself [12]. According to requirements in present robotic software architectures, being object-oriented component

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

based reusable software patterns, we developed such a framework, named Controlling Robots with CORBA [14] (CoRoBa), written in C++ and based on CORBA, a standardized and popular middleware. CoRoBa relies on wellknown Design Patterns [15] and provides a component based development model. Concerning the tuning and simulation of navigation algorithms, a Java based multi mobile robot simulator (MoRoS3D) exists, and integrates seamlessly in the framework as described in detail in [16]. 2.3.1. Goal seeking controller Given the polar coordinates of the target in the robot frame, a Sugeno type fuzzy controller calculates the turn angle allowing the robot to reach the target. The transfer function of the goal seeking controller representing the dependency of the turn angle on the goal distance and the goal angle (polar coordinates) is shown in Fig. 1. We can see that the turn angle to apply to the robot to reach the target is directly proportional to the goal angle and inversely proportional to the goal distance with more importance to the goal coordinates. The goal distance and goal angle are fuzzified into 4 and 11 Gaussian fuzzy sets, respectively. The turn angle can take 11 crisp value in the interval [-180°,180°].

N° 4

2008

Fig. 2. Diagram of the arrangement of the robot sensors. 2.3.3. Reinforcement learning for the obstacle avoidance controller Reinforcement learning is a control strategy in which the robot embedded in an environment attempts to maximize total reward in pursuit of a goal [12]. At any time step, the robot should take a decision to avoid the obstacles or to reach it goal. When the decision is taken and the action performed, the decision process receives a feedback in the form of a reward from the environment which indicates if a good or bad decision to take in attempting to achieve the goal. As the learning concerns only the obstacle avoidance controller, the action corresponds to the possible scalar outputs of the controller and the states correspond to the different combinations of the input fuzzy variables of the controller. The learning algorithm used in our application is as follows: 1. Initialize ( ) to 0 for all state and action 2. Perceive current state 3. Choose an action according to action value function 4. Carry out action in the environment. Let the next state be and the reward be . 5. Update action value function from , and :

Fig. 1. The transfer function of the goal seeking controller. 2.3.2. Obstacle avoidance controller The 0-order Takagi-Sugeno fuzzy controller used for obstacle avoidance is based on the distances to the obstacle detected by the sonar and infrared sensors as well as the estimated distance to the detected moving obstacles. The front ultrasonic and infrared sensors of the robot are grouped in 7 groups as depicted in Fig. 2. The sensor data from groups G1 to G5 and the goal angle are used as inputs to the obstacle avoidance controller. Where as the data from the sensors of the groups G6 and G7 are used only in the mediation process. The output of the controller is the turn angle, which ranges between [-180°,180°]. th

The distance di measured by the i sensor group is expressed as: For

i = 1,..,5, (25) th

where dij is the distance measured by the j sensor of the sensor group i. The distance measured by each sensor group Gi is fuzzified into three Gaussian fuzzy sets.

where is a learning rate parameter and discounting factor between 0 and 1. 6. Return to 2.

is a fixed

3. Experimental results Figure 3 shows an example of SLAM using the proposed algorithm with the second method for global map building. Black squares describe the positions where the feature matching has passed from a local frame to an other. The ellipses around the features on the original frame represent the estimated covariance. The ellipses are drowning in straight line around non matched feature and in dots for matched features. The simulation of the robot navigational behaviour is done with simulator MoRoS3D. A simulator as such increases safety when developing and testing algorithms. In MoRoS3D a robot can be placed in a 3D environment and interact with that environment in a manner similar to that of the robot in the real physical situation. Although MoRoS3D visualizes the entire surroundings of the robot, the robot software only ”sees” the information it collects through its sensors, just like with a physical robot. The MoRoS3D simulator provides simple interaction with the user and offers different virtual cameras including onArticles

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 2,

N째 4

2008

board and tracking ones. Simple distance sensors, such as Laser, US and IR, are simulated. Figure 4 shows the training of the path planning system in a realistic environment and Figure 5 shows some results of the developed path planning system in some realistic cases.

Fig. 3. Example of Mapping and localization in a real scene.

Fig. 4. Diagram of the arrangement of the robot sensors.

Articles

Fig. 5. Results of the navigation strategy, presented in the MoRoS3D multi robot simulator.

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

4. Conclusion We presented a monocular-based EKF SLAM method. The proposed approach for SLAM is an extension to the method introduced by Davison et al. [1] to the cases of large environments with possible perceptual aliasing. For this, the proposed approach uses a history memory which accumulates sensory evidence over time to identify places with a stochastic model of the correlation between map features. This has allowed also building a complete description of the scene as a global map composed of several size limited local maps. Based on the estimated map and its global position, the robot can fined a path and navigate without colliding with obstacles to reach a user defined goal using a control system based on adaptive fuzzy logic.

AUTHORS Sid Ahmed Berrabah*, Eric Colon - Department of Mechanics, Royal Military Academy, Avenue de la Renaissance 30, 1000 Brussels, Belgium. E-mail: sidahmed.berrabah@rma.ac.be. * Corresponding author

References [1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

A. J. Davison, I. D. Reid, N. D. Molton, O. Stasse, “MonoSLAM: Real-Time Single Camera SLAM”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, issue 6, June 2007, pp.10521067. S.A. Berrabah, G. De Cubber, V. Enescu, H. Sahli, “MRFBased Foreground Detection in Image Sequences from a Moving Camera”. In: IEEE International Conference on Image Processing (ICIP 2006), Atlanta, GA, USA, Oct. 2006, pp.1125-1128. A. J. Davison, Y. G. Cid, N. Kita, “Real-time 3D SLAM with wide-angle vision”. In: Intelligent Autonomous Vehicles, Lisboa-Portugal, Jul. 2004. J. Folkesson, P. Jensfelt, H. Christensen, “Graphical SLAM using vision and the measurement subspace”. In: IEEE/JRS -Intl Conf. on Intelligent Robotics and Systems (IROS), Edmonton-Canada, Aug 2005. F. Andrade-Cetto, A. Sanfelin, “Concurrent Map Building and Localization with landmark validation”. In: 16th International Conference on Pattern Recognition ICPR/02, 2002, vol.2. J. W. Fenwick, P. M. Newman, J. J. Leonard, “Cooperative Concurrent Mapping and Localization”. In: Proceedings of the 2002 IEEE International Conference on Robotics and Automation, May 2002, Washington, USA, pp. 1810-1817. S. Thrun, D. Fox, W. Burgard, “A probabilistic approach to concurrent Mapping and Localization for Mobile Robots”, Machine Learning, vol.31, no. 1-3, 1998, pp. 29-53. J.D. Trados, J. Neira, P. Newman, J. Leonard, “Robust mapping and localization in indoor environments using sonar data”, International Journal of Robotics Research, 2002, N. 21, pp. 311-330. M. W. M. Gamini Dissanayake, P. Newman, S. Clark, H. F.

[10] [11]

[12]

[13]

[14]

[15]

[16]

N° 4

2008

Durrant-Whyte, M. Csorba, “A Solution to the Simultaneous Localization and Map Building (SLAM) Problem”, IEEE Transactions on Robotic and Automation, 2001, vol. 17, no. 3, pp. 229-241. L.X. Wang, Adaptive Fuzzy Systems and Control, PTR Prentice Hall, 1994. A. Bonarini, “Delayed reinforcement, fuzzy Q-learning and fuzzy logic controllers”. In: Genetic Algorithms and Soft Computing, Herrera, F. and Verdegay, J.L. (Eds.), Physica-Verlag, 1996. M.S. Kim, S.G. Hong, J.J. Lee, “Self-Learning Fuzzy Logic Controller using Q-Learning”, Journal of Advanced Computational Intelligence and Intelligent Informatics, vol.4, no. 5, 2000, pp. 349-354. A. Gonzalez, R. Perez, A Learning System of Fuzzy Control Rules Based on Genetic Algorithms, Genetic Algorithms and Soft Computing, Studies in Fuzziness and Soft Computing, vol. 8, Physica-Verlag, September 1996, pp. 202-225. M. Henning, S. Vinoski, Advanced corba programming with C++, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1999. E. Colon, H. Sahli, and Y. Baudoin, “Coroba, a multi mobile robot control and simulation framework”, Special Issue on “Software Development and Integration in Robotics” of the International Journal on Advanced Robotics, vol. 3, 2006, no. 1, pp. 73-78. Thomas Geerinck, Eric Colon, Sid Ahmed Berrabah, and Kenny Cauwerts, “Tele-robot with shared autonomy: Distributed navigation development framework”, Integrated Computer-Aided Engineering (ICAE), vol. 13, no. 4, 2006, pp. 329-346.

Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N째 4

2008

DENSE 3D STRUCTURE AND MOTION ESTIMATION AS AN AID FOR ROBOT NAVIGATION Geert De Cubber

Abstract: Three-dimensional scene reconstruction is an important tool in many applications varying from computer graphics to mobile robot navigation. In this paper, we focus on the robotics application, where the goal is to estimate the 3D rigid motion of a mobile robot and to reconstruct a dense three-dimensional scene representation. The reconstruction problem can be subdivided into a number of subproblems. First, the egomotion has to be estimated. For this, the camera (or robot) motion parameters are iteratively estimated by reconstruction of the epipolar geometry. Secondly, a dense depth map is calculated by fusing sparse depth information from point features and dense motion information from the optical flow in a variational framework. This depth map corresponds to a point cloud in 3D space, which can then be converted into a model to extract information for the robot navigation algorithm. Here, we present an integrated approach for the structure and egomotion estimation problem. Keywords: outdoor mobile robots, behavior-based control, stereo vision and image motion analysis for robot navigation, modular control and software architecture (MCA).

1. Introduction Recovering 3D-information has been in the focus of attention of the computer vision community for a few decades now, but as so far no all-satisfying method has been found. Most attention in this area has been on stereovision-based methods, which use the displacement of objects in two (or more) images. The problem with these vision algorithms is that they require the matching of feature points, which is not easy for untextured surfaces. Where stereovision must be seen as a spatial integration of multiple viewpoints to recover depth, it is also possible to perform a temporal integration. The problem arising in this situation is known as the "Structure from Motion" (SfM) problem and deals with extracting 3-dimensional information about the environment from the motion of its projection onto a two-dimensional surface [4]. In general, there are two approaches to SfM. The first, feature based method is closely related to stereovision. It uses corresponding features in multiple images of the same scene, taken from different viewpoints. The basis for feature-based approaches lies in the early work of Longuet-Higgins [8], describing how to use the epipolar geometry for the estimation of relative motion. In this article, the 8-points algorithm was introduced. It features a way of estimating the relative camera motion, using the essential matrix, which constrains feature points in 14

Articles

two images. The first problem with these feature-based techniques is of course the retrieval of correspondences, a problem that cannot be reliably solved in image areas with low texture. From these correspondences, estimates for the motion vectors can be calculated, which are then used to recover the depth. An advantage of feature-based techniques is that it is relatively easy to integrate results over time, using bundle adjustment [12] or Kalman filtering [5]. Bundle adjustment is maximum likelihood estimators that consist in minimizing the re-projection error. It requires a first estimate of the structure and then adjusts the bundle of rays between each camera and the set of 3D points. The second approach for SfM uses the optical flow field as an input instead of feature correspondences. The applicability of the optical flow field for SfM calculation originates from the epipolar constraint equation which relates the optical flow u(u,v) to the relative camera motion (translation t and rotation ) and 3D structure, represented by the depth parameter d, in a non-linear fashion, as indicated by equation (1). (1) In [6], Hanna proposed a method to solve the motion and structure reconstruction problem by parameterizing the optical flow and inserting it in the image brightness constancy equation. More popular methods try to eliminate the depth information first from the epipolar constraint and regard the problem as an egomotion estimation problem. Bruss & Horn already showed this technique in the early eighties using substitution of the depth equation [3], while Jepson & Heeger used later algebraic manipulation to come to a similar formulation [7]. The current state-of-the art in SfM systems considers the construction of sparse feature-based scene representations, e.g. from points and lines. The main drawback of such systems is the lack of surface information, which restricts their usefulness, as the number of features is limited. In the past, optical flow - based SfM methods such as the famous Bruss & Horn [3] and Jepson & Heeger [7] algorithms were also mainly aimed at motion and structure recovery using very low-resolution optical flows. With the increase in available processing power, however, the SfM community is now trying to address the dense reconstruction problem. The optical flow based SfM approaches are more suited to address dense reconstruction problem, as they can go out from the optical flow over the whole image field. This leads to an approach as sketched by Figure 1 showing two main input paths to the dense reconstruction: sparse epipolar reconstruction and

Journal of Automation, Mobile Robotics & Intelligent Systems

dense optical flow estimation.

Fig. 1. The general approach of the proposed dense 3D reconstruction algorithm: merging sparse information (epipolar geometry) with dense information (optical flow). In order to bring the advantages of both sparse and dense SfM theorems together, we will try here to fuse both methods into an integrated structure recovery algorithm. In the context of this research work, the situation with a static scene and a dynamic observer is envisaged. The constraint of the static scene can be lifted by incorporating an independent motions segmentation pre-processing algorithm, segmenting the recorded images according to different motion patterns. The SfM analysis is then performed once for each of the motion patterns.

2. Sparse structure and motion estimation 2.1. Feature detection and matching Every sparse structure from motion approach starts off with feature detection and matching to acquire the necessary input data for subsequent processing steps. The aim is to estimate the image locations of a point belonging to a certain 3D structure in different camera views. The landmark or feature point detection was usually performed with the Harris corner detector, but recently, the Scale Invariant Feature Transform (SIFT) is used more and more. A complete use of the SIFT approach has been presented in [10] for reliable point matching between different views of an object or scene. SIFT features are located at scale-space maxima and minima of a difference of Gaussian function. Since the vector of gradients consists of differences of intensity values, it is invariant to affine changes in intensity. Due to these advantageous properties, the SIFT feature detector was selected for our application. The proposed feature detection and matching method aims at combining the advantages of the SIFT descriptor (robust detector and descriptor) with the advantages of feature tracking, where features can be tracked over longer time spans. On the subject of feature tracking, our choice went to an implementation of the Kanade-LucasTomasi (KLT) Feature Tracking algorithm as proposed by Stan Birchfield in [2]. The Kanade-Lucas-Tomasi (KLT) Feature Tracking algorithm uses the window-based technique proposed in [9], because it is simple, fast, and gives accurate results if windows are selected appropriately. A hybrid feature matching algorithm was set up, beginning with a SIFT feature detection and description process. A KLT-based tracker then tracks the detected features.

VOLUME 2,

N째 4

2008

2.2. Structure and motion estimation The first step of the proposed structure and motion estimation procedure consists of an estimation of the optimal frame rate using the Geometric Robust Information Criterion (GRIC), introduced by Torr in [11]. This step involves the computation of a scoring function evaluating the goodness of fit of the model. The optimal frame rate for each frame is set to be the lowest time step for which the GRIC-scoring function for the fundamental matrix model provides a higher value than the GRIC-scoring function for the homography model. To come to one consistent frame rate for the whole image sequence, a globally optimal frame rate is calculated by taking an average of the individual optimal time steps for each frame. In the second stage of reconstruction, estimating the trifocal tensors across image triplets performs threeview geometry reconstruction. The trifocal tensor is a 3 x 3 x 3 array of numbers that relate the coordinates of corresponding points or lines in three views. The trifocal tensor estimation algorithm takes as input 6 random correspondences across 3 views, which are used by a non-linear estimation method to compute a first estimate of the trifocal tensor. In a following stage, the support for the proposed trifocal tensor is measured by counting the number of matches that follow the projection model proposed by the given trifocal tensor. This process is repeated numerous times and the trifocal tensor estimate with the largest number of inliers is chosen. In a final step, the trifocal tensor is re-estimated with a linear method by only taking into account the inlier correspondences. After the trifocal tensors are estimated, the fundamental matrices and camera matrices can be calculated by decomposing the trifocal tensor. The fundamental matrix F encapsulates the intrinsic geometry between two views. It is a 3 x 3 matrix of rank 2. It is straightforward to compute the fundamental matrices F21 and F31 between the first and the other views from the trifocal tensor, once the epipoles e' and e'' are known, following equations (2) and (3). (2) (3) The camera matrix P expresses the action of a single projective camera on a point in space in terms of a linear mapping of a homogeneous 3D scene point to a homogeneous 2D image point . In homogeneous coordinates this mapping is written as . In the general case, the camera matrix is a 3 x 4 matrix of rank 3, made up of a camera calibration matrix K and a rotation matrix R and translation vector t, relating the camera position and orientation to the world coordinate system. The camera matrices can be calculated from the trifocal tensor by applying equations (4): (4)

Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N째 4

2008

The information enclosed in the estimated fundamental matrices is now employed to estimate the camera motion parameters. For this, the essential matrix , which is the specialization of the fundamental matrix to the case of normalized image coordinates, is calculated from the fundamental matrix. As the essential matrix can also be written as , it is clear that the rotation matrix and translation vector can then be estimated by applying singular value decomposition. A problem with this egomotion estimation process is that in general four different solutions are obtained. The correct estimate for rotation matrix and translation vector can be found back by imposing the constraint that reconstructed point locations must lie in front of both cameras. In the next step, the 3D structure data is reconstructed by triangulating all matched pixels to their 3D location. Nonlinear optimization is applied to estimate the point 3D location when multiple matches over time are available. Up until this point in the SfM estimation procedure, the 3D motion and 3D reconstructions between image triplets were unrelated. Multi view merging addresses this issue. Merging is achieved by estimating the relative scale between the estimated structures and estimating the space homography between the different cameras using a linear least squares method. This space homography is then applied to bring all reconstructions to the same projective basis. As the last step of the structure and motion estimation process, bundle adjustment is used to produce globally optimal 3D structure and camera motion estimates. This is achieved by minimizing the reprojection error, as expressed by equation (5), by a Levenberg-Marquadt nonlinear least squares algorithm, taking advantage of the sparse structure of the system matrices.

dense reconstruction process, as sketched by Figure 1, needs to be present in the constraint equations. However, only using this information would lead to problems at spatial (image) and temporal (movement) discontinuities. Therefore, an anisotropic smoothing term was added to preserve the depth discontinuities at image discontinuities. Here, we'll elaborate more on the different constraint equations that can be used for this purpose. The image brightness constraint is based upon the Lambertian assumption that corresponding pixels have equal grey values. To express this, Alvarez first derived a simplified expression for the disparity that is based upon the knowledge of the epipolar geometry, calculated before by the sparse structure and motion estimation algorithms. This formulation can be expressed as:

(5)

where the integration domain is the image field and m is a regularization parameter. This formulation can be introduced into the Euler-Langrange equation. Eventually, we retrieve:

3. Dense structure and motion estimation 3.1. The optical flow Optical flow is the distribution of apparent velocities of movement of brightness patterns in an image. Optical flow can arise from relative motion of objects and the viewer. Consequently, optical flow can give important information about the spatial arrangement of the objects viewed and the rate of change of this arrangement. The optical flow estimation technique applied here relates the image correspondence problem from optical flow to other modules such as segmentation, shape and depth estimation, occlusion detection and signal processing.

(6) where l1 and l2 represent two image frames and l is a depth parameter to be estimated. The constraint above does not contain any diffusion terms in feature space. To increase the numerical stability, we add a regularization term. This term has to ensure that discontinuities and smooth areas are well preserved by the reconstruction process. We chose to use the Nagel and Enkelmann regularization model, as this method has already been proven successful in a range of independent experiments. The regularization term has the following form: (7) Where D is a regularized projection matrix, leaving the energy functional to be minimized as: (8)

(9)

The Euler-Lagrange equation can be solved, provided that an initial condition is given, by calculating the asymptotic state. The initial condition is a back projected depth map, which we calculate by inserting into equation (1) the dense optical flow estimate and the estimated motion vectors calculated earlier.

4. Results 3.2. Dense reconstruction To reconstruct a dense depth field, it is necessary to maximize the information that can be retrieved out of the given data. To tackle the various data inputs and constraints imposed on the depth reconstruction, energy based methods are very well suited. Here, we follow the approach proposed by Alvarez in [1]. Alvarez proposes an energy-based approach to estimate a dense disparity map between two images. Each of the two input paths to the 16

Articles

4.1.

A comprehensive test & analysis environment The main problem in evaluating the performance of any 3D reconstruction algorithm is the absence of quality ground truth data. Available data on the Internet most often only consist of series of images with some camera data. In order to overcome this problem, we went out from an artificial 3D scene and added a well-defined

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

camera that we set up to follow a predefined trajectory. By doing so, we were able to control all variables - depth information, camera calibration data and camera motion - needing to be estimated by the structure estimation algorithms. We then made photorealistic renderings of the scene as seen by the camera at different time steps. These renderings serve as base data for the image processing algorithms. Also the depth information was exported at this stage by constructing depth maps at each time frame. This was achieved by rendering all data as a single Rich Pixel Format (RPF) file. RPF is a multilayered high precision data format developed for integrated 3D data storage. Figure 2 shows the 3D model along with the camera trajectory and some frames together with their depth map rendering, taken from a 40-frame sequence. Based on this data, it is now possible to compute for any pixel of an image the ground truth corresponding pixel in any of the cameras. This allows us to extract useful data to analyze the calculation chain of the algorithm. For feature matching between two camera views, ground truth data can be provided by projecting the point in the first camera to its 3D location and by back projecting this 3D point onto the image plane of the second camera. For epipolar geometry reconstruction, the ground truth camera matrices can be calculated based upon the ground truth motion data, which is also useful to analyze the accuracy of the egomotion estimation.

NÂ° 4

2008

4.2. Feature detection & matching Figure 3 shows the tracked correspondences, compared to the ground truth feature movement. Feature Matches for Camera 10

100

150

200

250 0

100

150

200

250

Fig. 3. Matches for the different feature points (dashed line), compared to the ground truth feature motion (dotted line). As can be noticed, the KLT-tracking algorithm performs very well. Only in the lower right quadrant of the image, the measurement somewhat differs from the ground truth movement due to the relatively large vertical motion field present in this quadrant, while in the other parts of the image, the motion field is mostly horizontal. 4.3. Sparse reconstruction As results for the sparse reconstruction algorithms, we only show the estimated motion vectors on Figure 4, as these results show the applicability of the proposed multi-view reconstruction technique. Figure 4 shows respectively the translation and rotation vector for some camera views and compare them with the ground truthvalue. To obtain these results, we imposed an extra check in the algorithm such that all vectors acquire the dominant sign computed for the sequence. Doing so makes the translation and rotation vector converge to within an acceptable error margin of the ground truthvalue, except for one single camera where the estimate for the rotation vector is seriously wrong.

Fig. 2. From left to right: the 3D wire frame and model and the camera trajectory used for the test environment (the original 3D model was developed by Jerome Vaucanson in AutodeskÂŽ 3D Studio MaxÂŽ), photorealistic rendering and associated depth map rendering for some frames of the Seaside sequence. This leads to a comprehensive testing and analysis workflow for the evaluation of SfM algorithms. Current approaches are mostly limited to the reconstruction one specific scene. It is firstly hard to quantify the accuracy of a 3D reconstruction on paper and secondly, this leads the way for tuning algorithms towards certain scenes. In our workflow, it is straightforward to change the 3D scene and to produce photorealistic images with ground truth data. It is our hope that with sound benchmarking techniques, the performance of different structure estimation techniques can be accredited in a more reliable fashion.

Fig. 4. Estimates of the 3D motion vectors for different camera views compared to the ground truth motion in white. Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

4.4. Dense reconstruction In order to preserve stability, we chose to use a semiimplicit numerical scheme to calculate the depth field iteratively. Figure 5 compares the obtained result from dense reconstruction to the ground truth depth map. It is clear that artefacts are still visible, but the relative depths can be discerned very well. Calculation time for this estimation is about 5 minutes on a 3.0GHz CPU. Ground truth depth map

[1]

[2]

[3]

[5]

[6] Depth map after iteration

[7]

[8]

[9]

[10]

Fig. 5. The ground truth depth map and the depth map retrieved after dense reconstruction. [11]

5. Conclusions and future work

AUTHOR Geert De Cubber - Royal Military Academy, Department of Mechanical Engineering (MSTA), Av. de la Renaissance 30, 1000 Brussels. E-mail: Geert.De.Cubber@rma.ac.be

Articles

2008

References

[4]

In this paper, I proposed an approach towards dense depth reconstruction. The approach aims to combine the strength of the more robust feature-based structure from motion approaches with the spatial coherence of dense reconstruction algorithms. To achieve this, a variation framework was set up, minimizing the epipolar reprojection error and the image brightness constraint, while preserving discontinuities in the depth field by introducing an anisotropic diffusion term. The dense optical flow information is back projected and serves as initial guess for the iterative solver. The resulting depth maps can serve as a very useful input for a robot navigation planner as they provide rich information about the environment. Using this data for robotic navigation tasks in outdoor environments will be one of the first issues to address with respect to future work.

N° 4

[12]

L. Alvarez, R. Deriche, J. Sanchez, and J. Weickert, “Dense disparity map estimation respecting image derivatives: a PDE and scale-space based approach”, Journal of Visual Communication and Image Representation, vol. 13(1/2), 2002, pp. 3-21. Stan Birchfield. Klt, An implementation of the kanadelucas-tomasi feature tracker, January 1997. Available at: http://www.ces.clemson.edu/stb/klt/. A.R. Bruss and Berthold K.P. Horn, “Passive navigation”, Computer Vision, Graphics and Image Processing, vol. 21, January 1983, pp. 3-20. A. Chiuso, P. Favaro, H. Jin and S. Soatto, “Structure from motion causally integrated over time”, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 24(4), 2002, pp. 523-535. H. Jin, P. Favaro, and S. Soatto, “A semi-direct approach to structure from motion”, The Visual Computer, vol. 19(6), October 2003, pp. 377-394. K.J. Hanna, “Direct multi-resolution estimation of egomotion and structure from motion”. In: Workshop on VisualMotion, October 1991, pp. 156-162. D.J. Heeger and A.D. Jepson, “Subspace methods for recovering rigid motion: Algorithm and implementation”, International Journal of Computer Vision, vol. 7(2), 1992, pp. 95-117. H.C. Longuet-Higgins, “A computer algorithm for reconstructing a scene from two projections”, Nature, no. 293(5828), September 1981, pp. 133-135. B. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision”. In: Int. Conf. on Artificial Intelligence, Vancouver, 1981, pp. 674-679. David G. Lowe, “Distinctive image features from scaleinvariant keypoints”, International Journal of Computer Vision, vol. 60(2), 2004, pp. 91-110. P. H. S. Torr, “Bayesian model estimation and selection for epipolar geometry and generic manifold fitting”. Int. J.Comput. Vision, vol. 50(1), 2002, pp. 35-61. B. Triggs, P.F. Mclauchlan, R.I. Hartley and A.W. Fitzgibbon, “Bundle adjustment - a modern synthesis, Lecture Notes in Computer Science, vol. 1883, 2000, pp. 298 - 372.

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N째 4

2008

A BEHAVIOUR-BASED CONTROL AND SOFTWARE ARCHITECTURE FOR THE VISUALLY GUIDED ROBUDEM OUTDOOR MOBILE ROBOT Daniela Doroftei, Eric Colon, Geert De Cubber

Abstract: The design of outdoor autonomous robots requires the careful consideration and integration of multiple aspects: sensors and sensor data fusion, design of a control and software architecture, design of a path planning algorithm and robot control. This paper describes partial aspects of this research work, which is aimed at developing a semiautonomous outdoor robot for risky interventions. This paper focuses on three main aspects of the design process: visual sensing using stereo vision and image motion analysis, design of a behaviourbased control architecture and implementation of modular software architecture. Keywords: control, mobile robot, software architecture.

1. Introduction For decades, autonomous robotics has been popular research area, yet the amount of real intelligent autonomous outdoor robots applied on the field is still very limited. The research project presented here has as a goal to develop intelligent autonomous robotic agents which can assist humans for various types of risky outdoor interventions (surveillance, crisis management, ...). The challenges for such a robotic system are tremendous and span various fields of research: from sensing and sensor fusion to modelling and control, map building and path planning, decision making and autonomy, to the final integration of all these components. Three of these aspects are investigated more profoundly in this paper: visual sensing, robot control and software architecture. The following paragraphs give an overview of the different existing algorithms and design choices for these different components. Robotic agents can rely on numerous types of sensors to gain knowledge about the environment or about itself and its position. These sensors include infrared sensors, ultrasound sensors, laser range scanners, GPS, inertial measurement units, etc. Cognitive science and biological examples pointed out the importance of visual sensing, which led to the application of computer vision algorithms like stereo vision [12], obstacle detection [6], person following [8], visual servicing [10] to robotics and it eventually also led to mixed paradigms like visual simultaneous localisation and mapping (VSLAM) [4]. However, integration of vision modules into control architecture for an autonomous mobile robot is more difficult than just adding the vision components. [17] This is due to the high bandwidth and processing requirements of vision sensors, which require a task-specific configuration of vision-based behaviours. Another draw-

back of many computer vision algorithms is the lack of stability and robustness when confronted with varying illumination conditions, as in outdoor situations, although illumination-invariant algorithms have been proposed [5]. For visual sensing, the Robudem robot is outfitted with a stereo camera system, consisting of two digital cameras. In order to maximize the information stream towards the navigation unit, two different visual processing techniques are used: stereo vision and image motion analysis. An autonomous mobile robot must be self-reliant to operate in complex, partially known and challenging environments using its limited physical and computational resources. Its control system must ensure in real time that the robot will achieve its tasks despite all these constraints. [11] One of the first robot control architectures was the Sense Model Plan Act (SMPA) paradigm. The primary drawback of this approach is that the series of stages through all sensor data must pass places an unavoidable delay in the loop between sensing and action. To counter this drawback, alternatives, such as the behaviour-based approach, were proposed. In behaviour-based control, the control of a robot is shared with a set of purposive perception-action units, called behaviours. [14] Based on selective sensory information, each behaviour produces immediate reactions to control the robot with respect to a particular objective, a narrow aspect of the robot's overall task such as obstacle avoidance or wall following. Behaviours with different and possibly incommensurable objectives may produce conflicting actions that are seemingly irreconcilable. Thus a major issue in the design of behaviour-based control systems is the formulation of effective mechanisms for coordination of the behaviours' activities into strategies for rational and coherent behaviour. This is known as the action selection problem. Numerous action selection mechanisms have been proposed over the last decade; a qualitative overview can be found in [7]. The behaviour-based controller presented here uses of statistical reasoning on the output data of each behaviour to determine the stability and reliability and therefore also the activity level of seven behaviours, each proposing a (different) velocity and turning setup value for the robot actuators. Today, robot control architectures become more and more complex, as human reasoning is mimicked. Moreover, there is a significant portion of robot functionality that is common to a large number of robotic systems in different application domains. Unfortunately, most functionality implementations are tied to specific robot hardware, processing platforms, and communication environments. Most research and development in software for Articles

Journal of Automation, Mobile Robotics & Intelligent Systems

robotic systems is based on proprietarily designed architectures invented from scratch each time. To avoid this problem, the choice of flexible, extendable and real-time capable software architecture is very important. This software architecture has to ease the use of reusable and transferable software components. Multiple software architectures, like Orocos [2], Player/Stage [9], Campout [15], CoRoBa [3] â&#x20AC;Ś have been proposed in the past, all with their strengths and weaknesses. The existence of such a multitude of software frameworks hasn't helped the standardisation of robot software architectures. In the course of this research project, the Modular Controller Architecture (MCA) [18] was used. MCA is a modular, network transparent and realtime capable framework tailored to the control of autonomous robots. The remainder of this paper is structured as follows: first the visual sensing algorithms are explained in detail; then a behaviour based robot control scheme is proposed after which the implemented software architecture is introduced.

2. Visual sensing 2.1. Stereo vision Stereo vision employs the difference in location between two cameras. This difference in space leads to two images where the same points can be found at different positions. The goal of stereo disparity estimation is finding the correct correspondences between image points from the left and right camera. For this, we employ the algorithm presented by Birchfield et al. in [1]. The algorithm matches individual pixels in corresponding scan line pairs while allowing occluded pixels to remain unmatched, and then propagates the information between scan lines. The algorithm handles large untextured regions, uses a measure of pixel dissimilarity that is insensitive to image sampling, and prunes bad search nodes to increase the speed of dynamic programming. The output of this algorithm is a dense depth map of the area in front of the cameras, as shown in Figure 1. On the depth map in Figure 1, nearby objects appear dark. The cross on top marks the location of the closest obstacle, which is the darkest point on the depth map and which corresponds here to the obstacle in front of the robot. The data-content of this dense depth map must now be reduced to be useful for the navigation controller. For this, we use the approach proposed by Schafer in [16]. Following this approach, the dense depth map is down projected onto the ground surface, such that it can be represented as a 2D line. This data is further reduced in

VOLUME 2,

NÂ° 4

dimensionality by calculating from the depth line the distance to the nearest obstacle on the left dl , in the middle dc, and on the right dr , respectively. 2.2. Image motion analysis Motion analysis can provide extra information about the environment. The rationale behind the usage of the image motion for navigation purposes is that when large image motion is detected, this is likely due to objects close to the camera (and thus close to the robot), which should trigger the obstacle avoidance module. On the other hand, when few image motions are detected, this means that the way in front of the camera is probably quite clear of obstacles. Multiple techniques stand at our disposal to estimate the image motion. These methods differ in their approach towards the main problem to be solved in image motion: the background estimation and subtraction process. As the camera system is installed on a moving robot system, background estimation is particularly difficult in this case, as it is very hard to build up a model of the background over a large amount of time. This constraint the use limits of traditional advanced background estimation techniques like kernel density estimators, mean shift or mixtures of Gaussians [13]. As a result, the frame difference between successive frames was employed to find back the moving objects. As expressed by equation (1), the motion mk for each pixel is robustly estimated by calculating the frame difference when the difference is above a certain threshold which is dependent on the robot velocity V. (1)

With c a constant describing the relation between robot speed and image motion. Figure 2. shows this image motion field as calculated by the right robot camera. The resulting motion field m is then summed over the whole image to obtain one single motion measure for the camera image. (2) This calculation is performed once for the left camera and once for the right camera image, leading to two distinct image motion estimates.

Left

Centre

Right

Fig. 1. Stereo Processing a) Left camera image; b) Right camera image; c) Dense depth map (white=far, dark = near); d) Depth line with nearest distances to obstacles on the left, in the middle and on the right. 20

Articles

2008

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 2,

N° 4

2008

Figure 2. Image motion field of right camera: a) Image at ti-1 ; b) Image at ti ; c) Image motion field.

3. Behaviour-based robot control 3.1. Behaviour-based framework The control architecture describes the strategy to combine the three main capabilities of an intelligent mobile agent: sensing, reasoning and actuation. These three capabilities have to be integrated in a coherent framework in order for the mobile agent to perform a certain task adequately. To combine the advantages of purely reactive and planner-based approaches, this research work aims at implementing a behaviour-based controller for autonomous navigation. For the Robudem robot, three main sensing capabilities are installed: odometry, stereo vision, image motion analysis. These three sensing capabilities are processed by separate behaviours. In this context, the odometry information is used for guiding the robot to a given goal position, while the visual sensors are used for obstacle avoidance. The main advantage of using behaviour-based approaches is that each of the different behaviours can be designed and implemented separately. For the Robudem robot, three main sensing capabilities are installed for now: odometry, stereo vision, image motion analysis. These three sensing capabilities are processed by separate behaviours. In this context, the odometry information is used for guiding the robot to a given goal position. The general scheme of the behaviour-based navigation module is shown in Figure 3.

Fig. 3. General scheme of the behaviour-based navigation module. The output of this behaviour-based navigation planner is a velocity V and turn angle T command to be sent to the robot. How these properties are estimated is explained in detail in the following section. 3.2. Behaviour design and fusion The velocity V and turn angle T sent to the robot are dependent on the individual input of each of the different behaviours: goal seeking, stereo vision and image motion. Therefore, each of these behaviours calculates its own velocity and turn angle for the robot to be performed. These values are then weighed according to the activity

level A of the specific behaviour according to the formulation of equation (3). The activity level of behaviour describes to which degree this behaviour is relevant for the calculation of the final robot command.

(3)

With V and T, respectively, the velocity and turning command for the robot; AE, AF, AR the activity level for, respectively, the Emergency Stop and the Front or Rear Steering behaviour; AS, AM, AP the activity levels for, respectively, the Stereo Vision, Image Motion and Goal Seeking behaviour; VS, VM, VP the Velocity commands from, respectively, the Stereo Vision, Image Motion and Goal Seeking behaviour; TS, TM, TP the turn angle commands from, respectively, the Stereo Vision, Image Motion and Goal Seeking behaviour. When the Emergency Stop is activated, no robot movement is possible, which is a security measure. The activity level for the front and rear steering decides on the drive mode, which will be adopted by the robot. The velocity and turn prescripts are calculated at behaviour level as follows: The stereo vision behaviour receives as data dl and dr, the distances to obstacles on the left and right side of the robot. The smaller the distance to obstacles, the more careful and slow the robot must move. The velocity VS is therefore directly proportional to the mean of the measured distances left and right and the turn angle TS is chosen to maximize the distance to obstacles. The image motion behaviour receives as data ml and mr, the movement measured by the left and the right camera. The more movement in the image, the more probable there are objects close to the robot, so the velocity should be lowered. The robot speed VM is as such inversely proportional to the image motion and the turn angle TM is chosen to minimize the image motion. The goal seeking behaviour receives as data the estimate of the robot position (xR , yR) and orientation qR as calculated by the odometry and robot kinematics. This position and orientation is compared to the desired goal position (xG , yG) and orientation qG and a velocity VP and turn angle TP prescript are calculated from this information. Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

A major issue in the design of behaviour-based control systems is the formulation of effective mechanisms for coordination of the behaviours' activities into strategies for rational and coherent behaviour. Therefore, the activity levels need to be calculated. As noted before, these activity levels should reflect the relevance of the specific behaviour. The principle behind the calculation of the activity levels is that the output of behaviour should be stable over time in order to trust it. Therefore, the degree of relevance or activity is calculated by observing the history of the output - a velocity and turn angle - of each behaviour. This history-analysis is performed by comparing the current output to a running average of previous outputs, which leads to a standard deviation, which is then normalized. For the stereo vision behaviour, these standard deviations are:

N째 4

2008

they can be developed on Linux-side and then transferred later to RT-Linux. As errors in RT-Linux lead to systemhangs this development strategy prevents from many reboot cycles and results in faster software development. Each MCA module has a structure as shown in Figure 4. and is determined by four connectors with the outside world: Sensor input (left below), Sensor output (left top), Control input (right top), Control output (right below). As a result sensor data streams up, control commands stream down. The Sensor input and output are connected through a Sense procedure which enables to process the sensor data and the Control input and output are connected through a Control procedure which enables to process the control commands. This modular structure is particularly convenient for behaviour-based architectures as the individual behaviours translate easily to corresponding MCA-modules. 4.2.

(4)

The proposed behaviour-based control architecture Figure 4. shows the MCA scheme for the behaviourbased controller subgroup. It consists of three main behaviours, controlled (fused) by a Behaviour Selector module.

With cV and cT two normalisation constants. The bigger this standard deviation, the more unstable the output values of the behaviour are, so the less they can be trusted. The same approach is followed for the image motion (subscript M) and the goal seeking (subscript P) behaviours. This leads to an estimate for the activity levels: (5) For stability reasons, the activity level is initialized at a certain value (in general 0.5) and this estimate is then iteratively improved.

Fig. 4. MCA scheme for the behaviour-based controller subgroup.

4. A modular software architecture 4.1. An introduction to MCA As control architectures which aim at mimic human thinking risk of becoming highly complex, the choice of a flexible, extendable and real-time capable software architecture is very important. This software architecture has to ease the use of reusable and transferable software components. The chosen software architecture, MCA (Modular Controller Architecture) as presented by Scholl in [18], achieves this by employing simple modules with standardized interfaces. They are connected via data transporting edges which is how the communication between the single parts of the entire controller architecture is managed. The main programs only consist of constructing modules that are connected via edges and pooled into a group. This results in an equal programming on all system levels. As modules can be integrated both on Windows, Linux and on RT-Linux without changes, 22

Articles

The sensory input received by the behaviour-based controller subgroup consists of the distances to obstacles: dl, dc, dr; the motion in the left and right camera ml , mr and the robot position (xR , yR) and orientation qR. This data is processed by the stereo vision, image motion and goal seeking behaviours, outputting a velocity and turning command. The BehaviourSelector module receives as input the output of the different behaviours and calculates the activity levels for each of these behaviours according to equations (5). All of the three main behaviours also send the velocity command which was calculated to the Velocity module, where a fusion of the data occurs, using the activity levels, calculated by the BehaviourSelector module. The EmergencyStop module, which can be triggered by the user or when an obstacle is detected within a security distance, ensures that the velocity command is only transferred to the robot in safe conditions. For the turning behaviour, a similar approach is

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

followed; only in this case there are two separate fusion modules where the navigation behaviours can send their results to. This is related to the mechanical structure of the Robudem robot, which has a two-by-two differential drive system, meaning front and back wheels can be given a different turning angle, allowing for highly flexible manoeuvring on difficult terrain. It is our aim to let the behavioural controller (BehaviourSelector) decide, which is the best drive mode given the terrain circumstances, by setting the activity levels for the FrontSteering and RearSteering modules. For now, the user can select on the graphical user interface the desired drive mode and the activity levels AF and AR are set accordingly.

N° 4

2008

5. Conclusions In this paper we presented three main aspects of the design process of an intelligent autonomous robotic agent for risky outdoor interventions: visual sensing, behaviour-based control and the software architecture. Multiple visual cues, stereo vision and image motion analysis, were integrated into the robot control and software architecture. Behaviour based control architecture was proposed, using statistical reasoning to solve the action selection problem. All components were implemented using modular software architecture to achieve a futureproof design. The integration of these aspects enables the robot to reach a designated goal while avoiding obstacles.

5. Results The achievements of this research project can be discussed by taking a look at the Graphical User Interface (GUI) in Figure 5., as it shows the important features of the Robudem controller. On the upper left, the stereo camera images are visible, which are rectified to facilitate the calculation of the depth map. The dense depth map is then post processed, as is shown on the images to the right of the original depth map. In the middle of the interface, the measurements of the abstract visual sensors stereo vision and image motion analysis are shown using coloured bars. These indicate for the stereo vision sensor the distances to obstacles on the left, middle and centre and for the image motion sensor the motion in the left and right camera image. At the right of the interface, the activity levels of the different behaviours are shown. As can be noticed, the BehaviourSelector has found here that the ImageMotion behaviour delivers more reliable results than the StereoVision behaviour. This is in this case due to the lack of texture in the camera images, which renders the dense depth map estimation less robust. As discussed before, the activity levels for the VelocitySteering, FrontSteering and RearSteering are user-decided by setting the drive mode.

AUTHORS Daniela Doroftei*, Eric Colon and Geert De Cubber Royal Military Academy, Department of Mechanical Engineering (MSTA), Av. de la Renaissance 30, 1000 Brussels. E-mails: {daniela.doroftei, eric.colon, geert.de.cubber} @rma.ac.be. * Corresponding author

References [1]

[2]

[3]

[4]

S. Birchfield, Klt: An implementation of the kanadelucas-tomasi feature tracker, January 1997. Available at: http://www.ces.clemson.edu/stb/klt/. H. Bruyninckx, “Open robot control software: the OROCOS project”, Proc. Int. Conf. on Robotics and Automation, vol. 3, 2001, pp. 2523-2528. E. Colon, H. Sahli, Y. Baudoin, “CoRoBa, a multi mobile robot control and simu-lation framework”, Int. J. of Adv. Robotic Systems, vol. 3, no. 1, 2006, pp. 73-78. A.J. Davison, I.D. Reid, “MonoSLAM: Real-Time Single Camera SLAM”, IEEE Trans. on pattern analysis and machine intelligence, vol. 29, no. 6, June 2007.

Fig. 5. Graphical User Interface of the Robudem navigation controller.

Articles

Journal of Automation, Mobile Robotics & Intelligent Systems

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

G. De Cubber, S. Berrabah, H. Sahli, “Color-Based Visual Servoing Under Varying Illumination Conditions”, Robotics and Aut. Systems, vol. 47, no. 4, 2004, pp. 225-249. G.N. DeSouza and A.C. Kak, “Vision for Mobile Robot Navigation: A Survey”, IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 2, Feb. 2002 pp. 237-267. D. Doroftei, Behavior based Robot Navigation Techniques - State of the Art. Tech. report & Lecture Notes of The Royal Military Academy, Belgium, October 2006. V. Enescu, G. De Cubber, K. Cauwerts, S. Berrabah, H. Sahli, M. Nuttin, “Active Stereo Vision-based Mobile Robot Navigation for Person Tracking”, Int. Conf. on Informatics in Control, Automation and Robotics, vol. 2, 2005, pp. 32-39. B. Gerkey, R.T. Vaughan, and A. Howard, “The Player/ Stage Project: Tools for Multi-Robot and Distributed Sensor Systems”. In: Int. Conf. on Advanced Robotics, Portugal, 2003. P. Hong, H. Sahli, E. Colon, Y. Baudoin, “Visual Servoing for Robot Navigation”. In: 3rd Int. Conf. on Climbing and Walking Robots, Germany, 2001, pp. 255 - 264. A.D. Medeiros, “A survey of control architectures for autonomous mobile robots”, J. Braz. Comp. Soc., vol. 4, no. 3, 1998. C. Park, S. Kim, J. Paik, “Stereo vision-based autonomous mobile robot”. In: Intelligent Robots and Computer Vision XXIII: Algorithms, Techniques, and Active Vision. Proceedings of the SPIE, vol. 6006, October 2005, pp. 256-264. M. Piccardi, “Background subtraction techniques: a review”. In: Proc. of the IEEE Int. Conf. on Systems, Man and Cybernetics, vol. 4, October 2004, pp. 3099-3104. P. Pirjanian, Behavior coordination mechanisms - stateof-the-art, Tech. Report IRIS-99-375, Institute of Robotics and Intelligent Systems, Univ. of Southern California, 1999. P. Pirjanian, T. Huntsberger, A. Trebi-Ollennu, H. Aghazarian, H. Das, S.S. Joshi, P.S. Schenker, “CAMPOUT: a control architecture for multirobot planetary outposts”. In: Proc. SPIE Conf. Sensor Fusion and Decentralized Control in Robotic Systems III, USA, Nov. 2000. H. Schafer, M. Proetzsch, K. Berns, “Stereo-VisionBased Obstacle Avoidance in Rough Outdoor Terrain”, International Symposium on Motor Control and Robotics, 2005. C. Schlegel, J. Illmann, H. Jaberg, M. Schuster, and R. Worz, “Integrating vision-based behaviors with an autonomous robot. Videre”, MIT-Press, ISSN 1089-2788, vol. 1, no. 6, Winter 2000. K.U. Scholl, J. Albiez, B. Gassmann. MCA - An Expandable Modular Controller Architecture. Forschungszentrum Informatik an der Universitaet Karlsruhe (FZI).

Articles

VOLUME 2,

N° 4

2008

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N째 4

2008

DESIGN AND MOTION SYNTHESIS OF MODULAR WALKING ROBOT MERO Ion N. Ion, Alexandru Marin, Adrian Curaj, Luige Vladareanu

Abstract: The walking robots are built to displace the loads on the not-aligned terrain. The mechatronic walking system protects much better the environment when its contact with the soil is discrete, a fact that limits the area that is crushed appreciatively. At the Polytechnic University of Bucharest a walking modular robot to handle farming tools has been developed. This walking robot has three twolegged modules. Every leg has three freedom degrees and a tactile sensor to measure the contact, which consists of the lower and upper levels. The body of the MERO (MEchanism RObot) walking robot carries a gyroscopic bearing sensor to measure the pitch and roll angles of the platform. The legs are powered by hydraulic drives and are equipped with potentiometric sensors. They are used to control the walking robot in the adaptability to a natural ground. A vehicle like that is Romanian Walking Robot MERO (Fig.1). Keywords: walking robot, modular walking robot, robot control, and shift mechanism.

2. Building of experimental MERO modular walking robot The creation of a autonomous walking robot, equipped with capabilities such as objects handling, shift, perception, navigation, learning, reasoning, data storing and intelligent checking, enabling them to carry out missions such as changing the parts multitude of a dynamic world, is a target focusing the activities by many scientific research teams of several countries worldwide. The MERO modular walking robot, made by the authors, is a multi-functional mechatronic system designed to carry out planned movements aimed at accomplishing several-scheduled target. The walking robot operates and completes tasks by permanently interacting with the environment where known or unknown physical objects and obstacles are. Its environmental interactions may be technological (by mechanical effort or contact) or contextual ones (route identification, obstacle avoidance, etc). The successful fulfilment of the mission depends both on the knowledge, the robot, through its control system has on the initial configuration of the working place, and by those obtained during its movement as well as.

1. Introduction Modular Mobil Walking Robot represents a special category of robots, characterised by having the power source embarked on the platform. This weight of this source is an important part of the total charge that the walking machine can be transported. That is the reason why the walking system must be designed so that the mechanical work necessary for displacement, or the highest power necessary for acting it, should be minimal. The major power consumption of a walking machine is divided into three different categories: - the energy consumed for generating forces required sustaining the body in gravitational field; in other word, this is the energy consumed to compensate the potential energy variation - the energy consumed by leg mechanism actuators for the walking robot displacement in acceleration and deceleration phases - the energy lost by friction forces and moments in kinematic pairs. The modular walking robot weight can be distributed optimally on the support areas by controlling the forces and the variation of the distance from the soil level, allowing robots to walk over young trees or other plants, growing along the passage area [4],[3]. The easiness in obstacle avoidance - tree stumps, logs, felled trees is another advantage of walking robot use.

Fig.1. General view MERO modular walking robot. The MERO modular walking robot is made up of the following parts: a) the mechanical system made up of one, two or three modules articulated and shaped according to the requirements of the movement on an uneven ground, the robot's shift system is thus built that it may accomplish many toes' trajectories, which can alter by each step. Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

b) the actuating system of feet has a hydraulic drive; c) the distribution system is controlled by 12 or 18 servo-valves, according to the robot's configuration; d) the energy feeding system; e) the system of data acquiring on the shift the system's configuration and the environment; f) the control panel processing signals received from the driving and the acquiring systems.

3. On the shift system mechanisms of the MERO modular walking robot For the walking robot to get high shift performances on an as different terrain configurations as possible, and for increasing the robot's mobility and stability, under such circumstances, it is required a very careful survey on the trajectory's control, which involves both to determine the coordinates of the feet's leaning points, as related to the robot's body, and the calculation of the platform's location during the walking, as against a set system of coordinates in the field. These performances are closely connected with the shift system's structure and the dimensions of the compound elements. For simplifying its moulding, it is accepted the existence of a point-shaped contact between the foot and the leaning area. Studying the real situations of the shift by walking, analysing different types of walking as well as the issues related to the quasi-static and dynamic stability of the walking, made us to design and manufacture a modular walking robot. The constructive solutions we adopted for the platforms and the shift system mechanisms, allow the robot to enjoy a maximal adaptability to the most different moving ways. It is obvious that, from this point of view, the shift system mechanism is the most important element of the mechanical system. The shift system mechanism of any walking robot is built so that we could achieve a multitude of the toes' trajectories. These courses may change according to the ground, at every step. Choosing a certain trajectory depends on the topography of the surface, which the robot is moving on. As one could already notice, during the time, the shift mechanism is the most important part of the walking robot and it has one or several degrees of freedom, contingent of the kinematic chain that its structure relies on. Considering the fact that the energy source is fixed on the robot's platform, the dimensions of the legs mechanism's elements are calculated using a multicritical optimisation proceeding, which includes several restrictions.

The objective function may express: the mechanical work needed for shifting the platform by one step, the maximum driving force needed for the leg mechanism, the maximum power required for shifting.

These objective functions can be considered separately or simultaneously. The minimalisation of the mechanical work consumed for defeating of the friction forces can be considered in the legs mechanisms' synthesis also by a multicritical optimisation. 26

Articles

N째 4

2008

The kinematic dimensions of the shift system mechanism elements are obtained as a result of several considerations and calculation taking into account the degree of freedom, the energy consumption, the efficiency, the potential distribution, the kinematic performances, the operation field and the movement regulating algorithm. The walking robot's leg (Fig.2) is made up of a 3R spatial mechanism, operated by three linear hydraulic motors. The working area's dimensions were induced from the condition that the robot is able to step over the obstacles whose dimensions were initially established. The leg mechanism together with the driving system, the location sensors, the contact and force sensors, all make up one module of the shift system called the leg of the robot.

Fig. 2. The leg mechanism of the MERO walking robot. Z 16

Z 12

Z 15

Z 11 Z 6

Z 14

1 2

Z2 _ 3 N1

Z 31

P2 z

_ T1 h

_ N6

Z _ 3 N5

_ T2 _ N3

_ N4

_ T3

_ T6

_ T5 _ T4

Fig. 3. The kinematic scheme of the MERO modular walking robot's mechanical system. The movement system of the walking robot, as designed in the Figure 2, has three degree of freedom and a quite simple mechanical structure, being made up of three serially connected bars and three linear hydraulic motors. Two analog-digital and one digital-analogue interfaces (Fig. 5) and a processing computer derived from a PC acquire the measured data. The values found as a result of the measurements, enabled us to draw the diagrams emphasizing the variation of the kinematic and

Journal of Automation, Mobile Robotics & Intelligent Systems

kinetostatic parameters of the MERO walking robot's movement system. When drafting the MERO modular walking robot, it was succeeded a drop in the power consumption by both methods, namely by reducing the power consumption by an optimal designing of the legs' mechanisms, the cut in the power used by the robot by balancing the weight forces of the platform and the legs' elements, with the help of the system for static balancing). Figure 3 shows one of the robot's legs, equipped with the balance springs. LED OX axis roller

Phototransistor

LED Main output signal Sphere OY axis roller Phototransistor LED

output signal

N째 4

2008

phase, the leg of the robot is above the walking surface and is moving so that it realizes the stability state on the whole of the walking robot. The walk of the robot is characterized by the order raise and seat of the legs and by the trajectory form of the theoretical support point in comparison with the platform. To establish the walking order it is needed to number the legs. The state of the leg (i) at a given time [1], [4] is described by a state's i function q (t), that has only two values, 0 and 1, as it follows:

On the interval [0, t1], the leg is in the support phase. On the interval [t1, t2], the leg is not leaning upon the support surface and it is in the transfer phase. On the interval [t2, t3], the leg is on the support surface again, etc. At a moment of time, the state of the walking robot with N legs is defined by a N-dimensional vector q, named the vector of the legs states. The vector's components qi, i =1,N , are formed by the functions of the legs' states, ordered by their

Pendulum

Main output signal 90 deg. dephased

VOLUME 2,

(1) Phototransistor

Fig. 4. Stance transducer.

Fig. 5. Block scheme of acquiring data system. In order to determine the walking robot's walking stance while moving (maintenance of horizontal position), we attached the platform a sensor made up of two incremental transducers aimed at perceiving the walking stance (Fig. 4). It determines the platform's leaning both at the sagittal and frontal level. The sensor is made up of a body hanged by a rigid rod to a sphere leaning on two rollers.

4. Mathematical modelling of gait for modular mobile walking robot A cycle of the movement of the leg of a walking robot has two phases: the support phase and the transfer phase. In the first phase, the leg's support part has a direct contact with the walking surface area. In the transfer

so that the first component of the vector defines the state of the leg 1, the second one is the state of the leg 2, etc. It is assumed that in any finite interval of time there is a finished number of moments that define the values of the functions qi(t). The q states, which appear at every change of the value of the function qi(t), are numbered chronologically as they are carried on. As a result, the walk of a walking robot is described by a succession of states (qj), j = 1, 2, .... Therefore, it has to be lifted away from the support phase, moved towards the advance direction of the mechatronic mobile system and then lay down, following another cycle with another leg. As the Modular Mobil Walking Robot has two or more legs, these moves have to be coordinated, so that the move is ensured in conditions of the stability of the system. In order to allow a theoretical approach of the gait of the mechatronic mobile system it is necessary to define a lot of terms. Let us consider a scheme of a hexapod mechatronic mobile system, numbering each leg. In order to achieve and manage a hexapod. Modular Mobil Walking Robot - it is necessary to know all the walking possibilities, because the selection of the legs number and its structure depends on the selected type of the gait. The selection of the type of gait is a very complicated matter, especially in the real conditions of walking on the rough terrain. The longitudinal stability margin, Sl is the shorter of the distances from the vertical projection of the centre of gravity to the front and rear boundaries of the support pattern, as measured along the direction of motion (see Figure 6). If certain obstacles occur on the walking surface, a special crossing gait must be used, after learning the dimensions of such obstacles. Articles

Journal of Automation, Mobile Robotics & Intelligent Systems

Depending on the type of the obstacle, its surpassing can be made by the precise arrangement of the legs in the permitted areas around the obstacle. In such a case, the periodic gait, named “follow the leader” is highly recommended In case of walking on an unarranged terrain, due to the great diversity of the obstacle dimensions and forms, precise walk is not recommended.

VOLUME 2,

N° 4

2008

The terrain used in the study is two-dimensional, unarranged terrain. The terrain is divided into many cells and each cell is about the size of a footprint. The cells are classified into two types: a permitted cell and a forbidden cell. A forbidden cell is not suitable for a foot to tramp on it due to weakness of the soil structure, a ditch, or other reasons. A collection of many forbidden cells is a forbidden area.

5. Hybrid control of MERO modular walking robot stability

Fig. 6. The longitudinal stability margin.

Fig. 7. The model a modular walking robot. Figure 7 shows the model of a modular hexapod walking robot. The body coordinate system x-y is attached to the body centre and the x-axis is aligned with the body longitudinal axis. The centre of gravity is coincident with the body centre. Each leg is assigned a number as is shown in the Fig. 7. Each leg is represented by a line segment which connects the foot point and hip point is considered unlimited (i.e. the workspace of each leg is unlimited).

Fig. 8. Computer graphics simulation the gaits of the modular walking robot. 28

Articles

Walking robots may have a lower or higher autonomy degree. This autonomy has in view the power source's supply capability but also orientation and perception capabilities as regards the terrain configuration, the robot is running upon and its decision making and the motion manner towards a target. To plan the movements is necessary only for the walking robots at a low autonomy degree and which move according to a previously scheduled program. Walking robots having a high autonomy as concerned the decision making, should benefit from appropriate driving programs and obviously from high-speed computers. To control the walking robot shift in structured or less structured environments we need the following specifical functions: - the environmental perception and shaping using a multisensorial system for acquiring data, - data collecting and defining the field configuration, - movement planning, - analysis of the scenes. Control of the robot stability entails that the vertical projection of the G system gravity centre on the supporting surface must be inside the supporting polygon. The geometric centre position O is defined as the diagonals intersection point of the polygon formed by the points, where the legs are connected to the platform, and G(GxGyG,zG) as the gravity centre position of robot. Maintaining the vertical of the centre of gravity in the support surface is increasingly difficult if the robot is moving on a slope. In this case maintaining the stability depends on the transported load (fi) and on the distance XC from the surface of the support points to the centre of gravity. Stability is obtained by reducing the XC component when the fi load increases taking into account the moving slope of the robot. A control method consists in evaluating the robot load and assigning a constant step value in order to obtain robot stability. As a result, there is obtained a high number of steps for a complete stability, depending on the robot's speed and the obstacles met by the robot, which can lead to the robot being overturned. A new control method, which practically eliminates instabilities and has a fast response of the control loop, is presented in Figure 9. It consists of a “multi-stage” fuzzy control (MS), which entails the realization of two fuzzy control loops, one in position and the other in force on two different decision stages in order to determine the distance XC from the surface of the support points to the centre of gravity.

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

MS fuzzy control has multiple rule bases, where the result of an inference of the rule base is transmitted to the next stage [2]. In this way the most important dimensions of the inference can be grouped into smaller sets and combined with base rules. In the MS structure, the results of the rule base of the control of position P are transmitted to the rule base of the PF position - force control.

N° 4

2008

By analysing the rule base, it can be observed that the force feedback is a function of the inference results of the fuzzy control P component. The rule base P is easily modified from a typical linear rule base, allowing for the replacement of all Zero (ZO) values, except the centre of the rule base. For a certain set of inputs, i.e. the measured XC, the evaluation of fuzzy rules produces a fuzzy membership

Fig. 9. Fuzzy “multi-stage” control (FMS) for MERO modular walking. In terms of control, based on the characteristics of the positioning functions, P is of high level, generally controls the system to avoid any instability of the robot and ensures system control should dynamic disturbances appear or if commands are generated and the main function is force. The control returns the functions base P when the system dynamic is settled. The task of the controller is to assign the measured XCP of the fuzzy variables, i.e. “positive great” (PM), and to evaluate the decision rules through inference. So that, it can finally establish the value of the output variable, i.e. the velocity as fuzzy variable, which best follows the controlled parameter. The configuration of the decision rules and of the fuzzy variables used in making the decision depends on the problem of specific control. The values of deflection detected through sensors are quantified in a number of points corresponding to the discourse universe elements, and then the values are assigned as grades of membership in a few fuzzy subsets. The relationship between the inputs, i.e. the measured deflections, and outputs, i.e. velocity, and the grades of membership can be defined in conformity to the operator's experiences and the demands of the robot charge. There are defined empirically the membership functions for all input and output elements. The fuzzy variables have been chosen as follows: NM - negative great, NM - negative medium, Nm - negative small, ZO zero, Pm - positive small, PM - positive medium, PM positive great.

set for system control. In order to take a concrete action, one of these values must be chosen. In this application, the control value with the highest degree of membership has been selected. The rules are evaluated at equal intervals, in the same way as a conventional control system. The result of logical inference also represents fuzzy values which are applied to the defuzzify mode. The defuzzify represents a transformation of the fuzzy variables defined on the output discourse universe in a numerical value. This processing is necessary because the control in the case of fuzzy regulators is done exclusively with crypted values. Choosing defuzzify method the weight centre of the area the defuzzify output is determined. Choosing a discreet discourse universe allows for using the multiprocessor systems for generation the fuzzy output variables with a reduced processing time. By applying the fuzzy logic control, there is obtained a linear passing, without discontinuities, in controlling the distance XC from the surface of the support points to the centre of gravity. Moreover, a fast response of the control loop is obtained while maintaining the stability of the robot.

6. Conclusions The MERO modular walking robot was developed at University ”Politehnica“of Bucharest in the MEROTEHNICA laboratory. Such module robot has two/four/six legs with three degrees of freedom each. The body of modular walking robot carries a gyroscopic attitude sensor to measure the pitch and roll angles of the body. Articles

Journal of Automation, Mobile Robotics & Intelligent Systems

The legs are powered by hydraulic drives and are equipped with joint angle potentiometer transducers. Each leg has three degree of freedom and a tactile sensor to measure the contact, which consists of lower and upper levels. The MERO type transducers used in walking robots offer both force control and robot protection. Each of the feet is equipped with stain gauged force sensing device optimised by finite element analysis. Each of the rotational pairs is closed-loop controlled by software servocontrolled by an external computer.

AUTHORS Luige Vladareanu - Institute of Solid Mechanics of Romanian Academy, Department of Dynamic Systems, C-tin Mille. 15, Bucharest 1, Romania. Ion N. Ion*, Adrian Curaj, Alexandru Marin - Politehnica University of Bucharest, Splaiul Independentei 313, Bucharest, Postal Code 060042, Romania. E-mail: ioni51@yahoo.com. * Corresponding author

References [1]

[2]

[3] [4]

[5]

[6]

[7]

[8]

[9]

Bessonov A.P., Umnov N.V., “The Analysis of Gaits in six-legged Robots according to their Static Stability”. In: Proc. Symp. Theory and Practice of Robots and Manipulators, Udine, Italy, 1973. Duan J.C., Chung F.L., “A Mamdani Type Multistage Fuzzy Neural Network Model”. In: IEEE Fuzzy Systems Conference 1998 - FUZZ'98, pp. 1253-1258. Hirose S., Three Basic Types of Locomotion in Mobile Robots, IEEE 1991, pp. 12-17. Ion I., Simionescu I., Curaj A., “Mobile Mechatronic System With Applications in Agriculture and Sylviculture”. In: The 2th - IFAC International Conference on Mechatronic Systems, 8th-12th December 2002, Berke-ley USA. Ion I., Stefanescu D.M., “Force Distribution in the MERO Four-Legged Walking Robot”. In: ISMCR'99 - Topical Workshop on Virtual Reality and Advanced Human-Robot Systems, vol. X, 10th-11th June 1999, Tokyo, Japan. Ion I., Simionescu I., Avramescu A., “Stability Analysis of Spider Gaits of Four legges Walking Machines”. In: 4th Int Workshop Robotics in the Alpes-Adria Region RAA'95, 6th-8th July 1995, Portschach, Austria. Ion I., Ungureanu M., Simionescu I., “Control System of MERO modular Walking Robot”. In: Proceedings of the International Conference on Manufacturing Systems, Bucharest, 10th-11th October 2002, ICMaS 2002, Editura Academiei, ISBN 973-27-0932-4, pp. 351-354. Simionescu I., Ion I., Ciupitu L., “Optimisation of Walking Robots Movement Systems”. In: Proceedings of the VII International Congress on the Theory of Machines and Mechanisms, 3rd-5th September 1996, Liberec, Czech Republic. Waldron K.J., “Force and Motion Management in Legged Locomotion”. In: Proceedings of 24th IEEE Conference on Decision and Control, Fort Lauderdale, 1985, pp. 12-17.

Articles

VOLUME 2,

N° 4

2008

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N째 4

2008

SAFE REACTION OF A ROBOT ARM WITH TORQUE SENSING ABILITY ON THE EXTERNAL DISTURBANCE AND IMPACT: IMPLEMENTATION OF A NEW VARIABLE IMPEDANCE CONTROL Dzmitry Tsetserukou, Hiroyuki Kajimoto, Naoki Kawakami, Susumu Tachi

Abstract: The paper focuses on control of a new anthropomorphic robot arm enabling the torque measurement in each joint to ensure safety while performing tasks of physical interaction with human and environment. A novel variable control strategy was elaborated to increase the robot functionality and to achieve human-like dynamics of interaction. The algorithm of impact control imparting reflex action ability to the robot arm was proposed. The experimental results showed successful recognition and realization of three different types of interaction: service task, co-operative task, and impact state. Keywords: human-robot interaction, anthropomorphic robot arm, variable impedance controller.

1. Introduction Many operations in specific environments (e.g. nuclear, in-space, underwater) are dangerous for human or difficult to access (e.g. micromanipulation, surgery). Recently, teleoperation is gaining increased attention by researchers interested in studying the possibilities to overcome the limited functionality of autonomous robots performing specific tasks, and to improve dexterous operations. We successfully developed master-slave robot systems (TELESAR I and TELESAR II) based on the telexistence concept and gradually improved performance and stability of teleoperation [1]. The operator cockpit has two 6-DOF exoskeleton-type structure arms with hands. The slave robot enables to perform complex interaction with objects in environment by means of 7-DOF arms and 8-DOF hands. The contact tasks of a tip of the end-effector with environment are provided by impedance control algorithm. A slave robot controlled by the operator mainly performs the manipulations in unknown and unstructured human environment. The operator has to move the slave robot arm to target position, and to perform manipulations while simultaneously avoiding obstacles. However, operators cannot accomplish these tasks in real time. Besides, they are not perfect in planning the collision-free motion in dynamic unstructured surroundings. Recent researches on teleoperation are focusing on automation of collision avoidance and contact task based on information from sensors, allowing thus operator to concentrate on the task to be performed. The Robonaut teleoperated robot was developed to assist in-space operations and work with humans in space exploration [2]. The developed manipulator hand has very good force sensing ability due to the 6-axis force sensors attached at

the fingertips and 19 Force Sensing Resistors. Using this force/tactile sensing ability, the operator moves the robot arm until the palm contacts an object, and then, grasping algorithm is activated. However, collision or expected contact with an environment may occur on the entire surface of the robot arm (e.g. forearm, elbow, upper arm, shoulder) producing interactive forces. Moreover, teleoperated robot in rescue and some other applications has to interact with human being in various cooperative tasks. In these fields, human-robot interaction represents a crucial factor for a robot design and imposes strict requirements on its behaviour and control in order to ensure safe interaction with environment and effectiveness while target task execution. Several effective methods on enhancement of contact detection ability of manipulator were reported. To avoid collisions in time-varying environment, Lumelsky et al. [3] proposed to cover manipulator with a sensitive skin capable of detecting nearby objects. As this device integrates a huge amount of small sensors incorporated on a tiny rigid platform, and requires complicated wiring and signal processing hardware, such devices have high cost and reliability issues. Besides, sensitive skin provides only contact pattern information. Finding the technical solution for trade-off between safety and performance is the target of the new manipulation technology. To cope with this issue, the active compliance control implying fast joint torque controlling based on measuring the applied external torque in each joint was developed. The first embodiment of torque measurement is the integration of a torque sensor into each joint of the manipulator. The impedance control generates compliant trajectory based on measured external torque information. Such approach has two main advantages: (1) sensor detects not only forces applied to the hand but also those exerted at other points on the manipulator, (2) and allows to increase the performance for fast movements by active vibration dam-ping. Several attempts have been made by researchers to improve joint torque control. Wu and Paul [4] proposed simple, wide bandwidth, torque servo system using strain-gauge-based joint torque sensor. The developed torque-controlled lightweight robot arm with high load to weight ratio is described in [5]. Each joint of the arm is facilitated with strain-gauge-based torque sensor, posi-tion sensor and piezzo-brake. Sakaki and Iwakane [6] proposed to use a compact motor drive with embedded magnetostrictive torque sensor for impedance control of the robot arm. The approach of torque measurement through the elasticity of the harmonic drive flexsplines allows keeping the same stiffness and mechanical structure of the robot [7]. This method requires the strain Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

gauges to be installed on the flexsplines. The crucial shortcomings of the torque measurement approaches mentioned above are discussed in the paper [8]. The alternative method for increasing the safety level of robot arms interacting with humans is intentionally introducing compliance at the mechanical design level. The main idea is decoupling the rotor inertia from the link inertia through using passive elasticity [9]. However, the robot control is complicated by many unknown parameters (e.g. actuator stiffness and damping). Furthermore, compliant transmission negatively affects the performance in term of increased oscillations and settling time. To realize the safe physical contact of entire robot arm structure with human and to guarantee the collision avoidance, our primary idea is concentrated on the design of a whole-sensitive robot arm (by using distributed optical torque sensors in each joint). When contact with environment occurs, manipulator automatically adjusts its dynamic parameters (stiffness and damping) according to the measured external torque and time derivative of torque. The newly developed anthropomorphic manipulator iSoRA (intelligent Soft Robot Arm) having 4-DOF arm and 8-DOF hand (Fig. 1) is capable to safely interact with environment wherever contact occurs on the arm surface.

2. Design of a new anthropomorphic robot arm From the safety point of view, to minimize injures in case of collision, most of the robot parts were manufactured from aluminium alloys to obtain as much lightweight structure as possible. The robot links were designed in round shape to reduce impact force. The distribution of the arm joints replicates the human arm structure in order to make it easy to operate using kinaesthetic sensation during teleoperation. To remove mechanical subsystems without disassembling the main structure when the failures do occur, we have been using a modular approach while designing anthropomorphic robot arm. Therefore, we selected CSF-series gear head type of Harmonic Drive instead of compact and lightweight component one. The Harmonic Drive offers such advantages as accurate positioning, high torque capability, torsional stiffness, and high single stage ratios. Developed robot arm has 4-DOF: Roll, Pitch, Yaw joints of a shoulder, and Pitch joint of an elbow. Each joint is equipped with optical torque sensor directly connected to the output shaft of Harmonic Drive. The sizes and appearance of the arm were chosen so that the sense of incongruity during interaction with human is avoided. We kept the arm proportions the same as in average height human: upper arm length L1 of 0.308 m; upper arm circumference of 0.251 m (diameter 0.080 m); forearm length L2 of 0.241 m; forearm circumference of 0.189 m (diameter 0.06 m). The 3D CAD model of the developed arm and coordinate systems based on Denavit-Harten-berg convention are represented in Fig. 2.

Torque sensor distribution in Shoulder Pitch and Roll Joints

Sensors in Yaw and Elbow Joints

Yaw

Z1 Y2

Articles

L1 J3

Y4 Z4 X4

X3 Z3 L2

Y5 Z5

Fig. 2. 3D CAD model and coordinate systems.

Y0 + X0 + Pitch Roll

The remainder of the paper is structured as follows.

2008

Section 2 describes the development of a whole-sensitive robot arm. The elaborated intelligent variable impedance control algorithm allowing contact state recognition and the experimental results are discussed in Section 3. In Section 4, we briefly conclude the paper.

Fig. 1. iSoRA robot arm. The distinctive features of our approach are as follows: 1) We use optical approach for torque measurement, which allows avoiding inherent shortcomings of strain-gauge-based and magnetostrictive force/torque sensors. We developed new optical torque sensors having high dependability, good accuracy (even in electrically noisy environment), low price, compact sizes, and easy manufacturing and calibration procedure [8]. 2) The stiffness of the proposed sensor is much smaller then previously proposed. So, when collision takes place, the injuries of human can be considerably reduced through compliant coupling between the motor rotor inertia and the link inertia. 3) The intelligent variable impedance control and reflexaction-based impact control were elaborated to realize safe, smooth and natural human-robot interaction.

N째 4

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

The motors are equipped with magnetic encoders having 512 pulses per revolution. To protect the sensor against influence of bending moment and axial force, the simple supported loaded shaft configuration was implemented using two sets of bearings. The principal specifications of the developed arm and kinematic parameter values are given in Table 1 and Table 2, respectively.

N째 4

2008

ters. The graphical representation of joint impedance control is given in Fig. 3.

Table 1. Principal specifications.

Fig. 3. Concept of the local impedance control. The desired impedance properties of i-th joint of manipulator can be expressed as: (2) where Jdi , Ddi , Kdi are the desired inertia, damping, and stiffness of i-th joint, respectively; is torque applied to i-th joint and caused by external and gravity forces, is the difference between the current position and desired one . The state-space presentation of the equation of local impedance control is written as follows:

Table 2. Denavit-Hartenberg parameters.

(3)

or:

3. The impedance control (4) 3.1. Joint impedance control The dynamic equation of an n-DOF manipulator in joint space coordinates (during interaction with environment) is given by:

where the state variable is defined as ; A, B matrices. After integration of (4), the discrete time presentation of the impedance equation is expressed as:

(1) where are the joint angle, the joint angular velocity, and the joint angle acceleration, respectively; is the symmetric positive definite inertia matrix; is the vector of Coriolis and centrifugal torques; is the vector of actuator joint friction forces; is the vector of gravitational torques; is the vector of actuator joint torques; is the vector of external disturbance joint torques. People can perform dexterous contact tasks in daily activities while regulating own dynamics according to time-varying environment. To achieve skilful human-like behaviour, the robot has to be able to change its dynamic characteristics depending on time-varying interaction forces. The most efficient method of controlling the interaction between a manipulator and an environment is impedance control. This approach enables to regulate response properties of the robot to external forces through modifying the mechanical impedance parame-

(5)

To achieve the fastest possible non-oscillatory response to the external force, we assigned the eigenvalues l1 and l2 of matrix A as real and equal . By using Cayley-Hamilton method for matrix exponential determination, we have:

(6)

(7)

where T is the sampling time; I is the identity matrix. There are several conflicting requirements on the Articles

Journal of Automation, Mobile Robotics & Intelligent Systems

choice of dynamics parameters of impedance model to provide effectiveness and functionality of robot in tasks of physical interactions fulfilled in co-operation with humans, and to ensure the collision avoidance. For example, while accomplishing service tasks for human in the autonomous mode, it is required to provide high stiffness (to ensure small position error during object handling) and high damping (for good velocity tracking). Realization of human following motion, mainly used in performing of cooperating tasks, also imposes specific requirements on desired impedance parameter selection. In this approach, by applying force to the robot arm, it is possible to intuitively operate the robot along the force and speed direction without considering any command signals. In the case of collision, the very small stiffness is obligatory to reduce the impact forces. Basically, there are two types of solution of this problem: adaptive and functional adjustments of impedance parameters. In the adaptive control, damping and stiffness of the system are gradually adjusted according to sensed dynamics and contact forces [10]. However, due to parametric uncertainties of the robot dynamics model, it is difficult to obtain the complete description of the dynamics. Therefore, model-based adaptive impedance control must rely on either repeated motions or time for adaptation to achieve convergence to the desired system parameters. For systems with more then one DOF, such approach hardly can be applied. In the functional approaches, current impedance parameters have predetermined relations to the current sensed variables. Generally, these methods presume determining the current stiffness and damping matrices functionally dependent on sensed variables. The main idea of the paper [11] is that the involving contact between slave robot and the environment can be classified according to the angle between commanded velocity and the contact force. It is supposed that force and velocity vectors are usually parallel when the impact occurs or when the object is being pushed. So, the functional dependency of stiffness and damping on angle between sensed force and velocity vectors was proposed. In real co-operation tasks and, especially, collisions, these vectors can not only be parallel, but also have indepenent arbitrary directions according to external force coming direction. Another line of variable impedance research is directed to estimation of human arm stiffness [12]. The proposed variable impedance controller varies a damping parameter of target impedance in proportion to an estimated value of the human arm stiffness. Despite the fact that robot can effectively follow the human arm motion, it cannot perform task autonomously. Besides, only impedance parameters of the robot end-effector can be adjusted. We elaborated a new methodology for impedance parameter adjustment based on the interaction mode and providing the dynamic stability of the system. 3.2.

Intelligent variable joint impedance control

3.2.1. Control of service task and human following motion The research on impedance characteristics of human arm shows that, while pushing or pulling the object naturally, human arm stiffness and damping behaviour can be 34

Articles

VOLUME 2,

N° 4

2008

approximated by exponential curves [13]. The first essential peculiarity of new control method is that we introduce the exponential functional dependency between sensed force and stiffness to impart the human-like damping and stiffness behaviour to the robot arm interacting with environment. The second main feature originates from the fundamental conflict in impedance selection with regard to current working conditions. We consider the threshold of external disturbance torque value to distinct the service task (with high stiffness and damping of joints) from human following motion tasks requiring low stiffness. This value can be chosen depending on the force necessary to accomplish the service task. We assigned the specific magnitude of to each joint of robot arm. The third distinctive contribution is the recognition of the collision based on the time derivative of joint torque. The procedure of impedance parameters selection is illustrated by the example of elbow joint as follows. On the first stage, the parameters of desirable impedance model of robot are computed for the case of service task accomplishment and average-level-contact-force of human-robot interaction. The desired stiffness Kd1 (for static equilibrium case) is calculated from Eq. (8) based on the maximum deflection value of joint angle caused by external torque while service task performance. (8)

It was defined that external torque of 1 Nm results in of 0.1 rad giving the Kd1 of 10 (Nm/rad). The desired damping is expressed as: (9) To realize fast non-oscillatory response on the external torque, we defined damping coefficient of 1.05. The value of desired inertia Jd1 of 0.1 (kg×m²) was assigned to realize fast response tracking. Thus, the value Dd1 of 2.1 (Nm×s/rad) was derived from (9). These parameters are valid till the interaction force does not cause the overload of robot arm. When sensed value of the torque is larger then threshold level, robot recognizes this condition as human following motion mode, and adjusts its dynamics parameters (stiffness and damping) in the same way as humans in order to provide smooth natural interaction. To realize such continuous change of dynamics, we are using exponential relation between external disturbance torque and desired stiffness: (10) where Kd2 is the desired stiffness on the second stage of interaction; m is the coefficient defining the level of decreasing of arm joint stiffness in response on increasing difference between external torque and threshold external torque value. The desired damping is adjusted to prevent force responses from being too sluggish while changing stiffness values, and to ensure contact stability: (11)

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

Then, variable joint impedance controller is described by:

(12)

To verify the theory and to evaluate the feasibility and performance of the proposed impedance controller, the experiment with new whole sensitive robot arm was conducted. To ensure the effectiveness of service task accomplishment, we decided to implement position-based impedance control (Fig. 4). In this algorithm, compliant trajectory generated by the impedance controller is tracked by the PD control loop.

NÂ° 4

2008

During the experiment, the interaction with arm was performed to exceed join torque threshold level ( of 0.6 Nm was assigned, = -1.155). The experimental results for the elbow joint â&#x20AC;&#x201C; applied torque, impedance trajectories with constant and variable coefficients, variable stiffness plot, variable damping plot, applied torque with different magnitudes and corresponding impedance trajectories with constant and variable coefficients â&#x20AC;&#x201C; are presented in Fig. 5, Fig. 6, Fig. 7, Fig. 8, Fig. 9, and Fig. 10, respectively. The experimental results show the successful realization of the variable joint impedance control. While contact with human, the robot arm generates compliant soft motion according to the sensed force. The plot presented in Fig. 6 shows that the variable impedance control

Fig. 4. Block diagram of position-based impedance control.

Fig. 5. External torque.

Fig. 7. Variable stiffness Kd.

Fig. 9. External torque.

Fig. 6. Impedance trajectories.

Fig. 8. Variable damping Dd.

Fig. 10. Impedance trajectories.

Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

provides softer trajectory to accomplish human following motion than the impedance control with constant coefficients. As we assigned the critically damped response of impedance model to disturbance force, output angle has ascending-descending exponential trajectory. As seen from the experimental results shown in Fig. 9 and Fig. 10, the proposed variable impedance control comprehensively distinguishes and processes the service tasks (torques under threshold level) and co-operative human following tasks (torques above threshold level). As the result of the experiments with variable impedance-controlled arm, tactile sensation of soft friendly interaction was really achieved. 3.2.2. Impact control In daily life, a person frequently contacts with an environment. The impact between a limb and environment is inevitable during interaction. In order to work in human daily environment, robot has to recognize the collision condition, and its control system should be able to quickly and smoothly guide manipulator to avoid excessively large impact force. Different impact control algorithms were proposed. Basically, they are focusing on preparing for impact in advance [14] and coping with impact during contact transience [15]. However, due to unexpected nature of collision and lack of the robust control strategy of impact effect minimization, the application area of such approaches is highly limited. We believe that realization of the robust impact control should originate from human reflex system. A reflex action is an automatic involuntary neuromuscular action elicited by a defined stimulus, and it can respond to external stimuli with small reaction time retracting the limbs away from the object. In our control algorithm, impact is processed as follows. When an unexpected colli-

N째 4

sion is detected, the impact control algorithm provides the pre-programmed reflex action of the robot arm. After accomplishment of a safe and smooth collision, the control system is returned to the original mode aimed at distinguishing the service and co-operating tasks. While analysing the results of collision experiments, we came to conclusion that the large contact forces, mainly used as criterion of collision, do not indicate the impact, while the time derivative of force value does. In the developed impact control system, the value of the time derivative of torque exceeding the assigned threshold is inter-preted as collision state. To realize reflexive human-like behaviour, the stiffness of the impedance model of the robot arm on the first stage of contact transient has to be reduced drastically. Hence, exponential relation between time derivative of external torque and desired stiffness coefficient Kd3 was defined with large coefficient of 0.7: (13) To attenuate the oscillations of the robot arm, which are inherent to collision, the larger constant damping coefficient of 1.25 was assigned. The value of desired damping coefficient is calculated from Eq. (9). In the second stage of contact transient, while time derivative of torque is negative, the high damping value reducing arm inertia effect is desired. For this case, the following relation between desired damping Dd3 and time derivative of torque was specified: (14) where 1.2).

is the weighting factor for damping (equals to

Fig. 11. Torque derivation.

Fig. 13. Variable damping Dd.

Fig. 15. Torque derivation.

Fig. 12. Variable stiffness.

Fig. 14. Impedance trajectories.

Fig. 16. Impedance trajectories.

Articles

2008

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

The threshold value of the time derivative of torque was assigned to 2.6 Nm/s. The experimental results for the elbow joint – time derivative of applied torque, desired collision stiffness Kd3, desired collision damping Dd3 plot, impedance trajectories with constant and collision variable coefficients, time derivative of torque for two different contact states and corresponding impedance trajectories – are presented in Fig. 11, Fig. 12, Fig. 13, Fig. 14, Fig. 15, and Fig. 16, respectively. From the plots of experimental results, it is apparent that, when impact is recognized (time derivative of torque becomes larger than threshold value), the stiffness and damping are decreasing drastically (Fig. 12, Fig. 13), allowing avoidance of large impact forces. On the second stage of interaction, when the impact force is decreasing, the damping coefficient of impedance model (Fig. 13) is increased to suppress the dynamic oscillations. The Fig. 15 indicates that nature of co-operation tasks (time derivative of torque has small and stretched in time scale value) completely differs from impact state (time derivative of torque has large spike and is shortened in time scale value). The developed control takes advantage of this feature to generate reflex-action-based motion.

AUTHORS Dzmitry Tsetserukou, Naoki Kawakami and Susumu Tachi - Department of Information Physics and Computing, The University of Tokyo, Japan. E-mails: dima_teterukov@ipc.i.u-tokyo.ac.jp, kawakami@star.t.u-tokyo.ac.jp, tachi@star.t.u-tokyo.ac.jp. Hiroyuki Kajimoto - Professor at Department of Human Communication, The Universityof Electro-Communications, Japan. E-mail: kajimoto@hc.uec.ac.jp.

2008

References [1]

[2]

[3]

[4]

[5]

[6]

4. Conclusion New whole-sensitive robot arm iSoRA was developed to provide human-like capabilities of contact task performing in a broad variety of environments. Each joint was facilitated with high-performance optical torque sensor. The intelligent variable impedance control and reflexaction-based impact control were elaborated to realize safe, smooth, and natural human-robot interaction. We introduced the exponential functional dependency between sensed force and stiffness to impart human-like damping and stiffness behavior to the robot arm. Experimental results of impact revealed that large contact forces, mainly used as criterion of collision, do not indicate the impact state, while the time derivative of force value does. The magnitude of time derivative of torque was used as an indicator of collision state. To realize reflexive human-like behavior, stiffness of the impedance model of robot arm was reduced drastically during collision transient. This was achieved by means of reciprocal functional dependency of stiffness on the value of time derivative of torque. The conventionally impedance-controlled robot can provide contacting task only at the tip of the end-effector with predetermined dynamics. By contrast, approaches developed by us provide delicate continuous safe interaction of all surface of the arm with environment.

N° 4

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

Tadakuma R., Asahara Y., Kajimoto H., Kawakami N., and Tachi S., “Development of anthropomorphic multiD.O.F. master-slave arm for mutual telexistence,” IEEE Trans. Visualization and Computer Graphics, vol. 11, Nov. 2005, no. 6, pp. 626-636. Diftler M.A., Culbert C. J., Ambrose R. O., Platt Jr. R., and Bluethmann W. J., “Evolution of the NASA/DARPA Robonaut control system”. In: Proc. 2003 IEEE Int. Conf. Robotics and Automation, Taipei, 2003, pp. 2543-2548. Lumelsky V. J. and Cheung E., “Real-time collision avoidance in teleoperated whole-sensitive robot arm manipulators,” IEEE Trans. Systems, Man, and Cybernetics, vol. 23, 1993, no. 1, pp. 194-203. C. H. Wu, Paul R. P., “Manipulator compliance based on joint torque control”. In: Proc. 19th IEEE Conf. Decision and Control, Alberquerque, NM, 1980, pp. 88-94. Hirzinger G., Albu-Schaffer A., Hahnle M., Schaefer I., and Sporer N., “On a new generation of torque controlled light-weight robots”. In: Proc. 2001 IEEE Int. Conf. Robotics and Automation, Seoul, Korea, 2001, pp. 3356-3363. Sakaki T., Iwakane T., “Impedance control of a manipulator using torque-controlled lightweight actuators”, IEEE Trans. Industry Applications, vol. 28, Nov. 1992, no. 6, pp. 1399-1405. Hashimoto M., Kiyosawa Y., Paul R. P., “A torque sensing technique for robots with harmonic drives”, IEEE Trans. on Robotics and Automation, vol. 9, Feb. 1993, no. 1, pp. 108-116. Tsetserukou D., Tadakuma R., Kajimoto H., Tachi S., “Optical torque sensors for implementation of local impedance control of the arm of humanoid robot”. In: Proc. 2006 IEEE Int. Conf. Robotics and Automation, Orlando, 2006, pp. 1674-1679. Bicchi A., Tonietti G., “Fast and soft arm tactics: dealing with the safety-performance trade-off in robot arms design and control”, IEEE Robotics and Automation Magazine, vol. 11, June 2004, no. 2, pp. 22-33. Carelli R., Kelly R., “Adaptive impedance/force controller for robot manipulators,” IEEE Trans. Automatic Control, vol. 36, Aug. 1991, no. 8, pp. 967-972. Dubey R. V., Chang T. F., Everett S. E., “Variable damping impedance control of a bilateral telerobotic system”, IEEE Control Systems Magazine, vol. 17, Feb. 1997, pp. 37-44. Tsumugiwa T., Yokogawa R., Hara K., “Variable impedance control based on estimation of human arm stiffness for human-robot cooperative calligraphic task”. In: Proc. 2002 IEEE Int. Conf. Robotics and Automation, Washington DC, 2002, pp. 644-650. Rahman M. M., Ikeura R., Muzutani K., “Investigating the impedance characteristics of human arm for development of robot to cooperate with human operators”, In: Proc. 1999 IEEE International Conference Systems, Man and Cybernetics, Tokyo, Japan, 1999, pp. 676-681. Walker I. D., “Impact configuration and measures for kinematically redundant and multiple armed robot system”, IEEE Trans. Robotics and Automation, vol. 10, Oct. 1994, no. 5, pp. 670-683. Volpe R., Khosla P., “A theoretical and experimental investigation of impact control for manipulators”, Int. Journal of Robotics Research, vol. 12, Aug.1993, no. 4, pp. 351-365. Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N째 4

2008

A NEW 3D SENSOR SYSTEM BY USING VIRTUAL CAMERA MODEL AND STEREO VISION FOR MOBILE ROBOTS Hyunki Lee, Min Young Kim and Hyungsuck Cho

Abstract: So many researches have been conducted to develop 3D sensing method for mobile robots. Among them, the optical triangulation, a well-known method for 3D shape measurement, is also based on active vision sensing principle for mobile robot sensor system, so that the measurement result is robust to illumination noises from environments. Due to this advantage it has been popularly used. However, to obtain the 3D information of environment needs a special scanning process and scanning actuators need. To omit this scanning process multi-line projection methods have been widely researched. However, they suffer from an inherent limitation: The results of multi-line projection method commonly have measurement errors because of 2p-ambiguity caused by regularly repeated multiline laser pattern. In this paper, to overcome 2p-ambiguity effectively, we introduce a novel sensing method for a 3D sensing system using multi-line projection and stereo cameras, based on the virtual camera model and stereovision algorithm. To verify the efficiency and accuracy of the proposed method, a series of experimental tests is performed. Keywords: virtual camera model, stereo vision, active sensor.

1. Introduction Nowadays a major research issue of mobile robots is development of environment sensing and recognition system for navigation and task execution. A variety of machine vision techniques [1-4] have been conducted for determination of 3D scene geometric information from 2D images. Although binocular and trinocular vision [5] sensors among them have been widely used as representative ones of passive sensor system, they still suffer from sudden changes of image intensity due to illumination noise, insufficient feature information on environment composed of plain surfaces and correspondence problem between multiple images. These reasons have made most mobile robot researches on 3D environment reconstruction deal with just straight lines, edges and corners as interesting features [2,3,6-8]. However, these features are saliently observed only in well-arranged and structured environments with simple polygonal objects or polygon-textured surfaces. This information may not be sufficient to describe the whole structure of 3D space [9]. In many approaches for mobile robot applications, a laser sensor has been frequently used for detail sensing beca-use of its robustness to the nature of the navigation environment and easy to feature extraction [10,11]. The LRF (laser range finder) measuring the time-of-flight 38

Articles

between the sensor and an object [12,13] is an example of such a laser sensor. The disadvantage is that it maintains the same resolution even at a short distance and needs still high cost with high power consumption, scanning process and heavy weight. To decrease the sensing time without degradation of the sensor resolution, it is necessary to develop a new visual sensor system different from the sensors mentioned above. The optical triangulation method has become a widely developed technique for 3D object shape measurements such as single laser line projection, multi-laser line projection, and colour laser line projection.

Fig. 1. The hardware system of an active vision system. In using single laser line projection, the method to acquire the 3D information of an object is simple and fast. However, a scanning process and a scanning actuator are required in order to acquire the entire 3D environment information, because at each measurement process the depth value is obtained only at the area exposed to the light source. Omitting this scanning process, multi-laser line projection and fringe pattern projection have been popularly used. However, these approaches typically have an inherent limitation of 2p-ambiguity, caused by fringe patterns or the use of laser-structured lights with regular frequencies. In this paper, a new sensing system and method are introduced with the aim of overcoming 2p-ambiguity by adopting principles of the trinocular vision method. [5] In demonstrating how this new technique can eliminate the 2p-ambiguity, the principles of the optical triangulation method are briefly reviewed first and then its limitations are discussed. In Section 2, the procedure of incorporating the trinocular vision method into the optical triangulation method is described in detail. In Section 3, a series of experiments is performed to investigate the performance of the proposed sensing system. Finally, in Section 4, conclusions are drawn regarding the validity of the proposed method.

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

1.1.

The principle of optical triangulation and its limitations Optical triangulation algorithm [14] is popular for 3D shape measurement. It uses the laser line or fringe pattern, and can measure the absolute depth value by analysing the geometric relationship between a ray of the camera and a ray of the light source. In Fig. 2(a) a point light source is an incident on a diffusely scattering surface under an angle q1. The resulting light spot on the surface, A, is imaged by a lens onto the detector at point a. The optical axis of the lens makes an angle q2 to the surface normal. Let's assume that the object moves with a distance s normal to its surface. From the figure, using simple trigonometric relations, we find that the corresponding movement of the images spot on the detector is given by

N째 4

2008

detected laser point is very easy whereas in the case of multi-laser line projection or fringe pattern projection the determination is very difficult. This phenomenon is known as 2p-ambiguity. As shown in Fig. 2(b), 2pambiguity is caused by multi-laser projection. We assume that multi-laser line is projected and among them one laser line scattered at the object surface is detected at point a in the detector. However, we cannot determine the exact depth value, as shown in Fig. 2(b). In Fig. 2(b), the scattered light from points A, B, and C can be measured at the same point, a, in the detector. Hence, the exact depth value cannot be determined. If a regular fringe pattern is projected, this phenomenon becomes even more problematic for depth measurement. Previous studies to solve the 2p-ambiguity problem Many studies have been conducted to solve the 2pambiguity problem. The first method proposed to overcome this limitation involves the use of structured-lighting sensors of sequential patterns codified in the time domain. [15] This method used several pattern coded fringe patterns and captures many fringe images, and thus requires much time to capture and obtain 3D depth information. Another means of dealing with this limitation is using a colour laser structured light [16,17] or intensity modulated laser structured light [18]. These two methods vary the intensity or colour of the fringe pattern to find a correspondence between the projection laser line and detected laser line. However, the modulated laser line introduces error depending on the characteristics of the surface of the object; if a red laser line is projected and the object colour is green or red, this red laser line cannot be detected in the detector. In addition, the hardware to project the multi-laser line is complex and expensive. Morano et al. [19] developed a structured-lighting method using pseudorandom array codes. This is an extension of the pseudorandom theory to the 2D case, and can take 3D information in a one-shot image using a colour projector. This approach typically involves a simple configuration composed of a camera and a projector that can illuminate the scene with various flexible shape patterns of light. However, the reflected pattern colour may be different from that of the projected pattern due to the different reflection properties of the object surface and differences of colour characteristics between the projector and camera. This may result in incorrect correspondence matching. Chen et al. [20] adopted a range data acquisition system using colour structuredlighting and stereovision for easy correspondence matching of the projected multi-line colour stripe pattern. By adding one more camera, they replaced the challenging problem of lighting-to-image correspondence by the more easily solved problem of image-to-image stereo correspondence. In a similar approach to that study, M.Y.Kim et al. [5] suggested a trinocular sensor system composed of two cameras and one pattern projector. For analysing 3D information from the sensor system, one pattern projector is assumed as a virtual camera. In order to solve the correspondence problem between the projected laser line 1.2.

(1) where m is the transversal magnification of the lens. By adopting the same principle, we can measure the depth value at the line area using a single laser line projector. (a) The principle of optical triangulation

(b) Limitation of the multi-laser line projection method

Fig. 2. Principle of optical triangulation and its limitation. If we want to obtain the whole 3D information of an object in the case of the single laser projection method, scanning process, wherein either a light source or object is moved by an actuator is required. This increases the sensing time and requires a costly scanning actuator. To solve this problem many researches have used a multilaser line projection method or fringe pattern projection method. In these cases, however, finding correspondence between the light spot on the object surface and the detected laser point poses a considerable challenge. In the case of the single laser projection method, as shown in Fig. 3, the determination of the correspondence between the light spot on the object surface and the

Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

and the detected laser line acquired in two camera images, a unique statistical way was proposed: voting phase and ballot counting phase based on trinocular vision theory. However, they did not focus on the 2pambiguity problem of a finely repeated multi-stripe laser pattern, and utilized only laser pattern images observed by two cameras and then analysed their line features. Therefore, in the present work, we review the 2p-ambiguity problem for a pattern projection system, and solve this problem by using not only laser pattern images but also stereo intensity images simultaneously. 1.3. Stereovision algorithm In order to explain the sensing algorithm, we must first discuss stereovision algorithm, because the proposed sensing system consists of three cameras and the relationship between these cameras is determined. Fig. 3 shows the general stereovision geometry. Here, the optical axes of two cameras are not parallel. In stereovision geometry, the corresponding points have the same intensity value. If points P1 and P2 are the corresponding points, we can draw the figure as shown in Fig. 3.

N째 4

2008

where eR is the vanishing point of camera 2; ML and MR are the perspective camera matrix of camera 2 and cath th mera 1, respectively, where ML(i, j) is the i row and j column component of the camera 2 matrix and ML and MR are already known through the calibration process; FL is the fundamental matrix, which indicates the relationship between two cameras; and (xR, yR) is the image coordinates of P1. The purpose of the stereo vision is to determine the depth value of the point P. [21] If we know that points P1 and P2 are the corresponding points and the coordinates of points P1 and P2 are (xL, yL) and (xR, yR), we can calculate the position X(XP , YP , ZP) of point P in the world coordinates.

(3)

where MLi and MRi are the i row of the camera matrix of camera 1 and camera 2, respectively. To solve equation (3), we perform SVD (Singular Value Decomposition) with matrix A, and the last column of the unit matrix becomes the solution of equation (3).

2. The proposed sensing method

Fig. 3. General stereovision geometry. In Fig. 3, O1 and O2 is the optical centre of camera 1 and camera 2, respectively, P1 and P2 are the camera image point of object point P at camera 1 and 2, respectively, and OXYZ are the world coordinates. The line between O1 and O2 is the baseline, and lines l1 and l2 indicate the epipolar line, which are the projection line of line O2P and line O1P at the image plane of cameras 1 and 2, respectively. The equation of the epipolar line l2 can be estimated as follows [21]: (2)

Our sensing method uses one more camera based on a typical optical triangulation method. One pattern projector is assumed as a virtual camera, and thus it is possible to acquire the effect of a trinocular vision system. Using this feature, depth information without 2p-ambiguity can be acquired. 2.1.

The hardware configuration of the sensing system Fig. 4(a) shows the basic concept of the proposed sensor system for three-dimensional surface profile measurement. The sensor system is composed of a multistripe laser pattern generator and two cameras. As shown in the figure, the laser beam generated by a multi-stripe laser pattern generator is projected onto a scene, and two cameras composing a stereovision system take the distorted laser pattern images of the scene. Fig. 4(b) shows that the pattern projector is assumed as a virtual camera. (a) The hardware set-up for proposed sensing system Laser pattern

laser pattern projector Multi-stripe projector multi-stripe laser pattern laser pattern

camera 2

Objects

Camera 2

camera 1 Camera 1

Floor floor

Articles

Distorted pattern

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

(b) Virtual camera model

NÂ° 4

2008

principle. Fig. 5(b) shows this process. The estimated 3D positions of candidate points are projected to the image plane of camera 1 by using the camera projection matrix of camera 1.

(6)

Fig. 4. The hardware set-up for proposed sensing algorithm.

(a) The first step of the proposed algorithm based on optical triangulation method

(b) The second step of the proposed algorithm based on stereo vision method

Fig. 5. The proposed algorithm based on the optical triangulation method and stereovision. 2.2. The proposed sensing algorithm The sensing algorithm consists of three steps. The first step is to find candidate points by using a virtual camera model and camera 2, which are the identical result with the typical optical triangulation method and are related with each other by 2p-ambiguity. First, we choose an arbitrary point, R1, in the image plane of camera 2, and then find an epipolar line at the virtual image, VEr1, as shown in the figure by using equation (2). Along the epipolar line VEr1, the points having the same intensity value with R1 are searched and the 3D positions of the candidate points are estimated by using these chosen points and R1 point by adopting equation (2). In Fig. 5(a) A, B, and C points are the obtained candidate points. The second step is to find a final matching point with R1 by adopting the stereovision

where S is the scale factor, X, Y, and Z are the coordinates of the candidate points (A, B, C) in the world coordinates, x1 and y1 are the image coordinates in the camera 1, and ML is the 3Ă&#x2014;4 camera projection matrix of camera 1, which consists of both intrinsic and extrinsic parameters. In the figure, a and g become the projection of candidate points A and C, respectively. Next, we can decide the matching point with position R1 by adopting a stereovision constraint. The main constraint of stereovision is that matching points in the left and right image have the same intensity. If the gpoint in the left image plane has the same intensity value with point R1 in Fig. 5(b), points g and R1 become matching points. However, the matching points are not found easily, and sometimes a number of matching points appear simultaneously for a given point. To overcome this problem, we check the stereo constraints (C1, C2, and C3) for these points, given in section 2.2. If the matching points satisfy these constraints, we can finally find the matching points. As noted previously, 2p-ambiguity is related to the correspondence problem of stereo images, and due to 2p-ambiguity we cannot find the correct depth value of the environment. However, in solving this correspondence problem the 2p-ambiguity problem can also be solved and we can obtain the correct depth value of the environment. The third step is to find the final depth value of point R1. By using this matching result, which is found at the second step, we can estimate the final depth value of point R1 by using equation (3). We repeat this process for other arbitrary points, and can obtain all depth values of the object.

3. Experimental Results The experimental system consists of a laser pattern projector and two cameras, as shown in Fig. 4(a). The world coordinates are located at the optical centre of the left camera and the optical axis of camera 2 is unparallel to camera 1. The reference plane is located in front of the sensor system, 938.10 mm, and the laser pattern width is 16mm. We can obtain a virtual fringe image, as shown in Fig. 6.

Fig. 6. Virtual laser pattern image by using virtual camera model. Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

Figs. 7, 8, and 9 show the sensing results of the proposed algorithm. In Fig. 7(a), a simple box is placed in front of the reference plane. The box size is 80×210×50 mm. As we can see in Figs. 7(c) and (d), the results of both optical triangulation and the proposed algorithm are almost identical, because the distance value between the front surface of the object and reference plane is less than that of the 2p-ambiguity. However, the 2p-ambiguity phenomenon can be seen in Fig. 8(c) from the result of optical triangulation.

(a) Object 3 for experiment

(b) Fringe images within the field of view

(d) Result using the proposed algorithm

Articles

2008

Next, a box with dimensions of 222×400×80 mm was placed in front of the reference plane, so the distance between the reference plane and the front surface of object 2 was 140 mm, as indicated in Fig. 8(a). In Fig. 8(c), the measured value, 38 mm, is different from the true value. However, the results of the value, as shown in Fig. 8(d), imply that the proposed algorithm can overcome the 2p-ambiguity. Likewise, the experimental results shown in Figs. 9(c) and (d) also sup-port that the proposed algorithm does not suffer from 2p-ambiguity.

(a) Object 2 for experiment

Fig. 7. Experimental results for object 2.

N° 4

Fig. 8. Experimental results for object 3.

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

(a) Object 4 for experiment

(b) Fringe images within the field of view

N° 4

2008

sing algorithm is introduced based on the typical optical triangulation method and stereovision technique. In our experiments, the proposed algorithm can overcome 2pambiguity whereas typical optical triangulation method cannot. In addition, the accuracy of the measured results of the proposed algorithm is similar to that of the typical optical triangulation method (typical optical triangulation method: 8 mm; the proposed algorithm: 4 mm). This indicates that the proposed algorithm, which combines the principle of the optical triangulation method with stereovision, maintains the accuracy of the former and can measure the absolute depth value without 2p-ambiguity. In order to assess the capacity of the proposed algorithm in terms of its performance in various practical applications, a series of experiments were performed under different object conditions, i.e. variation of object shape and surface texture. These experiments demonstrated the feasibility of successful perception. Thus, it is evident that the proposed sensing method can be utilized in many applications such as mobile robots, 3D inspection, etc.

AUTHORS Hyunki Lee* and Hyungsuck Cho - Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology, 373-1, Guseong-dong, Yuseong-gu, Daejeon 305-701, South Korea. Tel.: 82-42-8693253, 3213, Fax: 82-42-869-3210. E-mails: hklee@lca.kaist.ac.kr, hscho@lca.kaist.ac.kr. Minyoung Kim - Research Team of Vision System, Kohyoung Technology Co. Ltd. Seoul, South Korea. E-mail: mykim@lca.kaist.ac.kr. * Corresponding author (d) Result using the proposed algorithm References [1]

2. 3.

Fig. 9. Experimental results for object 4.

4. Conclusions

In this study, a novel sensing method for mobile robots is proposed based on the principle of optical triangulation and stereovision techniques to acquire more reliable information and to remove the 2p-ambiguity. The hardware system consists of two cameras and a pattern projector. In addition to this hardware, a novel-sen-

8. 9.

Valkenburg R.J., Mcivor A.M., “Accurate 3D measurement using a structured light system”, Image Vision Comput., vol. 16, 1998, pp. 99–110. Faugeras O., Three Dimensional Computer Vision: A Geometric Viewpoint, MIT Press, Cambridge, 1993. Ayache N., Artificial Vision for Mobile Robots: Stereo Vision and Multisensory Perception, MIT Press, Cambridge, 1991. Lee K.M., Kuo C.J., “Surface reconstruction from photometric stereo images”, J. Opt. Soc. Am. A, no. 10, 1993, pp. 855–867. Kim M. Y., Cho H., "An active trinocular vision system of sensing indoor navigation environment for mobile robots," Sensors and Actuators A: Physical, vol. 125, 2006, pp. 192–209. Zhang Z., Faugeras O., “A 3D world model builder with a mobile robot”, Int. J. Robot. Res., no. 11, 1992, pp. 269–285. Weckesser P., Dillmann R., “Modeling unknown environments with a mobile robot”, Robot. Auton. Syst., vol. 23, 1998, pp. 293–300. Cho H., Optomechatronic: Fusion of Optical and Mechatronic Engineerings, CRC Press, 2005 Everett H.R., Sensors for Mobile Robots: Theory and Application, A.K. Peters, Natick, 1995. Articles

Journal of Automation, Mobile Robotics & Intelligent Systems

10. 11.

12.

13.

14. 15.

16.

17.

18.

19.

20.

21.

Kanade T., Three-Dimensional Machine Vision, Kluwer Academic Publishers, Boston, 1989. Agapakis J.E., Katz J.M., Friedman J.M., Epstein G.N., “Vision-aided robotic welding: an approach and a flexible implementation”, Int. J. Robot. Res., no. 9, 1993, pp. 17–34. Suk M., Bhandarkar S.M., Three-Dimensional Object Recognition From Range Images, Springer-Verlag New York, Secaucus, 1992. Amann M., Bosch T., Lescure M., Myllyla R., Rioux M., “Laser ranging: a critical review of usual techniques for distance measurement”, Opt. Eng., vol. 40, 2001, pp. 10–19. Gasvik K.J., Optical Metrology, 2nd edition, John Wiley & Sons Ltd, 1995. Batlle J., Mouaddib E., Salvi J., “Recent progress in coded structured light as a technique to solve the correspondence problem: a survey”, Pattern Recogn., vol. 31, 1998, pp. 963–982. Schubert E., “Fast 3D object recognition using multiple color coded illumination”. In: Proceedings of ICASSP, vol. 4, 1997, pp. 3057-3060. Boyer K.L., Kak A.C., “Color encoded structured light for rapid active ranging”, IEEE Trans. Pattern Anal. Mach. Intell., no. 9, 1987, pp. 14–28. Lu C., Inokuchi S., “An Absolute Depth Range Measurement of 3-D Objects Based on Modulation Moiré Topography”. In: Proceedings of ICPR, vol. 15, 2000, pp. 754-757 Morano R.A., Ozturk C., Conn R., Dubin S., Zietz S., Nissanov J., “Structured light using Pseudorandom codes”, IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, 1998, pp. 322–327. Chen C.S., Hung Y.P., Chiang C.C., Wu J.L., “Range data acquisition using color structured lighting and stereo vision”, Image Vision Comput., vol. 15, 1997, pp. 445–456. Forsyth D. A., Ponce J., Computer Vision: A Modern Approach, Prentice Hall, 2003.

Articles

VOLUME 2,

N° 4

2008

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

NÂ° 4

2008

DEVELOPMENT OF A STABILIZED WIDEBAND PENDULUM-TYPE GRAVITY DIRECTION SENSOR UTILIZING AN ELECTROMAGNETIC CLUTCH AND BRAKE FOR ANTI-SWING Tokuji Okada, Akihiro Tsuchida, Tomofumi Mukaiyachi, Toshimi Shimizu

Abstract: This paper proposes the usefulness of combining an electromagnetic clutch and brake with a pendulum for antiswing for developing a stabilized wideband pendulum-type gravity direction sensor. The pendulum-type sensor is high in response to direction change but liable to swing even when the change stops, in general. Therefore, anti-swing control without degrading sensitivity of direction measurement is important. Continuous damping of the pendulum motion causes an error in a stationary condition unless the pendulum is free from damping repetitively in an appropriate term. To develop a quick response direction sensor without error, we use a magnetic device. Based on motion analysis of a double pendulum, we simulate behaviours of the pendulum-type sensor by actuating the magnetic device by using different signals in form, frequency and magnitude. Also, we fabricate a motion simulator having two rotation joints to evaluate the anti-swing effect of the magnetic device. In the experiment we demonstrate that the device is beneficial for suppressing pendulum vibration over a wideband for measuring resultant direction of the acceleration compound with motion and gravity on such a platform moving randomly. Also, we show that a triangular signal is better than a sinusoidal signal for making the settling time short; however difference is within a narrow margin. Keywords: anti-swing, electromagnetic clutch and brake, gravity direction sensor, pendulum, inclinometer.

1. Introduction Tilt angle is evaluated as an angular displacement from a certain direction. In general, tilt of a plane like a liquid surface or floor is evaluated by two angular values. A sensor of this type is called a 2-axis inclinometer. In either type, angular displacement is collected by evaluating electric potential using wired, conductive or magneto-resistive resistance, in general. Some other methods paying attention to the liquid level or position of a ball in a cylindrical tube or spherical vessel are reported [1]. A pendulum-type sensor has also been reported so far [2]. This type is sensitive to monitor motion change directly without the affect of offset and hysteresis. On the other hand, it is too delicate to reduce vibration. A servo controlled pendulum sensor can make friction torque zero since a gearing mechanism is not needed for the coupling potentiometer. Anti-swing techniques for carrying a load by crane have been proposed. In a special edition entitled â&#x20AC;&#x153;What's the essence of the vibration control?â&#x20AC;?, damping material

and new dampers are introduced [3]. There are two types of damping among these. First is the active type that cancels vibration by motors driven by external energy corresponding to the vibration power. Second is the passive type that transforms the vibration power into mechanical or electrical actuation for suppressing vibration without using external energy. At this time, two swingtypes having 2 axes and unit-type operating as a vibration absorber are considered as the active type. Actually, some of these are found in MR (Magneto-rheological) dampers [4], anti-sway based on switching in a suitable timing [5], feedback control for a gantry crane [6], use of the gyroscopic stabilising mechanism [7], crane control for anti-swing based on cable roll up or delayed feed back control [8], [9].

Fig. 1. Definition of parameters for motion analysis of a double pendulum. As for the passive type, gondola control using a swing damper [3], use of piezoelectric devices [10] are found for instance. In addition, the pendulum mechanism is applied to trains to improve the comfortableness of the ride [11]. This paper describes the development of a stabilized wideband pendulum-type gravity direction sensor utilizing an electromagnetic clutch and brake for anti-swing of a pendulum. We call the electromagnetic clutch and brake magnetic device for simplicity. The sensor is intended to stabilize a mobile robot's motion changing its position and acceleration dynamically on irregular terrain. For this purpose, the sensor is needed to measure the direction of resultant acceleration of motion and gravity with simplicity in assembling, signal processing and at reasonable cost. Actually, we propose a pendulumtype gravity sensor that measures the direction of resultant acceleration when the robot moves at high acceleration as an active-type device using the magnetic device instead of a gyroscope. This is because we noticed that a rigid pendulum arm is so sensitive to the robot's change in motion. Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

We analyze double pendulum motion since this pendulum simulates the measurement environment attaching the sensor to the robot's body while it moves. Simulated results are compared with experimental results to show validity of mathematical equations extracted in the analysis. We clarify the influences of signals in form, frequency and level for actuating the magnetic device to design a high performance pendulum-type gravity sensor.

N째 4

2008

(4)

All of these energies are summed up to have L (5)

2. Analysis of a pendulum motion A. Extraction of mathematical form Generally, an installed pendulum on a moving object is supposed to be a double pendulum as shown in Fig.1. In the following, we call first pendulum for the first rigid arm extending from the rotation centre O, and second pendulum for the next rigid arm connected serially to hang a mass which we call weight at the end. We analyze motion of the second pendulum in a vertical plane referred by horizontal and vertical axes of X and Y, respectively. Nomenclatures are as follows, where the subscript i stands for the pendulum position, say 1 for the first pendulum for instance. g - gravity acceleration li - length of the pendulum arm mi - mass of the pendulum arm Mi - mass of the pendulum weight S - settling time of the pendulum t - time elapsed (xi,yi) - mass centre of the pendulum weight (Xi,Yi) - mass centre of the pendulum arm qi - inclination/rotation angle of the pendulum (CCW from gravity direction is positive) f - angle toward which the mass M2 exists (CCW is positive similarly to qi) . w - angular velocity of the first pendulum(= q1) Let suppose that Ta means total kinetic energy of the mass Mi, then we have

where 2 A1 = l1 (3M1 +3M2 + m 1 +3m2)/6 2

(6)

A2 = l2 (3M1 + m2)/6

(7)

A3 = l 1 l 2(M2 + m2/2)

(8)

A4 = gl 1(M1 + M2 + m 1/2+ m2)

(9)

A5 = gl 2(M2 + m2/2)

(10)

At this moment, introducing a root rotation of the first pendulum with a constant angular velocity makes w constant. Substituting Eq. (5) into the Lagrange's equation results in the form [12]. (11) where c means a resistance coefficient under the case where resistive force occurs proportionally to the second pendulum's angular velocity. When the mass M2, called weight afterwards simply falls freely from the angle f[rad], w becomes zero. Therefore, (12) Unknown parameters q2 in Eqs. (11) and (12) are solved by the Runge-Kutta method for instance.

(1)

Also, supposing Tb is total translation energy of the arms, we can write

(2)

Assuming that the two pendulum arms are composed of uniform material yields total arm rotation energy Tq. (3) Concerned with potential energy expressed as V, it follows that

Articles

Fig. 2. Swing angles in the upper and lower parts depend on the condition when the clutch/brake is activated or not in the case when w takes zero and the mass M2 falls freely from f= -p/2 [rad].

Journal of Automation, Mobile Robotics & Intelligent Systems

Fig. 3. Analytical settling time versus l2 when the pendulum falls freely from f= -p/2 [rad].

VOLUME 2,

N° 4

2008

to swing ±p/3 [rad] from the datum direction f= -p/2 [rad], its motion is simulated and shown in Fig.4 under -2 such conditions that l2=6 [cm], m2=5.88×10 [N], -1 M2=1.568×10 [N], and swing frequency 1 [Hz]. The upper and middle parts are the results of q1 and q2, respectively. From these we notice that the second pendulum swings two times in one cycle swing of the first pendulum. The lower part shows the motion of the second pendulum when the second pendulum root is regulated from the vertical base by an intermittent signal in the period when the time elapses from 3 to 7 seconds under the same conditions mentioned before.

Fig. 5. Comparison of a simulated pendulum motion with and without anti-swing when the motion pendulum root rotates at a constant speed 60[rpm] under l2=5.4 [cm].

Fig. 4. Analytical results of the characteristics of q2 versus t when the pendulum l2 swings ±p/3 [rad] around the direction of f= -p/2 [rad]. B. Simulation of pendulum motion using numerical method B.1. In the case when pendulum root is immovable We estimate pendulum motion by solving q2 of the differential equation Eq. (12). Actually, suppose that the weight falls from a lateral direction(f= -p/2 [rad]) under such that l1=13.1 [cm], l2=7 [cm], m1=M1=0, -2 -1 m2=6.86×10 [N], and M2=1.568×10 [N]. Then characteristics of q2 versus t are shown in the upper part of Fig.2. Under the same conditions, we regulated the pendulum root by applying a brake-on and brake-off signal repetitively, i.e. lock and release, called intermittent signal, to the numerical method for locking the root with a duty ratio 0.5 and evaluated the settling time of the pendulum. Our settling time is defined as the time required making the pendulum stable within a permissible error of ±p/180 [rad]. Lower part of the figure shows the results when the signal has frequencies of 6, 12, and 18 [Hz]. From the figure we can confirm that the frequency 6[Hz] makes the time shorter in most cases. Settling time is also affected by pendulum length l2. The time using the intermittent signal of frequencies 5, 10, and 15 [Hz] is shown in Fig.3. These results imply that the settling time becomes shorter when the frequency is lower and the length is shorter. B.2. In the case when pendulum root swings Suppose that the first pendulum root is activated

Fig. 6. Comparison of a simulated pendulum with and without anti-swing when the pendulum root rotates at a constant speed under l2=7.0 [cm].

Fig. 7. Simulated motion of the double pendulum when w is constant. Swing angles of q2 are shown in free and controlled conditions at the upper and lower parts, respectively. B.3. In the case when the pendulum root rotates at a constant speed A double pendulum is formed when the arm l1 rotates at a constant speed. And the weight has a tendency to face in the direction of the resultant acceleration of gravity and motion. After calculating the relation of q2 versus t, weight position is collected by changing q1 with an increment of p/20 [rad], and displayed by black circles Articles

Journal of Automation, Mobile Robotics & Intelligent Systems

in Fig. 5(a). Rotation is considered under the condition such that l1=13.1 [cm], l2=5.4 [cm], m1=M1=0, -2 -1 m2=5.292×10 [N], M2=1.568×10 [N] and w= 2p [rad/s] (=60 [rpm]) in CCW. In such a case when -2 l2=7.0 [cm] and m2=6.86×10 [N], we have the results found in Fig. 6(a). From these figures, it is evident that the weight direction is greatly affected by the root rotation. And it is confirmed that the direction is dependent on the amount of w. On the other hand, the direction change of the weight is stabilized when the second pendulum root is regulated by the intermittent signal of 18 [Hz]. Actually, calculated results of the previous rotation are shown in Fig. 5(b) and Fig. 6(b). One can observe that the pendulum swing is drastically suppressed to prove the anti-swing effect. Similar conditions to get the results shown in Fig. 6 are applied to calculate q2 versus t characteristics shown in Fig. 7. The period from t=3 [s] to 7 [s] in the lower part of the figure shows the regulated results from the vertical base by the intermittent signal of 18 [Hz]. From this figure, we can confirm that the pendulum swing decreases sharply to pursue the anti-swing effect.

3. Fabrication of sensor and swing generator In order to certify the anti-swing effect quantitatively, it is necessary to develop an appropriate pendulumtype gravity direction sensor. Also, it is beneficial to devise a simple electro-mechanical generator producing a variety of vibration. Fabrication of these is described in the following.

Fig. 8. Outlook of the fabricated experimental set.

Fig. 9. Gravity sensor combined with the electro-magnetic clutch and brake. 48

Articles

VOLUME 2,

N° 4

2008

A. Construction of gravity direction sensor There is an overview of the experimental set in Fig. 8. Main components of the set are a pendulum-type gravity sensor and vibration generator on which the sensor is attached. The sensor is assembled by combining a pendulum with a magnetic device coaxially, and also with a potentiometer by a gearing mechanism. Actually, the clutch and brake of a diameter of 28 [mm] and thickness of 14 [mm] (Miki pulley AP05Y07), and potentiometer (Midorisokki CP-3M) are assembled to measure the angle q2 (see Fig. 9). The pendulum root is suspended on the rectangular frame through radial bearings. This root fixes a pendulum arm and guides a thin elastic disc of the magnetic device so that it can slide toward the axial direction freely. On the other hand, the device has coils fixed to the rectangular frame and attracts the disc with proportional force to electric current flowing through the coils. Therefore, the pendulum is free to move when no current flows. That is, the pendulum motion is regulated easily by an appropriate current so that the attractive force is controlled analogously from zero to a certain value. With a large force, the pendulum is locked from instantaneous rotation. B. Preparation of a vibration generator When the sensor is attached to a moving object, the sensor weight has a tendency to find a relaxed direction by rotating for pointing resultant direction of gravity and motion. The tendency has the advantage of controlling the table so that it can keep a vessel steady without spilling the liquid contents. When the sensor is attached to a slow motion robot, the direction is assumed as the gravity direction and is useful for controlling platforms for loading or seating horizontally on an irregular terrain. Basically, the principal aim of the sensor is to measure angular shift quantitatively from gravity direction even when the sensor is attached on an object moving with high acceleration in 3-D space. However, in our leg-type locomotion environment, the sensor should measure robot posture in the sagital plane (pitch) or colonal plane (roll) without the affect of vertical swing. Evidently, sudden change of motion disturbs the measurement. Principally, the sensor is composed of a pendulum having a single arm. And the pendulum root moves in a real measurement environment on a mobile robot. This is the reason why the double pendulum is discussed in section 2.A. This idea initiated us to fabricate a vertical vibration generator. This is overviewed like a gondola (Ferris wheel) since it has a rotating wheel around a horizontal axis and hangs a weight from the wheel rim pivot. The platform for carrying spectators is assumed to be a pendulum hanging from the weight. The hanged object is counter-balanced. The wheel centre axis is driven by a DC motor (SS23FLH50 24V) with reduction gear(1/50). According to the motor input signal, the first pendulum root can rotate CW and CCW. Also, it can swing around an assigned direction within a limited range. Original signals are produced by a signal generator (Iwatsu SG-4101). Amplified signals having a sinusoidal, triangular or rectangular form serve to drive the motor.

Journal of Automation, Mobile Robotics & Intelligent Systems

C. Sensor installation on the vibration generator Since the pendulum root is freely supported by radial bearing at the rim, the gearing mechanism of the potentiometer gives angular displacement of the pendulum arm from a certain angle on the rim body. Therefore, it is necessary to integrate angles of wheel rotation and potentiometer output to extract angular displacement of the arm from gravity direction, said to be the inclination angle. But this integration is time consuming and not good for real time measurement. To measure the inclination angle in real time, we propose the use of an immovable base as a part of the ground at the pivot joint because the base gives the angular index for understanding potentiometer output. In fact, we generated the base by connecting two same sized sprockets with a timing belt. One sprocket is fixed to the vibrator body located at the wheel centre. The other sprocket is attached to the cylinder supported at the pivot joint. The timing belt (Bridgestone S3M414) is efficient to generate the base at the joint (see Fig.8). The base having a cylindrical form is attached to the aforementioned rectangular tube on which the potentiometer is attached. Since the second pendulum root is free to rotate coaxially on the cylinder, potentiometer output means the exact data of our measurement. That is, the rectangular tube always keeps the same posture regardless of wheel rotation. A variety of robot vibration is produced at the pivot joint by transferring a variety of signals to a motor. At the same time the potentiometer can measure the inclination angle of the pendulum without disturbance. Therefore, the assembly is beneficial to collect data effectively and to evaluate swing suppression ability of the magnetic device directly without calculation even when a fullscaled robot is not available.

VOLUME 2,

NÂ° 4

2008

produce brake force and relaxing time. To apply this, we used a modified repetitive signal based on half-wave rectification. This rectification is easily performed by inserting a diode in the electric circuit for driving the device. The time interval is controlled by changing the frequency of an original signal. Actually, the positive part of a signal from the function generator (Iwatsu FG330) is rectified for producing a smooth brake force. Its negative part makes the brake force zero.

Fig.10. Experimental characteristics of q2 versus t when the weight falls freely from f= -p/2 [rad]. Analytical and experimental results are in the upper and lower parts, respectively.

4. Experimental results To certify the anti-swing effect of the magnetic device, pendulum motion is observed while its root swings or rotates, in addition to free fall of the weight from a fixed position even when the pendulum root remains at the same position. A. Driving signal of the magnetic device An appropriate amount of damping is needed to suppress pendulum vibration but it must be released at a certain interval to avoid steady error even when there is no vibration. Unfortunately, there is no way to determine release timing and interval for adapting to unknown vibration at this moment. Also, the damping should be gentle so that there is no shock to the pendulum. This implies that a pulse form signal is not good for driving the magnetic device; however we used the form as an intermittent signal in the simulation for calculation simplicity. We noticed that a repetitive signal like a sinusoidal wave is useful. However, the device produces an attractive force regardless of signal sign. This operation looks as if a full-wave rectified signal is inputted. Therefore, relaxing time generated by this plus and minus repetitive signal is momentary and not enough to track motion correctly. Our solution is to assign an equal interval to

Fig.11 Experimental characteristics of q2 versus t when the weight swings Âąp/3 [rad] from the datum direction f= -p/2 [rad]. B. Free fall of the weight In the case when the pendulum root is immovable in the direction q1= -p/3 [rad], we released the weight from the direction of f= -p/2 [rad] so that it falls freely. Its behaviour is observed for comparing motion with and without the brake force. Other motion is demonstrated by changing the signal in form, frequency and arm length. A storagescope (Iwatsu DS-8710) collected so much data easily. The upper Articles

Journal of Automation, Mobile Robotics & Intelligent Systems

part of Fig. 10 shows memorized wave forms concerned with q2 versus t under f= -p/2 [rad]. The lower part compares difference of q2 versus t when the magnetic device is driven by signals of pulsating, sinusoidal and triangular form for half-wave rectification. In the figure, a frequency of 6[Hz] has been proven to be better in Fig. 2. Evidently, pendulum swing is remarkably suppressed in each of the three signals. Triangular form is rather good in the three. C. In the case when pendulum root swings From the results shown in section 2.B, it is supposed that f influences the generation of many types of vibration. Obviously, one can say that the vibration becomes larger in the vicinity of f=p/2, p, and 3p/2 [rad]. This is clearly observed in our experiment. To show one example of this, let's suppose that swing frequency 1[Hz], halfwave rectified sinusoidal and triangular form of frequency 18[Hz], and f= -p/2 [rad], -p/3 < q2 [rad]< p/3, -2 -1 l2=6 [cm], m2=5.88×10 [N], M2=1.568×10 [N]. Then, time dependent pendulum motion is monitored by the storagescope (See Fig.11). The first and second parts in the upper area show q1 and q2 when there is no brake force, respectively. The first and second parts in the lower area show q2 when the signal to the magnetic device has half-wave rectified sinusoidal and triangular forms, respectively. Brake force is given in the period from t=3 to t=4. These results confirmed that the brake force prevents the arm from excessive swing. In addition, the triangular form is superior to the sinusoidal form; however difference is within a narrow margin. Also, it is confirmed that wave forms of q2 are deformed from those obtained in the simulation (see Fig. 4). These might be caused by the fact that w drifts since q1 is simply changed by a feed forward control without thinking about load difference between move-up rotation and move-down rotation. Other factors are pointed out in low motor response to control q1 because motor inertia and reduction gear ratio are not considered. However, it is true that the vibration caused by the pendulum root swing is drastically suppressed.

VOLUME 2,

N° 4

2008

D. In the case when pendulum root rotates In the experiment of irregular pendulum root rotation, the brake force operates effectively also to suppress pendulum vibration. However, the pendulum behaves different with the affect of a small angular error of arm direction in the initial state of rotation, and becomes chaotic to form different motion patterns.[12]. These motions are not shown because of the limited number of pages. Hori [13] verified the possibility of controlling this motion with stability by the method called OGY. In this paper, we show experimental results performed under the same conditions that were used to get the results in Figs. 6 and 7. However, the driving signal was changed from the intermittent to triangular forms, and the first pendulum rotates CW under such conditions that -1 -1 l2=3 [cm], m2=2.94×10 [N], and M2=9.8×10 [N]. Figs.12 and 13 show results without and with brake force, respectively. In these figures, weight positions are marked with white circles for ease of understanding q2. The front circular board is a big protractor to help in measuring q1 with the naked eye.

Fig. 13. Pendulum direction when the magnetic device is activated for anti-swing. Fig. 14 shows the collected wave form of q2 in the demonstration shown in Fig. 13 using signals of halfwave rectified sinusoidal and triangular forms. Brake force is not applied in the upper part, but applied in the middle and lower parts using the triangular form and sinusoidal forms, respectively. Advantages of the brake force are also verified here. Triangular form is superior to the sinusoidal form; however the difference is within a narrow margin. Magnitude of the driving signal in the aforementioned three cases has not been discussed so far; however fundamental experiments proved that the brake force increases as much electric current flows through the device. These preparatory experiments helped us to find the appropriate voltage that is determined by considering input impedance of the device.

Fig. 12. Pendulum direction when the magnetic device is inactive for free rotation. 50

Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N° 4

2008

ratio by considering inertial moment of the pendulum arm and weight. The proposed sensor will be helpful as a 3-D pendulum-type gravity sensor by supporting the pendulum root in a spherical joint. These subjects are part of our plan for the future.

AUTHORS Tokuji Okada* and Tomofumi Mukaiyachi - Niigata University, Department of Biocybernetics, Ikarashi 2-8050, Niigata-Shi, Japan 950-2181. E-mail: okada@eng.niigata-u.ac.jp. Akihiro Tsuchida and Toshimi Shimizu - Graduate School of Niigata University, Ikarashi 2-8050, NiigataShi, Japan 950-2181. * Corresponding author

References [1]

Fig. 14. Experimental motion of the double pendulum × motion when q1 is constant. Swing angles of q2 are shown at the upper and lower parts depending on inactive and active controls of the magnetic device, respectively.

[2]

5. Concluding remarks We proposed the construction of a pendulum-type gravity direction sensor using a potentiometer combined with a magnetic device. Also, we showed the design of the sensor with a vibration generator for evaluating sensor performance. In this paper, the magnetic device is fixed on the first arm but on the vertical base to avoid cable twist when the first arm rotates. In the simulation of sensing the gravity direction, we calculated the inclination angle of the pendulum arm and also verified that the calculated results became similar to those observed in a practical demonstration. In the experiment giving brake force to a pendulum root, we proved that the force was valuable to suppress pendulum vibration from both analytical and experimental considerations. Particularly, a signal of half-wave rectified sinusoidal form has the advantage of driving the device for making settling time short. The signal of a half-wave rectified triangular form was effective in its anti-swing performance; however the difference was so small. For making the settling time short we confirmed from the experiment that a pendulum having a short and long arm was valid for narrow and wide band vibration, respectively. Since the proposed pendulum-type sensor could measure gravity direction angle without the affect of vibration, its output might be utilized to control a robot seat for comfortable riding, for instance. Application into a servo loop will always keep the seat horizontal and guide normal equilibrium sense to a driver. The magnetic device was so small in size and power for suppressing vibration. The powered-up device will serve not only as a sensor but also as an actuator to suppress swing of a crane, for instance. To improve sensor performance for anti-swing and powered-up use as an actuator, it is worth while to determine signal frequency and duty

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11] [12] [13]

T. Okada, K. Kurosaki, K. Berns, R. Dillmann, “Measurement of resultant acceleration in 3D space based on sensing a metallic ball position on elastic layer located at inside of a spherical vessel”, Trans. of SICE, no. 787/796, 2005, pp. 41-10. Daniel Inaudi, Branko Glisic, “Development of a fiberoptic interferometric inclinometer”. In: Proc. SPIE Int. Symp. on Smart Structure and Materials, no. 4694, 2002, pp. 36-42. K. Suzuki, “Damping and passive vibration controlDevelopment of damping material and new dampers-”, J. of SICE, no. 37-8, 1998, pp. 531-540. C. Sakai, T. Terasawa, A. Sano, “Adaptive identification of MR damper with application to vibration control”, Trans. SICE, vol. 42, no. 10, 2006, pp. 1117-1125. R. Kondo, S. Shimahara, “Anti-sway control of a rotary crane via two-mode switching control”, Trans. SICE, vol. 41, no. 4, 2005, pp. 307-313. Yannick Aoustin, Alexander Formal'sky, “Simple antiswing feedback control for a gantry crane”, Robotica, vol. 21, 2003, pp. 655-666. O. Nishihara, H. Matsushita, S. Sato, “Vibration damping mechanisms with gyroscopic moments”, Trans. JSME, vol. 57C-534, 1991, pp. 497-503. Ho-Hoon Lee, “A new approach for the anti-swing control of overhead cranes with high speed load hoistling”, Int. J. of Control, vol.76, no. 15, 2003, pp. 1493-1499. Ziyad N. Masoud, Alih. Nayfeh, Nader A. Nayfeh, “Sway reduction on quay-side container cranes using delayed feedback controller: Simulations and experiments”, J. of Vibration and Control, vol. 11, no. 8, 2005, pp. 1103-1122. T. Fujita, H. Nomura, M. Yasuda, A. Matsuura, M. Tsuchiya, “Fundamental study of passive microvibration control with smart structure using piezoelectric devices”, Trans. JSME, vol. 66C-644, 2000, pp. 1097-1101. Technical report of Kawasaki Heavy Industry, Special edition on railway wagon, no. 160, 2004. M. Komuro, “Double pendulum and chaos”, J. of RSJ, vol. 15, no. 8, 1997, pp. 1104-1109. H. Shikano, Y. Hori, “Stabilizing control of double pendulum using reconstructed attractor”, Trans. IEE Japan, vol. 120D-1, 2000, pp. 142-147. Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N° 4

2008

PARALLEL ROBOT MOTION APPLIED TO FLIGHT SIMULATOR CONTROL LAW SYNTHESIS Cezary J. Szczepański

Abstract: The herein proposed method of synthesis of the parallel robot applied as flight simulator motion system control laws came from author's 20 year experience in developing, integrating and testing the control laws and their software dedicated to different simulators. The goal of the simulator motion control law synthesis at the proposed method is not minimizing cost function taken a priori at the beginning of that synthesis process but achieving positive assessment by the operator (e.g. pilot) team on the basis of simulator motion perception. Procedures adopted for those simulators FAT (Final Acceptance Tests) within proposed method were based on standard military equipment testing methods. Performing the final and the most important tests by the real device operators (pilots) was the new element here. The other important modification of the classical method was introducing the simulated object acceleration derivative into the filters controlling the simulator motion system. It appeared to be particularly effective in the cases of highly manoeuvrable airplane simulators. Keywords: simulator motion system, parallel robot.

1. Introduction The first simulators were equipped with the motion systems - they were flight simulators. Their function was to support the training of the future pilots. The first flight simulator [1] was the Model B without tail and engine sections. It was used in 1910 for the training of Wright's “Flyer B” piloting. It was supported in the way allowing banking, crucial for piloting that airplane. The cam was driven by electric motor, continuously changing the roll motion of the simulated airplane. When the student properly operated the controls, achieving the horizontal position of the wings was possible. After a few hours of such training the proper reaction habits appeared. The other training device constructed around year 1910, being one of the first simulators was shown in the Fig.1. It was manually driven and consisted of two halves of the barrel, mounted one on the top of the other. They were imitating the simulated airplane roll and pitch motion. The trainee goal was to keep the level reference bar in a horizontal position. Even in that pioneering time the importance of the motion stimuli for the motion object control was appreciated. They are of particular meaning when the human-operator is tightly connected with the controlled object, like it takes place at the airplane, space vehicle, car, railway locomotive or ship. For those objects 52

Articles

controlling human-operator uses the motion stimuli caused by the object motion. Those stimuli have the shape of acceleration, velocity, translation (linear or rotational). As the visual stimuli are not easy to replicate but easy to evaluate, then the motion stimuli are both difficult to replicate and evaluate. Their evaluation is usually more subjective than the other stimuli.

Fig. 1. "Antoinette" one of the first synthetic flight simulators. For the controlled moving objects, which contain their operator on-board, the motion stimuli are priceless information source on the object orientation and its changes. For the human they constitute the basic source of information on the motion. So that object simulator should imitate those stimuli in the way allowing the human-operator controlling the object in the simulator the same way as in the real device. For the simulator evaluation the formal, objective methods have been applied for years. As the result, formally correct simulator was received by the end user. Unfortunately the end user operators didn't accept the simulator, and particularly its motion system, as not similar to the real object. Those critical remarks have lead to the eliminating the motion systems from many types of the simulators, e.g. many flight and mission simulators operated by USAF. In such a situation the other way of simulator evaluation had to be found. The final goal was creating the simulator evaluation and testing method, which could lead to its acceptance by the end user operators. That problem stroke the author of the paper and his co-operators in the mid of the 80. of the previous century, when the work on “Iapetus” (Fig. 2) full mission simulator of the subsonic jet trainer airplane started. The work was continued in Poland for some years.

2. Anthropocentric algorithm of simulator motion control The object controlled by the system described in that paper is the parallel robot, called in the simulator world:

Journal of Automation, Mobile Robotics & Intelligent Systems

classical, synergetic, hexapod motion system. The reference systems adopted in that paper are standard ones; their detail description one can find in. [2] Also the kinematical and dynamic simulator motion platform equations will not be listed here, as they are well known. The matter of the paper is the new type algorithm of the wash out filters controlling the simulator platform motion. The anthropocentric algorithm is of the classical type. [3] It is nonlinear washout, which parameters are tuned up to perception of the selected group of the test humanoperators, e.g. pilots. Standard input data into that algorithm, calculated by the “simulated object dynamics” module are: both linear and rotational accelerations and velocities. During development of the “Iapetus” flight simulator motion system driving algorithms author has found, that the standard input data are not enough. The other additional signals were checked and tested, and among them the derivative of acceleration acting on the pilot appeared to be the best for our purpose. The same algorithm structure was applied into the simulators of the following objects: supersonic fighter-bomber airplane Su-22, medium multipurpose helicopter W-3WA “Sokol” and electric locomotive EP-09.

VOLUME 2,

N° 4

2008

The non-coordinated filters are developed for the following channels: - vertical (heave in aviation), - yaw. 2.1.1. Longitudinal motion The simulator moving platform accelerations which are to be performed in that channel, in the reference system connected with that platform, are the following: (1) where: - platform acceleration component imitating the linear acceleration of simulated object; - platform acceleration component imitating the rotational acceleration of simulated object. The platform acceleration component imitating the linear acceleration of simulated object, one can get from the formula: (2)

where: tp, zp - coefficients of the filter controlling the platform motion. Additional indexes precisely indicate their function and belonging to the particular filters. The platform acceleration component imitating the rotational acceleration of simulated object can be calculated using the formula: (3) It compensates the human-operator incorrect feeling of the specific force component at the simulator, coming out of the simulated object pitch acceleration simulation. Fig. 2. "Iapetus" research flight simulator. For the sake of avoiding the repeating of the equations, well known among their developers, only the new equations, characteristic for that method, will be presented here in detail. The scheme of proposed algorithm is presented in the Fig.3. After reading the input data calculated in the “object dynamics module” the first step of the algorithm is calculated. It is calculation of the specific force acting on the human-operator in the simulator. Next the specific force gradient is being calculated. After that, algorithm performs the translational and rotational motion filters calculations, and corrects the actuators required moves, at the end. 2.1. Translational motion filters In the anthropocentric algorithm, the coordinated filters equations are developed for the following motions: - longitudinal (surge in aviation) and pitch, - lateral (sway in aviation) and roll.

2.1.2. Lateral motion The simulator motion platform lateral acceleration, described in the reference system connected with that platform, has the form: (4) where: - platform acceleration component imitating lateral acceleration of the simulated object, - platform acceleration component imitating components of roll acceleration of the simulated object. The acceleration component is calculated from the high pass filter equation, where the input signal is the acceleration gradient, without gravity component: (5)

Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

The component of the motion platform lateral acceleration is described as following: (6)

N째 4

2008

2.1.4. Translational motion wash out filters The simplest description of the wash out filters can be achieved with the use of the inertial reference system O1x1y1z1. The translational accelerations of the motion platform in that system are as follows:

It compensates the human-operator incorrect perception of the specific force, which is connected with the imitating the simulated object roll acceleration.

(8)

2.1.3. Vertical motion The simulator motion platform vertical acceleration in the reference system connected with that platform is being calculated from the vertical acceleration gradient of the simulated object. There is no rotational motion component in that channel. The formula for calculating the is the following:

(9)

(10) (7) Then the wash out filters equations take the shape: (11)

(12)

(13)

where: - simulator motion platform accelerations after the wash out filters transformation. 2.2. Rotational motion filters 2.2.1. Pitch motion Simulator motion platform pitch angle as following:

is defined

(14) where: - platform pitch angle generated by the low pass filter for the slow-speed changing courses coming out of the platform pitch angle and acceleration component coming out of the gravity, - platform pitch angle generated by the high pass filter for the high-speed changing courses connected with the pitch acceleration of the simulated object.

Fig. 3. Simulator platform main calculation loop scheme. 54

Articles

Simulator motion platform pitch angle comes out of the coordinated simulation of the longitudinal and pitch motion of the simulated object. That angle reflects the platform pitch caused by the low-speed changing, long lasting, and longitudinal acceleration imitation. Low pass filter allows the changing of the platform pitch angle below the human-operator perception threshold. Comparing the longitudinal specific forces acting at the centre of reference systems connected with the simulated object and the simulator motion platform, one can achieve the equation:

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N째 4

2008

(15)

(20)

where: - simulator platform pitch angle before the filtering, which imitates the low-speed changing component of the longitudinal acceleration.

From the above equation one need to calculate the angle, using the value of angle calculated in the pitch channel. Next it goes to the equation for the low pass filter in the roll channel:

The equation (15) is used for calculation of the angle. The low pass filter equation is as follows:

(21)

(16) Low pass filter of the longitudinal motion After differentiating and implementing into the Eq. (16) one can achieve the equation for the angle:

Value of the formula:

one may calculate from the following

(22)

(17) but in practice more effective way is to use the Newton method for its calculation. High pass filter of the roll acceleration The component of the simulator motion platform roll angle is being calculated in the same way as the high pass component of the platform pitch angle. The high pass roll filter is the following: High pass filter of the pitch acceleration High pass component of the simulator pitch angle is felt by the human-operator at the majority of the simulated object motion control tasks as the disturbance of the specific force. Because of that, during simulation of those phases of the motion, that component has to be comparatively small. That is guaranteed by the high pass filter:

2.2.3. Yaw motion It is imitated in the uncoordinated way. The high pass filter in that channel goal is to dump the simulated object low frequency yaw acceleration changes. The filter equation has the form:

(18)

(24)

component

2.2.4. Rotational motion wash out filters As it was at the translational motion case, the most suitable is to transform the simulator platform velocities into the inertial reference system. Then they take the shape:

From that equation one can get the of the equation (14).

2.2.2. Roll motion The simulator motion platform roll imitates in a coordinated way the low-speed changing component of lateral specific force and roll acceleration of the simulated object. Platform roll angle is described by the following equation:

(23)

(25) (26)

(19) (27) where: - platform roll angle generated by the low pass filter for the low-speed courses coming out of theeeee platform roll caused by the gravity component, - platform roll angle generated by the high pass filter for the high-speed courses coming out of the simulated object roll acceleration.

Those are the input data for the simulator platform rotational motion wash out filters: (28)

(29) Low pass filter of lateral motion Comparing the low-speed changing components of the lateral specific forces acting at the centre of reference systems connected with the simulated object and the simulator motion platform, one can achieve the equation:

(30)

Articles

Journal of Automation, Mobile Robotics & Intelligent Systems

2.3.

Correction of the required actuator movement 2.3.1. Correction caused by the motion platform dynamics During the simulator operation, one can meet the case, when required move of the platform and/or its position at a certain calculation step could cause the motion dangerous for the platform itself. The most common case of that type is requirement of the move resulting with the exceeding the platform maximum load. That limit is usually caused by the limits of the equipment mounted on top of the platform, e.g. presentation system or computers. The necessity of implementing of those limits into the calculation of the required moves of the platform, introduces the non-holonomic constraints into the solved filter equation systems, making them much more complicated. That problem is usually omitted during the formal, literature analysis, but in the real system it needs to be solved for avoiding the equipment damage. The author's experience shows, that for some types of the simulated objects, with the “high dynamics” characteristics, when the quick and/or big simulator platform moves are required, quite effective solution is introduction of the second level gain coefficient into the filter equations. The values of dynamic parameters at which that second level coefficients are switching on into the calculations, should be established a priori and tested with the operators of the simulated object. 2.3.2. Correction caused by the actuator dynamics The next stage of the actuator moves correction is taking into consideration their characteristics. Each actuator has its own maximum values of the acceleration, velocity or displacement. Those limits cannot be crossed during the simulator work. So at each step of the actuator control, one has to check whether: acceleration gradient, acceleration, velocity and displacement limits for each of the actuator are not exceeded. If one of the limits could be crossed, then the required parameter (acceleration gradient, acceleration, velocity or displacement) of the endangered actuator has to be corrected up to the limit. If it is the case of a few endangered actuators parameters in the same control calculation step, one needs to correct the endangered parameters of each actuator to the same extent as the worst case actuator. During that correction process, one needs to remember the most important goal of the motion stimuli generation, i.e. the proper motion cue generation. Practically we can say that the motion system goal is to replicate the specific force vector in the simulator as it would be in the real object during the simulated phase of that object motion. The most important parameter of that vector is its sense, next direction and the less important its module. Keeping that in mind, we can say that the optimum way of actuators required moves correction is correcting all the actuators moves proportionally to the worst case actuator limit exceeding. In that way we can loose only the module of the simulated acceleration. 2.3.3. Identification of the platform real position It looks like the obvious step, but it is not taken into consideration during the simulator platform motion 56

Articles

VOLUME 2,

N° 4

2008

control algorithm analysis. Usually it is taken implicit assumption, that platform is able to perform any required motion. Unfortunately, in the real system that assumption cannot be accepted. In the real system one can meet loosing or distorting an actuator control signal. Sometimes one can also meet unexpected delays of the data transmission in the simulator computer system, which could destabilize the work of the motion system. For those reasons we need to check at every control step the real position of each actuator. If it is different than required one, the proper correction value needs to be added to the next step control signal. Of course, such corrected signal has to be subject of the corrections described above (par. 9 and 10).

3. Filter synthesis evaluation 3.1. Evaluation method During many years of simulators exploitation different methods of their evaluation were used. The best example for that are the flight simulators, as only for them there are existing formal standards and requirements. In the 80-ies and 90-ies years of the previous century there was a trend for developing the strict definition of the flight simulator quality with the use of a set of measurable, objective parameters, e.g. [4] Very often the final result of such an evaluation method was flight simulator fulfilling all or the most number of those formal requirements but evaluated by the pilots as “not similar to the real airplane”. The method applied by the author of that paper together with co-operators at the end of 80-ies of XX century, took different attitude. The goal was to get the flight simulator evaluated by the pilots as the “acceptable and similar to the real airplane”. For achieving that goal the simulator evaluation method was developed. The simulator motion system evaluation was performed in three phases: 1. initial validation of the motion platform control filters parameters, with the use of standard methods applied in the control theory, 2. during the company tests simulator was being evaluated by the test pilots or very experienced real device operator, like engineer of the electric locomotive, 3. during final acceptance (state) tests simulator was being evaluated by the pilots flying routinely, every day the simulated type of the airplane (also applies to other simulated objects types). That method appeared to be very effective way of calibration of all the simulator parameters, particularly simulator motion system. The result was the motion system control filters with “optimized” set of parameters, allowing for the simulating of the whole flight. envelope of the simulated object. Also the simulator itself was accepted by the end users as an “acceptable and similar to the real airplane”. As the example some chosen results of the “Iapetus” flight simulator tests will be presented in the paper. That simulator is still in use at the Military Institute of Aviation Medicine in Warsaw, as the research flight simulator. That

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

simulator imitates two engines, subsonic jet trainer airplane. For the simulator motion system tests the piloting tasks important from the training and critical from the platform motion control point of view, were adopted. They were the following: 1. turns: left next right stable turn with slow ascending and stable velocity, 2. loop, 3. rolls: slow, controlled rolls into both directions, 4. Take off. Filter parameters were calibrated initially for each of the task during the first validation phase, next some “optimization” among their values was performed and then during the company tests the real test pilots verified those values. The final values of the filter parameters were achieved with the help of the “mono-type” pilots of the simulated airplane. The set of input data, with the motion platform control filter parameters values checked and changed during the tests is presented in the Fig. 4.

N° 4

2008

the simulated flights. The example results of registered simulated motion flights show some substantial control mistakes made by the pilots. They allow for analysis of representing the accelerations by the simulator motion system during the complex phases of simulated flight. Those piloting mistakes simplify that analysis, because they produce the substantial values of cross components in combined control channels, confirming also the correctness of their action. They also show, that after loop completing pilot started to induce roll oscillations, which were caused by the incorrect performance of the loop itself. The full set of results showing the complexity of controlling of the simulator platform motion and therefore platform motion system synthesis is presented in Ref. 2. Platform displacements - LOOP

Fig. 4. Example input data set of filter parameters. Achieving such set of “optimized” values of control filter parameters required extensive tests to be done. The input data set “optimal” for one task appeared to be very “non optimal” for the other tasks. Achieving really good set of those values without the real human-operator cooperation would be very long, frustrating and ineffective way of performing that task. Simulator motion platform controls filter parameters values achieved during the company tests, with the test pilots participation were almost fully accepted by the mono type pilots. During that final tests phase only some slight changes into those values were introduced. But that final stage of test cannot be avoided. During that phase some changes of the simulator elements could be done, and then some additional calibrating of the filter parameters values would be necessary. Some example integrated results of the platform motion are presented in the Fig. 5; more results one can find in Ref. [2]. As the reference real airplane data, the flights simulated on the “Iapetus” simulator with the motion platform turned off were used. Of course, having the proper data from the real airplane flights would be the best, but acquiring them was impossible for such broad extent of the required data, because of the financial reasons. So there was taken the decision to use as a reference the data from flights on the simulator with the motion platform turned off. Those reference flights were chosen by the experienced pilots and researchers as the most similar to the ones, which would be performed on the real airplane in the environment conditions similar to

Fig. 5. Example results of loop simulation.

4. Conclusions Anthropocentric method of flight simulator motion system control law synthesis presented in the paper allows for effective development and implementation of such system into the simulators of different types of objects, not only airplane or helicopter. It was verified in practice during development, manufacturing, integrating and testing of the following simulators: - research flight simulator “Iapetus” of the subsonic jet trainer airplane, - mission simulator of the supersonic, swept wing fighter-bomber airplane Su-22, - electric locomotive EP-09, - mission simulator of the medium, multi purpose helicopter W-3WA “Sokol”. Above simulators represent the moving objects with broad range of dynamic properties, starting from electric locomotive able to pull heavy train or make “solo” ride, through subsonic jet airplane, and supersonic, high manoeuvrable airplane, ending at helicopter. For those all simulators their motion systems developed with the use of presented method were successfully tested and evaluated. Also their multi year exploitation proved their correctness, i.e. the human-operators control habits learned at the simulators were easily transferred into the controlling of the real objects. Articles

Journal of Automation, Mobile Robotics & Intelligent Systems

Also taking the acceleration derivative as input signals into the filter equations appeared to be the correct decision. It caused the minimization of the platform with the mounted on it simulator modules inertia influence on the quality of generated motion cues. Adopted structure of the control laws was positively verified for the all listed above simulators, imitating dynamically so different objects. Motion cues generated by those control laws were accepted by the humanoperators of all the simulated objects, and those laws differ only with the values of their parameters. The presented method of simulator motion system control law synthesis appeared to be the effective engineering tool for the developing, testing and validating of the simulator motion systems of any type of the dynamic object, which can be controlled by the human-operator.

AUTHOR Cezary J. Szczepański, D.Sc. Eng. - Associated Professor at Military University of Technology, ul. Kaliskiego 2, 00-908 Warsaw. E-mail: cjsz@poczta.onet.pl.

References [1]

[2]

[3]

[4]

Heintzman R.J., Determination of Force Cueing Requirements for Tactical Combat Flight Training Devices, Training Systems Product Group, Aeronautical Systems Center, Air Force Material Command, ASC-TR-97-5001, Wright Patterson AFB 1997, OH USA. Szczepański C., „Antropocentryczne systemy sterowania ruchem symulatorów” (Anthropocentric Control Systems of Simulator Motion), Prace Naukowe ITWL, no. 19, 2005, Warsaw, pp. 233 and furt. (in Polish with English abstract). Parrish R.V., Dieudonne J.E., Bowles R.L., Martin D.J.: Coordinated Adaptive Washout for Motion Simulators. Journal of Aircraft, vol. 12, no. 1, January 1975. Blackwood M.I., Airplane Flight Simulator Evaluation Handbook. International Standards for the Qualification of Airplane Flight Simulators, Hughes Redifussion Simulation Ltd., March 1993.

Articles

VOLUME 2,

N° 4

2008

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N° 4

2008

METHODS FOR CLASSIFICATION OF TACTILE PATTERNS IN GENERAL POSITION BY HELMHOLTZ'S EQUATION Jaromír Volf, Martin Dvorák, Josef Vlcek

Abstract: This paper describes new and original methods of tactile pattern recognition in general position applying the solution of Helmholtz's equation for a tactile transducer. Three groups of methods have been formed, based on: (a) calculation of the A matrix eigen value, with the matrix being formed either from the whole pattern, or from the limit points; (b) the scalar characteristic distribution of the components of the pattern's A matrix; (c) the geometrical properties of the A matrix of the pattern. The patterns have been classified into five groups. Keywords: pattern recognition, Helmholtz´s equation, tactile image, tactile information.

where l - eigen value of continuous pattern, F - pressure function on transducer area that F¹0. If the pattern is discretized by the sensor or transducers, Helmholtz's equation has to be solved in differential form. The continuous Laplace operator is transformed into the five-point one

or the nine-point one

1. Introduction The ability to make right decision and the orientation of intelligent robot in given surroundings are connected directly to the efficiency of the sensor system and the ability to process the obtained information. Here the processing of the tactile information is of major importance. Several processing methods may be used. This paper deals with the processing of two-dimensional tactile patterns by solving Helmholtz's equation to create the tactile image that is in general position. New methods of processing the primary tactile information can be used for the tactile transducer treated. Because of their universality, these methods can also be applied to other types of tactile transducers. In [6] solution of pattern recognition by basic pattern position is described. Then in this paper by general position.

2. Solution of the Helmholtz's equation If we consider the part of the tactile sensor that comes into contact with the object as a diaphragm deformed by pressure, where a definite function determines the pressure distribution on the diaphragm, the methods for computing elastic diaphragms may be analogously used for determining the tactile pattern. The use of Helmholtz's equation, a special type of general partial differential equation, seems to be especially advantageous. Let us identify the pattern limit by the border C and the internal pattern zone by R. There Helmoltz's equation may be written in the form: DF+lF=0 in R F=0 in C

where h - the separation of the sensor centers, I, J - the coordinates of the sensor centers. The matrix A characterizes the tactile pattern and is formed from the appropriate differential Laplace operator. The Helmholtz equation changes into the form:

which is solved in form: det(LE-A)=0 where h lh L A -

distance between sensors centre. eigen value of discretized pattern. minimum eigen value of matrix A. pattern matrix type NxN. vector of dimension N representing value of function F.

Three groups of methods are formed, based upon: 1. the computation of the eigen value of the matrix A: a) the matrix A was formed for the whole pattern; b) the matrix A was produced from the ground points after the internal points are filtered by use of the fiveor nine-point Laplace operator; Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

2. the scalar characteristic distribution of the components of matrix A of the tactile pattern; 3. the geometric properties of the matrix A. For all cases five pattern groups were selected: circles, rectangles, squares, rectangle triangles and isosceles triangles.

3. Methods using the eigen value of the matrix A The plane is considered to be the pattern space. Therefore two features are necessary to indicate the tactile pattern. The first is the minimum eigen value L of the matrix A; the other must be carefully selected. If we take into consideration that the eigen values depend upon the pattern area, the latter may be that area 2 which unit is h . The dependence of L5 (the minimum eigen value for the matrix A made by the five-point Laplace operator) upon the pattern area is the same for all patterns. Another feature may be the number N of activated sensors 'inside' the discretized pattern (in the area Rhr of the discretized pattern, see Fig. 1). Feature L5 or L9 (the minimum eigen value for the matrix A formed by the nine-point Laplace operator) may be chosen. In both cases the basic pattern groups are distinguishable. Kharkevich [4] has proved that the edges of the pattern bring more information than its area. This has also been proved in [5]. With this knowledge, the number of activated sensors P0 (black points) at the border Chr5 of the area Rhr (see Fig. 2) is chosen as the second feature. This border is computed from the matrix A formed by the five-point Laplace operator.

N째 4

2008

The matrix A can be formed by all points of the tactile pattern. Further improvement is reached if the inside points of the pattern are filtered off and the matrix A is formed from the five- or nine-point Laplace operator for border points only. These are obtained by applying both the operators mentioned. Different combinations of appropriate numbers and sensor numbers at the border may be formed.

4. Method using the scalar characteristic distribution of the components of matrix A If we start from the knowledge that the vertices and edges are the points containing the most information and that the inside pattern points bring less information, another method can be formulated. For the term "information brought by the pattern", the following relation may be presented by formula:

where a(i,j) N -

elements of matrix A, pattern sensor numbers (order of matrix A), coefficient of pattern ability.

This relation satisfies the condition presented in the introduction of Section 3. If we want to accent vertices and edges we use formula:

The information measure in the pattern is expressed by

where k -

the Laplace operator used for forming the matrix A, the norm coefficient of pattern ability.

5. Methods using the geometric properties of the matrix A Fig. 1. The discrete pattern of a circle.

Fig. 2. Border P05 (black points) of the tactile pattern of a circle. 60

Articles

The operator being the representation that in our case transforms the matrix of activated sensors to the matrix A, the geometric properties of this matrix may be investigated. The matrix A is always square and in order to get comparable results for different matrices, let us introduce a normalization, after which the matrix area will have unity dimension. If we suppose the elements of matrix A are the points in a plane, then after normalization the element with the coordinates (i,j) is transformed to (ii,jj). The normalization is given by the relations.

Considering that the matrix A is symmetrical, which is a disadvantage in this case, because its centrum of gravity

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

will always lie at the main diagonal, its lower triangular part will be used. The centrum of gravity coordinates may be expressed in two ways: in rectangular Cartesian coordinates T(x,y) or in polar coordinates T(V,a), where V is the distance from the origin of coordinates (ii, jj) and a is the angle between the axis jj and the connecting line between the origin and the centrum of gravity T (see Fig. 3). The elements of matrix A act as weighting coefficients during the computation. The centrum of gravity coordinates in the Cartesian system are determined according to the relations:

where N No

N° 4

2008

1. Determine pattern centrum of gravity:

where Lxi Lyi

distance i - point from axis y thorough coordinate basic origin, distance i - point from axis x thorough coordinate basic origin, activation sensors number.

2. Pattern completion by fictive points By computing and the pattern discretization, the various abnormality and teeth rise on patterns border. Then we insert other fictive points between original points that present active tactile sensors. By this way we increase texture and increase expressively rotate pattern quality. On Fig. 4 triplet of adjacent points (they are big and represent active tactile sensors) and created rectangle isosceles triangle we complete by next 18 points. the activated sensor number in the pattern area Rhr (dim A = N), elements number of down triangle normalized matrix A.

For polar coordinates we get

By analogy with the foregoing cases, the matrix A may also be formed from the five- or nine-point Laplace operator, for either all the activated sensors or for the reduced matrix. The individual Cartesian coordinates of the polar centrum of gravity T in combination with a number P05 of activated sensors may be selected for marks.

Fig. 4. Pattern completing by fictive points. 3. Sequential rotation of all pattern points In last step cycle is executed, in which all pattern points are rotated (real and fictive) around its centrum of gravity step by step about 0, 4, 8, ..., 180°. We calculate features in each from these angles. E. q. for each feature we obtain (180°/4°) + 1 = 46 values. From these values we used only minimal, maximal or difference between maximal and minimal feature value. After rotation calculation we decrease texture for next processing (calculation features). For next calculation we keep only points, which fulfill condition: x mod 5 = 0 where x, y mod -

Fig. 3. The centrum of gravity of the normalized matrix A.

6. Way of making rotation patterns Experimentally has been chosen pattern rotation about 4° in interval 180°. Self pattern rotation is executed by next rules:

together

y mod 5 = 0

are points coordinates (real and fictive) in are tactile sensor, residue after division (e.g. 6 mod 5 = 1).

From pattern with decrease texture we calculate all features and then we make next rotation about 4°. Analogously as irregularities and teeth rise on the pattern border by the pattern rotation from computer calculation, than holes rise inside of pattern from the same reason. Than we have to make three rules for correction turning pattern. Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

1. rule - if in axis y of tactile sensor area rises situation sensor - space - sensor then we insert sensor instead the space; 2. rule - if in axis x of tactile sensor area rises situation sensor - space - sensor then we insert sensor instead the space; 3. rule - if in diagonal of tactile sensor area rises situation sensor - space - sensor then we insert sensor instead the space.

N° 4

2008

For selecting of equilateral triangles from circles and squares the dependence min It5 = f(min P05). This dependence shows Fig. 7. The pattern classes of circles and squares are in area under curve and the pattern classes of triangles are up curve.

Using of these rules shows Fig. 5 Rule 1

Rule 2

X X - => X X X

X - X => X X X Fig. 7. Features dependence min It5 = f(min P05).

Rule 3 X

X -

=> X

X X

or X

X =>

X X

Fig. 5. Rules for holes correction in pattern.

7. Classification and the used features For classification we used net features. All were defined up. a9 - polar coordinate of centrum of gravity of normalize matrix A, - number of activate sensors in area Rhr, N - scalar characteristic distribution of the compoIt5 nents of matrix A, - number of active sensors on border Rhr, border P05 is created by 5 point Laplace´s operator.

By combination of up procedures we can select four pattern classes: rectangle triangles, equilateral triangles, rectangles and last group create squares and circles. For differentiation of circles from squares we used dependence dif It5 = f(difP05), see Fig. 8. By three features dependences (Fig. 6 to Fig. 8) we can right classify five pattern classes by probability dependent from inside sensors N. For inside sensors number N³35 is right pattern recognition more as 90 %, for N³100 is right pattern recognition about 99,9 %. Minimal size of pattern is N=20. In this case is right pattern recognition about 70%.

In front of these features we will write symbols min, max, dif. Min or max present minimal or maximal features value through its turn. Dif presents difference between maximal and minimal value. In first step, by classification was used, features dependence dif a9 = f(N). This dependence shows Fig. 6. Fig. 8. Features dependence dif It5 = f(min P05).

8. Conclusion This paper has dealt with the new and original methods for processing tactile information and distinguishing tactile patterns. These methods are based on the solution of Helmholtz's differential equation in discrete form.

ACKNOWLEDGMENTS This research has been supported by Research Programme MSM6840770015.

Fig. 6. Features dependence dif a9 = f(N). By two curves we select the graph to three parts. To area 1 the rectangle pattern class belongs, to area 2 rectangle triangles pattern class belongs and to area 3 remaining patterns classes (circles, squares, equilateral triangles). 62

Articles

AUTHORS Jaromír Volf*, Martin Dvorák, Josef Vlcek - Faculty of Mechanical Engineering, CTU in Prague, Technická 4, 166 07 Prague 6. E-mail: jaromir.volf@fs.cvut.cz. * Corresponding author

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 2,

N째 4

2008

References [1] [2] [3] [4] [5]

[6]

Dyga R., Chart P., Raspoznavanije obrazov i analyz scen, Mir: Moscow 1976. Forsythe G.E., Wasow W.R., Finite Difference Methods for Partial Differences Equations, Wiley: New York, 1967. Pugh A., Robot Sensors, vol. 2, Tactile and Nonvision Sensors, Springer, Berlin, 1986. Kharkevich A.A., "O cennosti informacii", Problemy Kibernetiky, no.4, 1960, pp. 53-57. Kanajev E.M. and Shnejder A.J., Principy postrojenija osjazatelnych racpoznajuscich ustrojstv, Mechanika Masin, Nauka, Moscow, 1974, pp. 29-32. Volf J., "Methods of Processing Tactile Information based on the Solution of Helmholtz's Equation", Sensors and Actuators A. Physical, vol 41, 1994, no. 1-3, ELSEVIER-SEQUOIA S.A.: Lausanne, Switzerland, SSDI 0924-4247(93)00648-N, pp. 174-179.

Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N째 4

2008

A HYBRID SYSTEM CONTROL APPROACH TO BIPED ROBOT CONTROL Yingjie Yin, Shigeyuki Hosoe

Abstract: Human being can decide how to auto-adapt his motion to environmental changes. Comparing with this, the automation of the motion of biped robots is still very inefficient. In this paper, we review the state of the art of biped motion control in the field of hybrid system control. The main motivation is to fix the understanding of the research field and clarify strong and weak points of the available approaches. The presentation is illustrated by a typical example of biped machines. Keywords: biped robot, MLD model, periodic motion, hybrid system control.

1. Introduction The motion of a biped robot is naturally described as a hybrid dynamical system. Its behaviour is determined by interacting continuous and discrete dynamics. Just like living systems change posture (such as running, walking, balancing) variously according to the purpose of work, control also changes according to it. However, the problems of hybrid dynamical systems are inherently difficult because of their combinatorial nature. In recent years, the control problems of biped motions have received considerable attention. The systems are tried to design from the hybrid system point of view. However, the understanding of hybrid systems is rather limited at present, and most of proposed approaches are schemes based on heuristic rules inferred from practical plant operation. New paradigm for the modelling of biped motions is required and systematic approaches for the synthesis of the biped system are expected. An important control objective for biped locomotion systems is to generate reference motion trajectories that are consistent with underlying principles of hybrid movement, and then control the robot to track it. In this trend, a series of papers have been published (see [9], [11], [12], and [13]), where the periodic orbits of the bipeds are approximated by polynomials for numerical simplicity, and numerical algorithms are proposed to generate the trajectories. There are designed feedback controllers that enforce the biped robots to track the pre-planned trajectories. The difficulties of this kind of approaches are that the movements are non-adaptable to environmental changes, and unexpected events are not pre-computable. Besides, the polynomial approximation of the joint trajectory may be relatively rough because of the dimension. In this meaning, despite the huge amount of works on the topic, walking/running control of biped robots is still inefficient especially in terms of stability, adapta64

Articles

bility and robustness. In contrast with this, human being can decide to auto-adapt his walking/running to environmental changes rather than predicting it a very long time ahead. A consideration inspired by human behaviour is to introduce the path changes on-line when stability and motion in environment are guaranteed. Towards this direction, in [2], the zero moment point (ZMP) was controlled for preserving the dynamic equilibrium. In [4], a trajectory, better adapted to the encountered situation, was chosen on-line amongst the sets of stored trajectories. In [10], a continuous set of parameterized trajectories was used as the candidate of choice. In [7], [8], [17], by optimizing the joint motion over a receding control horizon, biped robots are controlled without pre-computed reference trajectory such that the approaches are adaptable to the environment changes. In this paper we summarize some of the currently hybrid control ideas for the motion control of biped robots. The main motivation is to fix understanding of the research field and clarify strong and weak points of the available approaches. In the consideration of motion planning then control, we propose that a hybrid external system can be used for producing a periodic locomotion pattern of running instead of the polynomial approximation. In the consideration of on-line motion control without predescribed trajectory, our main proposal is to express the biped motion as a unified modelling from the point of view of the hybrid system, which is called the mixed logic dynamical (MLD) model. Based on the MLD model, the motion planning problem of biped motion is formulated as an optimal control problem where the change of discrete configurations defines gait patterns. Finally, we conclude the proposal and promote some potential trends from the aspects of system control theory for the biped motion control.

2. Modelling of the biped motion Consider a typical biped system shown in Fig.1. Motions are assumed to take place in the sagittal plane, consist of successive phases of single support, flight, and collision event without slip.

Fig. 1. A typical biped robot with three links.

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N째 4

2008

A. Dynamical equations of flight and stance In the flight phase, the dynamical model is obtained by the method of Lagrange as the models of (1) and (2) along the state transitions of (3) and (4) can be expressed as (1) (5) where qf = [q xcm ycm] is the generalized coordinates in T the flight phase, with q = [q1 q2 q3] the joint angles, and (xcm , ycm) the centre position of the mass. qb = [q1 q2]T is T the joint angles actuated in the flight phase, tf = [t1 t2] is the applied torque. m0 is the mass of the biped. Note that t3 is not applicable in flight, thus the joint q3 is under-actuated. T

In the stance phase, the end of the stance leg is fixed and the robot is fully actuated. The dynamical equation of stance is described as (2) where

B. Motion transitions An impact occurs when the advancing leg touches the ground, i.e. y2 = 0. Under the assumption of inelastic impact, the post-impact velocities of angular joints are discontinous,

(6) where

are the hyper-surfaces of motion switching, .

(5) and (6) formulate a hybrid system whose discontinuous behaviour is caused by a motion transition. D. A mixed logic dynamical model The hybrid system of biped robots can be represented as a mixed logic dynamical (MLD) model by introducing logic auxiliary variable , to associate the events of motion transition[17], (7) By (7), the system of (5), (6) can be equivalently transformed to the following relations by the similar way as in [17], (8)

where is the initial value of in the stance phase, is the terminal value of in the flight phase, respectively. is the terminal value of the generalized coordinates in the flight phase. The roles of the two legs are swapped after impact,

where is the initial value of in the stance phase. Therefore, the state transition from flight to stance is

where , . is the continuous auxiliary variable introduced for variable linearization. The details of the coefficient matrices are omitted here. The MLD model of (8) represents both the continuous motions and the discrete events in a unified framework, allows the synthesis of the hybrid system in a systematic way.

3. Progressing constraint (3) This transition takes place in an infinitesimal small length of time, hence there exists no double support phase. On the other hand, the transition from stance to flight is initiated by accelerating the stance leg off the ground. Suppose the stance-to-flight transition occurs at a pre-determined state where . The transition is continuous in position and velocity,

(4)

where C. State equation expression Introducing the state vector

A successful walking/running should be a stable and successive progress forward. For that, conditions of stable and successive walking/running have to be taken into account as constraints subject to the hybrid dynamical equations. A. Erected body The movable range of joint 1 is limited, (9) which results in the hip position of the robot above a positive level to avoid falling. B. Progression For propel the body in the intended direction with a progression rhythm, horizontal velocity of the swing toe should be kept positive. For that, we set (10) Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

where

N째 4

2008

and

C. Environment For walking/running on non-smooth ground, the surface is supposed to be known, expressed by a set of mathematical inequalities, (11) Then the impact occurrence condition becomes . In addition, the constraints of , have to be added for generating the biped motion. (9)~(11) can be included into the constraint inequality of (8). The resulted MLD model becomes a unified description for both the physical phenomena and the control constraints.

4. Motion control of biped robots by trajectory tracking The motion control of biped robots is frequently performed by tracking a pre-defined periodic orbit. For that, the problem is generally separated into two steps: (i) the synthesis of motion patterns[5], [9], [6] and (ii) the control of the robot to track the prescribed trajectory [12], [11]. The problems of these approaches are that the movements are non-adaptable to environment changes, and unexpected events are not pre-computable. A. The periodic orbit of running Denoted by , the periodic orbit of running consists of trajectories of the stance motion and the flight motion with impulse effect. The desired path of the stance motion is , which initiates from and terminates at . Since the takeoff of robots is continuous,

. Consequently, the duration of the flight phase and the progressing distance of one step in running is also specified. The curves of and which connect at the previous terminates can be chosen optimally by ensuring the control constraints of (9)~(11) while satisfying the boundary conditions of (12) and (13). In [9], [11], [12], [13], the and are approximated by a set of polynomials for numerical simplicity. Here we remark that the paths of and can also be considered as the trajectories generated by a hybrid external dynamical system by the technique of [1], [16]. The trajectories can be produced according to some theoretical instruction, and they can be much smooth including rich information. The results on this topic will be reported in our coming publication. B. Trajectory tracking For tracking the pre-planned periodic orbit, the controller design is accomplished by asymptotically regulating the tracking errors , to zero, (14) where is the first two elements of for the 3-link biped. The twice time derivation of along (6) is (15) The controller of (16) results (15) to be

(12) where is the initial of the flight phase. The position of the centre of mass at the takeoff point can be calculated by the terminal state of stance,

The desired path of the flight phase is which initiates from and terminates at . The ending of flight triggers the (next) initial of stance by impact event,

is used to transform the equation (5) to a normal form,

(13)

(17)

where , can be obtained by the time derivation of the following collision condition.

(18)

Therefore, the periodic orbit is a closed-route consisting of two curves of and . A pre-defined terminates of the stance phase, and , determine the terminates of the flight phase, 66

Then a finite-time control law of renders to zeros with finite time before takeoff. On the other hand, biped systems are underactuated in the flight phase. For the convenience of synthesis, a coordinate transformation

Articles

(19) (20)

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

For the 3-link robot shown in Fig1, the joint is unactuated in the flight phase. Thus , can be chosen to be

(22) is the 3rd row of the inertia matrix It can be confirmed by (1) that

(23) (22) also implies that

(24)

By (21), (23), (24), and (1), the equations of (19), (20) become

(25)

Note that (25) is the internal dynamics of the biped system in the flight phase, which can not be directly regulated by . By the coordinate transformation, the tracking error becomes , and it's twice time derivation is (18), i.e.,

For the tracking control of the flight motion, the controller of (26) results in . A PD control of exponentially at zeros,

will stabilize

Note that the value of is required for the generation of in (26). is the internal variable vector evolving along the internal dynamics of (25), and depends on the initial state of the flight phase. Concluding the previous discussion, we have the following results. Theorem: Suppose the biped robot is fully actuated in the stance phase, and a periodic orbit is pre-defined for the biped running. If the matrix and are invertible upon the periodic orbit, then the stance motion controller (16) and the flight motion controller (26) result in asymptotical tracking of the biped motion to the pre-defined periodic orbit, if and only if the settle

2008

time of the finite-time stabilizing feedback controller is shorter than the duration of the stance phase. The zero-error manifold of the hybrid system is

(21)

where is the angular momentum of the biped about its centre of mass,

N째 4

(27)

5. Motion control of biped robots without pre-defined trajectory Human being can decide his walking/running to autoadapt to environment changes rather than predicting it a very long time ahead. Inspired by this, an important control objective for biped locomotion systems is to determine the motion trajectory online for current state and changed environment, while guarantee the stable motion in environment. The advantage of motion control without the using of predicted trajectory is the adaptability of the motion to environment. The control problem of on-line walking adaptation has been studied by some researchers. In [2], [3], the zero moment point (ZMP) was controlled for preserving the dynamic equilibrium. However, this adaptation can only involve small changes of the original trajectory. In [4], a trajectory better adapted to the encountered situation was chosen on-line amongst the sets of stored trajectories. However, the switching from one trajectory to another may lead to unexpected effects in control. To cope with this problem, in [10], a continuous set of parameterized trajectories was used as the candidate of choice. The switches were suppressed, but the set of trajectories has to be defined beforehand. A consideration different with the previous approaches is to adapt the model predictive control (MPC) to the on-line walking adaptation. MPC is a control method for the problem of multivariable systems that are constrained in the state and/or control variables. In the past years, MPC is well applicated in refining or chemical industry, but few works are dedicated to process with short time response such as robotic system. In fact, for the problem of walking pattern synthesis of biped robot subject to unilateral constraints or disturbances due to an unstructured environment, MPC method is suitable and some experience has been obtained in [7], [8], and [17]. A. The model predictive control of biped walking In [7], the criterion used for minimization in the model predictive control (MPC) can be (28)

where

is the reference state where impact occurs. is the sampled state which can be predicted by the dynamics of (6) from the current state and applied control. denotes the sampling time. In the MPC approach, the future control inputs , are calculated which minimize the criterion of (28). Then, only is applied to the biped robot for the next sampling time which results in the updated state . The time is shifted from to , the length of the optimization horizon is Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

shortened to in [7], and the process of optimization is repeated. The approach [7] avoided to take into account the impact as an interior point of optimization horizon. Therefore, the effect of impact cannot be treated positively. Furthermore, the walking adaptability to environment is poor because it shortens the prediction horizon each iteration until to . In [8], the optimization horizon is kept constant. However, how to predict the occurrence of impact then compensate positively for the effect of impact by the MPC approach is not stated clearly. B. The MPC of biped walking based on the MLD model Remind that the MLD model (8) for the biped motion encapsulates phases of continuous motion, switching between types of motion, occurrence of impacts, and represents all of the features in a unified model. Therefore, based on the MLD model, the synthesis of a biped motion can be carried out in a systematic way; meanwhile the effect of impact can be taken into well account. According to this consideration, the criterion for minimization is as the following in [17]. (29)

N° 4

2008

6. Conclusions A review of the state of the art in the field of hybrid system control of biped robot has been undertaken in this paper. Despite the huge amount of works on the motion control of biped robot, the problem is still inefficient especially in terms of stability, adaptability and robustness. On the other hand, hybrid systems represent a highly challenging research area which has a lot of application in robotics. However, straight-forward application of available frameworks faces the limit of computational complexity and lucks the theoretical prediction of system properties. In this connection, approaches based on application of MLD look more attractive. Also, a human uses his predictive function based on an internal model together with his feedback function for motion, which is considered as a motor control model of a cerebellum [15]. Stimulated by this, a general theoretical studies for motion control of hybrid systems are reported in [16], [14] which are based on the MLD model or the piece-wise linear system model. Further developing this kind of theory will be useful for the realization of complex motion of bio-mimetic robots. Finally, we mention that biologically-inspired solution, as those based on the reinforcement learning paradigm, can be of high potential if the appropriate minimal simulation models can be constructed and validated. ACKNOWLEDGMENTS This work is supported by the Grants-in-Aid for Scientific Reserach of Monbu Kagakusho, Japan.

which is subject to the MLD model of (8). is the horizon for optimization and is kept to be constant. are weighting matrices. is the future state of predicted by the MLD model. is set as a desired state of pre-impact, which can be time-varying or time invariant. and are the desired auxiliary continuous and logical states corresponding to . In the case that an impact occurs within the horizon, the minimization of (29) implies that the state before the impact point is regulated to approach the pre-impact state, and the state after the impact point is controlled towards the impact of the next step. The optimal control is generally effective for impact occurred either within or outside the optimization horizon . The second term at the right side of (29) implies the minimal input control. The optimization of (29) is carried out over a receded horizon . The biped robot is controlled with neither precomputed reference trajectory nor switches at the algorithm level. Thus the resulted walking can easily autoadapts to environment changes. The MPC of the MLD system can be solved using powerful mixed integer quadric programming (MIQP) algorithm. Its solution corresponds to the objective-oriented optimization of the gaits. Simulation results are reported in [17], which show that the MLD model based MPC approach leaded to faster walking with smaller torque. Theoretically, the MLD model based MPC approach is also effective for the synthesis of the biped running. Numerical test is required to confirm its validation.

Articles

AUTHORS Yingjie Yin - Measurement & Instrumentation Engineering Div.5, Toyota Technical Development Corporation, 1-21 Imae, Hanamoto-cho, Toyota, Aichi, 470-0334 Japan. E-mail: yingjie.yin@mail.toyota-td.jp. Shigeyuki Hosoe - RIKEN-TRI Collaboration Center for Human-Interactive Robot Research, the Institute of Physical and Chemical Research (RIKEN), Moriyama-ku, Nagoya, 463-0003 Japan. E-mail: hosoe@bmc.riken.jp.

References [1] [2]

[3]

[4]

[5]

A. Isidori, Nonlinear Control Systems, 3nd ed., Springer, 1996. J. Park, H. Chung, ”Zmp compensation by on-line trajectory generation for biped robots”. In: Proc. of the IEEE Conf. on Systems, Man and Cybernetics (SMC99), 1999, pp. 960-965. J.H. Park, H. Chung, ”Hybrid control of biped robots to increase stability in locomotion”, Journal of Robotic Systems, vol. 17, no.4, 2000, pp. 187-197. J. Denk and G. Schmidt, ”Walking primitive synthesis for an anthropomorphic biped using optimal control techniques”. In: Proc. of the Inter. Conf. on Climbing and Walking Robots (CLAWAR), Karlsruhe, Germany, September 2001, pp. 819-826. C. Chevallereau, A. Formalsky, B. Perrin, ”Low energy cost reference trajectories for a biped robot”. In: Proceedings of the IEEE Inter. Conf. on Robotics and Automation (ICRA), May 1998, pp. 1398-1404.

Journal of Automation, Mobile Robotics & Intelligent Systems

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

VOLUME 2,

N° 4

2008

C.M. Chew, G.A. Pratt, ”A general control architecture for dynamic bipedal walking”. In: Proceedings of the IEEE Inter. Conf. on Robotics and Automation (ICRA), vol.4, San Francisco, CA, pp. 3989-3995, April 2000. H. van der Kooij, R. Jacobs, B.Koopman, F.van der Helm, ”An alternative approach to synthesizing bipedal walking”, Biological Cybernetics, vol. 88, 2003, pp. 46-59. C. Azevedo, P. Poignet, and B. Espiau, ”Artificial locomotion control: from human to robots”, Robotics and Autonomous Systems, vol. 47, 2004, pp. 203-223. C. Chevallereau and Y. Aoustin, ”Optimal reference trajectories for walking and running of a biped robot”, Robotica, vol. 19, 2001, pp. 557-569. C. Chevallereau and P. Sardain, ”Design and actuation optimization of a 4 axes biped robot for walking and running”. In: Proc. of the IEEE Inter. Conf. on Robotics and Automation (ICRA), San Francisco, CA, April 2000, pp. 3365-3370. J. W. Grizzle, G. Abba, and F. Plestan, ”Asymptotically stable walking for biped robots: Analysis via systems with impulse effects”, In: IEEE Trans. on Automatic Control, vol. 46, Jan. 2001, pp. 51-64. E. R. Westervelt, J. W. Grizzle, and D. E. Koditschek, ”Hybrid Zero Dynamics of Planar Biped Walkers”, In: IEEE Trans. on Automatic Control, vol. 48, Jan. 2003, pp. 42-56. C. Chevallereau, E.R. Westervelt, and J.W. Grizzle, ”Asymptotically Stable Running for a Five-Link, FourActuator, Planar Bipedal Robot”, Inter. Journal of Robotics Research, vol. 24, June 2005, pp. 431-464. E. Muramatsu, K. Watanabe, ”The stability condition of the 2-order piecewise linear systems based on the eigen value and eigen vector”, In: Trans. of the Institute of Systems, Control, and Information Engineers, vol. 18, 2005, no.2, pp. 70-79. M. Kawato, ”Internal models for motor control and trajectory planning”, Current Opinion in Neurobiology, no. 9, 1999, pp. 718-727. Y.J. Yin, S. Hosoe, ”Tracking Control of Mixed Logic Dynamical Systems”, In: IEICE Trans. on Fundamentals of Electronics, Communications and Computer Sciences, vol. E87-A, no.11, 2004, pp. 2929-2936. Y.J. Yin, S. Hosoe, ”Mixed Logic Dynamical Modeling and On Line Optimal Control of Biped Robot”. In: Proc. of IEEE/RSJ Inter. Conf. on Intelligent Robots and Systems, Oct. 2006, pp. 5895-5900.

Articles

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N° 4

2008

INFocus THE SPOTLIGHT on new n Robot for less than £24 At the first European conference on Artificial Life in Winchester Alexis Johnson from the University of Southampton's School of Electronics and Computer Science (ECS) described how he and his fellow students developed a platform of 25 (£24 each) robots capable of more than two hours of autonomy and with sufficient code capacity and processing power to run complex algorithms. The team employed motors normally used to vibrate mobile phones that are designed to be attached to circuit boards in the standard manufacturing process (by removing the need for manual assembly of the robots and bringing the cost of a swarm of robots within reach of a typical research project). Swarm robotics platforms are used for the investigation of emergent behaviour. They permit the study of swarm behaviour by physical simulation: providing real world constraints and experimental scope unattainable in software simulation alone. Long-term possible applications for swarm robotics are in earthquake scenarios, environmental monitoring, and the field of space science. Source: For further information about ALIFE XI, visit: http://www.alifexi.org/ Based of ScienceDaily.com

n Crash Avoidance Robotic Car Based on joint research with the Research Center for Advanced Science and Technology at University of Tokyo, Nissan has built the Biomimetic Car Robot Drive (BR23C). This inspired by flight of the bumblebee robotic micro-car recreates bee characteristics with the goal of producing a system that prevents collisions altogether. Unveiled at CEATEC 2008 Biomimetic Robot Car is equipped with a prototype collision avoidance system. The next-generation safety technology is modelled after the way that bees avoid crashing into each other. "The BR23C robotic car is positioned as the inner-most layer of this shield. We are expecting that this robotic car will support the development of future collisionavoidance technologies" said Mitsuhiko Yamashita, Executive Vice President in charge of research and development.

Source: http://www.nissan-global.com/EN/NEWS/2008/_STORY/08092601-e.html Nissan BR23C Biomimetic Robot Car 70

In the spotlight

Journal of Automation, Mobile Robotics & Intelligent Systems

VOLUME 2,

N째 4

2008

n Robot in a Maze The robot is named "All Right", because it solves the maze using right turns - writes its creator, David Cook. All Right robot has a very single-minded approach to navigation, powered by LiPoly (lithium polyester) batteries and an assortment of sensors including 9 photo detectors for floor sensing and line following, 4 more photo detectors used as quadrate encoders, a battery voltage sensor and an assortment of pushbuttons. While the robot can turn left when it wants to, it solves line-following mazes by making right turns whenever possible. The robot is approximately 18 cm wide x 15 cm deep x 12 cm tall; it weighs 725 grams.

Source: http://www.robotroom.com/Maze-Solving-Robot-AllRight.html

n Micro Turbines are becoming to reality The University of Maryland's A. James Clark School of Engineering MEMS Sensors and Acturators Lab developed a micro scale pump and turbo generator. In press release researcher Reza Ghodssi wrote, "for the first time, we have achieved a level of miniaturization for machines like that achieved over the last decades in electronics". The tiny turbine achieves rotational speeds of 87,000 RPM, supported on micro-ball bearings so small they are almost invisible to the naked eye. Funding the research, US Army hopes that micro-turbines will replace the 20lbs of bulky lithium ion batteries now carried by soldiers. The tiny generators could also power micro air vehicles (MAV), small UAVs, and other pint-sized robots.

Image: courtesy of University of Maryland

Source: More information, including a paper design, fabrication, and characterization of a Rotary Micromotor Supported on Microball Bearings and video of the turbine in action at: http://www.eng.umd.edu/media/pressreleases/pr092208_bearings.html

In the spotlight

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N° 4

2008

EVENTS WINTER 2008

November 17 – 19

ICERI 2008 – International Conference of Education, Research and Innovation, Madrid, Spain http://www.iated.org/iceri2008

December th

01 – 04

26 Army Science Conference, Orlando, Florida, United States. http://www.asc2008.com

01 – 05

21 Australasian Joint Conference on Artificial Intelligence, Auckland, New Zealand. http://www.ai08.org

08 – 10

ICPADS'08 – The 14 IEEE International Conference on Parallel and Distributed Systems Melbourne, Victoria, Australia http://www.deakin.edu.au/conferences/icpads2008/

10 – 13

International Conference on Distributed Computing and Internet Technologies, New Delhi, India. http://www.icdcit08.org

15 – 17

International Conference on Advances in Mechanical Engineering, Surat, Gujarat, India. http://www.svnit.ac.in

National Seminar on Fuzzy Theory and Applications, Selangor, Shah Alam, Malaysia. http://www.tmsk.uitm.edu.my/~nsfta08/

18 – 20

ICOM'08 – The 3rd International Conference on Mechatronics, Cultural Activity Center (CAC) International Islamic University Malaysia Kuala Lumpur, Malaysia http://www.iiu.edu.my/ICOM/2008/

20 – 22

ICACTE 2008 – International Conference on Advanced Computer Theory and Engineering, Phuket, Thailand http://www.icacte.org

DMAI 2008 – IEEE International Workshop on Data Mining and Artificial Intelligence, Khulna, Bangladesh. http://www.cs.mu.oz.au/~mrhassan/workshopindex.html

EVENTS WINTER 2009 - NEXT PAGE

Events

VOLUME 2,

Journal of Automation, Mobile Robotics & Intelligent Systems

N° 4

2008

EVENTS WINTER 2009

January 05 – 06

ICMCS 2009 – International Conference on Mathematics and Computer Science, Chennai, Tamilnadu, India. http://www.icmcs.in

27 – 28

International Conference on Advanced Computer Control, Singapore, Singapore. http://www.icacc.org

28 – 30

CCISE 2009 – International Conference on Computational Intelligent Systems Engineering, Dubai, United Arab Emirates. http://www.waset.org/wcset09/dubai/iccise/

February 25 – 27

ICCASE 2009 – International Conference on Control, Automation and Systems Engineering, Penang, Malaysia. http://www.waset.org/wcset09/penang/iccase/

10 – 12

ICEPAG 2009 – International Colloquium for Environmentally Preferred Advanced Power Generation, Newport Beach, California, United States. http://www.apep.uci.edu/ICEPAG2009

March 07 – 09

International conference on Future Networks, Bangkok, Thailand. www.icfn.org

18 – 20

IMECS 2009 – International MultiConference of Engineers and Computer Scientists 2009, Hong Kong, Hong Kong. http://www.iaeng.org/IMECS2009/

19 – 22

ICICIS 2009 – Fourth International Conference on Intelligent Computing and Information Systems, Cairo, Egypt. http://icicis.edu.eg

Events