Skill memory in biped locomotion Using perceptual information to predict task outcome
1
J. Andre1, C. Santos2, L. Costa3 Department of Industrial Electronics, University of Minho, joaocandre@dei.uminho.pt 2 Department of Industrial Electronics, University of Minho, cristina@dei.uminho.pt 3 Department of Production and System, University of Minho, lac@dps.uminho.pt
Abstract Robots must be able to adapt their motor behavior to unexpected situations in order to safely move among humans. A necessary step is to be able to predict failures, which result in behavior abnormalities and may cause irrecoverable damage to the robot and its surroundings, i.e. humans. In this paper we build a predictive model of sensor traces that enables early failure detection by means of a skill memory. Specifically, we propose an architecture based on a biped locomotion solution with improved robustness due to sensory feedback, and extend the concept of Associative Skill Memories (ASM) to periodic movements by introducing several mechanisms into the training workflow, such as linear interpolation and regression into a Dynamical Motion Primitive (DMP) system such that representation becomes time invariant and easily parameterizable. The failure detection mechanism applies statistical tests to determine the optimal operating conditions. Both training and failure testing were conducted on a DARwIn-OP inside a simulation environment to assess and validate the failure detection system proposed. Results show that the system performance in terms of the compromise between sensitivity and specificity is similar with and without the proposed mechanism, while achieving a significant data size reduction due to the periodic approach taken. Keywords Reinforcement learning · Bio-inspired · Skill Memory
robótica
4
artigo científico
2.ª Parte
condition in equation (16) is satisfied, then the null hypothesis of being a successful trial is not rejected and the sensor data is assumed to be in conformity with the expected values - there are no signs of failure conditions. If, on the other hand, ytrial(n) is out of the confidence bounds established in (16), then there is a high probability of failure occurring, or that movement objective is not achieved at the end of the trial. Whether or not the current trial is flagged as failure depends on the thresholds for failure detection: the minimum number of sensors M and the minimum number of consecutive instants failing N. Simply put, if the system detects at least M sensors failing for N instants consecutively, then it is predicted that, based on the previous experiences stored into the ASM, task execution will fail. 6.1. Detection Accuracy Failure detection was interpreted as a simple two-class classification problem, characterized by the sensitivity, considered to be the probability of detecting a failure on unsuccessful trials and, similarly, by a specificity value, the probability of rejecting failures on successful trials. Conclusions about the performance of the failure detection algorithm were based on a detection score computed from sensitivity and specificity values: (17)
6. FAILURE DETECTION In order to properly take advantage of the information stored into the ASM, we propose a system that monitors continuously the execution of a motor skill (in this case biped locomotion) and looks for deviations that could evolve into movement failures. The failure detection protocol we introduce in this work was inspired by Pastor et al. [20], but utilizes a more refined statistical analysis in order to achieve the best results possible. Once again we anchor the whole process on the phase values at any given time. At each instant n of the simulation, and , reconstructed from the trained DMPs, provide the ASM values for the correspondent phase (n), upon which a statistical z-test is performed. Thus, a tolerance interval is established for each sensor according to:
which can be interpreted as inversely proportional to the distance to the optimal operation point of maximal (100%) sensitivity and specificity - a higher detection score implies better performance when detecting failure conditions.
robótica 100, 3.o Trimestre de 2015
Figure 4. ROBOTIS DARwIn-OP humanoid robot in Webots.
(16)
where ytrial(n) is the sensor data of the current simulation; (n) is the phase at the current instant n in this trial, z =2.57 for a confidence level of 99%, and and represent the mean and standard deviation values stored into the ASM. If the
6.2. Optimal Parameterization M and N, thresholds for the number of sensors and number of consecutive time steps, respectively, have a significant impact on the performance of the failure detection system. With no a priori knowledge or practical know-how about how the sensor readings vary, it is hard to estimate the optimal values for these
national Journal of Humanoid Robots 8(2), 245–273 (2011); 11. Karssen, J.G.D., Wisse, M.: Fall detection of two-legged walking robots using multi-way principal components analysis. International Journal of Humanoid Robots 7(1), 73–93 (2010); 12. Kim, J., Kim, Y., Lee, J.: A Machine Learning Approach to Falling Detection and Avoidance for Biped Robots. In: SICE Annual Conference, pp. 562–567 (2011); 13. Kunihiro, O., Koji, T., , Yasuo, K.: Falling motion control for humanoid robots while walking. In: 7th IEEE-RAS International Conference on Humanoid Robotics (Humanoids 2007), pp. 306–311 (2007); 14. Matos, V., Santos, C.P.: Central Pattern Generators with Phase Regulation for the Control of Humanoid Locomotion. In: IEEE-RAS International Conference on Humanoid Robots. Osaka, Japan (2012); 15. Ogata, K., Terada, K., , Kuniyoshi, Y.: Real-time selection and generation of fall damage reduction actions for humanoid robots. In: 7th IEEE-RAS International Conference on Humanoid Robotics (Humanoids 2008), pp. 233– 238 (2008); 16. Olson, D., Delen, D.: Advanced Data Mining Techniques. In: Springer-Verlag (2008); 17. Owaki, D., Morikawa, L., Ishiguro, A.: Listen to body’s message: Quadruped robot that fully exploits physical interaction between legs. In: 2012 IEEE/ RSJ International Conference on Intelligent Robots and Systems, pp. 1950– 1955. Ieee, Vilamoura, Portugal (2012); 18. P.-B.Wieber: Viability and predictive control for safe locomotion. In: 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1103–1108 (2008); 19. Pastor, P., Hoffmann, H., Asfour, T., Schaal, S.: Learning and generalization of motor skills by learning from demonstration. 2009 IEEE International Conference on Robotics and Automation pp. 763–768 (2009); 20. Pastor, P., Kalakrishnan, M., Chitta, S., Theodorou, E., Schaal, S.: Skill learning and task outcome prediction for manipulation. 2011 IEEE International Conference on Robotics and Automation pp. 3828–3834 (2011); 21. Pastor, P., Kalakrishnan, M., Meier, F., Stulp, F., Buchli, J., Theodorou, E., Schaal, S.: From dynamic movement primitives to associative skill memories. Robotics and Autonomous Systems 61(4), 351–361 (2013); 22. Pastor, P., Kalakrishnan, M., Righetti, L., Schaal, S.: Towards Associative Skill Memories. In: 12th IEEE-RAS International Conference on Humanoid Robotics (Humanoids), pp. 309–315. Osaka, Japan (2012); 23. Pastor, P., Righetti, L., Kalakrishnan, M., Schaal, S.: Online movement adaptation based on previous sensor experiences. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2011); 24. Plagemann, C., Stachniss, C., Burgard, W.: Efficient failure detection for mobile robots using mixed-abstraction particle filters. In: European Robotics Symposium 2006, pp. 93–107 (2006); 25. Renner, R., Behnke, S.: Instability detection and fall avoidance for a humanoid using attitude sensors and reflexes. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2967–2973 (2006); 26. ROBOTIS: ROBOTIS DARWIN-OP e-Manual v1.23.00. http://support.robotis. com/en/product/darwin-op.htm (Aug. 2014); 27. Santos, C.P., Matos, V.: CPG modulation for navigation and omnidirectional quadruped locomotion. Robotics and Autonomous Systems 60(6), 912– 927 (2012); 28. Ruiz-del Solar, J., Moya, J., Parra-Tsunekawa, I.: Fall detection and management in biped humanoid robots. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2007) 2, 3323– 3328 (2010).
PUB
10. Kalyanakrishnan, S., Goswami, A.: Learning to predict humanoid fall. Inter-