ARTIGO TÉCNICO Luís Seabra Lopes1, Member, IEEE, António J.S. Teixeira1, Member, IEEE, Marcelo Quinderé2, Mário Rodrigues3 1 Dep. of Electronics, Telecommunications and Informatics, Universidade de Aveiro, Portugal. 2 IEETA, Universidade de Aveiro, Portugal. 3 Escola Superior de Tecnologia e Gestão de Águeda, Portugal
INTEGRATED CAPABILITIES FOR KNOWLEDGE ACQUISITION THROUGH SPOKEN LANGUAGE INTERACTION IN A MOBILE ROBOT ABSTRACT This paper describes an integrated set of capabilities developed to support knowledge acquisition in a mobile robot through spoken language interaction with its users. The robot is a prototype of an intelligent service robot, designed and developed having in mind hosting tasks in a building or event. The main modules involved in knowledge acquisition, here described, are concerned with spoken language understanding (based on a combination of deep and shallow methods), dialogue management and knowledge management. Question answering, supported by the knowledge management module, is based on deductive and inductive inference. The integration of these capabilities and the efficiency of the knowledge acquisition process are experimentally evaluated.
I. INTRODUCTION In recent years, robotics-related technologies have reached a remarkable level of maturity. Service robots can be found in a variety of fields, entertainment robots are popular, humanoid robot hardware and software are making substantial progress, etc. The next step seems to be to develop companion robots, meaning, intelligent service robots capable of performing useful work in close cooperation/interaction with humans [5][6]. It has been argued that this new generation of robots will need to comply with three criteria [29]. First, these robots must be animate, meaning that they should respond to changing conditions in their environment. This requires a close coupling of perception and action. Second, personal robots should be adaptable to different users and different physical environments. One of the basic capabilities in this respect is to be able to make decisions at the task-level. This is a central topic, since future robots are expected to be modular and reconfigurable, from a hardware point of view, and act in unstructured environments, which means that the number of action alternatives at the task-level will increase significantly, with respect to traditional automation-style robots. The adaptable criterion also implies the need for learning / knowledge acquisition capabilities [3]. Learning is important for robots to adapt to new environments, new users and new tasks. Learning should take place both at the sensorimotor level and at the task level. Finally, robots should be accessible, meaning that they should be able to explain their beliefs, motivations and intentions, and, at the same time, they should be easy to command and instruct. In most cases, accessibility will imply the use of spoken language communication, since it is the natural means of communication for human users. In order to meet the animate, adaptable and accessible criteria for intelligent service robots, it is, therefore, necessary to include in their design such basic capabilities as linguistic communication, reasoning, reactivity and learning. “Integrated Intelligence” is an emerging keyword that identifies an approach to building intelligent artificial agents in which the integration
[16]
robótica
of all those aspects of intelligence is considered [29]. Recent research works explore a variety of alternative paths, leading to architectures in which different functionalities are combined in different ways [5][6][25] [28][9][12] [15][35]. As mentioned, the adaptable criterion implies the acquisition of knowledge about environments, users and tasks. In the specific case of hosting, which is one of the scenarios envisaged in our project, the robot needs to acquire and provide knowledge to visitors. Such knowledge may include the physical location of people and facilities in a building, the functions of people in an organization, the conducted projects, etc. This paper describes an integrated set of capabilities developed to support knowledge acquisition in a mobile robot through spoken language interaction with its users. Usually, knowledge acquisition (KA) is understood as the process of accumulating semantic structures (in the classical artificial intelligence sense) for computer-based decision-support systems and includes interviews to experts and/or text analysis. Automating KA involves extracting knowledge from natural language interactions and/or from text. Unfortunately, KA from natural language can still be considered a relatively immature field [20]. There is little research on robots with symbolic knowledge processing capabilities. MARCO [16] is an agent capable of following free-form natural language route instructions in a virtual indoor environment. Using linguistic and spatial knowledge, it can infer implicit actions. However, it does not have a KA capability. Given that the scenarios in which our robot must act can be dynamic, it is not viable to encode all the required knowledge. The robot therefore needs KA capabilities. The main sources of such knowledge are the users themselves. In this context, it is necessary to integrate possibly contradictory knowledge from different sources. Related problems have been addressed in other contexts. Benferhat et al. [1] presented strategies for conflict resolution developed to deal with exception handling and iterated belief revision, but which could also be applied in merging information from different sources. A method to construct an integrated knowledge
REFERENCES [1] [2]
Despite the small size of the test, the results show that the new dialogue manager is able to handle pronoun resolution and has improved the performance of the robot on knowledge acquisition.
[3] [4] > @
VII. CONCLUSION The paper described Carl, an interactive robot built at the University of Aveiro. The paper focused on an integrated set of capabilities developed to support knowledge acquisition through spoken language interaction in Carl. Spoken language understanding follows a hybrid deep/shallow analysis approach that is, to a certain extent, robust with respect to misrecognition problems. The information state based dialogue manager is flexible and supports the main requirements of the robot Carl, namely reactivity, mixed initiative, grounding and ambiguity resolution. The knowledge acquisition and management module is able to accumulate (possibly contradictory) information obtained from different interlocutors, with different recognition confidences. When asked, Carl attempts to provide the most plausible reply given the accumulated information. An evaluation of these capabilities was presented above.
[6] [7] [8]
[9]
[10] [11]
[12] [13] [14]
Carl incorporates several commercial hardware components as well as several commercial and public domain software components. This development approach is central to the integrative philosophy of the project. The software architecture of Carl is based on a community of OAA agents running in two onboard computers. Carl is currently being evaluated from the point of view of usability. A set of usability tests were carried out, and the results are currently being processed.
[15] [16]
[17] [18] [19]
[20]
Future work in this project should continue to improve on several fronts, namely robustness of spoken language understanding, more sophisticated language generation and further exploration of the potential of the “Information State� approach to dialog management. Another possibility for future exploration is use Portuguese language in Carl or another similar robot. Yet another challenge, which we are already addressing in a parallel project, is to develop language acquisition capabilities for robots [27]. On a side note to this conclusion, we would like to mention that Carl, in its 10-years life, became a popular robot. It was demonstrated in several national and international events (including AAAI’2001 in Seattle, IROS’2002 in Lausanne, and RoboCup’2004 in Lisbon). It appeared on the cover of AI Magazine in 2002. It appeared on the website of ActivMedia Robotics for some time in 2002. It was more recently included in a selection of European robots that appear in a leaflet of EURON, the European Robotics Research Network. Last but not the least, it is frequently cited in scientific literature (e.g. [7] [8][11][12][14][15][19][35][37]).
[21]
[22] [23]
[24] [25] [26] [27] [28] [29] [30] [31]
[32]
ACKNOWLEDGMENTS This work has been funded by the University of Aveiro under a PhD grant to Marcelo QuinderĂŠ and a Research Assistant grant to MĂĄrio Rodrigues. The CARL project was initially funded by FCT under contract PRAXIS/P/ EEI/12121/1998 (“Communication, Action, Reasoning and Learning in Roboticsâ€?) and more recently under contract POSI/SRI/48794/2002 (“LanJXDJH *URXQGLQJ IRU +XPDQz5RERW &RPPXQLFDWLRQ} ZKLFK LV SDUWLDOO\ funded by FEDER.
[33]
[34] [35] [36] [37]
[22]
robĂłtica
Benferhat, S.; Kaci, S.; Le Berre, D.; and Williams, M. 2004. Weakening conflicting information for iterated revision and knowledge integration, Artificial Intelligence z Brazdil, P.; and Torgo, L. 1990. Knowledge Acquisition via Knowledge Integration, Current Trends in AI z $PVWHUGDP ,26 3UHVV Connell, J.H.; and Mahadevan, S. (edrs.) 1993. Robot Learning. Kluwer Academic Publishers. Daelemans, W.; Zavrel, J.; van den Bosch, A.; and van der Sloot, K. 2002. TiMBL: Tilburg Memory-Based Learner Reference Guide, version 4.2, Technical Report, Tilburg University. 'DXWHQKDKQ . 5RERWV ZH /LNH WR /LYH ZLWK" z $ 'HYHORSPHQWDO 3HUVSHFWLYH RQ D 3HUVRQDOL]HG Life-Long Robot Companion. In Proceedings of IEEE Ro-man 2004, 13th IEEE International Workshop on Robot and Human Interactive Communication z 2ND\DPD -DSDQ Dautenhahn, K.; Woods, S.; Kaouri, C.; Walters, M.; Lee Koay, K.; and Werry, I. 2005. What is a Robot &RPSDQLRQ z )ULHQG $VVLVWDQW RU %XWOHU" ,Q Proceedings of IROS2005. Edmonton, Alberta, Canada. Eliasson, K. (2005) Towards a Robotic Dialogue System with Learning and Planning Capabilities, Proc. IJCAI’2005 4th Workshop on Knowledge and Reasoning in Practical Dialogue Systems. Hanafiah, Z.M., C. Yamazaki, A. Nakamura and Y. Kuno (2004) Understanding inexplicit utterances using vision for helper robots, ICPR 2004. Proceedings of the 17th International Conference on Pattern Recognition. HĂźwel, S., B. Wrede, and G.Sagerer, Robust Speech Understanding for Multi-Modal Human-Robot Communication, Proc. 15th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN06), Hatfield, UK, September 6-8, 2006, p. 45-50. Jurafsky, D., and J. H. Martin. Speech and Language Processing. to be published, 2nd edition, 2007. Kiatisevi, P., V. Ampornaramveth and H. Ueno (2004) “Knowledge-based Interactive Robot: System Architecture and Dialogue Managerâ€?, PRICAI 2004: Trends in Artificial Intelligence: 8th Pacific Rim International Conference on Artificial Intelligence, Proceedings, Lecture Notes in Computer Science, vol. 3157, p. 673-682. Kiatisevi, P., V. Ampornaramveth, H. Ueno, (2006) A Frame-based Knowledge Software Tool for Developing Interactive Robots, International Journal of Artificial Life and Robotics, 10 (1), 18-28. Lappin, S., and H. J. Leass. An algorithm for pronominal anaphora resolution. Comp Ling. z 1994. Lauria, S., G. Bugmann et al (2001) Training Personal Robots Using Natural Language Instruction, IEEE Intelligent Systems, 16(5), 38-45. Lauria, S., G. Bugmann, T. Kyriacou, E. Klein (2002) “Mobile Robot Programming Using Natural Languageâ€?, Robotics and Autonomous Systems, 38 (3-4): 171-181. MacMahon, M. B. Stankiewicz, B. Kuipers (2006) Walk the Talk: Connecting Language, Knowledge, Action in Route Instructions, In Proc. 21st National Conf. Artificial Intelligence (AAAI-2006), Boston, 0$ S z Martin, D.; Cheyer, A.; and Moran, D. 1990. The Open Agent Architecture: a framework for building distributed software systems. Applied Artificial Intelligence z Michalski, R. S. 1994. Inferential Theory of Learning: Developing Foundations for Multistrategy Learning. Machine Learning: A Multistrategy Approach, Morgan Kaufman, 3-106. Ogura, Y., S. Ando, N. Hieda, H. Lim, and A. Takauishi, Macro-walking Instruction for Biped Humanoid Robot, Proc. 2003 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2003), 417-422. Potter, S. (2001) A Survey of Knowledge Acquisition from Natural Language, technology maturity assessment , Artificial Intelligence Applications Institute, University of Edinburgh. QuinderĂŠ, M., L. Seabra Lopes and A.J.S. Teixeira (2007) A Dialog Manager for an Intelligent Mobile Robot, Proc. Workshop on natural Language Processing and Cognitive Science (NLPCS-2007), Funchal, Madeira, 191-200. QuinderĂŠ, M., L. Seabra Lopes and A.J.S. Teixeira (2007) An Information State Dialogue Manager for a Mobile Robot, Proc. Interspeech’2007, Antwerp, Belgium, August 27-31, p. 162-165. Rodrigues, M.; Teixeira, A.; Seabra Lopes, L. 2004. An Hybrid Approach for Spoken Natural Language Understanding Applied to a Mobile Intelligent Robot. In Proceedings of Natural Language Understanding and Cognitive Science: NLUCS’2004 z 3RUWR 3RUWXJDO RosĂŠ, C. P.; and Lavie, A. 1998. LCFlex: An Efficient Robust Left-Corner Parser. Technical Report. University of Pittsburgh. Russell, S.; Norvig, P. 2003. Artificial Intelligence: A Modern Approach, 2nd ed. Prentice-Hall. Seabra Lopes, L. 2002. Carl: from Situated Activity to Language Level Interaction and Learning. In Proceedings of IEEE IROS’2002 z /DXVDQQH 6ZLW]HUODQG Seabra Lopes, L. and A. Chauhan (2007) How many Words can my Robot learn? An Approach and Experiments with One-Class Learning, Interaction Studies, 8(1), 53-81. Seabra Lopes, L.; and Connell, J. H. (edrs.) 2001a. Semisentient Robots (Special Issue of IEEE Intelligent Systems 16(5)), Computer Society. Seabra Lopes, L.; and Connell, J. H. 2001b. Semisentient Robots: Routes to Integrated Intelligence, IEEE Intelligent Systems z Seabra Lopes, L.; and Teixeira, A. 2000. Human-Robot Interaction through Spoken Language Dialogue. In Proceedings of IROS’2000 z -DSDQ Seabra Lopes, L.; Teixeira, A.; and QuinderĂŠ, M. 2005. A Knowledge Representation and Reasoning Module for a Dialogue System in a Mobile Robot. In Proceedings of 2nd International Workshop on Natural Language Understanding and Cognitive Science: NLUCS’2005 z 0LDPL 86$ Seabra Lopes, L.; Teixeira, A.; QuinderĂŠ, M.; and Rodrigues, M. 2005. From Robust Spoken Language Understanding to Knowledge Acquisition and Management. In Proceedings of Interspeech’2005, z . Lisbon, Portugal. Seabra Lopes, L.; Teixeira, A.; Rodrigues, M.; Gomes, D.; GirĂŁo, J.; Teixeira, C.; SĂŠnica, N.; Ferreira, L.; and Soares, P. 2003. A Robot with Natural Interaction Capabilities, In Proceedings of ETFA’2003, z Sowa, J.F. 2005. Semantic Networks. MIT Encyclopedia of Cognitive Science, http://www. jfsowa. com/pubs/semnet.htm Toptsis, I, A. Haasch, S. HĂźwel, J. Fritsch and G.A. Fink (2005) Modality Integration and Dialog Management for a Robotic Assistant, Proc. InterSpeech’2005, Lisbon, Portugal, p. 837-840. Traum, D., and S. Larsson. The Information State Approach to Dialogue Management. In Jan van Kuppevelt and Ronnie Smith, editor, Current and New Directions in Discourse and Dialogue. Kluwer, 2003. Vasudevan, S., S. Gächter, V. Nguyen, R. Siegwart (2007) Cognitive maps for mobile robots - an object based approach, Robotics and Autonomous Systems z
INTEGRATED CAPABILITIES FOR KNOWLEDGE ACQUISITION THROUGH SPOKEN LANGUAGE INTERACTION IN A MOBILE ROBOT
SLU/DM/KAM system on the second experiment. One can see that it has a high performance and confidence efficiency.