Interactions between Human and Robot – Case Study: WorkPartner-robot in the ISR 2004 Exhibition

Sami Ylönen, Mikko Heikkilä, Petri Virekoski

Automation Technology Laboratory, Helsinki University of Technology, P.O. Box 5500, 02015 HUT, Finland,

Abstract

WorkPartner is a mobile interactive service robot designed for lightweight outdoor tasks in co-operation with humans. WorkPartner participated in the ISR 2004 (35th International Symposium on Robotics) exhibition on CLAWAR stand in Paris 22 – 26 March 2004. During the five days a lot of information was collected about human-machine interaction. The robot communicated with humans using speech and gestures, and observed environment using vision system. The visitors seemed to get a very humane impression of the robot.

Keywords: human-machine interaction, vision system, speech, gestures

Introduction

Mobility of WorkPartner is based on a hybrid system, which combines the benefits of both legged and wheeled locomotion to provide at the same time good terrain negotiating capability and a large velocity range (see Fig. 1). The working tool is a two-hand human-like manipulator that can be used for manipulation or handling of tools. The user or operator can be physically present on the same site as the robot and communicate with it using speech and gestures, or he can use telepresence from another place and communicate via Internet.

The ultimate goal is a highly adaptive service robot. Some possible work tasks for the WorkPartner: garden work, guarding, picking trash, transferring lightweight obstacles, environment mapping. The WorkPartner project, its mechatronics design, hybrid locomotion and control system have been reported in six previous CLAWAR conferences [1], [2], [3], [4], [5], [6]. The purpose of present paper is to continue the series by introducing development that is done for its human interaction. The project is public and can be followed on the Web site: www.automation.hut.fi/IMSRI/workpartner.

Fig. 1. WorkPartner at the exhibition stand

The system diagram in figure 2 summarises the hardware structure of WorkPartner indicating also the main software environment and communication interconnections between the subsystems. [7]

Fig. 2. System diagram of the WorkPartner hardware

Interaction

WorkPartner was demonstrated on CLAWAR stand in the ISR 2004 exhibition. Demonstration consisted of movements and interaction like speech, gestures and eye contact. The robot was driven near the humans and it offered candies (Fig. 3). It was working under teleoperation. The operator drove the robot using a joystick and a teleoperation device for the humanlike manipulator (Fig. 4).

Appearance is very important for a robot that works interactively with humans. Many research organisations and companies are developing robots that look like humans, for example Honda has developed Asimo robot [8]. WorkPartner has a human like upper body, but for greater mobility, it has a platform with four wheeled legs.

Fig. 3. WorkPartner offering candies for the exhibition visitors

Fig. 4. Teleoperation of the manipulator

Communication methods of the robot

Usability of the robot is one of the most important research areas in the WorkPartner project. A robot that is designed to be working interactively with humans has to be easy to use by different people. Therefore it should be able to communicate in a way that is natural for humans. WorkPartner uses speech, gestures and eye contact.

Speech

Different things can be expressed easily by speech and it is the most natural way of communication between humans. In the exhibition WorkPartner had a couple of preprogrammed sentences that it spoke by a speech synthesis program. For example it said, “Take some candies”, “Greetings from Finland” and “What is your name”.

Gestures

The robot can make many different gestures using its arms, legs and head. For example it did dancing movements with the body, waved its hand and nodded its head.

Eye contact

WorkPartner has a CCD-camera in its turning head. It was in color tracking mode. Human face color was selected for the tracking. This way the robot was looking for the faces of the visitors and an eye contact was formed between human and the robot.

Human reactions

This chapter describes the most important results that we got in the exhibition. It was interesting to see how differently people reacted to the robot. Most humans were very interested, some were cautious and some were very familiar with it. Some communicated to the operator and some to the robot. Some acted with the robot like it was another human. Overall, the visitors seemed to get a very humane impression of the robot.

Humans were looking at the camera of the robot and most were smiling. This was like eye contact between humans and the robot. Figure 5 shows some human reactions seen by the camera of WorkPartner.

Fig. 5. Human reactions seen by the camera of the robot

People said “thank you” to WorkPartner quite often after it had given candies. Gestures toward the robot included smiling, shaking hands, blowing a kiss and even kissing the head of the robot. It looks that children humanize the robot more often than adults. Women regarded to WorkPartner more emotionally and men more technically.

Taking candies

Main work task of WorkPartner was giving candies for the visitors. The candies were in a paper bag that was held by the robot. Only a few visitors approached WorkPartner initially, but when it said, “Take some candies” and handed the bag towards them, many were encouraged to come and take sweets.

Media interest

WorkPartner was filmed to the news of four main TV-channels in France. Some magazines were interested in WorkPartner too.

Future of the WorkPartner-robot

The goal of the project is to have WorkPartner communicate autonomously with humans using natural methods such as speech, speech recognition and gestures and also to learn tasks. It has been a research project, but we are looking for partners to cooperate in prototyping and commercializing WorkPartner and/or its component systems.

Conclusions

The interactivity of the robot has a great impact on the behavior of humans. The robot appeared to be looking at humans by turning its head, made gestures by its arms and spoke. This induced some people to actually speak to the robot instead of its operator and also establish an eye contact with it.

It was a very nice experience to participate in the big exhibition with WorkPartner. We got much information concerning interactions between humans and robots. This information is utilized in our research. Collecting of interaction experiences will continue in the future.

References

1. Leppänen I., Salmi S. and Halme A., WorkPartner – HUT-Automations new hybrid walking machine, CLAWAR'98, Brussels 1998.

2. Halme A., Leppänen I. and Salmi S., Development of WorkPartner-robot – design of actuating and motion control system, CLAWAR'99, Portsmouth 1999.

3. Halme A., Leppänen I., Salmi S. and Ylönen S. Hybrid locomotion of a wheel-legged machine. CLAWAR'2000, Madrid 2000.

4. Halme A., Leppänen I., Montonen M., Ylönen S. Robot motion by simultaneous wheel and leg propulsion. 4th International Conference on Climbing and Walking Robots, Karlsruhe 2001.

5. Ylönen S., Halme A. Further development and testing of the hybrid locomotion of WorkPartner robot. 5th International Conference on Climbing and Walking Robots, Paris 2002.

6.  Luksch T., Ylönen S., Halme A. Combined Motion Control of the Platform and the Manipulator of WorkPartner Robot, 6th International Conference on Climbing and Walking Robots, Catania 2003.

7.  Halme A., Leppänen I., Suomela J., Ylönen S., Kettunen I. WorkPartner: Interactive Human-Like Service Robot for Outdoor Applications, International Journal of Robotics Research, Vol 22, July-August, 2003.

8.  Sakagami Y., Watanabe R., Aoyama C., Matsunaga S:, Higaki N., Fujimura K., The intelligent ASIMO: System overview and integration, IROS 2002, International Conference on Intelligent Robots and Systems, EPFL, Lausanne, Switzerland, pp. 2478-2483, October 2-4, 2002.