The Look That Hands Over the Floor: Gaze, Conversation and the Robot Stare

Where we point our eyes quietly organises who speaks next, and robots that get gaze right can steer a conversation without saying a word.

Eyes as traffic signals

Most of us have never thought about where we look while we talk, which is exactly why it works. Gaze is one of the most heavily used and least noticed signals in conversation. It tells others whether we are paying attention, whether we expect a reply, and sometimes who we expect it from.

The modern study of this begins with Adam Kendon's 1967 paper in Acta Psychologica. Kendon filmed pairs of people in conversation and analysed in detail where they looked and when. He described gaze as doing several jobs at once: monitoring the other person for feedback, regulating the flow of turns, and expressing emotion and engagement. One of his most cited observations was that speakers tended to look away as they began a long utterance and to look back towards their partner as they approached its end, as if checking in and offering the floor.

A pattern, not a law

It is worth being careful here. Kendon's account was based on a small number of conversations, and later researchers questioned how strongly and how universally gaze regulates turns. Looking at the listener at the end of a turn is a tendency, not a rule, and it interacts with culture, status, the task at hand and simple habit. In face-to-face conversation about objects, for instance, people spend much of their time looking at the objects rather than at each other.

For robot designers, that nuance matters. A robot that mechanically stares at the user to signal 'your turn' can feel interrogating rather than inviting. The question is not just whether to use gaze, but how to use it in a way that people read as natural.

Robots that assign roles with their eyes

One of the clearest demonstrations that robot gaze carries social meaning came from Bilge Mutlu and colleagues in 2009, working with the Robovie robot at ATR in Japan. Borrowing the sociologist Erving Goffman's idea of footing, the roles people take in an interaction, they designed gaze behaviours intended to cast each of two human participants as an addressee, a bystander or an overhearer.

In a laboratory study with 72 participants, the robot used gaze alone to place people in these roles. The participants' behaviour followed suit: those treated as addressees took more part in the conversation, while those treated as onlookers held back. The effects were not only behavioural. Addressees reported feeling more part of the group, and people cast as overhearers differed from others in how much they liked the robot. A few degrees of head and eye movement, in other words, decided who felt included.

The art of looking away

Looking at someone is only half of gaze. Humans also look away, and they do it for reasons. Averting gaze can signal that you are thinking, that you have not finished your turn, or simply that you are not trying to dominate the other person.

In 2014 Sean Andrist, Xiang Zhi Tan, Michael Gleicher and Mutlu built a model of conversational gaze aversion for robots, based on statistics of how people look away during talk, and implemented it on a NAO robot that has no movable eyes and must use its head instead. In a study with 30 participants the aversions were perceived as intentional. They made the robot seem more thoughtful and helped it hold the conversational floor: well-timed glances away discouraged people from interrupting while the robot was still mid-thought. It is a neat inversion of the usual goal. Sometimes the most human thing a robot can do is not look at you.

Gaze in groups

Gaze becomes even more powerful when several people are present, because it can single out one listener from many. Dan Bohus and Eric Horvitz at Microsoft Research tested this with an embodied conversational agent that combined speech, gaze and gesture to hold, release or hand over the floor in a shared task with two human participants. When the agent released the floor to a particular person during verbal confirmations, that person was the first to speak in about 86 per cent of cases.

This is where gaze moves from being a nicety to being infrastructure. In a group, a robot that cannot direct its attention clearly will create confusion about who is supposed to answer, and confusion quickly becomes silence or overlap.

Design lessons

Three practical points emerge. First, gaze must be timed to speech, not just aimed: the moment a robot shifts its eyes carries as much meaning as the direction. Second, a robot's face design shapes what it can say with gaze. Faces with expressive eyes, such as back-projected heads, can make subtle shifts that a rigid head cannot. Third, gaze is a perception problem as well as a display problem. A robot that wants to read gaze cues from people needs reliable tracking of heads and eyes in messy real-world settings.

Above all, gaze reminds us that conversation is not just an exchange of words. It is a shared activity carried out by bodies, and robots that ignore that will always seem slightly absent, however good their language.

References

1. Kendon, A. (1967). Some functions of gaze-direction in social interaction. Acta Psychologica, 26, 22-63. Link
2. Mutlu, B., Shiwa, T., Kanda, T., Ishiguro, H. and Hagita, N. (2009). Footing in human-robot conversations: how robots might shape participant roles using gaze cues. Proceedings of HRI 2009, 61-68. Link
3. Andrist, S., Tan, X. Z., Gleicher, M. and Mutlu, B. (2014). Conversational gaze aversion for humanlike robots. Proceedings of HRI 2014, 25-32. Link
4. Bohus, D. and Horvitz, E. (2010). Facilitating multiparty dialog with gaze, gesture, and speech. Proceedings of ICMI-MLMI 2010. Link

Feature drafted with AI from the published research listed above, for social-robotics.com.