Clever but Slow: Large Language Models and the Race Against the Awkward Pause

LLMs finally gave robots something worth saying, but they also made it harder to say it on time, and timing is half of what makes speech feel like conversation.

Why a second feels like forever

Earlier in this series we saw that people typically respond to each other within about 200 milliseconds, and that this speed depends on listeners planning their reply before the other person finishes. Against that baseline, a machine that takes two or three seconds to answer is not just slow. It sends a social signal. In human conversation, a delay before a response often foreshadows a refusal, a disagreement or trouble understanding. People hear robot latency through the same filter.

Robots have a related but distinct problem: they are judged on whether they seem to be paying attention at all. A silent, motionless robot after you have spoken could be thinking, could have misheard, or could have crashed. Uncertainty quickly becomes discomfort.

What the early experiments found

In 2008 Toshiyuki Shiwa, Takayuki Kanda and colleagues at ATR in Japan asked a simple question: how quickly should a communication robot respond? In their experiments, people's preference peaked when the robot responded after about one second. Interestingly, an instant response was not the most liked.

Because real systems cannot always manage a second, the team also tested a conversational filler, the Japanese equivalent of 'um' or 'well'. Using a filler softened people's impressions of a long wait. The lesson has proved durable. A robot that signals it has heard you and is working on an answer buys itself time, in the same way that a person who says 'hmm, good question' is not penalised for taking a moment.

Incremental thinking

Fillers are a patch. The deeper fix, explored by dialogue researchers since the late 2000s, is incremental processing: building systems that work on partial input rather than waiting for complete utterances. Gabriel Skantze and David Schlangen's 2009 work on incremental dialogue processing in a small task domain is one of the classic early examples. The idea is to let speech recognition, understanding and response planning run in parallel with the user's speech, much as human listeners do, so that the system can begin to respond as soon as it is appropriate.

Incremental architectures are harder to build. Hypotheses must be revised as more words arrive, and the system must be able to abandon a half-prepared reply if the user's turn takes an unexpected direction. But they are the only realistic route to human-like timing, because they move work out of the silence after the user stops and into the time while the user is still talking.

Enter the language models

Large language models transformed robot dialogue almost overnight. Before them, most social robots relied on scripted flows or narrow intent recognition, which worked within tight domains and broke as soon as conversation wandered. LLMs allowed open-ended talk on almost any topic, with fluent, context-aware replies. For the first time, a robot could plausibly chat.

The catch is that LLMs add latency at the worst possible point. A typical pipeline now involves speech recognition, then sending text to a model, waiting for the first tokens, then synthesising speech. Even with streaming output, the delay can stretch well beyond what people tolerate in face-to-face talk. And LLMs were trained on text, not on the rhythms of spoken interaction, so on their own they know nothing about when to speak.

Lessons from the field

A study by Bahar Irfan, Sanna Kuoppamäki and Skantze, published in Autonomous Robots in 2025, shows these problems vividly. They built a companion robot on Furhat using GPT-3.5 and refined it through participatory design with Swedish-speaking older adults. Early sessions were marked by frequent interruptions, slow and repetitive responses and superficial conversation. Even after improvements, problems remained, including hallucinations, out-of-date information and a failure to notice when people were disengaging. These caused frustration, confusion and worry among participants. The language model's fluency did not solve the problems of conversation as an activity.

Encouragingly, the same group has shown that the timing side can be improved by pairing LLMs with dedicated turn-taking models. In their 2025 HRI paper, Skantze and Irfan combined predictive models of turn ends with an LLM on Furhat, so the robot could begin preparing a reply before the user finished and avoid jumping in during pauses. Compared with a silence-based baseline, it reduced both response delays and interruptions, and participants preferred it.

The division of labour

The emerging picture is one of specialisation. Language models supply content: what to say. Turn-taking and backchannel models supply timing: when to say it, and how to show you are listening in the meantime. Fillers, gaze aversion and nods cover the gaps while the slower machinery catches up. Speech-native models that process audio directly may eventually blur these lines, but the underlying demands of conversation will not change.

The irony is hard to miss. For decades, robot dialogue was fast but dim. Now it can be bright but slow. The real achievement will be a robot that is both, and that is ultimately less a question of bigger models than of respecting the timing that humans have been negotiating, at around a fifth of a second per handover, for as long as we have talked.

References

1. Shiwa, T., Kanda, T., Imai, M., Ishiguro, H. and Hagita, N. (2008). How quickly should communication robots respond? Proceedings of HRI 2008, 153-160. Link
2. Skantze, G. and Schlangen, D. (2009). Incremental dialogue processing in a micro-domain. Proceedings of EACL 2009, 745-753. Link
3. Levinson, S. C. and Torreira, F. (2015). Timing in turn-taking and its implications for processing models of language. Frontiers in Psychology, 6, 731. Link
4. Irfan, B., Kuoppamäki, S. and Skantze, G. (2025). Between reality and delusion: challenges of applying large language models to companion robots for open-domain dialogues with older adults. Autonomous Robots, 49, 9. Link
5. Skantze, G. and Irfan, B. (2025). Applying general turn-taking models to conversational human-robot interaction. ACM/IEEE International Conference on Human-Robot Interaction (HRI 2025). Link

Feature drafted with AI from the published research listed above, for social-robotics.com.