The Voice in the Machine: Designing How a Robot Sounds

A robot's voice is often its most powerful social signal, triggering gender stereotypes, shaping trust and setting expectations the rest of the machine may not be able to meet.

The first thing people judge

Long before anyone has formed a view of a robot's intelligence, they have heard it speak. Within a sentence or two, listeners are making guesses about its age, gender, origin, competence and friendliness, the same snap judgements we make about people on the phone. For designers this makes voice one of the cheapest and most consequential choices on the spec sheet. It is also one of the easiest to get wrong, because the responses it triggers are largely automatic.

Machines are not gender neutral

The classic demonstration comes from Nass, Moon and Green, who in 1997 asked whether computers stripped of almost every social cue would still be stereotyped. In an experiment with forty participants, the only hint of gender came from the computer's recorded voice. Nobody was deceived about what they were dealing with: participants knew they were working with a machine. Even so, across all three gender stereotypes the researchers tested, people's judgements shifted in the stereotypical direction depending on whether the voice sounded male or female. The authors took this as evidence of how deeply rooted gender stereotyping is, so deeply that it extends to boxes on a desk.

For robot designers the implication is uncomfortable. Choosing a voice is not a neutral aesthetic decision. A feminine voice on a helpful assistant, or a masculine one on a security robot, may quietly reinforce the very stereotypes a product team would claim not to hold. Users will also bring those expectations to bear on how they rate the robot's performance.

Sounding like me

Voice does not only carry gender; it interacts with who is listening. In a 2012 HRI study, Eyssel and colleagues varied both the gender of a robot's voice and whether that voice sounded human-like or obviously synthetic. Participants felt more positive towards, and psychologically closer to, a robot whose voice matched their own gender. They also attributed more human qualities to the same-gender robot, but only when its voice sounded human-like. The authors suggested a projection mechanism: people may read their own characteristics into a machine that sounds a bit like them.

This is a useful corrective to the assumption that there is a single best voice. What counts as warm or credible depends on the listener, which raises awkward design questions for robots deployed to diverse publics.

Trust is a moving target

It is tempting to assume that a more natural voice simply makes a robot more trustworthy. Work by Torre, Goslin, White and Zanatto complicates that picture. Across three experiments using a repeated investment game, which measures trust through how much money people hand over, participants played with agents that had different accents or, in one experiment, a robot with either a natural or a synthetic voice. The agents then behaved either reliably or unreliably.

The striking result was an interaction: the voice that earned more trust when the agent behaved well was not the same one that earned more trust when it behaved badly. The researchers interpreted this as a congruency effect. A voice creates a first impression, and what matters next is whether the agent's behaviour confirms or violates it. A voice that promises a lot and then disappoints may be punished more than one that set lower expectations.

The expectation gap

This links to a broader concern in the field. A survey of voice in human-agent interaction by Seaborn and colleagues, published in ACM Computing Surveys in 2021, emphasises that voice and body are perceived together, and that people tend to apply the social psychology they use with humans to vocal machines. It also notes that attitudes to synthetic voices have shifted over time, alongside the spread of commercial voice assistants.

Put these findings together and a design principle emerges. A highly human voice on a machine with limited understanding writes a cheque the rest of the system cannot cash. The voice advertises social competence, and when the robot misunderstands a simple request, the fall is steeper. Modern neural speech synthesis makes it trivially easy to give any robot a voice indistinguishable from a person's, which makes this mismatch more likely, not less.

Choosing deliberately

None of this means robots should all sound like old-fashioned speech synthesisers. It means voice deserves the same scrutiny as any other part of the interaction design. Teams should ask what stereotypes a given voice might activate, whether the voice matches what the robot can actually do, and how it will be heard by different groups of users. Some designers deliberately choose voices that are clearly artificial yet pleasant, or harder to place on a gender scale, precisely to manage expectations.

The research does not hand us a perfect voice. What it shows, convincingly, is that people will hear a personality, a gender and a promise whether or not we meant to put them there.

References

1. Nass, C., Moon, Y. and Green, N. (1997). Are Machines Gender Neutral? Gender-Stereotypic Responses to Computers With Voices. Journal of Applied Social Psychology. Link
2. Eyssel, F., Kuchenbrandt, D., Bobinger, S., de Ruiter, L. and Hegel, F. (2012). 'If you sound like me, you must be more human'. Proceedings of the 7th ACM/IEEE International Conference on Human-Robot Interaction. Link
3. Torre, I., Goslin, J., White, L. and Zanatto, D. (2018). Trust in artificial voices. Proceedings of Technology, Mind, and Society. Link
4. Seaborn, K., Miyake, N. P., Pennefather, P. and Otake-Matsuura, M. (2021). Voice in Human-Agent Interaction. ACM Computing Surveys. Link

Feature drafted with AI from the published research listed above, for social-robotics.com.