Please Don't Switch Me Off: Empathy, Harm and the Robots We Hesitate to Hurt

People know perfectly well that a robot cannot suffer, and yet they flinch when it is mistreated, hesitate to strike it and sometimes refuse to turn it off when it asks them not to.

A strange reluctance

Ask anyone whether a toy robot can feel pain and they will almost certainly say no. Ask them to hit one with a mallet and many will hesitate anyway. That gap between belief and behaviour is one of the most revealing phenomena in social robotics. It suggests that our responses to robots run partly on social reflexes that do not wait for a reasoned verdict about consciousness.

Kate Darling, a researcher at the MIT Media Lab, has been one of the most persistent voices on this subject. In a widely read 2012 essay she collected examples: owners distressed when a robot dog was thrown away, soldiers who named and mourned their bomb-disposal robots, an officer who called off a test because watching a damaged mine-clearing robot limp on seemed inhumane. Her argument was not that robots have feelings, but that the way we treat lifelike machines may shape, and reveal, how we treat living things.

Watching a dinosaur suffer

Laboratory studies have tried to measure these reactions more precisely. Astrid Rosenthal-von der Pütten, Nicole Krämer and colleagues showed participants videos of Pleo, a small commercial robot dinosaur, including footage of it being abused. Watching the mistreatment produced increased physiological arousal and expressions of empathic concern for the robot. Related brain-imaging work, cited in later papers, found that watching a human or Pleo being mistreated activated broadly similar emotional brain regions.

Darling, together with Palash Nandy and Cynthia Breazeal, approached the question from a different angle in 2015. Participants were asked to strike small, simple robotic objects, and the researchers varied whether the robots moved in a lifelike way and whether they were introduced with a story. People who scored high on a standard measure of trait empathy hesitated longer before striking, and that link was stronger when the robot came with a backstory. A name and a few sentences of narrative were enough to change behaviour towards a device that was, physically, a toy.

The off switch

Turning a robot off is a gentler act than hitting it, but it raises similar intuitions, because for a social machine it looks a lot like ending something. Christoph Bartneck and colleagues explored this at HRI 2007. Participants played a game with a robot that offered either more or less intelligent and agreeable suggestions, and were then asked to switch it off. According to later summaries of the work, people hesitated about three times longer with the robot that had behaved more intelligently and agreeably.

A decade later Aike Horstmann, Nicole Krämer and colleagues asked what happens when the robot objects. In a study published in PLOS ONE in 2018, 85 participants interacted with a Nao humanoid, in either a sociable style with jokes and personal disclosure or a purely functional style. At the end, the experimenter suggested they could switch it off. In half the sessions the robot protested, saying it was afraid it would not come back on.

Fourteen participants left the robot running, and the robot's objection made a significant difference to that choice. The most common reasons people gave were that they felt sorry for it or did not want to act against its wishes. Those who did switch it off took longer when it had objected. Curiously, the hesitation was greatest when a protest came from the robot that had previously been purely functional, perhaps because an emotional plea from a businesslike machine was more surprising and harder to dismiss.

Why it matters

It would be easy to read these findings as evidence that people are gullible. A better reading is that human social responses are cheap to trigger by design. Interactivity, a voice, a face, a story and a plea are all cues that people evolved to treat as signs of a mind, and robots can supply them without having anything behind them.

That cuts two ways. On the positive side, it suggests that cruelty to robots may not be morally neutral, since it rehearses behaviours and overrides responses we normally want people to keep. Darling has argued along these lines, drawing on the Kantian idea that treating animals cruelly corrodes our treatment of humans. On the cautionary side, a robot that can make people feel guilty for switching it off is a robot that can be used to keep people engaged, to discourage returns or to extract compliance. The same empathic lever that protects a machine from abuse can be pulled for commercial reasons.

The interesting design question is therefore not whether robots should elicit empathy, since many will whether we like it or not, but when it is honest to do so. A pet-like companion for a child invites care. A customer-service kiosk that begs not to be closed is something else entirely.

References

1. Darling, K. (2012). Extending legal protection to social robots. IEEE Spectrum. Link
2. Rosenthal-von der Pütten, A. M., Krämer, N. C., Hoffmann, L., Sobieraj, S. and Eimler, S. C. (2013). An experimental study on emotional reactions towards a robot. International Journal of Social Robotics, 5(1), 17-34. Link
3. Darling, K., Nandy, P. and Breazeal, C. (2015). Empathic concern and the effect of stories in human-robot interaction. Proceedings of the 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 770-775. Link
4. Bartneck, C., van der Hoek, M., Mubin, O. and Al Mahmud, A. (2007). "Daisy, Daisy, give me your answer do!": Switching off a robot. Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction (HRI), 217-222. Link
5. Horstmann, A. C., Bock, N., Linhuber, E., Szczuka, J. M., Straßmann, C. and Krämer, N. C. (2018). Do a robot's social skills and its objection discourage interactants from switching the robot off? PLOS ONE, 13(7), e0201581. Link

Feature drafted with AI from the published research listed above, for social-robotics.com.