Social RoboticsColumbia robot face learns to lip-sync to speech and song
A 26-motor robotic face taught itself realistic mouth movements by watching its reflection and videos of people.
Researchers in Hod Lipson's Creative Machines Lab at Columbia Engineering, with PhD student Yuhang Hu leading the study, reported in Science Robotics a humanoid face that learns to move its lips in time with audio. The robot first practised random expressions in front of a mirror to learn how its 26 motors shape its mouth, then studied recordings of humans talking and singing. It could sync to speech in several languages and to songs without understanding the words, though sounds such as B and W remained difficult. Believable facial motion is a long-standing obstacle for humanoid social robots, and the team also flagged the ethical questions raised by machines that connect more convincingly with people.
This archive entry summarises reporting published elsewhere; follow the source link for the full story. The summary was drafted with AI from that reporting.