Social RoboticsGoogle unveils PaLM-E, a language model that takes in robot sensor data
The largest version, with 562 billion parameters, combined the PaLM language model with a large vision model to plan robot tasks.
Researchers at Robotics at Google described PaLM-E, a generalist model that feeds images, robot states and other sensor information into a large language model alongside text. Its biggest variant was built on the 540-billion-parameter PaLM and a 22-billion-parameter vision transformer, and it set a new best score on a visual question-answering benchmark without task-specific tuning. In tests with real robots it produced step-by-step plans that a lower-level control policy then carried out. The work showed how one model could connect language, perception and action, a foundation that conversational robots would later build on.
Source: Google Research Blog
This archive entry summarises reporting published elsewhere; follow the source link for the full story. The summary was drafted with AI from that reporting.