Sapienza pre-print splits a Unitree G1's audio stream into separate music and speech branches, each driving its own learned whole-body motion library

A Unitree G1 humanoid at Sapienza's ROCOCO lab listens to a continuous audio stream, decides whether it is music or speech, and routes each input to a different learned motion library. The robot picks coordinated arm, leg, torso, and head moves in real time, with no human selecting them.

The framework, detailed in an arXiv pre-print and on the ROCOCO lab's project page, splits the audio pipeline into two branches. The music branch uses audio fingerprinting and semantic embeddings to identify a track and align motion to its structure. The speech branch maps spoken input to a discrete library of imitation-learned skills, for direct human-robot interaction. Both branches feed a shared reinforcement-learning interface that schedules which skill to execute.

The authors report validation in simulation and on a single Unitree G1, a commercial humanoid used in many robotics labs, with consistent sim-to-real transfer. Code is on GitHub.

The work is a single-lab pre-print, and several limits shape the claim. The demonstration uses one commercial platform in a controlled setting, with no field deployment, no benchmark numbers in the visible abstract, and no comparison to other audio-driven humanoid systems. The dual-input split is the architectural contribution; independent reproduction, scaling beyond a single robot, and competitive placement remain unresolved.