The People Paid to Move Like Robots So Robots Can Learn to Move Like People

The cleaner wore a baseball cap. Mounted on the brim was a camera. Every wipe of a kitchen counter, every sweep under a couch cushion, every folded towel was recorded, and soon to be sold.

The cleaner worked for Shift, a New York City startup that on May 28 announced it would clean any apartment in the city for free. The offer, according to Understanding AI's reading of the company's scheduling site, was framed in the FAQ as "limited time" and may have been a publicity stunt. Either way, the data was the point: Shift was stockpiling first-person footage of household chores, the kind a robot would need to learn the same task.

That footage, multiplied across startups and continents, is becoming the raw material for the next wave of AI. Large language models were trained by predicting the next word across billions of tokens scraped from a web of text that already existed. Roboticists want the equivalent: an internet-scale corpus of everyday task demonstrations. It does not exist. The largest openly available dataset, ABC-130K, holds about 3,500 hours of recorded demonstrations, per the same reporting, three and a half orders of magnitude short of what the field treats as a useful training set. Someone has to perform every minute.

That someone, increasingly, is a paid human in a room being watched.

Three strategies have converged in the past year, and the same companies are running all of them. First: record humans doing tasks and convert the video into robot training data, the Shift model, but also the model of any startup that ships a camera to a cleaner, a warehouse worker, or a home cook. Second: hire humans to teleoperate robots directly, controlling arms and grippers from a remote station while the robot moves through a real kitchen, factory floor, or living room. Third: outfit humans with gloves, exoskeletons, or body rigs that force robot-like motion and capture richer data, including the wrist and finger positions a hat-mounted camera misses.

Scale AI, the data-labeling company, is reported to be experimenting with all three: recording humans, teleoperating robots, and piloting wearable rigs. The same article notes that Understanding AI's reporter, Timothy B. Lee, spent several months interviewing "dozens of founders, engineers, and robotics researchers" about the demonstration-data gap, and what he found was a field that has stopped pretending one tactic will work.

Some of that finding happened on the ground. In a University of Pennsylvania robotics lab, Lee tried collecting robot data himself and came away convinced that the manual, supervised nature of the work, the hours of hand-labeling, the failed takes, the precise resets between attempts, is the bottleneck the field does not yet have a tool to fix.

In a Chinese lab during the spring, he watched operators in VR headsets puppet humanoid robots (human-shaped machines built to stand, walk, and manipulate objects the way a person does) through ordinary tasks: opening a refrigerator door, sweeping a piece of trash into a dustpan, rearranging pillows on a couch. Each gesture by the human produced a paired robot trajectory, captured frame by frame and stored for later training. The lab was one of several in China using the same approach to scale up demonstration volume without sending a robot into a home.

By August, the work had a conference floor. The Actuate gathering in San Francisco hosted a notable density of data-collection startups, according to the same reporting, with founders describing parallel experiments in cap-cameras, teleoperation contracts, and wearable rigs. The clustering was the signal: the demonstration-data shortage is no longer a research problem, it is an industry.

The industry's stated end state is a data flywheel: robots reliable enough to operate mostly on their own, generating new training data as they perform useful work in the world. Once that flywheel turns, the cap-cameras and the VR puppeteers become transitional labor: paid, supervised, and slowly made redundant. Until then, every minute of robot training is a minute of human performance, on camera, on site, and one chore at a time.

The question to watch over the next twelve months is which of the three tactics scales. Teleoperation gives the cleanest data and the slowest throughput. Wearable rigs give the richest data and the highest equipment cost. Cap-cameras and free-cleaning stunts give the most footage and the least controlled scenes. The mix the industry settles on will determine whether the general-purpose robot arrives in 2028, 2030, or later, and whether the people doing the demonstrations today are remembered as a workforce or a footnote.