The brain moved before the body did

Until recently, teaching a robot a task meant writing it. An engineer described positions, forces and conditions, tested them, and shipped a program that did exactly one thing in exactly one place. Change the part, the lighting or the table height, and someone had to come back.

Robot foundation models replace the program with a policy: a neural network that takes camera images and the robot's own joint positions and outputs the next motion, twenty to fifty times a second. The policy is trained, not written, and what it is trained on is the interesting part.

Three sources of experience

Teleoperation. A person wears a headset and controllers, drives the robot through a task, and the whole episode is recorded: what the cameras saw, what the joints did. This is the highest-quality data and the most expensive, because it runs in real time on real hardware. Companies operate rooms of teleoperators purely to generate it.

Human video. There is an enormous amount of footage of people doing physical work. A model can learn from it what a task looks like and roughly how it decomposes, even though a human arm and a robot arm have different kinematics. This gives breadth cheaply and precision never.

Simulation. A physics engine can run thousands of copies of a robot in parallel, faster than real time, with randomised friction, mass, lighting and object shape. A policy trained across that randomisation learns to be robust rather than tuned, which is why sim-to-real transfer works at all. What simulation cannot model well is contact: the exact behaviour of a soft object, a sticky surface or a screw thread biting.

Why one model for many bodies matters

The older generation of learned controllers was trained per robot. A policy for one arm was useless on another with different link lengths. Current models take the robot's own configuration as part of the input, which means data collected on one machine improves behaviour on another. That turns a fleet into a single learning system, and it means the value of a robot company compounds with the number of hours its machines spend working, not with the number it sells.

It also explains a strategic move that confused people: several robot makers publish their models or their datasets. In a world where data compounds, a common data format is worth more than a proprietary one.

What these models are good and bad at

Good: tasks with visual variation and a forgiving tolerance. Sorting mixed items, placing objects on a shelf, opening a familiar type of door, folding a towel badly.

Bad: anything requiring precision beyond a millimetre or so, long sequences where an early error compounds, and situations outside the training distribution. A policy that has never seen a spill does not know that a floor can be slippery. It will not reason its way there; it will simply act as if the floor were dry.

The failure mode is characteristic and worth understanding: these models fail confidently and smoothly, executing a wrong motion with the same fluency as a right one. There is no hesitation to detect.

The compute question

Training a robot policy is cheap compared with training a frontier language model. Running one is harder than people expect, because it has to run on the robot, at low latency, on a power budget that competes with the motors. This is the same constraint that drives AI on the device, and it is why robot brains tend to be a small fast model on board with a larger one consulted occasionally.

Split architectures are now standard: a slow high-level model that decides what to do next in language-like steps, and a fast low-level policy that turns each step into motion. The high-level model can live in a data center. The low-level one cannot.

What to watch

Watch for results reported on tasks the team did not choose, in buildings they do not control. Watch for the first credible number on how much a task costs to teach, in teleoperation hours. And watch whether a policy trained on one company's fleet transfers to another company's robot, because that is the moment this becomes an industry rather than a set of projects.