Partnership and scope

Boston Dynamics dates the collaboration to October 2024, when it disclosed work with Toyota Research Institute aimed at making Atlas a more general-purpose humanoid. The collaboration uses Large Behavior Models, or LBMs, that combine broad knowledge with task-specific training. Boston Dynamics says policies trained across varied tasks can generalize and recover better than policies built for only one or a few tasks.

How Atlas learns from people

Operators wearing motion-tracking equipment demonstrate movements and manipulation tasks by controlling Atlas directly. Boston Dynamics describes the mapping between the operator and robot as one to one.

The earlier Atlas software joined model-based control, planning and separate perception models. According to Boston Dynamics, adding a task to that system generally required a robotics engineer or integrator to change code. The newer approach replaces most of that stack with one neural network trained on demonstrations, so each task does not need its own program.

The model receives camera images, proprioceptive data and a natural-language task description. Its outputs use the same command interface as the human-operated system. Training demonstrations span many tasks and sometimes multiple robot embodiments, which lets one policy learn several behaviors. Boston Dynamics says the model associates language prompts with those behaviors, giving it a chance to carry out instructions written in English.

The teleoperation interface supports full-body demonstrations such as stepping, balancing on one foot, reaching shelves, and throwing or catching objects. The team has also collected movements that demand coordinated posture changes, including reaching between the robot's legs. A later virtual-reality setup added two foot trackers so Atlas could match an operator's stance, support area and intent to step.

Policy and training pipeline

Boston Dynamics says its Atlas policy builds on TRI's LBM work. The company describes a 450-million-parameter diffusion transformer trained with a flow-matching objective. The policy uses images, proprioception and language prompts to produce commands for the full robot at 30 Hz.

Camera frames from the head and proprioception make up the policy's observations. Its outputs specify targets for both grippers, the neck, torso, hands and feet. This representation also works with the Atlas Manipulation Test Stand, an upper-body platform that shares Atlas hardware and software but omits the torso and lower-body components. Boston Dynamics says that common interface lets it pool training data from the two embodiments.

The company divides policy development into four stages: collect demonstrations on hardware and in simulation; process and label the data; train a network across the task set; and evaluate it on a test suite. The training mix includes data from full Atlas, the Manipulation Test Stand and TRI's Ramen dataset. Boston Dynamics also uses quality-assurance tools to review demonstrations, filter them and send feedback to operators.

Simulation serves several roles in the company's account: refining the teleoperation system, running unit and integration tests, training and evaluating policies, and supplying co-training data. Scott Kuindersma of Boston Dynamics also said simulated rollouts and reinforcement-learning policies could expose the LBM to motions beyond human teleoperation; he reported that simulated co-training data had already improved performance on the physical robot.

What the demonstrations show

The two research teams have built end-to-end, language-conditioned policies for long manipulation sequences. Boston Dynamics says the policies use Atlas's ability to step, place its feet precisely, crouch, shift its center of mass and avoid colliding with itself.

In the Spot Workshop sequence, Atlas completes three prompted subtasks. It moves folded Spot legs from a cart to a shelf. It places face plates into a bin retrieved from a lower shelf. It then transfers other Spot parts from a blue bin into a tilt truck.

The first policy versions did not recover when something unexpected happened, such as a dropped part or a closing bin lid. After the team added recovery demonstrations and retrained the network, Boston Dynamics said the policies handled those cases without algorithm or engineering changes.

A separate result applies specifically to the upper-body Atlas Manipulation Test Stand. With one language-conditioned policy, that platform handled tasks including tying rope, turning over a barstool, spreading a tablecloth and moving a 22 lb (9.9 kg) tire.

Boston Dynamics reports that it can raise policy execution speed by roughly 1.5 to 2 times with little performance loss on both the test stand and full Atlas. The company cautions that task dynamics can prevent that speed-up; its narrower conclusion is that some cases may exceed the pace captured through human teleoperation.

Hardware and research direction

Atlas has 78 degrees of freedom. Each gripper contributes seven, and two head-mounted HDR stereo cameras provide visual input for both operators and learned policies.

The teleoperation system combines Atlas's model-predictive controller with a custom virtual-reality interface. Operators receive stereo camera views plus robot-state information, tactile feedback and heads-up displays. The original setup used a headset, base stations, hand controllers and a chest tracker before the team added foot tracking for mobile manipulation.

TRI's Russ Tedrake said human demonstrations allow skills to be added quickly and that stronger LBMs can need fewer demonstrations for more robust behavior. Boston Dynamics lists tactile gripper-force control, faster dynamic manipulation, cross-embodiment data and vision-language systems for more open-ended reasoning among its next research directions.

Sources

This article was researched and fact-checked against the following sources: