Learning to Bridge Humanoid Robot Skills

1Sapienza University of Rome

Abstract

Policies learned for individual humanoid behaviors can now produce agile whole-body motions, but useful tasks often require several behaviors in sequence. Transitioning between independently trained skills, the source skill may leave the robot in a state from which the incoming skill cannot continue. This thesis investigates whether a single intermediate policy, called a bridge, can connect such skills while leaving them frozen. Given the robot’s recent motion, a short reference sequence from the incoming skill, and a time limit, the bridge must move the robot from the state left by one skill to a state from which the next can proceed.

We develop three bridge architectures, with each addressing limitations of the previous one: a goal-conditioned reinforcement-learning policy, a conditional variational autoencoder, and a diffusion planner paired with a universal tracker. The thesis describes all three architectures in detail; this page focuses on the third, which was the most promising and achieved the best results. The approaches use human-motion data retargeted to the robot. Quantitative evaluation focuses on the diffusion-based approach and three transitions between frozen skills of the Unitree G1, comparing it with direct switching and pair-specific transition policies. We also demonstrate the approach in long simulated tasks that chain many skills together.

Problem Definition

A skill is a controller that realizes a particular behavior. Each skill work reliably only from a set of states, called its initiation set. During a task, a high-level controller may require to switch to a different skill. At the request, the source skill has left the robot in state A, which may lie outside the initiation set of the incoming skill.

The bridge is a single policy shared across skill pairs. At a switch, it takes control and must bring the robot from A to a target state B before a specified deadline, after which the incoming skill takes control. The target is an entry point recorded during an execution of the incoming skill; the associated entry sequence shows how that skill continues from B. The bridge is conditioned on the source motion leading to A, this entry sequence, and the deadline. It does not observe the skills themselves and is trained on a separate motion corpus.

Timeline of a switch: source skill, bridge, incoming skill
Switch from walking to jumping. The source skill runs until a request at state A. The bridge controls the robot for h steps and hands over to the incoming skill at target state B.

Method

Motion Data

The bridge is trained on a separate corpus of kinematic trajectories and does not observe the skills it connects. The corpus is built from human motion capture in LAFAN1 and the CMU and KIT subsets of AMASS. We select locomotion and related motion, including walking, running, turning, stopping, crouching and jumping; AMASS intervals are selected using BABEL labels. We use GMR to retarget the selected motion to each robot and then ground it. A kinematic filter removes motion the bridge should not be required to reproduce, including long airborne phases and motions beyond the robot's kinematic limits. Motion graphs augment the corpus by stitching clips at similar frames. We divide the trajectories into shorter windows and mask a segment in the middle: frames before it form the history, and frames after it form the future. During training, the masked segment remains available as ground truth, so the bridge learns to generate the motion between the history and future. At deployment, the history is the motion the robot performed under the source skill, while the future is a recorded sequence from the incoming skill that describes its nominal motion after handover. The bridge generates the intervening motion. The future is a reference for the transition, not a guarantee that the robot will reproduce those states exactly; choosing a suitable future is a separate part of the problem.

Human poses next to the retargeted robot poses
Human poses from LAFAN1 (top) and AMASS (bottom), alongside their retargeted and grounded robot poses.
A stitched trajectory joining a walk and a jump
A stitched trajectory joins 0.4 s of walking motion to a jump clip; the difference between the segments is faded out over 0.2 s.

Diffusion Planner and Universal Tracker

The bridge separates trajectory generation from execution. A diffusion planner, trained on the motion corpus, receives the request and generates a complete kinematic trajectory from A to B by inpainting the frames between the history and future.

A universal tracker, trained with PPO to follow short kinematic trajectories, executes the plan on the simulated robot. It observes up to 32 control steps ahead; near the end of the bridge, this includes the incoming skill's entry sequence. It can account for small inconsistencies in the kinematic plan.

A plan of the diffusion planner from walking to jumping
A plan for a switch from walking to the crouch at the start of a jump, with a duration of 0.4 s. The planner generates the 20 control steps between A and B without first bringing the robot to rest.

Skills

Each skill is trained independently and kept frozen during the experiments. The skills are obtained using different methods: task-reward training, reference-motion tracking, or a combination of both.

Unitree G1

The walking skill follows commanded velocities and is trained with task rewards. The jumping skill tracks a 1.54 m forward jump that begins from a crouch. The kicking skill combines reference-motion tracking of a human football kick with rewards for ball contact. Four further skills track individual karate strikes: lunge punch, front kick, roundhouse kick and side kick.

Two additional reference-tracking skills cover rising from a seated position on the floor and sprinting forward. The sprint reference begins with the robot already in motion.

Walk
Jump
Kick
Lunge punch
Front kick
Roundhouse kick
Side kick

Booster T1

The walking skill follows a distance command and stops at the destination. The turning skill rotates in place to a commanded heading, and the pushing skill moves a box by a specified distance. All three are trained with task rewards and used in the Sokoban demonstration.

Walk
Push
Turn

Transitions

Each experiment evaluates one switch between two G1 skills under controlled conditions. The same bridge is used across all skill pairs.

Walking, Jumping and Kicking

An entry point from the incoming skill is placed ahead of the robot. The bridge is called at a specified distance from the entry point and is given a fixed deadline to reach it.

Walk to jump

The jump begins from a deep crouch, which the jumping skill has not entered from a walking state during training. The bridge brings the walking robot into the crouch and stops its forward motion at the take-off location.

Walk to kick

The entry point lies along the approach to the ball, while the kicking reference is already underway. Although the posture changes little, the robot's velocity and supporting-foot contact must match the entry sequence. The bridge therefore hands over control while the robot is still walking.

Jump to walk

The bridge takes control while the robot is still absorbing the landing and brings it to an entry point in the walking cycle.

Sit to sprint

The bridge takes control while the robot is rising from a seated position on the floor and brings it to an entry point in the moving sprint. The sprint skill then continues without resetting the robot.

Karate

Each clip below shows a transition between two individual karate skills. The bridge takes control as one strike ends and brings the robot to the starting pose of the next strike, placed at the robot's current location. The incoming skill then continues from that pose.

Lunge punch to front kick
Front kick to roundhouse kick
Roundhouse kick to side kick
Side kick to lunge punch

Comparison with Baselines

We compare the bridge with two baselines:

  • Instant switch (no-op): the incoming skill takes control as soon as the switch is requested, from the state left by the source skill.
  • Dedicated transition policy: the method of Lee et al., Composing Complex Skills by Learning Transition Policies (ICLR 2019). It keeps the skills frozen and inserts a learned controller between them. A separate policy is trained on executions for each skill pair, and that policy determines when to hand over control.

Walking, Jumping and Kicking

We measure the incoming skill's performance after handover relative to nominal executions from its own entry sequences. The bridge's performance depends on the distance to the entry point and the available duration. It succeeds in all 150 trials at the selected settings and keeps incoming-skill performance near nominal levels. Its transitions last 0.6 to 1.2 s before handover, compared with at most 0.18 s for those policies.

Walk to jump

Walk to kick

Jump to walk

At handover, the bridge's planar-velocity and yaw-rate errors are 0.098 m/s and 0.083 rad/s, close to nominal values of 0.089 m/s and 0.075 rad/s. Both errors are larger with direct switching (0.229 m/s and 0.222 rad/s) and the transition policy (0.315 m/s and 0.326 rad/s).

Karate

Each clip compares the bridge with instant switching for a different pair of karate skills.

Lunge punch to front kick
Front kick to roundhouse kick
Roundhouse kick to side kick
Side kick to lunge punch

Sitting to Sprinting

Both runs start from the same interrupted sit-to-stand motion. In one, control passes directly to the sprint skill; in the other, the bridge first moves the robot to the sprint entry point.

Sit to sprint

With direct switching, the robot falls shortly after the sprint skill takes control. The bridge reaches the sprint entry point, allowing the sprint skill to continue.

Demos

In each demonstration, a high-level controller selects the next skill and requests the switch.

Soccer, Unitree G1

The robot walks toward a fallen robot, jumps over it, proceeds to the ball and kicks it into the goal. A high-level controller follows this fixed sequence and places each entry point using geometric rules derived from the scene. The bridge connects the three skills while the robot remains in motion.

Karate, Unitree G1

The robot performs four karate strikes—a lunge punch, front kick, roundhouse kick and side kick—in sequence without being reset. After each strike, the bridge connects its closing guard to the opening guard of the next strike.

Sokoban, Booster T1

We deliberately use Sokoban as a simpler transition setting: the walking, turning and pushing skills begin from nearly stationary states and are generally compatible. This lets us demonstrate the bridge with a more complex high-level controller, a symbolic planner. The planner computes a valid solution to the maze before the robot moves; the controller then carries out this known plan one action at a time, using the three skills and requesting a bridge at each skill change. A purely reinforcement-learning approach would have to learn both how to solve the maze and how to execute its solution. The bridge primarily aligns the robot—for example, with a box before pushing—rather than enabling otherwise infeasible switches.