RoboTwin 2.0
50 tasks · Clean and randomized scenes. Compared with the published base-policy reference.
Training-Only Video Supervision
Future-video prediction improves a pretrained VLA during fine-tuning. At deployment, the video branch is removed and the original action pathway is retained.
01 / Overview
Future frames in robot demonstrations describe object motion and task progress beyond the action labels.
We turn those observations into an auxiliary learning signal for the shared VLA representation. The video branch is used only during training.
02 / Method
Two objectives shape a shared representation. Only the original action pathway is retained at deployment.
A trainable adapter connects the shared VLM to video cross-attention; a VACE condition network supplies current-frame visual information.
Flow-matching video loss is evaluated only on valid future latent blocks, alongside the original action objective.
The updated shared prefix and action expert retain the original action-generation pathway.
03 / Results
Improved action success after removing the video branch. Full benchmark comparisons are available below.
RoboTwin 2.0
50 tasks · Clean and randomized scenes. Compared with the published base-policy reference.
Real-world manipulation
Four tasks · 80 trials per policy. Matched demonstrations, training budget, and evaluation conditions.
RoboTwin 2.0 · LIBERO · LIBERO Plus
| Method / expert | RoboTwin Clean | RoboTwin Random | RoboTwin Average | LIBERO | LIBERO Plus |
|---|---|---|---|---|---|
| FastWAM | 77.76 | 1.92 | 39.84 | 97.60 | 52.56 |
| X-VLA | 67.96 | 20.95 | 44.46 | 98.10 | 71.40 |
| Robust-WAM | 81.20 | 34.40 | 57.80 | 97.90 | 58.90 |
| 4D-WAM | 81.50 | 41.80 | 61.65 | 98.60 | 71.01 |
| PrismWAM | 87.98 | 53.16 | 70.57 | 99.35 | 79.44 |
| π0.5 reference | 70.66 | 46.00 | 58.33 | 96.85 | 84.66 |
| π0.5 + DiT-2.2B (Random) | 84.84 | 57.52 | 71.18 | 98.05 | 85.15 |
π0.5 RoboTwin and standard LIBERO scores are official references; its LIBERO Plus score is the authors’ evaluation. RoboTwin Average equally weights Clean and Random; LIBERO averages four suites; Plus pools 10,030 scenarios. Cross-method scores follow the comparisons reported in the paper.
Five auxiliary video-expert configurations
| Method / expert | RoboTwin Clean | RoboTwin Random | RoboTwin Average | LIBERO | LIBERO Plus |
|---|---|---|---|---|---|
| Lightweight Transformer (Random) | 85.22 | 53.84 | 69.53 | 96.85 | 84.99 |
| DiT-5B (Pretrained) | 82.32 | 55.70 | 69.01 | 96.95 | 85.97 |
| DiT-5B (Random) | 83.64 | 51.24 | 67.44 | 97.60 | 85.12 |
| DiT-2.2B (Pretrained) | 81.34 | 54.74 | 68.04 | 97.80 | 84.96 |
| DiT-2.2B (Random) | 84.84 | 57.52 | 71.18 | 98.05 | 85.15 |
Random DiT-2.2B has the highest observed RoboTwin average among these configurations. The lightweight predictor leads on Clean, and pretrained DiT-5B leads on Plus. Configuration changes include architecture as well as capacity; these results do not isolate parameter count alone.
AgileX ALOHA · Four tasks · 20 trials per task and policy
| Task | Original π0.5 | Random DiT-2.2B |
|---|---|---|
| Stack Three Bowls | 35 | 75 |
| Hang Cup on Mug Tree | 10 | 15 |
| Place Wooden Block on Towel | 70 | 85 |
| Place Beverage and Lift Basket | 55 | 70 |
| Overall | 42.50 | 61.25 |
Both policies use the same demonstrations, training budget, instructions, and evaluation conditions. Each task has 100 demonstrations. Success requires completion within 30 seconds without human intervention; 34/80 trials succeed for the base policy and 49/80 for our variant.
04 / Real world
Four tasks on an AgileX ALOHA robot. Watch representative executions below.
Grasp and stack three bowls in sequence.
Open video ↗Coordinate both arms to grasp, transfer, and hang the cup.
Open video ↗Transfer the wooden block and place it on the towel.
Open video ↗Place the beverage in the basket, then grasp and lift the basket.
Open video ↗05 / Resources
Read the paper and cite this work.
@misc{cui2026futureprediction,
title={Future Prediction as Training-Only Video Supervision for Pretrained VLA Policies},
author={Hanwen Cui and Yuanyuan Li and Qing Guo and Zhida Zhang and Chao Chen and Xi Chen and Jie Cao},
year={2026}
}