Benchmark
RoboDojo – Real
Real-world builds of ten tasks from the RoboDojo simulation benchmark, two each for generalization, memory, precision, long-horizon and instruction-conditioned manipulation, run on a bimanual station and scored by milestone.
Read the task definitions10 tasks · setups, instructions and milestone scoringGet the demonstrations on Hugging Face500 episodes · 10 tasks · LeRobot · CC BY 4.0
Leaderboard
#ModelMethodScore
1
Fable 5.2 High FastZero-shotLLM38.5
2
Dream-1Fine-tunedWAM22.0
3
π0.5Fine-tunedVLA20.6
π0.5Fine-tunedVLA20.64
Dream-1Zero-shotWAM8.5
By task
| Task | ||||
|---|---|---|---|---|
| Generalization | ||||
| 39.0 | 54.5 | 51.0 | 18.5 | |
| 15.5 | 13.0 | 11.5 | 10.0 | |
| Memory | ||||
| 64.0 | 9.0 | 7.0 | 1.5 | |
| 23.0 | 5.0 | 1.0 | 4.0 | |
| Precision | ||||
| 37.0 | 17.0 | 9.0 | 0.0 | |
| 37.0 | 14.5 | 16.0 | 10.5 | |
| Long-Horizon | ||||
| 62.5 | 10.0 | 38.0 | 21.0 | |
| 31.0 | 35.0 | 15.0 | 10.0 | |
| Instruction-Conditioned | ||||
| 40.0 | 41.0 | 21.5 | 1.0 | |
| 36.5 | 21.0 | 36.0 | 8.5 | |
Zero-shotFable 5.2 High Fast
Fine-tunedDream-1
Fine-tunedπ0.5
Zero-shotDream-1
Zero-shot
Fable 5.2 High Fast
Episode 1 of 10
15/100
Recording
Download data · 833 MBInstruction
Place the car, shoe, toothbrush, and dual-face hammer into the open cardboard box. Point the car nose, shoe toe, toothbrush head, and hammer's yellow striking face to the left.
Layout
Scoring
| Score | Milestone | This episode |
|---|---|---|
| 0 | No milestone below is reached. | ✓ |
| 5 | A task object is grasped and lifted clear of the table. | ✓ |
| 15 | One object is packed and the gripper is open. | Reached |
| 30 | Two objects are packed. | |
| 45 | Three objects are packed. | |
| 60 | All four objects are packed. | |
| 80 | All four objects are packed and at least two are oriented. | |
| 100 | All four objects are packed and oriented, and the box is within its initial footprint. |