Source (Nov 2024): Lauren Ay - Advancing Robotics with VLA Models (Prelim Exam Talk)
From fixed skill libraries to single unified policies
- SayCan (early): predefined skill library with a trained policy and value function per skill. Doesn't scale - every new skill needs its own policy.
- RT-1: single transformer policy, 700 skills, 130K teleoperated episodes from 13 robots. 97% success rate. But actions are discretized into fixed bins.
- RT-2: co-finetune a VLM on internet-scale VQA data plus robot data. Actions still discrete tokens, but generalization jumps. Closed source.
- OpenVLA: open-source 7B model beat RT-2's 55B. Most of the gain came from better vision encoding (DINOv2 + SigLIP).
Diffusion and flow matching replaced discrete action bins with continuous control
- Octo: first to use denoising diffusion as the action head. Captures multimodal action distributions, outputs continuous control. Decouples transformer from action head.
- DexVLA: scaled the diffusion action expert to 1B params with multi-headed design per embodiment. Learned new skills on unseen robots with fewer than 100 demos.
- pi0: swapped diffusion for flow matching - learns a vector field via ODE instead of stochastic denoising. Faster inference, same smooth trajectories.
Hierarchical architectures decouple reasoning from execution
- pi0.5: pre-trains with discrete actions (faster), post-trains with flow matching for continuous output. Predicts high-level subtask first, then conditions actions on it. Better zero-shot generalization to unseen environments.
- HiRobot: System 1 (fast motor execution via VLA) and System 2 (deliberative reasoning via VLM). Handles mid-task user corrections and open-ended prompts.
- NaVILA: VLA as high-level reasoner outputting mid-level commands ("move forward 75cm"), separate RL locomotion policy for execution. Same VLA works across humanoid and quadruped by swapping the low-level controller.
Chain-of-thought reasoning before action improves success and enables human correction
- Visual CoT: model generates a subgoal image, conditions its action sequence on that image. Closed-loop. Beat baselines on the emerging LIBERO benchmark.
- Language CoT: model reasons explicitly through bounding boxes and motion primitives before executing. A single human correction during execution boosted task success by 48%.
- Open question: chain-of-thought helps, but the field still lacks standardized evaluation. Benchmarks are forming but not settled.