VLA fine-tuning breaks vertical scaling. The bottleneck is data loading, not compute.
VLA data is heterogeneous and CPU-heavy
VLA training data is heterogeneous - video frames, sensor readings, action labels, language instructions. Not simple text tokens.
- Each sample requires heavy preprocessing: decode video, align modalities, normalize formats
- This is CPU-bound work that happens before the GPU sees anything
- More GPUs = more demand on the same data pipeline = bottleneck gets worse, not better
More GPUs makes the bottleneck worse
Standard approach: bigger node, more GPUs per node.
- GPU utilization drops as you add GPUs - they're waiting on data
- 8-GPU node might see 40-50% utilization because CPUs can't feed them fast enough
- Throwing money at bigger machines makes the problem more expensive, not smaller
Decouple CPU and GPU scaling
Decouple CPU and GPU scaling. Scale them independently.
- Distributed data loading: dedicated CPU workers that preprocess and stream to GPU nodes
- Scale CPU workers based on data complexity, GPU workers based on model size
- Data pipeline becomes its own service, not a sidecar on the training node
- GPU utilization goes back up because the bottleneck moves to where you can address it
Training infra is now a competitive advantage
- Any team fine-tuning VLAs on proprietary data (robotics, autonomous vehicles, embodied AI)
- The foundation model is increasingly commodity - competitive advantage is in the fine-tuning data and how fast you can iterate on it
- Training infrastructure becomes a hiring priority for teams that haven't thought about it