From specialized perception models to unified VLAs like PI-0.5 - each transition moved where the hard engineering problems live.
The physical AI stack has gone through three distinct eras in a relatively short period. If you're working in robotics, autonomous vehicles, or any domain where models need to interact with the physical world, the arc matters. Not as history for its own sake, but because each transition redefined what engineering teams build themselves vs. what they inherit - and the latest shift is changing the infrastructure requirements in ways most teams aren't prepared for.
The perception era: many models, many pipelines
In the first era, the stack was a collection of specialized perception models. Object detection, semantic segmentation, depth estimation, lane detection - each had its own model, its own training pipeline, its own data requirements. The planning layer was hand-coded, consuming structured outputs from the perception stack and making decisions through rules and state machines.
The engineering effort lived in the perception layer. Each model needed its own dataset, its own evaluation metrics, its own deployment pipeline. A team might maintain a dozen models simultaneously, each with different training cadences and failure modes.
Where the work lived
Model teams owned individual components. System integration was a separate discipline, often undervalued. Data teams were fragmented across model boundaries.
The multimodal transition: fewer models, harder integration
The second era arrived when vision-language models proved that a single model could handle multiple perception tasks. Instead of a dozen specialized models, teams could use one or two foundation models for most perception needs. The model itself became less of the engineering challenge.
But the hard problem shifted. The gap between "model produces good outputs" and "robot does the right thing" was enormous. Connecting a VLM's semantic understanding to actual motor control required careful system design.
Where the work lived
System integration became the primary engineering challenge. Model teams shrank or merged. The new bottleneck was the interface between understanding and action.
The VLA era: end-to-end, new bottlenecks
Vision-Language-Action models like RT-2, Octo, and PI-0.5 collapse the entire stack. One model goes from raw observation (camera frames, proprioception) to motor actions. No separate perception layer. No hand-coded planner. The model is the system.
This sounds like it should make everything simpler. In some ways it does - fewer integration boundaries, fewer failure modes to debug across components. But it creates new problems that most teams aren't set up to handle.
The data problem explodes
VLA training data is heterogeneous in ways that LLM text data isn't. Video frames, sensor readings, action labels, language instructions - all need to be aligned, preprocessed, and served to the GPU. This is CPU-heavy work, and it doesn't scale linearly with GPU count.
Where the work lives now
Training infrastructure. Data pipeline engineering. The model architecture is increasingly commodity - the competitive advantage is in the training data and how fast you can iterate. Teams that haven't invested in training infrastructure are discovering it's now their primary bottleneck.