How Frontier AI Models Could Leapfrog Robotics Progress

How Frontier AI Models Could Leapfrog Robotics Progress

There’s been a lot of progress in robot models in the last few years. This work has made use of progress in language and multimodal models, including by using small vision-language models (VLMs) for language and image processing. But robotics progress is only weakly coupled to frontier model progress: robotics labs are mostly trying to scale their own models based on their own data. Because existing non-frontier VLMs are already sufficient at the semantic tasks they’re used for in robotics (like parsing language commands and identifying objects), the VLM is typically not the bottleneck to further progress. As a result, new frontier model releases don’t usually translate directly into better robot models.

However, it seems possible that future frontier models could generalize more strongly to robotics. Above a certain threshold of capability, they might be able to quickly enable a lot of progress in robot models, either by controlling robots themselves or by dramatically improving the way robot models are trained. Importantly, frontier models could do this in ways that bypass the current robotics scaling path (rather than just making the existing approach scale faster). This means that robotics scaling laws might underestimate how quickly robotics could improve.