After spending years racing to build ever-larger AI models, researchers and infrastructure providers are increasingly focused on a new problem: how to make those systems affordable enough to deploy at scale.
Sara Hooker, cofounder and CEO of startup AI lab Adaption, told the audience at Fortune Brainstorm Tech on Tuesday that most of today’s AI is what she called “monolithic”—or stuck in time. That is, once a model is trained, the model’s knowledge and capabilities are essentially fixed. If something changes in the world, or if the model learns something useful from users, that knowledge doesn’t automatically become part of the model.
“You need models that can evolve,” she explained, “otherwise you end up with massive inefficiencies.”
Still, for now, scale does matter—and the biggest models are not going away anytime soon, said Rodrigo Liang, CEO of AI chip company SambaNova, though there will be “plenty of room for more efficient models to come in.” For the time being, he explained, customers are left to struggle with the cost of scaling models; with energy-hungry infrastructure; and with finding enough AI chops.
But Hooker focused on what’s next, saying that we’re at an “inflection point with massive urgency to change that curve” or model size. Most people, she explained, intuitively understand that you shouldn’t just apply the same model to all problems. “Probably 90% of problems are very easy—many things that you do in bulk processing, for example, you shouldn’t be throwing a massive model at.”
She argued that future AI systems will need to adapt continuously to new information and rapidly change their behavior, rather than relying on repeated calls to a fixed model—a dynamic she said is contributing to the soaring API bills many companies are now experiencing. Today’s enterprises are deploying agents at scale, but those agents often aren’t learning from their mistakes, so companies are paying repeatedly—in compute, API calls, and infrastructure costs—for the same errors.
While model developers like Hooker are focused on building more capable and efficient AI systems, Liang said the industry’s immediate challenge is running today’s massive models efficiently enough to make real-world deployments economically viable. He argued that trillion-parameter models remain too expensive and power-hungry, and said SambaNova’s strategy is focused on delivering faster inference with lower power consumption through hardware specifically designed for large-model workloads.
“We’re getting two to 3x better than the [Nvidia] Blackwells [GPUs] on the exact same models, and so we think that at scale that’s the way to at least bring the cost down,” he said.
More from Fortune’s 25th Brainstorm Tech:
‘Not an Allbirds Moment’: Xbox’s new CEO says she is grounding the console in gaming roots not AI
