Save StorySave this storySave StorySave this story
Nvidia is the undisputed king of AI chips. But thanks to the AI it helped build, the champ could soon face growing competition.
Modern AI runs on Nvidia designs, a dynamic that has propelled the company to a market cap of well over $4 trillion. Each new generation of Nvidia chip allows companies to train more powerful AI models using hundreds or thousands of processors networked together inside vast data centers. One reason for Nvidia’s success is that it provides software to help program each new generation of chip. That may soon not be such a differentiated skill.
A startup called Wafer is training AI models to do one of the most difficult and important jobs in AI—optimizing code so that it runs as efficiently as possible on a particular silicon chip.
Emilio Andere, cofounder and CEO of Wafer, says the company performs reinforcement learning on open source models to teach them to write kernel code, or software that interacts directly with hardware in an operating system. Andere says Wafer also adds “agentic harnesses” to existing coding models like Anthropic’s Claude and OpenAI’s GPT to soup up their ability to write code that runs directly on chips.
Many prominent tech companies now have their own chips. Apple and others have for years used custom silicon to improve the performance and the efficiency of software running on laptops, tablets, and smartphones. At the other end of the scale, companies like Google and Amazon mint their own silicon to improve the performance of their cloud-computing platforms. Meta recently said it would deploy 1 gigawatt of compute capacity with a new chip developed with Broadcom. Deploying custom silicon also involves writing a lot of code so that it runs smoothly and efficiently on the new processor.
Wafer is working with companies including AMD and Amazon to help optimize software to run efficiently on their hardware. The startup has so far raised $4 million in seed funding from Google’s Jeff Dean, Wojciech Zaremba of OpenAI, and others.
Andere believes that his company’s AI-led approach has the potential to challenge Nvidia’s dominance. A number of high-end chips now offer similar raw floating point performance—a key industry benchmark of a chip’s ability to perform simple calculations—to Nvidia’s best silicon.
“The best AMD hardware, the best [Amazon] Trainium hardware, the best [Google] TPUs, give you the same theoretical flops to Nvidia GPUs,” Andere told me recently. “We want to maximize intelligence per watt.”
Performance engineers with the skill needed to optimize code to run reliably and efficiently on these chips are expensive and in high demand, Andere says, while Nvidia’s software ecosystem makes it easier to write and maintain code for its chips. That makes it hard for even the biggest tech companies to go it alone.
When Anthropic partnered with Amazon to build its AI models on Trainium, for instance, it had to rewrite its model’s code from scratch to make it run as efficiently as possible on the hardware, Andere says.
Of course, Anthropic’s Claude is now one of many AI models that are now superhuman at writing code. So Andere reckons it may not be long before AI starts consuming Nvidia software advantage.
“The moat lives in the programmability of the chip,” Andere says in reference to the libraries and software tools that make it easier to optimize code for Nvidia hardware. “I think it's time to start rethinking whether that's actually a strong moat.”
Besides making it easier to optimize code for different silicon, AI may soon make it easier to design chips themselves. Ricursive Intelligence, a startup founded by two ex-Google engineers, Azalia Mirhoseini and Anna Goldie, is developing new ways to design computer chips with artificial intelligence. If its technology takes off, a lot more companies could branch into chip design, creating custom silicon that runs their software more efficiently.
“We are going after the long poles of chip design—physical design and design verification,” says Mirhoseini, who is also an assistant professor at Stanford University, in reference to two of the main challenges involved with chip design.
Designing computer chips is one of the most consequential—and tricky—jobs on the planet. Chip engineers need to figure out how to arrange a vast number of components across a piece of silicon to optimize different functionality. After a chip is first designed, its performance has to be carefully tested and verified in an iterative process before the designs can be sent off to a foundry.
Nvidia’s designs are crucial for modern AI, with each new generation of chip allowing companies to train more powerful AI models using hundreds or thousands of processors networked together inside vast data centers.
Mirhoseini and Goldie developed a way for AI to optimize the layout of key components of computer chips while at Google. The approach transformed how Google designs its own processors, and it is now widely used in the industry to help arrange features on different chips.
Ricursive aims to go further, however, by automating more elements of chip design and integrating large language models into the process. The goal is to enable engineers to use natural language to describe changes or ask questions about a chip. Just as one can vibe code an app, perhaps eventually it will be possible to vibe design a chip.
Ricursive is still developing its technology, but Mirhoseini says the company has already shown that it can optimize more aspects of chip design.
The prospect of automating chip design in this manner has some investors salivating: Ricursive has raised $335 million at a $4 billion valuation in just a few months.
Goldie says it may ultimately be possible to have AI codesign both chips and algorithms to make them more powerful. She says that having AI tweak its own silicon and code could form a recursive kind of AI improvement. “We are moving into this new regime where we can just spend more compute to design faster and better chips—creating a kind of scaling law for chip design.”
What do you think of AI designing its own silicon? Leave a comment or send an email to [email protected] to let me know.
This is an edition of Will Knight’s AI Lab newsletter. Read previous newsletters here.
