AI chip startups collectively walked away with more than a billion dollars of new capital on Tuesday, showing that venture capitalists are still excited about the opportunity to challenge Nvidia’s dominance despite all the talk of an AI bubble.

MatX, which was founded in 2022 by Google engineers Reiner Pope and Mike Gunter, received the lion’s share of the cash. The startup raked in $500 million in a series B funding round led by VC firms Jane Street and Situational Awareness LP.

The startup aims to put out its first chip, an LLM-optimized accelerator called the MatX One, later this year. While many AI startups like Groq, dMatrix, and SambaNova have prioritized inference, Matx says its chip will do it all: pre-training, reinforcement learning, and inference prefill and decode.

Beyond this, concrete details on the chip remain thin. The company boasts the chip’s split systolic array will deliver the highest “FLOPS per mm2” and scale to “hundreds of thousands of chips.”

Speaking of scale, the company is certainly going to need plenty of silicon if it wants to cram the latest LLMs into SRAM.

Compared to the HBM used by AMD or Nvidia, SRAM is orders of magnitude faster. MatX expects its first chip will be able to deliver more than 2,000 tokens a second for a large 100-layer mixture of expert models.

The downside to SRAM is it isn’t very space-efficient. The largest dies today can only fit a few hundred megabytes of the stuff and still have room for compute.

MatX isn’t the first to go down this road. Cerebras got around this constraint by building a wafer-sized chip, while for Groq it was a numbers game: Want to run a bigger model? Just add (hundreds) more chips. MatX appears to be borrowing that same strategy.

However, unlike Groq or Cerebras, MatX will also use HBM — just not to store model weights if it can avoid it. Instead, MatX says, the HBM will be used to store the model’s key-value (KV) caches, which keep track of a model’s states across sessions. (You can think of KV caches as the model’s short term memory.)

By combining SRAM and HBM in this way, MatX believes its chip will be able to achieve both the sheer throughput of GPUs and the speed of SRAM-based designs. And now it’s got more money to try and prove it out.

Axelera scores $250M to scale AI from the edge to the datacenter

Also on Tuesday, Dutch startup Axelera announced it had raised a quarter of a billion dollars in a new funding round led by Innovation Industries to advance the development of its low-power RISC-V based AI accelerators.

Unlike MatX, which is going straight for Nvidia’s jugular, Axelera’s ambitions are far more modest. Its Europa and Metis AI accelerators are designed primarily for power-constrained edge workloads such as computer vision and robotics.

But by focusing on the edge first, the company aims to develop a compute architecture that can scale efficiently to take on any AI/ML task whether it’s running at the edge or in the datacenter.

The company has already shown progress toward this end. Axelera’s latest chip, Europa, boasts up to 629 TOPS of INT8, fed by 64GB of DRAM good for 200 GB/s of bandwidth. In terms of compute, this puts it on par with an Nvidia A100 while using less than a sixth the power at 45 watts. Having said that, it still trails the nearly six-year-old accelerator in memory capacity (80GB of HBM2E) and bandwidth 2 TB/s.

And that’s for a chip that’s still primarily designed for the edge. The company is working on a new chip designed to provide even greater performance. That chip, codenamed Titania, is being developed in partnership with the EU’s EuroHPC Digital Autonomy with RISC-V in Europe (DARE) program, which sees a domestic alternative to US chips for supercomputing.

SambaNova clinches $350M amid tie up with Intel

Finally, SambaNova received a $350 million cash infusion from Vista Equity, Cambium Capital, and Intel’s investment fund to bring its next-gen dataflow accelerators to market.

We’ve got all the details here, but in short, the funding was announced alongside a multi-year collab that’ll see the chip startup cram Chipzilla’s Xeons into its AI servers.

The company also disclosed a new AI accelerator, the SN50, which will be deployed by SoftBank in its Japanese datacenters starting later this year. ®