In AMD\u2019s latest bid to upset Nvidia’s dominance in AI hardware, the House of Zen has acquired AI chip company Taalas, which bakes model weights directly into silicon in a process that promises to boost inference performance by an order of magnitude or more.<\/p>\n
The deal, announced at market close on Thursday, appears to be framed in much the same context as Nvidia\u2019s $20 billion licensing deal <\/a>with Groq last December: make high-performance \u201cpremium\u201d inference services prized for AI agents, like code assistants, faster and cheaper to run. AMD didn\u2019t disclose the terms of the deal, but from what we understand, this is an actual acquisition rather than an acquihire.<\/p>\n Founded in 2023 and based in Toronto, Taalas\u2019 approach to inference is radically different from conventional GPUs or the dataflow architectures that underpin Groq LPUs or Cerebras’ waferscale accelerators. <\/p>\n A model-specific integrated circuit<\/p>\n The startup\u2019s chips don\u2019t rely on HBM to store the model weights but rather etch them directly into the silicon. In a sense, Taalas\u2019 chips are really model-specific integrated circuits or MSICs.<\/p>\n Perhaps more importantly, Taalas\u2019 tech isn\u2019t just conceptual. In February, the startup revealed<\/a> its first test chip fabbed on TSMC\u2019s 6nm process tech, which it called the HC1. Initial benchmarks saw the chip serve Meta\u2019s Llama 3.1 8B at a blistering 16,960 tokens a second \u2014 when announced last February, that was 48x faster than Nvidia’s GPUs and 8.5x faster than Cerebras’ accelerators.\u00a0<\/p>\n While Llama 3.1 is ancient by today\u2019s standards, having made its debut all the way back in mid 2024, the reticle-sized chip was really intended to prove the concept.\u00a0<\/p>\n Taalas has been incredibly secretive about how its chips actually work, but we know its processors are comprised of two main regions: the mask-ROM recall fabric where model weights are etched, and the SRAM recall fabric where KV caches and fine-tuning adapters are stored.<\/p>\n For its second-gen HC2 chip due out this summer, Taalas aims to boost parameter count to 20 billion parameters. That might not sound like much, but just like with GPUs for larger models, weights are simply distributed across multiple accelerators using pipeline parallelism.<\/p>\n At 20 billion parameters per chip, you\u2019d need just 50 accelerators to support a trillion-parameter model, and AMD just so happens to have a rack-scale compute platform and in-house system design team that can comfortably accommodate that.<\/p>\n That\u2019s quite a bit more space and power efficient than Nvidia\u2019s recently unveiled LPX systems, which would need a few dozen GPUs and at least 2,000 Groq LPUs<\/a> to serve the same model.<\/p>\n