OpenAI says it will deploy 750 megawatts worth of Nvidia competitor Cerebras’ dinner-plate sized accelerators through 2028 to bolster its inference services.<\/p>\n
The deal, which will see Cerebras take on the risk of building and leasing datacenters to serve OpenAI, is valued at more than $10 billion, sources familiar with the matter tell El Reg.<\/p>\n
By integrating Cerebras’ wafer-scale compute architecture into its inference pipeline, OpenAI can take advantage of the chip’s massive SRAM capacity to speed up inference. Each of the chip startup’s WSE-3 accelerators<\/a> measures in at 46,225 mm2 and is equipped with 44 GB of SRAM.<\/p>\n Compared to the HBM found on modern GPUs, SRAM is several orders of magnitude faster. While a single Nvidia Rubin GPU<\/a> can deliver around 22 TB\/s of memory bandwidth, Cerebras’ chips achieve nearly 1,000x that at 21 Petabytes a second.<\/p>\n All that bandwidth translates into extremely fast inference performance. Running models like OpenAI’s gpt-oss 120B, Cerebras’ chips can purportedly achieve<\/a> single user performance of 3,098 tokens a second as compared to 885 tok\/s for competitor Together AI, which uses Nvidia GPUs.<\/p>\n In the age of reasoning models and AI agents, faster inference means models can “think” for longer without compromising on interactivity.<\/p>\n “Integrating Cerebras into our mix of compute solutions is all about making our AI respond much faster. When you ask a hard question, generate code, create an image, or run an AI agent, there is a loop happening behind the scenes: you send a request, the model thinks, and it sends something back,” OpenAI explained in a recent blog post<\/a>. “When AI responds in real time, users do more with it, stay longer, and run higher-value workloads.”<\/p>\n However, Cerebras’ architecture has some limitations. SRAM isn’t particularly space efficient, which is why, despite the chip’s impressive size, they only pack about as much memory as a six-year-old Nvidia A100 PCIe card.<\/p>\n Because of this, larger models need to be parallelized across multiple chips, each of which are rated for a prodigious 23 kW of power. Depending on the precision used, the number of chips required can be considerable. At 16-bit precision, which Cerebras has historically preferred for its higher-quality outputs, every billion parameters ate up 2 GB of SRAM capacity. As a result, even modest models like Llama 3 70B required at least four of its CS-3 accelerators to run.<\/p>\n It’s been nearly two years since Cerebras unveiled<\/a> a new wafer scale accelerator, and since then the company’s priorities have shifted from training to inference. We suspect the chip biz’s next chip may dedicate a larger area to SRAM and add support for modern block floating point data types like MXFP4, which should dramatically increase the size<\/a> of the models that can be served on a single chip.<\/p>\n Having said that, the introduction of a model router<\/a> with the launch of OpenAI’s GPT-5 last summer should help mitigate Cerebras’ memory constraints. The approach ensures that the vast majority of requests fielded by ChatGPT are fulfilled by smaller cost-optimized models. Only the most complex queries run on OpenAI’s largest and most resource-intensive models.<\/p>\n