Chinese fintech giant Ant Group has open-sourced an inference framework for a relatively new type of artificial intelligence model that it said could make AI systems more efficient, surpassing a framework proposed by researchers at US chipmaking giant Nvidia.<\/p>\n
The Alibaba Group Holding affiliate said Monday that its framework, dInfer, was designed for diffusion language models \u2013 a newer class of models that generate outputs in parallel, unlike \u201cautoregressive\u201d systems used in large language models (LLMs) such as ChatGPT, which produce text sequentially from left to right. Alibaba owns the Post.<\/p>\n
Diffusion models are already widely used in image and video generation.<\/p>\n
Ant claimed dInfer was up to three times faster than vLLM, an open-source inference engine developed by researchers at the University of California, Berkeley, and 10 times faster than Nvidia\u2019s own framework Fast-dLLM.<\/p>\n
Autoregressive language models, from OpenAI\u2019s GPT-3.5 to DeepSeek\u2019s R1, have so far powered the chatbot boom due to their strength in understanding and generating human language. But researchers, especially in academic circles, have continued exploring diffusion language models in search of greater capabilities.<\/p>\n
Ant\u2019s push into alternative model paradigms underscores how China\u2019s Big Tech firms are stepping up efforts in algorithm and software optimisation to offset the country\u2019s disadvantages in advanced AI chips.<\/p>\n
According to internal tests on Ant\u2019s own diffusion model LLaDA-MoE, dInfer generated an average of 1,011 tokens per second on a code-generation benchmark called HumanEval, compared with 91 tokens per second for Nvidia\u2019s Fast-dLLM and 294 for Alibaba\u2019s Qwen-2.5-3B model optimised with vLLM.<\/p>\n","protected":false},"excerpt":{"rendered":"Chinese fintech giant Ant Group has open-sourced an inference framework for a relatively new type of artificial intelligence…\n","protected":false},"author":2,"featured_media":80268,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[365,363,364,2300,138,607,219,4431,263,111,43,139,69,2114,14095,2022,145,3040,377],"class_list":["post-80267","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-asia","tag-business","tag-china","tag-economy","tag-hong-kong","tag-lifestyle","tag-new-zealand","tag-news","tag-newzealand","tag-nz","tag-opinion","tag-south-china-morning-post","tag-sport","tag-technology","tag-us","tag-world"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/80267","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/comments?post=80267"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/80267\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media\/80268"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media?parent=80267"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/categories?post=80267"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/tags?post=80267"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}