{"id":76339,"date":"2025-08-18T01:51:21","date_gmt":"2025-08-18T01:51:21","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/76339\/"},"modified":"2025-08-18T01:51:21","modified_gmt":"2025-08-18T01:51:21","slug":"that-cheap-open-source-ai-model-is-actually-burning-through-your-compute-budget","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/76339\/","title":{"rendered":"That ‘cheap’ open-source AI model is actually burning through your compute budget"},"content":{"rendered":"

Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders. Subscribe Now<\/a><\/p>\n

A comprehensive new study<\/a> has revealed that open-source artificial intelligence models consume significantly more computing resources than their closed-source competitors when performing identical tasks, potentially undermining their cost advantages and reshaping how enterprises evaluate AI deployment strategies.<\/p>\n

The research, conducted by AI firm Nous Research<\/a>, found that open-weight models use between 1.5 to 4 times more tokens \u2014 the basic units of AI computation \u2014 than closed models like those from OpenAI<\/a> and Anthropic<\/a>. For simple knowledge questions, the gap widened dramatically, with some open models using up to 10 times more tokens.<\/p>\n

Measuring Thinking Efficiency in Reasoning Models: The Missing Benchmarkhttps:\/\/t.co\/b1e1rJx6vZ<\/a><\/p>\n

We measured token usage across reasoning models: open models output 1.5-4x more tokens than closed models on identical tasks, but with huge variance depending on task type (up to\u2026 pic.twitter.com\/LY1083won8<\/a><\/p>\n

\u2014 Nous Research (@NousResearch) August 14, 2025<\/a> <\/p>\n

\u201cOpen weight models use 1.5\u20134\u00d7 more tokens than closed ones (up to 10\u00d7 for simple knowledge questions), making them sometimes more expensive per query despite lower per\u2011token costs,\u201d the researchers wrote in their report published Wednesday.<\/p>\n

The findings challenge a prevailing assumption in the AI industry that open-source models offer clear economic advantages over proprietary alternatives. While open-source models typically cost less per token to run, the study suggests this advantage can be \u201ceasily offset if they require more tokens to reason about a given problem.\u201d<\/p>\n

AI Scaling Hits Its Limits<\/p>\n

Power caps, rising token costs, and inference delays are reshaping enterprise AI. Join our exclusive salon to discover how top teams are:<\/p>\n

Turning energy into a strategic advantage<\/p>\n

Architecting efficient inference for real throughput gains<\/p>\n

Unlocking competitive ROI with sustainable AI systems<\/p>\n

Secure your spot to stay ahead: https:\/\/bit.ly\/4mwGngO<\/a><\/p>\n

The real cost of AI: Why \u2018cheaper\u2019 models may break your budget<\/p>\n

The research examined 19 different AI models<\/a> across three categories of tasks: basic knowledge questions, mathematical problems, and logic puzzles. The team measured \u201ctoken efficiency\u201d \u2014 how many computational units models use relative to the complexity of their solutions\u2014a metric that has received little systematic study despite its significant cost implications.<\/p>\n

\u201cToken efficiency is a critical metric for several practical reasons,\u201d the researchers noted. \u201cWhile hosting open weight models may be cheaper, this cost advantage could be easily offset if they require more tokens to reason about a given problem.\u201d<\/p>\n

\"\"Open-source AI models use up to 12 times more computational resources than the most efficient closed models for basic knowledge questions. (Credit: Nous Research)<\/p>\n

The inefficiency is particularly pronounced for Large Reasoning Models (LRMs), which use extended \u201cchains of thought<\/a>\u201d to solve complex problems. These models, designed to think through problems step-by-step, can consume thousands of tokens pondering simple questions that should require minimal computation.<\/p>\n

For basic knowledge questions like \u201cWhat is the capital of Australia?\u201d the study found that reasoning models spend \u201chundreds of tokens pondering simple knowledge questions\u201d that could be answered in a single word.<\/p>\n

Which AI models actually deliver bang for your buck<\/p>\n

The research revealed stark differences between model providers. OpenAI\u2019s models, particularly its o4-mini<\/a> and newly released open-source gpt-oss<\/a> variants, demonstrated exceptional token efficiency, especially for mathematical problems. The study found OpenAI models \u201cstand out for extreme token efficiency in math problems,\u201d using up to three times fewer tokens than other commercial models.<\/p>\n

Among open-source options, Nvidia\u2019s llama-3.3-nemotron-super-49b-v1<\/a> emerged as \u201cthe most token efficient open weight model across all domains,\u201d while newer models from companies like Mistral showed \u201cexceptionally high token usage\u201d as outliers.<\/p>\n

The efficiency gap varied significantly by task type. While open models used roughly twice as many tokens for mathematical and logic problems, the difference ballooned for simple knowledge questions where efficient reasoning should be unnecessary.<\/p>\n

\"\"OpenAI\u2019s latest models achieve the lowest costs for simple questions, while some open-source alternatives can cost significantly more despite lower per-token pricing. (Credit: Nous Research)<\/p>\n

What enterprise leaders need to know about AI computing costs<\/p>\n

The findings have immediate implications for enterprise AI adoption, where computing costs can scale rapidly with usage. Companies evaluating AI models often focus on accuracy benchmarks and per-token pricing, but may overlook the total computational requirements for real-world tasks.<\/p>\n

\u201cThe better token efficiency of closed weight models often compensates for the higher API pricing of those models,\u201d the researchers found when analyzing total inference costs.<\/p>\n

The study also revealed that closed-source model providers appear to be actively optimizing for efficiency. \u201cClosed weight models have been iteratively optimized to use fewer tokens to reduce inference cost,\u201d while open-source models have \u201cincreased their token usage for newer versions, possibly reflecting a priority toward better reasoning performance.\u201d<\/p>\n

\"\"The computational overhead varies dramatically between AI providers, with some models using over 1,000 tokens for internal reasoning on simple tasks. (Credit: Nous Research)<\/p>\n

How researchers cracked the code on AI efficiency measurement<\/p>\n

The research team faced unique challenges in measuring efficiency across different model architectures. Many closed-source models don\u2019t reveal their raw reasoning processes, instead providing compressed summaries of their internal computations to prevent competitors from copying their techniques.<\/p>\n

To address this, researchers used completion tokens \u2014 the total computational units billed for each query \u2014 as a proxy for reasoning effort. They discovered that \u201cmost recent closed source models will not share their raw reasoning traces\u201d and instead \u201cuse smaller language models to transcribe the chain of thought into summaries or compressed representations.\u201d<\/p>\n

The study\u2019s methodology included testing with modified versions of well-known problems to minimize the influence of memorized solutions, such as altering variables in mathematical competition problems from the American Invitational Mathematics Examination (AIME)<\/a>.<\/p>\n

\"\"Different AI models show varying relationships between computation and output, with some providers compressing reasoning traces while others provide full details. (Credit: Nous Research)<\/p>\n

The future of AI efficiency: What\u2019s coming next<\/p>\n

The researchers suggest that token efficiency should become a primary optimization target alongside accuracy for future model development. \u201cA more densified CoT will also allow for more efficient context usage and may counter context degradation during challenging reasoning tasks,\u201d they wrote<\/a>.<\/p>\n

The release of OpenAI\u2019s open-source gpt-oss models<\/a>, which demonstrate state-of-the-art efficiency with \u201cfreely accessible CoT,\u201d could serve as a reference point for optimizing other open-source models.<\/p>\n

The complete research dataset and evaluation code are available on GitHub<\/a>, allowing other researchers to validate and extend the findings. As the AI industry races toward more powerful reasoning capabilities, this study suggests that the real competition may not be about who can build the smartest AI \u2014 but who can build the most efficient one.<\/p>\n

After all, in a world where every token counts, the most wasteful models may find themselves priced out of the market, regardless of how well they can think.<\/p>\n

Daily insights on business use cases with VB Daily<\/p>\n

If you want to impress your boss, VB Daily has you covered. We give you the inside scoop on what companies are doing with generative AI, from regulatory shifts to practical deployments, so you can share insights for maximum ROI.<\/p>\n

Read our Privacy Policy<\/a><\/p>\n

\n\t\t\t\t\tThanks for subscribing. Check out more VB newsletters here<\/a>.\n\t\t\t\t<\/p>\n

An error occured.<\/p>\n

\t\t\t\t\t\"\"\/<\/p>\n

\t\t\t