How an artificial intelligence (AI) model performs is critically important to the enterprises investing in and deploying the technology. But there\u2019s debate whether advancements in large language models are slowing.<\/p>\n
This debate centers around AI scaling laws.<\/p>\n
Popularized by OpenAI, the concept behind AI scaling laws is simply this: Larger models trained on more compute will yield better performance. OpenAI\u2019s 2020 paper<\/a>, \u201cScaling Laws for Neural Language Models\u201d\u00a0was the first influential paper to demonstrate scaling laws.<\/p>\n Google DeepMind\u2019s 2022 paper<\/a>, \u201cTraining Compute-Optimal Large Language Models\u201d\u00a0added a crucial insight: It demonstrated that data, instead of model size, and compute were the two key factors influencing a model\u2019s performance. Its Chinchilla model, which was less than half the size of GPT-3 but with four times more data, outperformed GPT-3.<\/p>\n \u201cOver the past few years, AI labs have hit on what feels like a winning strategy: scaling more parameters, more data, more compute,\u201d said Garry Tan<\/a>, president of startup accelerator Y Combinator<\/a>, on its video series<\/a> YC Decoded. \u201cKeep scaling your models and they keep improving.\u201d<\/p>\n But there are signs that initial leaps in performance are decelerating.<\/p>\n The two main fuels for scaling \u2014 data and computing \u2014 are becoming scarcer and more expensive, wrote Adnan Masood<\/a>, UST\u2019s<\/a> chief architect of AI and machine learning, in a blog post<\/a>. \u201cThese trends strongly indicate a plateau in the current trajectory of large language models.\u201d<\/p>\n For example, in knowledge quizzes, math word problems and coding tests, improvements are \u201cflattening,\u201d Masood said. He noted that in the knowledge quiz MMLU benchmark, GPT-3 recorded 43.9% and doubled to 86.4% for GPT-4 in 2023 but has since plateaued at 90% in 2024.<\/p>\n \u201cIf the old scaling laws are beginning to lose their edge, what comes next?\u201d Tan asked.<\/p>\n The answer from both Tan and Masood is that scaling laws are changing. AI model performance is still advancing, but it\u2019s now due to new techniques and not just making data and computing power larger.<\/p>\n That\u2019s why OpenAI introduced its o1 and o3 AI reasoning models after the GPT-series, Tan said. For the o, or omni, models, OpenAI used \u201cchain of thought\u201d techniques to get the model to think through its answers. The result was improved performance. (OpenAI skipped naming the model o2 because it was the name of a telecom provider.)<\/p>\n \u201cOpenAI researchers found that the longer o1 was able to think, the better it performed,\u201d Tan said. \u201cNow, with the recent release of its successor, o3, the sky seems to be the limit for this new paradigm of scaling LLMs.\u201d<\/p>\n Tan said o3 \u201csmashed benchmarks that were previously considered far out of reach for AI.\u201d<\/p>\n In the U.S., leading AI models retain the usage top spot for only about three weeks before being overtaken \u2014\u00a0especially by open\u2011source competitors, according to a June 2025 report<\/a> from Innovation Endeavors. The model cycle release remains fast \u2014\u00a0if no longer exponential. <\/p>\n Scaling laws are not yet dying, but the AI community is preparing for a future that emphasizes smarter architectures, reasoning\u2011driven models, and use of distributed data sources.<\/p>\n Read more: <\/p>\n What Businesses Should Ask When Choosing an AI Model<\/a><\/p>\n Court Rules Anthropic Doesn\u2019t Need Permission to Train AI With Books<\/a><\/p>\n