The next phase of global AI competition will hinge less on who can scale today\u2019s transformer models the fastest but more on who can reinvent the architecture to deliver comparable capability at a fraction of the power cost, according to a top United States scientist.\u00a0<\/p>\n
In simple terms, a transformer is the core AI architecture that learns patterns in vast amounts of data by weighing relationships between words or symbols, while a large language model (LLM) is a transformer trained at scale to generate and reason with human-like text. Popular LLM examples include ChatGPT and DeepSeek.<\/p>\n
\u201cI would like to see alternatives to the transformer model to give us this kind of thinking without the high energy use that we have now,\u201d\u00a0Jennifer Chayes, dean of the College of Computing, Data Science, and Society at the University of California, Berkeley, told Asia Times in an interview in Hong Kong.<\/p>\n
\u201cIt\u2019s very basic mathematical questions that will underlie that,\u201d she said. \u201cNobody knows exactly what to do there. People are trying different approaches.\u201d<\/p>\n
She added that some of the world\u2019s best computer scientists are spending time grappling with this question because of its enormous societal stakes, particularly around energy consumption and climate change, but acknowledged that these researchers are reluctant to devote themselves fully to such a difficult problem for fear that years of effort could yield little return and jeopardise their careers.<\/p>\n
Chayes praised China\u2019s DeepSeek for using \u201cknowledge distillation\u201d methods to train its AI models, noting that the process consumes far less energy than traditional AI training.\u00a0<\/p>\n
\u201cThat was very clever. That\u2019s an innovation,\u201d she said, adding that\u00a0distillation techniques are now being applied beyond mainstream AI development and into scientific research, including her own work in chemistry and materials science.<\/p>\n
\u201cNo matter how much money you have, you can\u2019t generate enough chemistry data to make a foundation chemistry model,\u201d she said. \u201cSo, how do you distill and post-train your AI models? How do you integrate the training with experiments in cases where there is very sparse data? These are huge foundational questions.\u201d<\/p>\n
Knowledge distillation \u2013 or, simply, distillation \u2013 is a commonly used AI training technique. It can be understood as a student who keeps asking questions to a knowledgeable teacher.\u00a0At some point, the student will be as smart as the teacher.<\/p>\n
On January 22, 2025, a group of DeepSeek researchers published<\/a> a paper stating that the training of the DeepSeek\u2011R1 model relied on distilled data from Alibaba\u2019s Tongyi Qianwen (Qwen) and Meta\u2019s Llama models. The team said the total training cost of DeepSeek\u2011R1 was about US$5.58 million, roughly 1.1% of the estimated US$500 million spent on training Meta\u2019s Llama 3.1.\u00a0<\/p>\n Brain drain in the US<\/p>\n Commenting on US chip export controls on China, Chayes said she did not believe the restrictions had had a negative impact on Chinese researchers. Instead, she argued that the pressure may have encouraged scientists in China to pursue more innovative ways to increase computing power and efficiency.<\/p>\n \u201cIf you\u2019re under pressure, you are going to make greater breakthroughs. And DeepSeek certainly did that,\u201d she said. \u201cI see that people are getting more computing power at Tsinghua University than they\u2019re being given at US universities.\u201d<\/p>\n At the same time, Chayes noted that many American universities face a different challenge when competing with Chinese counterparts, as they continue to lose talent to large technology companies, a dynamic that can leave academic researchers feeling disadvantaged in access to people and resources.<\/p>\n The debate over export controls has played out against a shifting policy backdrop. Reuters reported on January 30 that China had given DeepSeek approval to buy Nvidia\u2019s H200 AI chips, although Nvidia chief executive Jensen Huang said on the same day that his company had not received confirmation and that Beijing was still finalizing the license.<\/p>\n Last April, the US government banned<\/a> exports of Nvidia\u2019s H20 AI chips to China amid rising political tensions between Washington and Beijing, a restriction that was later reversed in July 2025. Now, Washington allows the export of the H200 chips to China.<\/p>\n New Shaw Prize category<\/p>\n Before joining UC Berkeley in 2020, Chayes spent more than two decades as a technical fellow at Microsoft, where she led major research programs and helped shape the company\u2019s long-term research agenda. She founded and led the Theory Group at Microsoft Research, and later established Microsoft Research New England and the New York City lab.\u00a0 She has also chaired the selection committees for the Turing Award.<\/p>\n She has recently accepted an invitation from Tony Chan, the former president of King Abdullah University of Science and Technology, to chair the selection committee for the Shaw Prize\u2019s newly established award in the computer science category. The move expands the Shaw Prize beyond its three long-standing fields of mathematics, astronomy and life science.<\/p>\n The selection committee brings together some of the most influential figures in the AI sector, including John Hennessy, chairman of the board of Alphabet Inc, which operates Google; Yann LeCun, former chief scientist of Meta AI and widely recognized as a \u201cgodfather of AI\u201d; and Harry Shum, the former executive vice president of Microsoft\u2019s AI and Research group.<\/p>\n Chayes stressed that the selection committee has deep knowledge of AI development in China and is well placed to assess research from mainland Chinese scientists alongside work from the rest of the world under the Shaw Prize\u2019s established nomination and review process.<\/p>\n \u201cHarry Shum has spent much of his career in China,\u201d she said. \u201cAnd personally, in 1997\u201398, I worked with (Taiwanese computer scientist) Kai-Fu Lee to open Microsoft Research Asia in Beijing.\u201d<\/p>\n She added that in the more than two decades leading up to her move to Berkeley in 2020, she mentored many computer scientists in Beijing and built extensive personal relationships with research leaders across China.<\/p>\n Chayes said she has worked closely with Chinese researchers, both in mainland China and in the United States, over the past decades, and has a positive image of them.<\/p>\n \u201cI feel that, on average, researchers from China work harder than those from the rest of the world. That\u2019s something that I love about them,\u201d she said. \u201cThey\u2019re really my type of people.\u201d<\/p>\n She said the Shaw Prize\u2019s first laureate in computer science, to be announced in spring 2027, will be selected purely on scientific merit and could come from anywhere in the world.<\/p>\n \u201cWe\u2019ve been discussing who the obvious candidates are. We want to make sure who gets nominated. Some of them are Chinese, some are from Europe and America. Some are Chinese in America,\u201d she said.<\/p>\n She cited a mainland Chinese postdoctoral researcher who had worked in North America and is now based in Hong Kong as an example of the kind of globally mobile scholar who could be considered.<\/p>\n Chayes also described<\/a> Ya-Qin Zhang, a chair professor at Tsinghua University and former president of Baidu, as an old friend, saying she first knew him when he was head of Microsoft Research Asia in 1998. She added that Chinese-born American computer scientist Fei-Fei Li is also a close friend, and that the two continue to work together on an AI expert panel appointed by California Governor Gavin Newsom in September 2024.<\/p>\n Read: Beijing to approve Nvidia H200 imports, flagging overreliance<\/a><\/p>\n Follow Jeff Pao on Twitter at\u00a0@jeffpao3<\/a><\/p>\n \t