![\"A](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2026\/03\/12ec3de9-1b53-48bc-b819-64096f35b776.jpg\")
Two years ago, ChatGPT was able to answer fewer than 3 per cent of questions in the exam correctlyAlamy<\/p>\nNot so long ago, the peak of artificial intelligence was a helpful, animated paperclip. <\/p>\n
But tech developers have claimed that large language models could outperform all human experts within a year \u2014 and they have built the test to prove it.\u00a0<\/p>\n
Humanity\u2019s Last Exam (HLE) is a bank of 2,500 closed-format questions designed to see how far AI is from the \u201cfrontiers of human expertise\u201d.\u00a0<\/p>\n
Two years ago, ChatGPT was able to answer fewer than 3 per cent of questions in the exam correctlyAlamy<\/p>\n
Covering about a hundred topics, which range from mythology and physiology to rocket science, the questions were designed to require at least PhD-level understanding.\u00a0Anyone \u2014 or anything \u2014 achieving even close to 100 per cent would be a \u201cuniversal expert\u201d. <\/p>\n
AI is racing towards that point at a startling pace.\u00a0<\/p>\n
\u201cWe wanted to create this close-ended academic benchmark, set to the frontier of expert humans, that only a handful of people on earth can really solve,\u201d said Calvin Zhang, the research lead at Scale, the AI company behind HLE.<\/p>\n
\u201cWe\u2019ve seen over the past few years insane progress on these language models. It\u2019s impressive, model builders have really done a great job at improving these reasoning models.\u201d<\/p>\n
During initial tests in late 2024, OpenAI\u2019s GPT-4o answered fewer than 3 per cent of HLE questions correctly. This proved that \u201ca marked gap\u201d remained between large language models (LLMs) and expert academics, researchers said.\u00a0<\/p>\n
Within months, Google Gemini<\/a> could answer 18.8 per cent of the exam correctly. Last month, its latest model scored 45.9 per cent.<\/p>\n Researchers at Scale and the Center for AI Safety, a non-profit organisation, built HLE to test both breadth of knowledge and depth of reasoning \u2014 examining, for example, if a model could both identify and apply the correct mathematical formula.<\/p>\n The creators issued a global appeal for submissions in September 2024, asking for short-response questions with unambiguous answers that could not easily be searched for on the web.<\/p>\n Encouraged by a $500,000 prize pot, experts from about 50 countries submitted 70,000 questions. If any existing model could answer a question, it was rejected, allowing submissions to be whittled down to a set of 13,000 questions. In the end, about 2,500 questions were selected, although some have since been removed or edited based on user feedback.\u00a0<\/p>\n The topics are deliberately eclectic: any successful LLM would have to know both the correct pronunciation of biblical Hebrew and the number of tendons attached to each bone of a hummingbird.<\/p>\n To prevent later models benefiting from answers being publicly discussed online, a technique known as \u201cbenchmark hacking\u201d, a significant number of HLE questions have been kept secret.\u00a0<\/p>\n Despite the difficulty of the questions and the safeguards against cheating, AI could get close to full marks within a year, Zhang believes. <\/p>\n Kate Olszewska, a product manager at Google DeepMind<\/a> agrees: \u201cIf we truly cared about this as the only thing in life, I think we could get to it pretty quickly.\u201d<\/p>\n![\"A](\"https:\/\/www.newsbeep.com\/nz\/wp-content\/uploads\/2026\/03\/2a3bc5f0-5823-48fa-a33e-996bb8935e41.jpg\")
The London headquarters of Google DeepMind, the lab behind GeminiJose Sarmento Matos\/Bloomberg<\/p>\n