{"id":317918,"date":"2026-03-07T16:51:09","date_gmt":"2026-03-07T16:51:09","guid":{"rendered":"https:\/\/www.newsbeep.com\/nz\/317918\/"},"modified":"2026-03-07T16:51:09","modified_gmt":"2026-03-07T16:51:09","slug":"researchers-create-humanitys-last-exam-to-test-the-limits-of-artificial-intelligence","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/nz\/317918\/","title":{"rendered":"Researchers Create \u2018Humanity\u2019s Last Exam\u2019 to Test the Limits of Artificial Intelligence"},"content":{"rendered":"

As artificial intelligence<\/a> has advanced<\/a> over the years, the methods used to measure<\/a> its capabilities<\/a> have become outdated. Tests that once challenged<\/a> advanced<\/a> AI models<\/a> are now being solved with ease, making it harder for researchers to pinpoint what current systems<\/a> are actually capable of.<\/p>\n

However, an international team of researchers has recently developed<\/a> a new exam designed to test the limits of modern AI systems<\/a>. Known as Humanity\u2019s Last Exam (HLE), the assessment includes 2,500 expert-level questions spanning disciplines from mathematics<\/a> and natural sciences<\/a> to ancient<\/a> languages<\/a> and humanities<\/a>. Details of the project and its results are outlined in a recent study<\/a> published in Nature<\/a>.<\/p>\n

Initial results indicate that even the most advanced AI models struggled with this exam. GPT-4o scored 2.7%, Claude 3.5 Sonnet 4.1%, and OpenAI\u2019s o1 model reached about 8% accuracy. More recent systems, such as Gemini 3.1 Pro and Claude Opus 4.6, improved to around 40-50% accuracy.<\/p>\n

When AI Outgrows Tests<\/p>\n

For years, researchers have used standardized tests to track AI capabilities. One well-known example is the Massive Multitask Language Understanding (MMLU) exam, which tests models in many academic subjects.<\/p>\n

Today, many advanced AI systems perform well on these exams, prompting questions about whether these tests still provide meaningful insights into the true capabilities of artificial intelligence.<\/p>\n

\u201cWhen AI systems start performing extremely well on human benchmarks, it\u2019s tempting to think they\u2019re approaching human-level understanding,\u201d said Dr. Tung Nguyen, an instructional associate professor of computer science and engineering at Texas A&M University<\/a> and a contributor to the new benchmark. \u201cBut HLE reminds us that intelligence isn\u2019t just about pattern recognition \u2014 it\u2019s about depth, context and specialized expertise.\u201d<\/p>\n

an Exam Beyond AI\u2019s Reach<\/p>\n

The development of Humanity\u2019s Last Exam involved nearly 1,000 researchers from diverse academic backgrounds. Each expert contributed questions that required advanced knowledge in their field, with every problem designed to have a single, verifiable answer.<\/p>\n

The questions reflect a vast range of human knowledge, from translating ancient Palmyrene inscriptions to identifying microscopic anatomical features in birds and analyzing phonological details in Biblical Hebrew pronunciation. The team tested each question against leading AI models. If a model could correctly answer a question, it was excluded from the final set. This process resulted in an exam specifically designed to be outside the reach of current AI technology.<\/p>\n

the Limits of Artificial Intelligence<\/p>\n

Even the best AI models missed many questions during initial trials of the exam, especially those that required specific knowledge or skills from different fields. This means that while AI is good at pattern recognition, it still has trouble with advanced, specialized knowledge.<\/p>\n

\u201cWithout accurate assessment tools, policymakers, developers and users risk misinterpreting what AI systems can actually do,\u201d Nguyen said.<\/p>\n

Although the name suggests a dramatic scenario, researchers created Humanity\u2019s Last Exam to better understand the strengths and weaknesses of current AI systems, not to warn about AI replacing human expertise.<\/p>\n

\t\t <\/p>\n

\t\t\t
\n\t\t\t\t\"Psilocybin\t\t\t<\/a>
\n\t\t
\n\t\t\t\t\t <\/p>\n

\u201cThis isn\u2019t a race against AI,\u201d Nguyen said. \u201cIt\u2019s a method for understanding where these systems are strong and where they struggle.\u201d<\/p>\n

Designing a Benchmark for the Future<\/p>\n

The team of specialists from various academic disciplines worked together to create a reliable way to test both computational ability and the wide range of knowledge that characterizes human expertise. Humanity\u2019s Last Exam represents one of the most ambitious attempts to benchmark the capabilities of advanced AI systems. Researchers have kept most of the questions private to preserve the integrity of the exam as AI systems continue to improve.<\/p>\n

\u201cWhat made this project extraordinary was the scale,\u201d Nguyen said. \u201cExperts from nearly every discipline contributed. It wasn\u2019t just computer scientists; it was historians, physicists, linguists, medical researchers. That diversity is exactly what exposes the gaps in today\u2019s AI systems \u2014perhaps ironically, it\u2019s humans working together.\u201d<\/p>\n

Austin Burgess is a writer and researcher with a background in sales, marketing, and data analytics. He holds a Master of Business Administration, a Bachelor of Science in Business Administration, and a Data Analytics certification. His work combines analytical training with a focus on emerging science, aerospace, and astronomical research.<\/p>\n","protected":false},"excerpt":{"rendered":"As artificial intelligence has advanced over the years, the methods used to measure its capabilities have become outdated.…\n","protected":false},"author":2,"featured_media":317919,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[365,363,4036,7705,21661,6301,11127,172177,111,139,69,13478,620,145],"class_list":["post-317918","post","type-post","status-publish","format-standard","has-post-thumbnail","category-technology","tag-ai","tag-artificial-intelligence","tag-chatgpt","tag-claude","tag-exam","tag-gemini","tag-humanity","tag-humanitys-last-exam","tag-new-zealand","tag-newzealand","tag-nz","tag-open-ai","tag-openai","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/317918","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/comments?post=317918"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/317918\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media\/317919"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media?parent=317918"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/categories?post=317918"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/tags?post=317918"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}