Researchers behind a new study say that the methods used to evaluate AI systems\u2019 capabilities routinely oversell AI performance and lack scientific rigor. <\/p>\n
The study, led by researchers at the Oxford Internet Institute in partnership with over three dozen researchers from other institutions, examined<\/a> 445 leading AI tests, called benchmarks, often used to measure the performance of AI models across a variety of topic areas. <\/p>\n AI developers and researchers use these benchmarks to evaluate model abilities<\/a> and tout technical progress<\/a>, referencing them to make claims on topics ranging from software engineering performance<\/a> to abstract-reasoning capacity<\/a>. However, the paper, released Tuesday, claims these fundamental tests might not be reliable and calls into question the validity of many benchmark results. <\/p>\n According to the study, a significant number of top-tier benchmarks fail to define what exactly they aim to test, concerningly reuse data and testing methods from pre-existing benchmarks, and seldom use reliable statistical methods to compare results between models.<\/p>\n Adam Mahdi, a senior research fellow at the Oxford Internet Institute and a lead author of the study, argued these benchmarks can be alarmingly misleading: \u201cWhen we ask AI models to perform certain tasks, we often actually measure completely different concepts or constructs than what we aim to measure,\u201d Mahdi told NBC News. <\/p>\n Andrew Bean, a researcher at the Oxford Internet Institute and another lead author of the study, concurred that even reputable benchmarks are too often blindly trusted and deserve more scrutiny.<\/p>\n \u201cYou need to really take it with a grain of salt when you hear things like \u2018a model achieves Ph.D. level intelligence,\u2019\u201d Bean told NBC News. \u201cWe\u2019re not sure that those measurements are being done especially well.\u201d<\/p>\n Some of the benchmarks examined in the analysis measure specific skills, like Russian or Arabic language abilities, while other benchmarks measure more general capabilities, like spatial reasoning and continual learning. <\/p>\n A core issue for the authors was whether a benchmark is a good test of the real-world phenomenon it aims to measure, or what the authors label as \u201cconstruct validity.\u201d Instead of testing a model on an endless series of questions to evaluate its ability to speak Russian, for example, one benchmark reviewed in the study measures a model\u2019s performance on nine different tasks, like answering yes-or-no questions using information drawn from Russian-language Wikipedia. <\/p>\n However, roughly half of the benchmarks examined in the study fail to clearly define the concepts they purport to measure, casting doubt on benchmarks\u2019 ability to yield useful information about the AI models being tested.<\/p>\n As an example, in the study the authors showcase a common AI benchmark called Grade School Math 8K (GSM8K), which measures performance on a set of basic math questions<\/a>. Observers often point to leaderboards<\/a> on the GSM8K benchmark to show that AI models are highly capable at fundamental mathematical reasoning, and the benchmark\u2019s documentation says it is<\/a> \u201cuseful for probing the informal reasoning ability of large language models.\u201d<\/p>\n Yet correct answers on benchmarks like GSM8K do not necessarily mean the model is actually engaging in mathematical reasoning, study author Mahdi said. \u201cWhen you ask a first grader what two plus five equals and they say seven, yes, that\u2019s the correct answer. But can you conclude from this that a fifth grader has mastered mathematical reasoning or arithmetic reasoning from just being able to add numbers? Perhaps, but I think the answer is very likely no.\u201d<\/p>\n Bean acknowledged that measuring nebulous concepts like reasoning requires evaluating a subset of tasks, and that such selection will invariably be imperfect. \u201cThere are a lot of moving pieces in these evaluations, and satisfying all of them requires balance. But this paper calls for benchmarks to clearly define what they set out to measure,\u201d he said. <\/p>\n \u201cWith concepts like harmlessness or reasoning, people oftentimes just throw the word around to pick something that falls near that category that they can measure and say, \u2018Great, now I\u2019ve measured it,\u2019\u201d Bean added.<\/p>\n In the new paper, the authors make eight recommendations and provide a checklist to systematize benchmark criteria and improve the transparency and trust in benchmarks. The suggested improvements include specifying the scope of the particular action being evaluated, constructing batteries of tasks that better represent the overall abilities being measured, and comparing models\u2019 performance via statistical analysis. <\/p>\n Nikola Jurkovic, a member of technical staff at the influential METR AI research center, commended the paper\u2019s contributions. \u201cWe need more rigor if we want to be able to interpret the results of AI benchmarks. This checklist is a starting point for researchers to check whether their benchmark will be insightful,\u201d Jurkovic told NBC News.<\/p>\n Tuesday\u2019s paper builds on previous research<\/a> pointing out flaws<\/a> in many AI benchmarks. <\/p>\n Last year, researchers from AI company Anthropic<\/a> advocated for increased statistical testing to determine whether a model\u2019s performance on a specific benchmark really showed a difference in capabilities or was rather just a lucky result given the tasks and questions included in the benchmark. <\/p>\n To attempt to increase the usefulness and accuracy of benchmarks, several research groups have recently proposed new series of tests that better measure models\u2019 real-world performance on economically meaningful tasks. <\/p>\n