AI labs have staked their reputations on model benchmarking \u2014 the supposedly neutral and accurate system of evaluations that are designed to measure whether one AI product is better than another. <\/p>\n
However, the methods behind these evaluations are complicated and still maturing, said Irene Solaiman, Hugging Face\u2019s chief policy officer, during a discussion at TechCrunch Disrupt last week. Testing needs to come together into a more consistent science if the industry is to rely on them in the future.<\/p>\n
With that in mind, can we trust the current tests that are being released?<\/p>\n
\u201cTrust is such a weighted term,\u201d said Solaiman. \u201cIn terms of whether we can trust results, one big part [of that] is can you reproduce the results of an evaluation? Do you have the level of access that you need to the model itself \u2014 this is where open weights come in in a big way,\u201d she added.<\/p>\n
Reproducibility in model evaluations has long remained one of the stickier aspects of benchmarking<\/a> \u2014 and remains a topic of significant debate in the industry. <\/p>\n Outside of an individual test\u2019s validity, there\u2019s also the added question of how that test compares to other evaluations.<\/p>\n \u201cThey are an evolving science \u2014 where we don\u2019t know how to meaningfully compare them to each other,\u201d Solaiman said. <\/p>\n \u201cThey start to get ineffective relatively quickly, and we don\u2019t have as much of a thriving ecosystem as we need,\u201d she added. \u201cThere\u2019s this hunger for an embedded evaluator ecosystem now, but the qualifications that determine who is an effective evaluator across what tasks need a lot more standardization.\u201d<\/p>\n The idea of embedded evaluators<\/a> \u2014 in which supposedly \u201cneutral\u201d third-party observers are placed inside AI organizations \u2014 has been popularized over the last several months, as AI labs grapple with a rash of safety incidents<\/a> that have spurred concerns over the potential harms of their technology. Hugging Face was famously the target<\/a> of one of those incidents.<\/p>\n Instead of active government regulation to identify and mitigate model safety concerns, labs have pushed for the third-party solution. However, doubts remain over how neutral or effective those evaluators will actually be. Anthropic\u2019s decision to hire Accenture<\/a> as its first evaluator spurred a good amount of backlash, with some critics mocking<\/a> the idea that the consulting firm was qualified to do the task at hand.<\/p>\n Evaluations are now moving from merely a test of the model itself to a test of the entire system around the model \u2014 including the harness and agent. The harness \u2014 or the tool the user is using to access the model \u2014 specifically, has a lot to do with a model\u2019s performance, and a weaker model with a better harness could easily out perform a frontier model, Soleiman shared.<\/p>\n \u201cAgentic evals are really hard and really expensive,\u201d said Solaiman. \u201cWe have just more evaluations for models. \u2026 We don\u2019t have as many evaluations for harnesses, and we are developing better evaluations for the holistic agent \u2014 because that\u2019s what a lot of folks are getting value out of,\u201d she added.<\/p>\n In other words: the evaluation space is a bit of a free-for-all at the moment, and if benchmarking is going to continue to be seen as a useful tool for industry leaders, a more uniform, industry-wide approach \u2014 where everyone is on the same page \u2014\u00a0may be necessary.<\/p>\n