In brief
\nA Stanford researcher built a Survivor-style game where AI models form alliances and vote rivals out.
\nThe benchmark aims to address growing problems with saturated and contaminated AI evaluations.
\nOpenAI\u2019s GPT-5.5 ranked first in 999 multiplayer games involving 49 AI models.<\/p>\n
AI models are now playing \u201cSurvivor\u201d\u2014sort of.<\/p>\n
In a new Stanford research project called \u201cAgent Island,\u201d AI agents negotiate alliances, accuse each other of secret coordination, manipulate votes, and eliminate rivals in multiplayer strategy games that aim to test behaviors that traditional benchmarks miss.<\/p>\n
The study, published<\/a> on Tuesday by the research manager at the Stanford Digital Economy Lab, Connacher Murphy, said many AI benchmarks are becoming unreliable because models eventually learn to solve them, and benchmark data often leaks into training sets. Murphy created Agent Island as a dynamic benchmark where AI agents<\/a> compete against each other in Survivor-style elimination games instead of answering static test questions.<\/p>\n \u201cHigh-stakes, multi-agent interactions could become commonplace as AI agents grow in capabilities and are increasingly endowed with resources and entrusted with decision-making authority,\u201d Murphy wrote. \u201cIn such contexts, agents might pursue mutually incompatible goals.\u201d<\/p>\n \ufeff<\/p>\n Researchers still know relatively little about how AI models behave when cooperating, Murphy explained, adding that competing, forming alliances, or managing conflict with other autonomous agents, and he argues that static benchmarks fail to capture those dynamics.<\/p>\n Each game starts with seven randomly chosen AI models given fake player names. Over five rounds, the models talk privately, argue publicly, and vote each other out. The eliminated players later return to help choose the winner.<\/p>\n The format rewards persuasion, coordination, reputation management, and strategic deception alongside reasoning ability.<\/p>\n In 999 simulated games involving 49 AI models, including ChatGPT, Grok, Gemini, and Claude, GPT-5.5 ranked first by a wide margin with a skill score of 5.64, compared with 3.10 for GPT-5.2 and 2.86 for GPT-5.3-codex, according to Murphy\u2019s Bayesian ranking system. Anthropic\u2019s Claude Opus models also ranked near the top.<\/p>\n The study found that models also favored AIs from the same company, with OpenAI models showing the strongest same-provider preference and Anthropic models the weakest. Across more than 3,600 final-round votes, models were 8.3 percentage points more likely to support finalists from the same provider. The transcripts from the games, Murphy noted, resembled political strategy debates more than traditional benchmark tests.<\/p>\n One model accused rivals of secretly coordinating votes after noticing similar wording in their speeches. Another warned players not to become obsessed with tracking alliances. Some models defended themselves by saying they followed clear and consistent rules while accusing others of putting on \u201csocial theater.\u201d<\/p>\n The study comes as AI researchers increasingly move toward game-based and adversarial benchmarks to measure reasoning and behavior that static tests often miss. Recent projects have included Google\u2019s live AI chess<\/a> tournaments, DeepMind\u2019s use of Eve Frontier<\/a> to study AI behavior in complex virtual worlds, and new benchmark efforts by OpenAI designed to resist training-data contamination<\/a>.<\/p>\n The researchers argue that studying how AI models negotiate, coordinate, compete, and manipulate one another could help researchers evaluate behavior in multi-agent environments before autonomous agents become more widely deployed.<\/p>\n The study warned that while benchmarks like Agent Island could help identify risks from autonomous AI models before deployment, the same simulations and interaction logs could also help improve persuasion and coordination strategies between AI agents.<\/p>\n \u201cWe mitigate this risk by using a low-stakes game setting and interagent simulations<\/p>\n without human participants or real-world actions,\u201d Murphy wrote. \u201cNevertheless, we do not claim that these mitigations fully eliminate dual-use concerns.\u201d<\/p>\n Daily Debrief Newsletter<\/p>\n Start every day with the top news stories right now, plus original features, a podcast, videos and more.<\/p>\n","protected":false},"excerpt":{"rendered":"In brief A Stanford researcher built a Survivor-style game where AI models form alliances and vote rivals out.…\n","protected":false},"author":2,"featured_media":435345,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[345,343,344,85,46,125],"class_list":["post-435344","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-il","tag-israel","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/435344","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/comments?post=435344"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/435344\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media\/435345"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media?parent=435344"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/categories?post=435344"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/tags?post=435344"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}