2025-12-12T21:26:30.691Z<\/p>\n
Share<\/p>\n
Facebook Email X<\/p>\n LinkedIn<\/p>\n Reddit<\/p>\n Bluesky<\/p>\n WhatsApp<\/p>\n Copy link<\/p>\n lighning bolt icon Impact Link<\/p>\n Save <\/p>\n Read in app<\/p>\n <\/a> <\/p>\n This story is available exclusively to Business Insider We just got a sobering picture of how often AI models get their facts straight. This week, Google DeepMind<\/a> introduced the FACTS Benchmark Suite<\/a>, which measures how reliably AI models produce factually accurate answers.<\/p>\n It tests models in four areas: answering factoid questions from internal knowledge, using web search effectively, grounding responses in long documents, and interpreting images. The best model, Google<\/a>‘s Gemini 3<\/a> Pro, reached 69% accuracy, with other leading models falling well below that.<\/p>\n For context, if any of the reporters I manage filed stories that were 69% accurate, I would fire them.<\/p>\n Beyond journalism, this number should matter to businesses betting on AI<\/a>. While models excel at speed and fluency, their factual reliability still lags far behind human expectations, especially in tasks involving niche knowledge, complex reasoning, or precise grounding in source material.<\/p>\n Even small factual errors can have outsized consequences in sectors such as finance, healthcare, and the law. This week, my talented colleague Melia Russell<\/a> looked at how law firms are handling the rise of AI models as a source of legal truth<\/a>. It’s messy: She recounts how one firm fired an employee because they filed a document riddled with fake cases after using ChatGPT<\/a> to draft it.<\/p>\n The FACTS benchmark is a warning but also a roadmap: by quantifying where and how models fail, Google hopes to accelerate progress. But for now, the takeaway is clear: AI is getting better, but it’s still wrong about one-third of the time.<\/p>\n Sign up for BI’s Tech Memo newsletter here<\/a>. Reach out to me via email at abarr@businessinsider.com<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"2025-12-12T21:26:30.691Z Share Facebook Email X LinkedIn Reddit Bluesky WhatsApp Copy link lighning bolt icon An icon in the…\n","protected":false},"author":2,"featured_media":182397,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[345,343,344,85,46,125],"class_list":{"0":"post-182396","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-il","12":"tag-israel","13":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/182396","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/comments?post=182396"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/182396\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media\/182397"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media?parent=182396"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/categories?post=182396"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/tags?post=182396"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n <\/p>\n
\n <\/a> <\/p>\n
\n An icon in the shape of a lightning bolt.<\/p>\n
\n Saved<\/p>\n
\n subscribers. Become an Insider<\/a>
\n and start reading now.
\n Have an account? Log in.<\/p>\n