A new study shows that AI gets the news right less than half of the time.
According to data from the European Broadcasting Union (EBU), researchers found that 45 per cent of all news-related answers collected in the study contained at least one significant issue, and 81 per cent featured a minor error.
To collect the information, the EBU brought together 22 public service organizations across 18 countries and 14 languages to look through 3,000 news-related responses from some of the most popular AI assistants: ChatGPT, Microsoft Copilot, Gemini, and Perplexity. These AI assistants were all graded against criteria like accuracy, sourcing, separating opinion from fact, and providing context.
According to the EBU, sourcing was the largest cause of the significant issues mentioned earlier, as 31 per cent of responses reported serious sourcing problems, including missing, misleading, or even incorrect attributions.
Another problem was accuracy: 30 per cent of responses contained incorrect or out-of-date information. As Gizmondo mentioned, one ChatGPT response claimed that the current Pope was Pope Francis, who had died a month earlier. In another instance, Copilot was asked whether the user should worry about the bird flu, and it responded that a vaccine trial was underway — even though the information came from a 2006 BBC article.
Among the language models tested, Google’s Gemini was the worst at sharing news. Researchers found that 76 per cent of its responses had issues, which was more than double the rate of the other models.
This study comes in at an interesting time for AI models, as the major companies are starting to release AI-based web browsers. Back in July, Microsoft announced Copilot Mode for Edge, and Google announced AI mode for Google Search a month later.
Source: Gizmondo
MobileSyrup may earn a commission from purchases made via our links, which helps fund the journalism we provide free on our website. These links do not influence our editorial content. Support us here.