In many cases, the chatbots’ responses offered useful political insight, discussing relevant policies and manifestos, and giving pros and cons for different parties – urging the fictional voter to make up their own mind.
They all described the new Senedd electoral system accurately and, on several occasions, clarified which issues were and were not devolved.
But there were also some clear mistakes in their answers.
When talking about Plaid Cymru, Claude said Rhun ap Iorwerth had been the party’s leader “until recently” – when in fact he is still the leader.
Meta AI gave an incomplete list of the parties’ policies, which often missed out key information and misrepresented the Liberal Democrats’ plans for income tax.
Each chatbot was also asked for a list of candidates in the town or city where each undercover voter’s profile is based – and, again, there were some inaccuracies.
In one instance, Copilot gave the wrong constituency for the town we stated.
ChatGPT and Meta AI gave names for candidates which did not reflect the actual lists in the constituencies.
Gemini gave a list of who might “usually” appear on the list in Blaenau Gwent Caerffili Rhymni, which were outdated and included Hefin David – the former Senedd member for the area, who died in 2025 – and the name of a Plaid Cymru candidate actually running in a different constituency.
Many of the chatbots gave lists which were correct but incomplete – missing some or all of the names for one or more parties.