The number of people who attended the “Unite the Kingdom” rally in London on 13 September has been widely contested. Organiser Stephen Yaxley-Lennon (better known as Tommy Robinson) wrongly claimed police had estimated three million people turned up. We fact checked this and found police had actually estimated that between 110,000 and 150,000 attended. Whilst measuring the exact size of a crowd is hard, the three million estimate appeared to be far too high, based on both the police figures and analysis from independent crowd size experts we spoke to.
While Google Search generated significant traffic to our article, it also blatantly misrepresented what we had said.
As you can see from the screengrab above, the way the content was represented was simply terrible. Instead of highlighting a summary of our fact check the search result displayed content from deep within the page, woefully out of context and suggesting that the “Unite the Kingdom” march was somehow about the Iraq war and attended by over a million people.
We decided to take a deeper look into what was going on in search results for our fact checks. Whilst it’s clear that misrepresentation of this kind is not happening with every search result, we have found these responses are inconsistent and at times directly misleading.
Join 72,330 people who trust us to check the facts
Subscribe to get weekly updates on politics, immigration, health and more.
To begin with it’s important to understand the elements of a written fact check. Looking at the same article about the rally in London, you can see our pages contain a number of elements. To address bad information, we first have to put the claim on our page so that we can debunk it. We try to do this as clearly as possible, listing what was claimed and what we concluded side by side. Under this we publish our fact check which is normally a few hundred words long, and something we would refer to as ‘body text’.
Using slightly different variants of the claim, the results in a Google search pulled content from different parts of the page and so presented our fact check in a variety of ways that ranged from correct to untidy to actively misleading.
We have since undertaken a small survey of how our recent fact checks appear in Google Search results and have found that in 10-15% of searches, Google is actively showing information in a way that explicitly suggests we have come to an opposite conclusion. Based on our own site usage, this means millions of people are now being shown incorrect information in Google Search results each year. That this information is being shown directly in association with our own website is deeply concerning.
Of course people do click on the results and come to our site to see the full picture, but this jumble of words representing our complex fact checking feels needlessly confusing.
All of this is also happening since Google removed support for the ClaimReview standard— a data format that was designed to ensure that this kind of confusion did not happen. We still use ClaimReview in our fact checks that specifically states what was claimed and what we’ve concluded.
When we approached Google recently to discuss their search results they told us: “We aim to surface relevant, high quality information in all our Search features and we continue to raise the bar for quality with ongoing updates and improvements. When issues arise – like if our features misinterpret web content or miss some context – we use those examples to improve and take appropriate action under our policies.”
We would strongly urge Google to reinstate support for open standards to respect the care, attention and effort that is put into writing fact checks. Without these standards we can only conclude that Google is actively vandalising the internet, which is quite an extraordinary thing to have to write about one of the world’s largest web platforms.
Related article