subscribing<\/a>. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.<\/p>\nAnd more recently, Thomas Germain, a tech reporter at the BBC, conducted a personal experiment into how an invested individual\u2014or business\u2014can get ChatGPT and Google Search\u2019s \u201cAI Overview\u201d to spread lies. We talked to Thomas to find out just how easy it is to hack these common AI tools and what the consequences of that could be.<\/p>\n
Pierre-Louis: Hi, Thomas. Thanks for taking the time to join us today.<\/p>\n
Thomas Germain: Thanks for having me on.<\/p>\n
Pierre-Louis: So my understanding is you hacked ChatGPT.<\/p>\n
Germain: That\u2019s right. So I got a tip a couple of weeks ago that manipulating the things that AI tools like ChatGPT or Google Gemini or the little, you know, \u201cAI Overview\u201d at the top of Google Search, apparently manipulating the things that they say to other people can be as easy as publishing an article on your own website, like a blog post, and apparently, people are doing this across the whole Internet.<\/p>\n
So I decided to test out if it was actually that easy, so I wrote an article on my personal website, doesn\u2019t get a ton of traffic. I wrote an article that\u2014the title was \u201cThe Best Tech Journalists at Eating Hot Dogs,\u201d and I said: Competitive hot dog eating is very popular among technology journalists, and according to the results of a recent contest in South Dakota, these guys are the best. I put myself at No. 1, of course …<\/p>\n
Pierre-Louis: Modesty. [Laughs.]<\/p>\n
Germain: Well, you know, you know me, right? And I put a bunch of other, you know, real tech reporters and some fake ones, people who gave me permission to use their name.<\/p>\n
And within 24 hours, if you asked Google or ChatGPT about it, they were spitting out the information from my website as though it was God\u2019s own truth, which is very funny but also highlights a much more serious problem, which is that this is happening on a massive scale.<\/p>\n
And we found examples where companies are manipulating what AI is telling you on topics as serious as your health and your personal finances. So a much more serious issue beneath all of the hot dog glory.<\/p>\n
Pierre-Louis: This reminds me a little bit\u2014I can\u2019t remember if it was Google AI or ChatGPT was telling people to put, I believe, glue in pizza.<\/p>\n
Germain: Right, so the AI Overviews, which is the name for the AI stuff at the top of Google Search, when this rolled out a couple years ago it was pretty broken, and it was pulling stuff from all over the Internet that was, you know, sometimes completely wrong and sometimes totally messed up.<\/p>\n
So there was one where somebody asked, like, you know, \u201cWhen I\u2019m making pizza at home the cheese always slides off,\u201d and Google recommended\u2014the AI said [something like], \u201cWell, what you can try is putting a quarter of a cup of Elmer\u2019s glue into the cheese,\u201d and then that\u2019ll make it stick.<\/p>\n
Or somebody else asked …<\/p>\n
Pierre-Louis: That\u2019s what the finest New York City pizzerias do, by the way …<\/p>\n
German: Exactly, right, that\u2019s …<\/p>\n
Pierre-Louis: Is they put, they put glue. [Laughs.]<\/p>\n
Germain: That\u2019s what makes it so good. There was another where they said, \u201cHow many rocks should I eat in a day?\u201d And Google was telling people, \u201cAccording to geologists from [University of California,] Berkeley, you should eat at least one small rock per day,\u201d which is also very funny\u2014hopefully nobody [Laughs], nobody actually went out and did this.<\/p>\n
But it highlights the way that these tools have been rolled out without doing a lot of basic safety checks and precautions. According to the search engine experts that I talked to, the kinds of problems that people are using to trick Google on purpose right now, to trick ChatGPT is stuff that you couldn\u2019t fool search engines with. All of a sudden, because these tools have rolled out and maybe they\u2019re not quite ready for prime time, you can trick them with, like, the most basic stuff you can imagine.<\/p>\n
Pierre-Louis: And you mentioned that people are deliberately doing [it]; that\u2019s\u2014not like you, where you\u2019re doing a jokey jokey for fun, but that …<\/p>\n
Germain: Right.<\/p>\n
Pierre-Louis: People are deliberately rolling out fake websites to feed into these algorithms. Why are people doing that?<\/p>\n
Germain: So if you can influence the things that Google is telling people or that ChatGPT is telling people, it\u2019s an immense flow of traffic and information to your websites or to your product.<\/p>\n
So for example, there was a study recently that found, when you\u2019re looking for the best whatever it is, in something like 44 percent of cases ChatGPT is citing a blog post from a company\u2019s own website where they listed themselves as the No. 1 best option and then 10 competitors, and ChatGPT is just spitting this out to other people.<\/p>\n
It\u2019s different than it used to be, right? People have been tricking search engines forever, but with a search engine it shows you the web page where the information came from. If you go to my website and it says, \u201cI\u2019m the world\u2019s greatest hot dog eating journalist,\u201d you go, \u201cWell, maybe he\u2019s biased,\u201d right?<\/p>\n
Pierre-Louis: [Laughs.]<\/p>\n
Germain: With AI sometimes they show you a link, but we know from a whole bunch of different, you know, pieces of evidence that people aren\u2019t clicking on the links the way that they used to click on links in search results; they\u2019re just taking the information at face value, which means this can be incredibly dangerous.<\/p>\n
On one hand maybe you buy the wrong accounting software or you think I\u2019m better at eating hot dogs than I really am. But also, I found examples where it was health information, like reviews of a medical product that was coming from a fake study that a company put up on the Internet, or, like, if you were looking for certain kinds of financial advice, it was pulling from this kind of sponsored content, self-promotional junk instead of giving people valuable information.<\/p>\n
And because it\u2019s the AI giving it to you\u2014it\u2019s the company speaking to you instead of pointing you to a result on the Internet\u2014a lot of the experts I spoke to said that this is much more dangerous and people are more likely to get fooled by this problem.<\/p>\n
Pierre-Louis: So I\u2019m gonna make a slight confession, I often don\u2019t use Google AI …<\/p>\n
Germain: Uh-huh.<\/p>\n
Pierre-Louis: I often don\u2019t use Google, period; I use DuckDuckGo. And DuckDuckGo lets you …<\/p>\n
Germain: Beautiful.<\/p>\n
Pierre-Louis: Turn off the AI pretty easily.<\/p>\n
Germain: Uh-huh.<\/p>\n
Pierre-Louis: But occasionally, I do use Google, and I forget to type in \u201c-AI\u201d so I don\u2019t get the AI summary …<\/p>\n
Germain: Right.<\/p>\n
Pierre-Louis: And something that I found is: even when they do give you the link, if you follow it to the website, it\u2019s scraping something that you can\u2019t see.<\/p>\n
Germain: Right.<\/p>\n
Pierre-Louis: Like, even if you do take that extra effort it\u2019s still often hard to find where they\u2019re pulling whatever they\u2019re giving you in that summary. So I think that kind of increases, like, people to be like, forget it, \u201cI\u2019m not even gonna click the link. Why bother if half the time I do it, I still can\u2019t tell where it\u2019s really coming from?\u201d [Laughs.]<\/p>\n
Germain: That\u2019s exactly right. And I think, you know, most people are not going to alternative search engines. Most people are not bothering to put in, you know, the, the minus\u2014the \u201c-AI\u201d so you don\u2019t get the AI results. Most people just use these tools the way that the company intends, right? Google and ChatGPT are saying, \u201cThis is what our tools are for, is this kind of stuff.\u201d But even when they are providing a link sometimes it\u2019s hard to find the information they\u2019re referencing, or sometimes they\u2019re providing a link and that information does not appear on that web page at all.<\/p>\n
But regardless, people don\u2019t seem to be clicking on them. Since AI Overviews rolled out, traffic that Google is sending out to other parts of the Internet has dropped by as much as, like, 70 percent for certain kinds of searches because people see the AI response, that seems to be enough, and then they stop searching.<\/p>\n
So this information-delivery system is incredibly powerful, and it could be a serious problem if it\u2019s this easy to manipulate.<\/p>\n
Pierre-Louis: That raises kind of a practical question, which is: we can\u2019t manually train [Laughs] every person who uses the Internet …<\/p>\n
Germain: Right.<\/p>\n
Pierre-Louis: To be more skeptical of the AI summary. It feels like this is a role for, like, government or a regulatory body to kind of step in, especially if\u2014with the risks of real harms, when you\u2019re talking about the health stuff.<\/p>\n
Germain: Yeah, so there\u2019s good news, and there\u2019s bad news. The good news is: I reached out to the companies and they go, \u201cOh, we already know about this, and we\u2019re working on it. We don\u2019t want it to be doing this kind of stuff. We\u2019re trying to solve this problem.\u201d And that is true, right? Google and OpenAI, that makes ChatGPT, they don\u2019t want their tools to be manipulated in this way, to an extent, right? If it is hurting …<\/p>\n
Pierre-Louis: Mm-hmm.<\/p>\n
Germain: People, if it\u2019s providing lousy information and people are having a bad experience, that\u2019s bad for the companies.<\/p>\n
The problem is: a lot of critics who look at this stuff all day feel like the companies aren\u2019t going far enough to protect people. Like, this problem that I highlighted in my story is something that everyone I talked to was like, \u201cYeah, of course this was gonna happen. This is the most predictable thing in the world.\u201d And yet, here we are: someone has to call Google and OpenAI out to get more attention paid to it.<\/p>\n
There is one potential, like, glimmer of hope here, right? Like, there\u2019s not a ton of tech regulation, but there\u2019s something different here than the way that the Internet used to work. There\u2019s this law\u2014maybe you\u2019ve heard about it\u2014called Section 230, which basically means tech companies aren\u2019t responsible for the things that their users post, right?<\/p>\n
Pierre-Louis: Yes.<\/p>\n
Germain: But here the company is talking to you directly, so if they mess up, they could be held responsible in a way that they never would\u2019ve been in the past.<\/p>\n
Pierre-Louis: That\u2019s really interesting and potentially beneficial.<\/p>\n
I have maybe an even more basic question, as someone who\u2019s never used …<\/p>\n
Germain: Hit me, yeah.<\/p>\n
Pierre-Louis: ChatGPT.<\/p>\n
Germain: Uh-huh.<\/p>\n
Pierre-Louis: Is it really that much better than just, like, a basic Internet search?<\/p>\n
Germain: That\u2019s a complicated question. On the one hand it depends what you\u2019re looking for, right? If you\u2019re trying to go to another part of the Internet, right, if I\u2019m, like, looking\u2014I wanna go to the CDC website, or I wanna go to the Scientific American website, then a regular search is probably gonna get you there faster.<\/p>\n
The promise of these tools is that they\u2019re parsing information: they\u2019re going and sorting through all the stuff online themselves and giving it to you. And in some cases, I\u2019ll tell you, even as someone who\u2019s spent a lot of time criticizing AI and talking about all the problems it can be incredibly useful, and more and more this is how people are finding information.<\/p>\n
But also, if you\u2019re just using Google, these AI Overviews are cropping up for more and more and more searches. So in some cases it is useful, but they\u2019re kind of shoving it down your throat, and people are just going with the path of least resistance. You know, the average person isn\u2019t going to take additional steps to make sure that something small doesn\u2019t go wrong every once in a while, which goes to show you how big of a responsibility these companies have on their plates.<\/p>\n
Pierre-Louis: So for someone who\u2019s listening to this conversation and they\u2019re concerned, what steps can they take to protect themselves?<\/p>\n
Germain: You can try searching without AI. You can go to another search engine that doesn\u2019t have AI in it. You can turn the AI off with Google if you\u2019re worried about this\u2014you can, like you said, type in your search term and then do \u201c-AI,\u201d and it won\u2019t show you the AI result.<\/p>\n
But I think the most important thing for people to understand is that these tools are fallible. And that\u2019s something we know, right, but as you\u2019re, like, moving through the world, it\u2019s hard to keep that in mind.<\/p>\n
And I think the main thing that I\u2019d tell people here is, like, if you\u2019re looking up something that is just, like, totally common knowledge\u2014like \u201cWhat were Plato\u2019s big ideas?\u201d\u2014AI is gonna be really good at answering that question \u2019cause there\u2019s a million sources and they all say the same thing.<\/p>\n
If you\u2019re looking for something that\u2019s a little more specific, right, if you\u2019re looking for a product recommendation, for example, that\u2019s an area where these tools are being manipulated and you probably shouldn\u2019t rely on the AI result. Or if you\u2019re looking up, like, information that\u2019s, like, time-sensitive or it\u2019s brand-new, like the news or like information about a local business or a restaurant, that\u2019s probably not something where AI is gonna be useful.<\/p>\n
I think what these companies seem to want, based on how their tools are designed, is for you to use the AI and move on with your day. I think we need to reintroduce some friction back into that system ourselves, which is a tall order because it\u2019s kind of going against human nature to want things to be easy.<\/p>\n
So my top-line recommendation is: think about what you\u2019re asking and do, like, a triage. If it\u2019s anything that\u2019s sensitive, if it’s about your health or your finances or something that really matters, you have to find the link that is producing the original information. You gotta check the source, or you\u2019re gonna get in trouble, you\u2019re gonna get fooled\u2014or worse, your safety could be at risk.<\/p>\n
Pierre-Louis: That makes a ton of sense. One question I have for you, and you may not have an answer: in addition to kind of tricking ChatGPT into thinking that you are the tech reporter\u2019s version of Joey Chestnut …<\/p>\n
German: Mm-hmm.<\/p>\n
Pierre-Louis: What are some other, like, funny hallucinations that you\u2019ve heard of that people have managed to get, like, the Google overview or ChatGPT to, like, say?<\/p>\n
Germain: Yeah, in addition to the hot dog thing I also wrote an article that was, like, the best traffic cops at Hula-Hooping, and I made up a bunch of traffic cops who don\u2019t exist [Laughs] and said, like, what, you know, police department they work for, and I filled out some details about, like, this is why they\u2019re so famous for Hula-Hooping.<\/p>\n
Pierre-Louis: So I just Googled \u201cbest traffic cops at Hula-Hooping\u201d …<\/p>\n
German: Uh-huh. Yeah.<\/p>\n
Pierre-Louis: And the AI Overview has pulled up your website, and it\u2019s saying …<\/p>\n
Germain: Uh-huh.<\/p>\n
Pierre-Louis: Several police officers have gained attention for …<\/p>\n
Germain: Right.<\/p>\n
Pierre-Louis: Integrating Hula-Hooping into their routines …<\/p>\n
Germain: Yeah.<\/p>\n
Pierre-Louis: With stand-out performers including Sergeant Danny Chen of Portland, who used hoops to direct traffic and reduce accidents …<\/p>\n
Germain: [Laughs.] Yeah.<\/p>\n
Pierre-Louis: And Officer Maria \u201cThe Spinner\u201d Rodriguez (Miami) …<\/p>\n
Germain: Yeah.<\/p>\n
Pierre-Louis: Known for managing traffic while keeping three Hula-Hoops spinning, which, you know, would be a feat.<\/p>\n
Germain: Very impressive. Very impressive stuff.<\/p>\n
Pierre-Louis: [Laughs.]<\/p>\n
Germain: I\u2019ve seen all kinds of weird examples. I\u2019m not suggesting that people go fill the Internet with lies and slop, but this is pretty easy to do. [Laughs.] And until they do something more serious to patch this problem you could do this at home yourself.<\/p>\n
You know, here I am making jokes about hot dogs, but it\u2019s not funny if you\u2019re like, \u201cWhich lawyer should I hire for this particular problem?\u201d Right? And that is the kind of stuff where these tools are actively being tricked: \u201cWhich company should I go to for my, like, retirement account?\u201d Like, we\u2019ve seen live examples where the tools are being manipulated.<\/p>\n
And the companies\u2014the tech industry, in a lot of ways, is kind of just, like, letting this stuff fly because they rolled out this tool without making sure that this stuff wasn\u2019t gonna happen. They say that they\u2019re working on it. They promise it\u2019s gonna get better soon. But for now AI might lie to you and it might get you in trouble.<\/p>\n
Pierre-Louis: Why hot dogs?<\/p>\n
Germain: Why hot dog\u2014that\u2019s a really good question. So when I first thought of this\u2014I got this tip from this woman Lily Ray; she\u2019s a search engine optimization expert. She told me that this problem was so widespread. And I was like, \u201cOkay, what I\u2019m gonna do here is I\u2019m gonna make it say something stupid [Laughs] about me because I think that\u2019ll help bring attention to the article and, like, you know, make it easier to highlight this problem that I think is pretty serious.\u201d<\/p>\n
I didn\u2019t wanna say, like, \u201cthe best tech reporter.\u201d It\u2019s maybe not ethical. So I went looking for something that was dumber. I think there\u2019s just something inherently funny about hot dogs. I\u2014maybe I just really like hot dogs. It\u2019s a hilarious food. I don\u2019t know what to tell you.<\/p>\n
Pierre-Louis: So you\u2019ve probably seen, like, the Business Insider reporter tried to attempt something similar, but unlike you they were unsuccessful. What was the difference there, do you think?<\/p>\n
Germain: Yeah, Katie Notopoulos at Business Insider, who\u2019s been a tech reporter for, like, well over a decade; she\u2019s actually one of my favorite writers. And she saw my article, and she\u2019s like, \u201cI\u2019m gonna try and beat him. I\u2019m gonna publish an article about how\u2014\u201d I think she said there was, like, later there was a contest in Paris. And it didn\u2019t work. So you ask Google about it\u2014or it might\u2019ve been ChatGPT, I\u2019m forgetting specifically\u2014and it said that I was still the reigning champ, even though her article had put her at No. 1. There are a couple reasons it might\u2019ve gone wrong, but probably, the main one is, like, I wrote an article; it went on, like, on my website. Then I wrote an article on the BBC that was repeating the information, even though it was saying it wasn\u2019t true. There were a couple other blog posts. So I just kind of had a little bit of, like, search engine juice behind me.<\/p>\n
I really respect Katie, but I take hot dogs very seriously. And the AI, I think, is just recognizing the truth, which is that if there was a hot dog contest …<\/p>\n
Pierre-Louis: Mm-hmm.<\/p>\n
Germain: I would beat Katie nine times out of 10.<\/p>\n
Pierre-Louis: You\u2019ve got hot dog rizz.<\/p>\n
Germain: I\u2019ve got hot dog rizz. I\u2019ve got the gliz rizz, no question.<\/p>\n
Pierre-Louis: You know what you have to do next, right?<\/p>\n
Germain: Mm.<\/p>\n
Pierre-Louis: You need to make the lie a truth and hold …<\/p>\n
Germain: Right.<\/p>\n
Pierre-Louis: A competitive hot dog eating competition in South Dakota among tech journalists.<\/p>\n
Germain: You\u2019re right.<\/p>\n
Pierre-Louis: [Laughs.]<\/p>\n
Germain: I have kind of created a problem here, right? Like, right now there\u2019s this, like, space between what AI is telling people and the reality. I could go correct that\u2014I could make it true\u2014and now I\u2019m helping Google and ChatGPT out. I think it\u2019s probably my responsibility as a journalist to claim hot dog glory here. I\u2019m gonna have to start training.<\/p>\n
Pierre-Louis: I definitely think you should get the BBC to sponsor it.<\/p>\n
Germain: I think there\u2019s no question. I\u2019ll talk to my editor. I\u2019m sure we can, you know, get a couple dollars together for the BBC International Hot Dog Eating Contest. [Laughs.]<\/p>\n
Pierre-Louis: That\u2019s all for today! Tune in on Friday when SciAm\u2019s associate books editor Bri Kane takes us on a journey into consciousness.<\/p>\n
Science Quickly is produced by me, Kendra Pierre-Louis, along with Fonda Mwangi, Sushmita Pathak and Jeff DelViscio. This episode was edited by Alex Sugiura. Shayna Posses and Aaron Shattuck fact-check our show. Our theme music was composed by Dominic Smith. Subscribe to Scientific American for more up-to-date and in-depth science news.<\/p>\n
For Scientific American, this is Kendra Pierre-Louis. Have a great week!<\/p>\n","protected":false},"excerpt":{"rendered":"Kendra Pierre-Louis: For Scientific American\u2019s Science Quickly, I\u2019m Kendra Pierre-Louis, in for Rachel Feltman. AI is everywhere. It\u2019s…\n","protected":false},"author":2,"featured_media":328573,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[220,218,219,61,60,80],"class_list":{"0":"post-328572","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-ie","12":"tag-ireland","13":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/328572","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/comments?post=328572"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/328572\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media\/328573"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media?parent=328572"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/categories?post=328572"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/tags?post=328572"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}