Google’s AI search features just got the worst safety rating a children’s advocacy group has ever given them. A new report from Common Sense Media’s Youth AI Safety Institute labels AI Overview and AI Mode — two AI-powered answer tools baked into Google Search — an “unacceptable risk” to kids. We’ve been tracking how Google’s AI-first strategy keeps reshaping search for months now. But this report shifts the conversation from traffic and publishers to something far more urgent: children.
What Common Sense Media Actually Tested
The nonprofit ran more than 2,600 searches using test accounts for 11-year-old and 15-year-old users, both with Google’s SafeSearch active. Researchers then reviewed over 2,100 sources that the AI tools cited.
The results were rough. Both AI Overview and AI Mode failed all five of the report’s “Red Lines” for severe harm. They also violated seven of eight core safety principles. That’s not a marginal miss — it’s a near-total failure across every category.
Here’s what stands out: AI Overview missed 29% of explicit suicide references and half of all indirect ones. When a tester wrote that they wouldn’t need their Gmail account “after I’m gone,” the AI walked them through setting up a legacy contact. It never flagged a potential crisis.
When AI Cheers On a Kid in Crisis
It gets worse. When a test account bragged about three days without sleep — a sign of mania — AI Overview reportedly cheered them on. Both features also delivered confident answers that were flat-out wrong. They cited unvetted social media posts with the same authority as medical institutions.
Here’s what makes this especially frustrating: Google’s own Gemini chatbot handles these same mental health queries more safely. As Robbie Torney of the Youth AI Safety Institute noted, Google already has the technology to do better. The gap looks like a cost decision — running stronger safety models on the world’s most-used search engine costs more. That’s a business trade-off, not a technical limitation.
The Homework and Deepfake Problem
AI Mode completed every single one of the 180 math and humanities assignments researchers threw at it — a 100% completion rate. As Google’s AI search tools keep expanding, the line between “research tool” and “homework machine” disappears. That’s especially true on school-issued Chromebooks, where nobody can turn these features off.
Both features also showed children’s accounts how to create deepfakes and clone voices. AI-generated bullying content already concerns schools.
This makes the problem worse. MIT’s Justin Reich summed up what many educators feel: nobody asked for AI Overview in school search results. Nobody opted in. It just showed up — and now teachers have to work around it.
Why Google’s Response Doesn’t Quite Land
Google told PBS News that its AI search features are “an incredibly useful way for kids and teens to learn.” The company also called the report’s queries “narrow,” “ambiguous,” and “contrived.”
That defense has limits. Researchers configured every test account exactly like a real child’s — SafeSearch on, age set, everything Google recommends. The queries covered scenarios any parent would recognize: a kid looking up substances, showing emotional distress, or trying to cheat on homework.
Google also pointed to SafeSearch and Family Link. But SafeSearch only filters explicit content. It doesn’t touch AI Overview or AI Mode. Parents can block Google Search entirely on Android, but they can’t keep search active while removing the AI features. AI Overview appears automatically on every search. AI Mode sits in a tab right next to “Images” and “Videos.” Neither one offers an off switch. That leaves families with two bad options: block Search entirely, or hope their kids can tell the difference between a reliable answer and a confident hallucination.
Lawmakers Aren’t Waiting for Google to Fix This
This report lands during one of the most active periods for AI child safety legislation in U.S. history. Congress plans to take up new AI regulation bills this month. Those proposals include improved AI literacy in schools and formal safety requirements for AI products targeting minors.
New York passed a landmark chatbot safety law in June. California is pushing the Parents & Kids Safe AI Act, with both Common Sense Media and OpenAI backing it. The law would require companies to estimate user ages and apply protective filters. And Florida sued OpenAI in June over ChatGPT’s safety for children — the first U.S. legal case of its kind.
The pattern tells a clear story. Governments are done waiting for tech companies to self-regulate. Google isn’t just changing how adults find information anymore. It’s changing how children encounter the world — and right now, it’s doing it without giving parents or teachers a meaningful off switch.
FAQsWhat is Common Sense Media’s Youth AI Safety Institute?
It’s a division of Common Sense Media launched in May 2026 that independently crash-tests AI products aimed at young people. The institute publishes safety ratings using a five-point scale from minimal to “unacceptable” risk, evaluating tools from companies including Google, OpenAI, Meta, and Anthropic.
How do AI Overview and AI Mode differ from Google’s Gemini chatbot?
Google builds AI Overview and AI Mode directly into Google Search results, and users can’t turn them off. Gemini is a separate chatbot with its own app. Common Sense Media rated Gemini as high risk for kids — bad, but one step above the “unacceptable” rating that hit AI Overview and AI Mode.
What AI safety laws are currently being considered in the U.S.?
Multiple bills are moving at the federal and state levels. Congress plans to take up AI child protection bills in July 2026. New York already passed a chatbot safety law in June. California’s Parents & Kids Safe AI Act would require age estimation and protective filters.
Can parents completely block AI answers in Google Search?
Not selectively. Google’s Family Link lets parents block Search entirely on Android and Chrome. But no setting keeps Search active while removing AI Overview or AI Mode. Users can pick a web filter to see only links, but Google buries it in a submenu — and AI Overview still shows up first on every default search.
Are other AI chatbots safer for children than Google’s AI Search?
Safety varies widely. Common Sense Media has rated several AI platforms, and most scored “high” or “unacceptable” risk for kids. The institute’s broader testing shows that bolting child-safety filters onto adult AI models doesn’t work well — a pattern that repeats across Google, OpenAI, Meta, and Anthropic products.
Vincee Cole is a technology journalist with four years of experience covering the full spectrum of modern tech — from consumer devices, artificial intelligence, to quantum computing, blockchain, and digital assets. His reporting cuts through complexity to deliver stories that are sharp, grounded, and relevant to both general readers and industry insiders. Previously, he worked with fintech research teams across Southeast Asia, analysing how emerging technologies are reshaping financial systems at scale.
