Parents in North Carolina this week filed a lawsuit against Google alleging that the company’s AI tools\u2014specifically Gemini via AI Mode on a school-issued Chromebook\u2014helped their 13-year-old daughter (identified as “Emma”) plan a near-suicide attempt.<\/p>\n
In the lawsuit, the Gizmodo online news site reported, parents claimed that Google\u2019s AI features allegedly gave tips to their daughter early this year on how to end her life and even \u201ccoached\u201d her to keep the truth of the situation from her family and friends.<\/p>\n
The Google products failed to trigger a crisis alert at any point, the parents claimed. The school\u2019s internet monitoring software eventually triggered an alert, and her parents were notified in March, thwarting a suicide attempt.<\/p>\n
\u201cWe are reviewing the claims in the lawsuit and we take this very seriously,\u201d a Google spokesperson told Gizmodo. \u201cWe\u2019ve worked extensively with medical professionals and academic experts to build features that help young people find critical support and trusted resources when they need it.\u201d<\/p>\n
\u201cOur systems aim to prevent users from engaging in harmful conversations, and when people\u2019s queries indicate they may be experiencing a crisis we show hotline information directly in Search to connect them with real-world support.\u201d<\/p>\n
According to Google, explicit content gets filtered out in Search results for students using Workspace: \u201cAny highly sensitive queries in Search are provided with information from trustworthy sources.\u201d<\/p>\n
Google says its crisis detection systems show hotline information \u201cwhen it finds that a user is contemplating self-harm.\u201d<\/p>\n
The company told Gizmodo that it extensively monitors for attempts to \u201cjailbreak\u201d these safety guardrails.<\/p>\n
\u201cWhat happened to Emma was no accident,\u201d the lawsuit claims, according to the Gizmodo report. \u201cThe ground was laid many months before, thousands of miles away from Emma\u2019s North Carolina home, where Google leaders concluded that they had a problem: the Silicon Valley titan was losing the battle for AI supremacy.\u201d<\/p>\n
Gizmodo is an American online news publication owned by Swiss digital media company Keleops Media.<\/p>\n
\u201cGoogle could have publicly explained that every time a student used their Chromebook to run a Google search, Gemini-powered AI features could be triggered and draw them into unsafe conversations that endangered their safety and well-being,\u201d the lawsuit claims. \u201cGoogle intentionally chose not to do so.<\/p>\n","protected":false},"excerpt":{"rendered":"Parents in North Carolina this week filed a lawsuit against Google alleging that the company’s AI tools\u2014specifically Gemini…\n","protected":false},"author":2,"featured_media":523017,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[29],"tags":[2007,2080,199885,199886,21829,199887,14624,1846,44271,6138,88,90,89,711,44999],"class_list":["post-523016","post","type-post","status-publish","format-standard","has-post-thumbnail","category-san-jose","tag-ai","tag-artificial-intelligence","tag-chrome","tag-chromebook","tag-gemini","tag-gismodo","tag-google","tag-lawsuit","tag-new-york-times","tag-north-carolina","tag-san-jose","tag-san-jose-headlines","tag-san-jose-news","tag-suicide","tag-suicide-prevention"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/523016","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/comments?post=523016"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/523016\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media\/523017"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media?parent=523016"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/categories?post=523016"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/tags?post=523016"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}