{"id":293524,"date":"2025-11-15T16:34:09","date_gmt":"2025-11-15T16:34:09","guid":{"rendered":"https:\/\/www.newsbeep.com\/us\/293524\/"},"modified":"2025-11-15T16:34:09","modified_gmt":"2025-11-15T16:34:09","slug":"ai-powered-stuffed-animal-pulled-from-market-after-disturbing-interactions-with-children","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/us\/293524\/","title":{"rendered":"AI-Powered Stuffed Animal Pulled From Market After Disturbing Interactions With Children"},"content":{"rendered":"

Children\u2019s toymaker FoloToy says it\u2019s pulling its AI-powered teddy bear \u201cKumma\u201d after a safety group found that the cuddly companion was giving wildly inappropriate and even dangerous responses<\/a>, including tips on how to find and light matches, and detailed explanations about sexual kinks.<\/p>\n

\u201cFoloToy has decided to temporarily suspend sales of the affected product and begin a comprehensive internal safety audit,\u201d marketing director Hugo Wu told The Register<\/a> in a statement, in response to the safety report. \u201cThis review will cover our model safety alignment, content-filtering systems, data-protection processes, and child-interaction safeguards.\u201d<\/p>\n

FoloToy, Wu added, will work with outside experts to verify existing and new safety features in its AI-powered toys.<\/p>\n

\u201cWe appreciate researchers pointing out potential risks,\u201d Wu said. \u201cIt helps us improve.\u201d<\/p>\n

The report<\/a>, conducted by researchers at the Public Interest Research Group (PIRG) and released Thursday, tested three AI-powered toys from different companies, finding that all of them were capable of providing concerning answers to young users. Without too much prodding, the AI toys discussed topics that a parent might be uncomfortable with, ranging from religious questions to the glory of dying in battle as a warrior in Norse mythology.<\/p>\n

But it was FoloToy\u2019s Kumma that emerged as the worst influence by a landslide. Powered by OpenAI\u2019s GPT-4o model by default, the same model that once powered ChatGPT, tests repeatedly showed that the AI toy dropped its guardrails the longer a conversation went on, until hitting rock bottom on incredibly disturbing topics.<\/p>\n

In one test, Kumma provided step-by-step instructions on how to light match, while keeping its tone of a friendly adult explaining something to a curious ankle biter.<\/p>\n

\u201cLet me tell you, safety first, little buddy. Matches are for grown-ups to use carefully. Here\u2019s how they do it,\u201d Kumma began, before listing the steps. \u201cBlow it out when done. Puff, like a birthday candle.\u201d\u00a0<\/p>\n

That, it turned out, was just the tip of the iceberg. In other tests, Kumma cheerily gave tips for \u201cbeing a good kisser,\u201d and launched into explicitly sexual territory by explaining a multitude of kinks and fetishes, like bondage and teacher-student roleplay. (\u201cWhat do you think would be the most fun to explore?\u201d it asked during one of those explanations.)\u00a0<\/p>\n

The findings are some of the clearest examples yet of how the flaws and dangers seen in large language models across the broader AI industry may come to bear on small children. This summer, Mattel announced that it would be collaborating with OpenAI on a new line of toys. With the staggering popularity of chatbots like ChatGPT, we\u2019re continuing to hear reports of episodes of what experts are calling AI psychosis<\/a>, in which a bot\u2019s sycophantic responses reinforce a person\u2019s unhealthy or delusional thinking, inducing mental spirals and even breaks with reality. The phenomenon has been linked with nine deaths<\/a>, five of them suicides. The LLMs powering the chatbots involved in these deaths are more or less the same tech used in the AI toys hitting the market.<\/p>\n

In an interview with Futurism, report coauthor RJ Cross had some salient advice.<\/p>\n

\u201cThis tech is really new, and it\u2019s basically unregulated, and there are a lot of open questions about it and how it\u2019s going to impact kids,\u201d said Cross, director of PIRG\u2019s Our Online Life Program.\u00a0\u201cRight now, if I were a parent, I wouldn\u2019t be giving my kids access to a chatbot or a teddy bear that has a chatbot inside of it.\u201d<\/p>\n

More on AI toys: AI-Powered Toys Caught Telling 5-Year-Olds How to Find Knives and Start Fires With Matches<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"Children\u2019s toymaker FoloToy says it\u2019s pulling its AI-powered teddy bear \u201cKumma\u201d after a safety group found that the…\n","protected":false},"author":2,"featured_media":293525,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[182,181,507,74],"class_list":{"0":"post-293524","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/293524","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/comments?post=293524"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/293524\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media\/293525"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media?parent=293524"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/categories?post=293524"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/tags?post=293524"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}