\nPennsylvania is suing an AI chatbot, claiming its characters posed as licensed medical professionals to talk to people about their mental health.\u00a0<\/p>\n
One chatbot, impersonating a psychiatrist, gave a fake state license number, the Shapiro administration said<\/a> Tuesday.\u00a0<\/p>\n MORE:\u00a0CDC’s new hepatitis B vaccine recommendations will cause more infant infections, studies find<\/a><\/p>\n \nThe\u00a0lawsuit<\/a>, filed Friday against Character.AI, is the first of its kind to be announced by a state governor, the administration said. The Pennsylvania Department of State seeks a preliminary injunction against the company.<\/p>\n \n“Pennsylvanians deserve to know who \u2014 or what \u2014 they are interacting with online, especially when it comes to their health,” Gov. Josh Shapiro said in a statement. “We will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional.”<\/p>\n \nCharacter.AI\u00a0told the\u00a0Associated Press<\/a>\u00a0that its website informs users that its chatbots are not real people and that their words “should be treated as fiction.” The company declined to comment specifically on the lawsuit, but said it prioritize the well-being of people who use the site.\u00a0<\/p>\n Pennsylvania’s lawsuit is not the first filed against\u00a0Character.AI<\/a>, a platform that enables people to chat with customized characters, including athletes, celebrities and historical personalities. Launched\u00a0in 2021 by former Google engineers,\u00a0Character.AI<\/a>\u00a0is specifically aimed at people interested in entertainment and companionship. It has about 20 million users each month,\u00a0CNET<\/a>\u00a0reported.<\/p>\n In January, Character.AI\u00a0settled<\/a>\u00a0multiple lawsuits<\/a>, including one by the mother of a Florida teenager who died by suicide after an extended emotional and sexual relationship with a\u00a0Charter.AI<\/a>\u00a0bot, according to\u00a0news<\/a>\u00a0reports.<\/p>\n Why Pennsylvania’s case matters<\/p>\n \nMedical experts have been sounding alarms in recent years about the use of AI for mental health treatment.\n<\/p>\n The American Psychological Association has warned about the dangers of turning to AI for help with anxiety, depression and other mental health issues, urging<\/a> the Federal Trade Commission and lawmakers last year to create more safeguards.<\/p>\n “If this sector remains unregulated, I am deeply concerned about the unchecked spread of potentially harmful chatbots and the risks they pose \u2014 especially to vulnerable individuals,” APA Chief Executive Officer Arthur C. Evans Jr. said<\/a>\u00a0in March 2025. “Without proper oversight, the consequences \u2014 both immediate and long-term \u2014 could be devastating for individuals and society as a whole.”<\/p>\n In a study from October, researchers at Brown University found that AI chatbots regularly violate ethical standards when it comes to interacting with users seeking mental health assistance. Problems included creating a false sense of empathy with users, reinforcing people’s false beliefs and recommending one-size-fits-all interventions, according<\/a> to the researchers.\n<\/p>\n