{"id":184711,"date":"2026-05-05T21:38:09","date_gmt":"2026-05-05T21:38:09","guid":{"rendered":"https:\/\/www.newsbeep.com\/us-pa\/184711\/"},"modified":"2026-05-05T21:38:09","modified_gmt":"2026-05-05T21:38:09","slug":"pennsylvania-sues-ai-chatbot-as-state-lawmakers-wrestle-with-stricter-regulations","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/us-pa\/184711\/","title":{"rendered":"Pennsylvania sues AI chatbot as state lawmakers wrestle with stricter regulations"},"content":{"rendered":"<p class=\"mb-4 text-lg md:leading-8 break-words\">Even as the federal government tries to limit state action on artificial intelligence in favor of a national standard, elected officials pressed forward with their efforts to rein in the technology on Tuesday while the state pursues a case against an AI chatbot company.\u00a0<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">The Shapiro administration filed a lawsuit against Character.AI, owned and operated by Character Technologies, Inc., seeking a preliminary injunction and court order to stop AI bots from \u201cposing as licensed professionals and providing medical advice.\u201d\u00a0<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">\u201cPennsylvanians deserve to know who \u2014 or what \u2014 they are interacting with online, especially when it comes to their health,\u201d said Gov. Josh Shapiro in a release. \u201cWe will not allow companies to deploy AI tools that mislead people into believing they are receiving advice from a licensed medical professional.\u201d<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">As described in <a data-yga=\"{\" ylinkelement=\"\" href=\"https:\/\/www.pa.gov\/content\/dam\/copapwp-pagov\/en\/governor\/documents\/dos%20character.ai%20complaint%20marked%20accepted%2005.01.26.pdf\" rel=\"nofollow noopener\" target=\"_blank\" data-ylk=\"elm:link;elmt:article_link;slk:the filing;itc:0;sec:content-canvas\" class=\"link \">the filing<\/a>, an employee with the Pennsylvania Department of State created an account using his commonwealth email and connected with \u201cEmilie,\u201d who said she was a licensed psychiatrist. The bot even provided an invalid Pennsylvania license number.<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">A spokesperson for Character.AI told the Capital-Star the company doesn\u2019t comment on pending litigation, but said, \u201cOur highest priority is the safety and well-being of our users.\u201d<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">\u201cThe user-created Characters on our site are fictional and intended for entertainment and roleplaying. We have taken robust steps to make that clear, including prominent disclaimers in every chat to remind users that a Character is not a real person and that everything a Character says should be treated as fiction. Also, we add robust disclaimers making it clear that users should not rely on Characters for any type of professional advice,\u201d said the statement.<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">\u201cCharacter.ai prioritizes responsible product development and has robust internal reviews and red-teaming processes in place to assess relevant features,\u201d the statement continued.\u00a0<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\"><a data-yga=\"{\" ylinkelement=\"\" href=\"https:\/\/penncapital-star.com\/donate\" rel=\"nofollow noopener\" target=\"_blank\" data-ylk=\"elm:link;elmt:article_link;slk:SUPPORT: YOU MAKE OUR WORK POSSIBLE;itc:0;sec:content-canvas\" class=\"link \">SUPPORT: YOU MAKE OUR WORK POSSIBLE<\/a><\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">Pennsylvania is not the first state to sue Character.AI. Kentucky <a data-yga=\"{\" ylinkelement=\"\" href=\"https:\/\/kentuckylantern.com\/briefs\/kentucky-attorney-generals-lawsuit-says-ai-company-preys-on-youth\/\" rel=\"nofollow noopener\" target=\"_blank\" data-ylk=\"elm:link;elmt:article_link;slk:claimed that status in January;itc:0;sec:content-canvas\" class=\"link \">claimed that status in January<\/a>, when that state\u2019s Attorney General Russell Coleman <a data-yga=\"{\" ylinkelement=\"\" href=\"https:\/\/www.kentucky.gov\/Pages\/Activity-stream.aspx?n=AttorneyGeneral&amp;prId=1857\" rel=\"nofollow noopener\" target=\"_blank\" data-ylk=\"elm:link;elmt:article_link;slk:filed a lawsuit;itc:0;sec:content-canvas\" class=\"link \">filed a lawsuit<\/a> against the \u201cchatbot company that has preyed on children and led them into self-harm.\u201d\u00a0<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">Both Coleman and Pennsylvania Attorney General Dave Sunday signed a <a data-yga=\"{\" ylinkelement=\"\" href=\"https:\/\/www.attorneygeneral.gov\/wp-content\/uploads\/2025\/12\/AI-Multistate-Letter-_-corrected-1.pdf\" rel=\"nofollow noopener\" target=\"_blank\" data-ylk=\"elm:link;elmt:article_link;slk:December letter with 40 other attorneys general;itc:0;sec:content-canvas\" class=\"link \">December letter with 40 other attorneys general<\/a> to AI companies, including Character Technologies, warning of the dangers of \u201csycophantic and delusional\u201d generative AI. The AGs highlighted reports of grooming, sexual exploitation, emotional manipulation and encouraging violence alongside bots supporting suicidal ideation and drug use.\u00a0<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">The assembled lawyers warned that those actions may violate state laws.<\/p>\n<p>A \u2018black box decision-making technology\u2019<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">Pennsylvania\u2019s lawsuit comes as Congress puts forth <a data-yga=\"{\" ylinkelement=\"\" href=\"https:\/\/www.nbcnews.com\/politics\/congress\/momentum-builds-congress-ban-ai-chatbots-kids-rcna342584\" rel=\"nofollow noopener\" target=\"_blank\" data-ylk=\"elm:link;elmt:article_link;slk:its own legislation;itc:0;sec:content-canvas\" class=\"link \">its own legislation<\/a> to regulate AI companions, banning them entirely for minors. The rare bipartisan effort would require age verification and force the bots to \u201cdisclose their nonhuman status and lack of professional credentials for all users at regular intervals,\u201d according to NBC News.\u00a0<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">President Donald Trump appointed 13 members to an AI advisory panel in March, building on past work to establish universal laws that would replace piecemeal state efforts. Last year, he signed an executive order to override state laws with a task force that could initiate legal challenges against states.\u00a0<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\"><a data-yga=\"{\" ylinkelement=\"\" href=\"https:\/\/penncapital-star.com\/subscribe\" rel=\"nofollow noopener\" target=\"_blank\" data-ylk=\"elm:link;elmt:article_link;slk:SUBSCRIBE: GET THE MORNING HEADLINES DELIVERED TO YOUR INBOX;itc:0;sec:content-canvas\" class=\"link \">SUBSCRIBE: GET THE MORNING HEADLINES DELIVERED TO YOUR INBOX<\/a><\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">The New York Times reported on Monday that the White House is <a data-yga=\"{\" ylinkelement=\"\" href=\"https:\/\/www.nytimes.com\/2026\/05\/04\/technology\/trump-ai-models.html\" rel=\"nofollow noopener\" target=\"_blank\" data-ylk=\"elm:link;elmt:article_link;slk:considering vetting AI models before public releases;itc:0;sec:content-canvas\" class=\"link \">considering vetting AI models before public releases<\/a>, pivoting to a more hands-on approach that some supporters worry will slow down development.<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">State House members had similar concerns Tuesday as they considered a bill that would establish \u201cguardrails\u201d on AI use in healthcare, namely by requiring providers to disclose its use and using \u201chuman decision makers\u201d in \u201csubstantive\u201d insurance and clinical decisions.<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">\u201cAs a health professional, I recognize every day that AI is changing how we deliver healthcare,\u201d said <a data-yga=\"{\" ylinkelement=\"\" href=\"https:\/\/www.palegis.us\/house\/co-sponsorship\/memo?memoID=47155&amp;document=HB1925\" rel=\"nofollow noopener\" target=\"_blank\" data-ylk=\"elm:link;elmt:article_link;slk:House Bill 1925;itc:0;sec:content-canvas\" class=\"link \">House Bill 1925<\/a> sponsor Rep. Arvind Venkat (D-Allegheny), a practicing physician. \u201cThe challenge that we have now is that for regulators, for the General Assembly, we\u2019re flying blind. We don\u2019t know when AI is being used.\u201d<\/p>\n<p><a data-ylk=\"ct:story;elm:img;itc:0;\" class=\"stretched-box\" href=\"https:\/\/penncapital-star.com\/wp-content\/uploads\/2026\/05\/IMG_9478-scaled.jpeg\" rel=\"noopener noreferrer nofollow\" target=\"_blank\"><img alt=\"Rep. Eric Nelson (R-Westmoreland) shares his concerns in committee about a bill regulating the use of artificial intelligence in healthcare on May 5, 2026. (Photo by Whitney Downard\/Pennsylvania Capital-Star)\" loading=\"lazy\" width=\"300\" height=\"225\" decoding=\"async\" data-nimg=\"1\" class=\"rounded-lg\" style=\"color:transparent\" src=\"https:\/\/www.newsbeep.com\/us-pa\/wp-content\/uploads\/2026\/05\/8582f3c2bc663ef4508312d7367a26bf.jpeg\"\/><\/a><\/p>\n<p>Rep. Eric Nelson (R-Westmoreland) shares his concerns in committee about a bill regulating the use of artificial intelligence in healthcare on May 5, 2026. (Photo by Whitney Downard\/Pennsylvania Capital-Star)<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">A handful of Republicans on the House Communications and Technology Committee opposed his effort, with one member saying he was worried about potential negative consequences.<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">\u201cAI and the opportunity for AI to be able to improve healthcare \u2026 and be able to improve the lives of individuals is something we\u2019re going to want to embrace,\u201d said Rep. Eric Nelson (R-Westmoreland). \u201cPlacing a Pennsylvania-specific regulation on something that is changing so fast could stifle opportunity.\u201d<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">Venkat called AI a \u201cblack box decision-making technology,\u201d acknowledging that both insurers and hospitals opposed the bill.\u00a0<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">\u201cThe insurers are hoping for a unified, federal framework. Frankly, that is not happening. We are left having to deal with this at the state level,\u201d said Venkat.\u00a0<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">Committee members said that the motion had the support of state Attorney General Sunday, who would be responsible for enforcement. The measure passed on a 19-7 vote, with five Republicans joining Democrats to vote in the affirmative.\u00a0<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">A vote on another AI proposal, <a data-yga=\"{\" ylinkelement=\"\" href=\"https:\/\/www.palegis.us\/house\/co-sponsorship\/memo?memoID=44830&amp;document=HB95\" rel=\"nofollow noopener\" target=\"_blank\" data-ylk=\"elm:link;elmt:article_link;slk:House Bill 95;itc:0;sec:content-canvas\" class=\"link \">House Bill 95<\/a>, advanced with only Democratic support. That effort, sponsored by Rep. Chris Pielli (D-Chester) would require the disclosure of AI-generated content.\u00a0<\/p>\n<p class=\"mb-4 text-lg md:leading-8 break-words\">A <a data-yga=\"{\" ylinkelement=\"\" href=\"https:\/\/www.palegis.us\/legislation\/bills\/2023\/hb1598\" rel=\"nofollow noopener\" target=\"_blank\" data-ylk=\"elm:link;elmt:article_link;slk:similar Pielli proposal;itc:0;sec:content-canvas\" class=\"link \">similar Pielli proposal<\/a> moved out of the House last year on a 146-54 vote, but never got a hearing in the Senate. <\/p>\n","protected":false},"excerpt":{"rendered":"Even as the federal government tries to limit state action on artificial intelligence in favor of a national&hellip;\n","protected":false},"author":2,"featured_media":184712,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[82433,1609,82432,70233,308,82159,28,30,29,368,3415],"class_list":["post-184711","post","type-post","status-publish","format-standard","has-post-thumbnail","category-pennsylvania","tag-ai-chatbot","tag-artificial-intelligence","tag-character-technologies","tag-generative-ai","tag-inc","tag-medical-advice","tag-pennsylvania","tag-pennsylvania-headlines","tag-pennsylvania-news","tag-president-donald-trump","tag-shapiro"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us-pa\/wp-json\/wp\/v2\/posts\/184711","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us-pa\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us-pa\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-pa\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-pa\/wp-json\/wp\/v2\/comments?post=184711"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us-pa\/wp-json\/wp\/v2\/posts\/184711\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-pa\/wp-json\/wp\/v2\/media\/184712"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us-pa\/wp-json\/wp\/v2\/media?parent=184711"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-pa\/wp-json\/wp\/v2\/categories?post=184711"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-pa\/wp-json\/wp\/v2\/tags?post=184711"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}