{"id":140802,"date":"2026-01-20T01:23:06","date_gmt":"2026-01-20T01:23:06","guid":{"rendered":"https:\/\/www.newsbeep.com\/us-ca\/140802\/"},"modified":"2026-01-20T01:23:06","modified_gmt":"2026-01-20T01:23:06","slug":"ca-orders-elon-musks-ai-firm-to-immediately-stop-sharing-sexual-deepfakes","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/us-ca\/140802\/","title":{"rendered":"CA Orders Elon Musk\u2019s AI Firm to Immediately Stop Sharing Sexual Deepfakes"},"content":{"rendered":"

California Attorney General Rob Bonta sent a cease and desist letter<\/a> to xAI ordering the Elon Musk company to stop creating and distributing nonconsensual sexual images.<\/p>\n

In the letter, Bonta cited \u201cnumerous examples of xAI taking ordinary, clothed images of women and children\u201d and allowing users \u201cto depict the people in suggestive and sexually explicit scenarios and \u2018undress\u2019 them, all without the subjects\u2019 knowledge or consent.\u201d<\/p>\n

Providing such imagery, Bonta wrote, violates state laws related to public decency<\/a> and a \u201cdeepfake\u201d pornography law that went into effect<\/a> just two weeks ago. Bonta said he appreciates that xAI took steps to address the issue in recent days, such as blocking Grok use<\/a> in places where generating images of people in bikinis is illegal, but he said the impact of those changes is unclear.<\/p>\n

Bonta sent the letter Friday. Two days prior, he announced an investigation into how and whether xAI and its social platform X broke the law by enabling the spread of naked or sexual imagery without consent. That came after an update to the company\u2019s Grok artificial intelligence tool to allow image editing last month.<\/p>\n

\u201cThe avalanche of reports detailing the non-consensual sexually explicit material that xAI has produced and posted online in recent weeks is shocking,\u201d Bonta wrote Wednesday. \u201cThis material\u2026 has been used to harass people across the internet. I urge xAI to take immediate action to ensure this goes no further.\u201d<\/p>\n

Bonta urged Californians who want to report depictions of them or their children undressed or committing sexual acts to visit oag.ca.gov\/report<\/a>. In a response emailed Wednesday, xAI did not address questions about the investigation.<\/p>\n

Research obtained by Bloomberg<\/a> found that X users utilizing Grok posted more non-consensual naked or sexual imagery than those of any other website. In a posting on X, Musk promised \u201cconsequences\u201d for people who made illegal content with the tool. On Friday, Grok limited image editing to paying subscribers.<\/p>\n

X and xAI appear to be violating the provisions of the pornographic deepfake ban, known as AB 621, said Sam Dordulian, who previously worked in the sex crimes unit of the Los Angeles District Attorney\u2019s Office but today works in private practice as a lawyer for people in cases involving deepfakes or revenge porn.<\/p>\n

Assemblymember Rebecca Bauer-Kahan, author of the law, told CalMatters in a statement last week that she reached out to prosecutors, including the attorney general\u2019s office and the city attorney of San Francisco, to remind them that they can act under the law. What\u2019s happening on X, Bauer-Kahan said, is what AB 621 was designed to address.<\/p>\n

\u201cReal women are having their images manipulated without consent, and the psychological and reputational harm is devastating,\u201d the San Ramon Democrat wrote. \u201cUnderage children are having their images used to create child sexual abuse material, and these websites are knowingly facilitating it.\u201d<\/p>\n

A global concern<\/p>\n

Bonta\u2019s inquiry also comes shortly after a call for an investigation<\/a> by Gov. Gavin Newsom, backlash from regulators in the European Union and India and bans on X in Malaysia, Indonesia, and potentially the United Kingdom. As Grok app downloads rise in Apple and Google app store<\/a>s, lawmakers and advocates are calling for the smartphone makers to prohibit the application.<\/p>\n

Why Grok created the feature the way it did and how it will respond to the controversy around it is unclear, and answers may not be forthcoming, since an analysis recently concluded<\/a> that it\u2019s the least transparent of major AI systems available today.<\/p>\n

Evidence of concrete harm from deepfakes is piling up. In 2024, the FBI warned that use of deepfake tools to extort young people is a growing problem<\/a> that has led to instances of self harm and suicide. Multiple audits have found that child sexual abuse material is inside the training data of AI models<\/a>, making them capable of generating vulgar photos. A 2024 Center for Democracy and Technology survey<\/a> found that 15% of high school students have heard of or seen sexually explicit imagery of someone they know at school in the past year.<\/p>\n

The investigation announced today is the latest action by the attorney general to push AI companies to keep kids safe. Late last year, Bonta endorsed a bill that would have prevented chatbots that talk about self harm and engage in sexually explicit conversations\u00a0 from interacting with people under 18. He also joined attorneys general from 44 other states in sending a letter that questions why companies like Meta and OpenAI allow their chatbots to have sexually inappropriate conversations with minors.<\/p>\n

California has passed roughly half a dozen laws since 2019 to protect people from deepfakes. The new law by Bauer-Kahan<\/a> amends and strengthens a 2019 law, most significantly by allowing district attorneys to bring cases against companies that \u201crecklessly aid and abet\u201d the distribution of deepfakes without the consent of the person depicted nude or committing sexual acts. That means the average person can ask the attorney general or the district attorney where they live to file a case on their behalf. It also increases the maximum amount that a judge can award a person from $150,000 to $250,000. Under the law, a public prosecutor is not required to prove that an individual depictured in an AI generated nude or sexual image suffered actual harm to bring a case to court. Websites that refuse to comply within 30 days can face penalties of $25,000 per violation.<\/p>\n

In addition to those measures, two 2024 laws (AB 1831<\/a> and SB 1381<\/a>) expand the state\u2019s definition of child pornography to make possession or distribution of artificially-generated child sexual abuse material illegal. Another required<\/a> social media platforms to give people an easy way to request the immediate removal of a deepfake, and defines the posting of such material as a form of digital identity theft. A California law limiting the use of deepfakes in elections was signed into law last year but was struck down by a federal judge last summer<\/a> following a lawsuit by X and Elon Musk.<\/p>\n

Future reforms<\/p>\n

Every new state law helps give lawyers like Dordulian a new avenue to address harmful uses of deepfakes, but he said more needs to be done to help people protect themselves. He said his clients face challenges proving violation of existing laws since they require distribution of explicit materials, for example with a messaging app or social media platform, for protections to kick in. In his experience, people who use nudify apps typically know each other, so distribution doesn\u2019t always take place, and if it does, it can be hard to prove.<\/p>\n

For example, he said, he has a client who works as a nanny who alleges that the father of the kids she takes care of made images of her using photos she posted on Instagram. The nanny found the images on his iPad. This discovery was disturbing for her and caused her emotional trauma, but since he can\u2019t use deepfake laws he has to sue on the basis of negligence or emotional distress and laws that were never created to address deepfakes. Similarly, victims told CNBC last year<\/a> that the distinction between creating and distributing deepfakes left a gap in the law in a number of U.S. states.<\/p>\n

\u201cThe law needs to keep up with what\u2019s really happening on the ground and what women are experiencing, which is just the simple act of creation itself is the problem,\u201d Dordulian said.<\/p>\n

California is at the forefront of passing laws to protect people from deepfakes, but existing law isn\u2019t meeting the moment, said Jennifer Gibson, cofounder and director of Psst<\/a>, a group created a little over a year ago that provides pro bono legal services to tech and AI workers interested in whistleblowing. A California law that went into effect Jan. 1<\/a> protects whistleblowers inside AI companies but only if they work on catastrophic risk that can kill more than 50 people or cause more than $1 billion in damages. If the law protected people who work on deepfakes, former X employees who detailed witnessing Grok generating illegal sexually explicit material last year to Business Insider<\/a> would, Gibson said, have had protections if they\u00a0 shared the information with authorities.<\/p>\n

\u201cThere needs to be a lot more protection for exactly this kind of scenario in which an insider sees that this is foreseeable, knows that this is going to happen, and they need somewhere to go to report to both to keep the company accountable and protect the public.\u201d<\/p>\n

Khari Johnson is a reporter with CalMatters.<\/p>\n","protected":false},"excerpt":{"rendered":"California Attorney General Rob Bonta sent a cease and desist letter to xAI ordering the Elon Musk company…\n","protected":false},"author":2,"featured_media":140803,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[29],"tags":[2080,9778,1611,71041,88,90,89,71042,11931,42518],"class_list":["post-140802","post","type-post","status-publish","format-standard","has-post-thumbnail","category-san-jose","tag-artificial-intelligence","tag-attorney-general-rob-bonta","tag-elon-musk","tag-online-images","tag-san-jose","tag-san-jose-headlines","tag-san-jose-news","tag-sexually-explicit-images","tag-x","tag-xai"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/140802","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/comments?post=140802"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/140802\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media\/140803"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media?parent=140802"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/categories?post=140802"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/tags?post=140802"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}