A disturbing and dangerous trend has surfaced on X, with users misusing the platform\u2019s AI tool, Grok, to morph photographs of women and children into sexually compromising images. The development has triggered global outrage and renewed concerns over AI-enabled sexual abuse.<\/p>\n
The trend began a couple of days ago and escalated on New Year\u2019s Eve, spreading rapidly across the platform. Users were seen issuing direct prompts to Grok to digitally manipulate images of women and children, turning ordinary photographs into explicit and abusive content. These images were then circulated widely without consent, exposing victims to humiliation, harassment, and harm.<\/p>\n
Women\u2019s rights activists and users across countries have been mounting intense pressure on Elon Musk to immediately fix the feature that allows such abuse. While X has reportedly hidden Grok\u2019s media feature, the misuse has not stopped. Images can still be morphed, shared, and accessed on the platform.
The trend has now reached Indian users on X, with experts warning that the issue goes far beyond online mischief or trolling. Cyber-safety specialists and gender-rights advocates say the morphing of images using AI amounts to a form of sexual violence, particularly when it involves women and children. They argue that such acts violate dignity, bodily autonomy, and consent, and can cause severe psychological trauma to victims whose images are weaponised without their knowledge.
The continued availability of morphed images on X, despite partial restrictions, has intensified criticism that the platform is failing to adequately protect users. Worried women users are deleting their pictures.<\/p>\n
Cyber-security expert Ritesh Bhatia told CNBC-TV18, \u201cWhy are we asking or expecting victims to be careful at all? This isn\u2019t about caution; it\u2019s about accountability. When a platform like Grok even allows such prompts to be executed, the responsibility squarely lies with the intermediary. Technology is not neutral when it follows harmful commands. If a system can be instructed to violate dignity, the failure is not human behaviour alone \u2014 it is design, governance, and ethical neglect. Creators of Grok need to take immediate action.\u201d<\/p>\n
Discussing legal remedies, cyber-law expert Adv. Prashant Mali told CNBC-TV18,\u00a0\u201cI feel this is not mischief \u2014 it is AI-enabled sexual violence. Victims have clear remedies under the IT Act, 2000, especially Sections 66E (violation of privacy) and 67\/67A (publishing or transmitting obscene or sexually explicit content), which squarely cover AI-generated morphed images even if no physical act occurred.<\/p>\n
\u201cUnder the Bharatiya Nyaya Sanhita, 2023, Section 77 (voyeurism) and allied provisions on sexual harassment and the dignity of women criminalise creation and circulation of such material, recognising harm to autonomy, not just physical exposure. Where the victim is a minor, POCSO is triggered immediately, with Sections 11, 12, 13, and 14 treating AI-generated sexualised images as aggravated sexual exploitation, regardless of \u2018virtual\u2019 excuses, making punishment swift and non-negotiable. Add to this the Intermediary Rules, which mandate rapid takedown and traceability.\u201d<\/p>\n
He further added,\u00a0\u201cThe legal framework is robust on paper. The real challenge lies in the speed of enforcement and digital-forensics capacity, not the absence of law. I also feel the defence of \u2018it was just an AI\u2019 will not survive judicial scrutiny.\u201d<\/p>\n
As calls grow louder for accountability, activists are demanding stricter controls on AI image tools, swift takedown mechanisms, and legal action against those generating and circulating abusive content. The Grok controversy has once again exposed the darker side of generative AI and raised urgent questions about whether social-media platforms are equipped \u2014 or willing \u2014 to prevent technology from being used as a tool of sexual harm.<\/p>\n","protected":false},"excerpt":{"rendered":"A disturbing and dangerous trend has surfaced on X, with users misusing the platform\u2019s AI tool, Grok, to…\n","protected":false},"author":2,"featured_media":386393,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,207363,207357,254,255,64,63,27089,207360,134028,3120,207359,161752,207361,207365,207362,207358,207364,105,188780,207356],"class_list":{"0":"post-386392","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-ai-ethics","10":"tag-ai-image-morphing","11":"tag-artificial-intelligence","12":"tag-artificialintelligence","13":"tag-au","14":"tag-australia","15":"tag-child-protection","16":"tag-cyber-law-india","17":"tag-deepfake-images","18":"tag-elon-musk","19":"tag-generative-ai-misuse","20":"tag-grok-ai","21":"tag-it-act","22":"tag-online-image-abuse","23":"tag-pocso-law","24":"tag-sexual-abuse-online","25":"tag-social-media-safety","26":"tag-technology","27":"tag-women-safety","28":"tag-x-platform"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/386392","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=386392"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/386392\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/386393"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=386392"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=386392"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=386392"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}