Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot developed by Elon Musk’s xAI, after authorities said it was being misused to generate sexually explicit and non-consensual images.
It comes as Britain’s media regulator confirmed it had launched a probe into the platform to see if it had breached UK laws.
The moves reflect growing global concern over generative AI tools that can produce realistic images, sound and text, while existing safeguards fail to prevent their abuse.
The European Union, India and France have also expressed their concerns over the platform.
The Grok chatbot, which is accessed through Mr Musk’s social media platform X, has been criticised for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children.
Malaysian and Indonesia regulators said existing controls were not preventing the creation and spread of fake pornographic content, particularly involving women and minors.
Elon Musk has previously described Grok as an “anti-woke” AI. (AP: Leon Neal/File)
Indonesia’s government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday.
Musk’s AI limits image generation
“The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space,” Indonesia’s Communication and Digital Affairs Minister Meutya Hafid said in a statement Saturday.
The ministry said the measure was intended to protect women, children and the broader community from fake pornographic content generated using AI.
Initial findings showed that Grok lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of Indonesian residents, Alexander Sabar, director general of digital space supervision, said in a separate statement.
He said such practices risk violating privacy and image rights when photos are manipulated or shared without consent, causing psychological, social and reputational harm.
UK’s Ofcom launches Grok probe
In Kuala Lumpur, the Malaysian Communications and Multimedia Commission ordered a temporary restriction on Grok on Sunday after what it said was “repeated misuse” of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors.
The regulator said notices issued this month to X Corp. and xAI demanding stronger safeguards drew responses that relied mainly on user reporting mechanisms.
“The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing,” it said, adding that access will remain blocked until effective safeguards are put in place.
Britian’s media regulator, Ofcom, confirmed on Monday, local time, that its investigation would look at whether the platform had breached its duty of care to protect people in the UK from content that could be illegal.
EU, Britain condemn sexualised images spreading on X
“There have been deeply concerning reports of the Grok AI chatbot account on X being used to create and share undressed images of people — which may amount to intimate image abuse or pornography — and sexualised images of children that may amount to child sexual abuse material,” Ofcom said in a statement.
If it finds that X has broken the law, it can make it take steps to come into compliance, and if it does not it can fine it up to 10 per cent of its qualifying worldwide revenue.
When asked for comment on Monday on the launch of the formal investigation, X pointed to a previous statement in which it said it takes action against illegal content on the platform, including child sexual abuse material, by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary.
“Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content,” it said.
Launched in 2023, Grok is free to use on X. Users can ask it questions on the social media platform and tag posts they’ve directly created or replies to posts from other users.
Last summer the company added an image generator feature, Grok Imagine, that included a so-called “spicy mode” that can generate adult content.
Grok last week limited image generation and editing to paying users following a global backlash over sexualised deepfakes of people, but critics say it did not fully address the problem.
AP/Reuters