Gaatha Sarvaiya would like to post on social media and share her work online. An Indian law graduate in her early 20s, she is in the earliest stages of her career and trying to build a public profile. The problem is, with AI-powered deepfakes on the rise, there is no longer any guarantee that the images she posts will not be distorted into something violating or grotesque.<\/p>\n
\u201cThe thought immediately pops in that, \u2018OK, maybe it\u2019s not safe. Maybe people can take our pictures and just do stuff with them,\u2019\u201d says Sarvaiya, who lives in Mumbai.<\/p>\n
\u201cThe chilling effect is true,\u201d says Rohini Lakshan\u00e9, a researcher on gender rights and digital policy based in Mysuru who also avoids posting photos of herself online. \u201cThe fact that they can be so easily misused makes me extra cautious.\u201d<\/p>\n
The consequence of facing online harassment is silencing yourself or becoming less active onlineTarunima Prabhakar, Tattle<\/p>\n
In recent years, India has become one of the most important testing grounds for AI tools. It is the world\u2019s second-largest market for OpenAI<\/a>, with the technology widely adopted across professions<\/a>.<\/p>\n But a report released on Monday<\/a> that draws on data collected by the Rati Foundation, a charity running a countrywide helpline for victims of online abuse, shows that the rising adoption of AI has created a powerful new way to harass women.<\/p>\n \u201cIt has become evident in the last three years that a vast majority of AI-generated content is used to target women and gender minorities,\u201d says the report, authored by the Rati Foundation and Tattle<\/a>, a company that works to reduce misinformation on India\u2019s social media.<\/p>\n In particular, the report found an increase in AI tools being used to create digitally manipulated images or videos of women \u2013 either nudes or images that might be culturally appropriate in the US, but are stigmatising in many Indian communities, such as public displays of affection.<\/p>\n The Indian singer Asha Bhosle, left, and journalist Rana Ayyub, who have been affected by deepfake manipulation on social media. Photograph: Getty<\/p>\n About 10% of the hundreds of cases reported to the helpline now involve these images, the report found. \u201cAI makes the creation of realistic-looking content much easier,\u201d it says.<\/p>\n There have been high-profile cases of Indian women in the public sphere having their images manipulated by AI tools: for example, the Bollywood singer Asha Bhosle<\/a>, whose likeness and voice were cloned using AI and circulated on YouTube. Rana Ayyub, a journalist known for investigating political and police corruption, became the target of a doxing<\/a> campaign last year that led to deepfake sexualised images<\/a> of her appearing on social media.<\/p>\n These have led to a society-wide conversation, in which some figures, such as Bhosle, have successfully fought for legal rights over their voice or image<\/a>. Less discussed, however, is the effect that such cases have on ordinary women who, like Sarvaiya, feel increasingly uncertain about going online.<\/p>\n \u201cThe consequence of facing online harassment is actually silencing yourself or becoming less active online,\u201d says Tarunima Prabhakar, co-founder of Tattle. Her organisation used focus groups for two years across India to understand how digital abuse affected society.<\/p>\n \u201cThe emotion that we have identified is fatigue,\u201d she says. \u201cAnd the consequence of that fatigue is also that you just completely recede from these online spaces.\u201d<\/p>\n For the past few years, Sarvaiya and her friends have followed high-profile cases of deepfake online abuse, such as Ayyub\u2019s, or that of the Bollywood<\/a> actor Rashmika Mandanna. \u201cIt\u2019s a little scary for women here,\u201d she says.<\/p>\n<\/p>\n Now, Sarvaiya hesitates to post anything on social media and has made her Instagram private. Even this, she worries, will not be enough to protect her: women are sometimes photographed in public spaces such as the metro, and those pictures can later appear online<\/a>.<\/p>\n \u201cIt\u2019s not as common as you would think it is, but you don\u2019t know your luck, right?\u201d she says. \u201cFriends of friends are getting blackmailed \u2013 literally, off the internet.\u201d<\/p>\n Lakshan\u00e9 says she often asks not to be photographed at events now, even those where she is a speaker. But despite taking precautions, she is prepared for the possibility that a deepfake video or image of her might surface one day. On apps, she has made her profile picture an illustration of herself rather than a photo.<\/p>\n \u201cThere is fear of misuse of images, especially for women who have a public presence, who have a voice online, who take political stands,\u201d she says.<\/p>\n skip past newsletter promotion<\/a><\/p>\n Sign up to Global Dispatch<\/p>\n Get a different world view with a roundup of the best news, features and pictures, curated by our global development team<\/p>\n Privacy Notice: Newsletters may contain information about charities, online ads, and content funded by outside parties. If you do not have an account, we will create a guest account for you on theguardian.com<\/a> to send you this newsletter. You can complete full registration at any time. For more information about how we use your data see our Privacy Policy<\/a>. We use Google reCaptcha to protect our website and the Google Privacy Policy<\/a> and Terms of Service<\/a> apply.<\/p>\n after newsletter promotion<\/p>\n Rati\u2019s report outlines how AI tools, such as \u201cnudification\u201d or nudify apps<\/a> \u2013 which can remove clothes from images \u2013 have made cases of abuse once seen as extreme far more common. In one instance it described, a woman approached the helpline after a photo she submitted with a loan application was used to extort money from her.<\/p>\n \u201cWhen she refused to continue with the payments, her uploaded photograph was digitally altered using a nudify app and placed on a pornographic image,\u201d the report says.<\/p>\n That photograph, with her phone number attached, was circulated on WhatsApp, resulting in a \u201cbarrage of sexually explicit calls and messages from unknown individuals\u201d. The woman told Rati\u2019s helpline that she felt \u201cshamed and socially marked, as though she had been \u2018involved in something dirty\u2019\u201d.<\/p>\n A fake video ostensibly showing Rahul Gandhi, the Indian National Congress leader, and India\u2019s finance minister, Nirmala Sitharaman, promoting a financial scheme. Photograph: DAU Secretariat<\/p>\n In India, as in most of the world, deepfakes operate in a legal grey zone \u2013 no specific laws recognise them<\/a> as distinct forms of harm, although Rati\u2019s report outlines several Indian laws that could apply to online harassment and intimidation, under which women can report AI deepfakes.<\/p>\n \u201cBut that process is very long,\u201d says Sarvaiya, who has argued that India\u2019s legal system remains ill equipped to deal with AI deepfakes. \u201cAnd it has a lot of red tape to just get to that point to get justice for what has been done.\u201d<\/p>\n Part of the responsibility lies with the platforms on which these images are shared \u2013 often YouTube, Meta, X, Instagram and WhatsApp. Indian law enforcement agencies describe the process of getting these companies to remove abusive content as \u201copaque, resource-intensive, inconsistent and often ineffective\u201d, according to a report released on Tuesday by Equality Now<\/a>, which campaigns for women\u2019s rights.<\/p>\n While Apple<\/a> and Meta have recently taken steps<\/a> to limit the spread of nudify apps, Rati\u2019s report notes several instances in which these platforms responded inadequately to online abuse.<\/p>\n