A Sydney high school senior has been referred to police by the NSW Department of Education for allegedly creating explicit deepfake pornographic images of his fellow students.<\/p>\n
eSafety Commissioner Julie Inman Grant said the “nudify” services had been used to “devastating effect” in Australian schools.<\/p>\n
“With this major provider blocking their use by Australians, we believe it will have a tangible impact on the number of Australian school children falling victim to AI-generated child sexual exploitation,” she said.<\/p>\n
Ms Inman Grant said eSafety took action against the UK company in September because it failed to put safeguards in place to prevent its services from being used to generate abusive material.<\/p>\n
She said the company was even marketing features like “undressing ‘any girl,’ and with options for ‘schoolgirl’ image generation and features such as ‘sex mode'”.<\/p>\n
<\/p>\n
Julie Inman Grant says a “cultural change” is needed to stop the concerning trend. (ABC News: Craig Hansen)<\/p>\n
Ms Inman Grant told the ABC that reports of deep fake image-based abuse had doubled in the past 18 months, but she believed this was “just the tip of the iceberg”.<\/p>\n
“What we’re seeing in a lot of these cases \u2026 these are teenage boys mostly who think it’s a bit of a laugh to harvest their female classmates’ photos from social media or elsewhere and generate these sexualised videos without really understanding the kind of long-term harm and humiliation and denigration they’re causing,” she said.<\/p>\n
Ms Inman Grant said, in her view, there must be a unified effort to create “cultural change” and stop the concerning trend.<\/p>\n
“We need to be supplementing our consent education and our respectful relationships education to really be able to reinforce that digital harms are real harms, and they have long-term, significant impacts on people,” she said.<\/p>\n