The two photos started out as typical teenage selfies: looking into the mirror, fully clothed. But once online predators had got hold of those pictures and ran them through an AI imaging tool, they had become the basis for extreme pornography videos.<\/p>\n
These examples come from the Report Remove<\/a> service, which allows children who have had explicit pictures of themselves distributed without their consent to flag the image confidentially and have it blocked or taken down from social media. Due to breakthroughs in AI, and the wide availability of AI models and nudification apps<\/a>, some under-18s are becoming victims without even being in contact with criminals.<\/p>\n These two victims show why the UK\u2019s National Crime Agency (NCA) and the Internet Watch Foundation (IWF), a child safety watchdog, have issued guidance telling parents to consider limiting visibility of family pictures by, for instance, making your social media profile private and just sharing photos of your children on \u201cclose friends\u201d groups.<\/p>\n Both organisations say they are not telling parents what to do but want them to be aware of the problem and what action to take. The data shows AI is an increasing source of child sexual abuse material. The amount of AI-generated child sexual abuse material [CSAM] found online rose by 14% last year, according to the IWF<\/a>, which identified 8,029 AI-made images and videos of realistic CSAM in 2025.<\/p>\n Dan Sexton, of the Internet Watch Foundation, feels uncomfortable telling parents not to post pictures of their children but feels he has no other option. Photograph: Graeme Robertson\/The Guardian<\/p>\n Dan Sexton, the IWF\u2019s chief technology officer, says he feels \u201cvery uncomfortable\u201d about telling parents not to put pictures of their children on public display but feels he has no other option. There is not enough protection against this technology, he says.<\/p>\n The UK government is bringing in restrictions, such as making it illegal to possess, create or distribute AI tools designed to generate child sexual abuse material. It is also giving tech companies and child protection agencies the power to test<\/a> whether artificial intelligence tools can produce child abuse images<\/a>.<\/p>\n Sexton acknowledges the long-term solution cannot be banishing photos from public social media accounts. He wants legislation for AI models that requires them to be \u201csafe by design\u201d, a principle voiced by online safety campaigners who want social media platforms to be built responsibly with user wellbeing as a core aim. \u201cPeople use the term safety by design but I am not seeing that here,\u201d he says.<\/p>\n The Department for Science, Innovation and Technology has been approached for comment.<\/p>\n