{"id":134960,"date":"2025-11-12T01:14:07","date_gmt":"2025-11-12T01:14:07","guid":{"rendered":"https:\/\/www.newsbeep.com\/ie\/134960\/"},"modified":"2025-11-12T01:14:07","modified_gmt":"2025-11-12T01:14:07","slug":"tech-companies-and-uk-child-safety-agencies-to-test-ai-tools-ability-to-create-abuse-images-artificial-intelligence-ai","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/ie\/134960\/","title":{"rendered":"Tech companies and UK child safety agencies to test AI tools\u2019 ability to create abuse images | Artificial intelligence (AI)"},"content":{"rendered":"

Tech companies and child protection agencies will be given the power to test whether artificial intelligence tools can produce child abuse images<\/a> under a new UK law.<\/p>\n

The announcement was made as a safety watchdog revealed that reports of AI-generated child sexual abuse material [CSAM] have more than doubled in the past year from 199 in 2024 to 426 in 2025.<\/p>\n

Under the change, the government will give designated AI companies and child safety organisations permission to examine AI models \u2013 the underlying technology for chatbots such as ChatGPT and image generators such as Google\u2019s Veo 3 \u2013 and ensure they have safeguards to prevent them from creating images of child sexual abuse<\/a>.<\/p>\n

Kanishka Narayan, the minister for AI and online safety, said the move was \u201cultimately about stopping abuse before it happens\u201d, adding: \u201cExperts, under strict conditions, can now spot the risk in AI models early.\u201d<\/p>\n

The changes have been introduced because it is illegal to create and possess CSAM, meaning that AI developers and others cannot create such images as part of a testing regime. Until now, the authorities have had to wait until AI-generated CSAM is uploaded online before dealing with it. This law is aimed at heading off that problem by helping to prevent the creation of those images at source.<\/p>\n

The changes are being introduced by the government as amendments to the crime and policing bill, legislation which is also introducing a ban on possessing, creating or distributing AI models<\/a> designed to generate child sexual abuse material.<\/p>\n

This week Narayan visited the London base of Childline, a helpline for children, and listened to a mock-up of a call to counsellors featuring a report of AI-based abuse. The call portrayed a teenager seeking help after he had been blackmailed<\/a> by a sexualised deepfake of himself, constructed using AI.<\/p>\n

\u201cWhen I hear about children experiencing blackmail online, it is a source of extreme anger in me and rightful anger amongst parents,\u201d he said.<\/p>\n

The Internet Watch Foundation, which monitors CSAM online, said reports of AI-generated abuse material \u2013 such as a webpage that may contain multiple images \u2013 had more than doubled so far this year. Instances of category A material \u2013 the most serious form of abuse \u2013 rose from 2,621 images or videos to 3,086.<\/p>\n

Girls were overwhelmingly targeted, making up 94% of illegal AI images in 2025, while depictions of newborns to two-year-olds rose from five in 2024 to 92 in 2025.<\/p>\n

Kerry Smith, the chief executive of the Internet Watch Foundation, said the law change could \u201ca vital step to make sure AI products are safe before they are released\u201d.<\/p>\n

\u201cAI tools have made it so survivors can be victimised all over again with just a few clicks, giving criminals the ability to make potentially limitless amounts of sophisticated, photorealistic child sexual abuse material,\u201d she said. \u201cMaterial which further commodifies victims\u2019 suffering, and makes children, particularly girls, less safe on and off line.\u201d<\/p>\n

Childline also released details of counselling sessions where AI has been mentioned. AI harms mentioned in the conversations include: using AI to rate weight, body and looks; chatbots dissuading children from talking to safe adults about abuse; being bullied online with AI-generated content; and online blackmail using AI-faked images.<\/p>\n

Between April and September this year, Childline delivered 367 counselling sessions where AI, chatbots and related terms were mentioned, four times as many as in the same period last year. Half of the mentions of AI in the 2025 sessions were related to mental health and wellbeing, including using chatbots for support and AI therapy apps.<\/p>\n","protected":false},"excerpt":{"rendered":"Tech companies and child protection agencies will be given the power to test whether artificial intelligence tools can…\n","protected":false},"author":2,"featured_media":134961,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[220,218,219,61,60,80],"class_list":{"0":"post-134960","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-ie","12":"tag-ireland","13":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/134960","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/comments?post=134960"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/134960\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media\/134961"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media?parent=134960"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/categories?post=134960"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/tags?post=134960"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}