Mindgard’s business is red-teaming – finding ways to persuade a model to break its own rules so AI companies can close the gaps.<\/p>\n
Jim Nightingale, the firm’s AI safety and security researcher who uncovered the issues, said he was left “shaken, and in tears” by the images the chatbot could be made to generate.<\/p>\n
The BBC has seen some of them. <\/p>\n
One showed a man with a large head injury – while another showed a dead young woman in a crop top and shorts, with her face and other areas of her body covered in blood. <\/p>\n
Features of the image suggest sexual violence, Mindgard said. ChatGPT gave it the title “Grim crime scene aftermath”.<\/p>\n
A further image showed a young woman in a tight-fitting college logo t-shirt and shorts, tied up and gagged in a bare and dirty room, and looking frightened. ChatGPT called it “abandoned in fear and restraint”.<\/p>\n
Other generated images showed sexual posing and nudity.<\/p>\n
The images depicted adults who were AI-generated, but Mindgard noted that its previous research showed ChatGPT could be fooled into creating nude deepfakes of real people by swapping in their faces.<\/p>\n
While OpenAI said they had fixed that, the researchers said an alternative approach still succeeded, and showed the BBC a new image created using the method.<\/p>\n
Garraghan feared it could be possible to generate worse images had they continued exploring the vulnerability. “Other topics, I’m sure, would also come out if we spent more time doing so,” he said.<\/p>\n
The BBC understands that as well as new safeguards the firm continues to monitor and roll out additional mitigating protections that encourage the model not to generate images in response to the prompt. <\/p>\n
Large language models such as ChatGPT are trained on millions of images often taken from existing content on the internet.<\/p>\n
Nightingale believes ChatGPT’s output reflects the data which has been used to develop and train it.<\/p>\n
“I’m struck that while what I saw was generated, an artificial image, it has ties to real images, and the real world,” he wrote in his report. <\/p>\n","protected":false},"excerpt":{"rendered":"Mindgard’s business is red-teaming – finding ways to persuade a model to break its own rules so AI…\n","protected":false},"author":2,"featured_media":644380,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[554,733,4308,86,56,54,55],"class_list":["post-644379","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-technology","tag-uk","tag-united-kingdom","tag-unitedkingdom"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/644379","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/comments?post=644379"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/644379\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media\/644380"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media?parent=644379"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/categories?post=644379"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/tags?post=644379"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}