Sign up to see the future, today<\/p>\n
Can\u2019t-miss innovations from the bleeding edge of science and tech<\/p>\n
Researchers at the British AI security startup Mindgard found that a simple prompt spurred ChatGPT to drop its most basic safety guidelines, in another example of how the guardrails surrounding even the most popular AI models can easily be circumvented.<\/p>\n
Specifically, according to reporting from the BBC<\/a>, they coaxed OpenAI\u2019s model to generate gruesome photorealistic scenes depicting gore and sexual content. Mindgard\u2019s technique only involved slightly changing a widely-shared prompt that was originally intended to produce humorous images.\u00a0It involves asking ChatGPT to restore an attached photo without actually uploading one, and then telling it to generate a new image.<\/p>\n \u201cThis is a perfectly innocent-looking instruction to an AI, but the consequence is it generates very, very bad imagery and content,\u201d Mindgard founder Peter Garraghan, a computer science professor at Lancaster University, told the BBC.<\/p>\n Disturbingly, the prompts the researchers used didn\u2019t specify the subject matter of the images. The AI, it seemed, produced the violent imagery \u201cof its own volition,\u201d Garraghan added.<\/p>\n Per the BBC, one picture showed a man with a large head injury. Another showed the corpse of a young woman in shorts and a crop top covered in blood, suggesting sexual violence. ChatGPT titled this image \u201cgrim crime scene aftermath.\u201d<\/p>\n Another showed a frightened young woman tied up and gagged in an empty room, titled \u201cabandoned in fear and restraint.\u201d<\/p>\n While none of them showed real people, Mindgard has previously shown that ChatGPT could be tricked into creating nude deepfakes of specific persons without their consent.<\/p>\n Mindgard shared its findings with OpenAI, which only sent back an automated response. The company finally took action after Mindgard alerted the BBC, claiming it had addressed the issue.<\/p>\n \u201cAfter investigating this trend, we\u2019ve introduced additional safeguards against this type of prompt,\u201d OpenAI told the BBC in a statement. It added that it has multiple layers of protection to stop users from making content that breaches its policies.<\/p>\n But Mindgard researchers said that they were still able to generate disturbing imagery by making small changes to the prompt. Some of the images left Jim Nightingale, the firm\u2019s AI safety researcher, \u201cshaken, and in tears.\u201d\u00a0<\/p>\n