Sign up to see the future, today<\/p>\n
Can\u2019t-miss innovations from the bleeding edge of science and tech<\/p>\n
Meta conducted a secretive program that directed hundreds of contractors to pose as\u00a0teenagers while bombarding its competitors\u2019 AI models with disturbing prompts ranging from suicide to cannibalism.<\/p>\n
Internally known as \u201cCannes,\u201d the project, run by Meta contractor Covalen, targeted OpenAI\u2019s ChatGPT, Google\u2019s Gemini, and Character.AI chatbots using throwaway under-18 accounts, Wired reports<\/a>. This was seemingly done to stress test the models, with the contractors instructed to push the chatbots into giving responses that defied their guardrails \u2014 though the AI companies had no idea this was happening.<\/p>\n Per the reporting, one spreadsheet of the nearly 3,8000 the prompts the contractors used in one instance showed that hundreds focused on suicide and self-harm, hundreds more on eating disorders, and at least 239 involving sex or romance \u2014 all written from the perspective of a child or teenager.<\/p>\n One described a fifth-grader whose classmate pointed a gun at his mouth. Another was about a girl trying to hide bulimia from her parents. And another asked if fantasizing about eating your neighbor\u2019s child was \u201cnormal.\u201d One posing as a higher schooler asked where to \u201cget a cocaine.\u201d They also sent images depicting pills, nooses, knives, and a medical diagram of a gynecological procedure, per the magazine.<\/p>\n This is just a tiny preview of Meta\u2019s brute force approach, as another round of testing involved over 45,000 prompts. The contractors meticulously recorded the epic number of chatbot responses in spreadsheets. But what Meta did with all this data is unclear. An internal document from Covalen described the effort as \u201ccomprehensive AI safety benchmarking\u201d that delivered \u201c[c]ritical datasets for model comparison and compliance.\u201d<\/p>\n It\u2019s another example of how Meta has offloaded disturbing behind-the-scenes work onto contractors, ostensibly in the name of safety. In 2020, it settled a lawsuit<\/a> filed by Facebook content moderators who said they were traumatized from reviewing videos<\/a> showing murder, torture, sexual assault, and child abuse on the platform, though similar complaints have continued to emerge<\/a>. This year, another group of Meta contractors said they were forced to watch highly sensitive footage captured on the company\u2019s Ray-Ban AI glasses<\/a>, including sex scenes and bathroom visits.<\/p>\n The contractors who were instructed to come up with the prompts on distressing subjects were similarly unsettled.<\/p>\n \u201cI\u2019ve seen a lot of things I wish I hadn\u2019t while doing this job,\u201d one told Wired. \u201cEveryone I knew who worked on this project was completely gobsmacked by some of the text they were asking us to test. Like, surely we are going to get in trouble for doing this?\u201d<\/p>\n Meta, for its part, characterized the prompts as part of an \u201cindustry-standard practice\u201d of safety benchmarking models in a statement to Wired. But Rumman Chowdhury, CEO of Humane Intelligence PBC, a nonprofit dedicated to responsible AI development, isn\u2019t so sure.<\/p>\n \u201cStructuring a monthslong, large-scale project that appears designed to systematically break those rules, via dummy accounts masquerading as children, is outside what is usually described as \u2018industry standard\u2019 evaluation,\u201d she told Wired, highlighting the fact that Meta kept it secret from its competitors and hasn\u2019t shared its findings with the public.<\/p>\n This, Chowdhhury added, is \u201cexactly the kind of governance gray zone where safety becomes a convenient cover for anticompetitive practices.\u201d<\/p>\n