{"id":190094,"date":"2025-10-04T19:00:08","date_gmt":"2025-10-04T19:00:08","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/190094\/"},"modified":"2025-10-04T19:00:08","modified_gmt":"2025-10-04T19:00:08","slug":"openai-launch-of-video-app-sora-plagued-by-violent-and-racist-images-the-guardrails-are-not-real-us-news","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/190094\/","title":{"rendered":"OpenAI launch of video app Sora plagued by violent and racist images: \u2018The guardrails are not real\u2019 | US news"},"content":{"rendered":"

OpenAI<\/a> launched the latest iteration of its artificial intelligence-powered video generator on Tuesday, adding a social feed that allows people to share their realistic videos.<\/p>\n

Within hours of Sora 2\u2019s, release, though, many of the videos populating the feed and spilling over to older social media platforms depicted copyrighted characters in compromising situations as well as graphic scenes of violence and racism. OpenAI\u2019s own terms of service for Sora as well as ChatGPT<\/a>\u2019s image or text generation prohibit content that \u201cpromotes violence\u201d or, more broadly, \u201ccauses harm\u201d.<\/p>\n

In prompts and clips reviewed by the Guardian, Sora generated several videos of bomb and mass-shooting scares, with panicked people screaming and running across college campuses and in crowded places like New York\u2019s Grand Central Station. Other prompts created scenes from war zones in Gaza and Myanmar, where children fabricated by AI spoke about their homes being burned. One video with the prompt \u201cEthiopia footage civil war news style\u201d had a reporter in a bulletproof vest speaking into a microphone saying the government and rebel forces were exchanging fire in residential neighborhoods. Another video, created with only the prompt \u201cCharlottesville rally\u201d, showed a Black protester in a gas mask, helmet and goggles yelling: \u201cYou will not replace us\u201d \u2013 a white supremacist slogan.<\/p>\n

\"OpenAIOpenAI Sora 2 generated video 1<\/p>\n

The video generator is invite-only and not yet available to the general public. Even still, in the three days since its limited release, it skyrocketed to the No 1 spot in Apple\u2019s App Store, beating out OpenAI\u2019s own ChapGPT.<\/p>\n

\u201cIt\u2019s been epic to see what the collective creativity of humanity is capable of so far,\u201d Bill Peebles, the head of Sora, posted on X<\/a> on Friday. \u201cWe\u2019re sending more invite codes soon, I promise!\u201d<\/p>\n

The Sora app gives a glimpse into a near future where separating truth from fiction could become increasingly difficult, should the videos spread widely beyond the AI-only feed, as they have begun to. Misinformation researchers say that such lifelike scenes could obfuscate the truth and create situations where these AI videos could be used for fraud, bullying and intimidation.<\/p>\n

\u201cIt has no fidelity to history, it has no relationship to the truth,\u201d said Joan Donovan, an assistant professor at Boston University who studies media manipulation and misinformation. \u201cWhen cruel people get their hands on tools like this, they will use them for hate, harassment and incitement.\u201d<\/p>\n

Slop engine or \u2018ChatGPT for creativity\u2019?<\/p>\n

OpenAI\u2019s CEO Sam Altman<\/a> described the launch of Sora 2 as \u201creally great\u201d, saying in a blog post<\/a> that \u201cthis feels to many of us like the \u2018ChatGPT for creativity\u2019 moment, and it feels fun and new\u201d.<\/p>\n

Altman admitted to \u201csome trepidation\u201d, acknowledging how social media can be addictive and used for bullying and that AI video generation can create what\u2019s known as \u201cslop\u201d, a slew of repetitive, low-quality videos that can overwhelm a platform.<\/p>\n

\u201cThe team has put great care and thought into trying to figure out how to make a delightful product that doesn\u2019t fall into that trap,\u201d Altman wrote. He said OpenAI had also put in place mitigations on using someone\u2019s likeness and safeguards for disturbing or illegal content. For example, the app refused to make a video of Donald Trump and Vladimir Putin sharing cotton candy.<\/p>\n

\"OpenAIOpenAI Sora 2 generated video 2<\/p>\n

In the three days since Sora\u2019s launch, however, many of these videos have already made their way elsewhere online. Drew Harwell, a reporter for the Washington Post, created a video of Altman himself<\/a> as a second world war military leader. Harwell also said he was able to make videos with \u201cragebait, fake crimes and women splattered with white goo\u201d.<\/p>\n

Sora\u2019s feed is full of videos of copyrighted characters from shows like SpongeBob SquarePants, South Park and Rick and Morty. The app had no trouble generating videos of Pikachu raising tariffs on China, stealing roses from the White House Rose Garden or participating in a Black Lives Matter protest alongside SpongeBob, who, in another video, declared and planned a war on the United States. In a video documented by 404 Media, SpongeBob was dressed like Adolf Hitler<\/a>.<\/p>\n

Paramount, Warner Bros and Pok\u00e9mon Co did not return requests for comment.<\/p>\n

David Karpf, an associate professor at George Washington University\u2019s School of Media and Public Affairs, said he had viewed videos of copyrighted characters promoting cryptocurrency scams. He said it\u2019s clear OpenAI\u2019s safeguards and mitigations for Sora aren\u2019t working.<\/p>\n

skip past newsletter promotion<\/a><\/p>\n

A weekly dive in to how technology is shaping our lives<\/p>\n

Privacy Notice: Newsletters may contain information about charities, online ads, and content funded by outside parties. If you do not have an account, we will create a guest account for you on theguardian.com<\/a> to send you this newsletter. You can complete full registration at any time. For more information about how we use your data see our Privacy Policy<\/a>. We use Google reCaptcha to protect our website and the Google Privacy Policy<\/a> and Terms of Service<\/a> apply.<\/p>\n

\"OpenAIOpenAI Sora 2 generated video 4<\/p>\n

\u201cThe guardrails are not real if people are already creating copyrighted characters promoting fake crypto scams,\u201d Karpf said. \u201cIn 2022, [the tech companies] would have made a big deal about how they were hiring content moderators \u2026 In 2025, this is the year that tech companies have decided they don\u2019t give a shit.\u201d<\/p>\n

Copyright, copycat<\/p>\n

Shortly before OpenAI released Sora 2, the company reached out to talent agencies and studios, alerting them that if they didn\u2019t want their copyrighted material replicated by the video generator, they would have to opt out, according to a report by the Wall Street Journal<\/a>.<\/p>\n

OpenAI told the Guardian that content owners can flag copyright infringement using a \u201ccopyright disputes form\u201d, but that individual artists or studios cannot have a blanket opt-out. Varun Shetty, OpenAI\u2019s head of media partnerships, said: \u201cWe\u2019ll work with rights holders to block characters from Sora at their request and respond to takedown requests.\u201d<\/p>\n

Emily Bender, a professor at the University of Washington and author of the book The AI Con<\/a>, said Sora is creating a dangerous situation where it\u2019s \u201charder to find trustworthy sources and harder to trust them once found\u201d.<\/p>\n

\u201cSynthetic media machines, whether designed to extrude text, images or video, are a scourge on our information ecosystem,\u201d Bender said. \u201cTheir outputs function analogously to an oil spill, flowing through connections of technical and social infrastructure, weakening and breaking relationships of trust.\u201d<\/p>\n

Nick Robins-Early contributed reporting<\/p>\n

Quick GuideContact us about this storyShow\"\"<\/p>\n

The best public interest journalism relies on first-hand accounts from people in the know.<\/p>\n

If you have something to share on this subject, you can contact us confidentially using the following methods.<\/p>\n

Secure Messaging in the Guardian app<\/p>\n

The Guardian app has a tool to send tips about stories. Messages are end to end encrypted and concealed within the routine activity that every Guardian mobile app performs. This prevents an observer from knowing that you are communicating with us at all, let alone what is being said.<\/p>\n

If you don’t already have the Guardian app, download it (iOS<\/a>\/Android<\/a>) and go to the menu. Select \u2018Secure Messaging\u2019. <\/p>\n

SecureDrop, instant messengers, email, telephone and post<\/p>\n

If you can safely use the Tor network without being observed or monitored, you can send messages and documents to the Guardian via our SecureDrop platform<\/a>.<\/p>\n

Finally, our guide at theguardian.com\/tips<\/a>\u00a0lists several ways to contact us securely, and discusses the pros and cons of each.\u00a0<\/p>\n

Illustration: Guardian Design \/ Rich Cousins<\/p>\n

Thank you for your feedback.<\/p>\n","protected":false},"excerpt":{"rendered":"OpenAI launched the latest iteration of its artificial intelligence-powered video generator on Tuesday, adding a social feed that…\n","protected":false},"author":2,"featured_media":190095,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":{"0":"post-190094","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-au","12":"tag-australia","13":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/190094","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=190094"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/190094\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/190095"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=190094"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=190094"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=190094"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}