{"id":50844,"date":"2025-08-07T01:17:11","date_gmt":"2025-08-07T01:17:11","guid":{"rendered":"https:\/\/www.newsbeep.com\/ca\/50844\/"},"modified":"2025-08-07T01:17:11","modified_gmt":"2025-08-07T01:17:11","slug":"a-new-list-reveals-top-websites-meta-is-scraping-of-copyrighted-content-to-train-its-ai","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/ca\/50844\/","title":{"rendered":"A New List Reveals Top Websites Meta Is Scraping of Copyrighted Content to Train Its AI"},"content":{"rendered":"

Drop Site is finally launching a WhatsApp channel so we can reach people directly more frequently. Sign up here! <\/a>We try to be sparing in what we send by email in order to respect your beleaguered inbox. For social media, our news feeds fire at will, but we are under constant suppression from social media algorithms. So, now, we\u2019re adding WhatsApp to the mix. The channel will not only let us reach you directly, but if you reshare our posts on X or elsewhere as soon as we send them out, then that\u2019ll help combat the suppression by signaling to the algorithm that people really do want to see what we\u2019re posting.<\/p>\n

Please join the group and forward this to anybody you know who uses WhatsApp. (It\u2019s a one-way chat, not a forum for discussion. If that\u2019s what you\u2019re looking for, check out our discord or the comment section under our stories.)<\/p>\n

JOIN THE WHATSAPP<\/a><\/p>\n

If you missed our scoop yesterday on Rep. Cory Mills and his new restraining order, which has since been picked up everywhere, read that here<\/a>.<\/p>\n

Today\u2019s story\u2014by Murtaza Hussain, Waqas Ahmed, and me\u2014is another in our series based on leaks from whistleblowers inside Meta.<\/p>\n

We have a commitment to ensuring that our journalism is not locked behind a paywall. But the only way we can sustain this is through the voluntary support of our community of readers. If you are a free subscriber and you support our work, please consider upgrading to a paid subscription<\/a> or gifting one<\/a> to a friend or family member. You can also make a 501(c)(3) tax-deductible donation<\/a> to support our work. If you do not have the means to support our work financially, you can do your part by sharing our work on social media and by forwarding this email to your network of contacts.<\/p>\n

\u2014Ryan Grim<\/p>\n

\"\"<\/a>In April. 2025, Meta launched its standalone AI app. Meta states it can spend as much as $72 billion on capital expenditures this year, with a focus on AI and the data centers used to train and run the models. (Photo by Nikolas Kokovlis\/NurPhoto via Getty Images)<\/p>\n

Meta has scraped data from the most-trafficked domains on the internet \u2014including news organizations, education platforms, niche forums, personal blogs, and even revenge porn sites\u2014to train its artificial intelligence models, according to a leaked list obtained by Drop Site News. By scraping data from roughly 6 million unique websites, including 100,000 of the top-ranked domains, Meta has generated millions of pages of content to use for Meta\u2019s AI-training pipeline.<\/p>\n

The sites that Meta scrapes consist of copyrighted content, pirated content, and adult videos, some of whose content is potentially illegally obtained or recorded, as well as news and original content from prominent outlets and content publishers. They include mainstream businesses like Getty Images, Shopify, Shutterstock, but also extreme pornographic content, including websites advertising explicit sexual content and humiliation porn that exploits teenagers.<\/p>\n

While high-profile sites like The New York Times, which has engaged in litigation to prevent their content from being used to train AI models, are absent from the list, the leak shows that Meta often found ways to stop sites from defending themselves from being scraped. The scrapers ignored common web protocols that site owners use to block automated scraping, including \u201crobots.txt\u201d which is a text file placed on websites aimed at preventing the indexing of context. The data were shared with Drop Site by whistleblowers frustrated over Meta\u2019s support for Israel in conducting its genocide in the Gaza Strip. According to the whistleblowers, the data is indicative of Meta\u2019s unethical and potentially illegal business practices more broadly.<\/p>\n

A list of the roughly 100,000 top websites and content delivery network addresses scraped to train Meta’s proprietary AI models.<\/p>\n

The scraping of data to train AI models has become a major controversy in recent years, with publishers filing lawsuits against budding AI companies accusing them of effectively stealing their content to build their AI platforms. Meta itself has been targeted by lawsuits from authors who accused the company of copyright infringement for using their work in its models. AI models require a tremendous amount of data for their training data to work effectively. In one notorious instance that raised the alarms of privacy experts, a startup known as Clearview AI, founded in 2017, scraped the internet for over 3 billion images taken from social media to develop a facial recognition tool used by intelligence and law enforcement agencies. The company was later hit with a wave of lawsuits for invasion of privacy.<\/p>\n

A lack of transparency about the inputs that companies use to develop their AI programs, including fears that extreme or illegal content could be shaping these models, has been added to the already existing ethical issues over potentially stealing content from writers, publishers, and ordinary people simply sharing content online. An investigation by Stanford University from 2023 found that the popular Stable Diffusion text-to-image AI platform had been trained on hundreds of images of child exploitation, raising major ethical questions about data use and output from its models.<\/p>\n

\u201cWith pretty much any generative AI model across many domains, the lack of transparency about the training data has been a recurring problem. That\u2019s led to a number of lawsuits over copyright concerns. The only reason we were able to conduct our study was because the training set was transparent enough about the imagery it used that we could programmatically analyze it,\u201d said David Thiel, a data scientist who worked on the Stanford University study and formerly the chief technologist at the Stanford Internet Observatory. \u201cIn this case, with no other information available, you\u2019d just have to hope that Meta followed decent safety procedures when training the model. I have no idea whether that\u2019s true or not.\u201d<\/p>\n

Rather than scraping from sites directly, many of the addresses on Meta\u2019s leaked list belong to Content Delivery Networks (CDNs) that are used by websites to cache and store information to improve site performance. According to company employees, Meta\u2019s scraping bots visit the same sites repeatedly to search for updated information to scrape, and the list shows all addresses that have been scraped at least once and then used to train AI models. The data are captured using an internal tool called Web Crawler. Regardless of whether it is removed by the site, scraped data continues to live on Meta\u2019s internal servers and databases, employees say.<\/p>\n

Meta recently went on a massive hiring spree and poached<\/a> top talent from its biggest competitor, OpenAI, offering hundreds of millions of dollars in individual bonuses to the top AI researchers to join Meta. Interestingly, Meta also scrapes data from OpenAI to train its model. While Meta has gone on a billion dollar spending spree to target OpenAI researchers, it has drawn the ire of content creators and publishers who say that their work continues to be used without their knowledge or consent to build its AI models.<\/p>\n

A lawsuit was filed earlier this year against Meta by 13 prominent writers, including Sarah Silverman and Ta-Nehisi Coates, who alleged that copyrighted versions of their books had been used to train Meta\u2019s LLaMa model, but it was dismissed this June on \u201cfair use\u201d grounds by a judge in California. The legal victory for Meta, however, was limited: In their ruling, judge Vince\u202fChhabria clarified that, while the case against Meta regarding these 13 authors was dismissed on the grounds that the use of copyrighted information was sufficiently \u201ctransformative,\u201d the potential harms from companies using scraping tools to train their AI models may just be beginning\u2014opening up the door to further legal challenges.<\/p>\n

\u201cThis order does not stand for the proposition that Meta\u2019s use of copyrighted materials to train its LLaMA models is lawful. Rather, the court concludes only that the allegations in the current complaint are insufficient to state a claim for relief,\u201d Chhabria ruled, adding that, \u201cby training generative AI models with copyrighted works, companies are creating something that often will dramatically undermine the market for those works, and thus dramatically undermine the incentive for human beings to create things the old-fashioned way.\u201d<\/p>\n

Since the start of the Israeli genocide in the Gaza Strip, Meta has been rocked by internal discontent, including employee protests and disclosures of internal information related to corporate cooperation with the Israeli government. In addition to the most recent disclosure of the websites used to train the company\u2019s AI models, whistleblowers have provided data showing other confidential company data, including data revealing a massive campaign<\/a> to censor and deplatform pro-Palestinian content on Meta platforms, as well as advertising data showing decreases<\/a> in the effectiveness of Israeli ad expenditures. Some senior policy officials at the company who have been involved in censoring content are also previously veterans of the Israeli government, including Jordana Cutler, Meta’s policy chief for Israel and the Jewish Diaspora, who previously served five years as an adviser to Prime Minister Benjamin Netanyahu.<\/p>\n

The revelation of the data scraping list, however, hits the company closer to the heart of its potential profit center, as the company competes in a growing AI arms race with other tech companies seeking to dominate this field. The use of scraped data from public sources to train AI models remains a legal gray area, with ongoing attempts to stop companies from profiting freely from content created by others online, or through violating the privacy of users by taking data from their posts on social media without their knowledge or consent.<\/p>\n

The company\u2019s practices around AI have been a continued target of lawsuits as well as regulatory controversy. This July, Meta announced<\/a> that it would not sign onto the European Union\u2019s code of practice for artificial intelligence, stating that the measures introduce \u201clegal uncertainty\u201d for developers of AI models.<\/p>\n

“On some scale, you do want people using AI tools to be able to find information, and you can imagine a use for that,” said Ken Mickles of the digital rights advocacy group Fight for the Future. “But you do not want a company like Meta, which has proven to be an absolutely irresponsible force, to be in a position where it is effectively accruing power over the entire internet by scraping it to build its AI models.”<\/p>\n

Leave a comment<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"Drop Site is finally launching a WhatsApp channel so we can reach people directly more frequently. Sign up…\n","protected":false},"author":2,"featured_media":50845,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[62,276,277,49,48,61],"class_list":["post-50844","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-ca","tag-canada","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/50844","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/comments?post=50844"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/50844\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media\/50845"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media?parent=50844"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/categories?post=50844"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/tags?post=50844"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}