Meta Delays the Terms That Let It Crawl Business Websites – www.thekeyword.co

One email. Everything that moved in ads, search and social, and what it means for your spend.
Join the newsletter and get our bitesized update straight to your inbox.
Google | Meta | ChatGPT| TikTok
Editor, The Keyword
Meta has moved its updated Commercial Terms from 30 October to 20 November.
A new clause lets Meta crawl websites linked to any business using its products.
Publishers asked UK and EU competition authorities to pause the terms.
Robots.txt appears in Meta’s statements but nowhere in the contract itself.
Its statement promises an explanation before 20 November and names no changes.
Meta has pushed back its updated Commercial Terms from 30 October to 20 November, after news publishers asked regulators in the UK and Europe to stop a clause that lets Meta crawl the websites of businesses that use its products. Meta’s updated terms page now carries the new date.
A Meta spokesperson told Reuters the company is “extending the effective date of our updated Commercial Terms and Self-Serve Ad Terms to 20 November to allow more time for us to explain to our partners and stakeholders what’s changing and what’s not.” The European Commission said it welcomed the delay amid discussions with the company, Reuters reported.
Under the new clause, a business using Meta’s products authorises Meta “to use automated software (such as web scrapers, bots, robots, spiders, crawlers, and user agents) and create test credentials” to access and crawl the websites associated with its products or services. Meta may also “periodically pull updates” from those sites.
The stated purpose is to “provide, develop, improve, research, and maintain advertising and business products.” The current Commercial Terms, in force since March 2024, contain no comparable permission and do not mention crawlers, scrapers or bots.
The clause is written for any business, not only publishers. An advertiser running ads on Facebook or Instagram would grant the same permission over the websites behind its products.
Almost every news organisation runs a Facebook Page, a business profile or an ad account on Meta. Publishers therefore read the clause as reaching all of their journalism, not only what they choose to post on Meta.
In the UK, the News Media Association asked the Competition and Markets Authority (CMA) to secure a pause. The European press publishers’ associations EMMA, ENPA, EPC and NME asked the European Commission to adopt interim measures, and in the US the News/Media Alliance condemned the terms.
On 22 October, WAN-IFRA’s letter to Meta brought those complaints together. The World Association of News Publishers represents more than 20,000 media brands in 120 countries, and its letter went to Joel Kaplan, Meta’s chief global affairs officer. It asks Meta to:
“For many local and independent newsrooms, Facebook and WhatsApp are how they reach their communities, so walking away is not an option,” said Irene Lanzaco, chair of WAN-IFRA’s Directors of Member Associations Committee.
Before the delay, a Meta spokesperson told Adweek that “the crawling terms we consolidated only applied to our ads and business products,” that the update “does not change how Meta uses web crawlers,” and that “content owners can use robots.txt to signal their preferences.” Meta has rejected the publishers’ claims, according to Reuters.
The updated terms do not mention robots.txt, the file a website uses to tell crawlers what they may read. The permission to crawl is written into the contract, while the commitment to respect that file sits in a spokesperson’s statement.
WAN-IFRA makes the same point. It also says Meta removed an explicit reference to AI training from the clause after the first criticism, and argues that the remaining wording still allows content to be used to develop Meta’s business products, which include generative AI services.
Meta’s statement promises to explain the terms before 20 November and names no change to the clause. The spokesperson added that Meta will use the time to set out how it uses web crawlers and to make clear that content owners can still use robots.txt to signal preferences, according to Reuters. WAN-IFRA CEO Stig Ørskov said on Friday: “This postponement is a welcome step, but it does not address our concerns.”
For a business with a Meta ad account, continuing to use Meta’s business tools after 20 November counts as accepting the terms. WAN-IFRA also ties the clause to Meta One, the subscription Meta opened to businesses in September, saying Meta is testing limits under which Facebook Pages must pay to post more than two links a month to their own websites. The same update adds an AI liability clause for advertisers, delayed to the same date.
Meta has moved the effective date of its updated Commercial Terms and Self-Serve Ad Terms from 30 October to 20 November 2026. It says the extra time is to explain to partners and stakeholders what is changing and what is not.
Businesses using Meta’s products authorise Meta to use automated software such as crawlers and bots, and to create test credentials, to access the websites associated with their products or services and to pull updates from them periodically. The terms say the purpose is to provide, develop, improve, research and maintain advertising and business products.
From 20 November, continuing to use Meta’s business tools counts as accepting the clause. Meta says site owners can still use robots.txt to signal their preferences, but the terms do not mention robots.txt, which is why publishers want it written in.
trusted by over 200,000 Marketers

source