The default applies only to crawlers that mix AI training with traditional search, only on ad supported pages, and only to new and free Cloudflare customers, leaving paid enterprise users and most existing sites untouched.
Cloudflare is installing itself as a commercial middleman between publishers and the AI companies that scrape the web, and it is using its position as one of the internet's largest network operators to do it. On July 1, the company announced a change to its default settings, branded "Content Independence Day": starting September 15, Cloudflare will block what it calls "mixed-use" crawlers from any web page that serves advertising, unless the site owner opts in. The new default applies to all new Cloudflare customers, every new site set up by an existing customer, and all existing free customers, according to the company's changelog and CEO Matthew Prince's blog post.
The mechanism matters more than the announcement. Cloudflare sits between millions of websites and the rest of the internet, handling traffic, security, and performance for roughly a fifth of the web, according to web technology survey firm W3Techs (W3Techs, July 2026). By switching the default for ad-supported pages from "allow unless blocked" to "block unless allowed," the company makes itself the place where any commercial deal between publishers and AI companies has to be struck. Cloudflare is doing more than offering an opt-out toggle. It is positioning itself as the paid broker between publishers and AI labs. TechCrunch, The Register, NBC News, and Forbes all frame the change as a forced commercial relationship rather than a simple policy update.
Cloudflare is openly targeting Google. Prince argued in his post that the world's largest search engine has access to "roughly 2x more information" than its AI competitors, because publishers cannot opt out of Google's crawlers without sacrificing discoverability in conventional search. That figure is Cloudflare's own framing, not an independent measurement, and the company is using it to argue the policy corrects a market imbalance. Google already offers a tool called Google-Extended that lets publishers block its AI products, including Gemini Apps and the Vertex API, from training on their content while staying visible in regular Google Search, according to Cloudflare's own summary of Google's position. The complication is that Googlebot, which powers Google Search, also feeds Google's AI Overviews and AI Mode, so the boundary between "search" and "AI" crawl traffic is blurrier than either company is making it sound.
The September 15 deadline is the verifiable hook, and the question of what counts as "mixed-use" will decide how much the policy actually changes. Cloudflare's framing carves out crawlers used purely for traditional search while catching crawlers that serve both AI training or inference and conventional indexing. Googlebot itself is not classified as "mixed-use" in Cloudflare's announcement, so the immediate bite against Google's AI features depends on how Cloudflare categorizes that bot going forward. The default change also does not touch Cloudflare's paid enterprise customers, who keep their current settings, which limits how widely the new rules will land on day one.
What to watch on September 15: which crawlers Cloudflare puts on its block list, how it handles Googlebot, and whether the publisher-AI partnerships the company is hinting at become revenue-sharing deals rather than press releases. The "Content Independence Day" framing suggests Cloudflare wants more than a one-off policy reset. It wants to be the place where the open web gets priced for AI.