Analysis of 7 billion OpenAI bot events reveals that publishers blocking GPTBot to prevent AI training may be missing the more consequential crawler: OAI SearchBot, which determines ChatGPT search visibility. After GPT 5's August 2025 launch, OAI SearchBot activity surged 3.5x…
When a website decides to keep OpenAI out, the standard move is to block GPTBot — the crawler that feeds content into AI training. But that block does not touch a separate crawler called OAI-SearchBot, and that distinction may be costing publishers the visibility they actually care about.
A new analysis from Botify, an enterprise SEO platform, and Chris Long, co-founder of the SEO consultancy Nectiv, examined roughly 7 billion OpenAI bot log events from November 2024 through March 2026. The finding: OAI-SearchBot activity — the crawler responsible for surfacing content in ChatGPT search answers — jumped 3.5 times after GPT-5 launched in August 2025, adding 2.2 billion events to the dataset. GPTBot, the training crawler, grew at a slower 2.9 times over the same period.
The result is a structural flip. Before GPT-5, OAI-SearchBot and GPTBot were running at roughly even volumes in Botify's dataset — a ratio of about 0.95 search events per training event. After GPT-5, the ratio moved to 1.14. OpenAI is now spending more time searching the web than feeding it into training, at least by this measure.
OpenAI's own developer documentation confirms the two crawlers operate independently. A site can allow OAI-SearchBot while blocking GPTBot, or vice versa. "Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers," the documentation states. GPTBot and OAI-SearchBot are separate entries in robots.txt, and disabling one does not disable the other.
That matters because publisher attention has been focused on GPTBot. Roughly 25 percent of the top 1,000 websites by traffic now block GPTBot, up from 5 percent in early 2023, according to LLM Clicks. Many of those sites made the calculation that keeping their content out of AI training was worth the trade. But blocking GPTBot says nothing about whether OAI-SearchBot can read the same pages — and OAI-SearchBot is the crawler that determines whether ChatGPT will surface a site in its search results.
"If you block GPTBot, you are not automatically excluded from ChatGPT search answers," Long wrote in the Botify analysis. "Sites that block OAI-SearchBot will not appear in ChatGPT search answers."
The stakes have risen since GPT-5. Healthcare sites saw OAI-SearchBot activity surge 740 percent post-launch; media and publishing sites saw a 701 percent increase. Travel, retail, and software verticals also saw significant jumps, ranging from 30 to 215 percent. No sector saw a decline. Botify's data comes from its enterprise clients — large websites in retail, technology, publishing, and travel — so it represents the upper end of crawler interaction, not a cross-section of the entire web.
There is a wrinkle in the data that complicates the narrative. While OAI-SearchBot was climbing, a different signal was dropping. ChatGPT-User events — the crawler activity that fires when a ChatGPT session fetches a page on behalf of a human user — fell 28 percent between December 2025 and March 2026. Long offers two possible readings: fewer ChatGPT sessions are triggering real-time page fetches, or OpenAI is relying more on its stored index and fetching pages less often in real time. Botify's team favors the index explanation. Either way, it suggests the relationship between crawler activity and actual user behavior is not straightforward.
OpenAI is also still tiny compared to Google as a crawler. In the most recent 30-day window Botify measured, Googlebot logged 18.2 billion events against 887 million from all OpenAI crawlers combined — about 4 percent of Google's crawl volume. A year earlier, that figure was 1.38 percent. The gap is closing quickly, but Google remains roughly 20 times larger in raw crawl volume, Search Engine Journal reports.
Botify is an SEO software company, and the analysis promotes its own crawl intelligence product. The dataset skews toward large enterprise websites, which have more sophisticated robots.txt configurations and more to lose from crawler misconfiguration. That is both a strength — precise log data — and a limitation: the findings may not translate directly to smaller sites or different verticals.
The underlying tension the data exposes is real, though. OpenAI is building ChatGPT into something that looks more like a search engine every quarter. OAI-SearchBot's growth rate, combined with the industry-specific surge in media and healthcare, suggests the crawler is becoming a primary path through which certain publishers are discovered inside ChatGPT. For those publishers, the question of whether their robots.txt allows OAI-SearchBot is suddenly a strategic decision, not a technical footnote.
The majority of SEO discussion has centered on GPTBot — what blocking it means, whether training on your content is a fair exchange for visibility. That debate is not wrong. But it has obscured a narrower, more immediate question: if you want your site to appear in ChatGPT search answers, GPTBot is irrelevant. OAI-SearchBot is the only crawler that counts.