AI crawlers / GPTBot
GPTBot
OpenAI · OpenAI model training · Training crawler
OpenAI's model-training crawler. Blocking it opts your content out of training, it does NOT remove you from ChatGPT search answers.
At a glance
Affects live AI answersNo
Affects search inclusion,
Used for model trainingYes
robots.txt tokenGPTBot
Source confidencedocumented · verified 2026-09-04
Your choice, this is a training control. Blocking it does not remove you from live AI answers.
What GPTBot is
GPTBot is the crawler OpenAI uses to collect content that may improve its generative AI foundation models. OpenAI states GPTBot is "used to make our generative AI foundation models more useful and safe," and that "disallowing GPTBot indicates a site's content should not be used in training generative AI foundation models."
GPTBot is a training control, not a discovery crawler. Blocking GPTBot does not remove you from ChatGPT's live search answers, that is governed by OAI-SearchBot. Whether to allow GPTBot is a separate decision about model training.
A common mistake is treating GPTBot as "the ChatGPT crawler." It is not the crawler that decides citation eligibility in ChatGPT search.
Controlling GPTBot in robots.txt
Add the matching group to the robots.txt at your site root.
Allow GPTBot:
User-agent: GPTBot
Allow: /Block GPTBot:
User-agent: GPTBot
Disallow: /Check what your live site actually allows with the Robots.txt + AI-Bot Tester.
Source
Verified on 2026-09-04 against OpenAI, Overview of OpenAI Crawlers. Confidence: documented, stated in the vendor’s own documentation.
The next question
Your page is AI-ready. But does AI actually cite you?
Being crawlable is step one. The AI Visibility Checker shows whether ChatGPT, Google AI Overviews, Perplexity and Gemini name and cite your brand, and who they name instead.
Keep going