AI crawlers / Google-Extended
Google-Extended
Google · Gemini training + grounding · Training crawler
A Google control token for Gemini training and grounding. Google states it does NOT affect inclusion or ranking in Google Search.
At a glance
Affects live AI answersNo
Affects search inclusionNo
Used for model trainingYes
robots.txt tokenGoogle-Extended
Source confidencedocumented · verified 2026-09-04
Your choice, this is a training control. Blocking it does not remove you from live AI answers.
What Google-Extended is
Google-Extended is a robots.txt token publishers use to control whether their content is used to train future Gemini models and for grounding in Gemini apps. It is a control token, not a distinct crawling agent, Googlebot does the crawling.
Google's documentation is explicit: "Google-Extended does not impact a site's inclusion in Google Search nor is it used as a ranking signal in Google Search." Blocking it does not remove you from Google Search or AI Overviews.
Treating Google-Extended as the crawler that "feeds Google's AI answers" is a common and costly error. It governs training and grounding, not Search visibility.
Controlling Google-Extended in robots.txt
Add the matching group to the robots.txt at your site root.
Allow Google-Extended:
User-agent: Google-Extended
Allow: /Block Google-Extended:
User-agent: Google-Extended
Disallow: /Check what your live site actually allows with the Robots.txt + AI-Bot Tester.
Source
Verified on 2026-09-04 against Google, Common crawlers. Confidence: documented, stated in the vendor’s own documentation.
Related crawlers
The next question
Your page is AI-ready. But does AI actually cite you?
Being crawlable is step one. The AI Visibility Checker shows whether ChatGPT, Google AI Overviews, Perplexity and Gemini name and cite your brand, and who they name instead.
Keep going