Google's Shared robots.txt Control for AI Training: What It Means
In short: Google announced on 30 October 2026 that it intends to support a cross-provider AI-training preference in robots.txt based on the IETF AIPREF draft, targeting 2027. It would let publishers express one common training preference while keeping company-specific controls such as Google-Extended. The shared standard is not live yet, so assess how it interacts with Gemini grounding and brand discovery before using blanket exclusions.
- Google intends to support a shared AI-training preference in robots.txt, targeting 2027.
- It is based on the IETF AIPREF draft and would sit alongside controls such as Google-Extended.
- It is not live yet.
- Blanket exclusions can affect how you appear in Gemini and AI answers. Decide on purpose.
What Google said
On 30 October 2026 Google said it plans to support a cross-provider AI-training preference in robots.txt, based on the IETF AIPREF draft, with a 2027 target. In practice, a publisher could state one training preference that many AI companies read, while still using company-specific controls such as Google-Extended.
Status
This is an intention, not a launch. Nothing changes in your robots.txt today.
Visibility versus protection
For a business that wants customers to find it, being used and cited by AI systems helps. Blocking everything can reduce how often you appear in Gemini and other AI answers. Keep two questions apart: may a model be trained on my content, and may my pages be used to answer questions and cited. Microsoft is separating these in Bing too: see Bing splits AI training and answer controls.
What to do now
- Open your robots.txt and note which AI crawlers you allow or block.
- Decide, per section of your site, what you want AI systems to do.
- Do not add blanket Disallow rules for AI crawlers unless you are sure you do not want to appear in AI answers.
- Review again when Google publishes the final syntax.
Want these checked on your own accounts? Request a free account review.
Get new marketing updates in your inbox.
We email you when we publish something worth your time. No spam, unsubscribe anytime.
Sources
- AI training control · Google Search Central
Facts in this article come from the sources above, checked on 1 November 2026. Platform features roll out in phases, so check your own account before acting. We update this page when official details change.