Quick answer: How do I rank in ChatGPT, Gemini, Claude and Perplexity?
- ChatGPT: allow OAI-SearchBot and OpenAI's search IP ranges; placement isn't guaranteed.
- Claude: allow Claude-SearchBot (search) and Claude-User (user-requested fetches).
- Perplexity: allow PerplexityBot; changes can take up to 24 hours.
- Gemini and Google AI: be indexed with a snippet; Google-Extended controls Gemini training and grounding in Gemini apps.
- Everywhere: answer-first pages, numbers and sources, consistent profiles, honest comparisons.
Key takeaways
- Each AI engine publishes its own crawler rules; blocking the wrong bot removes you from its answers.
- Search bots and training bots are separate for OpenAI, Anthropic and Perplexity; Google's Google-Extended also covers grounding in the Gemini apps.
- Google says its AI Overviews and AI Mode have no extra requirements beyond Search eligibility.
- Content that gets cited is specific: direct answers, statistics, quotations and sources.
- No engine guarantees placement; measure with referrals and repeated prompt checks.
"Rank in AI search" sounds like one job, but it's four sets of rules: OpenAI's for ChatGPT, Anthropic's for Claude, Perplexity's, and Google's for AI Overviews, AI Mode and the Gemini apps. Each company publishes which crawler it uses and what blocking it does, and the details differ in ways that matter. This guide puts the four side by side, from each company's own documentation read on 17 November 2026, then covers what makes content get cited by all of them. Jump to the rules side by side, the Google nuance or what gets cited.
- Engines, four sets of crawler rules
- 4
- Search bots to allow: OAI-SearchBot, Claude-SearchBot, PerplexityBot
- 3
- How long OpenAI and Perplexity say robots.txt changes can take
- 24 h
- Extra requirements for Google's AI features beyond Search
- 0
The rules, side by side
| Engine | Bot to allow for answers | Training bot (separate) | What the company says |
|---|---|---|---|
| ChatGPT | OAI-SearchBot | GPTBot | Allow OAI-SearchBot and OpenAI's published IPs; "placement is not guaranteed" |
| Claude | Claude-SearchBot, Claude-User | ClaudeBot | Disabling search or user bots "may reduce your site's visibility" |
| Perplexity | PerplexityBot (and Perplexity-User) | Not used for training | Allow it "to ensure your site appears in search results"; up to 24 hours to apply |
| Google AI Overviews and AI Mode | Googlebot | — | Indexed and eligible for a snippet; "no additional requirements" |
| Gemini apps | Googlebot + Google-Extended | Google-Extended | Google-Extended covers Gemini training and grounding in Gemini apps; not Search |
Table: AI search crawler rules by engine (each company's documentation, checked 17 Nov 2026)

The Google nuance: AI Overviews vs Gemini apps

Google has two different AI surfaces, with different controls:
- AI Overviews and AI Mode in Google Search use the Search index. Google says there are no additional requirements: be indexed and eligible to show with a snippet.
- The Gemini apps can use your content for grounding (providing Search-index content to the model at prompt time). Google says Google-Extended lets publishers manage whether content may be used for training future Gemini models and for grounding in the Gemini apps and Grounding with Google Search on Vertex AI. It "does not impact a site's inclusion in Google Search nor is it used as a ranking signal".
So blocking Google-Extended to opt out of training also opts you out of grounding in the Gemini apps. Google also says it intends to support the AIPREF shared training-preference signal in early 2027; check its documentation before changing robots.txt.
The robots.txt that works for most startups
If you want to be cited by all four engines (and you're comfortable with your public pages being used as their documentation describes), make sure none of these is disallowed:

- Googlebot — Google Search, AI Overviews and AI Mode
- OAI-SearchBot — ChatGPT search
- Claude-SearchBot and Claude-User — Claude's search and user-requested fetches
- PerplexityBot and Perplexity-User — Perplexity
- Google-Extended — only if you're happy with Gemini training and grounding; blocking it doesn't affect Search
- Your CDN or firewall — not challenging these bots by default
If you want to opt out of training, block GPTBot and ClaudeBot (and Google-Extended, accepting the grounding trade-off) while keeping the search bots allowed.
What gets cited by all four

- Answer first. One or two sentences that answer the question under each heading.
- Add numbers and sources. The GEO study (KDD 2024) found statistics, quotations and citations increased visibility in generative engines; keyword stuffing did poorly.
- One page per question. "What does X cost?", "X vs Y", "Best tool for [job]".
- Be corroborated. The same name, one-liner and facts on launch sites, directories, Crunchbase and a permanent profile.
- Keep it current. Dated facts, updated when they change.

How to track AI visibility
| Signal | Where |
|---|---|
| Referrals | chatgpt.com, perplexity.ai, claude.ai, gemini.google.com in analytics |
| Google AI traffic | Search Console Performance report, Web search type |
| Citations | The same ten buyer questions asked monthly in each engine |
| Accuracy | What each engine says about you; fix the sources if it's wrong |
Table: How to track visibility across AI search engines
Mistakes

- Blocking every AI bot — you also leave their answers
- Blocking Google-Extended without knowing the trade-off — it covers Gemini grounding too
- A bot firewall on by default — it can silently block search bots
- Vague marketing copy — nothing for engines to quote
- Inconsistent profiles — engines can't connect the facts
Ready to be cited?
How we know
Every crawler rule comes from the company's own documentation, read on 17 November 2026: OpenAI's crawler overview and ChatGPT search help article, Anthropic's help article on its bots, Perplexity's crawler page, and Google's AI features guide and common crawlers page. Content findings are from Aggarwal et al. (KDD 2024). No engine publishes its ranking formula. Science of Apps runs the Launchpad mentioned above.
Key terms, defined
- AI search engine: a system that answers questions with AI and cites web sources, such as ChatGPT search or Perplexity.
- Search bot vs training bot: separate crawlers for powering answers and for collecting training data.
- Grounding: giving a model fresh web content at the moment it answers, to improve accuracy.
- Google-Extended: Google's robots.txt token for Gemini training and grounding in Gemini apps.
- Citation: a link or attribution to your page inside an AI answer.
Frequently asked questions
How do I rank in ChatGPT?
Allow OAI-SearchBot in robots.txt and make sure your CDN accepts OpenAI's published search-bot IPs; OpenAI says that makes a site eligible for ChatGPT search, though placement isn't guaranteed. Then publish clear answer pages and keep consistent profiles on trusted sites.
How do I rank in Perplexity?
Allow PerplexityBot, which Perplexity says surfaces and links websites in its search results and isn't used for training foundation models. Perplexity says robots.txt changes can take up to 24 hours to apply. Then publish specific, well-sourced pages.
How do I appear in Claude's answers?
Allow Claude-SearchBot, which Anthropic says it uses to improve search result quality, and Claude-User, which fetches pages when users ask. Anthropic says disabling either may reduce your visibility. ClaudeBot is the separate training crawler.
How do I rank in Gemini and Google's AI Overviews?
For AI Overviews and AI Mode, Google says a page must be indexed and eligible to show with a snippet; there are no extra requirements. For the Gemini apps, note that Google-Extended controls whether your content is used for Gemini training and for grounding in Gemini apps, though it doesn't affect Google Search.
Can I appear in AI search but opt out of training?
With OpenAI, Anthropic and Perplexity, yes: allow their search bots and block their training bots (GPTBot, ClaudeBot). With Google, blocking Google-Extended opts out of Gemini training but also of grounding in the Gemini apps; it doesn't affect Google Search or AI Overviews.
What content do AI engines cite?
Pages that answer a specific question directly, with numbers, named sources and clear structure. Research on generative engines (the GEO study, KDD 2024) found adding statistics, quotations and citations increased visibility, while keyword stuffing did poorly.
How do I track AI search visibility?
Watch referrals from chatgpt.com, perplexity.ai, claude.ai and gemini.google.com in analytics; Google includes AI Overviews and AI Mode in Search Console's Web search type. Ask each engine the same set of buyer questions monthly and record whether you're cited.
Sources
- OpenAI — Overview of OpenAI crawlers (checked 17 Nov 2026)
- OpenAI Help Center — Searching the web with ChatGPT (checked 17 Nov 2026)
- Anthropic (Claude Help Center) — Does Anthropic crawl data from the web, and how can site owners block the crawler?
- Perplexity — Perplexity crawlers (checked 17 Nov 2026)
- Google Search Central — AI features and your website (checked 17 Nov 2026)
- Google — Google's common crawlers: Google-Extended (checked 17 Nov 2026)
- Aggarwal et al. — GEO: Generative Engine Optimization (arXiv 2311.09735, KDD 2024)
- Science of Apps — Launchpad: launch your product free
- Checked November 17, 2026. Figures that change — fees, salaries, prices — are reviewed every six months; if something is out of date, tell us.
Where to go from here
- How to get your brand mentioned in ChatGPT — the ChatGPT deep dive
- GEO for startups — what the research shows
- Organic visibility for startups — the five-step foundation
- Science of Apps - LaunchpadThe AI search optimization checklist: crawler access, content, corroboration and measurement for Google AI Overviews, ChatGPT, Claude and PerplexityA 2027 AI search optimization checklist: crawler access, answer-first content, corroboration and measurement for AI Overviews, ChatGPT, Claude and Perplexity.
- Science of Apps - LaunchpadLLM SEO vs traditional SEO side by side: what stays the same, what changes, which crawlers matter and how to measure each, using Google's and AI companies' own docsLLM SEO vs traditional SEO, side by side: the foundations that stay the same, what changes (crawlers, answers, citations, measurement) and what to skip.
- Career guidesGenerative engine optimization skills employers want in 2027: getting brands mentioned and cited in ChatGPT, Gemini, Perplexity and AI OverviewsGEO skills employers want in 2027: how AI assistants choose sources, quotable facts, authority and mentions, crawler access and tracking AI citations.
- Science of Apps - LaunchpadPerplexity SEO: how to show up in Perplexity answers, from allowing PerplexityBot and fixing your firewall to writing pages it citesHow to show up in Perplexity: allow PerplexityBot and its IP ranges, fix firewall rules, understand Perplexity-User, and write pages Perplexity cites.
- Science of Apps - LaunchpadHow ChatGPT, Google's AI features, Claude and Perplexity choose which products to recommend, from what they publish, and what that means for your startupHow ChatGPT, Google AI Overviews, Claude and Perplexity recommend products: search, fan-out, sources and corroboration, and how a startup gets included.
- Science of Apps - LaunchpadGenerative engine optimization for startups: what the research shows, and a plan to get cited by AI answersGEO for startups in 2027: what the original GEO research found, how to get cited by ChatGPT, Perplexity, Claude and Google AI, and a 30-day plan.
Get the ideas, not the noise.
Sign up with Google and we’ll send you short UX psychology tips by email — why people tap, stall and leave, and what to do about it. Unsubscribe anytime. Prefer something to keep? Get the free Laws of UX guide as a PDF.
Launch day ends. Your listing shouldn't.
A permanent, crawlable profile that states what you do in quotable words, the kind of independent page AI answers cite, plus your product tested end to end, including how findable you are on AI assistants.





