Science of Apps - Launchpad · AI search guide

How to rank in ChatGPT, Gemini,
Claude and Perplexity

To rank in AI search, first meet each engine's own rule: allow OAI-SearchBot for ChatGPT, Claude-SearchBot for Claude and PerplexityBot for Perplexity; for Google's AI Overviews and AI Mode, be indexed in Search, and don't block Google-Extended if you want to be used for grounding in the Gemini apps. Then publish specific, sourced answer pages and keep consistent profiles across trusted sites.

By , FounderUpdated 7 min read

Share

Quick answer: How do I rank in ChatGPT, Gemini, Claude and Perplexity?

  1. ChatGPT: allow OAI-SearchBot and OpenAI's search IP ranges; placement isn't guaranteed.
  2. Claude: allow Claude-SearchBot (search) and Claude-User (user-requested fetches).
  3. Perplexity: allow PerplexityBot; changes can take up to 24 hours.
  4. Gemini and Google AI: be indexed with a snippet; Google-Extended controls Gemini training and grounding in Gemini apps.
  5. Everywhere: answer-first pages, numbers and sources, consistent profiles, honest comparisons.

Key takeaways

  • Each AI engine publishes its own crawler rules; blocking the wrong bot removes you from its answers.
  • Search bots and training bots are separate for OpenAI, Anthropic and Perplexity; Google's Google-Extended also covers grounding in the Gemini apps.
  • Google says its AI Overviews and AI Mode have no extra requirements beyond Search eligibility.
  • Content that gets cited is specific: direct answers, statistics, quotations and sources.
  • No engine guarantees placement; measure with referrals and repeated prompt checks.

"Rank in AI search" sounds like one job, but it's four sets of rules: OpenAI's for ChatGPT, Anthropic's for Claude, Perplexity's, and Google's for AI Overviews, AI Mode and the Gemini apps. Each company publishes which crawler it uses and what blocking it does, and the details differ in ways that matter. This guide puts the four side by side, from each company's own documentation read on 17 November 2026, then covers what makes content get cited by all of them. Jump to the rules side by side, the Google nuance or what gets cited.

Engines, four sets of crawler rules
4
Search bots to allow: OAI-SearchBot, Claude-SearchBot, PerplexityBot
3
How long OpenAI and Perplexity say robots.txt changes can take
24 h
Extra requirements for Google's AI features beyond Search
0

The rules, side by side

EngineBot to allow for answersTraining bot (separate)What the company says
ChatGPTOAI-SearchBotGPTBotAllow OAI-SearchBot and OpenAI's published IPs; "placement is not guaranteed"
ClaudeClaude-SearchBot, Claude-UserClaudeBotDisabling search or user bots "may reduce your site's visibility"
PerplexityPerplexityBot (and Perplexity-User)Not used for trainingAllow it "to ensure your site appears in search results"; up to 24 hours to apply
Google AI Overviews and AI ModeGooglebot—Indexed and eligible for a snippet; "no additional requirements"
Gemini appsGooglebot + Google-ExtendedGoogle-ExtendedGoogle-Extended covers Gemini training and grounding in Gemini apps; not Search

Table: AI search crawler rules by engine (each company's documentation, checked 17 Nov 2026)

Claude Help Center article explaining Anthropic's three bots, ClaudeBot, Claude-User and Claude-SearchBot, and what happens when each is disabled
Anthropic's help article on its three bots and what blocking each one does.Image: Anthropic · Screenshot shown for commentary, 17 Nov 2026 · source

The Google nuance: AI Overviews vs Gemini apps

Google crawling documentation page Google's common crawlers, describing user agents including Google-Extended
Google's crawler documentation: Google-Extended controls Gemini training and grounding, not Search.Image: Google · Screenshot shown for commentary, 17 Nov 2026 · source

Google has two different AI surfaces, with different controls:

  • AI Overviews and AI Mode in Google Search use the Search index. Google says there are no additional requirements: be indexed and eligible to show with a snippet.
  • The Gemini apps can use your content for grounding (providing Search-index content to the model at prompt time). Google says Google-Extended lets publishers manage whether content may be used for training future Gemini models and for grounding in the Gemini apps and Grounding with Google Search on Vertex AI. It "does not impact a site's inclusion in Google Search nor is it used as a ranking signal".

So blocking Google-Extended to opt out of training also opts you out of grounding in the Gemini apps. Google also says it intends to support the AIPREF shared training-preference signal in early 2027; check its documentation before changing robots.txt.

The robots.txt that works for most startups

If you want to be cited by all four engines (and you're comfortable with your public pages being used as their documentation describes), make sure none of these is disallowed:

Checklist: The robots.txt that works for most startups — Googlebot; OAI-SearchBot; Claude-SearchBot and Claude-User; PerplexityBot and Perplexity-User; Google-Extended; Your CDN or firewall.
The robots.txt that works for most startups
  • Googlebot — Google Search, AI Overviews and AI Mode
  • OAI-SearchBot — ChatGPT search
  • Claude-SearchBot and Claude-User — Claude's search and user-requested fetches
  • PerplexityBot and Perplexity-User — Perplexity
  • Google-Extended — only if you're happy with Gemini training and grounding; blocking it doesn't affect Search
  • Your CDN or firewall — not challenging these bots by default

If you want to opt out of training, block GPTBot and ClaudeBot (and Google-Extended, accepting the grounding trade-off) while keeping the search bots allowed.

What gets cited by all four

Diagram of 5 steps for what gets cited by all four: 1. Answer first; 2. Add numbers and sources; 3. One page per question; 4. Be corroborated; 5. Keep it current.
What gets cited by all four
  1. Answer first. One or two sentences that answer the question under each heading.
  2. Add numbers and sources. The GEO study (KDD 2024) found statistics, quotations and citations increased visibility in generative engines; keyword stuffing did poorly.
  3. One page per question. "What does X cost?", "X vs Y", "Best tool for [job]".
  4. Be corroborated. The same name, one-liner and facts on launch sites, directories, Crunchbase and a permanent profile.
  5. Keep it current. Dated facts, updated when they change.
Science of Apps Launchpad homepage with the headline Launch your product. Get it found.
The Science of Apps Launchpad: permanent, crawlable product profiles.Image: Science of Apps · © Science of Apps · source

How to track AI visibility

SignalWhere
Referralschatgpt.com, perplexity.ai, claude.ai, gemini.google.com in analytics
Google AI trafficSearch Console Performance report, Web search type
CitationsThe same ten buyer questions asked monthly in each engine
AccuracyWhat each engine says about you; fix the sources if it's wrong

Table: How to track visibility across AI search engines

Mistakes

Checklist: Mistakes — Blocking every AI bot; Blocking Google-Extended without knowing the trade-off; A bot firewall on by default; Vague marketing copy; Inconsistent profiles.
Mistakes
  • Blocking every AI bot — you also leave their answers
  • Blocking Google-Extended without knowing the trade-off — it covers Gemini grounding too
  • A bot firewall on by default — it can silently block search bots
  • Vague marketing copy — nothing for engines to quote
  • Inconsistent profiles — engines can't connect the facts

Ready to be cited?

How we know

Every crawler rule comes from the company's own documentation, read on 17 November 2026: OpenAI's crawler overview and ChatGPT search help article, Anthropic's help article on its bots, Perplexity's crawler page, and Google's AI features guide and common crawlers page. Content findings are from Aggarwal et al. (KDD 2024). No engine publishes its ranking formula. Science of Apps runs the Launchpad mentioned above.

Key terms, defined

  • AI search engine: a system that answers questions with AI and cites web sources, such as ChatGPT search or Perplexity.
  • Search bot vs training bot: separate crawlers for powering answers and for collecting training data.
  • Grounding: giving a model fresh web content at the moment it answers, to improve accuracy.
  • Google-Extended: Google's robots.txt token for Gemini training and grounding in Gemini apps.
  • Citation: a link or attribution to your page inside an AI answer.

Frequently asked questions

How do I rank in ChatGPT?

Allow OAI-SearchBot in robots.txt and make sure your CDN accepts OpenAI's published search-bot IPs; OpenAI says that makes a site eligible for ChatGPT search, though placement isn't guaranteed. Then publish clear answer pages and keep consistent profiles on trusted sites.

How do I rank in Perplexity?

Allow PerplexityBot, which Perplexity says surfaces and links websites in its search results and isn't used for training foundation models. Perplexity says robots.txt changes can take up to 24 hours to apply. Then publish specific, well-sourced pages.

How do I appear in Claude's answers?

Allow Claude-SearchBot, which Anthropic says it uses to improve search result quality, and Claude-User, which fetches pages when users ask. Anthropic says disabling either may reduce your visibility. ClaudeBot is the separate training crawler.

How do I rank in Gemini and Google's AI Overviews?

For AI Overviews and AI Mode, Google says a page must be indexed and eligible to show with a snippet; there are no extra requirements. For the Gemini apps, note that Google-Extended controls whether your content is used for Gemini training and for grounding in Gemini apps, though it doesn't affect Google Search.

Can I appear in AI search but opt out of training?

With OpenAI, Anthropic and Perplexity, yes: allow their search bots and block their training bots (GPTBot, ClaudeBot). With Google, blocking Google-Extended opts out of Gemini training but also of grounding in the Gemini apps; it doesn't affect Google Search or AI Overviews.

What content do AI engines cite?

Pages that answer a specific question directly, with numbers, named sources and clear structure. Research on generative engines (the GEO study, KDD 2024) found adding statistics, quotations and citations increased visibility, while keyword stuffing did poorly.

How do I track AI search visibility?

Watch referrals from chatgpt.com, perplexity.ai, claude.ai and gemini.google.com in analytics; Google includes AI Overviews and AI Mode in Search Console's Web search type. Ask each engine the same set of buyer questions monthly and record whether you're cited.

Sources

  1. OpenAI — Overview of OpenAI crawlers (checked 17 Nov 2026)
  2. OpenAI Help Center — Searching the web with ChatGPT (checked 17 Nov 2026)
  3. Anthropic (Claude Help Center) — Does Anthropic crawl data from the web, and how can site owners block the crawler?
  4. Perplexity — Perplexity crawlers (checked 17 Nov 2026)
  5. Google Search Central — AI features and your website (checked 17 Nov 2026)
  6. Google — Google's common crawlers: Google-Extended (checked 17 Nov 2026)
  7. Aggarwal et al. — GEO: Generative Engine Optimization (arXiv 2311.09735, KDD 2024)
  8. Science of Apps — Launchpad: launch your product free
  9. Checked November 17, 2026. Figures that change — fees, salaries, prices — are reviewed every six months; if something is out of date, tell us.
Pass it on

Know someone who'd enjoy this?

Send it to them — one tap, no sign-up. Good ideas travel best between friends.

Keep exploring

Where to go from here

Join free

Get the ideas, not the noise.

Sign up with Google and we’ll send you short UX psychology tips by email — why people tap, stall and leave, and what to do about it. Unsubscribe anytime. Prefer something to keep? Get the free Laws of UX guide as a PDF.

Get the free Laws of UX PDFFree · no spam · unsubscribe anytime
Free, forever

Launch day ends. Your listing shouldn't.

A permanent, crawlable profile that states what you do in quotable words, the kind of independent page AI answers cite, plus your product tested end to end, including how findable you are on AI assistants.

Free PDF · 74 pages · 2 editions

The Laws of UX, in one field guide

60 research-backed principles in 6 categories — what each one means, why it works on the human brain and how to use it on your own screens. Written by Sushant Singh, founder of Science of Apps, who has scaled consumer apps to lacs of users and spent 10+ years leading product across startups and digital businesses in the US, EU & India.

Pick your edition
Free PDF · 90 pages · 2 editions

100+ Subscription Paywalls, decoded

110 real paywalls from top apps, taken apart, rated out of 5 and decoded with the Laws of UX — plus the frameworks, copy and checklists to ship your own. By Sushant Singh, founder of Science of Apps.

  • 110 paywalls rated
  • 34 Laws of UX
  • 12 frameworks

Which paywall scored 4.8 out of 5, and which scored 2.6? It’s all inside.

Pick your edition
Free PDF · 188 pages · 2 editions

Unmistakable: the logo book

How the world’s greatest logos are made, chosen and kept — and how to make yours. 98 short lessons and 19 true stories for founders, designers and the curious. By Sushant Singh, founder of Science of Apps.

  • 98 lessons
  • 19 true stories
  • 60+ real logos

Why did a $35 sketch become the Swoosh, and why did Gap lose its nerve in six days? It’s all inside.

Pick your edition