Quick answer: How do AI engines decide which products to recommend?
- Search at answer time: ChatGPT search, Claude, Perplexity and Google's AI features fetch web pages.
- Bots decide reach: OAI-SearchBot, Claude-SearchBot, PerplexityBot, Googlebot.
- Fan-out: Google runs related searches across subtopics for one answer.
- Sources: answers link the pages they used.
- Agreement: consistent facts across independent sites are easier to state confidently.
- Formula: none of them publishes one; placement isn't guaranteed.
Key takeaways
- An engine can't recommend what its search bot can't reach.
- Specific buyer questions get specific answers; generic pages rarely make the list.
- Independent sources matter: your own site is one voice.
- OpenAI says placement in ChatGPT search isn't guaranteed even when you allow its bot.
- Inauthentic mentions don't help much, says Google; real coverage does.
"What's the best tool for…?" is now a question people ask an AI assistant before they ask Google. The answer is a short list with links, and founders want to know how products get onto it. None of the engines publishes a formula, but each documents enough about how it searches and reads the web to explain the mechanics. This explainer puts those pieces together from OpenAI's, Google's, Anthropic's and Perplexity's own documentation, read on 17 November 2026. Jump to the pipeline, what makes a product recommendable or how to check.
The pipeline: from question to recommendation

- The question — "best invoicing app for freelancers in India".
- Search — the engine searches the web; Google may fan out into related sub-searches.
- Fetch — its search bot reads the pages it can reach.
- Summarise — it states what the sources say, often a short list.
- Link — it cites the pages it used.
| Engine | Bot that reads your pages | What the company says |
|---|---|---|
| ChatGPT search | OAI-SearchBot | Allow it to appear; "placement is not guaranteed" |
| Google AI Overviews and AI Mode | Googlebot | Indexed and snippet-eligible; may use query fan-out |
| Claude | Claude-SearchBot, Claude-User | Blocking them may reduce visibility |
| Perplexity | PerplexityBot, Perplexity-User | Allow it to appear in results; not used for training |
Table: How AI engines reach your pages (each company's documentation, checked 17 Nov 2026)
What makes a product recommendable

- Reachable — the engine's search bot can fetch your pages
- Specific — a page answering the exact buyer question, not a generic homepage
- Factual — prices, features, who it's for, dated
- Corroborated — independent sites describe you the same way
- Genuinely mentioned — Google says inauthentic mentions aren't as helpful as they seem
- Current — stale facts get dropped or repeated wrongly
The GEO study (KDD 2024) found that adding statistics, quotations and citations raised a source's visibility in generative engines, while keyword stuffing did poorly.
Why your competitor gets recommended
| Likely reason | Fix |
|---|---|
| Your pages block or challenge the bot | Allow the search bots; check the firewall |
| No page answers that exact question | Write one, answer first |
| Fewer independent sources describe you | Directories, profiles, genuine coverage |
| Inconsistent facts about you | One name, one-liner and price everywhere |
| Your facts are out of date | Update and date them |
Table: Common reasons a competitor is recommended instead (editorial)
How to check

- Ten buyer questions — asked in each engine, monthly
- Mentioned or not — and in what words
- Which sources are cited — those are the pages to be on
- Referrals — chatgpt.com, perplexity.ai, claude.ai, gemini.google.com in analytics
- Google — Search Console's Generative AI performance report

Ready to be recommendable?
How we know
Mechanics come from OpenAI's crawler and ChatGPT search pages, Google's AI features documentation and generative AI optimisation guide, Anthropic's crawler help article and Perplexity's crawler documentation, all read on 17 November 2026, plus the GEO paper (KDD 2024). The "why your competitor" table is our editorial reading; none of the engines publishes its selection formula.
Key terms, defined
- Search bot: a crawler fetching pages for answers.
- Query fan-out: running several related searches for one answer.
- Citation: a link to a source used in an answer.
- Corroboration: independent sources agreeing.
- Generative engine: an AI system that writes answers from sources.
Frequently asked questions
How does ChatGPT recommend products?
When ChatGPT searches the web, it reads pages its search crawler (OAI-SearchBot) can reach and summarises them with links. OpenAI says allowing OAI-SearchBot lets your pages appear in search results, but placement is not guaranteed.
How do Google AI Overviews choose which products to mention?
Google says AI Overviews and AI Mode may use query fan-out, running related searches across subtopics and sources, and show supporting links from pages that are indexed and eligible for a snippet. It doesn't publish how links are chosen.
Can I pay to be recommended by AI engines?
Not in their organic answers; none of these companies describes a paid route into answers. Be wary of services promising placement.
Why does an AI engine recommend my competitor and not me?
Usually because more independent pages describe the competitor clearly for that question, or because the engine's bot can't reach your pages. Check access first, then publish specific answers and earn consistent coverage.
Do reviews and directories affect AI recommendations?
They're among the independent sources engines can read. Google's guidance notes its AI features can show what's being said about products across the web, while inauthentic mentions aren't as helpful as they seem.
How do I check if AI engines recommend my product?
Ask each engine the ten questions your buyers ask, monthly, and note whether you're mentioned and what's said. Track referrals from chatgpt.com, perplexity.ai, claude.ai and gemini.google.com in analytics.
Sources
- OpenAI — Overview of OpenAI crawlers (checked 17 Nov 2026)
- OpenAI Help Center — Searching the web with ChatGPT (checked 17 Nov 2026)
- Google Search Central — AI features and your website (checked 17 Nov 2026)
- Google Search Central — Optimizing your website for generative AI features on Google Search (updated 10 Jul 2026)
- Anthropic (Claude Help Center) — Does Anthropic crawl data from the web, and how can site owners block the crawler?
- Perplexity — Perplexity crawlers (checked 17 Nov 2026)
- Aggarwal et al. — GEO: Generative Engine Optimization (arXiv 2311.09735, KDD 2024)
- Checked November 17, 2026. Figures that change — fees, salaries, prices — are reviewed every six months; if something is out of date, tell us.
Where to go from here
- Get recommended by ChatGPT — the practical steps
- Rank in AI search engines — each engine's rules
- AI search optimization checklist — everything to do
- Science of Apps - LaunchpadThe AI search optimization checklist: crawler access, content, corroboration and measurement for Google AI Overviews, ChatGPT, Claude and PerplexityA 2027 AI search optimization checklist: crawler access, answer-first content, corroboration and measurement for AI Overviews, ChatGPT, Claude and Perplexity.
- Science of Apps - LaunchpadHow to get your startup mentioned by Gemini, Claude and Grok: what each company documents about crawling and search, and the levers that work for all threeHow to get your startup mentioned by Gemini, Claude and Grok: what Google, Anthropic and xAI document about crawling, and the levers that work for all three.
- Science of Apps - LaunchpadHow to rank in ChatGPT, Gemini, Claude and Perplexity: each engine's own rules, side by side, and what works across all of themHow to rank in ChatGPT, Gemini, Claude and Perplexity in 2027: each engine's crawler rules side by side, and the content that gets cited by all four.
- Career guidesGenerative engine optimization skills employers want in 2027: getting brands mentioned and cited in ChatGPT, Gemini, Perplexity and AI OverviewsGEO skills employers want in 2027: how AI assistants choose sources, quotable facts, authority and mentions, crawler access and tracking AI citations.
- Science of Apps - LaunchpadGenerative engine optimization for startups: what the research shows, and a plan to get cited by AI answersGEO for startups in 2027: what the original GEO research found, how to get cited by ChatGPT, Perplexity, Claude and Google AI, and a 30-day plan.
- Science of Apps - LaunchpadHow to show up in Google AI Overviews and AI Mode: Google's actual requirements, query fan-out, the controls, and how to measure itHow to show up in Google AI Overviews and AI Mode: be indexed with a snippet, no extra requirements, Google's best practices, controls and how to track it.
Get the ideas, not the noise.
Sign up with Google and we’ll send you short UX psychology tips by email — why people tap, stall and leave, and what to do about it. Unsubscribe anytime. Prefer something to keep? Get the free Laws of UX guide as a PDF.
Launch day ends. Your listing shouldn't.
Be one of the sources engines read: a free, crawlable Launchpad profile with consistent facts, ranked by relevance and never by payment, plus a product test that checks your findability.





