Quick answer: Does llms.txt help SEO or AI search visibility?
- What it is: a Markdown file at /llms.txt summarising your site and linking key pages.
- Who proposed it: Jeremy Howard, published 3 September 2024 (v2 updated 2026).
- Format: an H1 with the site name is the only required part; a summary and link lists are optional.
- Google Search: doesn't use it, including AI Overviews and AI Mode.
- Where it can help: agents and coding tools reading your documentation.
- What helps more: crawlable pages with clear, specific answers.
Key takeaways
- llms.txt is a proposal, not a search-engine standard.
- Google lists llms.txt among things you don't need for its generative AI features.
- It isn't a crawler control: robots.txt still decides what bots may fetch.
- For a developer product with docs, a short llms.txt is cheap and can help agents.
- Don't spend time on it before your pages are crawlable and clear.
Every few months a new file promises to make AI engines notice you, and llms.txt is the one founders ask about most. It's a sensible idea with a narrow job, and the evidence on what it does for search is now unusually clear: Google has said in its own documentation that Search doesn't use it. This explainer covers what the file is, what Google says, where it can still help, and a short template, from the proposal itself and Google's guide, read on 17 November 2026. Jump to what it is, what Google says or when to add one.

What llms.txt is
| llms.txt | |
|---|---|
| Proposed by | Jeremy Howard, 3 September 2024 (v2 modified 10 August 2026) |
| Purpose | Help agents use a website: a brief summary plus links to key content |
| Location | /llms.txt at the root, or a subpath such as /docs/llms.txt |
| Required | An H1 with the site or project name |
| Optional | A blockquote summary, notes, H2 sections of link lists |
| Companion idea | Clean Markdown versions of pages at the same URL plus .md |
Table: The llms.txt proposal (llmstxt.org, checked 17 Nov 2026)
What Google says

Google's guide to optimising for its generative AI features (updated 10 July 2026) is direct: you don't need "new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them". It lists llms.txt among the "AEO/GEO hacks" you can ignore, alongside chunking content and chasing inauthentic mentions.
When llms.txt is still worth adding
| Situation | Worth it? |
|---|---|
| Developer docs that coding agents read | Yes: cheap, can help agents find the right pages |
| An API or SDK with many pages | Yes, with Markdown versions of key pages |
| A marketing site wanting Google visibility | No: Google Search doesn't use it |
| Before your pages are crawlable and clear | No: fix the pages first |
Table: When an llms.txt is worth it (editorial guidance)
A short template

- H1: your product name.
- Blockquote: one or two sentences on what it does and for whom.
- H2 "Docs": links to quick start, API reference, pricing, each with a short note.
- H2 "Optional": links agents can skip if short on space.
- Keep it current — an outdated map is worse than none.
What actually helps AI visibility

- Crawlable pages — allow Googlebot, OAI-SearchBot, Claude-SearchBot, PerplexityBot
- Clear answers — one question per section, answered first
- Specific facts — numbers, dates, sources
- Consistent profiles elsewhere — directories, Crunchbase, launch sites
- Structured data that matches the page — not special AI schema
The full list is in our AI search optimization checklist.

Ready to be found?
How we know
The format and history come from the llms.txt proposal (llmstxt.org); Google's position from its generative AI optimisation guide (updated 10 July 2026) and AI features documentation; crawler rules from OpenAI's and Perplexity's documentation. All read on 17 November 2026.
Key terms, defined
- llms.txt: a proposed Markdown summary file for AI agents.
- robots.txt: a file telling crawlers what they may fetch.
- Agent: software that uses websites on someone's behalf.
- Chunking: splitting content into small pieces, which Google says isn't needed.
- Markdown: a plain-text formatting syntax.
Frequently asked questions
What is llms.txt?
A proposal, published by Jeremy Howard in September 2024, to put a Markdown file at /llms.txt that gives AI agents a brief summary of a site and links to its most useful pages, ideally with clean Markdown versions of those pages.
Does Google use llms.txt?
Not for Search. Google's guide to optimising for generative AI features says you don't need machine-readable files, AI text files or Markdown to appear in Google Search, including its AI features, because Google Search doesn't use them.
Does llms.txt help with ChatGPT, Claude or Perplexity?
Their crawler documentation centres on robots.txt and their bots, not llms.txt. Being crawlable by OAI-SearchBot, Claude-SearchBot and PerplexityBot, with clear answers on your pages, is what their published guidance supports.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they may fetch. llms.txt is a summary and map of your content for agents; it doesn't grant or block access.
Should my startup add an llms.txt?
If you have developer documentation that agents and coding tools read, a short, maintained llms.txt is cheap and may help them. For search visibility, spend the time on crawlable, specific pages first.
What goes in an llms.txt file?
Per the proposal: an H1 with your site or project name (the only required part), an optional one-paragraph summary in a blockquote, optional notes, and H2 sections listing links to key pages with a short description of each.
Sources
- Jeremy Howard — The /llms.txt file, v2 (llmstxt.org; published 3 Sep 2024, modified 10 Aug 2026)
- Google Search Central — Optimizing your website for generative AI features on Google Search (updated 10 Jul 2026; checked 17 Nov 2026)
- Google Search Central — AI features and your website (checked 17 Nov 2026)
- Perplexity — Perplexity crawlers (checked 17 Nov 2026)
- OpenAI — Overview of OpenAI crawlers (checked 17 Nov 2026)
- Checked November 17, 2026. Figures that change — fees, salaries, prices — are reviewed every six months; if something is out of date, tell us.
Where to go from here
- Rank in AI search engines — what each engine actually uses
- How to show up in AI Overviews — Google's requirements
- AI search optimization checklist — what to do instead
- Science of Apps - LaunchpadPerplexity SEO: how to show up in Perplexity answers, from allowing PerplexityBot and fixing your firewall to writing pages it citesHow to show up in Perplexity: allow PerplexityBot and its IP ranges, fix firewall rules, understand Perplexity-User, and write pages Perplexity cites.
- Science of Apps - LaunchpadThe AI search optimization checklist: crawler access, content, corroboration and measurement for Google AI Overviews, ChatGPT, Claude and PerplexityA 2027 AI search optimization checklist: crawler access, answer-first content, corroboration and measurement for AI Overviews, ChatGPT, Claude and Perplexity.
- Science of Apps - LaunchpadHow to write a startup one-liner: the formula, the test Product Hunt uses, before-and-after examples, and why the same line everywhere helps search and AI enginesHow to write a startup one-liner: a simple formula, Product Hunt's tagline test, before-and-after examples, and why one consistent line helps AI search.
- Science of Apps - LaunchpadHow ChatGPT, Google's AI features, Claude and Perplexity choose which products to recommend, from what they publish, and what that means for your startupHow ChatGPT, Google AI Overviews, Claude and Perplexity recommend products: search, fan-out, sources and corroboration, and how a startup gets included.
- Science of Apps - LaunchpadChatGPT SEO for startups: how to get your product recommended and cited by ChatGPT, from OpenAI's own rulesChatGPT SEO for startups in 2027: what OpenAI says makes a site eligible for ChatGPT search, and how to get your product recommended and cited.
- Science of Apps - LaunchpadHow to show up in Google AI Overviews and AI Mode: Google's actual requirements, query fan-out, the controls, and how to measure itHow to show up in Google AI Overviews and AI Mode: be indexed with a snippet, no extra requirements, Google's best practices, controls and how to track it.
Get the ideas, not the noise.
Sign up with Google and we’ll send you short UX psychology tips by email — why people tap, stall and leave, and what to do about it. Unsubscribe anytime. Prefer something to keep? Get the free Laws of UX guide as a PDF.
Launch day ends. Your listing shouldn't.
What helps more than a text file: a free, crawlable Launchpad profile describing what you do in plain words, plus a product test that checks how findable you are on Google and AI assistants.





