Skip to content

llms.txt in 2026: What It Is, an Example and What It Does

llms.txt is a Markdown file at the root of a website that gives AI agents a short summary of the site and links to the key pages. In “llms.txt in 2026: What It Is, an Example and What It Does”, Pavlo Zhydkykh explains who reads the file, what Google says and when to publish one.

Pavlo Zhydkykh

Updated · 6 min read

A blue dot in the corner of a pale dot grid, with thin lines running from it to four solid dark dots, a short map to the pages that matter

Short answer

llms.txt is the file founders ask me about most since AI search took off. Some agencies sell it as a ranking trick; Google now says its search ignores the file. I add one to every site I build, this one included, so here is what the file is, an example, and what the data says it does and does not do.

Key takeaways

  • llms.txt is a Markdown file at /llms.txt that gives AI agents a short summary of a site and links to the key pages.[2]
  • Google Search ignores llms.txt: keeping one will neither harm nor help visibility or rankings in Google Search.[3]
  • In an Ahrefs study of 137,210 domains, 97% of llms.txt files received no requests in May 2026.[4]
  • robots.txt, the file that tells crawlers what pages to visit, was formally standardised as RFC 9309 in 2022.[1]
  • Pavlo Zhydkykh generates llms.txt from the site content on every build, with SEO and GEO basics included.

What is llms.txt?

llms.txt is a proposed convention for a Markdown file at /llms.txt that summarises a website for large language models and AI agents and links to the most useful pages. Jeremy Howard published the proposal in September 2024.

Wikipedia: robots.txt

“I publish llms.txt on every build because the file is cheap and harmless, but the real AI visibility work is crawlable pages and clear answers.”

Pavlo Zhydkykh · Web designer and developer since 2016

Last checked · 6 min read

What llms.txt is

llms.txt is a plain text file written in Markdown that sits at /llms.txt on a website. It tells an AI agent, in a few hundred words, what the site is and which pages hold the details. Jeremy Howard of Answer.AI proposed the format on 3 September 2024, and the proposal is now in its second version.

The problem the file solves is real. An HTML page wraps its text in navigation, scripts and footers, and an agent with a limited context window wastes tokens on all of it. llms.txt hands the agent a short map instead: a summary, then curated links, ideally to clean Markdown copies of the pages.

The format is simple. Only one part is required:

  • An H1 with the name of the site or project. This is the only required section.
  • A blockquote with a short summary that the rest of the file relies on.
  • Optional paragraphs or lists with more detail, but no headings.
  • H2 sections with lists of links, each written as [name](url) plus an optional note after a colon.
  • An H2 called "Optional" for secondary links an agent can skip when it needs a shorter context.

The proposal also suggests a Markdown copy of each important page at the same URL with .md added, and two link relations so agents can find both: rel="alternate" type="text/markdown" for the Markdown copy and rel="describedby" for the llms.txt file. A file can also sit in a subfolder, so /docs/llms.txt covers only the docs.

An llms.txt example

Here is a cut down version of the llms.txt file on this site. The full file lists every service, case study and article, and the site builds it from the same data as the pages, so it never goes out of date.

# Pavlo Zhydkykh

> Pavlo Zhydkykh is a web developer who designs and builds fast websites for AI, fintech and B2B startups on Framer and Astro + Sanity.

## Pages
- [Services](https://pzhydkykh.com/services/): all 24 services in 5 groups, with prices and timelines.
- [Pricing](https://pzhydkykh.com/pricing/): fixed prices, what is included and what costs extra.

## AI visibility
- [GEO](https://pzhydkykh.com/services/geo/): pages that ChatGPT, Perplexity, Claude, Gemini and Grok can read, trust and quote.

## Prices and timelines (USD)
- Landing page: from $1,200, 1 to 2 weeks
- Migration: from $2,200, 2 to 4 weeks

Three things make it useful. The summary names who the site is for and what it sells in one sentence. Every link carries a note, so an agent can pick the right page without opening all of them. And the facts an AI assistant gets asked most, such as prices and timelines, are in the file itself.

llms.txt vs robots.txt and sitemap.xml

The name makes people think llms.txt works like robots.txt. It does not. robots.txt, proposed by Martijn Koster in 1994 and standardised as RFC 9309 in 2022, tells crawlers what they may visit. llms.txt grants nothing and blocks nothing; it only describes.

robots.txtsitemap.xmlllms.txt
JobTells crawlers what they may visitLists every indexable pageSummarises the site and points to key pages
StatusInternet standard, RFC 9309The Sitemaps protocol, sitemaps.orgA proposal, no standards body
Main readerSearch and AI crawlersSearch enginesAI agents, mostly coding agents today
FormatPlain text rulesXMLMarkdown
Controls accessYes, by voluntary complianceNoNo

If you want to keep AI crawlers out, or let them in, that rule still lives in robots.txt. llms.txt is a reading guide that sits on top of it. To test whether the crawlers can reach and read your pages at all, see Can ChatGPT read your website?

Who actually reads llms.txt in 2026

The best data so far is a server log study by Ahrefs, published on 15 June 2026. Ahrefs looked at 137,210 domains in its web analytics product for May 2026:

  • 28% of the domains published an llms.txt file.
  • 97% of those files received zero requests that month.
  • 96% of the requests that did arrive came from bots.
  • GPTBot sent the most requests among AI tools, and Claude Code, a coding agent, came second.
  • AI retrieval bots, the ones that fetch pages to answer a live question, made 1.1% of the requests.

That matches what the proposal itself says: llms.txt files are used most heavily for software documentation, where coding agents follow them to API references. The AI labs publish one for their own developer docs, and so do developer platforms. When I checked on 26 October 2026, Anthropic's docs, Stripe's docs, Cloudflare's developer docs, Vercel and Sanity all served one. Company websites are catching up: when I checked the Forbes Fintech 50 on 11 November 2026, 23 of 47 sites published one, as I show in can AI assistants read the Forbes Fintech 50 websites.

Google's Chrome team also looks for the file. Lighthouse has an llms.txt audit among its agentic browsing checks. A missing file is marked "not applicable", because providing one is optional for now; the audit flags a server error when the file is requested.

What Google says about llms.txt

Google Search is clear. Its guide to optimizing for generative AI features, published in May 2026, lists llms.txt among the things you can ignore for Google Search, including AI Overviews and AI Mode. On 15 June 2026 Google added a note: keeping the file is fine for other services that use it, and it will neither harm nor help rankings in Google Search, because Google Search ignores it.

In the same guide Google says optimizing for generative AI search is still SEO: crawlable pages, a clear technical structure and content that goes beyond common knowledge. So llms.txt is not a ranking lever for Google. It is a courtesy for agents and assistants that choose to read it.

llms.txt best practices

If you publish one, these are the rules I follow on every build:

  1. Generate the file from the same content as the pages, at build time, so a price or a service never differs between the file and the site.
  2. Open with one sentence that says who you are, who you serve and what you sell.
  3. Link only to pages that answer real buyer questions: services, pricing, case studies, docs, the best articles.
  4. Give every link a short note after the colon; the note is what an agent reads to decide.
  5. Put the facts people ask about into the file: prices, timelines, locations, supported platforms.
  6. Keep it short enough to fit in a context window. The detail belongs behind the links.
  7. Serve it with a 200 status and a text content type, and check it after every deploy.
  8. Leave robots.txt in charge of access. Never list a page in llms.txt that robots.txt blocks.

Should a SaaS site publish one?

For most AI, fintech and B2B SaaS companies the honest answer is yes, but with low expectations. In my experience the file takes an hour, costs nothing to serve and does no harm. Who benefits depends on the site:

  • Developer docs and APIs: publish one, with Markdown copies of the pages. Coding agents are the readers the data actually shows.
  • Marketing site: publish a short one generated from your content, and do not expect leads from the file itself.
  • A site that is hard to crawl: fix the site first. llms.txt cannot make up for pages whose text only appears after JavaScript runs.

What I would not do is pay for an "llms.txt optimization" package, or keep a hand edited file that drifts out of date. A stale file with old prices is worse than none.

What moves AI visibility more

If the goal is to be named by ChatGPT, Perplexity or Google's AI Mode, the work that counts is the same work Google describes for search: pages the crawlers can read in plain HTML, clear answers to the questions buyers ask, structured data, fast loading and honest mentions on other sites. My Astro vs Webflow comparison explains why HTML that does not depend on JavaScript matters for AI crawlers.

That is what my GEO service and agent-ready websites work covers, and llms.txt is one line item in it. SEO and GEO basics, llms.txt included, come with every build: a landing page from $1,200 (1 to 2 weeks), a multi-page site from $3,000 (2 to 4 weeks), a migration from $2,200 (2 to 4 weeks). See pricing, or start with the free AI visibility check to see where AI assistants already mention you.

If you need this built

A landing page from Pavlo starts at $1,200 (1 to 2 weeks), a multi-page site at $3,000 (2 to 4 weeks). The price is fixed before work starts.

See all prices

Frequently asked questions

What is llms.txt?

llms.txt is a Markdown file at the root of a website, at /llms.txt, that gives AI agents a short summary of the site and links to its most useful pages. Jeremy Howard proposed it in September 2024.

Does Google use llms.txt?

No. Google's guide to generative AI features in Search says Google Search ignores llms.txt, and that keeping one will neither harm nor help your visibility or rankings.

Do ChatGPT and other AI assistants read llms.txt?

Rarely, so far. An Ahrefs log study of May 2026 found that 97% of llms.txt files got no requests at all, and AI retrieval bots made 1.1% of the requests that did arrive. Coding agents read the file more than AI search does.

Is llms.txt the same as robots.txt?

No. robots.txt is a standard that tells crawlers what they may visit. llms.txt controls nothing and blocks nothing; it only describes the site and points to key pages.

What should an llms.txt file contain?

An H1 with the site name, a one paragraph summary in a blockquote, and H2 sections of links with a short note on each. Add the facts buyers ask about, such as prices and timelines, and keep it short.

Should my startup website have an llms.txt file?

Yes, if it is generated from your content and kept current: it costs little and does no harm. Expect real use mainly on developer docs, and put the effort for AI visibility into crawlable pages, clear answers and mentions.

Pavlo Zhydkykh

Written by

Independent designer and developer in Eastern Europe. He builds websites on Framer and on Astro with Sanity for AI, fintech and B2B startups, and writes these guides from client work.

First published · Updated

Start a project
Halftone mouse cursor Free tool Does ChatGPT recommend you?

Free report in 48 hours · 5 AI engines

Checklist

6 llms.txt best practices for a startup site

The file is small, so every line counts. These are the rules Pavlo Zhydkykh follows when he adds llms.txt to a Framer or Astro + Sanity build.

  1. 1. Generate the file at build time

    Build llms.txt from the same content as the pages, so a price or a service never differs between the file and the site.

  2. 2. Open with one clear sentence

    The blockquote says who the company is, who the company serves and what the product does, in words a buyer would use.

  3. 3. Link only to pages that answer

    Services, pricing, case studies, docs and the best articles; leave out tag pages, legal pages and thin posts.

  4. 4. Note every link

    A short note after the colon is what an agent reads to decide which page to open.

  5. 5. Put the asked facts in the file

    Prices, timelines, locations and supported platforms belong in the file, not only behind a link.

  6. 6. Keep robots.txt in charge

    llms.txt grants no access; never list a page there that robots.txt blocks for the same crawler.

Fit check

Is an llms.txt file
right for you?

A good fit if

  • The company has developer docs or an API: coding agents are the readers the data actually shows.
  • The file can be generated from the site content, so Pavlo Zhydkykh keeps the file current with every deploy.
  • Buyers ask AI assistants about prices, timelines or integrations that the file can state plainly.
  • The site already serves the text in plain HTML, which Pavlo Zhydkykh checks first on every audit.

Not the right fit if

  • The goal is to rank higher in Google: Google Search ignores the file.
  • Nobody will update a hand written file, and stale prices are worse than no file.
  • The pages only show text after JavaScript runs: fix the site before adding llms.txt.
  • An agency sells "llms.txt optimization" as a separate package: Pavlo includes the file in every build.

Compared

llms.txt vs robots.txt

The names look alike, the jobs do not. Pavlo Zhydkykh sets up both on every build, and access rules always stay in robots.txt.

llms.txt vs robots.txt
Criterionrobots.txtllms.txt
JobTells crawlers what to visitSummarises the site and points to key pages
StatusInternet standard, RFC 9309 (2022)A proposal from 2024, no standards body
Main readerSearch and AI crawlersAI agents, mostly coding agents today
FormatPlain text rulesMarkdown
Controls accessYes, by voluntary complianceNo, the file only describes

In their words

llms.txt, in the words of the
proposal, Google and clients

Word for word, with the source linked: research, official documentation and the people Pavlo Zhydkykh has worked for.

“Agents are best served by concise, expert-level information gathered in a single, accessible location.”

llmstxt.orgThe /llms.txt fileSource ↗

“Doing so will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them.”

Google Search CentralOptimizing for generative AI featuresSource ↗

“And despite the filename, it is not a robots.txt-style directive: it controls nothing and blocks nothing.”

Ahrefsllms.txt study, June 2026Source ↗

“One of the best contractors I have worked with. Really thoughtful about communication. Went above and beyond in delivering the results that our team needs. Would 100% hire again for our projects.”

Client on UpworkFramer development · Framer Specialist contractSource ↗

“He's amazing in what he contributes, both design-wise and also from the ideas.”

Oliver BockOwner, The B2B Answer · Website design, 2025Source ↗

“Pavlo was a pleasure to work with on this project. The web and mobile designs he created were amazing. His communication was prompt which made for a smooth project. I highly recommend his services.”

Shawntae SpencerClient, QuickQuick Pay · Web and mobile design on UpworkSource ↗
Pavlo Zhydkykh

About the author

Pavlo Zhydkykh

Pavlo Zhydkykh is a web designer and developer in Eastern Europe who has built websites for AI, fintech and B2B startups since 2016, designing and building each project himself on Framer or Astro + Sanity. Clients include MStudio Asia, Silicon Data, Inturai, Accolade Pro, NimbleStax and Sancts.

  • Top Rated on Upwork
  • 100% Job Success
  • 10 years since 2016
  • 50+ sites shipped
  • 90+ PageSpeed at launch
Kulparkivska St. 200, Lviv 79021, Ukraine +380 93 610 62 68 pavlozhydkykh@gmail.com

Page reviewed by Pavlo Zhydkykh · Updated

AI visibility check

  1. 1Website
  2. 2Business
  3. 3Contact
  4. 4Check

Does ChatGPT recommend your website?

Pavlo Zhydkykh asks ChatGPT, Perplexity, Gemini, Claude and Grok 20 questions your buyers ask, and shows where you show up.

Free · report in 48 hours