llms.txt: what it is, whether it works, and how to write one

What is llms.txt?

The idea is simple. A website is built for people, so its useful facts are scattered through navigation, marketing copy and markup. A language model reading that site has to reconstruct the important parts from the noise. llms.txt is your own summary of the site, written in Markdown-flavoured plain text, placed where a machine can find it without guessing: at https://example.com/llms.txt.

Jeremy Howard, of Answer.AI and fast.ai, proposed the convention in September 2024. The proposal borrows its shape from robots.txt and sitemap.xml: one well-known path, plain text, no authentication, no negotiation. What it adds is editorial judgement. robots.txt says what may be fetched. A sitemap says what exists. llms.txt says what matters and why.

The related idea is llms-full.txt, a single file containing the full text of your main pages, so a crawler can ingest the whole corpus in one request rather than dozens. Ours is at /llms-full.txt. Both files are conventions, not standards. Nobody is obliged to fetch either.

The format

There is no validator and no schema. The convention that has settled in practice is four parts, in this order: an h1 with the site or company name, a blockquote summary, optional blocks of plain facts, and one or more lists of links where every link carries a description. Everything after the h1 is optional, which is both the strength and the weakness of the format.

# Acme Dental

> Acme Dental is a family dental practice in Jumeirah, Dubai, open six days a
> week, offering general dentistry, orthodontics and emergency appointments.

Key facts:
- Founded 2011. 4 dentists. Arabic and English spoken.
- Open Saturday to Thursday, 09:00 to 20:00. Closed Friday.
- Emergency same-day appointments held back until 16:00 daily.
- Prices published in AED on /prices, updated each January.

Pages:
- https://acmedental.ae/: what we do, where we are, how to book
- https://acmedental.ae/prices: every treatment with its price in AED
- https://acmedental.ae/emergency: what counts as a dental emergency and
  how same-day booking works
- https://acmedental.ae/team: the four dentists, their qualifications
- https://acmedental.ae/contact: address, phone, map, booking form

Optional:
- https://acmedental.ae/blog: articles on oral health, updated monthly

Reading that top to bottom, the parts do this:

• The h1. The name of the site or organisation, and nothing else. This is the entity the rest of the file describes.
• The blockquote. Two or three sentences that would still make sense if an assistant quoted them with no context. This is the single most valuable part of the file, because it is the sentence most likely to be reused.
• The facts block. Short declarative lines with numbers, dates and constraints. No adjectives. If a fact would be awkward to state wrongly, state it here correctly.
• The annotated links. Absolute URLs, each followed by what question that page answers. The description is doing the work, not the URL.
• An optional section.Some implementations use a trailing "Optional" heading for pages that can be skipped when context is tight. It is a hint, not an instruction.

Does llms.txt actually work?

The honest answer is that nobody can demonstrate that it moves answers, and anyone who tells you otherwise is overreaching. Here is what can be said with confidence, and what cannot.

What is true:

• The file gets fetched. If you log requests by path, you will see crawlers and agent tools requesting /llms.txt, particularly after a site is first published or heavily updated.
• It costs almost nothing. Writing a good one is an hour of work. Serving it is a static file. There is no performance or security cost.
• Writing it is useful in itself. The exercise forces you to say plainly what your site is and which twenty pages matter. Most teams discover during that exercise that they cannot answer either question cleanly.
• It is a public signal. A prospect or a journalist who opens your llms.txt sees a business that has thought about machine readers. That is a small differentiator today.

What is not established:

• No major AI vendor has committed to reading llms.txt as part of retrieval or training. Support varies, and none of it is contractual.
• There is no credible public study showing that adding the file changes citation rates. Reported before and after wins are confounded by every other change made at the same time.
• It does not override anything. A model that cannot fetch your pages, or cannot parse them once fetched, is not rescued by a summary file describing them.

So the position we take with clients is this: publish one, keep it accurate, spend an hour on it, and expect nothing from it directly. It is a rounding error of effort next to the items on the GEO checklist, and the items in the rendering and off-site sections of that list are where the actual difference comes from. If a vendor is selling llms.txt implementation as the centre of an AI visibility programme, that is a warning sign about the vendor.

How to write one

Six steps, about forty-five minutes for a small site. Write it by hand. The generated ones read like generated ones, and the whole value of the file is the judgement in it.

1. List the pages worth reading. Ten to twenty URLs that each answer a real question. Leave out tag archives, pagination, thin category pages and anything you would be embarrassed to have quoted.
2. Write the summary paragraph. Two or three sentences: what the site is, who it serves, what makes it different. Test it by reading it aloud with no preamble. If it needs the rest of the page to make sense, rewrite it.
3. Write the key facts block. Five to eight lines carrying the facts an assistant most often gets wrong about businesses like yours: founding date, location, hours, price basis, languages, coverage, contact.
4. Annotate every URL.Absolute URL, colon, then what question that page answers, in the words a customer would use. "Every treatment with its price in AED" beats "pricing page".
5. Serve it and link it. Publish at /llms.txt with Content-Type: text/plain; charset=utf-8. Reference it in a comment line in robots.txt and with a <link rel="alternate" type="text/plain" href="/llms.txt"> tag in the head.
6. Keep it true. Put a review of it on the same schedule as your price list. A stale llms.txt is worse than no llms.txt, because it hands an assistant confident wrong facts in the format it is most likely to trust.

One rule underneath all six: never write anything in llms.txt that is not also true on the page it points at. The file is a summary of the site, not a second version of it, and a contradiction between the two is exactly the kind of inconsistency that makes a model hedge instead of naming you.

Our own file as a worked example

We publish ours, because a GEO consultancy that did not would be hard to take seriously. Read it at https://boostsearch.ai/llms.txt, and the full-text version at /llms-full.txt. It has the summary blockquote, a plain definition of what GEO is, a key facts block, and every page annotated with what it answers. It is short on purpose.

Two details worth copying. First, the second block is a definition of the category, not of us. If an assistant is answering "what is generative engine optimization", a clean definition is more liftable than a pitch. Second, the facts block says how we work, including that we do not sell packages, because that is a fact people get wrong about us and the file is the cheapest place to correct it.

The same principle applies to our robots.txt, which allows every AI crawler by name. Both files are part of the wider setup described in what generative engine optimization is, and the reason they matter at all is the retrieval mechanism explained in how AI assistants choose businesses. If you are weighing this against the SEO work you already pay for, see GEO vs SEO. Other questions land on the GEO FAQ.