Is llms.txt Actually Useful for E-commerce?

A DTC coffee-gear brand selling into the US and Germany finds a LinkedIn screenshot in Slack: add llms.txt and ChatGPT will cite you. The founder wants it live Friday. The engineer asks where it sits, who owns it, and how anyone would know it worked. Nobody answers. Two weeks later it ships — four thousand sitemap URLs under a new heading. Six weeks later, the only /llms.txt request in the CDN log is the engineer’s curl.

llms.txt is not a switch that makes AI index your store; it is a manual you hand the model, and the work worth doing is choosing twenty pages that settle a buying question, not exporting forty thousand SKUs.

Three files, three jobs

Three root-level files get collapsed into one idea. They answer different questions; only one is binding.

File	Job	Audience	Binding?
robots.txt	Crawl permission and exclusions	Crawlers	Yes, mainstream crawlers follow it
sitemap.xml	Complete URL inventory, favors coverage	Crawlers	Convention, not enforcement
llms.txt	Curated entries and notes, favors selection	Models and agents	No; a proposal only

Proposed by Jeremy Howard in September 2024, llms.txt is a Markdown page at the root: H1 site name, one-line summary, H2 link groups, optional notes. An optional llms-full.txt can hold page text.

What matters is enforcement. robots.txt is read before a crawler enters; llms.txt is a note left on the desk after it is already inside. As of January 2026, no major AI engine’s public documentation lists it as a crawl or ranking input, so publishing one is not a switch and omitting one is not a penalty.

Worth doing anyway: being wrong costs forty lines in a repo, not a replatform, and writing it forces the team to name entry points, policies, and evidence in one place a person can read. Put it on the technical hygiene checklist, not the growth experiment list.

Three ways to write it and get nothing

A sitemap with a new title. Four thousand URLs, no groups, no notes. The model gets a second inventory and still guesses which page matters; copying it discards the one thing llms.txt adds.

A full-text dump of products and prices. Price, stock, and promotions move hourly, so a static copy is stale at deploy. A model quoting a three-week-old price is worse than one quoting nothing: the complaint, the return, and the one-star review land on you. Link anything that changes. What a PDP should expose instead is covered in what a product looks like inside an AI answer.

A marketing page with links. “Industry-leading,” “best-in-class,” no dimensions, no return terms. Models need checkable nouns and numbers; adjectives that collide with your own evidence get resolved by writing nothing.

All three add words, not information. llms.txt is a navigation layer, not a content surface.

Five steps, subtraction first

Writing the file is easy; deciding what to leave out is the work. Steps run in dependency order — if the first fails, the rest is decoration.

Step	Action	Owner	Pass test
1	Preconditions: no-JS-readable pages, no AI-crawler block, one canonical	Engineering	curl returns 200 with readable copy
2	Entry list, 30–80 URLs: brand, guides, products, policies, evidence	Content and ops	Each entry earns its line
3	Generate from content data, never hand-write	Engineering	Script re-runs; diff reviewable
4	Validate path, 200, text/plain; charset=utf-8, no redirects, links reachable	Engineering	Failure blocks release
5	Watch logs for reads, then retest the same prompts	Growth and engineering	A comparison, not a screenshot

Step two carries the value: a list is editorial judgment, not a directory. Pages a buyer opens before paying go first; pages marketing wants to push go last. A polished PDP is less quotable than three pages answering which size, how often the filter changes, and who pays return shipping. Multilingual stores group by language and link only that language’s canonical URL. Length has no hard limit, but past a few hundred lines it stops being an index.

Example: a curated list for a pet fountain brand

Example (fictional brand, domain, and specs; the structure is the point).

# Example Pet Fountain Co.

> Smart pet fountains for the US and Germany; removable pump housing, 12-month warranty.

## Brand and entity
- [About](https://example.com/about): founding year, factory, warranty entity

## Buying guides
- [Cat or dog](https://example.com/guides/cat-vs-dog): spout height, noise, capacity
- [Filter replacement](https://example.com/guides/filter): interval by water hardness

## Core products
- [UV fountain, 2 L](https://example.com/p/uv-2l): price, stock, warranty on the page

## Policies
- [Shipping](https://example.com/shipping)
- [Returns](https://example.com/returns)

## Evidence
- [Third-party reviews](https://example.com/press)

Three details carry it: price and stock stay links, every entry says what a reader learns, and only the core model appears; long-tail SKUs belong to the sitemap. The audience is also the new ops hire, and a manual a person cannot follow is one a model will not follow either.

How to tell whether anything reads it

Search server or CDN logs for GET requests to /llms.txt, then strip noise: your own curl, uptime monitors, and scanners, filtered by source network. Agents can be spoofed, so write “unknown” rather than claim certainty. Fix a threshold before you look; one example is several 200s from non-owned networks in each of two consecutive weeks.

Proving citation is harder: put a distinctive phrase only in the notes and search answers for it. Finding it is correlation, not causation, so judge impact with prompt-level citation share against a baseline, as in AI visibility baseline. Logs diagnose; they do not score.

Where this breaks

Crawl and render are not fixed. llms.txt does not override robots.txt or turn a JavaScript-rendered page into text. When preconditions fail, work the earlier layers of the technical foundation line first, starting with whether AI can render your store. The map of AI search entry points sorts levers by entry point.

The catalog is huge and prices move by the minute. Maintenance exceeds the benefit; invest in feeds, structured data, and PDPs instead.

Sales happen mostly inside a marketplace. Platform AI reads listings, reviews, and fulfillment metrics; a root file on your domain cannot reach that layer.

Someone expects a ranking switch. There is no such mechanism; a team that sells it internally as one will spend the next quarter explaining a flat quarter.

No owner, no file. An unowned manual goes stale within a quarter, and a stale manual is worse than none.

If you only make one change this month, pull 20 URLs from your sitemap — one brand, three category, eight product, four policy, four evidence — and curl each for a 200, one canonical, and readable copy with JavaScript off. Pass, and engineering generates the first /llms.txt from those records. Fail once, fix that page first.