llms.txt explained: what it is and do you need one?

What llms.txt is, how it differs from robots.txt and sitemaps, whether AI search engines use it, and how to write a simple, useful llms.txt file.

llms.txt is a proposed standard for a plain Markdown file at the root of your website that gives large language models a short summary of your site and links to your most useful pages. It is cheap to add and harmless when done well, but it is not a ranking lever: the major AI search engines have not confirmed that they use it. Treat it as tidy housekeeping, not a shortcut to AI visibility.

Below we cover what the file is, what it is not, whether your business needs one, and how to write a good one.

What llms.txt actually is

The proposal, published at llmstxt.org, describes a file located at /llms.txt on your domain. The idea is simple. Websites are built for human visitors, with navigation, scripts, pop-ups and layout code wrapped around the actual content. A language model trying to understand a site has to work through all of that. A short Markdown file can hand it the essentials directly.

The format is intentionally light:

  • An H1 with the name of the site or project (the only required part)
  • A short summary in a blockquote
  • Optional paragraphs or notes with more context
  • H2 sections containing lists of links, each with an optional short description
  • An optional section titled “Optional” for links that can be skipped when context is limited

The proposal also suggests offering clean Markdown versions of important pages, for example by adding .md to a page URL. That part is optional and less commonly done.

What llms.txt is not

Because the name looks like robots.txt, people often assume the two do similar jobs. They do not.

  • It is not an access control. llms.txt cannot allow or block any crawler. If you want to control which AI bots can fetch your pages, that is still the job of robots.txt.
  • It is not a sitemap. An XML sitemap lists the URLs you want search engines to discover and index. llms.txt is a curated, human-readable shortlist with context.
  • It is not a confirmed ranking signal. As of this writing, OpenAI, Google and Perplexity have not publicly stated that their search products use llms.txt to choose sources. Google has said there are no special requirements to appear in AI Overviews beyond normal search eligibility.
  • It is not a place for marketing claims. Stuffing the file with keywords or promotional language misses the point. If any tool does read it, clear facts are more useful than slogans.

If you want the bigger picture on how AI search picks sources, read our guide on how to get cited in ChatGPT. The factors that clearly matter, such as crawler access, indexing, answer-first content and brand consistency, all come before llms.txt.

Who is using it today

Adoption so far is strongest among developer tools, software documentation sites and API providers. That makes sense. Developers often paste documentation into AI coding assistants or point tools at a docs site, and a clean Markdown index makes that easier. Some documentation platforms now generate llms.txt automatically.

For general business websites, the picture is less clear. Some AI tools and agents may fetch the file when they come across it, and some site owners report seeing requests for it in their server logs. But fetching a file is not the same as using it to decide what to cite, and there is no public confirmation from the major AI search engines that it affects their answers.

The honest summary: llms.txt is a reasonable bet on where things might go, with limited confirmed payoff today.

Do you need one?

For most small and local businesses, llms.txt is a nice-to-have, not a priority. Here is a simple way to decide.

Add one now if:

  • You publish documentation, guides or a knowledge base that people use with AI tools
  • Your site has a lot of content and a clear set of pages you want to highlight
  • Your core SEO and crawler access are already in good shape and you want to cover the small extras

Put it lower on the list if:

  • Your robots.txt might be blocking search or AI crawlers and you have not checked
  • Your Google Business Profile or directory listings have inconsistent details
  • Your key service pages do not yet answer customer questions clearly

Those fixes have a much clearer effect on visibility. Once they are done, adding llms.txt takes very little time.

How to write a good llms.txt

A good file is short, accurate and points only to pages worth reading. Here is a simple example for a fictional dental practice:

# Riverside Family Dental

> Riverside Family Dental is a general and cosmetic dental practice in Plano, Texas, serving families since 2009. We offer preventive care, implants, Invisalign and same-day emergency appointments.

Located at 1200 Example Parkway, Plano, TX. Open Monday to Friday, 8am to 5pm. New patients welcome.

## Services

- [Dental implants](https://example.com/dental-implants): Single and multiple implants, process and timeline
- [Invisalign](https://example.com/invisalign): Clear aligner treatment for teens and adults
- [Emergency dentist](https://example.com/emergency-dentist): Same-day appointments for pain, swelling and broken teeth

## Patient information

- [Insurance and financing](https://example.com/insurance): Accepted plans and payment options
- [New patient guide](https://example.com/new-patients): What to expect at the first visit

## Optional

- [Blog](https://example.com/blog): Articles on oral health and treatment options

A few guidelines when writing yours:

  1. Lead with plain facts. The summary should say who you are, what you do and where, in one or two sentences.
  2. Link to canonical URLs. Use the live, preferred version of each page, not redirects or parameter URLs.
  3. Keep descriptions short. One line per link is enough to explain why it matters.
  4. Pick your best pages. This is a curated list. Ten strong links beat a hundred weak ones.
  5. Match your other sources. Business name, address and services should match your website, Google Business Profile and schema markup exactly.

If you would rather not build it by hand, our llms.txt generator creates a correctly formatted file from a few inputs.

Common mistakes to avoid

Most problems with llms.txt files we review come from treating the file as a dumping ground rather than a curated guide.

  • Listing every URL on the site. That is what your XML sitemap is for. A long, unprioritized list defeats the purpose of a short summary.
  • Linking to pages that redirect or no longer exist. Every link should load directly with a normal status code.
  • Contradicting your other sources. If the file says you are open on Saturdays and your Google Business Profile says you are not, you have created confusion, not clarity.
  • Writing promotional copy. Phrases like “the best dentist in Texas” add nothing a model can verify. State what you do, where and for whom.
  • Forgetting it exists. A file written once and never reviewed drifts out of date as the site changes.

How to publish and maintain it

Upload the file to the root of your domain so it loads at yourdomain.com/llms.txt. Open that URL in a browser to confirm it is public and displays as plain text. If you use a CDN or security plugin, make sure it is not blocking requests to the file.

Then add it to your maintenance routine. Review it whenever you add or retire services, move pages or change core details like hours or locations. A stale llms.txt with broken links sends the wrong signal to any tool that reads it.

It is also worth checking your robots.txt at the same time, since that file has a much clearer effect on whether AI search tools can reach your content at all. Our robots.txt tester checks whether specific crawlers are allowed.

Where to start

Check the basics first: confirm your robots.txt allows the search and AI crawlers you want, and make sure your main pages answer real customer questions near the top. Then spend half an hour writing a short, accurate llms.txt and upload it to your root domain. Review it each quarter alongside your other site files.

If you want help with the full picture, our AI search optimization service covers crawler access, content structure, structured data and files like llms.txt as part of one plan.

Frequently asked questions

Does llms.txt help my site rank in ChatGPT or Google?

There is no confirmed evidence that it does. llms.txt is a proposed standard, and the major AI search engines have not publicly committed to using it as a ranking or citation signal. Google has said no special files are needed to appear in AI Overviews. Treat llms.txt as a low-cost helper for tools that choose to read it, not as a ranking lever.

Is llms.txt the same as robots.txt?

No. robots.txt tells crawlers which parts of your site they may or may not access, and well-behaved crawlers follow those rules. llms.txt does not grant or block access to anything. It is a plain Markdown summary that points language models toward your most useful content. If you want to control AI crawler access, robots.txt is still the file to edit.

Where should the llms.txt file go?

The proposal says to place it at the root of your domain, so it is reachable at yourdomain.com/llms.txt, in the same location as robots.txt. It should be served as plain text and be publicly accessible without a login. If your site runs on a CMS that does not allow root files, check whether a plugin or your hosting settings can serve it there.

How often should I update my llms.txt file?

Update it whenever the pages it links to change in a meaningful way, such as new services, retired pages, moved URLs or changes to core business facts. For most small business sites that means reviewing it every few months and after any site restructure. A file pointing to broken or outdated pages is worse than not having one at all.

Shamim

Shamim

Founder of Serpastra. A full-stack SEO and digital marketer who spends most days on Local SEO for dental practices, AI search visibility and lead generation. More about Shamim.

Find out what is holding your rankings back

Send us your site. Within 2 business days you get a written audit, an AI visibility check and a short fix list ranked by impact. Free, with no contract attached.

Request my free audit