BYD TechThe short answer
llms.txt is a proposed standard: a Markdown file at the root of a website, such as example.com/llms.txt, that gives AI tools a short, clean overview of the site with links to its key pages. It was proposed by Jeremy Howard in September 2024. It is not a ranking factor, and Google says no AI text files are needed for its AI features. It is cheap to add and can help AI tools that read it, as long as it stays accurate.
Key points
- A Markdown summary of your site, at /llms.txt.
- Proposed in 2024. A convention, not an official web standard.
- Google says it is not needed for AI Overviews or AI Mode.
- Useful for AI tools and agents that choose to read it.
- Generate it from your real content, so it never drifts from the site.
What llms.txt is
Web pages are built for people and browsers: menus, scripts, styling. That makes them noisy for an AI model trying to understand a site quickly. The llms.txt proposal, published at llmstxt.org, suggests a single Markdown file that summarizes the site in plain text and links to clean versions of the important pages.
It sits beside robots.txt and sitemap.xml in spirit, but does a different job. robots.txt says what may be crawled. The sitemap lists pages. llms.txt explains what the site is about.
What goes in it
- A heading with the name of the site or company.
- A short summary in a quoted block.
- A few paragraphs of key context: what you do and for whom.
- Sections of links to the most important pages, each with a one-line description.
- Optionally, a fuller companion file, llms-full.txt, with the content of key pages.
Does it help?
Be realistic. Google’s documentation on AI features says you do not need to create new machine-readable files or AI text files to appear in AI Overviews or AI Mode. The llms.txt site itself does not claim that search engines use it for ranking.
Where it can help is with AI tools and agents that look for it, for example coding assistants reading documentation, or assistants asked to research a company. It also makes you write down, in one place, exactly what your company does and does not claim, which is useful in itself.
Doing it well
- Generate it from the same content as your pages, so it updates with them.
- Keep it short. It is a guide to the site, not a copy of it.
- State facts plainly: what you do, who for, how to reach you.
- Never put claims in it that your pages do not make.
- Make sure your robots.txt and CDN let AI crawlers fetch it.
BYD Tech’s own llms.txt is generated from the site’s content, which keeps it in step with the pages. The wider practice of being cited by AI answers is in Generative engine optimization, and we build sites this way in website development.
Sources
Checked when this guide was written.
