Skip to main content

Adding llms.txt to a Docusaurus Site

The llms.txt standard provides AI systems with a curated, Markdown-formatted summary of your website's content. This guide walks through adding an llms.txt file to a Docusaurus site using the static directory approach. No plugins required.

What You'll Accomplish​

By the end of this guide, your Docusaurus site will serve an llms.txt file at its root URL (e.g., https://yoursite.com/llms.txt) that AI systems can use to understand your site's structure, content, and purpose.

Prerequisites​

  • A working Docusaurus site (v2 or v3)
  • Familiarity with the /static directory in Docusaurus
  • An understanding of what content your site offers (you'll be summarizing it)

No npm packages, build plugins, or configuration changes are required.

Background: What llms.txt Is​

The llms.txt standard was proposed by Jeremy Howard in late 2024. A site's /llms.txt file is a Markdown document that contains a structured summary: the site's title, a description (as a blockquote), and categorized sections with links to key pages, each annotated with a brief description of what they contain.

The format looks like this:

# Site Title

> A one-paragraph description of what this site is and who it's for.

## Section Name

- [Page Title](https://yoursite.com/page-url): Brief description of what this page covers.
- [Another Page](https://yoursite.com/another): What this one is about.

## Optional

- [Less Critical Resource](https://yoursite.com/resource): Description.

The ## Optional section signals content that an AI system can skip if it's working within a tight context window. Everything above it is considered essential.

For the full spec, see Jeremy Howard's original proposal.

Step 1: Create the File​

Docusaurus serves everything in the /static directory as-is at the site root. A file at static/llms.txt becomes available at https://yoursite.com/llms.txt after build. No routing configuration needed.

Create the file:

static/llms.txt

Step 2: Write the Header​

Start with an H1 title and a blockquote description. The title should be your site's name. The description should be a concise, factual summary of what the site contains and who it's for. Write this for a machine audience: clear, specific, and free of marketing language.

# My Documentation Site

> Technical documentation for the Acme Widget API, including getting started
> guides, endpoint references, authentication workflows, and SDK examples.
> Intended audience: developers integrating with the Acme platform.

The description is your elevator pitch to an AI system. If a language model reads only this paragraph, it should know whether your site is relevant to a given query.

Step 3: Organize Your Content into Sections​

Group your pages into logical sections using H2 headers. Each section contains a list of links, where each link has a title and a colon-separated description. Common section patterns include:

By content type:

## Guides

- [Getting Started](https://yoursite.com/docs/getting-started): Step-by-step setup instructions for new users.

## API Reference

- [Authentication](https://yoursite.com/docs/api/auth): OAuth2 flow, API keys, and token management.

By topic area:

## Authentication

- [OAuth2 Guide](https://yoursite.com/docs/oauth2): Complete walkthrough of the OAuth2 integration.
- [API Keys](https://yoursite.com/docs/api-keys): Creating, rotating, and revoking API keys.

The choice depends on how your site is organized and what grouping would be most useful to an AI system trying to find relevant content.

Step 4: Write Useful Descriptions​

The description after each link is the most important part of the file. AI systems use these descriptions to decide which pages to fetch when they need more detail. A good description answers: "What will I learn if I follow this link?"

Weak descriptions:

- [Authentication](https://yoursite.com/docs/auth): Authentication documentation.
- [FAQ](https://yoursite.com/docs/faq): Frequently asked questions.

Strong descriptions:

- [Authentication](https://yoursite.com/docs/auth): OAuth2 flow, API key management, token refresh, and rate limiting by auth method. Includes code examples in Python, JavaScript, and C#.
- [FAQ](https://yoursite.com/docs/faq): Answers to common integration questions including webhook retry behavior, idempotency keys, sandbox vs. production differences, and error code explanations.

The difference matters. A strong description tells the AI system what the page contains specifically, so it can decide whether to fetch it without guessing.

Step 5: Add the Optional Section​

Content in the ## Optional section is explicitly deprioritized. AI systems with limited context windows can skip this section and still have a complete picture of your site. Use it for supplementary content: changelogs, contributor guides, blog archives, or resources that are useful but not essential.

## Optional

- [Changelog](https://yoursite.com/changelog): Version history and release notes.
- [Blog](https://yoursite.com/blog): Technical articles about the platform and industry.
- [Contributing Guide](https://yoursite.com/contributing): How to contribute to the project.

Step 6: Use Full URLs​

Use absolute URLs (starting with https://) for all links, not relative paths. AI systems fetching your llms.txt file may not have the context to resolve relative URLs, and the llms.txt spec expects fully qualified URLs.

## Correct

- [Guide](https://yoursite.com/docs/guide): Description.

## Incorrect

- [Guide](/docs/guide): Description.

Step 7: Build and Verify​

Run your Docusaurus build and verify the file is served correctly:

npx docusaurus build

After building, check that llms.txt exists in the build output:

ls build/llms.txt

If you're running the dev server (npx docusaurus start), you can verify the file is accessible at http://localhost:3000/llms.txt.

Docusaurus-Specific Considerations​

Broken link checking: Docusaurus's strict link checker (onBrokenLinks: 'throw') does not validate links inside files in the static/ directory. Your llms.txt links won't be checked during build. Verify them manually or with an external link checker after deployment.

The plugin alternative: A Docusaurus plugin called docusaurus-plugin-llms-txt exists that auto-generates llms.txt from your docs structure. The manual approach described here is recommended instead, for two reasons. First, auto-generation produces generic descriptions; it can list your pages, but it can't write the kind of contextual descriptions that make llms.txt useful. Second, the manual approach forces you to think about what content matters most, which is the whole point of the standard.

Updating the file: llms.txt is a static file. When you add, remove, or restructure pages on your site, update static/llms.txt to match. Consider adding a reminder to your content publishing checklist. A stale llms.txt file that lists pages which no longer exist is worse than no llms.txt file at all.

Companion file (llms-full.txt): The spec also defines an optional llms-full.txt file that includes the full Markdown content of every page listed in llms.txt, concatenated into a single file. For small-to-medium Docusaurus sites, this can be useful; it gives AI systems everything in one fetch. For larger sites, the file becomes unwieldy. If you want to create one, place it at static/llms-full.txt using the same approach.

Expected Outcome​

After following this guide, your site serves an llms.txt file that:

  • Lives at https://yoursite.com/llms.txt
  • Contains a structured summary with your site title, description, and categorized page links
  • Uses full URLs and descriptive annotations for every link
  • Separates essential and optional content
  • Requires no build plugins or configuration changes

Troubleshooting​

The file doesn't appear after build. Verify the file is in the static/ directory (not src/ or docs/). Docusaurus copies static/ contents to the build root verbatim.

The file appears but isn't served in development. Restart the dev server after adding new files to static/. The dev server watches for changes but may not pick up new files added while it's running.

Links return 404 after deployment. Double-check that your URLs use the correct base path. If your site is deployed at https://example.com/docs/ (with a baseUrl of /docs/), your llms.txt links need to include that prefix: https://example.com/docs/getting-started, not https://example.com/getting-started.

Further Reading​