LLMs.txt Explained

Your guide to the web’s new LLM-ready content standard

You might’ve seen various dev tools adding LLMs.txt support to their docs recently. This proposed web standard is quickly gaining adoption, but what is it exactly and why does it matter?

While robots.txt and sitemap.xml are designed for search engines, LLMs.txt is optimized for reasoning engines. It provides information about a website to LLMs in a format they can easily understand.

So, how did LLMs.txt go from proposal to industry trend practically overnight?

How Mintlify Popularized LLMs.txt

On November 14th, Mintlify added LLMs.txt support to their docs platform. In one move, they made thousands of dev tools’ docs LLM-friendly, like Anthropic and Cursor.

Anthropic and others quickly posted on X about their LLMs.txt support. More Mintlify-hosted docs joined in, creating a wave of visibility for the proposed standard.

The momentum sparked new community sites and tools. @ifox created .

Mot, who made dotenvx, built and shared an that scrapes your website and creates the file for you.

What exactly is an LLMs.txt file?

LLMs.txt is a markdown file with a specific structure. The specification defines two distinct files:

1. /llms.txt: A streamlined view of your documentation navigation to help AI systems quickly understand your site’s structure
2. /llms-full.txt: A comprehensive file containing all your documentation in one place

/llms.txt

The file must start with an H1 project name, followed by a blockquote summary. Subsequent sections use H2 headers to organize documentation links. The “Optional” section specifically marking less critical resources.

# Project Name
> Brief project summary

Additional context and important notes

## Core Documentation
- [Quick Start](url): Description of the resource
- [API Reference](url): API documentation details

## Optional
- [Additional Resources](url): Supplementary information

For a simple example, see llmtxt.org’s own .

/llms-full.txt

While /llms.txt provides navigation and structure, /llms-full.txt contains the complete documentation content in markdown.

# AI Review (Beta)

AI Review is a feature that allows you to review your recent changes in your codebase to catch any potential bugs.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/cursor/images/advanced/review.png" alt="AI Review" />
</Frame>

You can click into individual review items to see the full context in the editor, and chat with the AI to get more information.

### Custom Review Instructions

In order for AI Review to work in your favor, you can provide custom instructions for the AI to focus on. For example,
if you want the AI to focus on performance-related issues, you could put:

```
focus on the performance of my code
```

This way, AI Review will focus on the performance of your code when scanning through your changes.

### Review Options

Currently, you have a several options to choose from to review:

*   `Review Working State`
    *   This will review your uncommitted changes.
*   `Review Diff with Main Branch`
    *   This will review the diff between your current working state and the main branch.
*   `Review Last Commit`
    *   This will review the last commit you made.

The above snippet is from Cursor’s /llms-full.txt file. See the full file on .

How to generate LLMs.txt files

There are several different tools you can use to create your own:

• : A tool by dotenvx’s creator Mot that generates llms.txt using your site’s sitemap.xml.
• was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.
  Vollständiger Original-Artikel

  Den kompletten Beitrag mit allen Details direkt auf towardsdatascience.com lesen.

  ↗ Original-Artikel auf towardsdatascience.com lesen