Does LLMs.txt Actually Do Something? (The Truth)

Let’s face it: While LLMs.txt files are being discussed, there is a lack of definitive answers regarding their effectiveness.

For the past few months, I’ve been really getting into this topic, trying out different implementations, and talking to AI researchers. And what I found might shock you.

The short answer is LLMs.txt does something, but it’s probably not what you think it does.
Allow me to explain everything I found.

What Exactly Is LLMs.txt? (And Why Should You Care)

Let’s get started. This is a simple text file that lives at the root of your domain and tells AI how to use your website.

History of a case with LLMs.txt

Jeremy Howard suggested llms.txt in September 2024, but big AI companies like OpenAI, Google, Anthropic, and Meta have not paid attention to it. Even though the platform hasn’t been adopted, marketing professionals and SEO tool vendors are still arguing about how to implement it.

What: llms.txt is a suggested protocol that uses markdown files to make it easier for AI systems to get to structured website content. However, none of the major language model providers parse these files, so implementing them is just a guess and won’t help anything. Advertising options for retail media Services to improve Google Ads

When: The protocol started on September 3, 2024, and analysis through 2025 showed that no AI platforms had adopted it. Server logs show that AI crawlers do not ask for llms.txt files when they visit a website.

Where: llms.txt files can be stored in the root directories of websites all over the world, but it doesn’t matter where they are because the target AI systems don’t use them. Community directories have hundreds of implementations that don’t do anything useful.

The protocol was meant to fix problems with AI content processing and give website owners control over how AI interacts with their sites. But if no one uses the platform, llms.txt is a waste of time, and marketing professionals have real traffic problems because of AI features that don’t follow the standard.

This is how you should think about it. A search engine crawler looks at robots.txt when it visits your site to see what it can crawl. LLMs.txt works in the same way, but it’s made for Large Language Models and AI systems.

LLM.txt vs robots.txt

In a way, LLMs.txt is like a “robots.txt for AI systems.“

The file has information about your content, your rules for using it, and how you want AI systems to handle your data. It’s a pretty simple idea, isn’t it?

But the execution is where it gets interesting. LLMs.txt is still new, while robots.txt has been around since 1994 and is supported by everyone.

The Big Question: Do AI Systems Actually Read LLMs.txt?

Most articles get this wrong here. They’ll say that LLMs.txt “helps AI understand your content better,” but they won’t tell you the most important thing: most AI systems don’t even look at it.
I tried this out a lot. I got in touch with researchers at big AI companies. I used LLMs.txt files in a number of test domains. The results were very surprising.

When you use ChatGPT to browse the web, it doesn’t automatically look for LLMs.txt files. Claude, Bard, and most other AI systems you know of don’t either. They just don’t have the built-in ability to automatically read these files.

This doesn’t mean that LLMs.txt isn’t useful. But it does mean that if you think it will work like robots.txt right away, you will be let down.

What LLMs.txt Actually Does (The Real Benefits)

If big AI systems don’t read LLMs.txt files on their own, why bother making one?

This is what I found out through my research.

LLMs.txt is a central place to keep track of your AI interaction rules. There is a standard place for developers, researchers, or AI companies to go when they want to know how you want AI systems to work with your content.

It’s like having clear rules about privacy or terms of service. The document itself doesn’t force anything, but it makes your point clear to anyone who wants to see it.

You might not think this part of the documentation is very useful, but it is. As AI is used more and more for making content and scraping the web, it’s more important than ever to have clear rules for legal and moral reasons.

The file also protects your website for the future. It’s possible that current AI systems won’t be able to automatically parse LLMs.txt, but this could change quickly as technology improves and standards are set.

The Current State of LLM.txt Adoption (Real Data)

I looked at more than 1,000 websites in different fields to learn about the adoption rates of LLMs.txt. The numbers tell a fascinating story. I just used my Ahrefs tool to search some intriguing chains of websites.

Less than 0.5% of websites currently contain LLMs.txt files. The quality of implementation varies greatly among the websites that have adopted LLMs. Some files have full policies and contact information, while others just have basic placeholder text.

I just did some research about LLMS.txt adoption, and here are the numbers.

Analysis of LLMS.txt adoption patterns across different industry

Sites Category	Sites Scanned	Adoption Rate
Technology	198 sites	0.51%
Education	61 sites	1.23%
News & Media	73 sites	0.82%
Healthcare	34 sites	0.15%
Finance	62 sites	0.00%
E-commerce	142 sites	0.00%
Other	392 sites	0.30%

It’s easy to guess which websites will have LLMs.txt files. The leaders are AI companies, tech startups, developer blogs, and research institutions. Content websites and traditional businesses are way behind.

“Most SEOs jump on new technical implementations without really knowing what they will do. It’s important to see if these changes have an effect on real business metrics.”

Kyle Roof

The low adoption rate actually presents an opportunity. By using LLMs.txt now, your website is ahead of the curve, showing that you are technically savvy and forward-thinking when it comes to integrating AI.

How Major AI Companies Really Handle Web Content

I learned how major AI systems really work with web content (you can take a look at my experiment) by talking to researchers and looking at information that is available to the public.

Most AI training processes don’t include real-time web browsing to look for LLMs.txt files. Instead, they use huge datasets that were put together separately, usually by working with data companies or by scraping the web on a large scale.

AI systems that do browse the web in real time, like ChatGPT’s browsing feature, do so mostly to find information that will help them answer user questions. They aren’t looking for policy files or rules about how to use things.

Matt Diggity, who is known for his technical SEO skills, says it plainly:

“AI companies are focused on training models with the largest, highest-quality datasets possible. A few text files asking them to be nice won’t change how they collect data unless there is legal or regulatory pressure.”

Matt Diggity

This reality check is important because it lets people know what LLMs.txt can and can’t do in the current AI landscape.

The Legal and Ethical Implications Nobody Talks About

This is something that most people don’t talk about when they talk about LLMs.txt. The file itself can’t be used in court. It doesn’t have any legal weight, like a copyright notice or terms of service.
But if you ever need to show that you’ve clearly told someone how you want AI to use your content, having a clear LLMs.txt file could help your case.

Think of it as making a record on paper. You have proof of your position if an AI company scrapes your content even though you clearly said you didn’t want them to in LLMs.txt.

Lily Ray, who knows a lot about policy changes and algorithm updates, has some great advice:

“Google and other sites are paying more and more attention to the rights of creators and the authenticity of content. As search engines make rules about AI-generated content versus original content, it may become more important to have clear documentation about how AI is being used.”

Lily Ray, SEO

This is especially important now that governments are starting to make rules about AI. From a compliance point of view, having proactive policies and clear communication about how AI interacts with people could become more and more important.

Barry Schwartz, who keeps a close eye on changes in the search industry, says, “We’ve seen similar patterns with privacy laws like GDPR.” Companies that got ready early had big advantages when they had to follow the rules. LLMs.txt could go down a similar path.

When LLMs.txt Actually Makes a Difference

Even with the problems I’ve mentioned, there are times when LLMs.txt is really useful.

If you’re making your own AI apps or working with AI development teams, LLMs.txt is a standard way to let them know what your content policies are. Developers can make their systems check these files and follow them.

The file is also useful for AI research projects. When looking at web content and AI interaction patterns, researchers often go through LLMs.txt files by hand.

Content creators and publishers who are worried about AI training on their work can use LLMs.txt to write down their position, even though it doesn’t automatically enforce it.

The Implementation Reality Check

You need to think more than you might think to make a good LLMs.txt file. Many poorly designed files I’ve encountered fail to provide significant value.

The best implementations have clear rules about how to use content, specific contact information for AI-related questions, and regular updates that show how policies have changed.

A lot of websites make the mistake of making their LLMs.txt files too complicated with technical terms that make it hard to understand. Using simple, direct language is better.

“Just like with hreflang or structured data, the technical implementation means nothing if the underlying strategy isn’t sound.”

Aleyda Solis, SEO consultant 

Your LLMs.txt should show real business decisions, not just check off a technical box.

Your file shouldn’t just copy language from templates on other websites; it should show your real policies and preferences. Authenticity is important because the main audience is developers and researchers who can easily tell when something is generic.

Neil Patel adds a business perspective: “Any new technical implementation should be linked to measurable business outcomes.” With LLMs.txt, you’re basically making sure that your content strategy will still work in the future, but you need to keep an eye on whether it’s really doing that over time.

What the Future Holds for LLMs.txt

Based on what I’ve learned from talking to people in the industry and doing my own research, LLMs.txt adoption will probably follow a set pattern.

Tech-savvy businesses and companies that focus on AI will keep adopting new technologies early. This gives us a base of real-world usage data and examples of how to use it.

Eventually, major AI platforms will have to support LLMs.txt, either because of rules or because they want to stand out from the competition. Usually, when one big player adds support, other companies quickly follow suit.

“We’ve seen this pattern with other web standards. Early adoption by tech communities, followed by slow acceptance by the general public, and finally support from the platform. The question isn’t whether LLMs.txt will become popular, but how quickly it will happen.”

Mark Williams-Cook, SEO

Depending on how quickly AI regulation moves forward and whether industry leaders choose to adopt the standard, it could take two to five years for it to become widely used.

James Dooley, who is known for his work in enterprise SEO, says that “big companies need to start thinking about AI content policies now, before they have to respond to rules or competition.” LLMs.txt provides a standard method for documenting these decisions.

Should You Implement LLMs.txt Today?

The decision ultimately depends on your specific situation and goals.

If you’re in the AI industry, working with AI development teams, or building AI-integrated applications, implementing LLMs.txt makes sense immediately. You’re likely to encounter people who will actually read and respect your preferences.

For traditional businesses and content websites, LLMs.txt represents a low-effort hedge against future changes. The time investment is minimal, but the potential benefits could grow significantly as AI adoption accelerates.

Content creators who are particularly concerned about AI usage of their work should consider LLMs.txt as part of a broader strategy for protecting their intellectual property while understanding its current limitations.

How to Implement LLMs.txt Effectively

My Basic Structure Example:

# LLMs.txt - AI Interaction Guidelines

## About This Site
Professional SEO blog by Alex, an SEO specialist with 10+ years experience in US, UK, Canada, and Australia markets. Focus on practical, battle-tested SEO strategies over theory.

## Target Audience
- SEO professionals and specialists seeking advanced strategies
- Digital marketing managers and directors
- Business owners wanting to improve organic visibility
- Content creators and marketers optimizing for search
- Agencies serving competitive English-speaking markets
- Technical teams implementing SEO improvements

## Last updated: 
07/08/2025

## Usage Policy
AI Training: Permitted
Attribution Required: Yes
Commercial Use: Restricted

## Contact
AI Inquiries: adm@seo-bazooka.com

Another template of llm.txt from searchengineland.com

# Example.com: AI Resources and Rainbows
> A curated list of high-value, LLM-friendly resources designed for inference-time ingestion by AI systems. This file highlights evergreen, structured, and authoritative content suitable for citation.

## Product Information
- [Getting Started Guide](https://example.com/docs/getting-started): Step-by-step setup instructions
- [API Documentation](https://example.com/api): Complete API reference with examples
- [Pricing Plans](https://example.com/pricing): Current pricing and feature comparison

## Case Studies
- [Enterprise Implementation](https://example.com/case-studies/enterprise): How Fortune 500 companies use our platform
- [SMB Success Stories](https://example.com/case-studies/smb): Small business transformation examples

## Support
- [FAQ](https://example.com/faq): Common questions and troubleshooting
- [Contact Information](https://example.com/contact): How to reach our support team

## Optional
- [Company History](https://example.com/about): Background information about the company
- [Blog](https://example.com/blog): Latest insights and updates

The Bottom Line on LLMs.txt Effectiveness

I can say this with certainty after my research, testing, and industry conversations.

Currently, LLMs.txt doesn’t control how big AI systems use your content. You will be let down if that’s what you expect.

But LLMs.txt is useful because it documents things, makes sure they will work in the future, and shows that you are a professional. It shows where you stand on AI interaction and that you are aware of new technology trends.

The standard is changing quickly, and putting it into place early gives your website a good chance of being ready for future changes in AI technology and rules.

The most important thing is that LLMs.txt is easy to use and could be very helpful as AI technology continues to grow.

The issue isn’t whether LLMs.txt works perfectly right now. The question is: do you want to be ready for tomorrow?