On 3 December 2025, Google quietly added an LLMs.txt file to its official Search Central developer documentation portal, despite having repeatedly dismissed the file as useless just months earlier. This was spotted by a SEO professional Lidia Infante which later confirmed that this discovery was made by Crystal Carter on BlueSky.
https://bsky.app/profile/crystalontheweb.bsky.social
For a long time, Google dismissed LLMs.txt. One of their own search advocates declared, “No AI system currently uses llms.txt.”
So this shift raises big questions:
- Is Google signalling an interest in AI‑based discovery or indexing behaviour?
- Are we entering a phase where AI‑powered search (via large language models) begins to treat LLMs.txt as a guide? Much like sitemap.xml works for web crawlers?
If that turns out to be true, ignoring LLMs.txt could be a missed opportunity.
What Is LLMs.txt?
Think of LLMs.txt as a “sitemap for AI.” Instead of listing every URL (like sitemap.xml does), it gives a curated list of your most important pages. Those you consider “core content.”
- It lives at yourdomain.com/llms.txt.
- It’s written in plain text or Markdown, easy for AI systems and humans alike.
- It signals to large language models (LLMs) like ChatGPT, Gemini, and Claude which pages you want them to prioritise.
In short:
- Robots.txt = “who can crawl.”
- Sitemap.xml = “show me everything.”
- LLMs.txt = “here’s the best stuff, especially for AI tools.”
The Plot Thickens
Later, on the same day, Google happened to remove the llms.txt file from its developer documentation. This raised questions about whether this was intentional or was it Google trolling us? No one really can say…
The Truth (So Far): LLMs.txt Is Still a Grey Zone
Here’s the catch: adoption is far from universal.
- As of mid‑2025, Google itself said no major AI system was using LLMs.txt.
- Early audits of server logs show that many AI bots still ignore the file entirely; only traditional crawlers appeared in most cases.
- SEO specialists describe the LLMs.txt hype as “more panic than proof.”
- Google removing the LLMs.txt file the same day
LLMs.txt might matter later. Right now, in late 2025, it remains optional, experimental, and speculative.
Google’s own file at:
- developers.google.com/search/docs/llms.txt
It lists key SEO docs like crawl budget management and structured data guides. Before being pulled shortly after discovery, suggesting a platform-wide auto-add rather than Search Central endorsement.
John Mueller’s “hmmn :-/” reply to questions about it fuels the confusion, with no official policy shift announced.
Should You Add LLMs.txt?
Most blogs and most SEO influencers might suggest you to test it with low effort. It’s one file, no commitment, right? Wrong.
John Mueller himself said that LLMs.txt is useless.
This means that even when added, it’s not going to give you any great results.
Our analogy says that many SEO executives, companies and freelancers are looking for a hack. A savior technique that they might use and suddenly everything is ok.
With the rise of new known terms like AEO and GEO, we have started to believe that a LLMS.txt file will save us while we all know good quality content that has semantic relevance along with good quality backlinks still matter the most.
So should you add LLMs.txt? It’s an option. Not a necessity.
Best Practices from Google Search Central
Google’s brief LLMs.txt highlighted essentials like crawl budget management for large e-commerce sites and structured data for products.
Stick to proven signals:
- Helpful content
- Mobile-first
- Core Web Vitals
Explore our SEO services to audit your crawl setup and prioritise what AI will value next. Real results start with sites Google already loves.
In summary, Google’s LLMs.txt blip confirms it’s no priority, yet. Focus on fundamentals, watch for bot shifts in Search Console. Ready for a no-fluff audit? Contact Cloudex Marketing to align your site for whatever AI throws next.
