Why this Ahrefs llms.txt study matters for brands
As an expert branding curator, I recommend this concise, data-driven analysis on llms.txt adoption. Ahrefs studied logs from 137,000 domains and found 97 percent of files received zero requests. Only about 1,100 of roughly 38,000 valid files saw any traffic, mostly from non AI bots. The surprising audience includes SEO audit tools, profiling crawlers, and coding agents, not retrieval bots that power AI citations. This matters for brands planning content signals to emerging AI agents, and for teams automating llms.txt with CMS tools. The piece also flags prompt injection risks and measurement limits, so stakeholders should audit autogenerated files now.
Read this post to understand which bots actually fetch llms.txt, and why most files stay invisible. You will get clear figures, practical implications, and lens on how audits and scanners shape the file ecosystem. Learn why AI retrieval bots barely requested files, and why coding agents dominate real use cases. The study also compares Lighthouse audits, SEO tools, and research crawlers, revealing surprising traffic patterns. For brand teams and content strategists, the analysis offers immediate next steps and risk signals to act on. Dive into the original reporting for the data tables, methodology details, and nuanced takeaways I recommend sharing.
Source: www.searchenginejournal.com
