Should we publish llms.txt and allow AI crawlers?
Separates three decisions usually argued as one — crawler access, llms.txt, and licensing — and says which has an established effect.
What this checks
Separates three decisions usually argued as one — crawler access, llms.txt, and licensing — and says which has an established effect.
When to ask this
Best once you have some organic traffic and at least one AI-visibility measurement on file — useful across Some Traction and Heavy Traction stages. This is a live diagnosis over your own data, not a static report, so there's nothing to check before you've been measured at least once.
When this can't anchor to your data
This always returns its answer on the three decisions — crawler access, llms.txt, and licensing — nothing about your account makes it withhold the rules. It anchors once a crawl state exists for your site; without that:
No on-page crawl is stored for this site, so the per-agent state is unread here. Server logs are separately not ingested at all, so the volume those agents actually fetch is unknown either way.
How to ask it in chat
"should we publish llms.txt and allow ai crawlers"
What you'll get back
A direct answer computed from measurements already on file — no extra cost, no new data is purchased to answer this.