The nqzai crawler

Last updated: September 14, 2026

nqzai fetches web pages on behalf of its customers. If you found this page from a User-Agent string in your server logs, this explains what we were doing and how to stop us.

What it is, and is not

nqzai is a marketing platform. It fetches a page when one of our customers asks for a specific piece of work — checking whether a link to their site is still live, reading a page they named, or looking at a company's homepage while researching a contact.

• It is not a general-purpose crawler. We do not follow links to discover new pages, and we build no index of the web.
• It is not continuous. Requests happen when a customer runs an action. There is no background schedule that revisits your site.
• It does not log in. We send no credentials, submit no forms, and make no attempt to pass authentication, a paywall, or any other access control.

How to identify us

Every request to a site we do not own carries one agent:

Mozilla/5.0 (compatible; nqzai-bot/1.0; +https://nqz.ai/bot; purpose)

Match on nqzai-bot and you have matched all of it. The last word says which feature made the request, so your log — and an email to us — can say exactly what we were doing:

• linkcheck — confirming a link to a customer's site still exists
• sitemap and publish-detect — reading sitemap.xml
• site-check — checking a site's basic availability
• rag-readiness, content-quality, aeo and geo-scorecard — assessing how a page reads to AI answer engines
• ai-search-prompt-generator — our free public tool, fetching the address a visitor typed in
• website-health — our free public whole-site health check, fetching sitemap URLs and same-origin pages up to its 500-page cap
• product-scan — reading a customer's own site to learn what they sell
• read-url — fetching one page a customer asked about in chat, to report what it contains
• content-verify — fetching a page an article we drafted cites, to check the claim against its source
• contact-research — looking at a company's homepage while researching a contact
• commoncrawl — reading the public Common Crawl index, not your site
• crawler-access-check — checking whether a site's firewall lets AI search crawlers in (see below)

One check names another crawler first

Our GEO scorecard and agent-readiness check ask whether a site's firewall turns away AI crawlers that its robots.txt allows. A firewall rule that matches a crawler's name only fires on a request carrying that name, so this check sends one request per crawler (OpenAI's OAI-SearchBot and GPTBot, PerplexityBot, ClaudeBot) with that crawler's name first and ours after it:

Mozilla/5.0 (compatible; OAI-SearchBot/1.3; +https://openai.com/searchbot; via nqzai-bot/1.0; +https://nqz.ai/bot; crawler-access-check)

It runs only when someone checks that site with one of those tools, sends one request per crawler, and still names nqzai-bot, so blocking nqzai-bot covers it. We are not those crawlers, and the request says so.

Before 14 September 2026 the same requests carried twelve different names under two brands, including OutboundBot and Outbound-Bot from an earlier product name. If you blocked one of those, block nqzai-bot now instead.

Two of our requests look like a browser, and that is worth saying plainly

Two narrow paths send a desktop Chrome User-Agent rather than the agent above: the liveness check that asks only whether a page still responds, and the step that follows a redirect chain. Both do it for the same reason — many sites and firewalls reject unfamiliar agents outright, and a verifier that reported those rejections as "this page is dead" would give our customers false results about your site.

We are telling you because the alternative is worse. If you block nqzai-bot and still see occasional requests, that is why, and it is not an attempt to evade you. If you want those stopped as well, email us and we will exclude your domain.

robots.txt

We do not currently read your robots.txt before making these requests. We are not claiming an exemption — it is simply not implemented today, and we would rather say so than imply a courtesy we do not extend. Blocking by user agent, or emailing us, both work now.

How much we ask for

• A request times out after 10–15 seconds and we read at most a few megabytes.
• We read HTML only. We do not fetch images, scripts, or stylesheets.
• A link-verification run fetches at most 200 pages in total, at most 8 at a time.
• That cap is a total, not a per-site one, so a customer whose links come mostly from a single domain can produce a burst at that domain. If our traffic is a problem for you, the fastest fix is to tell us.

Blocking us

Block nqzai-bot at your web server, firewall, or CDN and those requests stop. To have your domain excluded entirely — including the two browser-agent paths — email palash@nqz.ai with the domain. We will action it and confirm.

If our traffic has caused you a problem, we would like to know. The same address reaches a person.

Contact

Questions about our crawler, or want your domain excluded? Email palash@nqz.ai.