AureoCatalogBot

How to Recognise It

Every request carries this User-Agent, including robots.txt, sitemaps, product feeds, pages and images:

The token to use in robots.txt is AureoCatalogBot. Requests come from the machines that run catalog imports rather than from a fixed set of IP addresses, so the User-Agent is the way to address it.

What It Respects

• It reads your robots.txt before anything else and follows the group for AureoCatalogBot, or the * group when there is none. The rules apply to every request, including each redirect it follows.
• A declared Crawl-delay is honoured, up to 10 seconds, and it then makes one request at a time.
• If robots.txt itself answers with a 429 or a server error, it treats the whole site as off limits for that run, as RFC 9309 asks.
• A refusal (401, 403 or 429) is accepted. It never retries with a different identity.

How Your Content Is Used

Pages become catalog entries: product facts, an image, and a link to your page. A language model reads each page to pull out those facts. Nothing it reads is used to train a model. In Content Signals terms, that is search and ai-input at the reference level, never ai-train.

Blocking or Slowing It

To keep it out entirely, add this to your robots.txt:

User-agent: AureoCatalogBot
Disallow: /

To slow it down or keep it out of part of your site instead:

User-agent: AureoCatalogBot
Crawl-delay: 5
Disallow: /account/

Pages Fetched Through Cloudflare

Some pages are fetched for us by Cloudflare's crawler, which identifies itself as CloudflareBrowserRenderingCrawler/1.0 and follows robots.txt rules written for that name. A rule for AureoCatalogBot does not reach those requests, so to cover both, name both or use the * group.

Contact

If the crawler is causing trouble, or you would like your products listed or corrected, email [email protected]. A person reads every message.