For webmasters

The AD2SPIDER crawler

If you got here from your logs, this is what you need to know.

What it is

AD2SPIDER is a desktop application for technical SEO audits. Consultants, agencies and marketing teams install it on their own computer to review websites, usually their own or a client's.

Visits don't come from our servers: they come from the computer of the person running the crawl, with their IP address. We neither see nor store what it crawls.

How it identifies itself

By default, every request carries this user agent:

AD2SPIDER/<version> (+https://ad2spider.com/bot/)

Users can change it, as with any other desktop crawler.

How it behaves

  • Respects robots.txt by default, following RFC 9309.
  • Limits concurrent requests and speed per host.
  • Retries carefully on 429 and 5xx errors.
  • Never submits forms or performs actions on your site: it only reads pages.

How to block it

Add this to your robots.txt:

User-agent: AD2SPIDER
Disallow: /

If you see abusive use, write to hola@ad2spider.com with the date, time and an excerpt of your logs.