A noindex left in a template or one extra line in robots.txt can keep a whole section out of Google. The hard part is seeing it: the page works perfectly for users, and only a crawl reveals which signals it sends to search engines.
AD2SPIDER combines the meta robots tag, the X-Robots-Tag header, the canonical, robots.txt and the status code of every URL to work out its technical eligibility: whether it can be indexed and, if not, why. Without Search Console data it does not claim what is indexed; it tells you what would prevent it.
Indexability: noindex and robots warnings
Real problems worth fixing.
Page with noindex
- Why it matters
- The page asks not to be indexed (meta robots or X-Robots-Tag). Fine if intended; otherwise it will not appear in Google.
- How to fix it
- If it should rank, remove the noindex directive from the source shown.
Conflicting indexing directives
- Why it matters
- Some sources say index and others noindex. Google applies the most restrictive one, which may not be what you expect.
- How to fix it
- Keep a single consistent directive across meta robots, meta googlebot and X-Robots-Tag.
Unrecognized robots directive
- Why it matters
- There is a directive search engines do not recognize (possible typo, e.g. 'no-index'). It is ignored.
- How to fix it
- Fix the directive to a valid value (noindex, nofollow, none, max-snippet…).
Page with nofollow
- Why it matters
- The page asks not to follow any of its links: search engines may not discover or value the linked pages.
- How to fix it
- Remove page-level nofollow unless intended; use rel=nofollow on specific links only.
Indexability: noindex and robots information
Data to understand the site; not errors in themselves.
Canonicalised to another URL
- Why it matters
- The canonical points elsewhere: this page proposes indexing the other one. Normal for variants; a mistake if this is the main version.
- How to fix it
- Confirm the canonical points to the right version; if this is the main one, use a self-referencing canonical.