Check any URL against both Googlebot and AI crawlers. Scans robots.txt rules, meta directives, content quality, structured data, and social tags. Get separate visibility scores for traditional search and AI search with plain language recommendations.
Four areas scored out of 100:
Results saved in this session. Click history to re-run.
A page can be blocked from search results in two places: the robots meta tag in its HTML and the robots header sent by the server. Most publishers check one and assume the rest. When the two disagree, search engines combine them, and the strictest rule wins. A single leftover directive can quietly remove a page from the index.
The failure modes are easy to miss. A staging page that inherited noindex, a header added by the CDN for one content type, or a plugin that injects nofollow across the site. Each one is invisible on the page itself, which is why the page still looks fine to visitors while disappearing from search.
The stakes compound with site migrations and redesigns. When robots settings change during a move, pages can end up blocked without anyone noticing until traffic drops. A checker that shows every directive and its source turns a mysterious ranking loss into a fixable configuration issue.
Enter the URL and run the scan. The tool fetches the page, reads the robots meta tag in the HTML, and inspects the robots HTTP header from the response. Each source is shown separately, so you can see exactly where every directive lives.
The verdict panel combines the two sources and tells you the effective state: indexable, noindexed, or followed or nofollowed. That combined view is what matters, because it is how crawlers actually read the page. Any conflict between the sources is called out explicitly.
Fix the source that is wrong, then rescan to confirm the effective state matches your intent. Use the checker after every deployment, migration, or plugin change that touches indexing. Thirty seconds of checking beats a month of puzzling over lost traffic.