Skip to main content
Bethemesh
Web & SEO

Local Indexability Analyzer

Analyze local indexability directives in HTML and an optional X-Robots-Tag without network requests.

  • Clipboard

Embed this widget

Customize the result, check the live preview, then copy the code.

Preview

Embed code type

Code to copy

Responsive and automatically resized by Bethemesh.

HTML code

Indexability report

Was this tool useful?

How does this tool work?

This analyser tells you whether a page can be indexed based on its HTML code and, if you provide it, its X-Robots-Tag HTTP header. It combines the directives of the robots and googlebot meta tags, compares the canonical with the page URL and lists the reasons for any block, without contacting the site.

The tool reads the <meta name="robots"> tag, the <meta name="googlebot"> tag and the X-Robots-Tag value you enter, then merges them: the most restrictive directive wins, as it does at Google.

If noindex appears in any of these sources, the page is reported as not indexable and the reason is added to the report. nofollow sets followLinks to false.

The canonical is resolved against the page URL, so a relative canonical such as /shoes/women becomes a full address. If it differs from the URL entered, a warning says so.

The report adds warnings for a missing canonical, title or meta description, and states its limit: neither robots.txt nor the live HTTP status is checked.

How to use this tool

  1. Paste the HTML

    At least the page’s <head> section, in the HTML code field.

  2. Enter the page URL

    It is used to resolve and compare the canonical.

  3. Add the X-Robots-Tag header

    Optional: copy its value from the Network tab of the browser developer tools.

  4. Analyze

    Read indexable, followLinks, reasons and warnings in the indexability report.

Use cases

Understand a page missing from Google

Check that no noindex is hiding in the template, an SEO plugin or the server header.

Check filtered pages

Make sure a URL with sorting parameters canonicalises to the main page.

Validate a staging environment

Confirm that a staging site is set to noindex before making it reachable.

Examples

Filtered page set to noindex

The page sorted by price is set to noindex and its canonical points to the version without parameters: it will not be indexed, but its links are still followed. The report gives the reason (noindex) and two warnings: canonical different from the URL entered and missing meta description.

Input

HTML code: <head><title>Chaussures femme</title><meta name="robots" content="noindex, follow"><link rel="canonical" href="/chaussures/femme"></head>
Page URL: https://example.com/chaussures/femme?tri=prix

Result

"indexable": false, "followLinks": true, "canonical": "https://example.com/chaussures/femme", "canonicalMatches": false,
"robots": { "meta": "noindex, follow", "googlebot": null, "xRobotsTag": null }

Blocked by HTTP header

The HTML has no directive, but the X-Robots-Tag header alone excludes the page and stops its links from being followed. This is a common method for PDFs and private documents.

Input

HTML code: <head><title>Facture 2026-114</title><meta name="description" content="…"><link rel="canonical" href="https://example.com/factures/2026-114"></head>
Page URL: https://example.com/factures/2026-114
X-Robots-Tag: noindex, nofollow

Result

"indexable": false, "followLinks": false, "canonicalMatches": true,
"robots": { "meta": null, "googlebot": null, "xRobotsTag": "noindex, nofollow" }

Tips and best practices

  • A page blocked by robots.txt is not crawled, so Google never sees its noindex. To deindex a page, leave crawling open until the noindex has been read.
  • Do not combine noindex with a canonical to another page: the two signals contradict each other. Pick one.
  • A canonical that differs from the URL is normal on a filtered or paginated page; it becomes suspicious on a page you want in the results.
  • To read the X-Robots-Tag header, open the developer tools, go to the Network tab, click the document and look at the response headers.

Sources and references

Frequently asked questions

What is the difference between meta robots and X-Robots-Tag?

The meta robots tag sits in the HTML; the X-Robots-Tag header is sent by the server with the HTTP response. They accept the same directives, but the header also works for non-HTML files such as PDFs and images.

Why doesn’t the tool check robots.txt?

It downloads nothing: it only analyses the HTML and header you provide. To test a robots.txt rule, use the robots.txt report in Search Console or our robots.txt generator.

Does a different canonical block indexing?

No, it is a hint, not a directive. Google often picks the declared canonical but can choose another URL. The tool therefore reports it as a warning, not as a blocking reason.

Does the googlebot tag override robots?

For Google, the two add up and the most restrictive directive applies. The tool does the same: a noindex in either makes the page non-indexable.

Is my code sent to a server?

No. The analysis runs in your browser.

Related tools

Keep going with these tools

Tools similar or complementary to this one.

Similar tools

HTML SEO Analyzer

Analyze key SEO signals in HTML locally.

Web & SEOFeaturedUse this tool

robots.txt generator

Create a clear robots.txt file to guide web crawlers.

Web & SEOFeaturedUse this tool

Read next