Sitemap robots crawlability auditor

Check sitemap URLs against pasted robots rules.

On your device · No account

Privacy details

Sitemap robots crawlability auditor processes your input in this browser. Inputs and results are excluded from analytics. Recent tools save tool names, visit counts and last-visit times. Saved tools store only their names. Draft saving is optional and stays on this device; delete saved text with the draft controls. Loading text from a URL is optional and contacts that server; its URL and your IP address are visible to the server. An optional “Use in” action keeps a handoff in this tab. The next tool removes it on arrival and ignores it if more than a minute has passed.

Copy link includes the inputs you choose to share in the address. Anyone with that link can reopen them. Files and generated results are excluded. Password and key tools share settings only.

Your input

robots.txt for the sitemap origin

Options and local file

Crawler product token

Open a text file on this device

Optional settings apply only to this tab.

Your result

Reset reloads the tool with its starting values and releases open files and device controls. Saved drafts stay on this device until you delete them.

Links include your inputs and settings. Anyone with the link can see them, so leave out private text. Files and generated results are left out; random draws run again.

Keyboard shortcuts

Updated 2026-09-20

Sitemap robots crawlability auditor explained

Paste a sitemap and the robots.txt text for its origin to see which listed URLs the selected crawler may request under those rules. The report identifies the matching rule and duplicates. Specific crawler groups take priority over the wildcard group; equally specific Allow rules win ties.

Next: URL parser

A worked example

Disallow: /private/ blocks /private/draft for the wildcard group. Allow: /private/public$ permits exactly that path when it is the most specific match.

Before you use the result

A local rules check, not a live crawl or indexing verdict. It does not inspect HTTP responses, noindex, canonical tags, JavaScript or access restrictions. Sitemap indexes must be opened as individual child sitemaps first. All input URLs must share one origin; no URL is contacted.

Next: XML formatter

Cite this page

Citation style

ToolOctopus. “Sitemap robots crawlability auditor.” Updated 2026-09-20. https://tooloctopus.com/sitemap-robots-crawlability-auditor.

Add an access date if your instructions require one.

Citation formatting uses citeproc-js by Frank Bennett and Citation Style Language styles. Licence and source code.