AI Crawler Checker — Robots.txt Policy for Six Bots
Check declared robots.txt permissions for six AI crawlers at URL paths
Description as published in the official MCP registry.
Maintain this server? Get the badge
What we measured
| Endpoint | https://mcp.apify.com/?tools=hermeskd/ai-crawler-access-check | registry |
|---|---|---|
| Website | apify.com/hermeskd/ai-crawler-access-check | registry |
| Handshake | Answers the MCP handshake; tool list not available without sign-in (10 ms) | our check, 2026-11-17 |
| Protocol version | 2026-07-28 | our check, 2026-11-17 |
| Registry version | 1.0.2 · active · updated 2026-11-03 | registry |
Signed reports
No signed reports yet. A report carries a signed decision record from the reporter's gate, so it shows that the gate allowed a call, not just an opinion; it does not prove which server answered. How to file one.
Connect
claude mcp add --transport http ai-crawler-access-check https://mcp.apify.com/?tools=hermeskd/ai-crawler-access-checkCommands are built from the registry entry. Check the publisher's documentation before giving any server access to your data.
Related servers
- Robots.txt & AI Crawler Checker robots.txt tester (RFC 9309), AI crawler access (GPTBot, ClaudeBot...) and sitemap check.
- ActableSite AI Crawler Monitor Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt…
- Robots.txt Policy Lint {"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}}…
- Broken Links Checker — Crawl Pages, Assets & Redirect Chains Find broken links on a website: crawls its pages and reports 404s, errors and redirect…
- Hyades Render LaTeX math as pure Unicode text art for terminals, code comments, and emails.
- Decant: convert CSV, XLSX and PDF to JSON; web page extraction that honours robots.txt Hosted MCP: CSV/XLSX to JSON rows and text-PDF pages; private uploads or URLs, no API key.
Is this your server?
Show the result of our latest check where your users look first. The badge is a live image: it changes when our check does, and it links back to this page.
[](https://openforallofus.com/tools/io.github.kriton-dev/ai-crawler-access-check)<a href="https://openforallofus.com/tools/io.github.kriton-dev/ai-crawler-access-check"><img src="https://openforallofus.com/badge/io.github.kriton-dev/ai-crawler-access-check.svg" alt="Agora: MCP handshake check" height="20"></a>The badge reports one thing: the result of our last handshake check. A server with no remote endpoint shows as "runs locally". How we check.
Agents can read this page as data: MCP endpoint https://openforallofus.com/api/mcp, tool get_tool with name io.github.kriton-dev/ai-crawler-access-check.