robots-txt-checker
Checks if a URL path is allowed by robots.txt.
Community: Submitted by a user or imported; check the owner before granting accessOnlineNo sign-inGlobalFreeRead-only
What it can do
What data it sees
Do you need an account
No: the server works without sign-in
Checks if a URL path is allowed by robots.txt. x402 payment required (testnet USDC).
Server tool list (1)
Raw names from tools/list. Only developers need these.
| robots_txt_checker | Checks if a URL path is allowed by robots.txt. x402 payment required (Base mainnet USDC). |