Crawlie

Technical SEO + AI-search audits for agents.

От сообществаРаботаетБез входаГлобальныйБесплатноМожет изменять данные

Что умеет

  • Crawl Site: Run a hosted technical-SEO + GEO audit of a whole website in Crawlie Cloud. Returns a compact digest (headline, scores, prioritized top fixes, issues grouped by rule with sample URLs) plus
  • Audit Url: Audit a single URL in Crawlie Cloud (no crawling beyond the page). Returns the compact SEO + GEO digest and a reportId. Counts as one crawl against your plan.
  • Audit Urls: Audit an explicit list of URLs in Crawlie Cloud (no link following). One report for the whole batch — use for spot-checking a set of pages. Each page audited counts against your plan's cre

Какие данные видит

Нужен ли аккаунт

Не нужен: сервер работает без входа

Technical SEO + AI-search audits for agents. Crawl any website and get a structured report your agent can actually work with — broken links, redirect chains, missing titles/canonicals, duplicate content, structured data, and GEO (generative engine optimization) readiness: is your site answerable by ChatGPT, Claude, and Perplexity?

What your agent can do

  • Audit a single URL, a list, or crawl an entire site
  • Slice saved reports without re-crawling — top fixes by impact, affected URLs per issue, link graph, GEO gaps
  • Explain any issue — why it matters and how to fix it, with agent-actionable output
  • Monitor continuously — create a project, set a crawl schedule, diff reports to catch regressions after deploys
  • Gate releases — run a pre-launch check and fail on broken links, 5xx, or noindex slips

Try asking

  • "Crawl mysite.com and give me the top 10 fixes by impact"
  • "What changed since the last crawl of my docs site?"
  • "Is my site ready to be cited by AI search engines?"

Sign in with OAuth — no API key to paste. Free to start. Also available as an open-source desktop crawler at crawlie.co.

Список инструментов сервера (18)

Технические названия из tools/list. Нужны только разработчикам.

crawl_siteRun a hosted technical-SEO + GEO audit of a whole website in Crawlie Cloud. Returns a compact digest (headline, scores, prioritized top fixes, issues grouped by rule with sample URLs) plus a reportId you can re-slice later without re-crawling. Long crawls stream progress; if one exceeds the request window you get a jobId to poll with crawl_status. Counts as one crawl against your plan.
audit_urlAudit a single URL in Crawlie Cloud (no crawling beyond the page). Returns the compact SEO + GEO digest and a reportId. Counts as one crawl against your plan.
audit_urlsAudit an explicit list of URLs in Crawlie Cloud (no link following). One report for the whole batch — use for spot-checking a set of pages. Each page audited counts against your plan's credits.
crawl_statusPoll a crawl started by crawl_site that exceeded the request window. Returns progress while running, or the full digest + reportId once done.
list_reportsList saved hosted reports for your team (id, url, scores, page counts).
get_reportLoad a saved hosted report. Compact digest by default; set includeIssues/includePages for the full flat lists.
top_fixesPrioritized fixes for a saved report, optionally scoped to a category (e.g. category='geo'). Operates on the latest report by default.
geo_gapsAI-readiness (GEO) issues for a saved report, grouped by rule with sample URLs. Latest report by default.
affected_urlsList URLs flagged by a specific rule in a saved report (e.g. rule='geo-no-author').
diff_reportsCompare two saved crawls of the same site: score deltas, pages added/removed, and issues that newly appeared or were resolved. Use it to verify fixes landed between crawls.
link_graphInternal-link analytics for a saved report: top authority pages (internal PageRank), biggest hubs, orphan pages, dead ends, max click depth, and reciprocal-link counts. Latest report by default.
explain_issueWhy a Crawlie rule matters, how to fix it, and its impact — e.g. rule='geo-no-llms-txt'. Use list_rules to browse rule ids.
list_rulesThe catalog of built-in SEO/GEO/accessibility checks Crawlie runs (rule id, title, category, severity), optionally filtered by category. Use explain_issue for the full why/how-to-fix text.
whoamiThe signed-in Crawlie Cloud account: email, active team, plan limits, and this period's crawl/credit usage.
list_projectsSaved sites (projects) for your team, with their crawl schedule, alert settings, last scores and next run time.
create_projectSave a site as a project so it can be re-crawled on a schedule with regression alerts. Set the schedule here or later with set_schedule. Scheduling requires a paid plan; the project itself is free.
set_scheduleUpdate a project's re-crawl schedule and regression alerts (email and/or a Slack/Discord-compatible webhook). Alerts fire when a scheduled crawl's health drops or new error-severity issues appear. Scheduling requires a paid plan.
project_trendScore history for a monitored project: health/GEO/accessibility, errors, warnings and page counts per crawl, oldest first. Use it to see whether a site is improving.