crw

Web search and web scraping for AI agents in one fast, single-binary MCP server.

CommunityOnlineNo sign-inGlobalFreeRead-only

What it can do

  • Crw Scrape: Scrape one URL to markdown, HTML, or links.
  • Crw Crawl: Start an async site crawl; returns a job id to poll with crw_check_crawl_status.
  • Crw Check Crawl Status: Poll an async crawl job and retrieve its pages.

What data it sees

Do you need an account

No: the server works without sign-in

Web search and web scraping for AI agents in one fast, single-binary MCP server. crw searches the web, scrapes any URL to clean markdown or HTML, maps a site's links, runs bounded recursive crawls, and extracts structured JSON from pages. Built-in anti-bot handling, JavaScript rendering, and proxy rotation recover content that a plain fetch and lighter scrapers miss. Open source (AGPL), local-first, and zero-config via npx, or use this managed cloud endpoint. A leaner, self-hostable alternative to Firecrawl and Tavily for agent web access, docs research, and structured data extraction.

How to connect

How to connect crw to Claude

The server address to paste into Claude settings:

https://crw--fastcrw.run.tools

  1. Open Claude (claude.ai or the desktop app).
  2. Go to Settings → Connectors.
  3. Click Add custom connector.
  4. Paste the server address copied below into Remote MCP server URL and click Add.
  5. In a chat, click + → Connectors and switch the new connector on.

Custom connectors are available on Free, Pro, Max, Team and Enterprise plans (Free is limited to one). On Team and Enterprise an organization Owner adds the connector first under Organization settings → Connectors.

Authorization

After you click Add, a sign-in window for the service opens. Sign in with your own account and approve access. Claude never sees your password.

How to connect crw to ChatGPT

The server address to paste into ChatGPT settings:

https://crw--fastcrw.run.tools

  1. Open ChatGPT in a browser (chatgpt.com). A Plus, Pro, Business, Enterprise or Edu plan is required.
  2. Turn on developer mode once: Settings → Apps → Advanced settings → Developer mode.
  3. Open Settings → Connectors and click Create.
  4. Fill in the form: Name (anything), Description (one line about what the service does), MCP server URL (copy it below).
  5. Under Authentication choose OAuth if the service requires sign-in, otherwise None. Click Create.
  6. In a new chat open + → Apps/Connectors and enable the connector.

OpenAI has renamed this section before (Connectors → Apps/Plugins). If the label differs, search settings for "developer mode". On Business/Enterprise workspaces an admin must allow custom connectors first.

Authorization

On first use ChatGPT opens the service's sign-in window. Sign in and approve access.

How to connect crw to Cursor

The server address to paste into Cursor settings:

https://crw--fastcrw.run.tools

Fastest: click Open in Cursor below and confirm the prompt.

Manually:

  1. In Cursor open Settings → Cursor Settings → MCP and click Add new global MCP server.
  2. Paste the JSON copied below into ~/.cursor/mcp.json (per project: .cursor/mcp.json in the repo root).
  3. Save the file. The server appears in the MCP list; authorize it there if asked.

Authorization

If the service needs sign-in, an authorize button appears next to the server in the MCP list.

Server tool list (7)

Raw names from tools/list. Only developers need these.

crw_scrapeScrape one URL to markdown, HTML, or links.
crw_crawlStart an async site crawl; returns a job id to poll with crw_check_crawl_status.
crw_check_crawl_statusPoll an async crawl job and retrieve its pages.
crw_mapDiscover URLs on a site via sitemap and/or a short crawl. Returns a URL list only, no page content.
crw_extractExtract structured JSON from URLs via a prompt and/or JSON schema. Returns results directly. Needs an LLM.
crw_cancel_extractRequest cancellation of an extract job. Returns the canonical status; cancelling remains non-terminal until the claimed URL settles.
crw_searchSearch the web for current information, news, facts, or docs. Use whenever the answer may depend on up-to-date or external information. Returns ranked results (url/title/description/snippet); optionally scrape each result inline.