agent-web — URL to LLM-ready markdown: a polite, robots-respecting web page reader (free)

URL to clean markdown for LLMs: a polite, robots.txt-respecting web reader.

CommunityOnlineNo sign-inGlobalFreeRead-only

What it can do

    What data it sees

    Do you need an account

    No: the server works without sign-in

    URL to clean markdown for LLMs: a polite, robots.txt-respecting web reader. Free, no API key

    How to connect

    How to connect agent-web — URL to LLM-ready markdown: a polite, robots-respecting web page reader (free) to Claude

    The server address to paste into Claude settings:

    https://agent-web.foomworks.workers.dev/mcp

    1. Open Claude (claude.ai or the desktop app).
    2. Go to Settings → Connectors.
    3. Click Add custom connector.
    4. Paste the server address copied below into Remote MCP server URL and click Add.
    5. In a chat, click + → Connectors and switch the new connector on.

    Custom connectors are available on Free, Pro, Max, Team and Enterprise plans (Free is limited to one). On Team and Enterprise an organization Owner adds the connector first under Organization settings → Connectors.

    Authorization

    After you click Add, a sign-in window for the service opens. Sign in with your own account and approve access. Claude never sees your password.

    How to connect agent-web — URL to LLM-ready markdown: a polite, robots-respecting web page reader (free) to ChatGPT

    The server address to paste into ChatGPT settings:

    https://agent-web.foomworks.workers.dev/mcp

    1. Open ChatGPT in a browser (chatgpt.com). A Plus, Pro, Business, Enterprise or Edu plan is required.
    2. Turn on developer mode once: Settings → Apps → Advanced settings → Developer mode.
    3. Open Settings → Connectors and click Create.
    4. Fill in the form: Name (anything), Description (one line about what the service does), MCP server URL (copy it below).
    5. Under Authentication choose OAuth if the service requires sign-in, otherwise None. Click Create.
    6. In a new chat open + → Apps/Connectors and enable the connector.

    OpenAI has renamed this section before (Connectors → Apps/Plugins). If the label differs, search settings for "developer mode". On Business/Enterprise workspaces an admin must allow custom connectors first.

    Authorization

    On first use ChatGPT opens the service's sign-in window. Sign in and approve access.

    How to connect agent-web — URL to LLM-ready markdown: a polite, robots-respecting web page reader (free) to Cursor

    The server address to paste into Cursor settings:

    https://agent-web.foomworks.workers.dev/mcp

    Fastest: click Open in Cursor below and confirm the prompt.

    Manually:

    1. In Cursor open Settings → Cursor Settings → MCP and click Add new global MCP server.
    2. Paste the JSON copied below into ~/.cursor/mcp.json (per project: .cursor/mcp.json in the repo root).
    3. Save the file. The server appears in the MCP list; authorize it there if asked.

    Authorization

    If the service needs sign-in, an authorize button appears next to the server in the MCP list.

    Server tool list (5)

    Raw names from tools/list. Only developers need these.

    read_urlFetch one publicly reachable URL and return clean, LLM-ready markdown (title + page description + word count + markdown). HTML pages are extracted to markdown — including HTML tables, which become GitHub-flavored Markdown tables; URLs pointing straight at a Markdown or plain-text document (raw READMEs, llms.txt, docs) are passed through verbatim. Polite by design: honors the origin's robots.txt for our user-agent, identifies honestly, read-only GET, never bypasses anti-bot/CAPTCHA/paywalls. Free. JavaScript-rendered pages are not supported yet.
    read_url_previewLike read_url but returns only the page description + the first ~600 characters of the markdown (title + word count + a truncation flag) — a cheap way to check a page's relevance before pulling the full text. Free.
    render_previewFree discovery stub for the screenshot/PDF render lane. Same robots/SSRF guards as read_url. Currently returns a 503 (no charge) until Cloudflare Browser Rendering is provisioned on this account.
    render_screenshotPAID (x402): returns x402 payment instructions for a PNG screenshot of a publicly reachable URL, rendered via a real, robots-respecting headless browser. Use render_preview (free) first. Currently soft-skips 503 until Cloudflare Browser Rendering is provisioned.
    render_pdfPAID (x402): returns x402 payment instructions for a PDF render of a publicly reachable URL, via a real, robots-respecting headless browser. Currently soft-skips 503 until Cloudflare Browser Rendering is provisioned.