搜尋引擎與 AI 爬蟲工具箱

Name: 搜尋引擎與 AI 爬蟲工具箱
Author: tw-search-seo

檢查網站爬蟲設定、驗證爬蟲 IP、IndexNow 提交與 YouTube SEO 檢查。

CommunityOnlineNo sign-inGlobalFreeRead-only

What it can do

What data it sees

Do you need an account

No: the server works without sign-in

檢查網站爬蟲設定、驗證爬蟲 IP、IndexNow 提交與 YouTube SEO 檢查。

How to connect

How to connect 搜尋引擎與 AI 爬蟲工具箱 to Claude

The server address to paste into Claude settings:

https://tw-search-seo.panda198271.workers.dev/mcp

Open Claude (claude.ai or the desktop app).
Go to Settings → Connectors.
Click Add custom connector.
Paste the server address copied below into Remote MCP server URL and click Add.
In a chat, click + → Connectors and switch the new connector on.

Custom connectors are available on Free, Pro, Max, Team and Enterprise plans (Free is limited to one). On Team and Enterprise an organization Owner adds the connector first under Organization settings → Connectors.

Authorization

After you click Add, a sign-in window for the service opens. Sign in with your own account and approve access. Claude never sees your password.

How to connect 搜尋引擎與 AI 爬蟲工具箱 to ChatGPT

The server address to paste into ChatGPT settings:

https://tw-search-seo.panda198271.workers.dev/mcp

Open ChatGPT in a browser (chatgpt.com). A Plus, Pro, Business, Enterprise or Edu plan is required.
Turn on developer mode once: Settings → Apps → Advanced settings → Developer mode.
Open Settings → Connectors and click Create.
Fill in the form: Name (anything), Description (one line about what the service does), MCP server URL (copy it below).
Under Authentication choose OAuth if the service requires sign-in, otherwise None. Click Create.
In a new chat open + → Apps/Connectors and enable the connector.

OpenAI has renamed this section before (Connectors → Apps/Plugins). If the label differs, search settings for "developer mode". On Business/Enterprise workspaces an admin must allow custom connectors first.

Authorization

On first use ChatGPT opens the service's sign-in window. Sign in and approve access.

How to connect 搜尋引擎與 AI 爬蟲工具箱 to Cursor

The server address to paste into Cursor settings:

https://tw-search-seo.panda198271.workers.dev/mcp

Fastest: click Open in Cursor below and confirm the prompt.

Manually:

In Cursor open Settings → Cursor Settings → MCP and click Add new global MCP server.
Paste the JSON copied below into ~/.cursor/mcp.json (per project: .cursor/mcp.json in the repo root).
Save the file. The server appears in the MCP list; authorize it there if asked.

Authorization

If the service needs sign-in, an authorize button appears next to the server in the MCP list.

Server tool list (14)

Raw names from tools/list. Only developers need these.

build_search_urls	一次產生 Google、Bing、Yahoo 奇摩、百度、YouTube 的進階搜尋網址（關鍵字、完全符合、排除字、限定網站、檔案類型、時間）。
search_operators	搜尋指令（site:、filetype:、intitle:、before: 等）在 Google、Bing、Yahoo、百度的支援對照與已失效指令。
generate_robots_txt	產生 robots.txt：allow_all、block_training（封鎖 AI 訓練保留搜尋）、block_all_ai、block_all，可加不抓路徑與 Sitemap。
test_robots_txt	依 RFC 9309 測試某個 User-Agent 能否抓取某路徑。
check_site_crawlers	檢查網站的 robots.txt、Sitemap、llms.txt，以及 Googlebot、bingbot、GPTBot、ClaudeBot、PerplexityBot 等是否被封鎖，附改善建議。
ai_crawler_list	AI 與搜尋引擎爬蟲 User-Agent 名單：業者、用途、是否遵守 robots.txt、官方 IP 清單。
verify_crawler_ip	驗證 IP 是否真的來自 Googlebot、bingbot、GPTBot、ClaudeBot、PerplexityBot、Applebot、Baiduspider 等（官方 IP 清單＋反查 DNS）。
serp_preview	預覽 Google 搜尋結果標題與描述，估算中文像素寬度與截斷位置。
generate_meta_tags	產生 title、description、canonical、Open Graph、Twitter 卡片與 JSON-LD。
youtube_seo_check	檢查 YouTube 章節時間碼（0:00 起、至少 3 段、每段 10 秒）與標題、說明、標籤、主題標籤數量上限。
indexnow_submit	用 IndexNow 通知 Bing、Yandex、Naver 等搜尋引擎網址已更新（需先在網站放金鑰檔，最多 100 個網址）。
search_engine_market_share	台灣與全球搜尋引擎市佔率（Google、Bing、Yahoo、百度等，StatCounter 最新月份）。
search_submit_guide	如何讓 Google、Bing、Yahoo、百度、YouTube、AI 搜尋收錄網站：工具、步驟與 API。
geo_citation_audit	AEO／GEO 引用準備度健檢：檢查網頁能否被 ChatGPT、Perplexity、Google AI 摘要等答案引擎引用（爬蟲可讀、開頭直接回答、問句標題、表格、外部與權威來源、更新日期、JSON-LD citation、llms.txt、Markdown 版、Content-Digest），給分數與優先修正清單。