agentcheck
Synthetic checks, nightly regression replay and model-drift alerts for AI agents
Community: Submitted by a user or imported; check the owner before granting accessOnlineAPI key requiredGlobalFreeRead-only
What it can do
What data it sees
Do you need an account
An API key from the service settings is required
Synthetic checks, nightly regression replay and model-drift alerts for AI agents
Server tool list (8)
Raw names from tools/list. Only developers need these.
| agentcheck_get_status | Current status of a public monitored target: overall state, uptime over 24h/7d/30d, last check time, last nightly scores, open incidents and the badge/status URLs. Use the owner (GitHub login) and target slug from the status page URL https://agentwares-agentcheck.vercel.app/<owner>/<slug>. No API key needed. |
| agentcheck_get_pricing | Machine-readable pricing for agentcheck: tiers with monthly USD price, target limits, check interval and features, plus per-run add-ons. Same data as /pricing.json. No API key needed. |
| agentcheck_create_target | Enroll something to monitor: an http endpoint (JSON or OpenAI-style chat), a remote MCP server (Streamable HTTP url) or an A2A agent (origin with /.well-known/agent-card.json). Pass `checks` to create checks in the same call (POST /api/v1/probe proposes three). Returns the target id, the public status page, the badge SVG URL and a README snippet. The first check runs on the next minute tick; call agentcheck_run_now to run immediately. Free tier: 1 target, hourly; Starter+: 5-minute checks. Requires an API key. |
| agentcheck_add_check | Add a check to one of your targets. A check runs an input (http path/prompt, mcp tool call, a2a message) on a schedule and judges the answer with a golden: exact, contains, regex and json_schema cost nothing; rubric and baseline use the LLM judge (baseline = same outcome as the last known-good answer). Returns the check id. Requires an API key. |
| agentcheck_run_now | Run every check of one of your targets immediately (outside the schedule) and return pass/fail per check with latency, judge cost and any incident opened or closed. Use it right after enrolling, or to confirm a fix. Requires an API key. |
| agentcheck_record | Record one production interaction with your agent (the prompt or messages, the final answer, the tools it called, optionally the model) as a trace on a target. Call it from your agent or from a proxy in front of it after each task; promote a good trace with agentcheck_promote_trace to replay it nightly and catch regressions. Requires an API key. |
| agentcheck_promote_trace | Turn an imported or recorded trace into a replayable check whose golden is the recorded outcome (tool sequence + final-answer rubric). The check runs daily and in the nightly replay (Pro). Returns the check. Requires an API key. |
| agentcheck_list_incidents | Open and recent incidents across your targets (or one target): when they opened/closed, the failing check and the cause. Requires an API key. |