numguard

Verify a number before your agent asserts it.

CommunityDegradedNo sign-inGlobalFreeRead-only

What it can do

  • Verify Backtest: Is a strategy's Sharpe real, or the luckiest of many tried? Deflated Sharpe Ratio: pass the observed per-period Sharpe `sr`, sample length `T`, and `n_trials` = how many strategy/para
  • Verify Subset Win: A 'we lead on subset/metric/checkpoint X' claim, corrected for how many you could have picked it from (look-elsewhere / multiple comparisons). Pass the raw p-value and the number of
  • Verify Model Gap: Is the accuracy gap between two models real, or below what the test set can resolve? Pass items-per- model `n` and the two accuracies. Returns the gap, its significance, and the mini

What data it sees

Do you need an account

No: the server works without sign-in

Verify a number before your agent asserts it. Catch inflated backtest(Deflated Sharpe Ratio), false eval "wins" (multiple-testing correction), unreal model gaps, and biased LLM judges — then attach a signed, portable receipt proving it was checked. Generous free tier, then pay-per-call in USDC over x402. Built for agent traders and eval builders.

How to connect

How to connect numguard to Claude

The server address to paste into Claude settings:

https://numguard--ipezygj2.run.tools

  1. Open Claude (claude.ai or the desktop app).
  2. Go to Settings → Connectors.
  3. Click Add custom connector.
  4. Paste the server address copied below into Remote MCP server URL and click Add.
  5. In a chat, click + → Connectors and switch the new connector on.

Custom connectors are available on Free, Pro, Max, Team and Enterprise plans (Free is limited to one). On Team and Enterprise an organization Owner adds the connector first under Organization settings → Connectors.

Authorization

After you click Add, a sign-in window for the service opens. Sign in with your own account and approve access. Claude never sees your password.

How to connect numguard to ChatGPT

The server address to paste into ChatGPT settings:

https://numguard--ipezygj2.run.tools

  1. Open ChatGPT in a browser (chatgpt.com). A Plus, Pro, Business, Enterprise or Edu plan is required.
  2. Turn on developer mode once: Settings → Apps → Advanced settings → Developer mode.
  3. Open Settings → Connectors and click Create.
  4. Fill in the form: Name (anything), Description (one line about what the service does), MCP server URL (copy it below).
  5. Under Authentication choose OAuth if the service requires sign-in, otherwise None. Click Create.
  6. In a new chat open + → Apps/Connectors and enable the connector.

OpenAI has renamed this section before (Connectors → Apps/Plugins). If the label differs, search settings for "developer mode". On Business/Enterprise workspaces an admin must allow custom connectors first.

Authorization

On first use ChatGPT opens the service's sign-in window. Sign in and approve access.

How to connect numguard to Cursor

The server address to paste into Cursor settings:

https://numguard--ipezygj2.run.tools

Fastest: click Open in Cursor below and confirm the prompt.

Manually:

  1. In Cursor open Settings → Cursor Settings → MCP and click Add new global MCP server.
  2. Paste the JSON copied below into ~/.cursor/mcp.json (per project: .cursor/mcp.json in the repo root).
  3. Save the file. The server appears in the MCP list; authorize it there if asked.

Authorization

If the service needs sign-in, an authorize button appears next to the server in the MCP list.

Server tool list (11)

Raw names from tools/list. Only developers need these.

verify_backtestIs a strategy's Sharpe real, or the luckiest of many tried? Deflated Sharpe Ratio: pass the observed per-period Sharpe `sr`, sample length `T`, and `n_trials` = how many strategy/parameter variants were tested before reporting this one. Optionally the return `skew`/`kurt`. For agent traders verifying backtest data before they trust (or publish) it.
verify_subset_winA 'we lead on subset/metric/checkpoint X' claim, corrected for how many you could have picked it from (look-elsewhere / multiple comparisons). Pass the raw p-value and the number of comparisons tested.
verify_model_gapIs the accuracy gap between two models real, or below what the test set can resolve? Pass items-per- model `n` and the two accuracies. Returns the gap, its significance, and the minimum detectable effect.
verify_judge_biasIs an LLM-judge / metric preference real, or just longer/first/same-family? Pass the count of verdicts the tested side won and the total. Exact binomial vs chance.
calibrate_judgeCheck an LLM judge against ground truth on a labelled slice. Pass aligned booleans: the judge's verdicts and the known-correct answers. Returns agreement and whether the judge's errors lean one direction (over-crediting = the length/self-preference failure mode).
audit_leaderboardAudit a whole leaderboard from per-item results. `results` maps each model to the list of item-ids it solved (or a {item: score} dict). Returns rank confidence intervals + whether #1 is statistically real.
issue_receiptVerify a claim server-side and hand back a portable, signed receipt (Ed25519) of numguard's OWN verdict. numguard **recomputes** the verdict from your inputs — it never signs a result you supply — so the receipt is real proof the claim was checked, not just an assertion. Anyone can verify it with only the public key.
verify_guard_traceRecompute the BEHAVIOURAL-guard verdict over an agent's action-trace and hand back a portable, signed receipt (Ed25519) — proof the run was guarded and what the guard decided. Catches what a code scanner can't: a cross-call exfiltration chain (read a secret → later send it to a non- allowlisted host) or an action taken right after ingesting untrusted content (prompt-injection consequence). numguard **recomputes** the verdict from the trace — it never signs a verdict you supply — so the receipt is real evidence, verifiable by anyone with only the public key. Use when: an agent needs to PROVE a run passed the behavioural guard (compliance, audit, handing verified work to another party).
verify_backtest_seriesRun the checks a Deflated-Sharpe pass STILL misses — on the actual returns series. Catches same-bar look-ahead, autocorrelation-inflated Sharpe (HAC), regime dependence / cherry-picked windows, drawdown & tail fantasy, one-lucky-epoch fragility (block bootstrap), overfitting beyond n_trials (PBO), and volatility-clustering. Returns a combined verdict + the checks that flagged. Pass `positions`+`asset_returns` for the leakage check, `turnover` for cost, `candidates` (a matrix) for PBO.
balanceYour remaining free calls and prepaid credit balance.
pricingMachine-readable price list (credits; 1 credit = $0.01) and the free-tier size, so an agent can decide before it calls. Also returns the wallet-native x402 rail an agent can pay at once its free tier is used.