numguard
Verify a number before your agent asserts it.
Community: Submitted by a user or imported; check the owner before granting accessOnlineNo sign-inGlobalFreeRead-only
What it can do
- Verify Backtest: Is a strategy's Sharpe real, or the luckiest of many tried? Deflated Sharpe Ratio: pass the observed per-period Sharpe `sr`, sample length `T`, and `n_trials` = how many strategy/para
- Verify Subset Win: A 'we lead on subset/metric/checkpoint X' claim, corrected for how many you could have picked it from (look-elsewhere / multiple comparisons). Pass the raw p-value and the number of
- Verify Model Gap: Is the accuracy gap between two models real, or below what the test set can resolve? Pass items-per- model `n` and the two accuracies. Returns the gap, its significance, and the mini
What data it sees
Do you need an account
No: the server works without sign-in
Verify a number before your agent asserts it. Catch inflated backtest(Deflated Sharpe Ratio), false eval "wins" (multiple-testing correction), unreal model gaps, and biased LLM judges — then attach a signed, portable receipt proving it was checked. Generous free tier, then pay-per-call in USDC over x402. Built for agent traders and eval builders.
Server tool list (11)
Raw names from tools/list. Only developers need these.
| verify_backtest | Is a strategy's Sharpe real, or the luckiest of many tried? Deflated Sharpe Ratio: pass the observed per-period Sharpe `sr`, sample length `T`, and `n_trials` = how many strategy/parameter variants were tested before reporting this one. Optionally the return `skew`/`kurt`. For agent traders verifying backtest data before they trust (or publish) it. |
| verify_subset_win | A 'we lead on subset/metric/checkpoint X' claim, corrected for how many you could have picked it from (look-elsewhere / multiple comparisons). Pass the raw p-value and the number of comparisons tested. |
| verify_model_gap | Is the accuracy gap between two models real, or below what the test set can resolve? Pass items-per- model `n` and the two accuracies. Returns the gap, its significance, and the minimum detectable effect. |
| verify_judge_bias | Is an LLM-judge / metric preference real, or just longer/first/same-family? Pass the count of verdicts the tested side won and the total. Exact binomial vs chance. |
| calibrate_judge | Check an LLM judge against ground truth on a labelled slice. Pass aligned booleans: the judge's verdicts and the known-correct answers. Returns agreement and whether the judge's errors lean one direction (over-crediting = the length/self-preference failure mode). |
| audit_leaderboard | Audit a whole leaderboard from per-item results. `results` maps each model to the list of item-ids it solved (or a {item: score} dict). Returns rank confidence intervals + whether #1 is statistically real. |
| issue_receipt | Verify a claim server-side and hand back a portable, signed receipt (Ed25519) of numguard's OWN verdict. numguard **recomputes** the verdict from your inputs — it never signs a result you supply — so the receipt is real proof the claim was checked, not just an assertion. Anyone can verify it with only the public key. |
| verify_guard_trace | Recompute the BEHAVIOURAL-guard verdict over an agent's action-trace and hand back a portable, signed receipt (Ed25519) — proof the run was guarded and what the guard decided. Catches what a code scanner can't: a cross-call exfiltration chain (read a secret → later send it to a non- allowlisted host) or an action taken right after ingesting untrusted content (prompt-injection consequence). numguard **recomputes** the verdict from the trace — it never signs a verdict you supply — so the receipt is real evidence, verifiable by anyone with only the public key. Use when: an agent needs to PROVE a run passed the behavioural guard (compliance, audit, handing verified work to another party). |
| verify_backtest_series | Run the checks a Deflated-Sharpe pass STILL misses — on the actual returns series. Catches same-bar look-ahead, autocorrelation-inflated Sharpe (HAC), regime dependence / cherry-picked windows, drawdown & tail fantasy, one-lucky-epoch fragility (block bootstrap), overfitting beyond n_trials (PBO), and volatility-clustering. Returns a combined verdict + the checks that flagged. Pass `positions`+`asset_returns` for the leakage check, `turnover` for cost, `candidates` (a matrix) for PBO. |
| balance | Your remaining free calls and prepaid credit balance. |
| pricing | Machine-readable price list (credits; 1 credit = $0.01) and the free-tier size, so an agent can decide before it calls. Also returns the wallet-native x402 rail an agent can pay at once its free tier is used. |