agentreliability

Sourced answers on testing, benchmarking and auditing autonomous AI agents — every claim bound to a registered source, with its own confidence and evidence…

CommunityOnlineNo sign-inGlobalFreeRead-only

What it can do

  • Search: Full-text search over the knowledge graph. Matching ignores accents and apostrophes, so query in the user's own words; every hit carries its relevance score and the fields it matched. Use this
  • Get Entity: Fetch one knowledge object by id, with its claims and the sources each claim cites. Use this once search, answer or get_topic has given you an id.
  • Get Topic: List the knowledge objects carrying a tag (topics are content-backed tags). Use this to browse a known topic; use search when you have a question rather than a tag, and get_overview to see

What data it sees

Do you need an account

No: the server works without sign-in

Sourced answers on testing, benchmarking and auditing autonomous AI agents — every claim bound to a registered source, with its own confidence and evidence tier.

Read-only MCP access to the agent-reliability corpus: 38 knowledge objects and 136 claims on agent evals, harnesses, benchmarks and release gates. No claim without a registered source.

How to connect

How to connect agentreliability to Claude

The server address to paste into Claude settings:

https://agentreliability--agentreliability.run.tools

  1. Open Claude (claude.ai or the desktop app).
  2. Go to Settings → Connectors.
  3. Click Add custom connector.
  4. Paste the server address copied below into Remote MCP server URL and click Add.
  5. In a chat, click + → Connectors and switch the new connector on.

Custom connectors are available on Free, Pro, Max, Team and Enterprise plans (Free is limited to one). On Team and Enterprise an organization Owner adds the connector first under Organization settings → Connectors.

Authorization

After you click Add, a sign-in window for the service opens. Sign in with your own account and approve access. Claude never sees your password.

How to connect agentreliability to ChatGPT

The server address to paste into ChatGPT settings:

https://agentreliability--agentreliability.run.tools

  1. Open ChatGPT in a browser (chatgpt.com). A Plus, Pro, Business, Enterprise or Edu plan is required.
  2. Turn on developer mode once: Settings → Apps → Advanced settings → Developer mode.
  3. Open Settings → Connectors and click Create.
  4. Fill in the form: Name (anything), Description (one line about what the service does), MCP server URL (copy it below).
  5. Under Authentication choose OAuth if the service requires sign-in, otherwise None. Click Create.
  6. In a new chat open + → Apps/Connectors and enable the connector.

OpenAI has renamed this section before (Connectors → Apps/Plugins). If the label differs, search settings for "developer mode". On Business/Enterprise workspaces an admin must allow custom connectors first.

Authorization

On first use ChatGPT opens the service's sign-in window. Sign in and approve access.

How to connect agentreliability to Cursor

The server address to paste into Cursor settings:

https://agentreliability--agentreliability.run.tools

Fastest: click Open in Cursor below and confirm the prompt.

Manually:

  1. In Cursor open Settings → Cursor Settings → MCP and click Add new global MCP server.
  2. Paste the JSON copied below into ~/.cursor/mcp.json (per project: .cursor/mcp.json in the repo root).
  3. Save the file. The server appears in the MCP list; authorize it there if asked.

Authorization

If the service needs sign-in, an authorize button appears next to the server in the MCP list.

Server tool list (8)

Raw names from tools/list. Only developers need these.

searchFull-text search over the knowledge graph. Matching ignores accents and apostrophes, so query in the user's own words; every hit carries its relevance score and the fields it matched. Use this whenever you have a question rather than an id, then follow up with get_entity.
get_entityFetch one knowledge object by id, with its claims and the sources each claim cites. Use this once search, answer or get_topic has given you an id.
get_topicList the knowledge objects carrying a tag (topics are content-backed tags). Use this to browse a known topic; use search when you have a question rather than a tag, and get_overview to see which tags exist.
answerAnswer a question from the corpus. Returns the matched object's claims with sources and confidence — never an unsourced answer. Use this when the user asked a question in words; use search when you want to see the candidates yourself.
get_sourcesThe instance's whole source registry, or just the sources cited by one object. Use this to check what a claim rests on, or to judge the corpus before trusting it.
get_relatedGraph neighbours of an object: outgoing and incoming relations, each with its relation type. Use this after get_entity to widen an answer with adjacent objects.
get_latestMost recently verified knowledge objects (freshness signal). Use this to judge how current the corpus is, or to see what changed since you last read it.
get_overviewCorpus overview: what this instance knows, counts by type, published tags, freshness. Use this first when you land here and do not yet know whether this corpus can answer your question.