rubrkit

MCP-native AI evaluation: rubric audits, eval suites, and proof reports for AI/LLM output.

CommunityOnlineNo sign-inGlobalFreeRead-only

What it can do

    What data it sees

    Do you need an account

    No: the server works without sign-in

    MCP-native AI evaluation: rubric audits, eval suites, and proof reports for AI/LLM output.

    How to connect

    How to connect rubrkit to Claude

    The server address to paste into Claude settings:

    https://www.rubrkit.com/api/v1/mcp

    1. Open Claude (claude.ai or the desktop app).
    2. Go to Settings → Connectors.
    3. Click Add custom connector.
    4. Paste the server address copied below into Remote MCP server URL and click Add.
    5. In a chat, click + → Connectors and switch the new connector on.

    Custom connectors are available on Free, Pro, Max, Team and Enterprise plans (Free is limited to one). On Team and Enterprise an organization Owner adds the connector first under Organization settings → Connectors.

    Authorization

    After you click Add, a sign-in window for the service opens. Sign in with your own account and approve access. Claude never sees your password.

    How to connect rubrkit to ChatGPT

    The server address to paste into ChatGPT settings:

    https://www.rubrkit.com/api/v1/mcp

    1. Open ChatGPT in a browser (chatgpt.com). A Plus, Pro, Business, Enterprise or Edu plan is required.
    2. Turn on developer mode once: Settings → Apps → Advanced settings → Developer mode.
    3. Open Settings → Connectors and click Create.
    4. Fill in the form: Name (anything), Description (one line about what the service does), MCP server URL (copy it below).
    5. Under Authentication choose OAuth if the service requires sign-in, otherwise None. Click Create.
    6. In a new chat open + → Apps/Connectors and enable the connector.

    OpenAI has renamed this section before (Connectors → Apps/Plugins). If the label differs, search settings for "developer mode". On Business/Enterprise workspaces an admin must allow custom connectors first.

    Authorization

    On first use ChatGPT opens the service's sign-in window. Sign in and approve access.

    How to connect rubrkit to Cursor

    The server address to paste into Cursor settings:

    https://www.rubrkit.com/api/v1/mcp

    Fastest: click Open in Cursor below and confirm the prompt.

    Manually:

    1. In Cursor open Settings → Cursor Settings → MCP and click Add new global MCP server.
    2. Paste the JSON copied below into ~/.cursor/mcp.json (per project: .cursor/mcp.json in the repo root).
    3. Save the file. The server appears in the MCP list; authorize it there if asked.

    Authorization

    If the service needs sign-in, an authorize button appears next to the server in the MCP list.

    Server tool list (41)

    Raw names from tools/list. Only developers need these.

    rubrkit_search_api_docsSearch the canonical docs/api Markdown guides and OpenAPI operation summaries.
    rubrkit_read_api_docRead a docs/api Markdown guide or OpenAPI source by slug, file name, or docs/api path.
    rubrkit_read_openapi_contractRead the canonical docs/api/openapi.yaml contract.
    rubrkit_create_artifact_bundleCreate an artifact bundle through /api/v1/artifact-bundles. Requires artifact_bundles:write for API keys.
    rubrkit_list_artifact_bundlesList artifact bundles through /api/v1/artifact-bundles. Requires artifact_bundles:read for API keys.
    rubrkit_read_artifact_bundleRead artifact bundle details, files, and file tree. Requires artifact_bundles:read for API keys.
    rubrkit_delete_artifact_bundleArchive an artifact bundle (soft delete) through DELETE /api/v1/artifact-bundles/{artifactBundleId}. The bundle moves to the archived state and can be restored or permanently deleted later. Requires artifact_bundles:write for API keys.
    rubrkit_hard_delete_artifact_bundlePermanently delete an archived artifact bundle and all of its files, versions, and run history through DELETE /api/v1/artifact-bundles/{artifactBundleId}/permanent. This is irreversible and only works on bundles that are already archived. Requires artifact_bundles:write for API keys.
    rubrkit_upload_artifact_filesUpload one or more text files into an artifact bundle. Requires files:write for API keys.
    rubrkit_list_artifact_filesList files in an artifact bundle. Requires files:read for API keys.
    rubrkit_read_artifact_fileRead the latest file metadata and content. Requires files:read for API keys.
    rubrkit_update_artifact_fileWrite a new editable version for a file. Requires files:write for API keys.
    rubrkit_start_auditStart an async audit job. Audit results are cached per account by a content hash of their inputs; pass force: true to bypass the cache and run a fresh audit (the same as the CLI's --no-cache flag). Requires audits:run for API keys.
    rubrkit_list_auditsList persisted audit history. Requires artifact_bundles:read for API keys.
    rubrkit_read_auditRead a single audit run. Requires artifact_bundles:read for API keys.
    rubrkit_apply_audit_rewriteApply a completed audit's AI rewrite to the bundle files (a new version per changed file). Requires audits:run for API keys.
    rubrkit_convert_to_rubr_flowStart an async rubr_flow conversion job. Requires rubr_flow:convert for API keys.
    rubrkit_list_rubr_flow_conversionsList conversion history. Requires artifact_bundles:read for API keys.
    rubrkit_read_rubr_flow_conversionRead one conversion report. Requires artifact_bundles:read for API keys.
    rubrkit_poll_jobRead async job progress. Requires jobs:read for API keys.
    rubrkit_list_job_eventsRead append-only job progress events. Requires jobs:read for API keys.
    rubrkit_list_artifact_bundle_versionsList bundle-level version events. Requires artifact_bundles:read for API keys.
    rubrkit_restore_artifact_bundle_versionRestore files from a prior artifact-bundle version by creating new file versions. Requires artifact_bundles:write and files:write for API keys.
    rubrkit_list_file_versionsList immutable versions for a file. Requires files:read for API keys.
    rubrkit_restore_file_versionRestore a prior file version by creating a new version. Requires files:write for API keys.
    rubrkit_run_evalsStart an async eval run. Requires evals:run for API keys. Set mode: "advanced" with baselineVersionId and candidateVersionId to run a statistically-backed A/B (repeats defaults to 3, max 5).
    rubrkit_list_evalsList eval history. Requires artifact_bundles:read for API keys.
    rubrkit_read_evalRead one eval run, including the `stats` statistical verdict on advanced-mode runs (empty for standard runs). Requires artifact_bundles:read for API keys.
    rubrkit_export_proof_reportStart an async proof report job with export storage enabled by default. Requires evals:run for API keys.
    rubrkit_read_proof_reportRead one proof report. Requires artifact_bundles:read for API keys.
    rubrkit_list_golden_casesList the golden cases for an artifact bundle via /api/v1/artifact-bundles/{artifactBundleId}/golden-cases. Requires artifact_bundles:read (or artifacts:pull) for API keys.
    rubrkit_add_golden_caseAdd a golden case (a real edge-case input with a confirmed-correct output) to an artifact bundle via POST /api/v1/artifact-bundles/{artifactBundleId}/golden-cases. Requires artifact_bundles:write for API keys.
    rubrkit_retire_golden_caseRetire a golden case via DELETE /api/v1/golden-cases/{goldenCaseId}. Requires artifact_bundles:write for API keys.
    rubrkit_create_drift_monitorCreate a drift monitor pinned to a file version via POST /api/v1/artifact-bundles/{artifactBundleId}/drift-monitors. Requires artifact_bundles:write for API keys.
    rubrkit_list_drift_monitorsList the drift monitors for an artifact bundle via /api/v1/artifact-bundles/{artifactBundleId}/drift-monitors. Requires artifact_bundles:read (or artifacts:pull) for API keys.
    rubrkit_read_drift_observationsList the recorded observations for a drift monitor via /api/v1/drift-monitors/{driftMonitorId}/observations. Requires artifact_bundles:read (or artifacts:pull) for API keys.
    rubrkit_run_drift_checkQueue a drift check immediately instead of waiting for the monitor's schedule, via POST /api/v1/drift-monitors/{driftMonitorId}/check. The first check on a new monitor calibrates its baseline. Costs one audit per repeat. Requires artifact_bundles:write for API keys.
    rubrkit_set_drift_monitor_statusPause or resume a drift monitor via PATCH /api/v1/drift-monitors/{driftMonitorId}. Requires artifact_bundles:write for API keys.
    rubrkit_record_outcomeRecord a production outcome event (input feature, metric, and/or label) for an artifact bundle via POST /api/v1/artifact-bundles/{artifactBundleId}/outcomes. Requires artifact_bundles:write for API keys.
    rubrkit_read_input_driftCompare recent production inputs against a prior reference window (population stability index) for an artifact bundle via GET /api/v1/artifact-bundles/{artifactBundleId}/drift/input. Requires artifact_bundles:read (or artifacts:pull) for API keys.
    rubrkit_read_validity_driftCorrelate rubric scores against recorded outcome metrics for an artifact bundle to check whether the rubric is still predictive via GET /api/v1/artifact-bundles/{artifactBundleId}/drift/validity. Requires artifact_bundles:read (or artifacts:pull) for API keys.