SystemIntelPilot
Enterprise AI operator evaluation MCP server.
Community: Submitted by a user or imported; check the owner before granting accessOnlineNo sign-inGlobalFreeCan modify data
What it can do
- Get Pilot Status: Get pilot status overview — cohort size, observation count, date range, data quality, active interventions. Computed from raw observations. Data is from a 50-operator synthetic pilot
- Get Operator Profile: Get operator profile — operator details, measurements (5 canonical metrics computed from raw token observations with values, percentiles, status), and benchmark availability. Ope
- Get Cohort Distribution: Get cohort metric distribution — min, p10, p25, median, p75, p90, max, mean, std, and outliers for a given metric across the 50-operator cohort. Computed from raw observations
What data it sees
Do you need an account
No: the server works without sign-in
Enterprise AI operator evaluation MCP server. 27 tools (22 read + 5 write) for measuring, benchmarking, diagnosing, and interv ening on how human operators use AI tools across 5 canonical metrics.
Server tool list (27)
Raw names from tools/list. Only developers need these.
| get_pilot_status | Get pilot status overview — cohort size, observation count, date range, data quality, active interventions. Computed from raw observations. Data is from a 50-operator synthetic pilot (labeled synthetic). |
| get_operator_profile | Get operator profile — operator details, measurements (5 canonical metrics computed from raw token observations with values, percentiles, status), and benchmark availability. Operator IDs are pseudonymous (e.g., op_001). Data is synthetic. |
| get_cohort_distribution | Get cohort metric distribution — min, p10, p25, median, p75, p90, max, mean, std, and outliers for a given metric across the 50-operator cohort. Computed from raw observations. |
| get_composite_score | Get developmental composite score (0-100) for an operator. Computed from raw metrics normalized via reference percentiles. Labeled DEVELOPMENTAL, not PERSONNEL. Weighted: leverage 30%, yield 30%, token_snr 20%, construction 20%. Data is synthetic. |
| get_composite_score_summary | Get cohort composite score summary — count, min, max, median, mean, Q1, Q3. Computed from per-operator scores. No individual rankings exposed. Label is DEVELOPMENTAL. |
| get_diagnostics | Get operator diagnostics — pattern detections and diagnoses computed from divergence analysis. All diagnoses are HYPOTHESIS, never fact. |
| get_data_quality | Get data quality summary — completeness, coverage, validity across the cohort. Computed from raw observations. |
| find_usage_operation_divergence | Find operators with usage-operation divergence. Computes usage percentile from raw token totals and compares to yield percentile. Returns all 50 operators with divergence class (LOW_USAGE_HIGH_OPERATION, HIGH_USAGE_LOW_OPERATION, etc.). |
| get_workflow_fit | Get workflow fit analysis — operator/workflow fit scores across workflow stages. |
| get_intervention_status | Get all interventions — 12 active interventions with operator IDs, catalog IDs, reason patterns, target metrics, start dates, followup periods, and synthetic outcomes. |
| list_pilot_options | List available pilot options — 5 canonical metrics, 15 eval families, 13 benchmark classes, 5 intervention types. |
| validate_pilot_configuration | Validate a pilot configuration before deployment. Returns valid status with warnings and errors. |
| compare_operator_to_reference | Compare an operator to a reference population. Returns benchmark selection, comparison group, and metric comparison. Computed from raw metrics and reference field. |
| get_executive_dashboard | Get executive dashboard info — the dashboard is a self-contained HTML file generated by the CLI (enterprise export dashboard --output file.html). |
| verify_change | Verify a measured change after intervention — pre/post comparison. Results are ASSOCIATION, never CAUSATION. |
| create_pilot_configuration | Generate a pilot configuration from parameters. |
| assign_intervention | Assign a targeted intervention to an operator. REQUIRES AUTHORIZATION. Contact pilots@mos2es.org for pilot access. |
| close_intervention | Close an intervention with outcome notes. REQUIRES AUTHORIZATION. |
| create_experiment | Create an experiment configuration. REQUIRES AUTHORIZATION. |
| record_workflow_observation | Record a workflow fit observation. REQUIRES AUTHORIZATION. |
| attach_outcome_dataset | Attach external outcome dataset for join analysis. Outcome joins are ASSOCIATION, never CAUSATION. REQUIRES AUTHORIZATION. |
| get_operator_system_decomposition | Two-way ANOVA-style decomposition partitioning metric variance into operator effect, system effect, and operator×system interaction. Computed from raw observations grouped by platform. Shows whether operator capability or system choice drives performance. |
| get_lineage_chain | Get the full lineage chain for an operator: STATE_A → BI_ACTION → AAI_TRANSFORMATION → BI_REDIRECTION → AAI_EXTENSION → COMMITTED_STATE → OUTCOME. Built from raw lineage and outcome data. |
| get_lineage_summary | Get lineage summary across the cohort — total lineages, workflow breakdown, average micro-eval metrics, outcomes linked. Computed from raw lineage data. |
| get_outcome_correlation | Correlate micro-eval metrics with outcome quality scores and cycle times through lineage. Computed via Pearson r from raw lineage + outcome data. Results labeled ASSOCIATION with evidence grade OBSERVATIONAL, never CAUSATION. |
| get_org_topology | Organization-level AI topology map — team-level metric distributions, median canonical metrics per team, capability concentration (Gini coefficient), platform adoption, single-point-of-failure detection, cross-team complementarity. Computed from raw measurements. |
| get_operator_similarity | Nearest-neighbor operator search using percentile-rank normalization and Euclidean distance across 5 canonical metrics. Computed from raw measurements. Returns comparable operators/cohorts, NOT personality matching. |