Terradev
Terradev is a cross-cloud compute control plane for AI workloads, not just a provisioning wrapper.
Community: Submitted by a user or imported; check the owner before granting accessOnlineNo sign-inGlobalFreeRead-only
What it can do
- Provision Gpu: Provision GPU instances for optimal parallel efficiency
- Preflight Report: Generate full preflight validation report with pass/warn/fail per check. Covers GPU drivers, CUDA, NCCL, RDMA, network, disk, and Docker.
- Preflight Gpu Check: GPU-specific preflight validation: NVIDIA drivers, CUDA version, GPU count, NCCL, NVLink topology, NCU stall-signature profiling, and adversarial config verification (V1-V3).
What data it sees
Do you need an account
No: the server works without sign-in
Terradev is a cross-cloud compute control plane for AI workloads, not just a provisioning wrapper.
Combines quoting, provisioning, topology optimization, training orchestration, inference tuning, and cost analytics in one CLI, with an accelerated idempotent runtime underneath.
Model agnostic. Dataset agnostic. GPU agnostic. Provider agnostic. The only thing Terradev is not agnostic about is correctness: it enforces topology, idempotency, and sequencing.
17 providers are available BYOAPI.
Amazon Web Services, Microsoft Azure, Baseten, Crusoe Cloud, DigitalOcean, E2E Networks, Gcore, Google Cloud Platform, Hugging Face, Hyperstack, InferX, Latitude.sh, RunPod, SiliconFlow, TensorDock, Vast.ai, YottaLabs
Go to one of them and get a token or secrets package. These are stored locally in your secrets vault that can be provisioned through the secrets command.
DISCLAIMER Terradev is an Apache 2.0 open source tool, is BYOAPI, does not charge any markup, and does not have relationships with any providers. Implementers are responsible for what they put in payloads, Terradev doesn't log request content, only public metadata. Usage in production systems is at the implementer's risk.
Server tool list (60)
Raw names from tools/list. Only developers need these.
| provision_gpu | Provision GPU instances for optimal parallel efficiency |
| preflight_report | Generate full preflight validation report with pass/warn/fail per check. Covers GPU drivers, CUDA, NCCL, RDMA, network, disk, and Docker. |
| preflight_gpu_check | GPU-specific preflight validation: NVIDIA drivers, CUDA version, GPU count, NCCL, NVLink topology, NCU stall-signature profiling, and adversarial config verification (V1-V3). |
| preflight_network_check | Network-specific preflight validation: RDMA availability, InfiniBand status, inter-node bandwidth, latency matrix, firewall rules. |
| local_scan | Scan local machine and network for available GPU devices. Returns total VRAM pool for local-first provisioning. |
| status | View all instances and costs with state optimization |
| manage_instance | Manage GPU instances (stop/start/terminate) |
| analytics | Get cost analytics |
| optimize | Find cheaper alternatives for running instances |
| setup_provider | Get setup instructions for a provider |
| configure_provider | Configure provider credentials |
| train | Launch distributed training on provisioned GPU nodes. Supports torchrun, deepspeed, accelerate, and megatron. Use from_provision='latest' to auto-resolve node IPs from your last provision command. |
| preflight | Pre-training validation: GPU availability, NCCL, RDMA, drivers across all nodes. |
| stage | Compress, chunk, checksum, and position datasets near compute. Supports local paths, S3/GCS URIs, HTTP URLs, and HuggingFace dataset names. Returns staging plan with agent recommendations. |
| up | CLI-native provisioning with manifest cache and drift detection. Use --fix-drift to detect and auto-fix drifted infrastructure. |
| rollback | Explicit versioned rollback. Format: job@version (e.g., llama3@v3). |
| manifests | List cached manifests and versions for jobs. |
| run_workflow | Run a declarative YAML workflow that chains multiple Terradev commands (provision → preflight → train → monitor → checkpoint). Returns step-by-step execution status with cost estimates and confirmation gates for expensive operations. |
| active_context | Get current Terradev state: running training jobs, active instances, spend-to-date, alerts. Call this on session start to resume context from previous sessions. |
| governance_request_consent | Request user consent for data movement across cloud regions. GDPR/SOC2 compliant consent tracking with audit trail. |
| governance_record_consent | Record a consent response (granted or denied) for a pending consent request. |
| governance_evaluate_opa | Evaluate OPA (Open Policy Agent) policies for data access. Checks region restrictions, classification rules, and compliance requirements. |
| governance_move_data | Move data with full governance audit trail. Requires prior consent and OPA policy approval. Tracks integrity, encryption, and compliance. |
| governance_movement_history | Get data movement audit log. Filter by user, dataset, or time range. |
| governance_compliance_report | Generate comprehensive compliance report: consent stats, policy evaluations, data movements, violations. For GDPR/SOC2/HIPAA audits. |
| inferx_deploy | Deploy model to InferX serverless platform |
| inferx_status | Check InferX endpoint status |
| inferx_list | List deployed InferX models |
| inferx_optimize | Get cost analysis for inference endpoints |
| inferx_configure | Configure InferX serverless platform credentials. |
| inferx_delete | Delete an InferX model deployment. |
| inferx_usage | Get InferX account usage statistics: requests, cost, GPU hours, latency. |
| inferx_quote | Get InferX pricing quotes for a GPU type. |
| infer_route | Semantic-aware inference routing. Analyzes query content across 6 signal dimensions (modality, complexity, domain, language, safety, keywords), applies NUMA-aware endpoint scoring, and selects the optimal inference endpoint. Uses DAG parallel execution for signal extraction. |
| infer_route_disagg | Disaggregated Prefill/Decode routing (DistServe architecture). Splits LLM inference into compute-bound prefill phase (routed to FLOPS-optimized GPUs like H100 SXM) and memory-bound decode phase (routed to bandwidth-optimized GPUs like MI300X). Tracks KV cache handoffs between endpoint pairs. |
| infer_failover | Run health checks and auto-failover for inference endpoints. If a primary endpoint is unhealthy and has a backup configured, traffic automatically shifts to the backup provider. |
| gpu_topology | GPU NUMA topology report with intra-GPU XCD (Accelerated Compute Die) awareness. Models MI300X (8 XCDs, 192GB HBM3), MI300A (6 XCDs, 128GB), H200 (unified 141GB HBM3e), H100 (80GB). Reports PCIe locality (PIX/PXB/PHB/SYS), GPU-NIC pairing, SR-IOV VF status, and generates XCD-aware NCCL/AITER environment variables. |
| vllm_start | Start a vLLM inference server on a remote instance via SSH/systemd. Supports Multi-LoRA, Sleep Mode, KV Offloading, Speculative Decoding. |
| vllm_stop | Stop a vLLM server on a remote instance. |
| vllm_inference | Test inference against a running vLLM endpoint (completions or chat). |
| vllm_info | Get vLLM server info: loaded models, config, and health status. |
| vllm_sleep | Put a vLLM server to sleep. Level 1: offload to CPU (fast wake). Level 2: discard weights (minimal RAM). |
| vllm_wake | Wake a sleeping vLLM server. For Level 2 sleep, also reloads weights and resets prefix cache. |
| sglang | Complete SGLang optimization stack with workload-specific auto-tuning for 7 workload types: agentic chat, batch inference, low latency, MoE models, PD disaggregated, structured output, and RAG. |
| sglang_start | [DEPRECATED] Use sglang action='start' instead. Start an SGLang inference server on a remote instance. |
| sglang_stop | [DEPRECATED] Use sglang action='stop' instead. Stop an SGLang server on a remote instance. |
| sglang_inference | [DEPRECATED] Use sglang action='inference' instead. Test inference against a running SGLang endpoint. |
| ollama_list | List models available on an Ollama server. |
| ollama_pull | Pull a model to an Ollama server on a remote instance. |
| ollama_generate | Generate text using an Ollama model (non-chat completions). |
| ollama_chat | Chat with an Ollama model using the chat/completions API. |
| ollama_model_info | Get detailed information about an Ollama model (parameters, template, license). |
| vllm_auto_optimize | Automatically optimize vLLM configuration by analyzing workload patterns. Selects optimal settings for the 6 critical knobs based on live endpoint metrics or sample request files. |
| vllm_analyze | Analyze a running vLLM server's workload and return specific optimization recommendations with before/after comparisons. |
| vllm_benchmark | Benchmark a vLLM endpoint with concurrent requests. Returns throughput (req/s), success rate, and total latency. |
| ollama_ps | List currently running Ollama models. |
| langfuse_configure | Configure Langfuse credentials (public key, secret key, host URL). |
| langfuse_test | Test Langfuse connectivity and list accessible projects. |
| langfuse_traces | List recent LLM traces from Langfuse. |
| langfuse_trace | Get a single Langfuse trace with all observations/spans. |