# Agentic AI & Agent Runtime

> Warp's coding-agent surface: the ai crate (agent orchestration, actions, skills, indexing), the app-side agent conversation and management models, MCP tool transport, and the computer_use automation layer. Stays provider-neutral — built-in or bring-your-own CLI agents.

- Repository: warpdotdev/warp
- GitHub: https://github.com/warpdotdev/warp
- Human wiki: https://grok-wiki.com/public/wiki/warpdotdev-warp-a55b6d0c09b5
- Complete Markdown: https://grok-wiki.com/public/wiki/warpdotdev-warp-a55b6d0c09b5/llms-full.txt

## Source Files

- `crates/ai/src/lib.rs`
- `crates/ai/src/agent/mod.rs`
- `crates/ai/src/agent/orchestration_config.rs`
- `crates/mcp/src/lib.rs`
- `crates/computer_use/src/lib.rs`
- `app/src/ai/agent_conversations_model.rs`

---

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:
- [crates/ai/src/lib.rs](crates/ai/src/lib.rs)
- [crates/ai/src/agent/mod.rs](crates/ai/src/agent/mod.rs)
- [crates/ai/src/agent/action/mod.rs](crates/ai/src/agent/action/mod.rs)
- [crates/ai/src/agent/orchestration_config.rs](crates/ai/src/agent/orchestration_config.rs)
- [crates/ai/src/skills/mod.rs](crates/ai/src/skills/mod.rs)
- [crates/ai/src/skills/skill_provider.rs](crates/ai/src/skills/skill_provider.rs)
- [crates/ai/src/index/mod.rs](crates/ai/src/index/mod.rs)
- [crates/mcp/src/lib.rs](crates/mcp/src/lib.rs)
- [crates/mcp/src/sse_transport/mod.rs](crates/mcp/src/sse_transport/mod.rs)
- [crates/computer_use/src/lib.rs](crates/computer_use/src/lib.rs)
- [app/src/ai/agent_conversations_model.rs](app/src/ai/agent_conversations_model.rs)
- [app/src/ai/agent_conversations_model/entry.rs](app/src/ai/agent_conversations_model/entry.rs)
</details>

# Agentic AI & Agent Runtime

Warp's coding-agent surface is split across several Rust crates and an app-side model layer. The `ai` crate owns the provider-neutral vocabulary of what an agent can *do* — its action types, action results, orchestration configuration, skills, and codebase index. The `app` layer owns the *management* of agent conversations and tasks (local interactive, ambient, and cloud-synced). Two supporting crates handle external surfaces: `mcp` provides a legacy SSE transport for connecting to Model Context Protocol servers, and `computer_use` provides a cross-platform screen/keyboard/mouse automation layer used by the agent's computer-use actions.

A defining property of this design is **harness neutrality**: the agent runtime is not bound to a single model provider or CLI. Orchestration and child-agent launches carry a `harness_type` string that selects between Warp's own harness (`oz`) and bring-your-own CLI agents (`claude`, `opencode`, `gemini`, `codex`), and skills are discovered from many vendor-specific directories. This page maps those modules, their data contracts, and how a tool call flows from the agent model through to local execution or an external service.

## Architecture at a Glance

The runtime separates the *action vocabulary* (in `crates/ai`) from *execution surfaces* (MCP servers, the local computer, child harnesses) and from *conversation management* (in `app`).

```mermaid
flowchart TB
    subgraph app["app/src/ai (management layer)"]
        ACM["AgentConversationsModel\n(polls + caches tasks)"]
        ENTRY["AgentConversationEntry\nprovenance + harness + capabilities"]
    end
    subgraph ai["crates/ai (action vocabulary)"]
        ACT["AIAgentActionType\n(tool-call catalog)"]
        ORCH["OrchestrationConfig\nmodel_id + harness_type + mode"]
        SKILL["skills::SkillProvider\nmulti-vendor discovery"]
        INDEX["index\n(file_outline / embeddings)"]
    end
    subgraph surfaces["Execution surfaces"]
        MCP["crates/mcp\nSSE transport to MCP servers"]
        CU["crates/computer_use\nActor: mouse/keyboard/screenshot"]
        HARNESS["Child harnesses\noz / claude / opencode / gemini / codex"]
    end
    ACM --> ENTRY
    ACM -.reads.-> ORCH
    ACT -->|CallMCPTool / ReadMCPResource| MCP
    ACT -->|UseComputer / RequestComputerUse| CU
    ACT -->|RunAgents / StartAgent| HARNESS
    ORCH -->|matches_active_config| ACT
    ACT -.SearchCodebase.-> INDEX
    ACT -.ReadSkill.-> SKILL
```

Sources: [crates/ai/src/lib.rs:1-16](crates/ai/src/lib.rs), [crates/ai/src/agent/mod.rs:1-10](crates/ai/src/agent/mod.rs), [crates/ai/src/agent/action/mod.rs:30-175](crates/ai/src/agent/action/mod.rs), [app/src/ai/agent_conversations_model.rs:508-535](app/src/ai/agent_conversations_model.rs)

## The `ai` Crate

`crates/ai/src/lib.rs` is the crate root and declares the public module surface: `agent`, `skills`, `index`, `document`, `project_context`, `workspace`, plus credential helpers (`api_keys`, `aws_credentials`) and `llm_id` (Sources: [crates/ai/src/lib.rs:1-16](crates/ai/src/lib.rs)). The `agent` module re-exports the action and result types, citations, and file-location grouping, and owns `orchestration_config` (Sources: [crates/ai/src/agent/mod.rs:1-10](crates/ai/src/agent/mod.rs)).

### Action Catalog (`AIAgentActionType`)

`AIAgentActionType` is the central enum describing every tool call an agent can request. It is provider-agnostic — it describes *intent* (read files, run a command, call an MCP tool, edit documents, drive the computer, orchestrate child agents), not how any specific model encodes those calls. Each variant has matching helpers: `cancelled_result` maps a variant to its cancelled `AIAgentActionResultType`, and `user_friendly_name`/`Display` provide human- and log-facing labels.

| Action variant | Surface / responsibility |
|---|---|
| `RequestCommandOutput`, `WriteToLongRunningShellCommand`, `ReadShellCommandOutput` | Terminal/PTY execution with read-only/risky/pager hints |
| `ReadFiles`, `RequestFileEdits`, `Grep`, `FileGlob` / `FileGlobV2` | Local filesystem access and edits |
| `SearchCodebase` | Semantic/code search backed by the `index` module |
| `CallMCPTool`, `ReadMCPResource` | MCP server tools/resources (optional `server_id`) |
| `UseComputer`, `RequestComputerUse` | `computer_use` automation (actions + screenshot params) |
| `ReadSkill` | Loads a `SkillReference` from a skill provider |
| `ReadDocuments`, `EditDocuments`, `CreateDocuments` | Agent-managed documents |
| `StartAgent`, `SendMessageToAgent`, `RunAgents` | Child-agent orchestration |
| `AskUserQuestion`, `SuggestPrompt`, `TransferShellCommandControlToUser` | Human-in-the-loop interactions |

Sources: [crates/ai/src/agent/action/mod.rs:30-175](crates/ai/src/agent/action/mod.rs), [crates/ai/src/agent/action/mod.rs:304-435](crates/ai/src/agent/action/mod.rs)

The `WriteToLongRunningShellCommand` path encodes how input is delivered to an interactive PTY via `AIAgentPtyWriteMode` (`Raw`, `Line`, `Block`); `Line` mode prepends `^A` (beginning-of-line) and submits with platform-aware `CR`/`LF`, and `Block` mode wraps the bytes in bracketed-paste markers when enabled (Sources: [crates/ai/src/agent/action/mod.rs:769-820](crates/ai/src/agent/action/mod.rs)).

### Orchestration & Harness Neutrality

When an agent orchestrates child agents (`RunAgents`) or launches a single child (`StartAgent`), the run carries run-wide configuration: `model_id`, `harness_type`, and an `execution_mode` (`Local` vs `Remote { environment_id, worker_host }`). The harness string is the neutrality seam — `harness_proto_to_string`/`harness_type_to_proto` map between the proto `Harness` oneof and client-side identifiers for `oz`, `claude` (ClaudeCode), `opencode`, `gemini`, and `codex` (Sources: [crates/ai/src/agent/orchestration_config.rs:184-214](crates/ai/src/agent/orchestration_config.rs)).

`OrchestrationConfig` mirrors a proto message but uses Rust-native types so view/model code stays free of proto imports, and `OrchestrationConfigStatus` (`None`/`Approved`/`Disapproved`) records the user's approval state (Sources: [crates/ai/src/agent/orchestration_config.rs:5-50](crates/ai/src/agent/orchestration_config.rs)). The function `matches_active_config` decides whether a `run_agents` call can auto-launch without re-prompting: empty fields on the call are treated as "inherit from config" and therefore match, while `execution_mode` variants must agree (Local vs Remote) (Sources: [crates/ai/src/agent/orchestration_config.rs:64-96](crates/ai/src/agent/orchestration_config.rs)).

`StartAgentExecutionMode` makes BYOC explicit in its `Local` variant: a `None` `harness_type` selects "the legacy embedded local child-agent flow," while `Some(harness_type)` "selects a third-party CLI harness to launch locally." Its `Remote` variant additionally carries an `auth_secret_name` — a managed secret name forwarded as the credential for non-Oz harnesses, with `None` meaning the remote environment falls back to its own ambient credentials (Sources: [crates/ai/src/agent/action/mod.rs:221-278](crates/ai/src/agent/action/mod.rs)).

```text
RunAgents (batched)                StartAgent (single child)
  base_prompt + skills               Local { harness_type: Option<String>,
  model_id / harness_type                    model_id: Option<String> }
  execution_mode:                    Remote { environment_id, harness_type,
    Local | Remote{env,host,CU}              model_id, auth_secret_name, ... }
  agent_run_configs[ {name,prompt} ]
        |
        v
  matches_active_config(req, cfg) --> auto-launch? (skip confirmation card)
```

Sources: [crates/ai/src/agent/action/mod.rs:177-219](crates/ai/src/agent/action/mod.rs), [crates/ai/src/agent/orchestration_config.rs:64-96](crates/ai/src/agent/orchestration_config.rs)

### Skills (Multi-Vendor Discovery)

The `skills` module discovers reusable instruction packs from many vendor directories rather than one proprietary location. `SkillProvider` enumerates supported origins (`Warp`, `Agents`, `Claude`, `Codex`, `Cursor`, `Gemini`, `Copilot`, `Droid`, `Github`, `OpenCode`), and `SKILL_PROVIDER_DEFINITIONS` maps each to a relative path such as `.agents/skills`, `.warp/skills`, or `.claude/skills`. The list order defines precedence (first = highest priority), surfaced via `provider_rank` (Sources: [crates/ai/src/skills/skill_provider.rs:30-147](crates/ai/src/skills/skill_provider.rs)). `SkillScope` distinguishes `Home` (e.g. `~/.agents/skills`), `Project`, and `Bundled` skills shipped with Warp (Sources: [crates/ai/src/skills/skill_provider.rs:58-66](crates/ai/src/skills/skill_provider.rs)). Public entry points include `parse_skill`/`parse_bundled_skill`, `read_skills`, and `SkillReference` (Sources: [crates/ai/src/skills/mod.rs:1-15](crates/ai/src/skills/mod.rs)).

This portable, file/directory-based discovery is what keeps skills BYOC-friendly: a skill is a folder under a recognized provider path, not a vendor-hosted catalog dependency.

### Codebase Index

The `index` module backs the `SearchCodebase` action. It splits into a native path (gated behind the `local_fs` feature) and a `wasm` path, with a bounded Rayon thread pool (`MAX_PARALLEL_THREADS = 2`) for indexing work, and exposes `build_outline`, `Outline`, and `Symbol` for file outlines plus repo-metadata types (Sources: [crates/ai/src/index/mod.rs:1-53](crates/ai/src/index/mod.rs)). A `full_source_code_embedding` submodule and `DEFAULT_SYNC_REQUESTS_PER_MIN = 600` indicate embedding-based search with a rate-limited sync (Sources: [crates/ai/src/index/mod.rs:3-6](crates/ai/src/index/mod.rs)).

## App-Side Conversation & Agent Management

`AgentConversationsModel` is "a unified interface for reading both local and ambient agent conversations (i.e. conversations & tasks)," responsible for polling for new tasks and maintaining local state; it backs both the agent management view and the conversation list view (Sources: [app/src/ai/agent_conversations_model.rs:508-535](app/src/ai/agent_conversations_model.rs)). It caches `tasks` and `conversations`, tracks per-window active data consumers to decide when to poll, and is gated behind `FeatureFlag::AgentManagementView` — when disabled it returns an inert empty model (Sources: [app/src/ai/agent_conversations_model.rs:569-584](app/src/ai/agent_conversations_model.rs)).

The model is resilient to network failure via a three-state `TaskFetchState` (`InFlight`, `PermanentlyFailed`, `TransientlyFailed`) with distinct cooldowns — 2s for transient (5xx/408/429/network) and 60s for permanent (401/403/404), deliberately not refusing forever in case permissions change mid-session (Sources: [app/src/ai/agent_conversations_model.rs:63-93](app/src/ai/agent_conversations_model.rs)). It also coalesces real-time refresh bursts with `RtcTaskRefreshThrottleState`, keeping the *earliest* timestamp in a burst because `updated_after` is a lower bound (Sources: [app/src/ai/agent_conversations_model.rs:95-121](app/src/ai/agent_conversations_model.rs)).

Each entry's origin is captured by `AgentConversationProvenance` (`LocalInteractive`, `AmbientRun`, `CloudSyncedConversation`), with `AgentConversationBackingData` flags for which data sources contributed and `AgentConversationCapabilities` describing which actions (open, share, delete, fork, cancel) are exposed under current navigation policy (Sources: [app/src/ai/agent_conversations_model/entry.rs:141-165](app/src/ai/agent_conversations_model/entry.rs)). Entries carry an optional `Harness` from `warp_cli::agent`, defaulting to `Harness::Oz` when unknown, so the management view stays harness-aware across both ambient tasks and synced conversations (Sources: [app/src/ai/agent_conversations_model/entry.rs:103-103](app/src/ai/agent_conversations_model/entry.rs), [app/src/ai/agent_conversations_model/entry.rs:355-361](app/src/ai/agent_conversations_model/entry.rs)).

## MCP Tool Transport

The `mcp` crate is intentionally small at its root: `lib.rs` exposes only `sse_transport` (Sources: [crates/mcp/src/lib.rs:1](crates/mcp/src/lib.rs)). That module is a "legacy SSE client transport for MCP, preserved from the rmcp fork after upstream removed SSE transport support in v0.11.0," allowing Warp to keep connecting to MCP servers that only speak the older Server-Sent-Events protocol (Sources: [crates/mcp/src/sse_transport/mod.rs:1-3](crates/mcp/src/sse_transport/mod.rs)). It re-exports the client building blocks — `SseClient`, `SseClientConfig`, `SseClientTransport`, `SseTransportError`, plus retry policies `ExponentialBackoff`, `FixedInterval`, and `NeverRetry` (Sources: [crates/mcp/src/sse_transport/mod.rs:9-11](crates/mcp/src/sse_transport/mod.rs)). The agent reaches MCP servers through the `CallMCPTool` and `ReadMCPResource` actions, each optionally scoped by a `server_id: Option<Uuid>` (Sources: [crates/ai/src/agent/action/mod.rs:95-109](crates/ai/src/agent/action/mod.rs)).

## Computer-Use Automation Layer

`crates/computer_use` provides the OS-level automation behind the `UseComputer` and `RequestComputerUse` actions. Platform-specific implementations are selected at compile time (`mac/`, `linux/`, `windows/`) with a `noop` fallback, and the public surface is a single `Actor` trait whose `perform_actions` executes a slice of `Action` values and returns an `ActionResult` (Sources: [crates/computer_use/src/lib.rs:1-58](crates/computer_use/src/lib.rs)).

```mermaid
classDiagram
    class Actor {
        <<trait>>
        +platform() Option~Platform~
        +perform_actions(actions, options) Result~ActionResult, String~
    }
    class Action {
        <<enum>>
        Wait MouseDown MouseUp MouseMove
        MouseWheel TypeText KeyDown KeyUp
    }
    class Options {
        screenshot_params: Option~ScreenshotParams~
    }
    class ActionResult {
        screenshot: Option~Screenshot~
        cursor_position: Option~Vector2I~
    }
    Actor --> Action : consumes
    Actor --> Options : with
    Actor --> ActionResult : returns
    ActionResult --> Screenshot
```

`Action` covers waits, mouse down/up/move/wheel, text typing, and key down/up, with `Key` supporting either a platform-specific `Keycode` or a `Char` (with a documented Windows BMP limitation) (Sources: [crates/computer_use/src/lib.rs:60-105](crates/computer_use/src/lib.rs)). Screenshots are optional and constrained by `ScreenshotParams` (`max_long_edge_px`, `max_total_px`, optional `region`); `ScreenshotRegion::validate` rejects negative or zero/negative-area regions before capture (Sources: [crates/computer_use/src/lib.rs:127-184](crates/computer_use/src/lib.rs)). The bridge from the action vocabulary lives in `UseComputerRequest`, which holds an `action_summary`, a `Vec<computer_use::Action>`, and optional `computer_use::ScreenshotParams` (Sources: [crates/ai/src/agent/action/mod.rs:742-756](crates/ai/src/agent/action/mod.rs)).

## Provider Neutrality Summary

The agent runtime stays provider-neutral at three concrete layers verified above: the **harness seam** maps a free-form `harness_type` string to built-in (`oz`) or external CLI agents (`claude`, `opencode`, `gemini`, `codex`) and forwards a named auth secret rather than hard-wiring a provider's credentials; **skills** are discovered from a ranked list of vendor directories on disk; and **MCP transport** connects to any conforming SSE server by URL with configurable retry. The `ai` crate deliberately describes agent *intent* (`AIAgentActionType`) rather than any single model's wire format, so adding or swapping a harness, model, or skill source does not change the action vocabulary or the app-side `AgentConversationsModel` that manages the resulting conversations and tasks (Sources: [crates/ai/src/agent/action/mod.rs:30-175](crates/ai/src/agent/action/mod.rs), [crates/ai/src/agent/orchestration_config.rs:184-214](crates/ai/src/agent/orchestration_config.rs)).

> Note on this run: investigation used native repository search/read tools only. The Compound Engineering knowledge profile was applied as portable wiki-shaping guidance (Page Shape / QA Review heuristics from a *bundled* skill snapshot); no `docs/solutions/**` solved-problem notes or `STRATEGY.md` were present in this checkout, so neither source class contributed content. All implementation claims above are grounded in the cited repository files.