# Network tiers and host proxy > Traffic confinement across allowlist, open, and offline tiers, host-side TLS SNI and HTTP inspection via devvm _proxy, and per-VM CA rotation. - Repository: devvmhq/devvm - GitHub: https://github.com/devvmhq/devvm - Human docs: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853 - Complete Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/llms-full.txt ## Source Files - `internal/netpolicy/tier.go` - `internal/netpolicy/netpolicy.go` - `internal/netpolicy/rules.go` - `internal/proxy/proxy.go` - `internal/proxy/policy.go` - `internal/proxy/sni.go` - `internal/trust/ca.go` --- --- title: "Network tiers and host proxy" description: "Traffic confinement across allowlist, open, and offline tiers, host-side TLS SNI and HTTP inspection via devvm _proxy, and per-VM CA rotation." --- Every project VM routes egress traffic through a dedicated host-side HTTP/HTTPS proxy process (`devvm _proxy`) managed by `internal/netpolicy` and `internal/proxy`. On confined tiers (`internet`, `allowlist`, and `none`), the backend isolates the VM's hypervisor or hostagent process so the guest cannot make direct outbound TCP, UDP, or DNS connections; all traffic must authenticate to the VM's loopback proxy on the host. Each VM also receives an isolated ECDSA P-256 certificate authority (`internal/trust`) paired with a pre-installed successor CA, enabling the host proxy to terminate TLS on port `443` for secret-bound hosts, substitute credential placeholders in flight, and rotate signing keys without restarting the proxy or VM. ## Network tiers and backend confinement A VM's network tier (`netpolicy.Tier`) is persisted in the `"net"` section of `~/.local/share/devvm/vms//vm.json` (`netpolicy.Net`). When a VM's state omits the tier, `netpolicy.Of` defaults to `allowlist` (`netpolicy.TierDefault`). If `vm.json` cannot be parsed or contains an unrecognized tier string, `netpolicy.TierOf` fails closed to `none` (`TierNone`). | Tier | Rank | Confined (`Tier.Confined()`) | Backend network (`Tier.Net()`) | Built-in policies (`PolicySet.Builtin`) | User policy files | Internal address floor | | --- | --- | --- | --- | --- | --- | --- | | `open` | 0 (lowest) | `false` | `backend.NetDirect` | None (`Config.Open = true`) | Ignored | Disabled (reaches host `localhost` and LAN) | | `internet` | 1 | `true` | `backend.NetProxy` | `network/internet.policy` (`**:*`) | `Denies` only (`!rule`), plus `secrets.policy` | Enforced unless an explicit IP/CIDR rule matches | | `allowlist` | 2 (default) | `true` | `backend.NetProxy` | `layers/base.policy`, `shared-tools/.policy`, `layers/.policy` | All rules (`Files`) and `secrets.policy` | Enforced unless an explicit IP/CIDR rule matches | | `none` | 3 (strictest) | `true` | `backend.NetProxy` | Agent policies only (`shared-tools/claude.policy`, `codex.policy`, `opencode.policy`) | `Denies` only (`!rule`), plus `secrets.policy` | Enforced unless an explicit IP/CIDR rule matches | Every tier starts `devvm _proxy` and configures `HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY=localhost,127.0.0.1,::1`, and `NODE_USE_ENV_PROXY=1` inside the guest. On `open`, the VM is not sandboxed to the proxy, but requests that use the proxy still receive secret injection, Nix binary caching, and VM link routing. ### State persistence in `vm.json` The `"net"` section in `vm.json` stores the configured tier alongside generated per-VM credentials and self-check status (`netpolicy.Net`): ```json { "net": { "tier": "allowlist", "made": { "check": "ok", "proxy_key": "a8f9...", "proxy_salt": "4c1e9b2d..." } } } ``` - `proxy_key`: Random password generated once per VM via `backend.NewProxyPassword`. Every request to `devvm _proxy` must send `Proxy-Authorization: Basic` with username `devvm` (`proxy.ProxyUser`) and this password. - `proxy_salt`: 16-byte (32-hex-character) random value generated via `crypto/rand` (`netpolicy.Salt`). Appended to secret placeholders inside the VM so a leaked placeholder from another VM or host process cannot be exchanged for credentials. - `check`: Result of the post-start confinement verification (`ok` or `unenforced`). ### OS-level hypervisor confinement When `Tier.Confined()` is `true`, the backend requires the `egress-proxy` capability (`backend.NetProxy`) and restricts the VM process before boot: On macOS, the Lima driver (`internal/backend/lima`) launches the VM's `limactl` hostagent under `/usr/bin/sandbox-exec` using a generated Seatbelt profile at `~/.local/share/devvm/vms//run/sandbox.sb` (`lima.profile`): ```scheme (version 1) (allow default) (deny network-outbound) (allow network-outbound (remote unix-socket (subpath "/Users/you/.local/share/devvm/lima/devvm-..."))) (allow network-outbound (remote unix-socket (path-literal "/Users/you/.local/share/devvm/run/ssh-agent.sock"))) (deny mach-lookup (global-name "com.apple.dnssd.service")) (allow network-outbound (remote ip "localhost:60022")) (allow network-outbound (remote ip "localhost:54321")) (deny network-bind) (allow network-bind (local tcp "localhost:60022")) (allow network-bind (local unix-socket (subpath "/Users/you/.local/share/devvm/lima/devvm-..."))) ``` - **Outbound restriction:** Denies all outbound IP and Unix socket connections except the VM's SSH port (`localhost:`), the VM's host proxy port (`localhost:`), Unix sockets inside the VM's Lima directory, and the configured SSH agent socket. - **DNS suppression:** Blocks Mach lookups to `com.apple.dnssd.service`, disables Lima's `hostResolver`, and points guest DNS at an unreachable address so guest DNS queries fail immediately and all name resolution occurs inside `devvm _proxy`. - **Bind restriction:** `NoBinds` prevents the Lima hostagent from binding arbitrary host ports; guest port forwarding is handled exclusively by `devvm _ports`. - **Guest proxy endpoint:** The guest reaches the proxy at `http://devvm:@192.168.5.2:`. On Linux, Lima starts QEMU through a wrapper generated in `~/.local/share/devvm/run/qemu/` (`devvm _qemu`). - **Argument validation:** The wrapper inspects Lima's QEMU invocation and refuses to start (`qemu_refused`) unless there is exactly one `-netdev user,id=net0,...` argument and no additional network devices. - **SLIRP restriction:** Appends `restrict=on` to the user-mode `-netdev` configuration, blocking all guest IP traffic and SLIRP DNS resolution. - **Guest forward:** Adds a single `guestfwd=tcp:192.168.5.4:3128-cmd:...` rule bridged to the host proxy at `127.0.0.1:`. - **Guest proxy endpoint:** The guest reaches the proxy at `http://devvm:@192.168.5.4:3128`. ### Startup sequence and `SelfCheck` During `starter.bringUp` (`internal/vm/start.go`), `netpolicy.Proxy.Prepare` and `netpolicy.Proxy.SelfCheck` run before guest setup scripts or user shells execute: `Proxy.secretsReady` invokes `trust.InitCA` on `~/.local/share/devvm/vms//run/ca/`, excludes the CA directory from macOS Time Machine backups (`platform.ExcludeFromBackup`), generates the VM's `ProxySalt` if absent, and writes `secrets.json` and `secrets.policy`. `Proxy.Prepare` builds `proxy.Config` and compares its SHA-256 (`cfg.Sum()`) against the running proxy's `~/.local/share/devvm/vms//run/proxy.json`. If the VM is stopped or the config sum changed (`Proxy.stale`), `Proxy.Stop` sends `SIGTERM` (followed by `SIGKILL` after 2 seconds if still alive), writes the new `proxy.json` (`0o600`), spawns `devvm _proxy -config `, and waits up to 5 seconds (`10 * 500ms`) for `proxy.status` to report the bound `127.0.0.1:` address. - **Confined tiers (`internet`, `allowlist`, `none`):** `enforced(ctx, b, vm)` first checks `b.Running(ctx, vm).Confined` on the host to verify the hypervisor process is the one `devvm` launched under confinement. Next, `Direct(ctx, b, vm)` runs `curl --noproxy * -sS -m 3 -o /dev/null https://1.1.1.1` inside the VM, which must fail. - If `run.Confined` is `true` but `Direct` succeeds, the VM has an unauthorized route out; `SelfCheck` records `"check": "unenforced"`, stops the VM immediately, and fails with `net_direct_route`. - If `run.Confined` is `false` (for example, started via `limactl start` or switched from `open` while running), `SelfCheck` records `"unenforced"` and prompts to restart the VM confined (`net_unenforced_restart`), or stops the VM when non-interactive (`net_unenforced_stop`). - **Open tier (`open`):** `checkOpen` sources `/etc/environment` inside the VM and runs `curl -sSf -m 3 -o /dev/null --proxy "$HTTPS_PROXY" http://devvm.proxy/blocked`. If the proxy fails to respond, `devvm` emits a `net_proxy_unreachable` warning and continues. ### Tier transitions Changing a VM's tier with `devvm network ` (`vm.Net.SetTier`) enforces link and group invariants: - **Between confined tiers (`internet`, `allowlist`, `none`):** Applies immediately to a running VM by regenerating `proxy.json` and restarting `devvm _proxy` on the same loopback port. - **Into or out of `open`:** Requires a VM restart because hypervisor confinement (`sandbox-exec` or QEMU `restrict=on`) is fixed at VM boot. If the VM is running, `devvm` offers to restart it immediately; if declined, a VM moving from a confined tier to `open` remains confined to its proxy (which switches to `"open": true` rules) until the next `devvm restart`. - **Link and group constraints:** Refuses tier changes if the VM belongs to a group (`GroupTierFixed`) or if the new tier would violate the link rule requiring every target VM to be on at least as strict a tier as its client (`tierLinked`, checked via `Tier.AtLeast`). --- ## Host proxy architecture (`devvm _proxy`) `internal/proxy` is a standalone, stdlib-only package (with `internal/daemon`) executed via the hidden `devvm _proxy -config ` command (`internal/cli/proxy.go`). All runtime state for a proxy instance lives in a `proxy.server` struct with zero package-level mutable state. ```mermaid flowchart TD subgraph Guest["Project VM Guest"] Client["HTTP / HTTPS / SSH Client"] end subgraph HostProxy["Host Process: devvm _proxy (internal/proxy)"] Auth["authorized()
Proxy-Authorization: Basic devvm:<key>"] HostCheck["splitHostPort() & HostOK()
lowercase, trim trailing dot, reject shorthand IPs"] Reserved{{"host == devvm.proxy?"}} ReservedSrv["serveReserved()
/blocked, /blocked/new, /ca.pem, /nix/"] ConnCap["connects chan (maxConnects = 1024)
HTTP 503 if full"] Links{{"Matches links.json?
(*.devvm.internal / link name)"}} LinkSock["Unix socket in 127.77.0.0/16
(no rules, no SNI check, no secrets)"] Policy["policy.verdict(host, port)
1. BrokenDeny check
2. Deny rules (!rule)
3. Allow rules"] Floor["egress.floorCheck()
Resolve DNS or check floorUpstream()
Block loopback, RFC1918, CGNAT, NAT64, hostAddrs"] InterceptCheck{{"Port 443 + CA loaded +
(bound secret or OAuth capture)?"}} SNIPeek["tunnel() + peekClientHello()
Verify TLS SNI == CONNECT host
before sending client bytes"] MITM["intercept() + forwarder()
Mint leaf cert via ca.signer()
Check Host header (421 if mismatch)
refuseMinting() for api.github.com
secrets.inject() / captureResponse()"] end subgraph UpstreamNet["Host Egress / Upstream"] CorpProxy["Upstream Proxy (optional)
HTTPS_PROXY / HTTP_PROXY"] Target["Remote Destination"] end Client -->|"CONNECT or GET http://..."| Auth Auth -->|"missing/bad -> 407"| HostCheck HostCheck -->|"invalid -> 400"| Reserved Reserved -->|"yes (HTTP)"| ReservedSrv Reserved -->|"no (CONNECT)"| ConnCap ConnCap -->|"slot acquired"| Links Links -->|"yes"| LinkSock Links -->|"no"| Policy Policy -->|"denied -> 403"| Floor Floor -->|"internal IP -> 403"| InterceptCheck InterceptCheck -->|"no"| SNIPeek InterceptCheck -->|"yes"| MITM SNIPeek --> CorpProxy MITM --> CorpProxy CorpProxy --> Target ``` ### Proxy configuration validation and lifecycle `netpolicy.Proxy.start` writes `~/.local/share/devvm/vms//run/proxy.json` with mode `0o600` before launching `devvm _proxy -config `. `proxy.ReadConfig` opens the file with `O_RDONLY | O_NONBLOCK | O_NOFOLLOW` (`readPrivate`) and enforces: - Regular file (not a symlink, FIFO, or device), owned by `os.Geteuid()`, with no group or world permissions (`fi.Mode().Perm() & 0o077 == 0`), and size $\le 1\text{ MiB}$ (`configMax`). - Strict JSON decoding (`DisallowUnknownFields`) with `"version": 1` (`ConfigVersion`). - Non-empty `password` containing no whitespace, `listen` set to a valid loopback address (`127.0.0.1:` or `[::1]:`), `open` mutually exclusive with `builtin_policies`/`policies`/`deny_policies`, paired `ca`/`ca_key` and `ca_next`/`ca_next_key`, and valid `captures` entries. Once listening, `serve` writes its bound `127.0.0.1:` address to the status file in `DEVVM_DAEMON_STATUS` (`daemon.Ready`) and spawns `watchLifeline`, which polls the Lima hostagent PID file (`Config.Lifeline`, `ha.pid`) every 3 seconds (`lifelineTick`) after waiting up to 15 minutes (`lifelineWait`) for the hostagent to start. When the hostagent exits, `devvm _proxy` closes its listener and terminates. ### `proxy.Config` schema reference Configuration schema version. Must be `1` (`proxy.ConfigVersion`). Loopback `ip:port` to bind (`127.0.0.1:`, or port `0` for build proxies). VM identifier written to every JSON-lines event in `network.jsonl`. Per-VM proxy secret (`netpolicy.Key`). Validated in constant time (`subtle.ConstantTimeCompare`) against `Proxy-Authorization: Basic` with user `devvm`. Ordered asset paths in `assets.FS` (for example, `layers/base.policy`, `shared-tools/node.policy`) loaded once at proxy startup (`readBuiltins`). Any missing file or invalid line aborts proxy startup. Ordered file paths on the host re-read on change (`watched[policyFile]`) where both allow and deny rules apply. Ordered file paths on the host evaluated after `policies` where allow rules are stripped and only deny rules (`!rule`) and broken denies apply (used on `internet` and `none` tiers). When `true`, disables all policy rules and the internal address floor. Cannot be combined with `builtin_policies`, `policies`, or `deny_policies`. Paths to the VM's current and successor CA certificate (`0o644`) and ECDSA private key (`0o600`) files in `vms//run/ca/`. Path to `vms//run/secrets.json` (`proxy.BindingList`), hot-reloaded on modification. OAuth token endpoints (`agent`, `host`, `path`) intercepted on port `443` to capture agent logins onto the host. Corporate proxy configuration (`https`, `http`, `no_proxy`, `secret`), PEM bundle of user CAs from `~/.config/devvm/ca`, and build-proxy flag allowing unresolved hostnames matched only by `**` to be forwarded upstream. Host directory (`~/.local/share/devvm/cache/nix-cache`) and byte cap (default `10 GiB`, `NixCacheDefault`) for serving `http://devvm.proxy/nix/`. Path to `vms//run/links.json` and the socket directory `~/.local/share/devvm/run`. SHA-256 digests of the `devvm` binary and both CA certificates. Included in `proxy.json` so `Config.Sum()` changes whenever `devvm` is upgraded or CAs rotate, triggering an automatic proxy restart. --- ## Policy files, rule grammar, and the address floor ### Policy evaluation order `netpolicy.Policies` constructs the `PolicySet` for a VM based on its tier, image layers, and enabled shared tools. On `allowlist`, policies are concatenated in this order: | Order | Source | Path / Asset | Scope | | --- | --- | --- | --- | | 1 | Base built-in policy | `assets/layers/base.policy` | OS package mirrors (Ubuntu, Debian, Alpine), `mise-versions.jdx.dev`, Sigstore/GitHub TUF roots, `nodejs.org`, and GitHub (`github.com`, `github.com:22`, `api.github.com`, `codeload.github.com`, `raw.githubusercontent.com`, `objects.githubusercontent.com`, `release-assets.githubusercontent.com`, `lfs.github.com`) | | 2 | Shared tool policies | `assets/shared-tools/.policy` | Registries and APIs for each shared tool in the VM (`node`, `python`, `rust`, `go`, `dotnet`, `claude`, `codex`, `opencode`) | | 3 | Built-in layer policies | `assets/layers/.policy` | Built-in image layers (`docker`, `nix`, `desktop`) | | 4 | Custom layer policies | `~/.config/devvm/layers/.policy` | User-authored layers included in the VM's image | | 5 | Global user policy | `~/.config/devvm/policy` | Rules managed with `devvm network allow\|deny\|rm -g` | | 6 | Per-VM user policy | `~/.local/share/devvm/vms//policy` | Rules managed with `devvm network allow\|deny\|rm` in the project folder | | 7 | Bound secrets policy | `~/.local/share/devvm/vms//run/secrets.policy` | `:443` rules generated for bound secrets, agent logins, and agent OAuth token endpoints | When `policy.verdict(host, port)` evaluates a request (`internal/proxy/policy.go`): 1. If `policy.open` is `true`, the request is immediately allowed with rule `"open"`. 2. If any watched policy file contains a **broken deny** (`BrokenDeny`), the proxy refuses all requests (`Refused: true`). 3. Every rule across all policies is scanned for a matching **deny** (`r.deny && match(r)`). If any deny matches, the request is blocked with `"denied by " + r.text`. Denies always beat allows regardless of file order. 4. Every rule is scanned for a matching **allow** (`!r.deny && match(r)`). The first matching allow rule permits the request to proceed to the address floor check. 5. Otherwise, the request is blocked with `"not on the allowlist"`. ### Rule syntax and `parseRule` validation `proxy.ParsePolicy` strips UTF-8 BOMs, splits on `#` comments, trims whitespace, and validates each non-empty line with `parseRule`: | Pattern | Port behavior | Description | | --- | --- | --- | | `example.com` | `80` and `443` (`port == 0`) | Exact hostname match (case-insensitive; one trailing dot stripped). | | `*.example.com` | `80` and `443` | Matches any subdomain at any depth (does not match bare `example.com`). | | `**` | `80` and `443` | Matches any hostname or public IP address. Does **not** bypass the internal address floor. | | `10.0.0.5` or `10.0.0.0/8` | `80` and `443` | Matches an IPv4/IPv6 address or CIDR prefix (and NAT64-embedded IPv4 addresses). Required to reach internal/private addresses past the floor. | | `:22` or `[fd00::1]:8080` | Port `22` / `8080` only | Restricts the rule to a single TCP port (`1`–`65535`). | | `:*` or `**:*` | Any port (`1`–`65535`) | Matches all valid TCP ports (`anyPort = -1`). | | `!` | Same as `` | Deny rule; takes precedence over any allow rule. | **Fail-closed broken denies (`proxy.BrokenDeny`):** A line that is not a valid rule (`!l.OK`) is normally logged to `proxy.log` and skipped. However, if an invalid line starts with `!` (for example, `! example.com` with a space or `!*.10.0.0.1`), `BrokenDeny` returns `true` and the proxy denies **all** traffic for that VM until the line is fixed or removed. ### Hot-reloading policy files (`watched`) User policy files, `secrets.json`, and `links.json` are wrapped in `watched[T]` (`internal/proxy/watched.go`). On every request, `watched.get()` runs `os.Stat` (or `os.Lstat` for private files) and re-reads the file when: - Its modification time (`mtime`), byte size, or inode changed since the last read, **or** - The last read's `mtime` was within 2 seconds of the read timestamp (preventing missed updates when two atomic writes land inside a single filesystem `mtime` tick). Policy files are read via `proxy.ReadRegular` with `O_NONBLOCK` and a `1 MiB` cap (`PolicyMax`), preventing a FIFO or `/dev/zero` symlink from hanging or exhausting memory. If a policy file is deleted (`fs.ErrNotExist`), it contributes zero rules; if reading fails for any other reason (such as permissions or `ErrNotRegular`), `watched` retains the last known rules (`keep: true`) so a transient read error never drops a deny rule. ### Hostname normalization and the internal address floor Before checking policy rules, `splitHostPort` lowercases the target host and strips one trailing dot, and `proxy.HostOK` rejects malformed or ambiguous targets with `HTTP 400 Bad Request`: - Rejects IPv6 zone identifiers (`fe80::1%eth0`), empty labels (`evil..test`, `.evil.test`, `evil.test..` which becomes `evil.test.` after stripping one dot), and names longer than 253 characters. - Rejects hostnames whose final label is purely numeric or starts with `0x` (such as `127.1`, `10.1`, `2130706433`, or `0x7f000001`), preventing `inet_aton` shorthand addresses from bypassing the address floor at an upstream corporate proxy. After a hostname or IP passes `policy.verdict`, `egress.resolve` resolves the hostname on the host via `net.DefaultResolver.LookupNetIP` and filters the returned IPs against the **address floor** (`hostAddrs.internal`): - **Host interfaces (`hostAddrs`):** Refreshed every 30 seconds via `net.InterfaceAddrs()`. - **Non-public ranges:** Loopback (`127.0.0.0/8`, `::1`), RFC 1918 / RFC 4193 private addresses, link-local unicast and multicast, interface-local multicast, unspecified (`0.0.0.0`, `::`), CGNAT (`100.64.0.0/10`), `"this network"` (`0.0.0.0/8`), IETF protocol assignments (`192.0.0.0/24`), benchmarking (`198.18.0.0/15`), reserved (`240.0.0.0/4`), local-use NAT64 (`64:ff9b:1::/48`), IPv4-compatible IPv6 (`::/96`), and site-local IPv6 (`fec0::/10`). - **NAT64 unwrapping:** Any address in `64:ff9b::/96` (`nat64IPv4`) is judged by its embedded 32-bit IPv4 address, so `64:ff9b::7f00:1` (`127.0.0.1`) is treated as internal and matches IPv4 deny/allow rules. An internal address is dialed only if an explicit IP/CIDR allow rule matches it (`pol.addrAllowed(a, port)`) and no IP/CIDR deny rule matches (`!pol.addrDenied(a, port)`). `egress.dial` dials the resolved `netip.AddrPort` values directly rather than passing the hostname to `net.Dial`, eliminating DNS rebinding between policy evaluation and connection establishment. --- ## TLS SNI inspection, secret injection, and OAuth capture ### Unintercepted `CONNECT` tunnels and SNI verification When a `CONNECT host:port` request is allowed and does not require TLS interception, `server.tunnel` dials the target and splices bytes bidirectionally (`internal/proxy/splice.go`, `internal/proxy/sni.go`): 1. **Server-first compatibility:** Bytes from the upstream server flow to the VM immediately after `200 Connection Established`, so protocols where the server speaks first (such as SSH or SMTP) deliver their banner without waiting for the client. 2. **Client-side TLS peek (`peekClientHello`):** On the client-to-upstream stream, the proxy peeks at the first byte (`r.Peek(1)`). If it is not `0x16` (TLS Handshake record), or the second byte in the 5-byte header is not `0x03`, client bytes are forwarded immediately without delay. 3. **In-memory SNI parsing (`sniOf`):** If the client stream begins with a TLS record, `peekClientHello` reads the full record (`5 + length` bytes) and passes it to `tls.Server` backed by a read-only `helloConn` buffer whose `GetConfigForClient` callback extracts `ClientHelloInfo.ServerName` and aborts the handshake. 4. **Domain-fronting check (`sniMatches`):** - For a DNS hostname in `CONNECT`, the TLS SNI (with any trailing dot stripped) must match `host` case-insensitively. - For an IP literal in `CONNECT`, the TLS SNI must be empty or match the IP literal. - If the SNI does not match, `server.tunnel` logs a `blocked` event (`TLS name "" doesn't match `) and closes the connection before a single byte of the `ClientHello` reaches the remote server. ### TLS interception and secret injection on port 443 Project VMs never receive raw tokens or API keys. Instead, guest environments and agent configuration files receive salted placeholders (such as `CLAUDE_CODE_OAUTH_TOKEN`, `GH_TOKEN`, or JWT-shaped placeholders in `~/.codex/auth.json` and `~/.local/share/opencode/auth.json`), and `vms//run/secrets.json` lists the active `proxy.Binding` rules: ```json { "version": 1, "bindings": [ { "name": "github", "kind": "secret", "hosts": ["api.github.com", "github.com", "codeload.github.com", "raw.githubusercontent.com"], "header": "Authorization", "placeholder": "gho_devvm_proxy_placeholder_4c1e9b2d..." } ] } ``` When a `CONNECT` request targets port `443` (`s.interceptPorts[443]`) and matches either a bound secret (`len(s.secrets.forHost(host)) > 0`) or an OAuth capture host (`s.captures.has(host)`), `server.intercept` terminates TLS: 1. **Client TLS handshake:** Uses `s.ca.leafFor(host)` to present a leaf certificate signed by the VM's active CA, negotiating `h2` or `http/1.1` with a 30-second handshake timeout. 2. **Host header pinning:** Every HTTP request inside the tunnel must have a `Host` header matching the `CONNECT` host. If a client sends a request for another host over the same HTTP/2 or keep-alive connection, the proxy responds with `421 Misdirected Request`. 3. **Credential-minting guard (`refuseMinting`):** On `api.github.com`, `mintsCredential` blocks requests that would use an injected GitHub token to create persistent credentials that outlive `devvm secret unuse`: - `POST /user/keys`, `POST /user/ssh_signing_keys`, `POST /user/gpg_keys` (account keys) - `POST /repos/{owner}/{repo}/keys` and `POST /repositories/{id}/keys` (deploy keys) - `POST /.../{repos,repositories,orgs,organizations,enterprises}/.../actions/runners/{registration-token,generate-jitconfig}` (self-hosted runners) - `POST` or `PATCH` to `/.../{repos,repositories,orgs,organizations}/.../hooks` (webhooks) 4. **In-process secret lookup and header injection (`secrets.inject`):** - For each matching `Binding`, if the request's `b.Header` (such as `Authorization`) contains `b.Placeholder` directly or base64-encoded inside a `Basic ` credential, `s.secrets.value(b)` resolves the secret in-process via `IO.Secret` (`secrets.Store.Get`). - Lookups are serialized under `valueMu` with a 20-second timeout (`lookupTimeout`), routed through the proxy's configured upstream corporate proxy (`WithRoute`) and custom CAs (`WithRoots`), and cached in memory for **5 minutes** (`valueTTL` for `KindSecret`) or **30 seconds** (`loginTTL` for `KindLogin`). - Touching or rewriting `secrets.json` (such as via `devvm secret unuse` or `devvm secret set`) triggers the `watched` `changed` callback, immediately flushing cached secret values even for already-open tunnels. ### OAuth token capture (`internal/proxy/capture.go`) Every VM's `proxy.Config` includes `agentlogin.Endpoints()` in `captures`: - `auth.openai.com` at `/oauth/token` (`codex`, `opencode`) - `platform.claude.com` at `/v1/oauth/token` (`claude`) - `console.anthropic.com` at `/v1/oauth/token` (`claude`) When an agent logs in inside the VM, `server.forwarder` intercepts port `443` to those hosts: - **Path canonicalization:** Any `POST` whose cleaned path (stripped of `;param` segments and matched case-insensitively) equals the capture path is rewritten to the canonical endpoint path with query strings removed. - **Request inspection (`captureRequest`):** Reads the request body up to `64 KiB` (`captureRequestMax`, with at most 8 concurrent reads via `captureReads`). If a VM sends its own placeholder refresh token (`devvm-proxy-managed`), the proxy short-circuits the request locally and returns fresh placeholders from the host store; if Codex performs an `id_token` exchange for an `openai-api-key`, the proxy substitutes the real `id_token` from the host store. - **Response capture (`captureResponse`):** Strips `Accept-Encoding` on outbound requests so Go's transport decompresses gzip transparently, drops `1xx` informational responses (`no1xx`) and `101` upgrades, reads `2xx` responses up to `1 MiB` (`captureReplyMax`), persists the real tokens to `~/.local/share/devvm/logins//` on the host, rewrites `vms//run/secrets.json`, and returns a sanitized JSON body to the VM containing salted placeholders, `"refresh_token": "devvm-proxy-managed"`, and an expiry in `2099`, keeping only `Content-Type`, `Cache-Control`, `Pragma`, and `Date` response headers. - **Unmatched path guard (`guard`):** Any other `2xx` response from a capture host (outside static HTML, CSS, JS, image, or font assets) whose body contains `"access_token"`, `"refresh_token"`, or `"id_token"` is refused with `502 Bad Gateway` so an alternate endpoint on the same host cannot leak raw tokens into the VM. --- ## Per-VM CA lifecycle and rotation Each VM has its own private Certificate Authority in `~/.local/share/devvm/vms//run/ca/` (`internal/trust/ca.go`). Because no two VMs share a CA, a compromised CA key in one VM's run directory cannot sign certificates trusted by any other VM or by the host. ```mermaid sequenceDiagram autonumber participant Launch as devvm launch (trust.InitCA) participant Disk as vms//run/ca/ participant Guest as VM Guest (/usr/local/share/ca-certificates) participant Proxy as devvm _proxy (proxy.ca) Launch->>Disk: Lock vms//run/ca/lock Launch->>Disk: Write ca.key (0600) & ca.pem (0644)
NotBefore: T0 - 1h, NotAfter: T0 + 30d Launch->>Disk: Write next.key (0600) & next.pem (0644)
NotBefore: T0 + 23d - 1h, NotAfter: T0 + 53d Launch->>Guest: assets/guest/ca.sh installs
devvm-proxy.crt (ca.pem) & devvm-proxy-next.crt (next.pem) Launch->>Proxy: Start with CA + CANext Note over Proxy: Days 0..23: signer(now) uses ca.pem
Mints 24h leaf certs (LRU cache max 1024) Note over Proxy: Day 23 (next.NotBefore + 1h):
signer(now) switches to next.pem automatically
(no VM launch or proxy restart needed) Launch->>Disk: Next launch after Day 23:
Promotes next -> ca, generates new next ``` ### CA timing constants and rotation rules `trust.InitCA` manages two ECDSA P-256 certificate/key pairs under `~/.local/share/devvm/vms//run/ca/lock`: | Constant / File | Value / Mode | Behavior | | --- | --- | --- | | `caLife` | `30 * 24 * time.Hour` (30 days) | Validity duration of each generated CA certificate (`NotAfter - NotBefore = caLife + caSkew`). | | `caRenew` | `7 * 24 * time.Hour` (7 days) | Renewal threshold before `cur.NotAfter`. When `cur` has $\le 7\text{ days}$ remaining, `InitCA` promotes `next` to `ca`. | | `caSkew` / `successorSkew` | `1 * time.Hour` | Clock-skew buffer. A new `cur` starts at `now - 1h`; `next` starts at `cur.NotAfter - 7d - 1h`, and the proxy begins signing with `next` at `next.NotBefore + 1h` (`cur.NotAfter - 7d`). | | `ca.pem` / `ca.key` | `0o644` / `0o600` | Current CA certificate and ECDSA P-256 private key. Key is always written before certificate so an interrupted write fails `tls.X509KeyPair` and is regenerated. | | `next.pem` / `next.key` | `0o644` / `0o600` | Successor CA certificate and private key, installed alongside `ca.pem` in the VM's trust store on every launch. | Each generated CA sets `IsCA: true`, `BasicConstraintsValid: true`, `MaxPathLenZero: true`, `KeyUsageCertSign | KeyUsageCRLSign`, and a unique 128-bit random serial number embedded in `Subject.CommonName` (`"devvm proxy CA "`) so `cur` and `next` never collide on subject name lookups. ### Leaf certificate minting and expiration handling Inside `devvm _proxy` (`internal/proxy/secrets.go`): - **Automatic signer switch (`ca.signer`):** At timestamp `now`, if `now >= next.NotBefore + 1h`, `signer(now)` returns `next`; otherwise it returns `cur`. Because both certificates were installed in the VM at launch, a long-running VM continues verifying intercepted TLS connections for at least `30 days - 1 hour` after its last `devvm` launch without restarting `devvm _proxy`. - **Leaf caching (`ca.mint`):** Mints a 24-hour leaf certificate (`DNSNames: []string{host}`, clamped to the signing CA's `NotBefore` and `NotAfter`) using a single per-CA ephemeral ECDSA P-256 `leafKey`. Keeps up to `1024` (`leafCacheMax`) leaf certificates in an LRU list; cached leaves are reused while they have more than 1 hour remaining (or until they reach the CA's `NotAfter`). - **Expired CA refusal (`caEndedRefusal`):** If a VM runs for more than 30 days without a `devvm` launch so that `now >= signer(now).NotAfter`, the proxy refuses intercepted `CONNECT` requests on port `443` with `403 Forbidden` (`Refused: true`) and logs: ```text devvm's CA has ended: open the VM again (devvm in its folder) for a new one ``` Running `devvm` in the project folder rotates the CAs, restarts the proxy, and installs the new certificates in the guest. ### Guest and custom CA installation (`assets/guest/ca.sh`) At every launch, `sync.sh` runs `assets/guest/ca.sh` as root with a directory containing `devvm-proxy.crt` (`ca.pem`), `devvm-proxy-next.crt` (`next.pem`), and any user CAs (`devvm-extra-1.crt`, ...) validated by `trust.Read` from `~/.config/devvm/ca`: 1. Copies changed certificates into `/usr/local/share/ca-certificates/` (`0o644`) and deletes any removed `devvm-*.crt` files. 2. Runs `update-ca-certificates --fresh` (or updates `/etc/ssl/certs/ca-certificates.crt` directly while preserving the original bundle in `/var/lib/devvm-ca` on minimal images without `update-ca-certificates`) and notifies `/etc/ca-certificates/update.d` hooks (such as Java's `jks-keystore`) of removed certificates. 3. Writes the combined PEM bundle of devvm-managed CAs to `/etc/ssl/devvm.pem` (`trust.Bundle`). 4. Exports `trust.Env` across all guest sessions so runtimes that bypass the system store trust both the system bundle and `/etc/ssl/devvm.pem`: - `NODE_EXTRA_CA_CERTS=/etc/ssl/devvm.pem` - `SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt` - `REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt` - `PIP_CERT=/etc/ssl/certs/ca-certificates.crt` - `UV_SYSTEM_CERTS=1` --- ## Reserved endpoints, upstream proxies, and logging ### Reserved `http://devvm.proxy` endpoints Plain HTTP requests sent to the proxy with `Host: devvm.proxy` are handled internally by `server.serveReserved` (`internal/proxy/proxy.go`): | Endpoint | Method | Response | | --- | --- | --- | | `http://devvm.proxy/blocked` | `GET` | Plain-text list of the last 50 blocked requests with `devvm network allow ` hints, or `nothingBlocked` explaining that a DNS/connection error for an allowed host means the tool bypassed `$HTTPS_PROXY`. Queried by `devvm-why` inside the VM. | | `http://devvm.proxy/blocked/new?after=N` | `GET` | Returns the total block counter on line 1 followed by deduplicated block notices since block `N`. Used by the guest shell prompt hook. | | `http://devvm.proxy/ca.pem` | `GET` | Concatenated PEM certificates of the VM's current CA and next CA (never private keys). | | `http://devvm.proxy/nix/...` | `GET`, `HEAD` | Shared host-filled Nix binary cache (`internal/proxy/nixcache.go`) when the VM image includes the `nix` layer. Serves `nix-cache-info` (priority `30`), `<32-base32>.narinfo`, and `nar/<52-base32>.nar[.xz\|.zst\|.bz2]`. Fetches misses from `https://cache.nixos.org` only when `cache.nixos.org:443` is allowed, verifies the pinned `cache.nixos.org-1` Ed25519 signature on each `narinfo` and the SHA-256 `FileHash`/`FileSize` on each NAR, and enforces an LRU disk cap (`nix-cache.size`, default `10 GiB`) under `~/.local/share/devvm/cache/nix-cache/.lock`. | ### Corporate upstream proxy routing (`internal/proxy/upstream.go`) `netpolicy.Proxy.Upstream` reads `HTTPS_PROXY`, `HTTP_PROXY`, and `NO_PROXY` from the host environment, or falls back to the `[proxy]` table in `~/.config/devvm/config.toml` (never mixing the two). - Only `http://` upstream proxies are supported (`ParseUpstream` returns `ErrUpstreamScheme` for `https://` or `socks5://`). - `config.toml` URLs cannot contain inline passwords; if a `[proxy]` URL includes a username (`http://alice@proxy.example.com:8080`), `Upstream.Secret` is set to `"proxy"`, and `devvm _proxy` looks up the `proxy` secret saved via `devvm secret set proxy` at startup. - When routing through an upstream proxy, `egress.floorUpstream` still resolves the hostname locally with a 3-second timeout and blocks the request if any returned IP is internal or denied. If the host resolver cannot resolve the name, `floorUpstream` forwards it to the corporate proxy only if an explicit hostname or `*.domain` rule allows it (`pol.byName(host, port)`), refusing unresolved names matched only by `**` (`errUnresolved`) unless `UpstreamUnresolved` is set (used only by image build proxies). ### Logs and `devvm network` commands Each VM maintains two log files in `~/.local/share/devvm/vms//logs/` (mode `0o600`), rotated at `10 MiB` (`proxy.LogLimit`): - `network.jsonl` (rotated by renaming to `network.jsonl.1` under an `flock` lock): JSON-lines decision log written by `devvm _proxy` and `devvm _links`. Records `tunnel`, `intercept`, `forward`, `blocked`, `login`, `nix-cache`, `link`, `link_refused`, `link_target_stopped`, and `link_limit` events. Never logs secret values or upstream proxy credentials. - `proxy.log` (rotated by copying to `proxy.log.1` and truncating in place): Diagnostic stderr log of `devvm _proxy`, including loaded policy counts, skipped/broken rules, and upstream DNS or connection errors withheld from the guest. ```bash title="Inspect tier, rules, and logs" # Show the folder's VM tier and ordered policy rules devvm network # Switch the VM to a stricter or wider tier devvm network none devvm network allowlist # Add or remove per-VM or global (-g) policy rules devvm network allow api.example.com '*.cdn.example.com:8443' devvm network deny '!telemetry.example.com' devvm network rm api.example.com devvm network allow -g pypi.org # Inspect recent events or deduplicated blocked hosts devvm network log -n 100 devvm network log --follow devvm network log --blocked ``` ```json title="devvm network --json" { "version": 1, "policy": "allowlist", "allowlist": [ { "builtin": "layers/base.policy", "rules": [ "ports.ubuntu.com", "github.com", "github.com:22", "api.github.com" ], "not_rules": [] }, { "file": "/Users/you/.local/share/devvm/vms/71a99b1076d92c/policy", "rules": [ "api.example.com" ], "not_rules": [] } ] } ``` ```json title="devvm network log --blocked --json" { "version": 1, "blocked": [ { "last": "2026-10-09T10:01:00Z", "count": 2, "host": "example.com", "port": 443, "allow": "example.com" } ] } ``` ## Related pages Run Claude Code, Codex, and opencode inside VMs, capture OAuth logins, and bind host secrets to allowed hosts. Route traffic between VMs over 127.77.0.0/16 sockets, enforce link tier rules, and forward guest ports to the host. Syntax, subcommands, and flags for devvm network, secret, agent, git-protect, clone, ssh-agent, link, and group. Inspect blocked requests with devvm network log, configure corporate proxies and custom CAs in ~/.config/devvm/ca, and recover disk space.