# devvm Documentation > Technical documentation and CLI/configuration reference for devvm, an isolated per-project virtual machine sandbox on Lima with read-only shared toolchains, coding agents, and host-enforced network and Git protections. ## Context Links - [Agent index](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/llms.txt) - [Human interactive docs](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853) - [GitHub repository](https://github.com/devvmhq/devvm) ## Repository Metadata - Repository: devvmhq/devvm - Generated: 2026-11-07T23:01:02.161Z - Updated: 2026-11-08T01:21:57.983Z - Runtime: Antigravity CLI - Format: Documentation - Pages: 23 ## Page Index - 01. [Overview](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/01-overview.md) - Public entry points, per-project VM sandbox architecture, runtime assumptions, and primary documentation routes. - 02. [Installation](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/02-installation.md) - Install devvm on macOS and Linux via Homebrew or the signed release installer, verify Minisign signatures, and validate host readiness with devvm doctor. - 03. [Quickstart](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/03-quickstart.md) - Configure initial defaults with devvm setup, launch a project VM with auto-detected toolchains and agents, and verify isolation in the guest shell. - 04. [VM lifecycle and state](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/04-vm-lifecycle-and-state.md) - How project paths map to VM names and 8-hex hashes, state persistence in vm.json, layout version checks, and in-place vs. remake transitions. - 05. [Images, layers, and content keys](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/05-images-layers-and-content-keys.md) - Deterministic SHA-256 key computation for base images, built-in and custom layers (docker, nix, desktop), and read-only EROFS shared tool images. - 06. [Network tiers and host proxy](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/06-network-tiers-and-host-proxy.md) - Traffic confinement across allowlist, open, and offline tiers, host-side TLS SNI and HTTP inspection via devvm _proxy, and per-VM CA rotation. - 07. [Host and Git isolation boundaries](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/07-host-and-git-isolation-boundaries.md) - Read-only .git mount protection, symlink and hardlink containment, set-id file audits, secret placeholder injection, and isolated repo clones. - 08. [Backend contract and Lima driver](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/08-backend-contract-and-lima-driver.md) - The four-role backend.Backend interface (Host, Images, Machines, Housekeeping), capability requirement checks (Caps), and the Lima driver implementation. - 09. [Configure coding agents and credentials](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/09-configure-coding-agents-and-credentials.md) - Run Claude Code, Codex, and opencode inside VMs, import and save agent settings, complete OAuth or API-key logins, and bind host secrets to allowed hosts. - 10. [Manage toolchains and custom layers](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/10-manage-toolchains-and-custom-layers.md) - Add, remove, and update shared mise toolchains (node, python, rust, go, dotnet) and author custom image layers with network policies via devvm layer new. - 11. [Use devcontainer.json and OCI base images](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/11-use-devcontainer.json-and-oci-base-images.md) - Translate .devcontainer/devcontainer.json properties into VM plans and build bootable VM base images from OCI/Docker container references with --from. - 12. [Connect VMs with links, groups, and port forwards](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/12-connect-vms-with-links-groups-and-port-forwards.md) - Forward guest ports to host 127.0.0.1, publish ports and link VMs under *.devvm hostnames, and join VMs into mutually reachable groups. - 13. [Connect SSH, VS Code, and VNC desktop](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/13-connect-ssh-vs-code-and-vnc-desktop.md) - Use generated ~/.ssh/config entries, configure per-VM SSH agents, launch VS Code Remote-SSH via devvm code, and connect to the VNC desktop relay. - 14. [Isolate work with repository clones](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/14-isolate-work-with-repository-clones.md) - Create VMs with independent Git clones using devvm --clone, switch existing VMs between mounted folders and clones, and fetch VM branches to the host. - 15. [CLI reference: VM and workspace commands](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/15-cli-reference-vm-and-workspace-commands.md) - Syntax, flags, exit codes, and examples for devvm (up), ls, restart, stop, update, rm, adopt, size, ports, host, code, desktop, and history. - 16. [CLI reference: network, security, and agent commands](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/16-cli-reference-network-security-and-agent-commands.md) - Syntax, subcommands, and flags for devvm network, secret, agent, git-protect, clone, ssh-agent, link, and group. - 17. [CLI reference: images, storage, and host commands](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/17-cli-reference-images-storage-and-host-commands.md) - Syntax, subcommands, and flags for devvm build, rebuild, images, layer, tools, space, gc, retention, setup, doctor, upgrade, completion, and limactl. - 18. [Configuration (config.toml) and --json schema reference](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/18-configuration-config.toml-and---json-schema-reference.md) - Complete reference for ~/.config/devvm/config.toml keys, settings.json precedence, environment variables, state directory layout, and version 1 --json payloads. - 19. [Diagnose host, boot, and SSH failures](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/19-diagnose-host-boot-and-ssh-failures.md) - Resolve devvm doctor host problems, Ubuntu cloud image boot fallbacks, SSH host-key verification errors, run logs, and moved project directories with devvm adopt. - 20. [Troubleshoot network policies, CAs, and disk space](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/20-troubleshoot-network-policies-cas-and-disk-space.md) - Inspect blocked requests with devvm network log, configure corporate HTTP/HTTPS proxies and custom CAs in ~/.config/devvm/ca, and recover disk space. - 21. [Manage disk retention, garbage collection, and upgrades](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/21-manage-disk-retention-garbage-collection-and-upgrades.md) - Configure automatic retention policies for run logs, agent history exports, unused tools, orphaned state, and Nix caches, and upgrade devvm installations. - 22. [Codebase invariants, testscript stubs, and E2E suite](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/22-codebase-invariants-testscript-stubs-and-e2e-suite.md) - Enforce single-owner and layer rules in internal/structure, write deterministic CLI testscript tests with internal/stubtest, and run real-VM E2E suites. - 23. [Pinning, release signing, and supply-chain verification](https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/23-pinning-release-signing-and-supply-chain-verification.md) - Maintain SHA-256 download pins and Ubuntu PGP archive checks with scripts/bump.sh, build and sign releases with Minisign, and publish artifacts. ## Source File Index - `.goreleaser.yaml` - `assets/base.yaml` - `assets/oci/build.sh` - `assets/prelude.sh` - `assets/shared-tools/build.sh` - `assets/shared-tools/mise.toml` - `cmd/devvm/main.go` - `install.sh` - `internal/agentlogin/restore.go` - `internal/backend/backend.go` - `internal/backend/caps.go` - `internal/backend/guard.go` - `internal/backend/lima/check.go` - `internal/backend/lima/lima.go` - `internal/backend/lima/oci.go` - `internal/backend/lima/seatbelt.go` - `internal/backend/lima/start.go` - `internal/build/build.go` - `internal/build/layer.go` - `internal/cli/agent.go` - `internal/cli/agentlogin.go` - `internal/cli/app.go` - `internal/cli/args.go` - `internal/cli/build.go` - `internal/cli/clone.go` - `internal/cli/code.go` - `internal/cli/desktop.go` - `internal/cli/doctor.go` - `internal/cli/flags.go` - `internal/cli/gc.go` - `internal/cli/gitprotect.go` - `internal/cli/group.go` - `internal/cli/images.go` - `internal/cli/json.go` - `internal/cli/layer.go` - `internal/cli/link.go` - `internal/cli/ls.go` - `internal/cli/network.go` - `internal/cli/ports.go` - `internal/cli/restart.go` - `internal/cli/retention.go` - `internal/cli/rm.go` - `internal/cli/root.go` - `internal/cli/script_test.go` - `internal/cli/secret.go` - `internal/cli/setup.go` - `internal/cli/size.go` - `internal/cli/space.go` - `internal/cli/sshagent.go` - `internal/cli/tools.go` - `internal/cli/up.go` - `internal/cli/update.go` - `internal/cli/upgrade.go` - `internal/desktop/desktop.go` - `internal/desktop/relay.go` - `internal/devcontainer/devcontainer.go` - `internal/devcontainer/reread.go` - `internal/fetch/fetch.go` - `internal/from/from.go` - `internal/from/registry.go` - `internal/gitprotect/gitprotect.go` - `internal/gitprotect/own.go` - `internal/hostgit/hostgit.go` - `internal/hostsafety/hostsafety.go` - `internal/image/key.go` - `internal/image/recipe.go` - `internal/image/tool.go` - `internal/netpolicy/log.go` - `internal/netpolicy/netpolicy.go` - `internal/netpolicy/rules.go` - `internal/netpolicy/tier.go` - `internal/paths/dirs.go` - `internal/paths/vm.go` - `internal/ports/ports.go` - `internal/project/project.go` - `internal/proxy/links.go` - `internal/proxy/policy.go` - `internal/proxy/proxy.go` - `internal/proxy/secrets.go` - `internal/proxy/sni.go` - `internal/secrets/secrets.go` - `internal/settings/companyproxy.go` - `internal/settings/config.go` - `internal/settings/retention.go` - `internal/settings/settings.go` - `internal/settings/values.go` - `internal/sshagent/sshagent.go` - `internal/sshconfig/sshconfig.go` - `internal/state/layout.go` - `internal/state/vm.go` - `internal/structure/owners_test.go` - `internal/structure/rules_test.go` - `internal/structure/structure_test.go` - `internal/stubtest/stubtest.go` - `internal/tools/share.go` - `internal/tools/tools.go` - `internal/trust/ca.go` - `internal/trust/trust.go` - `internal/ubuntu/archive.go` - `internal/ubuntu/pgp.go` - `internal/ui/runlog.go` - `internal/upgrade/install.go` - `internal/upgrade/minisign.go` - `internal/upgrade/source.go` - `internal/upgrade/upgrade.go` - `internal/vm/agentimport.go` - `internal/vm/agentlogin.go` - `internal/vm/agents.go` - `internal/vm/agentsave.go` - `internal/vm/clone.go` - `internal/vm/detect.go` - `internal/vm/devcontainer.go` - `internal/vm/env.go` - `internal/vm/exports.go` - `internal/vm/finder.go` - `internal/vm/groups.go` - `internal/vm/housekeep.go` - `internal/vm/lifecycle.go` - `internal/vm/links.go` - `internal/vm/move.go` - `internal/vm/parts.go` - `internal/vm/remake.go` - `internal/vm/start.go` - `internal/vm/tools.go` - `internal/vm/up.go` - `internal/vmname/vmname.go` - `internal/vmssh/checked.go` - `internal/vmssh/forwards.go` - `internal/vmssh/hostkeys.go` - `mise.toml` - `packaging/homebrew/devvm.rb` - `packaging/key.go` - `README.md` - `scripts/bump.sh` - `scripts/publish.sh` - `scripts/release.sh` - `tests/e2e/sandbox_test.go` --- ## 01. Overview > Public entry points, per-project VM sandbox architecture, runtime assumptions, and primary documentation routes. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/01-overview.md - Generated: 2026-11-07T21:40:58.913Z ### Source Files - `cmd/devvm/main.go` - `internal/cli/root.go` - `internal/cli/app.go` - `internal/vm/env.go` - `README.md` --- title: "Overview" description: "Public entry points, per-project VM sandbox architecture, runtime assumptions, and primary documentation routes." --- `devvm` is a Go CLI (`cmd/devvm`) that provisions and manages one isolated virtual machine per project directory on macOS and Linux using Lima (`internal/backend/lima`). Each project VM clones a content-addressed Ubuntu or OCI base image, mounts shared toolchains and coding agents read-only from host EROFS images, confines outbound network traffic through a per-VM host proxy (`devvm _proxy`), and stores all runtime state under `~/.local/share/devvm` and `~/.config/devvm` without writing files into the project workspace. ## System architecture `devvm` separates CLI command dispatch and terminal interaction (`internal/cli`, `internal/ui`) from VM lifecycle orchestration (`internal/vm`), host-side security boundaries (`internal/netpolicy`, `internal/proxy`, `internal/gitprotect`, `internal/secrets`), and the hypervisor driver contract (`internal/backend`). ```mermaid flowchart TB subgraph CLI["CLI & UI Layer (cmd/devvm, internal/cli, internal/ui)"] Main["cmd/devvm/main.go\nplatform.CloseInheritedOnExec() + ui.Main"] Root["internal/cli.NewRoot\nCobra command tree + checkCommands"] App["internal/cli.app\nLazy singletons (made[T])"] end subgraph Core["VM & Policy Orchestration (internal/vm, internal/netpolicy)"] VMEnv["vm.Env\nfinder, starter, Launch, Lifecycle, Tools, Links"] Starter["starter.bringUp\nhostChecks, requirements, confinementFor"] ProxyMgr["netpolicy.Proxy\ndevvm _proxy daemon + SelfCheck"] Forwarder["vmssh.Forwards\ndevvm _ports SSH loopback forwarder"] end subgraph State["Host State & Config (internal/paths, internal/state, internal/settings)"] ConfigDir["~/.config/devvm\nconfig.toml, layers/, policy, ca/, agents/"] DataDir["~/.local/share/devvm\nsettings.json, vms//vm.json, images/, tools/, logins/"] end subgraph BackendLayer["Backend Contract (internal/backend)"] Registry["backend.Registry & Review.Require\nCapability guard (Caps)"] Lima["lima.Backend\nHost, Images, Machines, Housekeeping"] end subgraph Guest["Project VM Sandbox (Lima Guest)"] MountRW["Project Folder Mount (read-write)\nor Independent Git Clone (--clone)"] ToolsRO["Shared Tools Mount (read-only)\nEROFS images at /opt/devvm/tools"] AgentRO["Agent Settings Mounts (read-only)\nPlaceholder credentials"] end Main --> Root --> App App --> VMEnv VMEnv --> Starter Starter --> ProxyMgr Starter --> Forwarder Starter --> Registry --> Lima VMEnv --> ConfigDir VMEnv --> DataDir Lima --> Guest Guest -- "HTTP/TLS via 127.0.0.1 proxy port" --> ProxyMgr ``` ## Public entry points and execution flow ### Binary entry point (`cmd/devvm/main.go`) The `devvm` binary serves both user-facing commands and internal background daemons. Execution begins in `cmd/devvm/main.go`: ```go title="cmd/devvm/main.go" func main() { platform.CloseInheritedOnExec() os.Exit(ui.Main(func(ctx context.Context, u *ui.UI) int { return cli.Execute(ctx, u, os.Args[1:]) })) } ``` 1. **Descriptor hygiene:** `platform.CloseInheritedOnExec()` marks any file descriptors above standard input (`0`), standard output (`1`), and standard error (`2`) close-on-exec so child processes (`limactl`, `ssh`, `git`, or background daemons) never inherit leaked descriptors. 2. **Terminal & signal lifecycle:** `ui.Main` constructs the run-scoped `*ui.UI` instance (`internal/ui`), traps `SIGINT`, `SIGTERM`, and `SIGHUP` to cancel the root context, waits for any active `LoggedCmd` subprocess to finish, executes the LIFO cleanup stack (`AddCleanup`), and returns the process exit code to `main`, which holds the sole `os.Exit` call in the repository. 3. **Command execution:** `cli.Execute` builds the Cobra command tree via `NewRoot(u)` and runs pre-execution validation before dispatching `RunE`. ### Service initialization (`internal/cli/app.go`) `NewRoot` creates a single `app` struct that holds lazy-initialized services wrapped in `made[T]`. Each service is constructed at most once per CLI invocation when requested by a command: | Service accessor | Underlying type | Responsibility | | --- | --- | --- | | `a.dirs()` | `paths.Dirs` | Resolves `$HOME`, `$XDG_DATA_HOME/devvm`, `$XDG_CONFIG_HOME/devvm`, `$DEVVM_LIMA_HOME`, and `$XDG_CACHE_HOME` | | `a.secrets()` | `*secrets.Store` | Manages host-stored API tokens, proxy passwords, and OAuth credentials (macOS Keychain or `~/.local/share/devvm/secrets`) | | `a.registry()` | `*backend.Registry` | Registers VM backends (`lima`), applies corporate proxy settings, and enforces capability checks | | `a.tools()` | `*tools.Env` | Manages `mise.lock` resolution and read-only shared tool image builds | | `a.vmEnv()` | `*vm.Env` | Wires `finder`, `starter`, `Launch`, `Lifecycle`, `Tools`, `Groups`, `Links`, `Proxy`, and `Forwarders` for project VM operations | | `a.builder()` | `*build.Env` | Builds and rebuilds base and layer VM images on the default backend | ### Pre-run invariants and exit codes Every command registered under `NewRoot` passes through two gates before its `RunE` handler executes: 1. **Static argument and flag validation (`checkCommands` in `internal/cli/args.go`):** Wraps every command's `Args` validator to reject blank arguments, enforce mutually exclusive flag groups (`exclusive`), and validate flag formats (`--cpus`, `--memory`, `--disk`, `--ports`, `--network`, `--tools`, `--agents`, `--layers`, `--lifetime`, `--older-than`, `--to`) prior to touching host state or looking up a VM. 2. **Workspace privacy and layout version check (`PersistentPreRunE` in `internal/cli/root.go`):** - Calls `u.DataOnly()` when `--json` is passed so informational messages route to `stderr` and `stdout` emits only versioned JSON (`"version": 1`). - Enforces `0700` permissions on the config and data directories via `state.MakePrivate`. - Verifies `~/.local/share/devvm/version` and `~/.config/devvm/version` via `state.CheckLayout` (skipped only for `help`, `version`, `completion`, `upgrade`, and hidden `_` commands). - Runs background retention cleanup (`autoRetain`) when due. | Exit code | Meaning | Trigger condition | | --- | --- | --- | | `0` | Success | Command completed without error | | `1` | Runtime failure | `RunE` or `checkLayout` returned an error (`ranError`), or an interactive prompt was refused under `--no-input` / non-TTY stdin (`msg.Refusal`) | | `2` | Usage error | Unknown command/flag, invalid flag value, mutually exclusive flags, or wrong argument count (`usageCode`) | | `130` | Interrupted | Process received `SIGINT`, `SIGTERM`, or `SIGHUP`, ran registered cleanups, and exited | ### Command surface organization Running `devvm` with no subcommand executes `up` (`internal/cli/up.go`), opening an interactive shell at the same path inside the current folder's VM after creating and starting the VM if needed. Subcommands are organized into six functional groups in `internal/cli/help.go`, alongside internal daemon entry points: | Category | Commands | Purpose | | --- | --- | --- | | **Your VM** | `devvm` (root `up`), `restart`, `stop`, `update`, `rm`, `adopt`, `ls`, `code`, `desktop`, `host`, `history` | Create, enter, stop, delete, adopt moved folders, list VMs, connect editors/VNC, and export agent history | | **Change your VM** | `tools`, `size`, `ports`, `network`, `ssh-agent`, `git-protect`, `clone`, `secret`, `layer add`, `layer rm` | Reconfigure a VM's mounted toolchains/agents, CPU/RAM/disk, port forwards, network tier, `.git` protection, clone mode, secrets, and image layers | | **Links between VMs** | `link`, `group` | Publish guest ports and connect VMs across `*.devvm` hostnames or mutually reachable groups | | **Images** | `images`, `build`, `rebuild`, `layer`, `layer new` | Inspect, build, and rebuild base and layer images, or scaffold custom layers in `~/.config/devvm/layers` | | **Disk space** | `space`, `gc`, `retention` | Report disk usage across VMs/images/tools/caches, garbage-collect unused artifacts, and configure automatic retention policies | | **This machine** | `setup`, `agent`, `doctor`, `upgrade`, `completion`, `limactl` | Configure global defaults, import/login/save coding agent state, verify host dependencies, self-upgrade, and pass commands to Lima | | **Internal daemons** | `_proxy`, `_ports`, `_links`, `_qemu`, `_refresh-tools`, `_msg`, `version` | Hidden subprocess entry points spawned by `devvm` for per-VM HTTP/TLS proxying, SSH port forwarding, inter-VM links, Linux QEMU netns wrapping, and daily tool checks | ## Per-project VM sandbox architecture ### Project path to VM identity mapping `devvm` maps each project folder to a deterministic VM name and state directory (`internal/project/project.go`, `internal/vmname/vmname.go`): - **Physical path resolution:** The project directory is resolved to its physical path (`pwd -P` / `paths.Physical`). Paths containing commas or control characters are rejected (`project.BadPathError`) because Lima splits mount options on commas. - **VM naming grammar (`devvm--`):** - `` is derived from `filepath.Base(dir)` converted to lowercase `a-z0-9` with non-alphanumeric runs replaced by `-`, trimmed to at most 20 characters (`slugMax = 20`, minimum 4 characters, falling back to `root`). - `` is the first 14 lowercase hexadecimal digits (56 bits) of the SHA-1 digest of the physical directory path (`vmname.HashLen = 14`). This 14-hex length distinguishes project VMs from image build VMs (`devvm-img-<12-hex>`). - **Parent folder resolution:** When run inside a subdirectory that does not have its own VM, `finder` searches parent directories up to (but excluding) `$HOME` and `/` unless `--here` is passed (`--parent` forces using an ancestor VM). - **Moved folder detection:** Each VM's `vm.json` records both the physical path (`Dir`) and the filesystem device/inode identity (`Folder` via `platform.FolderID`). If a project folder is moved or replaced on the host, `starter.bringUp` detects the mismatch via `folderChanged` and refuses to mount an untrusted replacement path; `devvm adopt` rebinds a VM to a moved directory. ### Layered images and read-only shared tools Project VMs do not install toolchains or agents into their writable root disks on creation: - **Base and layer images (`internal/image`, `internal/build`):** Every VM clones an image identified by a deterministic SHA-256 content key (`12` hex chars). The `base` image starts from a pinned Ubuntu cloud image (or an OCI container image via `--from`) and optionally stacks built-in layers (`docker`, `nix`, `desktop`) or custom user layers (`~/.config/devvm/layers/.sh`). Adding or removing layers (`devvm layer add`, `devvm layer rm`), switching `--from`, or running `devvm update` performs a remake (`internal/vm/remake.go`), preserving the VM's `/home` directory across the new image while keeping the old machine aside as `devvm--old` until the remake succeeds. - **Shared toolchains and coding agents (`internal/tools`, `assets/shared-tools`):** - **Supported toolchains:** `node`, `python`, `rust`, `go`, `dotnet` (auto-detected from project files such as `package.json`, `pyproject.toml`, `Cargo.toml`, `go.mod`, `*.csproj` / `*.sln`, or `.devcontainer/devcontainer.json`). - **Supported coding agents:** `claude` (Claude Code), `codex` (Codex), `opencode`. - Toolchains and agents are built inside throwaway VMs into compressed EROFS images stored in `~/.local/share/devvm/tools` and mounted read-only into project VMs. Writes such as `npm install -g` or `cargo install` are redirected to the guest's `$HOME` (`~/.npm-global`, `go_set_gobin = false`). ### Filesystem, Git, network, and credential isolation `starter.bringUp` (`internal/vm/start.go`) enforces four isolation boundaries before any shell or command runs inside a VM: 1. **Workspace mount vs. repository clone:** - **Mounted folder (default):** Only the project directory is mounted read-write at the identical path inside the VM. `project.Unsafe` refuses to create or start a VM whose project path is, contains, or resides inside `$HOME`, `~/.config/devvm`, `~/.local/share/devvm`, or `$DEVVM_LIMA_HOME`. - **`.git` protection (`internal/gitprotect`):** On mounted folders, `devvm` locks `.git` hooks, config, and attributes on the host so guest processes cannot plant malicious Git hooks or filters that execute on the host. On Linux hosts, `checkSetID` audits the project folder for setuid/setgid files when starting or stopping a VM. - **Isolated clone (`devvm --clone`):** The VM mounts no project folder from the host; instead, `devvm` clones the repository into the VM's disk over SSH and configures a `devvm` Git remote on the host so changes are pulled only via explicit `git fetch devvm`. 2. **Network confinement tiers (`internal/netpolicy/tier.go`):** - `allowlist` (default): Confines the VM to `backend.NetProxy`. Guest traffic can reach only the host-side `devvm _proxy` listener, which enforces built-in base/tool/layer policies, global rules (`~/.config/devvm/policy`), per-VM rules (`~/.local/share/devvm/vms//policy`), and secret target hosts. - `none`: Confines the VM to `backend.NetProxy` and permits only traffic to configured coding agent endpoints, bound secret hosts, and linked VMs (`*.devvm`). - `internet`: Confines the VM to `backend.NetProxy` and permits public internet hosts while blocking private/loopback ranges and applying user `deny` rules. - `open`: Starts the VM on `backend.NetDirect` with `HTTP_PROXY`/`HTTPS_PROXY` pointed at `devvm _proxy` for credential injection without rule filtering. - After starting a VM, `Proxy.SelfCheck` verifies from inside the guest that direct connections bypassing the proxy fail on confined tiers, stopping the VM immediately if confinement does not hold. 3. **Host-side secret and login substitution (`internal/secrets`, `internal/agentlogin`):** - Coding agent settings in `~/.config/devvm/agents/` are mounted read-only or copied into the guest. - OAuth logins (`~/.local/share/devvm/logins/`) and user secrets (`devvm secret`) never enter the VM in plaintext. Guest files receive random per-VM salted placeholders (`netpolicy.Salt`), and the host-side `devvm _proxy` performs TLS termination using a per-VM CA (`~/.local/share/devvm/vms//run/ca`) to inject the real credentials only on requests to authorized destination hosts. 4. **SSH host-key pinning and agent isolation (`internal/vmssh`, `internal/sshagent`):** - `vmssh.FromTarget` pins each VM's SSH host keys per boot in `vms//run/known_hosts` and verifies all host-to-guest SSH connections, port forwarders (`devvm _ports`), and inter-VM links (`devvm _links`). - By default, host SSH agent forwarding is disabled; users can enable `devvm`'s isolated SSH agent (`devvm ssh-agent`) or forward their host `SSH_AUTH_SOCK`. ## Runtime assumptions and host layout ### Backend contract (`internal/backend/backend.go`) All hypervisor interactions go through `backend.Backend`, which comprises `Name()`, `Caps()`, and four role interfaces: - `Host`: checks host readiness (`Problems`), VM name length limits (`NameMax`), storage locations (`Location`), and minimum free disk space (`DiskDirs`, `Floors`). - `Images`: builds and deletes base, layer, and shared-tool EROFS images (`BuildImage`, `BuildTools`, `PinOrigin`). - `Machines`: creates, starts, stops, deletes, renames, lists, inspects, and opens shells/SSH sessions in VMs (`CreateVM`, `Start`, `Running`, `Stop`, `Delete`, `Exec`, `Shell`, `SSH`). - `Housekeeping`: reports VM/cache disk usage (`Usage`), identifies untracked VMs (`Strays`), and cleans build caches (`DevvmCache`, `DeleteCache`). Callers verify required features (`backend.Feature`) via `Registry.Require(b, vm, features...)` before invoking backend methods. If a backend lacks a required capability or relies on an unreviewed emulation, `devvm` fails immediately with an actionable message rather than silently degrading isolation. ### Host requirements - **Operating system and architecture:** macOS 13 or newer (`arm64` or `x86_64`) or Linux (`arm64` or `x86_64` with `/dev/kvm` access and QEMU). - **Lima runtime:** Lima 2.2 or newer (`limactl` on `PATH`). `devvm` isolates its Lima instances under `$DEVVM_LIMA_HOME` (default `~/.local/share/devvm/lima`) and ignores the ambient `LIMA_HOME` environment variable so external Lima configurations cannot alter `devvm` guests. - **Host readiness check:** `devvm doctor` inspects the host through `Backend.Problems` and reports missing dependencies or permission issues without installing software automatically. ### Host directory layout and configuration precedence `devvm` reads and writes state exclusively under the XDG config and data directories (`internal/paths/dirs.go`): :::files ~/.config/devvm/ # $XDG_CONFIG_HOME/devvm (mode 0700) ├── version # Config layout version ├── config.toml # User-authored global defaults (read-only to devvm) ├── policy # Global network allow/deny rules for all VMs ├── ca/ # Custom/corporate root CAs trusted by VMs and builds ├── layers/ # User-defined layer scripts (.sh and .policy) └── agents/ # Imported coding agent settings, skills, and MCP configs ~/.local/share/devvm/ # $XDG_DATA_HOME/devvm (mode 0700) ├── version # Data layout version ├── settings.json # CLI-managed global defaults (devvm setup, -g flags) ├── ssh_config # Generated SSH config included from ~/.ssh/config ├── mise.lock # Resolved versions of shared toolchains and agents ├── tools/ # Read-only EROFS images (-.erofs) and records ├── images/ # Image build records, locks, rebuild counters, ubuntu.json ├── packages/ # Content-addressed .deb cache for base image builds ├── logins/ # Host-only coding agent OAuth tokens and auth files ├── secrets/ # Fallback secret files on hosts without OS Keychain ├── lima/ # Isolated Lima home ($DEVVM_LIMA_HOME) ├── vms/ │ └── <14-hex-hash>/ # Per-project VM directory │ ├── vm.json # VM state (folder path/ID, image, tools, net, ports, size) │ ├── policy # Per-VM network policy rules │ ├── secrets.policy # Auto-generated rules for bound secret hosts │ └── run/ # Per-boot CA keys, known_hosts, proxy/forwarder PIDs & logs ├── run/ # Global locks (claims.lock, create-.lock), sockets, QEMU wrapper ├── exports/ # Agent history archives (.tar.gz and .folder metadata) ├── cache/ # Host mise binaries, shared nix-cache, --from digests, release.json └── logs/ # Command run logs (runs/) and daily tools-check.log ::: Global defaults for new VMs (`network`, `ports`, `size.cpus`, `size.memory`, `size.disk`, `features`, `agents`, `layers`, and `retention.*`) follow strict precedence in `internal/settings/settings.go`: 1. **CLI flags** (`--network`, `--ports`, `--cpus`, `--memory`, `--disk`, `--tools`, `--agents`, `--layers`) 2. **`~/.config/devvm/config.toml`** (edited manually; `devvm` never overwrites it and refuses `-g` commands that conflict with keys defined in `config.toml`) 3. **`~/.local/share/devvm/settings.json`** (updated by `devvm setup`, `devvm size -g`, `devvm ports -g`, `devvm network -g`, and `devvm retention`) 4. **Built-in defaults** (`network = "allowlist"`, `ports = "all"`, image default CPU/RAM/disk) ## Next Install devvm on macOS and Linux via Homebrew or the signed release installer, verify Minisign signatures, and validate host readiness with devvm doctor. Configure initial defaults with devvm setup, launch a project VM with auto-detected toolchains and agents, and verify isolation in the guest shell. How project paths map to VM names and hashes, state persistence in vm.json, layout version checks, and in-place vs. remake transitions. Deterministic SHA-256 key computation for base images, built-in and custom layers (docker, nix, desktop), and read-only EROFS shared tool images. Traffic confinement across allowlist, open, and offline tiers, host-side TLS SNI and HTTP inspection via devvm _proxy, and per-VM CA rotation. Read-only .git mount protection, symlink and hardlink containment, set-id file audits, secret placeholder injection, and isolated repo clones. The four-role backend.Backend interface (Host, Images, Machines, Housekeeping), capability requirement checks (Caps), and the Lima driver implementation. Complete reference for ~/.config/devvm/config.toml keys, settings.json precedence, environment variables, state directory layout, and version 1 --json payloads. --- ## 02. Installation > Install devvm on macOS and Linux via Homebrew or the signed release installer, verify Minisign signatures, and validate host readiness with devvm doctor. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/02-installation.md - Generated: 2026-11-07T21:44:12.029Z ### Source Files - `install.sh` - `packaging/homebrew/devvm.rb` - `packaging/key.go` - `internal/cli/doctor.go` - `internal/upgrade/minisign.go` - `internal/backend/lima/check.go` --- title: "Installation" description: "Install devvm on macOS and Linux via Homebrew or the signed release installer, verify Minisign signatures, and validate host readiness with devvm doctor." --- `devvm` ships as a static Go binary (`CGO_ENABLED=0`) with all guest provisioning scripts, base Lima templates (`assets/base.yaml`), built-in layer definitions, and default network policies embedded directly into the executable via `assets.FS`. You can install `devvm` on macOS and Linux (`arm64` and `amd64`) through the `devvmhq/tap` Homebrew tap or the standalone `install.sh` release script, verify release artifacts against the maintainer's Ed25519 Minisign public key, and validate hypervisor and host readiness with `devvm doctor`. ## Host requirements `devvm` runs virtual machines through its `lima` backend (`internal/backend/lima`). Every command that interacts with VMs runs preflight host checks (`host.Problems`) and stops immediately if a required host capability is missing, without installing host software unprompted. | Requirement | Minimum / Value | Verification command | Automated by `install.sh` | | --- | --- | --- | --- | | Operating system | macOS 13 or newer (`minMacOS = "13"`) | `sw_vers -productVersion` | No | | Architecture | `arm64` (Apple silicon) or `amd64` (Intel) | `uname -m` | No | | Lima | Lima 2.2 or newer (`minLima = "2.2"`) | `limactl --version` | Yes (`brew install lima` or tarball into `$DEVVM_PREFIX`) | | Recommended resources | 4 CPUs, 8 GiB RAM, 8 GiB free disk | `sysctl hw.ncpu hw.memsize` | No (warns only; smaller hosts scale down base VM size) | | OpenSSH | OpenSSH 8.7+ (included in macOS 13+) | `ssh -V` | No (required for `ssh ` and editor integration) | | Docker Desktop | Optional | `docker version` | No (only required for `--from` with local-only Docker images) | | Requirement | Minimum / Value | Verification command | Automated by `install.sh` | | --- | --- | --- | --- | | Operating system | Linux (`arm64` / `aarch64` or `amd64` / `x86_64`) | `uname -s && uname -m` | No | | Lima | Lima 2.2 or newer (`minLima = "2.2"`) | `limactl --version` | Yes (`brew install lima` or tarball into `$DEVVM_PREFIX`) | | QEMU | `qemu-system-aarch64` or `qemu-system-x86_64`, plus `qemu-img` | `qemu-img --version` | Yes (`apt-get`, `dnf`, `pacman`, or `zypper` via `sudo`) | | KVM device access | Read/write access to `/dev/kvm` | `test -r /dev/kvm -a -w /dev/kvm` | Yes (`sudo usermod -aG kvm $USER`; requires re-login) | | Recommended resources | 4 CPUs, 8 GiB RAM, 8 GiB free disk | `nproc && free -g` | No (warns only; smaller hosts scale down base VM size) | | OpenSSH | OpenSSH 8.7+ | `ssh -V` | No (required for `ssh ` and editor integration) | | Docker Engine | Optional | `docker version` | No (only required for `--from` with local-only Docker images) | On Apple silicon Macs running an `x86_64` shell under Rosetta, `install.sh` queries `sysctl -in hw.optional.arm64` and installs the native `darwin-arm64` binary. ## Install `devvm` Install `devvm` using either Homebrew or the release installer script. Do not install both simultaneously; having two copies on `PATH` can cause `brew upgrade` to update the inactive binary while the other remains first on `PATH`. The `devvmhq/tap` repository provides two mutually exclusive formulas built from `packaging/homebrew/devvm.rb`: - `devvm`: tracks stable releases (`x.y.z`). - `devvm-rc`: tracks the newest release or prerelease (`x.y.z-rc.N`, `x.y.z-beta.N`). Both formulas declare `depends_on "lima"`, installing Lima automatically if it is not already present. ```bash brew trust --formula devvmhq/tap/devvm brew install devvmhq/tap/devvm ``` To follow release candidates and betas instead of stable releases: ```bash brew trust --formula devvmhq/tap/devvm-rc brew install devvmhq/tap/devvm-rc ``` The standalone `install.sh` script requires `curl`, `tar`, and either `sha256sum` or `shasum`. It installs `devvm` into `~/.local/lib/devvm/` and symlinks `~/.local/bin/devvm` without requiring `sudo`. ```bash curl -fsSL https://github.com/devvmhq/devvm/releases/latest/download/install.sh | bash ``` Pass flags to `install.sh` after `bash -s --`: ```bash curl -fsSL https://github.com/devvmhq/devvm/releases/latest/download/install.sh | bash -s -- --yes ``` ### Installer options and environment variables `install.sh` accepts the following command-line flags and environment variables: Answer yes to every interactive dependency repair prompt (`lima`, `qemu`, and `kvm-access`). Required when running non-interactively without a controlling `/dev/tty` if host dependencies are missing. Install a specific release or prerelease tag (for example, `0.1.0` or `0.2.0-rc.1`) instead of resolving the latest release. Equivalent to setting `DEVVM_VERSION`. Install only the `devvm` binary archive and skip running `devvm doctor` dependency checks and remediation prompts. Bypass the Homebrew collision check (`check_brew`) and install a second copy of `devvm` even when a Homebrew keg exists in `Cellar/devvm` or `Cellar/devvm-rc`. Base installation directory. `install.sh` extracts release archives to `$DEVVM_PREFIX/lib/devvm/` and links the executable at `$DEVVM_PREFIX/bin/devvm`. Target release version to install when `--version` is not passed on the command line. Base GitHub repository URL used to resolve `/releases/latest` and download release archives. Base GitHub repository URL used when downloading Lima release tarballs on hosts without Homebrew. ### Installer execution flow and filesystem layout When `install.sh` runs, it performs five sequential operations: `check_host` verifies that the OS/architecture pair is supported (`darwin` or `linux` on `arm64` or `amd64`) and checks `PATH` for `curl`, `tar`, and `sha256sum` or `shasum`. Next, `check_brew` inspects `HOMEBREW_CELLAR`, `HOMEBREW_PREFIX/Cellar`, `/../Cellar`, `/opt/homebrew/Cellar`, `/usr/local/Cellar`, and `/home/linuxbrew/.linuxbrew/Cellar` for `devvm` or `devvm-rc` keg directories without executing `brew`. If a keg exists and `--force` is not set, `install.sh` exits with status `1`. `resolve_version` determines the target version from `--version` / `DEVVM_VERSION`, the `release_version` stamped into the release's `install.sh`, or the HTTP redirect target of `$DEVVM_REPO_URL/releases/latest` (rejecting prerelease tags on `releases/latest`). It downloads `devvm---.tar.gz`, `SHA256SUMS`, and `SHA256SUMS.minisig` over HTTPS (`--proto =https --proto-redir =https`), verifies the Minisign signature when `minisign` is installed, and compares the archive's SHA-256 digest against `SHA256SUMS`. `install_devvm` extracts the archive into a temporary directory, moves it to `$DEVVM_PREFIX/lib/devvm/..new.$$`, points `$DEVVM_PREFIX/bin/devvm` at the staged binary, replaces `$DEVVM_PREFIX/lib/devvm/`, and updates the symlink to `$DEVVM_PREFIX/lib/devvm//devvm`. Older version directories under `$DEVVM_PREFIX/lib/devvm/` are removed unless `ps -A -ww -o command=` shows a running process (such as a VM's `devvm _proxy` or `devvm _qemu` wrapper) still executing from that path. Unless `--no-deps` is passed, `install_deps` runs `devvm doctor --json`, parses the `problems` array with `awk`, and prompts on `/dev/tty` (or auto-accepts with `--yes`) to install missing dependencies (`lima`, `qemu`, or `kvm-access`). `path_hint` checks whether `$DEVVM_PREFIX/bin` is in `PATH`. If missing, it prints the `export PATH` snippet for `~/.zshrc` or `~/.bashrc`. If present, it resolves `command -v devvm` with `readlink -f` and warns if another `devvm` binary precedes `$DEVVM_PREFIX/bin/devvm`. :::files ~/.local/ ├── bin/ │ ├── devvm -> ~/.local/lib/devvm//devvm │ └── limactl # Present when Lima is installed via tarball ├── lib/ │ └── devvm/ │ └── / │ ├── devvm # Static Go binary with embedded assets │ ├── README.md │ ├── CHANGELOG.md │ └── LICENSE └── share/ ├── devvm/ # Runtime state, images, VMs, and Lima home (created on first run) └── lima/ # Present when Lima is installed via tarball ::: ## Release signing and verification Every release is built from a signed Git tag verified against `packaging/release-signer` by `scripts/release.sh`. Before publishing, `scripts/release.sh` stamps `release_version` and `minisign_key` into `dist/install.sh`, appends the SHA-256 checksum of `install.sh` to `SHA256SUMS`, and signs `SHA256SUMS` with Minisign using the trusted comment `devvm `. ```mermaid flowchart TD subgraph Repo["Repository & Build (scripts/release.sh)"] PubKey["packaging/minisign.pub
Key ID: 8ADA44EC275BCCA3"] Embed["packaging/key.go
//go:embed *"] PubKey --> Embed end subgraph Release["GitHub Release Assets"] Script["install.sh
(stamped release_version & minisign_key)"] Sums["SHA256SUMS
(archives + install.sh digests)"] Sig["SHA256SUMS.minisig
(ED BLAKE2b-512 + global signature)"] Archive["devvm-<version>-<os>-<arch>.tar.gz"] end subgraph Tap["devvmhq/homebrew-tap"] Formula["Formula/devvm.rb & devvm-rc.rb
(pinned platform URLs & SHA-256)"] end PubKey -. "verifies" .-> Sig Sig -- "authenticates" --> Sums Sums -- "verifies SHA-256" --> Script Sums -- "verifies SHA-256" --> Archive Formula -- "verifies SHA-256 out-of-band" --> Archive Embed -- "compiled into devvm binary for devvm upgrade" --> Archive ``` ### Embedded Minisign public key The public key in `packaging/minisign.pub` (embedded into the `devvm` binary by `packaging/key.go`) is: ```text untrusted comment: minisign public key 8ADA44EC275BCCA3 RWSjzFsn7ETaijKJ2y7F4DtZobXmyv+LmdclJwZP/KmAkLUwOCzWF/2b ``` ### Verify `install.sh` before execution Piping `curl ... | bash` relies on the `install.sh` served by GitHub Releases to carry the public key. To eliminate trust in the release server before executing any shell code, verify `SHA256SUMS` and `install.sh` manually against the public key above: ```bash base=https://github.com/devvmhq/devvm/releases/latest/download curl -fsSL -O "$base/install.sh" -O "$base/SHA256SUMS" -O "$base/SHA256SUMS.minisig" ``` Requires `minisign` (`brew install minisign` or your Linux package manager): ```bash minisign -Vm SHA256SUMS -P RWSjzFsn7ETaijKJ2y7F4DtZobXmyv+LmdclJwZP/KmAkLUwOCzWF/2b ``` Expected output: ```text Signature and comment signature verified Trusted comment: devvm ``` ```bash macOS (shasum) shasum -a 256 -c --ignore-missing SHA256SUMS ``` ```bash Linux (sha256sum) sha256sum -c --ignore-missing SHA256SUMS ``` Expected output: ```text install.sh: OK ``` ```bash bash install.sh ``` ### Built-in signature verification during upgrades While `install.sh` delegates signature verification to the external `minisign` CLI if installed (and falls back to SHA-256 verification with a warning if `minisign` is absent), `devvm upgrade` verifies signatures natively in Go (`internal/upgrade/minisign.go` and `internal/upgrade/source.go`) using the key compiled into the running binary (`packaging.MinisignKey()`): 1. **Algorithm enforcement**: `PublicKey.Verify` accepts only prehashed `ED` signatures (Ed25519 over the BLAKE2b-512 digest of `SHA256SUMS`) and rejects legacy unhashed `Ed` signatures. 2. **Key ID matching**: The 8-byte key ID in the signature header must match `8ADA44EC275BCCA3`. 3. **Global signature and downgrade protection**: It verifies the Ed25519 global signature over `signature || trusted_comment` and requires the trusted comment to equal `devvm ` exactly, preventing a valid `SHA256SUMS` from an older release from being replayed for a different version. ## Validate host readiness with `devvm doctor` Run `devvm doctor` to inspect backend capabilities, host requirements, proxy certificate configuration, and workspace health: ```bash devvm doctor ``` `devvm doctor` exits with code `0` when all backend requirements are satisfied (even if warnings are present) and exits with code `1` when one or more backend problems exist. ### Problem keys and automated fixes `internal/backend/lima/check.go` evaluates the host and returns structured `backend.Problem` entries. Both `devvm doctor` and `install.sh` use these problem keys: | Problem key | Condition checked | Suggested `fix` command | `install.sh` action | | --- | --- | --- | --- | | `os` | `runtime.GOOS` is neither `darwin` nor `linux` | `null` | Prints message | | `arch` | `runtime.GOARCH` is neither `arm64` nor `amd64` | `null` | Prints message | | `macos` | macOS major version is below `13` | `null` | Prints message | | `lima` | `limactl` is missing from `PATH`, fails `limactl --version`, or is older than `2.2` | With `brew` on `PATH`: `brew install lima`, `brew reinstall lima`, or `brew upgrade lima`. Without `brew`: `null` | Prompts to run `brew` fix, or downloads and verifies the latest Lima release tarball into `$DEVVM_PREFIX` | | `qemu` | On Linux, `qemu-system-aarch64` (`arm64`) or `qemu-system-x86_64` (`amd64`) or `qemu-img` is missing | First package manager found on `PATH` (see table below), or `null` | Prompts to run `fix` via `/dev/tty` (`sudo`) | | `kvm` | On Linux, `/dev/kvm` does not exist (`os.Stat` fails) | `null` (enable virtualization in BIOS/UEFI or host hypervisor) | Prints message | | `kvm-access` | On Linux, `/dev/kvm` exists but `os.OpenFile("/dev/kvm", os.O_RDWR, 0)` fails | `sudo usermod -aG kvm ` | Prompts to run `fix`, then instructs you to log out and back in | | `lima-config` | `~/.local/share/devvm/lima/_config/default.yaml` or `override.yaml` exists | `null` (remove global config files from devvm's isolated Lima home) | Prints message | On Linux hosts missing QEMU, `qemuInstall` in `internal/backend/lima/check.go` selects the install command for the first package manager found on `PATH`: | Package manager | Architecture | Generated `fix` command | | --- | --- | --- | | `apt-get` | `amd64` | `sudo apt-get install -y --no-install-recommends qemu-utils ipxe-qemu qemu-system-x86 ovmf` | | `apt-get` | `arm64` | `sudo apt-get install -y --no-install-recommends qemu-utils ipxe-qemu qemu-system-arm qemu-efi-aarch64` | | `dnf` | `amd64` / `arm64` | `sudo dnf install -y qemu-kvm qemu-img` | | `pacman` | `amd64` / `arm64` | `sudo pacman -S --noconfirm qemu-base` | | `zypper` | `amd64` / `arm64` | `sudo zypper --non-interactive install qemu-kvm` | ### Non-fatal warnings and notices `devvm doctor` also checks for non-fatal conditions. Warnings are printed to stderr (or collected in the `warnings` array with `--json`) and never cause `devvm doctor` to exit with a non-zero code: - **Shadowed binary on `PATH`**: The first `devvm` found via `exec.LookPath("devvm")` resolves (`filepath.EvalSymlinks`) to a different file than `os.Executable()`. - **Low disk space**: The volume hosting `~/.local/share/devvm/lima` has less free space than the base build floor (`8 GiB`, defined in ` Lima.Floors().Base`). - **Undersized host**: Host RAM is below `8 GiB` (`recommendedMemoryGiB = 8`) or CPU count is below `4` (`recommendedCPUs = 4`). - **Invalid network policy lines**: Lines in `~/.config/devvm/policy`, `~/.config/devvm/layers/*.policy`, or `~/.local/share/devvm/vms/*/policy` that are not valid rules (`netpolicy.NotRules`). - **Stray entries in `vms/`**: Entries in `~/.local/share/devvm/vms/` that are not valid VM state directories (`state.VMFolderProblems`). - **Corporate proxy CA notice**: When `HTTPS_PROXY` or `config.toml`'s `proxy.https` is configured and `~/.config/devvm/ca` contains no `.crt`/`.pem` certificates, `devvm doctor` prints an informational notice reminding you to place TLS-inspecting proxy root CAs in `~/.config/devvm/ca`. ### Machine-readable output (`devvm doctor --json`) Passing `--json` outputs a versioned JSON payload to stdout and always exits `0` unless internal directory resolution fails. ```bash Command devvm doctor --json ``` ```json Output { "version": 1, "backend": "lima", "problems": [], "warnings": [], "docker": true, "features": { "lima": { "agent-folders": "native", "agent-forwarding": "native", "agent-toggle": "native", "build-confinement": "native", "build-image": "native", "clone": "native", "cpus": "native", "disk": "native", "docker-layer": "native", "egress-proxy": "native", "from": "native", "link": "native", "memory": "native", "nix-layer": "native", "ports": "emulated", "ports-all": "emulated", "project-mount": "native", "ssh": "native", "tools-share": "native" } }, "release": { "latest": "0.1.0", "checked": "2026-11-07T12:00:00Z" } } ``` JSON schema version, always `1`. Default backend used for new VMs (`"lima"`). Blocking backend requirements that are missing on this host. Each item contains `key` (`string`), `fix` (`string` or `null`), and `message` (`string`). Non-blocking warning messages (`[]string`) for `PATH` shadowing, low disk space, fewer than 4 CPUs or 8 GiB RAM, malformed policy lines, or stray directories in `vms/`. Whether the `docker` CLI is present on `PATH` (used only when building `--from` a local Docker image not present in a remote registry). Map of backend name to its 19 capability support levels (`"native"`, `"emulated"`, `"unreviewed"`, or `"none"`). On `lima`, `ports` and `ports-all` are `"emulated"` (reviewed and active via `vmssh.Forwards`), and all other 17 features are `"native"`. Result of the most recent background daily release check (`~/.local/share/devvm/release-check.json`), containing `latest` (`string`) and `checked` (RFC 3339 UTC timestamp), or omitted/`null` if no check has completed yet. ## Next Configure initial defaults with `devvm setup`, launch your first project VM, and verify isolation inside the guest shell. Resolve `devvm doctor` host problems, KVM permissions, Ubuntu cloud image boot fallbacks, and SSH host-key errors. Upgrade `devvm` installations with `devvm upgrade`, switch between stable and prerelease channels, or uninstall cleanly. Inspect how `scripts/release.sh` verifies signed tags, builds release archives, and signs `SHA256SUMS` with Minisign. --- ## 03. Quickstart > Configure initial defaults with devvm setup, launch a project VM with auto-detected toolchains and agents, and verify isolation in the guest shell. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/03-quickstart.md - Generated: 2026-11-07T21:46:36.788Z ### Source Files - `internal/cli/up.go` - `internal/cli/setup.go` - `internal/vm/up.go` - `internal/vm/detect.go` - `internal/vm/parts.go` - `internal/vm/start.go` --- title: "Quickstart" description: "Configure initial defaults with devvm setup, launch a project VM with auto-detected toolchains and agents, and verify isolation in the guest shell." --- `devvm` (the root command, implemented by `Launch.Up` in `internal/vm/up.go`) provisions an isolated, per-folder virtual machine cloned from a shared base image, mounts the host project directory read-write with `.git` protection locked, mounts shared EROFS toolchain and coding agent images read-only at `/opt/devvm/tools`, starts a per-VM host HTTP/TLS proxy (`devvm _proxy`), and replaces the CLI process with an interactive guest shell at the matching project path. Global defaults for the base image features and new VM resources, network tier, coding agents, and layers are persisted in `~/.local/share/devvm/settings.json` via `devvm setup` or overridden declaratively in `~/.config/devvm/config.toml`. ## Prerequisites Before creating a project VM, verify that the host hypervisor, Lima runtime (`limactl`), and filesystem permissions are ready: ```sh devvm doctor ``` `devvm doctor` inspects host virtualization support, Lima installation, `~/.ssh/config` integration, and state directory permissions without installing host software unprompted. If `devvm` is not yet installed, see [Installation](/installation). ## Configure initial defaults with `devvm setup` Running `devvm setup` configures the global defaults saved to `~/.local/share/devvm/settings.json`. If you run `devvm` in a project folder before running `devvm setup`, `creation.firstRun` automatically prompts for default VM size (`firstRunSize`), default coding agents (`firstRunAgents`), and `base` image features (`build.Choice.Choose`) before displaying the new VM plan. ### Interactive and non-interactive setup In an interactive terminal without flags, `devvm setup` prompts for five settings in order, starting from the current effective defaults (`settings.Read`): 1. **Base image features** (`features`): Optional shell and CLI utilities built into `base` (`cli-tools`, `fzf`, `omb`). Until explicitly chosen, all three features are enabled by default. 2. **Default VM size** (`size.cpus`, `size.memory`): Default CPU count and memory in GiB for new VMs (saved as `devvm size -g` saves them). Defaults to `base` fitted to the host: `min(4, hostCPUs)` CPUs and `8` GiB memory (or `hostGiB / 2` when `8 + 2 > hostGiB`), with a `100` GiB minimum disk. 3. **Default network tier** (`network`): Egress confinement mode for new VMs (`allowlist`, `internet`, `open`, or `none`; defaults to `allowlist`). 4. **Default coding agents** (`agents`): Shared coding agents mounted into every new VM (`claude`, `codex`, `opencode`, or none). 5. **Default image layers** (`layers`): Built-in (`docker`, `nix`, `desktop`) or custom layers from `~/.config/devvm/layers/` stacked bottom-first onto `base` for every new VM. ```sh title="Interactive terminal prompts" devvm setup ``` ```sh title="Non-interactive flag configuration" devvm setup --features cli-tools,fzf,omb --agents claude,codex --layers none ``` ```sh title="Minimal base image with no extras" devvm setup --features none --agents none --layers none ``` ### `devvm setup` options and base features Passing `--features`, `--agents`, or `--layers` applies the specified comma-separated list (or `none`) immediately without opening interactive pickers, leaving unspecified settings unchanged. Comma-separated list of `base` image features (`cli-tools`, `fzf`, `omb`) or `none`. Rejects empty strings (`--features ''`). Changing features marks `base` and dependent layer images for rebuild on next use. Comma-separated list of coding agents (`claude`, `codex`, `opencode`) or `none` that new VMs receive by default. Existing VMs keep their current agents. Comma-separated, bottom-first list of layers (`docker`, `nix`, `desktop`, or user layers in `~/.config/devvm/layers/`) or `none` that new VMs receive by default. | Component type | Identifier | Contents and behavior | | --- | --- | --- | | Base feature | `cli-tools` | `bat`, `fd`, `eza`, `zoxide` (`z`), `btop`, `tree`, `tldr` (`tealdeer`) | | Base feature | `fzf` | `fzf` package with `Ctrl-R` history and `Ctrl-T` file search wired into `~/.bashrc.d/fzf.sh` | | Base feature | `omb` | Pinned `oh-my-bash` commit with themed prompt, completions, and aliases | | Coding agent | `claude` | Claude Code shared EROFS tool image symlinked at `~/.local/bin/claude` | | Coding agent | `codex` | OpenAI Codex CLI shared EROFS tool image with file-backed auth store and analytics disabled | | Coding agent | `opencode` | opencode CLI shared EROFS tool image with placeholder auth import | | Built-in layer | `docker` | Docker Engine (`dockerd`) inside the VM with container proxy configuration | | Built-in layer | `nix` | Multi-user `nix-daemon` with the host-shared Nix binary cache (`http://devvm.proxy/nix`) | | Built-in layer | `desktop` | XFCE desktop environment and VNC server reachable via `devvm desktop` | Settings defined in `~/.config/devvm/config.toml` take precedence over `~/.local/share/devvm/settings.json`. When a key is set in `config.toml`, `devvm setup` skips its interactive prompt and refuses any flag that attempts to overwrite that key (`set_in_config`). ## Launch a project VM Running `devvm` with no subcommand executes `Launch.Up`. For a folder that does not yet have a VM, `creation.plan` inspects the directory, merges defaults and flags, builds any missing base, layer, or shared tool images, creates the VM, and opens a shell. ```mermaid sequenceDiagram autonumber actor User participant CLI as devvm (Launch.Up) participant Plan as creation.plan / detectFiles participant Build as build.Builds & tools.Versions participant Starter as starter.bringUp & Proxy participant Guest as VM (guest/sync.sh) User->>CLI: devvm (in ~/some-project) CLI->>Plan: Acquire folder creation lock & inspect project Plan->>Plan: Run firstRun (if unconfigured) & detectFiles(dir) Plan-->>User: Display summary & ask "Customize it? [y/N]" User-->>Plan: Confirm (Enter) CLI->>Build: Build base/layer image chain & shared tool .erofs images CLI->>CLI: Write state to vms//vm.json & b.CreateVM CLI->>Starter: Lock .git protection, start devvm _proxy, b.Start Starter->>Starter: Run Proxy.SelfCheck & start port forwarder CLI->>Guest: Stream tar payload & run guest/sync.sh Guest->>Guest: Mount /opt/devvm/tools, install CAs, set (devvm) prompt CLI->>Guest: Exec interactive shell in ~/some-project ``` ### Auto-detected toolchains and project markers When `--tools` is omitted, `detectFiles` (`internal/vm/detect.go`) scans the top-level entries of the project folder in a deterministic order and enables the corresponding shared toolchain for each matched file or glob pattern. If `.devcontainer/devcontainer.json` or `.devcontainer.json` is present (and `--no-devcontainer` is not passed), `devcontainers.prepare` also maps supported devcontainer features into the default toolchains, layers, and forwarded ports. | Part | Type | Top-level marker files (`detectFiles`) | Included binaries (`image.SharedTools`) | | --- | --- | --- | --- | | `node` | Toolchain | `package.json`, `bun.lock`, `bun.lockb` | `node`, `pnpm`, `bun` | | `python` | Toolchain | `pyproject.toml`, `requirements.txt`, `uv.lock`, `setup.py`, `Pipfile` | `python`, `uv` | | `rust` | Toolchain | `Cargo.toml` | `rust` (`rustup`, `cargo`, `rustc`) | | `go` | Toolchain | `go.mod` | `go` | | `dotnet` | Toolchain | `*.sln`, `*.slnx`, `*.csproj`, `*.fsproj`, `global.json` | `dotnet` | | `nix` | Layer (picker hint) | `flake.nix`, `shell.nix`, `devenv.nix`, or `default.nix` containing `mkShell` | Annotates `nix` in the interactive picker with the marker filename; does not auto-enable the layer unless selected or required by `devcontainer.json` | ### Step-by-step first launch Change into your project directory and run `devvm`: ```sh cd ~/some-project devvm ``` `devvm` refuses to mount unsafe directories—specifically your home directory (`~`), `~/.config/devvm`, `~/.local/share/devvm`, the Lima home directory, or any directory that contains them or sits inside `devvm`'s own directories—unless `--unsafe-folder` is explicitly passed or `--clone` is used. Before building images or allocating a disk, `creation.choose` prints the computed VM name (`devvm--<14-hex-hash>`) and the exact configuration sources: ```text ==> This folder's new VM, devvm-some-project-71a99b1076d92c, gets: toolchain node, for package.json agent claude, from devvm setup layers: none size: 4 CPUs, 8 GiB of memory and a 100 GiB disk network tier: allowlist ports: all Customize it? [y/N] ``` Press `Enter` (or `n`) to create the VM as shown, or answer `y` to open the interactive multi-section picker for toolchains, coding agents, layers, network tier, forwarded ports, CPUs, memory, and disk size. If command-line flags specify every configurable part, or if standard input is not a terminal, `devvm` proceeds without prompting. On the first run, `devvm` builds the `base` image (downloading and verifying the pinned Ubuntu cloud image if not cached) and any required layer or `.erofs` shared tool images, then creates and boots the VM: ```text ==> building image base (e9f517dddef1) a few minutes, and Ubuntu's cloud image to download the first time (details: ~/.local/share/devvm/logs/runs/20261024-090616-83541.log) ==> image base ready ==> creating devvm-some-project-71a99b1076d92c from base ==> starting devvm-some-project-71a99b1076d92c ``` If you interrupt (`Ctrl+C`) before the first shell opens, `undoCreate` automatically deletes the unfinished VM, removes its state directory, and unlocks `.git` protection. If your VM includes a coding agent (`claude`, `codex`, or `opencode`), authenticate on the **host** machine (or import existing host credentials) so the host proxy can inject credentials in transit without storing real tokens inside the VM: Run `devvm agent login` in a host terminal inside the project directory: ```sh devvm agent login claude ``` Copy your host's existing agent settings and logins into `~/.config/devvm/agents/` and the host secret store: ```sh devvm agent import ``` ### Root `devvm` (`up`) creation flags Flags passed to `devvm` apply when creating a new VM (or select which folder's VM to open): Comma-separated list of toolchains (`node`, `python`, `rust`, `go`, `dotnet`) or `none`. Overrides auto-detection from project files. Comma-separated list of coding agents (`claude`, `codex`, `opencode`) or `none`. Overrides the `devvm setup` / `config.toml` default. Comma-separated, bottom-first list of layers (`docker`, `nix`, `desktop`, or custom layers) or `none`. Replaces the `devvm setup` default. Network tier for the new VM: `allowlist`, `internet`, `open`, or `none`. Defaults to `devvm network -g` / `config.toml`, else `allowlist`. Number of virtual CPUs (`N`) for the new VM. Must not exceed host CPU count. Memory allocation in `GiB` (such as `16` or `16G`). Must not exceed `max(hostGiB - 2, hostGiB / 2)`. Disk size in `GiB`, at least the base image size (`100` GiB). VM disks can grow later via `devvm size`, but never shrink. Guest ports forwarded to host `127.0.0.1`: `all`, `none`, or a comma-separated list of ports and ranges such as `5173,8000-8010` (default `all`). Create the VM with an independent Git clone inside the VM disk at the project path instead of mounting the host project folder. Mutually exclusive with `--parent`. Create a folder-mounted VM without locking `.git` hooks and configuration on the host. Forward your host SSH agent into the VM while connected so guest commands can use host SSH keys. Build the VM's base image from an OCI/Docker container image reference (such as `node:24` or `alpine:3.21`) instead of devvm's Ubuntu cloud image. Ignore `.devcontainer/devcontainer.json` and `.devcontainer.json` when planning the VM. Create or open a dedicated VM for the current subdirectory rather than reusing an ancestor directory's VM. Open an ancestor directory's existing VM in the current subdirectory even when this subdirectory also has a VM. ## Verify guest isolation in the VM shell On every launch, `syncer.sync` (`internal/vm/sync.go`) streams an in-memory tar archive to the guest and executes `assets/guest/sync.sh` before opening your interactive shell. The shell prompt is prefixed with `(devvm)` via `~/.bashrc.d/devvm-prompt.sh` and starts in the exact host path (`~/some-project`). Run the following checks inside the `(devvm)` guest shell to verify each isolation boundary: In a standard mounted VM, only the project folder is mounted read-write from the host; the rest of `~` resides on the VM's own virtual disk. On macOS, `gitprotect` locks `.git` files (`config`, `hooks`, `info`, `objects`, `refs`) with `uchg` user immutable flags on the host so a compromised guest process cannot install malicious Git hooks or alter repository config: ```sh pwd ls -la .. touch .git/hooks/pre-commit ``` Expected behavior: - `pwd` prints the host project path (for example, `/Users/you/some-project`), while `ls -la ..` shows only the parent path stub on the VM root filesystem—none of your sibling host directories or host `~` dotfiles exist in the guest. - `touch .git/hooks/pre-commit` fails with `Operation not permitted` (or `Read-only file system`). Shared toolchains and coding agents are mounted from host `.erofs` images into `/opt/devvm/tools//` by `/usr/local/sbin/devvm-tools` and wired into `/etc/mise/config.toml`: ```sh df -T /opt/devvm/tools/*/* mise ls ``` Expected behavior: - Each active tool (`node`, `python`, `rust`, `go`, `dotnet`, `claude`, `codex`, `opencode`) is mounted as a read-only `erofs` filesystem under `/opt/devvm/tools/`. - Global package installs or tool self-updates cannot mutate the shared image; `go install` writes to `~/go/bin` (`go_set_gobin = false`) and npm global installs write to `~/.npm-global`. On the default `allowlist` tier, guest traffic is confined to the host proxy (`$HTTP_PROXY` / `$HTTPS_PROXY`), which permits only hosts in the base, toolchain, agent, layer, global, and per-VM allowlists. Test an unallowed external host and inspect the block report with `devvm-why`: ```sh curl -I https://example.com devvm-why ``` Expected behavior: - The request to `https://example.com` is rejected by the proxy (`403 Forbidden`), and the `_devvm_blocked` prompt hook (or running `devvm-why`) prints the blocked host along with the exact host command required to permit it: `devvm network allow example.com`. Inspect the guest environment and agent login files to confirm that no plaintext tokens exist inside the VM: ```sh cat ~/.config/devvm/env ``` Expected behavior: - Secret variables (such as `CLAUDE_CODE_OAUTH_TOKEN` or `GH_TOKEN`) and agent auth files (`~/.claude.json`, `~/.codex/auth.json`) contain salted placeholder strings rather than real credentials. The host proxy substitutes the real secret only over TLS connections to the bound destination host signed by the VM's proxy CA (`/etc/ssl/devvm.pem`). ## Everyday workspace lifecycle commands Once a project VM exists, subsequent `devvm` invocations in the project folder (or any of its subdirectories) reuse and start the existing VM in seconds: ```sh devvm # Open a shell in this folder's VM, starting it if stopped devvm --clone # Create a VM with an isolated repo clone instead of a folder mount devvm tools # Interactively add or remove shared toolchains and coding agents devvm code # Open the project inside the VM in VS Code over Remote-SSH devvm stop # Stop this folder's VM and its host proxy devvm ls # List all project VMs, their status, image, and network tier devvm rm # Delete this folder's VM after confirmation, exporting agent history first ``` ## Next How project paths map to VM names and hashes, vm.json state sections, and in-place vs. remake updates. Authenticate Claude Code, Codex, and opencode, import and save agent settings, and bind host secrets. Traffic confinement across allowlist, internet, open, and none tiers, plus devvm _proxy inspection. Read-only .git mount protection, unsafe folder checks, secret placeholder injection, and --clone mode. --- ## 04. VM lifecycle and state > How project paths map to VM names and 8-hex hashes, state persistence in vm.json, layout version checks, and in-place vs. remake transitions. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/04-vm-lifecycle-and-state.md - Generated: 2026-11-07T21:50:10.585Z ### Source Files - `internal/vmname/vmname.go` - `internal/project/project.go` - `internal/state/vm.go` - `internal/state/layout.go` - `internal/vm/lifecycle.go` - `internal/vm/remake.go` - `internal/vm/finder.go` --- title: "VM lifecycle and state" description: "How project paths map to VM names and 8-hex hashes, state persistence in vm.json, layout version checks, and in-place vs. remake transitions." --- `devvm` isolates each project directory in a dedicated virtual machine whose identity, configuration, and runtime processes are tracked outside the project tree under `~/.local/share/devvm` and `~/.config/devvm`. The packages `internal/vmname`, `internal/project`, `internal/state`, and `internal/vm` govern how physical host paths map to deterministic VM names and state directories, how `vm.json` persists core attributes and package-specific sections under OS file locks, and how VMs transition through creation, startup, in-place reconfiguration, full disk remakes, and teardown. ## Project path mapping and VM naming Every project VM name follows the grammar enforced by `internal/vmname` and constructed by `internal/project`: ```text devvm-- │ │ └── 14 lowercase hex digits (56 bits of SHA-1 of physical path) │ └───────── 4 to 20 chars of sanitized folder basename ([a-z0-9-]) └─────────────── Fixed prefix ("devvm-") ``` While short descriptions sometimes refer to truncated path hashes, `vmname.HashLen` is `14` hex characters (56 bits of SHA-1). This 14-character suffix distinguishes project VMs from image build VMs, which use 12-character keys (`devvm-img-<12-hex>`), even if a project folder is named `img-...`. ### Name grammar and hash derivation `project.New(dir, nameMax)` and `project.Nth(dir, nameMax, n)` construct a `project.Folder` from a physical directory path (`pwd -P` / `filepath.EvalSymlinks`): 1. **Path validation**: `project.Nth` rejects any path containing a comma (`,`, which breaks Lima mount option parsing) or any Unicode control character, returning `*project.BadPathError`. 2. **Slug generation (`project.Slug`)**: The base name of the directory (`filepath.Base(dir)`) is lowercased. Any character outside `a-z` and `0-9` is replaced with `-`, and leading or trailing dashes are stripped. If the resulting string is empty (for example, `/` or a name consisting only of punctuation), the slug defaults to `"root"`. 3. **Length bounding**: The slug is capped at `slugMax = 20` characters (or `nameMax - len("devvm-") - 1 - 14` if the backend imposes a stricter `NameMax()`), with trailing dashes stripped after truncation. If the backend's `NameMax()` leaves fewer than `slugMin = 4` characters for the slug, `project.Nth` returns `project.ErrNoRoom`. 4. **Hash computation (`project.Hash` and `hashNth`)**: - For the initial VM at a path (`n == 0`), `project.Hash(dir)` computes `sha1.Sum([]byte(dir))` and takes the first 14 hex characters (`vmname.HashLen = 14`). - If a previous VM created for `dir` was moved to a new directory, that moved VM retains its original name and `vms//` directory. When a new VM is subsequently created at the original `dir`, `finder.newName` increments `n` (from `0` up to `maxNames = 100`) and hashes `dir + "\x00" + strconv.Itoa(n)` until it finds a hash whose `vms//` directory holds no state. | Function / Constant | Package | Behavior | | --- | --- | --- | | `vmname.Prefix` | `internal/vmname` | Constant `"devvm-"` prefix required on all project VM names. | | `vmname.HashLen` | `internal/vmname` | Constant `14` (lowercase hex digits at the end of a project VM name). | | `vmname.IsVM(name)` | `internal/vmname` | Matches `^devvm-[a-z0-9]([a-z0-9-]*[a-z0-9])?-[0-9a-f]{14}$`. | | `vmname.HashOf(name)` | `internal/vmname` | Extracts the trailing 14-hex hash used as the directory name under `vms//`. | | `vmname.SlugOf(name)` | `internal/vmname` | Extracts `` from `devvm--`, which also serves as the guest hostname. | | `vmname.Short(name)` | `internal/vmname` | Returns `devvm-`, the short SSH/editor alias candidate when unclaimed by another VM. | | `vmname.IsAlias(alias, name)` | `internal/vmname` | Validates that `alias` is either `Short(name)` or `name` itself, preventing arbitrary host patterns in `~/.ssh/config`. | | `vmname.Aside(name)` | `internal/vmname` | Returns `devvm--old`, the temporary machine name used while remaking a VM. Does not match `IsVM`. | ### Refused and unsafe project folders Because a standard VM mounts its project folder read-write, `project.Unsafe` refuses to create or move a VM into directories that would expose host credentials, `devvm` state, or other VM disks: - **`Unsafe.Holds(dir)`**: Refuses `dir` if it is or contains `$HOME`, `$XDG_CONFIG_HOME/devvm`, `$XDG_DATA_HOME/devvm`, or `$DEVVM_LIMA_HOME` (for example, running `devvm` in `/` or `$HOME`), unless `--unsafe-folder` is explicitly passed on creation. - **`Unsafe.Inside(dir)`**: Refuses `dir` if it is nested inside the config folder, data folder, or Lima home (which could let a guest rewrite layer build scripts or host binaries). --- ## Target resolution and folder identity When a command acts on a project VM, `finder.target(ctx, need)` in `internal/vm/finder.go` locates the VM state and queries the backend once. ### Folder resolution rules (`--here` and `--parent`) `finder.folders` resolves which VM applies to the current physical working directory (`finder.dir()`) according to `FolderRule`: | Rule | CLI Flag | Resolution behavior | | --- | --- | --- | | `FolderAuto` | *(default)* | Uses the current directory's own VM if `vm.json` or a backend instance exists for it. Otherwise walks parent directories (`project.Parents`, nearest first, stopping before `$HOME` and `/`) and uses the first parent with a valid `vm.json` and backend instance, **unless** the current directory or an intermediate directory below that parent contains its own `.git` directory or file. | | `FolderHere` | `--here` (or `--clone`) | Uses only the current directory's own VM; never walks parent directories. | | `FolderParent` | `--parent` | Skips the current directory's VM and any intermediate `.git` boundaries to use the nearest parent directory's VM, or fails with `no_parent_vm`. | When `finder.parentVM` walks upward through `project.Parents(dir, home)`, it only passes over parent directories that have **no** `vm.json`. It stops at the first parent that has a `vm.json`. If that parent's folder identity no longer matches (`!sameFolder`) or its backend lacks the VM, the search returns `nil` rather than continuing to an outer parent. This prevents a VM in an outer folder from replacing a nested project folder through its read-write mount and causing commands in the nested folder to fall through to the outer VM. ### Physical folder identity (`FolderID`) `state.VM` stores both the physical path (`Dir`) and a platform-specific identity string (`Folder`, produced by `platform.FolderID(dir)`), which encodes the filesystem device, inode number, and creation/birth time. This pair distinguishes four directory states: 1. **Same folder (`sameFolder(f, st) == true`)**: Both `st.Dir == f.Dir` and `st.Folder == platform.FolderID(f.Dir)` match. 2. **Replaced folder (`t.other == true`)**: A VM state exists for `f.Dir`, but `st.Folder` differs (the directory was deleted and recreated, or replaced with another repository). Normal commands refuse with `vm_other_folder`. Only `devvm adopt` (which updates `st.Folder` and resets `.git` protection state) or `devvm rm` (which deletes the old VM without exporting from the missing folder) can act on it. 3. **Moved folder (`finder.moveHere`)**: The current directory has no VM of its own, but exactly one finished, non-clone VM in `Inventory.States()` has `st.Folder == id` at a different `st.Dir` whose old path is now gone (`FolderGone`) or a symlink resolving to the new path. If `platform.UniqueFolderID` confirms the filesystem does not reuse deleted inodes (APFS/HFS+ on macOS; ext4, XFS, Btrfs, tmpfs, ZFS, F2FS, or bcachefs on Linux with birth time) and the current user owns the directory, `devvm` prompts to move the VM (`move_vm`, or moves automatically with `devvm adopt --force`). Moving stops the VM if running, updates `st.Dir` and `st.Folder`, and records the previous path in the `moved` section of `vm.json` so the next guest setup relocates Claude Code session history. 4. **Missing or disconnected folder (`FolderOf(st)`)**: Evaluated with a 5-second timeout (`folderWait = 5 * time.Second`) so hanging network shares default to `FolderThere`: - `FolderThere` (`"there"`): Path exists, or `os.Lstat` fails with an error other than `fs.ErrNotExist`, or times out. - `FolderGone` (`"gone"`): Nothing exists at `st.Dir`, and the nearest existing parent directory resides on the **same** device ID recorded in `st.Folder`. - `FolderNotConnected` (`"not_connected"`): Nothing exists at `st.Dir`, and the nearest existing parent directory resides on a **different** device ID (for example, an unmounted external drive or network volume). `devvm rm ` requires interactive confirmation and ignores `--force` when a folder is `not_connected`. --- ## State persistence and directory layout `devvm` never writes state files into project directories. All configuration and runtime state reside in `paths.Dirs`: - **Config directory (`Dirs.Config`)**: `$XDG_CONFIG_HOME/devvm` (default `~/.config/devvm`), mode `0700`. Holds user-authored `config.toml`, global `policy`, custom `layers/`, trusted `ca/`, and imported `agents//` settings. - **Data directory (`Dirs.Data`)**: `$XDG_DATA_HOME/devvm` (default `~/.local/share/devvm`), mode `0700`. Holds `settings.json`, `ssh_config`, `vms//`, `images/`, `tools/`, `packages/`, `exports/`, `logs/`, `cache/`, `run/`, and `lima/` (unless overridden by `$DEVVM_LIMA_HOME`). ### Layout version checks Before running any stateful command (excluding `help`, `version`, `--version`, `completion`, and internal `_` subcommands), `state.CheckLayout(ctx, d)` verifies the layout version of both `Dirs.Data` and `Dirs.Config` under an exclusive file lock on `~/.local/share/devvm/version.lock`: 1. `state.Layout` is currently `1`. 2. `CheckLayout` reads `~/.local/share/devvm/version` and `~/.config/devvm/version`. 3. If either file contains a non-integer or a number `< 1`, `CheckLayout` fails with `*state.CorruptError` (`not a layout version`). 4. If either file records a version `> state.Layout`, `CheckLayout` aborts immediately with `msg.LayoutNewer`, preventing an older `devvm` binary from corrupting state written by a newer release. 5. If a `version` file is missing or `< state.Layout`, `CheckLayout` runs any registered `Migration` functions sequentially from `min(v) - 1` and writes `"1\n"` atomically to both `version` files. ### Per-VM directory structure and locks Each project VM stores all of its host-side files in `~/.local/share/devvm/vms//` (`paths.VM`), except Unix domain sockets, which are placed in `~/.local/share/devvm/run/` with short hash names to stay within macOS's 104-byte `sun_path` limit (`paths.SocketMax = 104`, accounting for OpenSSH appending 17 bytes to temporary control sockets). :::files ~/.local/share/devvm/ ├── version ├── version.lock ├── making.lock ├── groups.lock ├── retention.lock ├── tools.lock ├── ssh_config ├── settings.json ├── run/ │ ├── claims.lock │ ├── create-.lock │ ├── port-.ctl │ ├── link-.sock │ ├── link-.ctl │ ├── build-ports/ │ └── builds/ └── vms/ └── <14-hex-hash>/ ├── vm.json ├── lock ├── policy ├── home.tar.gz ├── run/ │ ├── remake.lock │ ├── known_hosts │ ├── started │ ├── sandbox.sb │ ├── proxy.json │ ├── proxy.pid │ ├── proxy.status │ ├── ports.lock │ ├── ports.pid │ ├── ports.status │ ├── links.json │ ├── links.pid │ ├── secrets.json │ ├── secrets.policy │ └── ca/ │ ├── lock │ ├── ca.pem │ ├── ca.key │ ├── next.pem │ └── next.key └── logs/ ├── network.jsonl ├── proxy.log ├── ports.log └── links.log ::: All locks use OS-level `flock` (`platform.LockFile` / `platform.TryLockFile`) on open file descriptors so the kernel automatically releases them if a `devvm` process crashes or receives `SIGKILL`. | Lock path | Acquisition mode | Scope and ordering rules | | --- | --- | --- | | `version.lock` | Blocking exclusive | Held during `state.CheckLayout` startup verification and migrations. | | `run/create-.lock` | Non-blocking (`TryLockFile`) | Held from before `devvm` prompts for a new VM plan until `makeMachine` records `created` in `vm.json`, and by `devvm rm`, `devvm adopt`, and `devvm gc --orphans`. Lives outside `vms//` so deleting/recreating the VM directory does not unlink the lock file. Removed on release. | | `vms//lock` | Blocking exclusive | Held by `state.VM.Create`, `Update`, `Claim`, `Remove`, and `RemoveFolder`. Only one VM's `lock` is held at a time. | | `run/claims.lock` | Blocking exclusive | Held while allocating resources that must be unique across all VMs (SSH aliases via `sshconfig.ChooseAlias`, and host ports for SSH, proxy, and VNC desktop via `ports.Allocator`). Always acquired **after** `vms//lock` (`VM.Claim`), or alone for stateless build VMs (`state.Claims`). | | `vms//run/remake.lock` | Non-blocking (`TryLockFile`) | Held throughout `Lifecycle` remake operations (`layer add`, `layer rm`, `update`, `clone`) and `finishPending`. Concurrent attempts fail immediately with `remake_running`. | | `making.lock` | Shared / non-blocking exclusive | Held shared (`tools.Making.Hold`) while creating a VM, updating VM tools, or remaking with reread tools; tried exclusively by `devvm gc --tool-images` and tool retention so active builds never have their tool images garbage-collected before `vm.json` records them. | ### `vm.json` schema `vms//vm.json` is written atomically via `state.WriteFile` (write to a temporary file in the same directory, `chmod 0600`, and `os.Rename`). Because every write replaces the file whole, reads (`state.ReadHash`, `state.ReadVM`, `state.ReadAll`) require no lock. Every `vm.json` contains top-level core fields (`state.core`) followed by alphabetically sorted package sections (`map[string]json.RawMessage`). Decoding (`VM.decode`) fails with `*state.CorruptError` if the file is not a JSON object or is missing any required core field (`version >= 1`, `id`, valid `name` matching `vmname.IsVM`, `image`, `dir`, or `backend`), or if `ReadAll` finds a `name` whose trailing hash does not match its enclosing `vms//` folder. #### Core fields Schema version of `vm.json`, fixed at `1` (`vmVersion = 1`). 32-character hex string (128 random bits from `crypto/rand`) generated on `VM.Create`. Allows links and concurrent commands (`stillGone`, `sameCreation`) to distinguish a recreated VM in the same folder from a previous VM that had the same name. Full project VM name (`devvm--<14-hex-hash>`). Chosen once at creation for the target backend and never changed. Short SSH host alias (`devvm-`) assigned under `claims.lock` by `sshconfig.ChooseAlias` if no other VM had claimed it when the VM was created. Validated via `vmname.IsAlias`. Target image name: `"base"` or layers joined with `+` (for example, `"docker+desktop"`). Recorded image build chain key sequence from `Builder.Chain(img).String()`, used by `Builds.Outdated(st.Chain)` to detect when the base image or any layer in the VM's stack has been rebuilt or updated. Pinned OCI container image reference (`repo@sha256:...`) when the VM's base image was built with `--from` or a `devcontainer.json` `image` property; omitted for the default Ubuntu cloud image. Physical host path of the project folder. Updated in place when a moved folder is adopted. Platform folder identity (`platform.FolderID(dir)` encoding device, inode, and birth time) used to detect folder replacement (`folderChanged`, `sameFolder`) and directory moves (`movedFrom`). Name of the backend managing the VM (for example, `"lima"`). Once recorded, the VM is always operated through this backend regardless of global backend flags. Filesystem path where the backend stores its data (`backend.LocationOf(b)`), used to detect if the backend home (such as `DEVVM_LIMA_HOME`) moved. UTC timestamp set by `creation.created` immediately after `b.CreateVM` succeeds. If `created` is zero/omitted (`!st.Finished()`), VM creation did not finish and `refuseUnfinished` blocks the VM from starting. Version string of the `devvm` binary that originally created the VM. Preserved across subsequent updates and remakes. #### Package sections Each subsystem reads its own section via `state.Section[T](st, name)` and mutates it inside `st.Update` or `st.Claim` via `state.SetSection(st, name, val)`. Encoding a value that marshals to `{}` or `null` deletes the section key from `vm.json`. | Section key | Owner package | Go type | Stored fields and purpose | | --- | --- | --- | --- | | `net` | `internal/netpolicy` | `netpolicy.Net` | `tier` (`allowlist`, `internet`, `open`, `none`) and `made` (`proxy_key`, placeholder `salt`, and `checked` self-check state). | | `git` | `internal/gitprotect` | `gitprotect.InVM` | `mode` (`on`, `strict`, `off`, `clone`), `next` (mode queued while the VM was running), and `made` (locked `.git` identity, linked git dirs, and config/hook handoff hash). | | `ports` | `internal/ports` | `ports.Ports` | `forward` (`""` for all ports `>= 1024`, `"none"`, or comma-separated ports/ranges) and `host` (`ssh`, `proxy`, and `desktop` loopback ports allocated under `claims.lock`). | | `agent` | `internal/sshagent` | `sshagent.InVM` | `mode` (`off`, `devvm`, `on`) and `made` (`toggle` capability flag, plus `run` ID and `mode` of the current boot). | | `tools` | `internal/tools` | `tools.InVM` | `names` (toolchains and coding agents) and `made` (`files` EROFS/tar image filenames, `libc`, and `glibc` version). | | `secrets` | `internal/secrets` | `secrets.InVM` | Bound secret names and allowed target host patterns for proxy placeholder substitution. | | `size` | `internal/vm` | `vm.Size` | Requested `cpus`, `memory` (GiB), and `disk` (GiB). Empty strings use the base image's defaults. | | `clone` | `internal/vm` | `vm.cloneInVM` | `on` (`true` for `--clone` VMs) and `made` (`remote` name added to host repo and `seeded` boolean). | | `devcontainer` | `internal/vm` | `vm.devcontainerInVM` | Source file path and SHA-256, container env vars, lifecycle commands, `parts` / `gave` (toolchains and layers), `ports` / `ports_gave`, `image`, `from_flag`, and `made` (`created` and `started` run markers). | | `desktop` | `internal/vm` | `vm.desktopInVM` | VNC relay password generated for `devvm desktop`. | | `links` | `internal/vm` | `vm.linksInVM` | Published TCP ports (`publish`), outbound VM links (`to`), and `group` membership. | | `remake` | `internal/vm` | `vm.remakeInVM` | Active only during a VM remake: target `image`, `command`, `home`, `no_export`, `made`, `clone`, `done`, `reread`, and `was` rollback snapshot. | | `moved` | `internal/vm` | `vm.movedInVM` | Original project path (`from`) preserved across folder moves until guest setup migrates Claude Code project history. | --- ## VM lifecycle state machine ```mermaid stateDiagram-v2 [*] --> Planned: devvm (up) acquires run/create-.lock Planned --> Unfinished: st.Create writes vm.json (created = zero) Unfinished --> [*]: Interrupt cleanup (undoCreate) or devvm rm Unfinished --> Stopped: b.CreateVM succeeds -> st.Update sets created Stopped --> Running: starter.bringUp (hostChecks, Proxy.Prepare, b.Start, SelfCheck) Running --> Stopped: Lifecycle.Stop / devvm restart / SelfCheck failure Running --> RemakeAside: Remake starts (acquires remake.lock, exports history & home.tar.gz, renames to devvm--old) Stopped --> RemakeAside: Remake starts (acquires remake.lock, renames to devvm--old) RemakeAside --> RemakeNew: b.CreateVM makes new machine & starter.bringUp starts it RemakeAside --> Stopped: Failure/Interrupt before restoreHome -> undo / putBack renames aside back RemakeNew --> Stopped: Failure/Interrupt before restoreHome -> undo deletes new & renames aside back RemakeNew --> RemakeDone: restoreHome succeeds -> committed=true, records new image/chain & remake.done=true RemakeDone --> Running: dropOld deletes devvm--old, clears remake section & home.tar.gz Stopped --> [*]: Lifecycle.Rm / RmVM -> b.Delete & vmFolders.teardown Running --> [*]: Lifecycle.Rm / RmVM -> exportFirst, b.Delete & vmFolders.teardown ``` ### Creation (`Launch.Up` and `creation.create`) When `devvm` runs in a directory without an existing VM (or with an unfinished VM that the user agrees to recreate via `remake_unfinished`), `Launch.makeHeld` acquires `run/create-.lock` and executes `creation.create`: `creation.plan` parses flags (`--tools`, `--agents`, `--layers`, `--network`, `--ports`, `--cpus`, `--memory`, `--disk`, `--clone`, `--ssh-agent`, `--from`, `--no-devcontainer`), evaluates `.devcontainer/devcontainer.json` if present, runs first-run default prompts if needed, verifies backend feature capabilities via `Registry.Require`, checks free disk space against `Floors.VM`, and offers to import the newest agent history export for the folder (`offerHistory`). While holding a shared lock on `making.lock`, `creation.create` refreshes `mise.lock` if the daily check is due, builds the target VM image (`build.Builds.Build`), and builds any missing read-only EROFS/tar tool images in `~/.local/share/devvm/tools/`. `vmFolders.clear` removes any leftover runtime files from a previously deleted VM. `st.Create` acquires `vms//lock` and `run/claims.lock`, selects the SSH alias, and writes `vm.json` with all requested sections (`net`, `git`, `sshagent`, `tools`, `ports`, `size`, `clone`, `devcontainer`) but with `Created` left zero. If `devvm` crashes after this point before `CreateVM` finishes, `st.Finished()` is `false` and `refuseUnfinished` prevents the half-created VM from ever starting without its network confinement. `creation.makeMachine` allocates the host SSH loopback port under `claims.lock`, invokes `b.CreateVM(ctx, key, spec)`, and calls `creation.created` to set `st.Created = time.Now().UTC()`. Once `Created` is written, `makeHeld` releases `run/create-.lock`. `starter.bringUp` locks `.git` (unless `--clone`), starts the per-VM host proxy, boots the VM, pins its SSH host keys in `vms//run/known_hosts`, starts the port forwarder (`devvm _ports`), and runs `Proxy.SelfCheck`. `creation.create` then imports any accepted agent history archive, runs `setUpVM` (syncing guest environment, CAs, secret placeholders, `~/.local/share/devvm/ssh_config`, links, initial `--clone` git push, and `devcontainer.json` commands), and replaces the `devvm` process with the guest shell (`platform.Exec`). If `devvm` is interrupted (`SIGINT`, `SIGTERM`, `SIGHUP`) before the first shell opens, `undoCreate` deletes the backend VM, unlocks `.git` or removes the host git remote, and runs `vmFolders.teardown` to remove `vms//`. ### Unified startup (`starter.bringUp`) Every operation that starts a VM or interacts with a running one—including `devvm`, `devvm restart`, `devvm host`, `devvm code`, `devvm history export`, `devvm rm` clone checks, and remake verification—goes through `starter.bringUp` in `internal/vm/start.go`: 1. **Unfinished check**: Calls `refuseUnfinished(vm, st)` and aborts with `vm_unfinished` if `st.Created.IsZero()`. 2. **Agent mount reconciliation (`o.agentDirs`)**: On interactive launches, `updateAgentDirs` compares the VM's mounted agent settings directories against `~/.config/devvm/agents//`. If the VM is already running and has pending next-start changes (size, SSH agent forwarding, agent mounts, moved project path, or a transition from a confined tier to `open`), `restartForNextStart` lists the pending changes and prompts whether to restart immediately. 3. **Three-point folder replacement guard (`folderChanged`)**: For stopped VMs that mount their project folder, `bringUp` verifies that `filepath.EvalSymlinks(st.Dir) == st.Dir` and `platform.FolderID(st.Dir) == st.Folder` at three points: - Before running `hostChecks` (so `.git` locking never follows a symlink planted by another VM). - Immediately before `b.Start`. - Immediately after `b.Start` returns (if the folder was swapped during boot, `bringUp` immediately stops the VM and proxy and fails with `project_folder_changed_stopped`). 4. **Host safety and `.git` protection (`hostChecks`)**: Verifies existing `.git` locks (`Protector.Check`), applies any queued `git-protect` mode change (`gitprotect.Next`) once the VM is stopped, and on Linux hosts scans for setuid/setgid files (`hostsafety.SetIDFiles`). 5. **Backend capability check**: Verifies via `Registry.Allowed` / `Require` that the backend supports all features required by the VM's current state (`startNeeds`). 6. **Start or re-confine**: - **If stopped (`starter.start`)**: Clears old `known_hosts`, ensures `devvm`'s SSH agent has keys if `mode == "devvm"`, prepares and starts the host proxy (`Proxy.Prepare`), checks link targets (`checkStartLinks`), constructs `backend.Want` (size, project mount + `ProjectID`, agent directory mounts, `AgentForwarding`), calls `b.Start(ctx, vm, want, conf)`, starts the host port forwarder (`forwards().Start`), and records the boot's `Run.ID` and SSH agent mode in `vm.json`. - **If already running (`runningConfinement`)**: Re-prepares the proxy if rules or secrets changed (stopping the VM via `Proxy.Unproxied` if the proxy fails), reloads expired SSH agent keys, and warns about any settings waiting for `devvm restart` (`nextStart`). 7. **Network self-check (`Proxy.SelfCheck`)**: Verifies end-to-end from inside the guest that traffic is routed through the VM's proxy and that disallowed connections fail on confined tiers; stops the VM immediately if confinement fails. ### Stopping and deleting VMs - **`Lifecycle.Stop` (`devvm stop`)**: Stops the backend instance (`t.stop`), stops the host proxy (`Proxy.Stop`), stops the port forwarder (`Forwarders.Stop`), stops active link sessions (`linkRuns.stop`), runs `starter.hostChecks` while stopped (applying any queued `git-protect` mode and checking `.git` and Linux set-id files), and rewrites `~/.local/share/devvm/ssh_config` without the stopped VM. - **`Lifecycle.Rm` and `Lifecycle.RmVM` (`devvm rm [VM]`)**: 1. Acquires `run/create-.lock` (`holdForRm`) so a VM currently being created cannot be deleted mid-flight. 2. Prompts for confirmation when deleting from a subfolder (`delete_parent_vm`, never bypassed by `--force`), when the folder was replaced (`delete_other_folder_vm`, never bypassed by `--force`), when the folder is on an unconnected drive (`delete_not_connected_folder_vm`, never bypassed by `--force`), or on normal deletion (`rm_confirm`, bypassed by `--force`). 3. For finished `--clone` VMs (unless `--no-clone-check` or `t.other`), boots the VM if needed and runs `cloneCheck` to inspect up to 100 linked worktrees (`cloneWorktreesMax = 100`) for uncommitted changes and unfetched commits, offering to fetch branches into `saved//` on the host repo. 4. Unless `--no-export`, if the VM has coding agents configured, boots the VM and streams agent transcripts/session history into `~/.local/share/devvm/exports/-.tar.gz` (capped at `historyMax = 2 GiB`) alongside a `.folder` metadata file recording the project path and network tier. If the export fails, `devvm rm` aborts and keeps the VM (`export_failed_kept`); if it exceeds 2 GiB, `devvm` warns (`history_too_large`) and proceeds. 5. Calls `b.Delete(ctx, vm)` and `vmFolders.teardown(ctx, t, false)` to stop the proxy, port forwarder, and link sessions, unlock `.git` or remove the clone remote (saving any unique remote-tracking commits under `saved//...`), remove inbound links from other VMs' `vm.json` files, delete any leftover `devvm--old` machine, delete `vms//` under `vms//lock`, and update `ssh_config`. --- ## In-place transitions vs. VM remakes `devvm` separates configuration changes by whether they can be applied to a running process on the host, applied inside the guest on the next shell launch, applied by editing the backend VM configuration on the next boot, or require replacing the root disk from a new image (`remaking.remake`). | Change category | Commands | When it takes effect | Mechanism | | --- | --- | --- | --- | | **Immediate (running VM)** | `devvm ports`, `devvm network allow/deny/rm`, `devvm network ` (confined tiers), `devvm secret`, `devvm link`, `devvm group` | Immediately | Updates `vm.json` (or `policy`) and restarts `devvm _ports`, reloads `devvm _proxy`, or updates link sessions without rebooting the guest. | | **Next shell launch (in-place)** | `devvm tools`, `devvm tools add/rm/update` | Immediately as the command opens the guest shell (new agent config dir mounts require restart) | Builds missing EROFS/tar tool images in `~/.local/share/devvm/tools/`, updates `tools` in `vm.json`, and mounts them inside the guest via `syncer.sync`. | | **Next VM stop** | `devvm git-protect on/strict/off` (when VM is running) | Next `devvm stop` or `devvm restart` (prompts to restart now) | Saves `git.next` in `vm.json` so host `.git` file flags (`chflags`/`chattr`) are never mutated while guest processes may race them. | | **Next VM start (`backend.Want`)** | `devvm size`, `devvm ssh-agent`, `devvm network open` (from confined), `devvm adopt` (moved folder), new agent store folders | Next `devvm` start or `devvm restart` (prompts to restart now if running) | `starter.start` passes `backend.Want` to `b.Start`, which compares and edits the stopped VM's Lima configuration in a single `limactl edit` before booting. | | **Full VM remake (`remaking.remake`)** | `devvm layer add`, `devvm layer rm`, `devvm update` (when image chain or `devcontainer.json` changed), `devvm clone on/off` | Replaces the VM machine and opens a new shell | Sets the old machine aside as `devvm--old`, creates a new machine from the target image, migrates the sanitized home folder, and deletes the old machine. | When a running VM is switched from a confined network tier (`allowlist`, `internet`, or `none`) to `open` via `devvm network open`, the VM remains attached to its sandboxed proxy network (`NetProxy`) until its next reboot, but its proxy immediately switches to open rules (allowing all hosts and local addresses). `starter.nextStart` warns that direct networking (`NetDirect`) will apply on the next `devvm restart`. ### Remake execution and crash recovery (`internal/vm/remake.go`) Because image layers (`docker`, `nix`, `desktop`, and custom layers in `~/.config/devvm/layers/`), base image rebuilds, and `--clone` mount topologies are baked into the VM's root disk or creation-time mount spec, changing them requires remaking the VM (`remaking.remake`). #### Pre-flight checks and locking `Lifecycle.toRemake` locates the folder's VM, prompts if acting on a parent folder's VM (`remake_parent_vm_ask`), and acquires a non-blocking lock on `vms//run/remake.lock` (`remaking.hold`). Under the lock, it re-reads `vm.json`: - If an interrupted remake is already recorded in `vm.json` (`remakePending`), `finishPending` completes it first. - It refuses VMs that do not exist, are unfinished (`vm_unfinished`), belong to another folder (`vm_other_folder`), or are `--clone` VMs (unless the command is `devvm clone on/off`). - `remaking.check` verifies before prompting that all layers in the target image exist, that the backend supports all required features (`Registry.Require`), and that host disk space satisfies `Floors.VM`. #### Step-by-step remake sequence Once confirmed interactively (or via `--force`), `remaking.remake` executes six interrupt-safe phases: `remaking.build` builds the target image (`build.Builds.Build`) and resolves its `chainOf` (`key` and `chain`). If `devvm update` detected a changed `devcontainer.json` (`p.Reread != nil`), it holds `making.lock` and builds any new shared tools required by the reread configuration. A failure here leaves the existing VM untouched. Unless `--no-export` (`p.NoExport`), `exportFirst` exports coding agent transcripts to `~/.local/share/devvm/exports/`. Unless `--clean-home` (`!p.Home`), `saveHome` starts the VM if stopped, runs `tar -czf - --one-file-system --ignore-failed-read .` in the guest's `~`, and pipes the stream through `copyHome` on the host into `vms//home.tar.gz`. Any failure in export (other than exceeding 2 GiB) or home archiving aborts the remake before the VM is modified. `remaking.remake` writes the `remake` section (`remakeInVM`) to `vm.json`. `setAside` stops the VM, stops its proxy, port forwarder, and link session, and renames the backend instance from `devvm--` to `devvm--old` (`vmname.Aside`). Registering `remaking.undo` on the UI cleanup stack ensures any subsequent error or interrupt rolls back to the aside machine. `finish` calls `creation.makeMachine` to create a new backend VM under the original name `devvm--` with the preserved state settings. `st.Update` sets `remake.made = true`, snapshots the previous `from`, `tools`, `ports.forward`, and `devcontainer` state into `remake.was` (`madeWith`), applies any reread `devcontainer.json` or clone mode switch (`switchedTo`), and resets `devcontainer.made` so `postCreateCommand` runs on the new disk. For `devvm clone off`, `mountFolder` removes the old clone remote (saving unmerged commits) and locks `.git`. `starter.bringUp` then boots and self-checks the new machine. Unless `--clean-home`, `restoreHome` streams `vms//home.tar.gz` into the running guest via `tar -xzf - --skip-old-files`, preserving files installed by the new image while restoring user files. Once `restoreHome` succeeds, `committed.Store(true)` disables rollback, and `st.Update` records the new `st.Image`, `st.Chain`, and `remake.done = true`. `dropOld` deletes `devvm--old` from the backend, cleans up host artifacts from a previous clone mode (`leftBehind`: unlocking `.git` after `devvm clone on` or clearing the `clone` section after `devvm clone off`), removes the `remake` section from `vm.json`, deletes `vms//home.tar.gz`, and opens the guest shell (`launcher.open`). #### Home archive sanitization (`copyHome` and `homeSkips`) Because the guest home directory is untrusted input streamed from the VM, `copyHome` in `internal/vm/home.go` never extracts the archive on the host. Instead, it decompresses and re-streams the tar entries in memory into `vms//home.tar.gz`, enforcing strict filters: - Strips leading `./`, cleans paths, and drops `.`, empty paths, absolute paths, `..` traversals, and paths containing control characters (`< 0x20` or `0x7f`). - Drops entries under `homeSkips`: `.config/devvm` (guest env and secret placeholders), `.ssh/authorized_keys` (managed by the backend), `.cache`, agent login files (`historyExcludes`), and read-only mounted or store-copied agent configuration paths (`agentDirs` and `agentFiles`). - Rejects any entry whose parent path traverses a previously kept symlink (`throughLink`), preventing symlink traversal outside `~` on extraction. - Keeps only regular files (`tar.TypeReg`), directories (`tar.TypeDir`), symlinks (`tar.TypeSymlink`), and hard links (`tar.TypeLink`) whose target is a regular file already keptEarlier in the stream. Device nodes, FIFOs, and sockets are dropped. - Masks file modes with `0777` (stripping setuid, setgid, and sticky bits) and clears user/group ownership headers. #### Rollback (`undo` / `putBack`) and crash recovery (`finishPending`) If an error or signal interrupts `remaking.remake` before `committed.Store(true)` (Step 5): 1. `undo` invokes `putBack`, which verifies `devvm--old` still exists, stops processes for `devvm--`, deletes the partially created new machine, and renames `devvm--old` back to `devvm--`. 2. If `remake.made` was set, `putBack` reverses any `clone` mount changes (`unmountFolder` / `switchedBack`) and restores `remake.was` (`from`, `tools`, `ports.forward`, `devcontainer`). 3. `clear` removes the `remake` section from `vm.json` and deletes `home.tar.gz`. If `devvm` was killed with `SIGKILL` or `putBack` failed, `remakeInVM` remains in `vm.json`. While `remakePending(st)` is true, read/modify commands fail with `remake_pending` and `devvm gc --orphans` skips the VM. The next `devvm`, `devvm layer add/rm`, `devvm update`, or `devvm clone` in that folder automatically calls `finishPending`: | Recorded state when `finishPending` runs | Recovery action | | --- | --- | | `remake.done == true` | New machine already has its home directory and updated `image`/`chain`; calls `dropOld` to delete `devvm--old` and clear `remake`. | | `devvm--old` exists, or original machine exists with `remake.made == false` | Calls `putBack` (if `devvm--old` exists) and `clear`, re-runs `remaking.check`, and restarts `remaking.remake` from Step 1. | | `devvm--old` is already gone and `remake.done == false` | The new machine never opened a shell; `makeNew` deletes any partial new machine, rebuilds the target image, and runs `finish` using the preserved `vms//home.tar.gz` (or warns with `remake_home_lost` if missing). | ## Related pages How base images, built-in and custom layers, and shared EROFS tool images compute deterministic SHA-256 build keys. How the four-role backend.Backend interface and Lima driver implement CreateVM, Start, Rename, and confinement. Create VMs with independent Git clones using --clone, switch modes with devvm clone, and rescue unfetched branches. Complete reference for ~/.config/devvm/config.toml keys, settings.json precedence, and state directory layout. Flags, arguments, and exit codes for devvm, ls, restart, stop, update, rm, adopt, size, and ports. Troubleshoot corrupt state files, unfinished VMs, replaced folders, and moved project directories. --- ## 05. Images, layers, and content keys > Deterministic SHA-256 key computation for base images, built-in and custom layers (docker, nix, desktop), and read-only EROFS shared tool images. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/05-images-layers-and-content-keys.md - Generated: 2026-11-07T21:53:56.789Z ### Source Files - `internal/image/key.go` - `internal/image/recipe.go` - `internal/image/tool.go` - `internal/build/build.go` - `internal/build/layer.go` - `assets/base.yaml` - `assets/prelude.sh` --- title: "Images, layers, and content keys" description: "Deterministic SHA-256 key computation for base images, built-in and custom layers (docker, nix, desktop), and read-only EROFS shared tool images." --- `devvm` identifies, caches, and invalidates VM disk images (`internal/image`, `internal/build`) and shared tool EROFS archives (`internal/tools`) using deterministic 12-hex-digit SHA-256 content keys. Every build artifact is keyed by a length-prefixed hash of its complete recipe—including the virtualization backend, parent image key, embedded script prelude (`assets/prelude.sh`), provisioning scripts, pinned upstream digests, and per-target rebuild counters—so any change to a script, feature selection, or base image automatically cascades to dependent layers without mutating existing VMs. ## Content key architecture Both VM disk images (`image.Input.Key`) and shared tool EROFS images (`image.ToolFile`) compute their 12-character lowercase hexadecimal key (`^[0-9a-f]{12}$`) through `image.keyHash` in `internal/image/key.go`. ### Length-prefixed SHA-256 framing To prevent canonicalization collisions between adjacent fields or multi-step scripts, `keyHash` frames every tag and value with an 8-byte big-endian unsigned length prefix (`uint64`) before writing bytes to the underlying SHA-256 state: 1. `newKeyHash()` initializes a `sha256.New()` digest and immediately writes the versioned header field `("format", "devvm image key 1")`. 2. Each `field(tag, value)` or `bytes(tag, value)` call writes `[8-byte big-endian len(tag)][tag bytes][8-byte big-endian len(value)][value bytes]`. 3. Recipe steps (`k.step(tag, s)`) write the step role tag (`"origin"` or `"step"` with `s.Name`), `"script"` (`s.Data`), `"args"` (`strconv.Itoa(len(s.Args))`), each `"arg"` value in order, and `"root"` (`"true"` or `"false"`). 4. `sum()` encodes the 32-byte SHA-256 digest as hexadecimal and truncates it to the first 12 hex characters (`keyLen = 12`). ```go title="internal/image/key.go" const keyFormat = "devvm image key 1" const keyLen = 12 func (k keyHash) write(b []byte) { var n [8]byte binary.BigEndian.PutUint64(n[:], uint64(len(b))) k.h.Write(n[:]) k.h.Write(b) } func (k keyHash) step(tag string, s Step) { k.field(tag, s.Name) k.bytes("script", s.Data) k.field("args", strconv.Itoa(len(s.Args))) for _, a := range s.Args { k.field("arg", a) } k.field("root", strconv.FormatBool(s.Root)) } ``` ### Key domain models ```mermaid classDiagram class Input { +string Backend +Name Image +string Parent +string From +string FromDigest +Recipe Recipe +string Missing +string Rebuild +Key() string } class Recipe { +[]Step Origin +[]Step Steps } class Step { +string Name +[]byte Data +[]string Args +bool Root } class Chain { +String() string +Key() string } class Link { +string Name +string Key } class Tool { +string Name +string Version +[]byte LockEntry +string Libc +string Base +string Format } Input *-- Recipe : hashes Recipe *-- Step : Origin / Steps Chain *-- Link : ordered base-first Tool ..> keyHash : ToolFile(assets, Tool) Input ..> keyHash : Key() ``` ### VM image key inputs (`image.Input`) `image.Input.Key()` hashes fields in a fixed sequence. Optional fields (`From`, `Missing`, `Rebuild`) are omitted from the hash stream when empty so default builds do not carry empty tags. Backend identifier returned by `Backend.Name()` (for example, `"lima"`). Ensures images are never shared across different virtualization backends. Normalized image name (`image.Name`): `"base"`, a single layer such as `"docker"`, or a `+`-joined layer stack bottom-first such as `"docker+nix"`. 12-hex-digit key of the parent image in the build chain, or `""` for `base`. Because every layer hashes its parent key, any change in `base` or an lower layer changes every key above it. Included only when `From != ""`. Hashes the user-supplied `--from` container reference (`from`) and its resolved registry manifest digest or Docker image ID (`from-digest`). Backend-specific origin steps (`Recipe.Origin`), present only on `base`. On Lima, this includes the `ubuntu-image` pin (`release`, `file`, `sha256`), optional `oci/build.sh`, `oci/bootify.sh`, and `oci/kernel.sh` conversion steps when `--from` is set, the `disk` size (`"100GiB"`), and the raw bytes of `assets/base.yaml`. Ordered guest provisioning scripts (`Recipe.Steps`) executed inside the build VM. Included only when a custom layer's `.sh` file was deleted from `~/.config/devvm/layers/` after being referenced (`".sh"`). Forces the key to diverge from any valid script content so recorded chains become outdated. Included only when `Rebuild != ""`. Holds the `-<8hex>` counter from `~/.local/share/devvm/images/rebuilds.json` for `(backend, from, image)`. **What image keys intentionally exclude:** Host CPU count and memory fitted by `buildSize` (`4` CPUs and `8 GiB` RAM capped to host limits), custom CA certificates in `~/.config/devvm/ca/`, company HTTP proxy credentials, and network allowlist files (`*.policy`) are excluded from image keys. Custom CAs and build proxy environment variables are injected transiently during build script execution and removed before the image VM stops; `*.policy` files are read on the host by `devvm _proxy` when a project VM runs. --- ## Script prelude (`assets/prelude.sh`) Every script shipped inside `devvm`'s embedded `assets.FS` that executes under `bash` is loaded through `image.OwnScript(own, name)`. `OwnScript` concatenates `assets/prelude.sh` and the target script into a single byte slice before storing it in `Step.Data` or hashing it in `ToolFile`. ```go title="internal/image/recipe.go" const preludeFile = "prelude.sh" func OwnScript(own fs.FS, name string) ([]byte, error) { prelude, err := fs.ReadFile(own, preludeFile) if err != nil { return nil, fmt.Errorf("reading devvm's %s: %w", preludeFile, err) } script, err := fs.ReadFile(own, name) if err != nil { return nil, fmt.Errorf("reading devvm's %s: %w", name, err) } return slices.Concat(prelude, script), nil } ``` ### Prelude runtime exports `assets/prelude.sh` sets no shell options of its own and runs only `uname -m` and `command -v apk`: | Identifier | Values / Signature | Behavior | | --- | --- | --- | | `$devvm_arch` | `arm64`, `amd64` | Maps `aarch64 \| arm64` to `arm64` and `x86_64 \| amd64` to `amd64`. Exits with status `1` on any other architecture. | | `$devvm_libc` | `musl`, `gnu` | Set to `musl` when `command -v apk` succeeds (Alpine guests); `gnu` otherwise. | | `devvm_sha` | `devvm_sha ` | Resolves `_sha256__musl` when `$devvm_libc == musl` and that variable is non-empty; otherwise resolves `_sha256_`. Returns status `1` if the variable is unset. | Scripts that depend on `prelude.sh` verify its presence on their first line after `set -euxo pipefail` via parameter expansion guards (`: "${devvm_arch:?}" "${devvm_libc:?}"`). - **Scripts prefixed with `prelude.sh` (`image.OwnScript`):** `layers/base.sh`, `layers/apt.sh`, `layers/desktop.sh`, `layers/docker.sh`, `layers/nix.sh`, `features/cli-tools.sh`, `features/fzf.sh`, `features/omb.sh`, `shared-tools/build.sh`, and `oci/build.sh`. - **Scripts that do not receive `prelude.sh`:** User-authored custom layers in `~/.config/devvm/layers/.sh`, `assets/oci/bootify.sh` (which runs under the container's `/bin/sh`), and `assets/oci/kernel.sh`. --- ## Base image recipe and package replay `base` is the root image of every chain. `build.Env.basePlan` constructs the `base` recipe by combining backend `Origin` steps with guest `Steps`. ```mermaid flowchart LR subgraph Origin["Recipe.Origin (Lima backend)"] O1["ubuntu-image
release + file + sha256"] O2["oci/build.sh + bootify.sh + kernel.sh
(--from only, root=true)"] O3["disk
100GiB"] O4["base.yaml
assets/base.yaml"] O1 --> O2 --> O3 --> O4 end subgraph GuestSteps["Recipe.Steps (in-VM bash)"] S1["apt.sh begin
(root=true)"] S2["base.sh
(root=false)"] S3[".sh ...
(cli-tools, fzf, omb)"] S4["apt.sh end
(root=true)"] S1 --> S2 --> S3 --> S4 end Origin --> GuestSteps ``` ### Lima origin steps and Ubuntu cloud image pinning On the `lima` backend, `images.Origin` (`internal/backend/lima/lima.go`) produces the following `Recipe.Origin` sequence: 1. **`ubuntu-image`:** Single argument formatted by `ubuntu.PinArg` from `~/.local/share/devvm/images/ubuntu.json` (` `). On the first build, `PinOrigin` fetches `cloud-images.ubuntu.com` for Ubuntu 26.04 (`resolute`), verifies `SHA256SUMS.gpg` in Go against `assets/layers/ubuntu-cloud.gpg` (fingerprint `D2EB44626FDDC30B513D5BB71A5D6C4C7DB87C81`), and pins the newest dated release (`release-YYYYMMDD`) under `~/.local/share/devvm/images/locks/ubuntu`. If a newly pinned Ubuntu image fails to boot during the same run (`backend.ErrBoot`), `Builds.fallBack` warns (`origin_no_boot`), pins the newest release older than the failed one (`backend.PinOlder`), and retries once. 2. **OCI conversion steps (`--from` only):** When building a `--from` container base, `convertSteps` appends three steps: - `oci/build.sh` (`Root: true`, `OwnScript` with `prelude.sh`, `Args: [digest, ref, username, uid, "/home/.guest", "20G"]`) - `oci/bootify.sh` (`Root: false`, raw file bytes) - `oci/kernel.sh` (`Root: false`, raw file bytes) 3. **`disk`:** Single argument `"100GiB"` (`backend.BaseDiskGiB`). 4. **`base.yaml`:** Raw bytes of `assets/base.yaml`, which sets `param.internal_netplanOptional: "true"` and disables system and user `containerd`. ### Base provisioning (`assets/layers/base.sh`) and features Between `apt.sh begin` and `apt.sh end`, `base` runs `layers/base.sh` followed by the user's selected features (`internal/build/features.go`): - **Core packages & pruning (`layers/base.sh`):** - Installs `git`, `curl`, `wget`, `ca-certificates`, `build-essential` (`build-base` on Alpine), `unzip`, `jq`, `ripgrep`, `erofs-utils`, `sqlite3` (`sqlite` on Alpine), and the newest `libicu[0-9]+` (`icu-libs` on Alpine). - Installs `/usr/local/bin/devvm-fetch`, which enforces SHA-256 (or SHA-512 when the expected hash is 128 hex characters) verification on all script downloads. - On `gnu` (Ubuntu), marks essential server utilities manual (`bc`, `file`, `htop`, `lsof`, `nano`, `rsync`, `screen`, `strace`, `tcpdump`, `time`, `tmux`, `mtr-tiny`, `pciutils`, `usbutils`, `inetutils-telnet`, `dmidecode`, `lshw`, `linux-image-virtual`), purges unnecessary server daemons (`snapd`, `multipath-tools`, `modemmanager`, `udisks2`, `unattended-upgrades`, `fwupd`, `rsyslog`, `bpftrace`, `apport`, `plymouth`, etc.), and masks `apt-daily.timer` and `apt-daily-upgrade.timer`. - Installs pinned `mise` (`mise_version=2026.11.3`) to `/usr/local/bin/mise`, configures `/etc/profile.d/devvm-tools.sh`, `/etc/systemd/system/devvm-tools.service` (or `/etc/init.d/devvm-tools` on OpenRC), `/usr/local/bin/xdg-open`, `/etc/profile.d/devvm-telemetry.sh`, GitHub SSH host keys in `/etc/ssh/ssh_known_hosts`, and `/etc/ssh/sshd_config.d/00-devvm.conf` (restricting remote port forwards to `127.77.*:*`). - **Optional base features (`assets/features/*.sh`):** Configured via `devvm setup` or `features` in `~/.config/devvm/config.toml` (defaulting to all features until chosen). Sorted alphabetically in the recipe: | Feature | Script | Installed components and pins | | --- | --- | --- | | `cli-tools` | `assets/features/cli-tools.sh` | `bat`, `fd` (`fd-find`), `zoxide`, `btop`, `tree` via `apt-get`/`apk`, plus pinned `eza` (`0.23.5`) and `tealdeer` (`1.9.1`, installed as `/usr/local/bin/tldr`) via `devvm-fetch`. | | `fzf` | `assets/features/fzf.sh` | `fzf` via `apt-get`/`apk` and bash keybindings in `~/.bashrc.d/fzf.sh`. | | `omb` | `assets/features/omb.sh` | Clones `oh-my-bash` and checks out pinned commit `94c2977eda06f230aafa9628fcd869123f01896b` with `DISABLE_AUTO_UPDATE="true"`. | If `config.toml` or `settings.json` names a feature that has no script in `assets/features/`, `basePlan` records it in `planned.unknown`, omits it from `Recipe.Steps` (and therefore the key), and logs a warning (`unknown_feature`) when building `base`. ### Deterministic `apt` package recording and replay (`internal/build/packages.go`) For default Ubuntu `base` builds (`From == ""`), `devvm` guarantees that rebuilding or recreating the same `base` key installs identical `.deb` packages offline: When `~/.local/share/devvm/packages/sets/.json` does not exist: 1. Host streams `in/mirrors` (containing `ubuntu-mirror` from `config.toml`, if configured) to `/var/cache/devvm/packages/in/mirrors` inside the build VM. 2. `apt.sh begin` enables `APT::Keep-Downloaded-Packages "true"`, `APT::Get::Always-Include-Phased-Updates "true"`, and `Acquire::GzipIndexes "false"`, rewrites `/etc/apt/sources.list.d/*.sources` to try `in/mirrors` before the official archive, and runs `apt-get update --error-on=any` (exiting `75` on unreachable sources). 3. After `base.sh` and feature scripts complete, `apt.sh end` copies every `*_InRelease` and `*_Packages` file to `/var/cache/devvm/packages/out/.InRelease` and `.Packages`, verifies every downloaded `.deb` against `apt-cache show`, writes `/var/cache/devvm/packages/out/Packages`, and moves each archive to `out/.deb`. 4. Host reads the tar stream from `out/` and validates it in Go (`internal/build/packages.go`): - Verifies clearsigned `InRelease` signatures against `assets/layers/ubuntu-archive.gpg` (fingerprint `F6ECB3762474EDA9D21B7022871920D1991BC93C`). - Checks suite names (`resolute`, `resolute-updates`, `resolute-security`, `resolute-backports`; refuses `resolute-proposed`) and anti-rollback `Date:` headers (`resolute` $\ge$ `2026-04-23`; other pockets no more than 24 hours older than the pinned Ubuntu cloud image date). - Verifies every `.Packages` SHA-256 and size against its signed `InRelease`, verifies that all 4 components (`main`, `restricted`, `universe`, `multiverse`) for `resolute-updates` and `resolute-security` were returned, and checks with `ubuntu.CompareVersions` that each recorded `.deb` is the newest version across `resolute`, `resolute-updates`, and `resolute-security`. 5. Host commits `.deb` files to `~/.local/share/devvm/packages/.deb` and writes `~/.local/share/devvm/packages/sets/.json` under `~/.local/share/devvm/images/locks/packages`. When `~/.local/share/devvm/packages/sets/.json` exists: 1. Before creating the VM, `cache.checkCached` verifies the size and SHA-256 of every `.deb` in `~/.local/share/devvm/packages/.deb`. If any file is missing or modified, the build aborts with `packages_missing`. 2. Host streams `in/Packages` and all `.deb` files into `/var/cache/devvm/packages/in/`. 3. `apt.sh begin` moves `/etc/apt/sources.list` and `/etc/apt/sources.list.d/*` aside to `/etc/apt/devvm-sources` and writes `deb [trusted=yes] file:/var/cache/devvm/packages/in ./` to `/etc/apt/sources.list.d/devvm-replay.list`. 4. `base.sh` and features install packages strictly from local disk without touching the network. 5. `apt.sh end` restores the original apt sources, deletes `/var/lib/apt/lists/*`, and runs `apt-get clean`. --- ## Layers and image chains Layers stack on top of `base` in a linear parent-child chain. `image.Name` joins layer names bottom-first with `+`: for example, `docker+nix` builds `docker` on `base`, then clones `docker` to build `nix`. ### Naming rules and layer resolution `image.LayerName` (`internal/image/key.go`) and `build.CheckLayerName` (`internal/build/layer.go`) enforce strict naming rules because image names map to filesystem paths and Lima instance names: - **Grammar:** `^[a-z0-9][a-z0-9-]*$` (lowercase ASCII letters, digits, and hyphens, starting with a letter or digit). - **Reserved names:** `"base"`, built-in layer names (`"desktop"`, `"docker"`, `"nix"`), and shared tool names (`"node"`, `"python"`, `"rust"`, `"go"`, `"dotnet"`, `"claude"`, `"codex"`, `"opencode"`) are rejected when creating a custom layer (`layer_name_reserved`). Passing a shared tool name to `devvm build` or `--layers` returns `tool_not_layer`. - **Normalization (`image.NameOf` / `image.ParseName`):** Strips `"base"`, empty segments, and duplicate layer names while preserving first-seen bottom-up order. `base+docker+nix+docker` normalizes to `docker+nix`; `base` alone has `Layers() == nil` and `Parent() == ""`. ### Layer recipe structure Every layer image (`name != "base"`) has an empty `Recipe.Origin` (its VM is cloned from ` ImagePrefix + spec.Parent` via `limactl clone --tty=false --mount-none`) and exactly two `Recipe.Steps`: 1. **`apt.sh` (`Root: true`, `Args: ["lists"]`):** Runs `assets/layers/apt.sh` with `prelude.sh`. Because `base` deletes `/var/lib/apt/lists/*` at the end of its build, `apt.sh lists` runs `apt-get update --error-on=any` if no `*_Packages` lists exist so the layer script can install distribution packages directly. 2. **`.sh` (`Root: false`, `Args: nil`):** Runs as the VM's default user (with passwordless `sudo`). For built-in layers, `Env.LayerData` loads `assets/layers/.sh` via `image.OwnScript` (prepending `assets/prelude.sh`). For custom layers, `Env.LayerData` reads `~/.config/devvm/layers/.sh` verbatim without `prelude.sh`. ### Built-in layers (`desktop`, `docker`, `nix`) | Layer | Script & Policy | Backend capability | Provisioning and pinning details | | --- | --- | --- | --- | | `desktop` | `assets/layers/desktop.sh`
`assets/layers/desktop.policy` | None (uses standard SSH forwarding) | Installs Xfce4 (`xfce4-session`, `xfwm4`, `xfce4-panel`, `xfdesktop4`, `xfce4-settings`, `xfce4-terminal`, `thunar`), TigerVNC (`tigervnc-standalone-server`, `tigervnc-tools`), `dbus-x11`, `xauth`, `socat`, and DejaVu fonts from the distribution archive. Installs `/usr/local/bin/devvm-desktop`, which starts `Xvnc` on display `:1` bound exclusively to Unix socket `/run/devvm-desktop/vnc.sock` (`0600`, `-rfbport -1 -nolisten tcp`). | | `docker` | `assets/layers/docker.sh`
`assets/layers/docker.policy` | `docker-layer` (`backend.FeatDockerLayer`) | On `gnu`, fetches Docker's apt keyring via `devvm-fetch` (pinned `docker_gpg_sha256`) and installs pinned versions of `docker-ce` (`29.9.3`), `docker-ce-cli` (`29.9.3`), `docker-ce-rootless-extras` (`29.9.3`), `containerd.io` (`2.4.2`), `docker-buildx-plugin` (`0.38.1`), `docker-compose-plugin` (`5.7.0`), and `docker-model-plugin` (`1.2.9`). On `musl` (Alpine), installs `docker`, `docker-cli-buildx`, and `docker-cli-compose` and enables the OpenRC service. Adds `$USER` to the `docker` group. | | `nix` | `assets/layers/nix.sh`
`assets/layers/nix.policy` | `nix-layer` (`backend.FeatNixLayer`) | Fetches pinned Nix (`2.35.4`), `direnv` (`2.38.2`), and `nix-direnv` (`3.2.0`) via `devvm-fetch`. Runs Nix's multi-user installer with `experimental-features = nix-command flakes`, `trusted-users = root`, `sandbox = true`, and `sandbox-fallback = false` (stripping build-time proxy env vars so they are not baked into `nix-daemon`), verifies `/nix/store` entries are owned by `root`, adds `/nix/var/nix/profiles/default/bin` to `/etc/sudoers.d/devvm-nix` and `/etc/profile.d/devvm-tools.sh`, and configures `~/.config/direnv/lib/devvm-nix.sh` to store direnv layouts inside `~/.cache/direnv/layouts/` instead of the host project folder. | ### Custom layers (`devvm layer new`) Running `devvm layer new ` calls `build.NewLayer` (`internal/build/layer.go`), which atomically links two files into `~/.config/devvm/layers/`: - `~/.config/devvm/layers/.sh`: Bash provisioning script template (`set -euo pipefail`). - `~/.config/devvm/layers/.policy`: Network allowlist rules read at VM runtime on the `allowlist` tier (never hashed into the image key). ### Chain propagation and staleness detection `build.Env.plan` computes the `image.Chain` (`[]image.Link`) from `base` up to the target image. The chain is serialized as space-separated `name=key` pairs in `~/.local/share/devvm/images//record.json` and in each project VM's `~/.local/share/devvm/vms//vm.json`: ```text base=1a2b3c4d5e6f docker=7a8b9c0d1e2f docker+nix=3a4b5c6d7e8f ``` ```mermaid flowchart TD subgraph Cached["Recorded Chain in vm.json"] B1["base = 1a2b3c4d5e6f"] --> D1["docker = 7a8b9c0d1e2f"] D1 --> DN1["docker+nix = 3a4b5c6d7e8f"] end subgraph Recomputed["Current Chain (after editing docker.sh or devvm rebuild docker)"] B2["base = 1a2b3c4d5e6f
(unchanged — reused)"] --> D2["docker = 9f8e7d6c5b4a
(new script/rebuild counter)"] D2 --> DN2["docker+nix = 5c4d3e2f1a0b
(Parent key changed)"] end D1 -. "now.Key() != l.Key" .-> D2 DN1 -. "now.Key() != l.Key" .-> DN2 ``` When `Builds.Outdated(chain)` checks a VM's recorded chain, it recomputes the current key for each link in the chain. If any link's current key differs from the recorded key: - Launching the VM prints a warning (`vm_outdated`) naming the changed images (`docker, docker+nix`) without modifying the running VM. - Running `devvm update` rebuilds any unbuilt images in the chain and remakes the VM from the updated image while preserving the VM's home directory. - On Linux hosts where Lima uses copy-on-write qcow2 overlays (`l.overlays()`), `BuildImage` strips write permissions (`chmod a-w`) from the finished image's `disk` file (`~/.local/share/devvm/lima/devvm-img-/diffdisk`) so backing images cannot be mutated. --- ## Shared tool EROFS images and keys Toolchains (`node`, `python`, `rust`, `go`, `dotnet`) and coding agents (`claude`, `codex`, `opencode`) are not baked into VM disk layers. Instead, each tool version is packaged into an immutable, lz4hc-compressed EROFS filesystem image stored in `~/.local/share/devvm/tools/--.erofs` and mounted read-only across VMs. ### Tool definitions (`image.SharedTools`) `internal/image/tool.go` defines the fixed ordering and group membership for all shared tools: | Tool name | Kind | Lock members (`GroupTools`) | Auto-detect project files (`Detect`) | Installer (`Installer`) | | --- | --- | --- | --- | --- | | `node` | `Toolchain` | `node`, `pnpm`, `bun` | `package.json`, `bun.lock`, `bun.lockb` | `Mise` | | `python` | `Toolchain` | `python`, `uv` | `pyproject.toml`, `requirements.txt`, `uv.lock`, `setup.py`, `Pipfile` | `Mise` | | `rust` | `Toolchain` | `rust` | `Cargo.toml` | `Rustup` | | `go` | `Toolchain` | `go` | `go.mod` | `Mise` | | `dotnet` | `Toolchain` | `dotnet` | `*.sln`, `*.slnx`, `*.csproj`, `*.fsproj`, `global.json` | `DotnetSDK` | | `claude` | `Agent` | `claude` | — | `Mise` | | `codex` | `Agent` | `codex` | — | `Mise` | | `opencode` | `Agent` | `opencode` | — | `Mise` | ### EROFS tool filename and key inputs (`image.ToolFile`) `image.ToolFile(assets, t)` validates the tool name (`^[a-z][a-z0-9]*$`), version (`^[0-9A-Za-z][0-9A-Za-z._+-]*$`), and libc constraints, then computes `--.erofs` using `newKeyHash()` with the following length-prefixed fields: ```go title="internal/image/tool.go" k := newKeyHash() k.field("tool", t.Name) k.field("version", t.Version) k.bytes("lock-entry", t.LockEntry) script, err := OwnScript(assets, "shared-tools/build.sh") // ... k.bytes("file shared-tools/build.sh", script) settings, err := miseSettings(assets) // ... k.bytes("mise-settings", settings) if installerOf(t.Name) == Mise { v, err := MiseVersion(assets) // ... k.field("mise", v) } k.field("libc", t.Libc) if t.Base != "" { k.field("base", t.Base) } else { k.field("base", "devvm") } if t.Format != "" { k.field("format", t.Format) } return t.Name + "-" + t.Version + "-" + k.sum() + ".erofs", nil ``` | Hash field tag | Value hashed | Why it is included | | --- | --- | --- | | `"format"` | `"devvm image key 1"` | Initialized automatically by `newKeyHash()`. | | `"tool"` | `t.Name` (e.g., `"node"`, `"pnpm"`, `"claude"`) | Distinguishes individual lock members (`node` builds separate EROFS images for `node`, `pnpm`, and `bun`). | | `"version"` | `t.Version` | Exact resolved upstream version from `~/.local/share/devvm/mise.lock`. | | `"lock-entry"` | `t.LockEntry` (`tools.Lock.Entry`) | Canonical JSON serialization (`json.Marshal`, keys sorted) of the tool's first `[[tools.]]` table in `mise.lock`, including all platform download URLs and checksums. Ignores TOML formatting, comments, or key order. | | `"file shared-tools/build.sh"` | `OwnScript(assets, "shared-tools/build.sh")` | Concatenation of `assets/prelude.sh` and `assets/shared-tools/build.sh` (which also holds `rustup_version` and `rustup_sha256_*` pins). | | `"mise-settings"` | `miseSettings(assets)` | Canonical JSON serialization of the `[settings]` table in `assets/shared-tools/mise.toml` (`{"experimental":true,"python":{"compile":false}}`). Ignores comments and `[tools]` / `[tool_alias]` tables. | | `"mise"` | `MiseVersion(assets)` (e.g., `"2026.11.3"`) | **Included only when `installerOf(t.Name) == Mise`.** Extracted by regex `(?m)^mise_version=([0-9A-Za-z._-]+)$` from `assets/layers/base.sh`. Omitted for `rust` (`Rustup`) and `dotnet` (`DotnetSDK`) so bumping `mise` does not invalidate tools `mise` does not install. | | `"libc"` | `"gnu"` or `"musl"` | Separates glibc and musl builds. | | `"base"` | `"devvm"` (for `gnu`) or `--from` digest (for `musl`) | `gnu` tool images always build inside `devvm`'s own Ubuntu `base` (`t.Base == ""`, hashed as `"devvm"`), allowing Ubuntu VMs and `--from` Debian/Ubuntu VMs to share the same `gnu` tool images without trusting container code during tool builds. `musl` tool images build inside the `--from` Alpine base and require `t.Base` to be that image's digest. | | `"format"` | `t.Format` (omitted when `""`) | Backend-specific EROFS format (`Backend.ToolFormat()`). Lima returns `""` (compressed EROFS via `mkfs.erofs -zlz4hc --all-root`). | `image.ToolFile` returns an error if `t.Libc == "musl"` and `t.Base == ""` (`"a musl tool image needs its base's digest"`), or if `t.Libc == "gnu"` and `t.Base != ""` (`"a gnu tool image builds in devvm's own base, which has no digest in its key"`). Furthermore, `Share.buildsFor` checks the base image's `record.json` before cloning the build VM and refuses the build with `tools_wrong_base` if the base image record does not match these invariants. ### Tool image build isolation and `GLIBC_` version scanning When `Share.Build` (`internal/tools/share.go`) builds missing `.erofs` files: 1. **Share lock:** Acquires `~/.local/share/devvm/tools.lock` (`LockShare`). 2. **Per-tool throwaway VM:** For each missing tool image, creates an isolated temporary directory `~/.local/share/devvm/tools-build.` containing `.build/mise.toml` and `.build/mise.lock`, clones `base` into `devvm-tools-build` with only that temporary directory mounted writable at `/mnt/devvm-tools` (never mounting `~/.local/share/devvm/tools`), and runs `assets/shared-tools/build.sh`. 3. **Installation & EROFS packing (`assets/shared-tools/build.sh`):** - Installs into `/opt/devvm/tools//` (using `rustup-init` with `rust-src` and `rust-analyzer` for `rust`, `devvm-fetch` with the lock's SHA-512 SDK tarball and a `metadata/workloads//userlocal` marker for `dotnet`, or `mise install --shared /opt/devvm/tools ` plus `corepack enable yarn` on `node` for `Mise` tools). - On `gnu` builds, runs `glibc_needs` across all installed ELF files (excluding `testdata` directories) using `readelf -W -V` to extract the highest `GLIBC_2.x` symbol version requirement into `/mnt/devvm-tools/.glibc`. - Packs `/opt/devvm/tools//` with `mkfs.erofs -zlz4hc --all-root /mnt/devvm-tools/.tmp ` and renames it to `/mnt/devvm-tools/`. 4. **Host verification and record commit (`Share.moveIn`):** After `devvm-tools-build` is deleted, the host verifies with `os.Lstat` that `` and `.glibc` are regular files (`tool_bad_output` otherwise), hashes `` with `image.SHA256File`, moves `` into `~/.local/share/devvm/tools/`, and writes `~/.local/share/devvm/tools/.json` (`state.ToolImage`). 5. **Compatibility check (`Share.Runs`):** Before mounting `gnu` tool images into a `--from` VM whose base image recorded a `glibc` version (`getconf GNU_LIBC_VERSION`), `Share.Runs` compares the base's `glibc` against each tool record's `glibc` via `image.GlibcOlder`. If the `--from` base has an older glibc than the tool requires, `devvm` refuses with `tool_needs_glibc`. 6. **Guest mounting (`assets/guest/tools.sh`):** Inside project VMs, `/usr/local/sbin/devvm-tools` reads `/etc/devvm/tools` (` ` per line), attaches `/mnt/devvm-tools/` to a read-only loop device with `losetup --find --show --read-only --direct-io=on`, and mounts it with `mount -t erofs -o ro` at `/opt/devvm/tools//`. Any unmounted tool is written to `/run/devvm/tools-unmounted` and aborts VM setup with `tools_unmounted`. --- ## State layout and metadata schemas All image and tool records live under `~/.local/share/devvm` (`paths.Dirs.Data`) and custom layer definitions live under `~/.config/devvm` (`paths.Dirs.Config`). :::files ~/.config/devvm/ ├── config.toml └── layers/ ├── .sh # Custom layer script (hashed in Recipe.Steps) └── .policy # Custom layer allowlist (runtime only, not hashed) ~/.local/share/devvm/ ├── making.lock # Shared lock held while a VM selects/builds tools ├── mise.lock # Resolved upstream tool versions and platform checksums ├── oci-build.lock # Exclusive lock for --from container conversions ├── tools.lock # Exclusive lock for writing to tools/ or mise.lock ├── cache/ │ ├── from-digests.json # Cached --from registry manifest digests │ └── mise//mise # Pinned host mise binary for resolving mise.lock ├── images/ │ ├── rebuilds.json # Per-(backend, from, image) rebuild counters │ ├── ubuntu.json # Pinned Ubuntu 26.04 cloud image release & SHA-256 │ ├── locks/ │ │ ├── # Per-image build/delete lock │ │ ├── packages # Package cache commit lock │ │ ├── rebuilds # rebuilds.json update lock │ │ └── ubuntu # ubuntu.json pin lock │ └── / │ └── record.json # Written last when image build succeeds (state.Image) ├── packages/ │ ├── .deb # Content-addressed Ubuntu .deb archive │ └── sets/ │ └── .json # Recorded package set for base image └── tools/ ├── --.erofs # Compressed read-only EROFS filesystem image └── --.erofs.json # Tool image metadata record (state.ToolImage) ::: ### JSON record schemas ```json title="images//record.json (state.Image)" { "name": "docker+nix", "chain": "base=1a2b3c4d5e6f docker=7a8b9c0d1e2f docker+nix=3a4b5c6d7e8f", "from": "", "digest": "https://cloud-images.ubuntu.com/releases/26.04/release-20260423/ubuntu-26.04-server-cloudimg-arm64.img@sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef", "libc": "gnu", "glibc": "", "built": "2026-11-07T15:00:00Z", "made_by": "0.1.0" } ``` ```json title="images/rebuilds.json (state.Rebuilds)" { "rebuilds": [ { "backend": "lima", "from": "", "name": "base", "counter": "20261107150405-a1b2c3d4" } ] } ``` ```json title="tools/--.erofs.json (state.ToolImage)" { "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "libc": "gnu", "glibc": "2.28", "base": "", "format": "", "backend": "lima", "built": "2026-11-07T15:05:00Z", "made_by": "0.1.0" } ``` ### Concurrency and lock ordering `devvm` coordinates concurrent processes using OS file locks (`flock` via `internal/platform`) on files outside the directories that guests mount: | Lock path | Scope and deadlock prevention | | --- | --- | | `~/.local/share/devvm/images/locks/` | Held exclusively by `Builds.lockBuild` while building `` and by `Env.lockFree` while deleting ``. A second `devvm` building the same key waits (printing `image_build_waiting` after 3s) and then observes `built == true`. If a previous build crashed before writing `record.json`, the lock is free and the next build deletes the `leftover` instance (`image_leftover`) before rebuilding. | | `~/.local/share/devvm/making.lock` & `~/.local/share/devvm/tools.lock` | Processes creating or updating a VM take `making.lock` **shared**, then acquire `tools.lock` **exclusively** inside `Share.Build`, keeping `making.lock` until `vm.json` records the VM's tool images. `devvm gc --tool-images` and automatic retention acquire `tools.lock` **exclusively** first, then attempt a non-blocking exclusive lock (`TryLockFile`) on `making.lock`. Because `gc` never blocks on `making.lock`, the two-lock protocol cannot deadlock and never deletes a current tool image while a new VM is being created. | ## Related pages Add, remove, and update shared mise toolchains and author custom image layers with network policies via devvm layer new. Translate .devcontainer/devcontainer.json properties into VM plans and build bootable VM base images from OCI/Docker container references with --from. The four-role backend.Backend interface (Host, Images, Machines, Housekeeping), capability requirement checks (Caps), and the Lima driver implementation. Syntax, subcommands, and flags for devvm build, rebuild, images, layer, tools, space, gc, retention, and setup. --- ## 06. Network tiers and host proxy > Traffic confinement across allowlist, open, and offline tiers, host-side TLS SNI and HTTP inspection via devvm _proxy, and per-VM CA rotation. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/06-network-tiers-and-host-proxy.md - Generated: 2026-11-07T21:58:10.608Z ### Source Files - `internal/netpolicy/tier.go` - `internal/netpolicy/netpolicy.go` - `internal/netpolicy/rules.go` - `internal/proxy/proxy.go` - `internal/proxy/policy.go` - `internal/proxy/sni.go` - `internal/trust/ca.go` --- title: "Network tiers and host proxy" description: "Traffic confinement across allowlist, open, and offline tiers, host-side TLS SNI and HTTP inspection via devvm _proxy, and per-VM CA rotation." --- Every project VM routes egress traffic through a dedicated host-side HTTP/HTTPS proxy process (`devvm _proxy`) managed by `internal/netpolicy` and `internal/proxy`. On confined tiers (`internet`, `allowlist`, and `none`), the backend isolates the VM's hypervisor or hostagent process so the guest cannot make direct outbound TCP, UDP, or DNS connections; all traffic must authenticate to the VM's loopback proxy on the host. Each VM also receives an isolated ECDSA P-256 certificate authority (`internal/trust`) paired with a pre-installed successor CA, enabling the host proxy to terminate TLS on port `443` for secret-bound hosts, substitute credential placeholders in flight, and rotate signing keys without restarting the proxy or VM. ## Network tiers and backend confinement A VM's network tier (`netpolicy.Tier`) is persisted in the `"net"` section of `~/.local/share/devvm/vms//vm.json` (`netpolicy.Net`). When a VM's state omits the tier, `netpolicy.Of` defaults to `allowlist` (`netpolicy.TierDefault`). If `vm.json` cannot be parsed or contains an unrecognized tier string, `netpolicy.TierOf` fails closed to `none` (`TierNone`). | Tier | Rank | Confined (`Tier.Confined()`) | Backend network (`Tier.Net()`) | Built-in policies (`PolicySet.Builtin`) | User policy files | Internal address floor | | --- | --- | --- | --- | --- | --- | --- | | `open` | 0 (lowest) | `false` | `backend.NetDirect` | None (`Config.Open = true`) | Ignored | Disabled (reaches host `localhost` and LAN) | | `internet` | 1 | `true` | `backend.NetProxy` | `network/internet.policy` (`**:*`) | `Denies` only (`!rule`), plus `secrets.policy` | Enforced unless an explicit IP/CIDR rule matches | | `allowlist` | 2 (default) | `true` | `backend.NetProxy` | `layers/base.policy`, `shared-tools/.policy`, `layers/.policy` | All rules (`Files`) and `secrets.policy` | Enforced unless an explicit IP/CIDR rule matches | | `none` | 3 (strictest) | `true` | `backend.NetProxy` | Agent policies only (`shared-tools/claude.policy`, `codex.policy`, `opencode.policy`) | `Denies` only (`!rule`), plus `secrets.policy` | Enforced unless an explicit IP/CIDR rule matches | Every tier starts `devvm _proxy` and configures `HTTP_PROXY`, `HTTPS_PROXY`, `NO_PROXY=localhost,127.0.0.1,::1`, and `NODE_USE_ENV_PROXY=1` inside the guest. On `open`, the VM is not sandboxed to the proxy, but requests that use the proxy still receive secret injection, Nix binary caching, and VM link routing. ### State persistence in `vm.json` The `"net"` section in `vm.json` stores the configured tier alongside generated per-VM credentials and self-check status (`netpolicy.Net`): ```json { "net": { "tier": "allowlist", "made": { "check": "ok", "proxy_key": "a8f9...", "proxy_salt": "4c1e9b2d..." } } } ``` - `proxy_key`: Random password generated once per VM via `backend.NewProxyPassword`. Every request to `devvm _proxy` must send `Proxy-Authorization: Basic` with username `devvm` (`proxy.ProxyUser`) and this password. - `proxy_salt`: 16-byte (32-hex-character) random value generated via `crypto/rand` (`netpolicy.Salt`). Appended to secret placeholders inside the VM so a leaked placeholder from another VM or host process cannot be exchanged for credentials. - `check`: Result of the post-start confinement verification (`ok` or `unenforced`). ### OS-level hypervisor confinement When `Tier.Confined()` is `true`, the backend requires the `egress-proxy` capability (`backend.NetProxy`) and restricts the VM process before boot: On macOS, the Lima driver (`internal/backend/lima`) launches the VM's `limactl` hostagent under `/usr/bin/sandbox-exec` using a generated Seatbelt profile at `~/.local/share/devvm/vms//run/sandbox.sb` (`lima.profile`): ```scheme (version 1) (allow default) (deny network-outbound) (allow network-outbound (remote unix-socket (subpath "/Users/you/.local/share/devvm/lima/devvm-..."))) (allow network-outbound (remote unix-socket (path-literal "/Users/you/.local/share/devvm/run/ssh-agent.sock"))) (deny mach-lookup (global-name "com.apple.dnssd.service")) (allow network-outbound (remote ip "localhost:60022")) (allow network-outbound (remote ip "localhost:54321")) (deny network-bind) (allow network-bind (local tcp "localhost:60022")) (allow network-bind (local unix-socket (subpath "/Users/you/.local/share/devvm/lima/devvm-..."))) ``` - **Outbound restriction:** Denies all outbound IP and Unix socket connections except the VM's SSH port (`localhost:`), the VM's host proxy port (`localhost:`), Unix sockets inside the VM's Lima directory, and the configured SSH agent socket. - **DNS suppression:** Blocks Mach lookups to `com.apple.dnssd.service`, disables Lima's `hostResolver`, and points guest DNS at an unreachable address so guest DNS queries fail immediately and all name resolution occurs inside `devvm _proxy`. - **Bind restriction:** `NoBinds` prevents the Lima hostagent from binding arbitrary host ports; guest port forwarding is handled exclusively by `devvm _ports`. - **Guest proxy endpoint:** The guest reaches the proxy at `http://devvm:@192.168.5.2:`. On Linux, Lima starts QEMU through a wrapper generated in `~/.local/share/devvm/run/qemu/` (`devvm _qemu`). - **Argument validation:** The wrapper inspects Lima's QEMU invocation and refuses to start (`qemu_refused`) unless there is exactly one `-netdev user,id=net0,...` argument and no additional network devices. - **SLIRP restriction:** Appends `restrict=on` to the user-mode `-netdev` configuration, blocking all guest IP traffic and SLIRP DNS resolution. - **Guest forward:** Adds a single `guestfwd=tcp:192.168.5.4:3128-cmd:...` rule bridged to the host proxy at `127.0.0.1:`. - **Guest proxy endpoint:** The guest reaches the proxy at `http://devvm:@192.168.5.4:3128`. ### Startup sequence and `SelfCheck` During `starter.bringUp` (`internal/vm/start.go`), `netpolicy.Proxy.Prepare` and `netpolicy.Proxy.SelfCheck` run before guest setup scripts or user shells execute: `Proxy.secretsReady` invokes `trust.InitCA` on `~/.local/share/devvm/vms//run/ca/`, excludes the CA directory from macOS Time Machine backups (`platform.ExcludeFromBackup`), generates the VM's `ProxySalt` if absent, and writes `secrets.json` and `secrets.policy`. `Proxy.Prepare` builds `proxy.Config` and compares its SHA-256 (`cfg.Sum()`) against the running proxy's `~/.local/share/devvm/vms//run/proxy.json`. If the VM is stopped or the config sum changed (`Proxy.stale`), `Proxy.Stop` sends `SIGTERM` (followed by `SIGKILL` after 2 seconds if still alive), writes the new `proxy.json` (`0o600`), spawns `devvm _proxy -config `, and waits up to 5 seconds (`10 * 500ms`) for `proxy.status` to report the bound `127.0.0.1:` address. - **Confined tiers (`internet`, `allowlist`, `none`):** `enforced(ctx, b, vm)` first checks `b.Running(ctx, vm).Confined` on the host to verify the hypervisor process is the one `devvm` launched under confinement. Next, `Direct(ctx, b, vm)` runs `curl --noproxy * -sS -m 3 -o /dev/null https://1.1.1.1` inside the VM, which must fail. - If `run.Confined` is `true` but `Direct` succeeds, the VM has an unauthorized route out; `SelfCheck` records `"check": "unenforced"`, stops the VM immediately, and fails with `net_direct_route`. - If `run.Confined` is `false` (for example, started via `limactl start` or switched from `open` while running), `SelfCheck` records `"unenforced"` and prompts to restart the VM confined (`net_unenforced_restart`), or stops the VM when non-interactive (`net_unenforced_stop`). - **Open tier (`open`):** `checkOpen` sources `/etc/environment` inside the VM and runs `curl -sSf -m 3 -o /dev/null --proxy "$HTTPS_PROXY" http://devvm.proxy/blocked`. If the proxy fails to respond, `devvm` emits a `net_proxy_unreachable` warning and continues. ### Tier transitions Changing a VM's tier with `devvm network ` (`vm.Net.SetTier`) enforces link and group invariants: - **Between confined tiers (`internet`, `allowlist`, `none`):** Applies immediately to a running VM by regenerating `proxy.json` and restarting `devvm _proxy` on the same loopback port. - **Into or out of `open`:** Requires a VM restart because hypervisor confinement (`sandbox-exec` or QEMU `restrict=on`) is fixed at VM boot. If the VM is running, `devvm` offers to restart it immediately; if declined, a VM moving from a confined tier to `open` remains confined to its proxy (which switches to `"open": true` rules) until the next `devvm restart`. - **Link and group constraints:** Refuses tier changes if the VM belongs to a group (`GroupTierFixed`) or if the new tier would violate the link rule requiring every target VM to be on at least as strict a tier as its client (`tierLinked`, checked via `Tier.AtLeast`). --- ## Host proxy architecture (`devvm _proxy`) `internal/proxy` is a standalone, stdlib-only package (with `internal/daemon`) executed via the hidden `devvm _proxy -config ` command (`internal/cli/proxy.go`). All runtime state for a proxy instance lives in a `proxy.server` struct with zero package-level mutable state. ```mermaid flowchart TD subgraph Guest["Project VM Guest"] Client["HTTP / HTTPS / SSH Client"] end subgraph HostProxy["Host Process: devvm _proxy (internal/proxy)"] Auth["authorized()
Proxy-Authorization: Basic devvm:<key>"] HostCheck["splitHostPort() & HostOK()
lowercase, trim trailing dot, reject shorthand IPs"] Reserved{{"host == devvm.proxy?"}} ReservedSrv["serveReserved()
/blocked, /blocked/new, /ca.pem, /nix/"] ConnCap["connects chan (maxConnects = 1024)
HTTP 503 if full"] Links{{"Matches links.json?
(*.devvm.internal / link name)"}} LinkSock["Unix socket in 127.77.0.0/16
(no rules, no SNI check, no secrets)"] Policy["policy.verdict(host, port)
1. BrokenDeny check
2. Deny rules (!rule)
3. Allow rules"] Floor["egress.floorCheck()
Resolve DNS or check floorUpstream()
Block loopback, RFC1918, CGNAT, NAT64, hostAddrs"] InterceptCheck{{"Port 443 + CA loaded +
(bound secret or OAuth capture)?"}} SNIPeek["tunnel() + peekClientHello()
Verify TLS SNI == CONNECT host
before sending client bytes"] MITM["intercept() + forwarder()
Mint leaf cert via ca.signer()
Check Host header (421 if mismatch)
refuseMinting() for api.github.com
secrets.inject() / captureResponse()"] end subgraph UpstreamNet["Host Egress / Upstream"] CorpProxy["Upstream Proxy (optional)
HTTPS_PROXY / HTTP_PROXY"] Target["Remote Destination"] end Client -->|"CONNECT or GET http://..."| Auth Auth -->|"missing/bad -> 407"| HostCheck HostCheck -->|"invalid -> 400"| Reserved Reserved -->|"yes (HTTP)"| ReservedSrv Reserved -->|"no (CONNECT)"| ConnCap ConnCap -->|"slot acquired"| Links Links -->|"yes"| LinkSock Links -->|"no"| Policy Policy -->|"denied -> 403"| Floor Floor -->|"internal IP -> 403"| InterceptCheck InterceptCheck -->|"no"| SNIPeek InterceptCheck -->|"yes"| MITM SNIPeek --> CorpProxy MITM --> CorpProxy CorpProxy --> Target ``` ### Proxy configuration validation and lifecycle `netpolicy.Proxy.start` writes `~/.local/share/devvm/vms//run/proxy.json` with mode `0o600` before launching `devvm _proxy -config `. `proxy.ReadConfig` opens the file with `O_RDONLY | O_NONBLOCK | O_NOFOLLOW` (`readPrivate`) and enforces: - Regular file (not a symlink, FIFO, or device), owned by `os.Geteuid()`, with no group or world permissions (`fi.Mode().Perm() & 0o077 == 0`), and size $\le 1\text{ MiB}$ (`configMax`). - Strict JSON decoding (`DisallowUnknownFields`) with `"version": 1` (`ConfigVersion`). - Non-empty `password` containing no whitespace, `listen` set to a valid loopback address (`127.0.0.1:` or `[::1]:`), `open` mutually exclusive with `builtin_policies`/`policies`/`deny_policies`, paired `ca`/`ca_key` and `ca_next`/`ca_next_key`, and valid `captures` entries. Once listening, `serve` writes its bound `127.0.0.1:` address to the status file in `DEVVM_DAEMON_STATUS` (`daemon.Ready`) and spawns `watchLifeline`, which polls the Lima hostagent PID file (`Config.Lifeline`, `ha.pid`) every 3 seconds (`lifelineTick`) after waiting up to 15 minutes (`lifelineWait`) for the hostagent to start. When the hostagent exits, `devvm _proxy` closes its listener and terminates. ### `proxy.Config` schema reference Configuration schema version. Must be `1` (`proxy.ConfigVersion`). Loopback `ip:port` to bind (`127.0.0.1:`, or port `0` for build proxies). VM identifier written to every JSON-lines event in `network.jsonl`. Per-VM proxy secret (`netpolicy.Key`). Validated in constant time (`subtle.ConstantTimeCompare`) against `Proxy-Authorization: Basic` with user `devvm`. Ordered asset paths in `assets.FS` (for example, `layers/base.policy`, `shared-tools/node.policy`) loaded once at proxy startup (`readBuiltins`). Any missing file or invalid line aborts proxy startup. Ordered file paths on the host re-read on change (`watched[policyFile]`) where both allow and deny rules apply. Ordered file paths on the host evaluated after `policies` where allow rules are stripped and only deny rules (`!rule`) and broken denies apply (used on `internet` and `none` tiers). When `true`, disables all policy rules and the internal address floor. Cannot be combined with `builtin_policies`, `policies`, or `deny_policies`. Paths to the VM's current and successor CA certificate (`0o644`) and ECDSA private key (`0o600`) files in `vms//run/ca/`. Path to `vms//run/secrets.json` (`proxy.BindingList`), hot-reloaded on modification. OAuth token endpoints (`agent`, `host`, `path`) intercepted on port `443` to capture agent logins onto the host. Corporate proxy configuration (`https`, `http`, `no_proxy`, `secret`), PEM bundle of user CAs from `~/.config/devvm/ca`, and build-proxy flag allowing unresolved hostnames matched only by `**` to be forwarded upstream. Host directory (`~/.local/share/devvm/cache/nix-cache`) and byte cap (default `10 GiB`, `NixCacheDefault`) for serving `http://devvm.proxy/nix/`. Path to `vms//run/links.json` and the socket directory `~/.local/share/devvm/run`. SHA-256 digests of the `devvm` binary and both CA certificates. Included in `proxy.json` so `Config.Sum()` changes whenever `devvm` is upgraded or CAs rotate, triggering an automatic proxy restart. --- ## Policy files, rule grammar, and the address floor ### Policy evaluation order `netpolicy.Policies` constructs the `PolicySet` for a VM based on its tier, image layers, and enabled shared tools. On `allowlist`, policies are concatenated in this order: | Order | Source | Path / Asset | Scope | | --- | --- | --- | --- | | 1 | Base built-in policy | `assets/layers/base.policy` | OS package mirrors (Ubuntu, Debian, Alpine), `mise-versions.jdx.dev`, Sigstore/GitHub TUF roots, `nodejs.org`, and GitHub (`github.com`, `github.com:22`, `api.github.com`, `codeload.github.com`, `raw.githubusercontent.com`, `objects.githubusercontent.com`, `release-assets.githubusercontent.com`, `lfs.github.com`) | | 2 | Shared tool policies | `assets/shared-tools/.policy` | Registries and APIs for each shared tool in the VM (`node`, `python`, `rust`, `go`, `dotnet`, `claude`, `codex`, `opencode`) | | 3 | Built-in layer policies | `assets/layers/.policy` | Built-in image layers (`docker`, `nix`, `desktop`) | | 4 | Custom layer policies | `~/.config/devvm/layers/.policy` | User-authored layers included in the VM's image | | 5 | Global user policy | `~/.config/devvm/policy` | Rules managed with `devvm network allow\|deny\|rm -g` | | 6 | Per-VM user policy | `~/.local/share/devvm/vms//policy` | Rules managed with `devvm network allow\|deny\|rm` in the project folder | | 7 | Bound secrets policy | `~/.local/share/devvm/vms//run/secrets.policy` | `:443` rules generated for bound secrets, agent logins, and agent OAuth token endpoints | When `policy.verdict(host, port)` evaluates a request (`internal/proxy/policy.go`): 1. If `policy.open` is `true`, the request is immediately allowed with rule `"open"`. 2. If any watched policy file contains a **broken deny** (`BrokenDeny`), the proxy refuses all requests (`Refused: true`). 3. Every rule across all policies is scanned for a matching **deny** (`r.deny && match(r)`). If any deny matches, the request is blocked with `"denied by " + r.text`. Denies always beat allows regardless of file order. 4. Every rule is scanned for a matching **allow** (`!r.deny && match(r)`). The first matching allow rule permits the request to proceed to the address floor check. 5. Otherwise, the request is blocked with `"not on the allowlist"`. ### Rule syntax and `parseRule` validation `proxy.ParsePolicy` strips UTF-8 BOMs, splits on `#` comments, trims whitespace, and validates each non-empty line with `parseRule`: | Pattern | Port behavior | Description | | --- | --- | --- | | `example.com` | `80` and `443` (`port == 0`) | Exact hostname match (case-insensitive; one trailing dot stripped). | | `*.example.com` | `80` and `443` | Matches any subdomain at any depth (does not match bare `example.com`). | | `**` | `80` and `443` | Matches any hostname or public IP address. Does **not** bypass the internal address floor. | | `10.0.0.5` or `10.0.0.0/8` | `80` and `443` | Matches an IPv4/IPv6 address or CIDR prefix (and NAT64-embedded IPv4 addresses). Required to reach internal/private addresses past the floor. | | `:22` or `[fd00::1]:8080` | Port `22` / `8080` only | Restricts the rule to a single TCP port (`1`–`65535`). | | `:*` or `**:*` | Any port (`1`–`65535`) | Matches all valid TCP ports (`anyPort = -1`). | | `!` | Same as `` | Deny rule; takes precedence over any allow rule. | **Fail-closed broken denies (`proxy.BrokenDeny`):** A line that is not a valid rule (`!l.OK`) is normally logged to `proxy.log` and skipped. However, if an invalid line starts with `!` (for example, `! example.com` with a space or `!*.10.0.0.1`), `BrokenDeny` returns `true` and the proxy denies **all** traffic for that VM until the line is fixed or removed. ### Hot-reloading policy files (`watched`) User policy files, `secrets.json`, and `links.json` are wrapped in `watched[T]` (`internal/proxy/watched.go`). On every request, `watched.get()` runs `os.Stat` (or `os.Lstat` for private files) and re-reads the file when: - Its modification time (`mtime`), byte size, or inode changed since the last read, **or** - The last read's `mtime` was within 2 seconds of the read timestamp (preventing missed updates when two atomic writes land inside a single filesystem `mtime` tick). Policy files are read via `proxy.ReadRegular` with `O_NONBLOCK` and a `1 MiB` cap (`PolicyMax`), preventing a FIFO or `/dev/zero` symlink from hanging or exhausting memory. If a policy file is deleted (`fs.ErrNotExist`), it contributes zero rules; if reading fails for any other reason (such as permissions or `ErrNotRegular`), `watched` retains the last known rules (`keep: true`) so a transient read error never drops a deny rule. ### Hostname normalization and the internal address floor Before checking policy rules, `splitHostPort` lowercases the target host and strips one trailing dot, and `proxy.HostOK` rejects malformed or ambiguous targets with `HTTP 400 Bad Request`: - Rejects IPv6 zone identifiers (`fe80::1%eth0`), empty labels (`evil..test`, `.evil.test`, `evil.test..` which becomes `evil.test.` after stripping one dot), and names longer than 253 characters. - Rejects hostnames whose final label is purely numeric or starts with `0x` (such as `127.1`, `10.1`, `2130706433`, or `0x7f000001`), preventing `inet_aton` shorthand addresses from bypassing the address floor at an upstream corporate proxy. After a hostname or IP passes `policy.verdict`, `egress.resolve` resolves the hostname on the host via `net.DefaultResolver.LookupNetIP` and filters the returned IPs against the **address floor** (`hostAddrs.internal`): - **Host interfaces (`hostAddrs`):** Refreshed every 30 seconds via `net.InterfaceAddrs()`. - **Non-public ranges:** Loopback (`127.0.0.0/8`, `::1`), RFC 1918 / RFC 4193 private addresses, link-local unicast and multicast, interface-local multicast, unspecified (`0.0.0.0`, `::`), CGNAT (`100.64.0.0/10`), `"this network"` (`0.0.0.0/8`), IETF protocol assignments (`192.0.0.0/24`), benchmarking (`198.18.0.0/15`), reserved (`240.0.0.0/4`), local-use NAT64 (`64:ff9b:1::/48`), IPv4-compatible IPv6 (`::/96`), and site-local IPv6 (`fec0::/10`). - **NAT64 unwrapping:** Any address in `64:ff9b::/96` (`nat64IPv4`) is judged by its embedded 32-bit IPv4 address, so `64:ff9b::7f00:1` (`127.0.0.1`) is treated as internal and matches IPv4 deny/allow rules. An internal address is dialed only if an explicit IP/CIDR allow rule matches it (`pol.addrAllowed(a, port)`) and no IP/CIDR deny rule matches (`!pol.addrDenied(a, port)`). `egress.dial` dials the resolved `netip.AddrPort` values directly rather than passing the hostname to `net.Dial`, eliminating DNS rebinding between policy evaluation and connection establishment. --- ## TLS SNI inspection, secret injection, and OAuth capture ### Unintercepted `CONNECT` tunnels and SNI verification When a `CONNECT host:port` request is allowed and does not require TLS interception, `server.tunnel` dials the target and splices bytes bidirectionally (`internal/proxy/splice.go`, `internal/proxy/sni.go`): 1. **Server-first compatibility:** Bytes from the upstream server flow to the VM immediately after `200 Connection Established`, so protocols where the server speaks first (such as SSH or SMTP) deliver their banner without waiting for the client. 2. **Client-side TLS peek (`peekClientHello`):** On the client-to-upstream stream, the proxy peeks at the first byte (`r.Peek(1)`). If it is not `0x16` (TLS Handshake record), or the second byte in the 5-byte header is not `0x03`, client bytes are forwarded immediately without delay. 3. **In-memory SNI parsing (`sniOf`):** If the client stream begins with a TLS record, `peekClientHello` reads the full record (`5 + length` bytes) and passes it to `tls.Server` backed by a read-only `helloConn` buffer whose `GetConfigForClient` callback extracts `ClientHelloInfo.ServerName` and aborts the handshake. 4. **Domain-fronting check (`sniMatches`):** - For a DNS hostname in `CONNECT`, the TLS SNI (with any trailing dot stripped) must match `host` case-insensitively. - For an IP literal in `CONNECT`, the TLS SNI must be empty or match the IP literal. - If the SNI does not match, `server.tunnel` logs a `blocked` event (`TLS name "" doesn't match `) and closes the connection before a single byte of the `ClientHello` reaches the remote server. ### TLS interception and secret injection on port 443 Project VMs never receive raw tokens or API keys. Instead, guest environments and agent configuration files receive salted placeholders (such as `CLAUDE_CODE_OAUTH_TOKEN`, `GH_TOKEN`, or JWT-shaped placeholders in `~/.codex/auth.json` and `~/.local/share/opencode/auth.json`), and `vms//run/secrets.json` lists the active `proxy.Binding` rules: ```json { "version": 1, "bindings": [ { "name": "github", "kind": "secret", "hosts": ["api.github.com", "github.com", "codeload.github.com", "raw.githubusercontent.com"], "header": "Authorization", "placeholder": "gho_devvm_proxy_placeholder_4c1e9b2d..." } ] } ``` When a `CONNECT` request targets port `443` (`s.interceptPorts[443]`) and matches either a bound secret (`len(s.secrets.forHost(host)) > 0`) or an OAuth capture host (`s.captures.has(host)`), `server.intercept` terminates TLS: 1. **Client TLS handshake:** Uses `s.ca.leafFor(host)` to present a leaf certificate signed by the VM's active CA, negotiating `h2` or `http/1.1` with a 30-second handshake timeout. 2. **Host header pinning:** Every HTTP request inside the tunnel must have a `Host` header matching the `CONNECT` host. If a client sends a request for another host over the same HTTP/2 or keep-alive connection, the proxy responds with `421 Misdirected Request`. 3. **Credential-minting guard (`refuseMinting`):** On `api.github.com`, `mintsCredential` blocks requests that would use an injected GitHub token to create persistent credentials that outlive `devvm secret unuse`: - `POST /user/keys`, `POST /user/ssh_signing_keys`, `POST /user/gpg_keys` (account keys) - `POST /repos/{owner}/{repo}/keys` and `POST /repositories/{id}/keys` (deploy keys) - `POST /.../{repos,repositories,orgs,organizations,enterprises}/.../actions/runners/{registration-token,generate-jitconfig}` (self-hosted runners) - `POST` or `PATCH` to `/.../{repos,repositories,orgs,organizations}/.../hooks` (webhooks) 4. **In-process secret lookup and header injection (`secrets.inject`):** - For each matching `Binding`, if the request's `b.Header` (such as `Authorization`) contains `b.Placeholder` directly or base64-encoded inside a `Basic ` credential, `s.secrets.value(b)` resolves the secret in-process via `IO.Secret` (`secrets.Store.Get`). - Lookups are serialized under `valueMu` with a 20-second timeout (`lookupTimeout`), routed through the proxy's configured upstream corporate proxy (`WithRoute`) and custom CAs (`WithRoots`), and cached in memory for **5 minutes** (`valueTTL` for `KindSecret`) or **30 seconds** (`loginTTL` for `KindLogin`). - Touching or rewriting `secrets.json` (such as via `devvm secret unuse` or `devvm secret set`) triggers the `watched` `changed` callback, immediately flushing cached secret values even for already-open tunnels. ### OAuth token capture (`internal/proxy/capture.go`) Every VM's `proxy.Config` includes `agentlogin.Endpoints()` in `captures`: - `auth.openai.com` at `/oauth/token` (`codex`, `opencode`) - `platform.claude.com` at `/v1/oauth/token` (`claude`) - `console.anthropic.com` at `/v1/oauth/token` (`claude`) When an agent logs in inside the VM, `server.forwarder` intercepts port `443` to those hosts: - **Path canonicalization:** Any `POST` whose cleaned path (stripped of `;param` segments and matched case-insensitively) equals the capture path is rewritten to the canonical endpoint path with query strings removed. - **Request inspection (`captureRequest`):** Reads the request body up to `64 KiB` (`captureRequestMax`, with at most 8 concurrent reads via `captureReads`). If a VM sends its own placeholder refresh token (`devvm-proxy-managed`), the proxy short-circuits the request locally and returns fresh placeholders from the host store; if Codex performs an `id_token` exchange for an `openai-api-key`, the proxy substitutes the real `id_token` from the host store. - **Response capture (`captureResponse`):** Strips `Accept-Encoding` on outbound requests so Go's transport decompresses gzip transparently, drops `1xx` informational responses (`no1xx`) and `101` upgrades, reads `2xx` responses up to `1 MiB` (`captureReplyMax`), persists the real tokens to `~/.local/share/devvm/logins//` on the host, rewrites `vms//run/secrets.json`, and returns a sanitized JSON body to the VM containing salted placeholders, `"refresh_token": "devvm-proxy-managed"`, and an expiry in `2099`, keeping only `Content-Type`, `Cache-Control`, `Pragma`, and `Date` response headers. - **Unmatched path guard (`guard`):** Any other `2xx` response from a capture host (outside static HTML, CSS, JS, image, or font assets) whose body contains `"access_token"`, `"refresh_token"`, or `"id_token"` is refused with `502 Bad Gateway` so an alternate endpoint on the same host cannot leak raw tokens into the VM. --- ## Per-VM CA lifecycle and rotation Each VM has its own private Certificate Authority in `~/.local/share/devvm/vms//run/ca/` (`internal/trust/ca.go`). Because no two VMs share a CA, a compromised CA key in one VM's run directory cannot sign certificates trusted by any other VM or by the host. ```mermaid sequenceDiagram autonumber participant Launch as devvm launch (trust.InitCA) participant Disk as vms//run/ca/ participant Guest as VM Guest (/usr/local/share/ca-certificates) participant Proxy as devvm _proxy (proxy.ca) Launch->>Disk: Lock vms//run/ca/lock Launch->>Disk: Write ca.key (0600) & ca.pem (0644)
NotBefore: T0 - 1h, NotAfter: T0 + 30d Launch->>Disk: Write next.key (0600) & next.pem (0644)
NotBefore: T0 + 23d - 1h, NotAfter: T0 + 53d Launch->>Guest: assets/guest/ca.sh installs
devvm-proxy.crt (ca.pem) & devvm-proxy-next.crt (next.pem) Launch->>Proxy: Start with CA + CANext Note over Proxy: Days 0..23: signer(now) uses ca.pem
Mints 24h leaf certs (LRU cache max 1024) Note over Proxy: Day 23 (next.NotBefore + 1h):
signer(now) switches to next.pem automatically
(no VM launch or proxy restart needed) Launch->>Disk: Next launch after Day 23:
Promotes next -> ca, generates new next ``` ### CA timing constants and rotation rules `trust.InitCA` manages two ECDSA P-256 certificate/key pairs under `~/.local/share/devvm/vms//run/ca/lock`: | Constant / File | Value / Mode | Behavior | | --- | --- | --- | | `caLife` | `30 * 24 * time.Hour` (30 days) | Validity duration of each generated CA certificate (`NotAfter - NotBefore = caLife + caSkew`). | | `caRenew` | `7 * 24 * time.Hour` (7 days) | Renewal threshold before `cur.NotAfter`. When `cur` has $\le 7\text{ days}$ remaining, `InitCA` promotes `next` to `ca`. | | `caSkew` / `successorSkew` | `1 * time.Hour` | Clock-skew buffer. A new `cur` starts at `now - 1h`; `next` starts at `cur.NotAfter - 7d - 1h`, and the proxy begins signing with `next` at `next.NotBefore + 1h` (`cur.NotAfter - 7d`). | | `ca.pem` / `ca.key` | `0o644` / `0o600` | Current CA certificate and ECDSA P-256 private key. Key is always written before certificate so an interrupted write fails `tls.X509KeyPair` and is regenerated. | | `next.pem` / `next.key` | `0o644` / `0o600` | Successor CA certificate and private key, installed alongside `ca.pem` in the VM's trust store on every launch. | Each generated CA sets `IsCA: true`, `BasicConstraintsValid: true`, `MaxPathLenZero: true`, `KeyUsageCertSign | KeyUsageCRLSign`, and a unique 128-bit random serial number embedded in `Subject.CommonName` (`"devvm proxy CA "`) so `cur` and `next` never collide on subject name lookups. ### Leaf certificate minting and expiration handling Inside `devvm _proxy` (`internal/proxy/secrets.go`): - **Automatic signer switch (`ca.signer`):** At timestamp `now`, if `now >= next.NotBefore + 1h`, `signer(now)` returns `next`; otherwise it returns `cur`. Because both certificates were installed in the VM at launch, a long-running VM continues verifying intercepted TLS connections for at least `30 days - 1 hour` after its last `devvm` launch without restarting `devvm _proxy`. - **Leaf caching (`ca.mint`):** Mints a 24-hour leaf certificate (`DNSNames: []string{host}`, clamped to the signing CA's `NotBefore` and `NotAfter`) using a single per-CA ephemeral ECDSA P-256 `leafKey`. Keeps up to `1024` (`leafCacheMax`) leaf certificates in an LRU list; cached leaves are reused while they have more than 1 hour remaining (or until they reach the CA's `NotAfter`). - **Expired CA refusal (`caEndedRefusal`):** If a VM runs for more than 30 days without a `devvm` launch so that `now >= signer(now).NotAfter`, the proxy refuses intercepted `CONNECT` requests on port `443` with `403 Forbidden` (`Refused: true`) and logs: ```text devvm's CA has ended: open the VM again (devvm in its folder) for a new one ``` Running `devvm` in the project folder rotates the CAs, restarts the proxy, and installs the new certificates in the guest. ### Guest and custom CA installation (`assets/guest/ca.sh`) At every launch, `sync.sh` runs `assets/guest/ca.sh` as root with a directory containing `devvm-proxy.crt` (`ca.pem`), `devvm-proxy-next.crt` (`next.pem`), and any user CAs (`devvm-extra-1.crt`, ...) validated by `trust.Read` from `~/.config/devvm/ca`: 1. Copies changed certificates into `/usr/local/share/ca-certificates/` (`0o644`) and deletes any removed `devvm-*.crt` files. 2. Runs `update-ca-certificates --fresh` (or updates `/etc/ssl/certs/ca-certificates.crt` directly while preserving the original bundle in `/var/lib/devvm-ca` on minimal images without `update-ca-certificates`) and notifies `/etc/ca-certificates/update.d` hooks (such as Java's `jks-keystore`) of removed certificates. 3. Writes the combined PEM bundle of devvm-managed CAs to `/etc/ssl/devvm.pem` (`trust.Bundle`). 4. Exports `trust.Env` across all guest sessions so runtimes that bypass the system store trust both the system bundle and `/etc/ssl/devvm.pem`: - `NODE_EXTRA_CA_CERTS=/etc/ssl/devvm.pem` - `SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt` - `REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt` - `PIP_CERT=/etc/ssl/certs/ca-certificates.crt` - `UV_SYSTEM_CERTS=1` --- ## Reserved endpoints, upstream proxies, and logging ### Reserved `http://devvm.proxy` endpoints Plain HTTP requests sent to the proxy with `Host: devvm.proxy` are handled internally by `server.serveReserved` (`internal/proxy/proxy.go`): | Endpoint | Method | Response | | --- | --- | --- | | `http://devvm.proxy/blocked` | `GET` | Plain-text list of the last 50 blocked requests with `devvm network allow ` hints, or `nothingBlocked` explaining that a DNS/connection error for an allowed host means the tool bypassed `$HTTPS_PROXY`. Queried by `devvm-why` inside the VM. | | `http://devvm.proxy/blocked/new?after=N` | `GET` | Returns the total block counter on line 1 followed by deduplicated block notices since block `N`. Used by the guest shell prompt hook. | | `http://devvm.proxy/ca.pem` | `GET` | Concatenated PEM certificates of the VM's current CA and next CA (never private keys). | | `http://devvm.proxy/nix/...` | `GET`, `HEAD` | Shared host-filled Nix binary cache (`internal/proxy/nixcache.go`) when the VM image includes the `nix` layer. Serves `nix-cache-info` (priority `30`), `<32-base32>.narinfo`, and `nar/<52-base32>.nar[.xz\|.zst\|.bz2]`. Fetches misses from `https://cache.nixos.org` only when `cache.nixos.org:443` is allowed, verifies the pinned `cache.nixos.org-1` Ed25519 signature on each `narinfo` and the SHA-256 `FileHash`/`FileSize` on each NAR, and enforces an LRU disk cap (`nix-cache.size`, default `10 GiB`) under `~/.local/share/devvm/cache/nix-cache/.lock`. | ### Corporate upstream proxy routing (`internal/proxy/upstream.go`) `netpolicy.Proxy.Upstream` reads `HTTPS_PROXY`, `HTTP_PROXY`, and `NO_PROXY` from the host environment, or falls back to the `[proxy]` table in `~/.config/devvm/config.toml` (never mixing the two). - Only `http://` upstream proxies are supported (`ParseUpstream` returns `ErrUpstreamScheme` for `https://` or `socks5://`). - `config.toml` URLs cannot contain inline passwords; if a `[proxy]` URL includes a username (`http://alice@proxy.example.com:8080`), `Upstream.Secret` is set to `"proxy"`, and `devvm _proxy` looks up the `proxy` secret saved via `devvm secret set proxy` at startup. - When routing through an upstream proxy, `egress.floorUpstream` still resolves the hostname locally with a 3-second timeout and blocks the request if any returned IP is internal or denied. If the host resolver cannot resolve the name, `floorUpstream` forwards it to the corporate proxy only if an explicit hostname or `*.domain` rule allows it (`pol.byName(host, port)`), refusing unresolved names matched only by `**` (`errUnresolved`) unless `UpstreamUnresolved` is set (used only by image build proxies). ### Logs and `devvm network` commands Each VM maintains two log files in `~/.local/share/devvm/vms//logs/` (mode `0o600`), rotated at `10 MiB` (`proxy.LogLimit`): - `network.jsonl` (rotated by renaming to `network.jsonl.1` under an `flock` lock): JSON-lines decision log written by `devvm _proxy` and `devvm _links`. Records `tunnel`, `intercept`, `forward`, `blocked`, `login`, `nix-cache`, `link`, `link_refused`, `link_target_stopped`, and `link_limit` events. Never logs secret values or upstream proxy credentials. - `proxy.log` (rotated by copying to `proxy.log.1` and truncating in place): Diagnostic stderr log of `devvm _proxy`, including loaded policy counts, skipped/broken rules, and upstream DNS or connection errors withheld from the guest. ```bash title="Inspect tier, rules, and logs" # Show the folder's VM tier and ordered policy rules devvm network # Switch the VM to a stricter or wider tier devvm network none devvm network allowlist # Add or remove per-VM or global (-g) policy rules devvm network allow api.example.com '*.cdn.example.com:8443' devvm network deny '!telemetry.example.com' devvm network rm api.example.com devvm network allow -g pypi.org # Inspect recent events or deduplicated blocked hosts devvm network log -n 100 devvm network log --follow devvm network log --blocked ``` ```json title="devvm network --json" { "version": 1, "policy": "allowlist", "allowlist": [ { "builtin": "layers/base.policy", "rules": [ "ports.ubuntu.com", "github.com", "github.com:22", "api.github.com" ], "not_rules": [] }, { "file": "/Users/you/.local/share/devvm/vms/71a99b1076d92c/policy", "rules": [ "api.example.com" ], "not_rules": [] } ] } ``` ```json title="devvm network log --blocked --json" { "version": 1, "blocked": [ { "last": "2026-10-09T10:01:00Z", "count": 2, "host": "example.com", "port": 443, "allow": "example.com" } ] } ``` ## Related pages Run Claude Code, Codex, and opencode inside VMs, capture OAuth logins, and bind host secrets to allowed hosts. Route traffic between VMs over 127.77.0.0/16 sockets, enforce link tier rules, and forward guest ports to the host. Syntax, subcommands, and flags for devvm network, secret, agent, git-protect, clone, ssh-agent, link, and group. Inspect blocked requests with devvm network log, configure corporate proxies and custom CAs in ~/.config/devvm/ca, and recover disk space. --- ## 07. Host and Git isolation boundaries > Read-only .git mount protection, symlink and hardlink containment, set-id file audits, secret placeholder injection, and isolated repo clones. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/07-host-and-git-isolation-boundaries.md - Generated: 2026-11-07T22:02:25.642Z ### Source Files - `internal/gitprotect/gitprotect.go` - `internal/gitprotect/own.go` - `internal/hostsafety/hostsafety.go` - `internal/hostgit/hostgit.go` - `internal/vm/clone.go` - `internal/secrets/secrets.go` --- title: "Host and Git isolation boundaries" description: "Read-only .git mount protection, symlink and hardlink containment, set-id file audits, secret placeholder injection, and isolated repo clones." --- `devvm` isolates host filesystems, Git repositories, and credentials from untrusted code running with passwordless `sudo` inside project VMs across six package boundaries: `internal/project` (mount boundary validation), `internal/gitprotect` (immutable `.git` hooks and config locking plus repository ownership checks), `internal/hostgit` (hardened host-side `git` subprocess execution), `internal/hostsafety` (Linux set-user-id and set-group-id file audits), `internal/secrets` with `internal/agentlogin` (salted placeholder injection and settings redaction), and `internal/vm` (isolated `--clone` repositories and tar stream sanitization). ## Host and guest boundary architecture A standard project VM mounts only its project directory read-write at the same path, plus shared EROFS tool images and agent configuration directories (`~/.local/share/devvm/agents/`) read-only. It never mounts the host home directory (`~`), `~/.ssh`, or `devvm` state directories. ```mermaid flowchart TB subgraph Host["Host Machine"] direction TB CLI["devvm CLI & Starter (internal/vm)"] GitProtect["Git Protector (internal/gitprotect)
chflags uchg / nouchg"] HostGit["Hardened Git Runner (internal/hostgit)
GIT_CONFIG_COUNT & GIT_SSH_COMMAND"] HostSafety["Host Safety Audit (internal/hostsafety)
SetIDFiles & SSHRemote"] SecretStore["Secrets & Login Store (internal/secrets)
macOS Keychain / 0600 files"] Proxy["Per-VM Host Proxy (devvm _proxy)
TLS SNI & Header Injection"] ProjectDir[("Host Project Folder
Read-Write Mount + Locked .git")] end subgraph Guest["Project VM (Untrusted Guest Root)"] direction TB GuestWorktree["Mounted Worktree or
Isolated Clone (devvm --clone)"] GuestEnv["Guest Environment & Agent Configs
Salted Placeholders + devvm-redacted"] end CLI --> GitProtect CLI --> HostSafety CLI --> HostGit CLI --> SecretStore GitProtect -- "Locks hooks, config,
worktrees, submodules" --> ProjectDir HostSafety -- "Scans setuid/setgid
(skips .git & other devices)" --> ProjectDir ProjectDir <-->|"virtiofs / 9p ( or omitted in --clone )"| GuestWorktree HostGit <-->|"Pinned SSH receive-pack / upload-pack"| GuestWorktree SecretStore -- "Writes bindings &
per-VM salt" --> Proxy SecretStore -- "In-memory tar stream
(sync.sh)" --> GuestEnv GuestEnv -- "HTTPS with placeholder" --> Proxy ``` ### Unsafe project directory refusals Before creating or starting a VM that mounts a project folder, `project.Unsafe` (`internal/project/project.go`) resolves symlinks via `paths.Physical` and rejects directories that would expose host credentials, `devvm` binaries, layer build scripts, or other VMs' disks: | Check method | Condition | Refused paths | Threat prevented | | --- | --- | --- | --- | | `Unsafe.Holds(dir)` | `paths.Within(paths.Physical(p), dir)` | `$HOME`, `~/.config/devvm`, `~/.local/share/devvm`, `~/.local/share/devvm/lima` (and `/`) | Mounting the user home directory, secret stores, proxy CA private keys, or VM disks read-write into the guest. | | `Unsafe.Inside(dir)` | `paths.Below(dir, paths.Physical(p))` | Subdirectories below `~/.config/devvm`, `~/.local/share/devvm`, or the Lima home | A VM inside `~/.config/devvm/layers` rewriting build scripts executed by future image builds, or modifying host binaries in the data directory. | | `project.Nth(dir, ...)` | Path contains `,` or `unicode.IsControl` | Any path containing a comma or control character (`BadPathError`) | Breaking Lima mount option parsing (which splits on commas) or line-oriented guest setup scripts. | Passing `--unsafe-folder` overrides `Unsafe.Holds` and `Unsafe.Inside` with a warning. A `--clone` VM does not mount the host project folder at all, so `devvm` does not apply unsafe-folder mount refusals to `--clone` VMs. ### Folder identity and nested project replacement checks `devvm` records the project directory's physical path (`st.Dir`) and filesystem identity (`st.Folder` via `platform.FolderID`, which combines device, inode, and birth time where supported) in `vm.json`. When a project with its own VM sits inside a parent folder that also has a VM, code inside the parent VM can replace the nested project directory with a symlink (for example, pointing to `$HOME`). `starter.bringUp` and `starter.start` (`internal/vm/start.go`) guard against this with `folderChanged(st)`: 1. Before running host checks on a stopped VM, `bringUp` verifies `filepath.EvalSymlinks(dir) == dir` and `platform.FolderID(dir) == st.Folder`. If either fails, startup aborts with `project_folder_changed`. 2. Immediately before `b.Start` and again immediately after `b.Start` returns, `starter.start` re-evaluates `folderChanged(st)`. If the directory was swapped during boot, `devvm` immediately stops the VM and proxy and returns `project_folder_changed_stopped`. --- ## Read-only `.git` mount protection Because the project directory is mounted read-write, a guest process could write `.git/hooks/post-checkout` or set `core.fsmonitor`, `core.hooksPath`, `core.sshCommand`, or `filter.*.clean` in `.git/config` to execute arbitrary code on the host during the next host `git` invocation. A read-only bind mount inside the guest cannot prevent this because guest `root` can `umount` it. On macOS, `gitprotect.Protector` (`internal/gitprotect/gitprotect.go`) locks executable and configuration paths inside `.git` on the host filesystem using the BSD user-immutable flag (`chflags uchg`), which guest `root` cannot clear across `virtiofs`. Linux hosts do not provide an unprivileged user-immutable file flag that blocks guest writes across `9p` or `virtiofs`. On Linux, `Protector.CanLock` (`platform.HasImmutableFlag`) is `false`: `devvm` warns with `git_protect_needs_macos_new` when creating a VM for a repository, sets the mode to `off`, and refuses `devvm git-protect on` or `strict`. Use `devvm --clone` on Linux hosts to isolate `.git`. ### Protected paths and pre-created placeholders `Protector.paths` and `Protector.lock` discover and lock the following paths across the top-level `.git` directory, `.git/modules/**` (any directory containing `HEAD`, `objects/`, and `refs/`), and working-tree submodules discovered via `git -c core.fsmonitor=false ls-files -s -z` (mode `160000`, up to `maxDepth = 8`): | Path pattern | Lock behavior when present | Behavior when missing | | --- | --- | --- | | `/hooks` | Locked recursively (`chflags -R uchg` without following symlinks) | Created empty (`os.Mkdir(path, 0o700)`) and locked so the VM cannot create `.git/hooks` later. | | `/config` | Locked (`chflags uchg`) if regular file | Skipped until created. | | `/config.worktree` | Locked when `extensions.worktreeConfig` is `true` | Created empty (`O_WRONLY\|O_CREATE\|O_EXCL`, mode `0600`) and locked when `extensions.worktreeConfig` is `true`. | | `/worktrees//commondir` | Locked if regular file | Skipped. | | `/worktrees//config.worktree` | Locked when `extensions.worktreeConfig` is `true` | Created empty and locked when `extensions.worktreeConfig` is `true`. | | `.git` (when a regular file) | Locked (`chflags uchg`) in a linked worktree | N/A | ### Protection modes and state schema A VM's `.git` protection state is stored under the `"git"` section (`gitprotect.InVM`) in `~/.local/share/devvm/vms//vm.json`: ```json { "mode": "on", "next": "strict", "made": { "protected": true, "git_id": "16777220:4829103:1730990000", "git_dirs": [ ".git/modules/sub", "vendor/sub/.git" ], "delegations": "9a4f2c81e03b7d12" } } ``` | Mode | Description | | --- | --- | | `on` | Default on macOS (unless `--no-git-protect` is passed). Locks all discovered hooks, config, worktree, and submodule paths while the VM is stopped. | | `strict` | Locks everything `on` locks, and temporarily unlocks `.git/config` to set `diff.ignoreSubmodules = all` before re-locking it. Prevents host `git status` and `git diff` from reading newly added submodule configs before the next VM stop/launch locks them. Switching back to `on` or `off` unsets `diff.ignoreSubmodules`. | | `off` | Unlocks all currently discovered paths plus every path previously recorded in `made.git_dirs` (so submodules removed from `.git/index` by the VM are also unlocked). | | `clone` | Set automatically for `--clone` VMs, which mount no host `.git` directory. Cannot be changed with `devvm git-protect`. | ### Race-free locking lifecycle and launch audits `devvm` never modifies `uchg` flags while a VM is running because a process in the VM could race the check and swap a directory for a symlink. - **Changing mode on a running VM:** `devvm git-protect ` saves the target mode in `git.next` (`gitprotect.SetNext`) and prompts to restart the VM immediately. If declined or run with `--no-input`, `devvm` applies `git.next` the next time it finds the VM stopped (`starter.hostChecks` or `devvm stop`). - **TOCTOU check on every flag change:** `Protector.flag` checks `realInDir(path)` (verifying the path is below `p.Dir`, is not a symlink via `os.Lstat`, and its parent resolves without symlinks via `filepath.EvalSymlinks`), records `before := devIno(path)`, applies `Chflags`, and then re-verifies `devIno(path) == before && p.realInDir(path)`. If the inode moved during the call, `Protector.flag` immediately clears `nouchg` at the destination and warns with `git_lock_raced`. - **Audits on launch and stop (`Protector.Check`):** On every launch of an existing VM, history export, and `devvm stop`, `protect` compares the live `.git` tree against `InVM.Made` and warns when it detects conditions a file lock cannot prevent: - **Replaced `.git` (`git_replaced`):** `Made.GitID` no longer matches `folderID(.git)` (the VM renamed `.git` and created a new one). `protect` warns and stops further lock updates so the user can inspect `.git`. - **`.git/commondir` present (`git_commondir`) or stray worktrees (`git_worktree_stray`):** `.git/commondir` exists in `.git`, or `.git/worktrees//commondir` does not resolve back to the repository's physical `.git` directory. - **New repository or submodule (`git_appeared`, `git_embedded_new`):** `.git` or a submodule git directory appeared since the last lock. - **Unlocked hooks (`git_hooks_unlocked`):** A `hooks` directory is unlocked and contains regular files other than `*.sample`. - **Working-tree hooks path (`git_hooks_path`):** `core.hooksPath` resolves inside the project directory. - **Delegations into the project tree or outside it (`git_include_inside`, `git_hook_link_inside`, `git_gitdir_outside`):** `include.path` or `includeIf.*.path` targets a file inside the project folder, a hook under `hooks/` is a symlink pointing into the project folder, or a submodule's `gitdir:` points outside the project folder. Hashed in `Made.Delegations` (first 16 hex chars of SHA-256) so unchanged delegations are only reported on creation/`on` or when changed. ```sh title="Inspect and switch .git protection" # Show current mode (on, strict, off, or clone) devvm git-protect devvm git-protect --json # Enable strict mode (sets diff.ignoreSubmodules=all in locked .git/config) devvm git-protect strict # Temporarily unlock .git on a stopped VM to edit .git/config on the host devvm stop devvm git-protect off git remote add upstream git@github.com:org/repo.git devvm git-protect on ``` --- ## Hardened host Git execution (`internal/hostgit`) `devvm` runs `git` on the host to read `user.name` and `user.email`, inspect `.git` and submodule indexes, verify `--clone` repositories, and push or fetch branches for `--clone` VMs. Because a mounted project repository (or a nested repository inside a parent VM's mount) can be modified by the guest, a structure test in `internal/structure` enforces that **every** host `git` invocation in `devvm` goes through `internal/hostgit`. ### Sanitized environment and config overrides `hostgit.Command(ctx, args...)` constructs an `*exec.Cmd` with a scrubbed environment (`environ`) and high-precedence `GIT_CONFIG_COUNT` environment key-value pairs that override any `.git/config`, worktree config, or global config: | Layer | Variable or config key | Value set by `hostgit` | Purpose | | --- | --- | --- | --- | | Environment | `GIT_*` (inherited) | Stripped completely | Prevents caller environment variables (`GIT_DIR`, `GIT_WORK_TREE`, `GIT_CONFIG_GLOBAL`, `GIT_INDEX_FILE`, `GIT_SSH_COMMAND`) from redirecting `git`. | | Environment | `PATH` | Filtered to `filepath.IsAbs(d)` only | Prevents `git` from executing binaries (such as `ssh` or helpers) from relative `PATH` entries inside the repo working tree. | | Environment | `GIT_TERMINAL_PROMPT` | `0` | Prevents interactive credential or host-key prompts on the terminal. | | Environment | `GIT_NO_LAZY_FETCH` | `1` | Prevents partial clones (`extensions.partialClone`) from connecting to a malicious promisor remote during `rev-list` or `status`. | | Environment | `GIT_ALLOW_PROTOCOL` | `ssh` | Overrides even per-protocol `protocol..allow=always` settings in `.git/config`; blocks `file://`, `ext::`, `http://`, `https://`, and `git://`. | | Environment | `GIT_SSH_COMMAND` / `GIT_SSH_VARIANT` | `false` / `ssh` | Disables network transport and overrides `core.sshCommand` and `ssh.variant` unless explicitly replaced by `clones.gitSSH`. | | `GIT_CONFIG_*` | `core.fsmonitor` | `false` | Disables `fsmonitor` daemon or Watchman hooks on `status` and `ls-files`. | | `GIT_CONFIG_*` | `core.hooksPath` | `os.DevNull` (`/dev/null`) | Disables all directory-based Git hooks. | | `GIT_CONFIG_*` | `core.alternateRefsCommand` | `true` | Neutralizes custom commands used to list refs in alternate object stores. | | `GIT_CONFIG_*` | `protocol.allow` / `protocol.ssh.allow` | `never` / `always` | Restricts transport protocols to SSH at the config level. | | `GIT_CONFIG_*` | `maintenance.auto` / `gc.auto` | `false` / `0` | Prevents background `git maintenance` or `git gc` from spawning on the host. | | `GIT_CONFIG_*` | `hook..enabled` | `false` (all 27 `githooks(5)` events) | Disables config-defined hooks (`hook.*`) introduced in newer Git versions. | When `hostgit.Command` is called with `-C `, `repoOff(ctx, dir)` first runs a hardened `git -C config --name-only --get-regexp ^(filter|hook)\.` query and appends config overrides for every named hook (`hook..enabled=false`) and every named filter driver (`filter..clean=""`, `filter..smudge=""`, `filter..process=""`, `filter..required=false`). In addition, `hostgit.Status(ctx, dir, args...)` always passes `--no-optional-locks` and `--ignore-submodules=all`. --- ## Symlink and hardlink containment Because a VM can plant symlinks or hardlinks inside mounted folders, agent configuration directories, tar streams, or `.git`, `devvm` enforces strict path resolution and containment across five distinct boundaries. ### 1. Path resolution primitives (`internal/paths/resolve.go`) `internal/paths` defines three canonical resolution strategies before comparing paths with `paths.Within(path, dir)` or `paths.Below(path, dir)`: - `paths.Physical(path)`: Resolves symlinks along the deepest existing prefix of `path` and joins the non-existent remainder as named. Used for checking project folders against `devvm`'s own directories (`project.Unsafe`) and checking where a not-yet-created directory will land. - `paths.PhysicalParent(path)`: Resolves symlinks in `filepath.Dir(path)` while keeping `filepath.Base(path)` unresolved. Used when the final path component may itself be a symlink that `devvm` must not follow, or a file the VM could create (such as paths under `.git`, recorded key paths, and Seatbelt profile rules). - `filepath.EvalSymlinks(path)`: Requires the entire path to exist and resolves all symlinks. Used for existing directories (`devvm` working directory, `devvm link` targets, `mise` binary paths). ### 2. `os.Root` confinement for host file reads Whenever `devvm` reads files from a directory that could contain untrusted symlinks, it opens the directory with Go's `os.OpenRoot` (`*os.Root`), which prevents symlink traversal (`..` or absolute links) outside the root at the OS syscall level: - **`.devcontainer/devcontainer.json`:** Opened via `os.Root` with `O_NONBLOCK` (so a FIFO planted by a VM does not block `devvm`), verified as a regular file on the open file descriptor, and capped at 1 MiB. - **`devvm agent import` (`internal/vm/agentimport.go`):** Opens `~/.claude`, `~/.codex`, or `~/.config/opencode` as an `*os.Root`. Regular files and symlinks that resolve to regular files *within* that `os.Root` are read and written as regular files into `~/.local/share/devvm/agents/`. Symlinks escaping the root, symlinks to directories, and symlinked login files (`~/.codex/auth.json`, `~/.claude.json`, etc.) are skipped with `agent_import_link_left_out`. Pruning stale files in the store (`pruneStore`) also operates through `os.OpenRoot`. ### 3. `devvm agent save` tar stream validation (`internal/vm/agentsave_read.go`) When `devvm agent save` reads modified agent settings back from a running VM, `AgentSave.read` streams a `tar` archive from the guest and validates every entry in memory via `readSent`: - **Resource caps:** Total stream capped at `saveMaxBytes` (128 MiB), stderr at `saveMaxStderr` (1 MiB), individual files at `saveMaxFile` (16 MiB), and total tar headers at `saveMaxEntries` (20,000). Exceeding any cap aborts the save before touching the host store. - **Path hygiene (`entryName`):** Every tar header name must be relative, clean (`path.Clean(name) == name`), contain no `..` segments, be valid UTF-8, and consist solely of graphic characters (`unicode.IsGraphic`, blocking terminal escape sequences and bidirectional overrides in diffs). - **Allowlisted hierarchy (`listedPath`):** Every entry must match `agentFilesOf` or `agentDirsOf` (or `opencode`'s `.config/opencode` excluding `node_modules` and lockfiles), and its parent directory must already exist in the stream as a real directory (`kind == 'd'`), rejecting entries placed beneath a symlinked directory. - **Bounded symlink resolution (`sent.resolve`):** Symlinks inside settings folders are never written to the host store as symlinks. Instead, `sent.resolve` follows relative symlinks in memory within the same top-level settings folder (`top`) for at most `saveLinkHops = 8` hops through real directories in the stream. If the chain resolves to a regular file inside `top`, it is saved as a regular file copy; otherwise, the symlink is skipped with `agent_save_link_left_out` and the existing store entry is preserved. ### 4. Remake home directory tar filter (`internal/vm/home.go`) When `devvm layer add`, `devvm layer rm`, `devvm update`, or `devvm clone` remakes a VM, `saveHome` streams the old VM's home directory (`tar -czf - --one-file-system --ignore-failed-read .`) through `copyHome` in Go memory without ever unpacking it on the host: ```go title="internal/vm/home.go" name, ok := homePath(h.Name) if !ok || skipped(name, skips) || throughLink(name, kept) || !keepType(h, kept) { continue } out := &tar.Header{ Typeflag: h.Typeflag, Name: name, Mode: h.Mode & 0o777, ModTime: h.ModTime, Format: tar.FormatPAX, } ``` - **Path and symlink parent check:** `homePath` drops `.`, absolute paths, `..` prefixes, and control characters (`< 0x20` or `0x7f`). `throughLink(name, kept)` walks `path.Dir(name)` upward and drops any entry whose ancestor was previously recorded in `kept` as `tar.TypeSymlink`, preventing `tar -xzf` in the new VM from following a parent symlink outside `~`. - **Hardlink containment (`keepType`):** Only `tar.TypeReg`, `tar.TypeDir`, `tar.TypeSymlink`, and `tar.TypeLink` (hardlinks) are kept; FIFOs, block/char devices, and sockets are dropped. A `tar.TypeLink` entry is kept **only** if its `h.Linkname` passes `homePath` and names a `tar.TypeReg` file that was already kept earlier in the same stream (`kept[target] == tar.TypeReg`), blocking hardlinks to `/etc/shadow` or files outside the kept home tree. - **Mode and owner stripping:** `Mode` is masked with `0o777` (stripping `setuid`, `setgid`, and `sticky` bits), and `Uid`/`Gid`/`Uname`/`Gname` are omitted. - **Setup exclusions (`homeSkips`):** `.config/devvm`, `.ssh/authorized_keys`, `.cache`, agent login files, and mounted/copied agent settings are stripped on the host and regenerated fresh after `tar -xzf - --skip-old-files` restores the archive in the new VM. ### 5. History import symlink and SQLite sanitization (`internal/vm/agents.go`) - **Archive path and symlink filter (`historyImport`):** When importing an agent history `.tar.gz` inside a VM, `historyImport` lists archive members with `tar -tzf "$t/export" --absolute-names --quoting-style=escape` and aborts with `devvm-history-unsafe` if any path falls outside `historyPaths` (`.claude`, `.claude.json`, `.codex`, `.local/share/opencode`, `.local/state/opencode`) or contains `..`. After unpacking into a temporary directory with `--no-same-owner --no-same-permissions --anchored` and excluding `importSkips()`, `find "$t/x" -type l` deletes every symlink whose target starts with `/`, is `..`, or contains `../`, `/..`, or `/../`. - **Opencode SQLite WAL login scrubbing (`opencodeClean`):** Because opencode 2 stores sessions and credentials together in `.local/share/opencode/opencode*.db` in WAL mode, both `historyExport` and `historyImport` exclude raw `.db`, `.db-wal`, `.db-shm`, and `.db-journal` files and run `devvm_opencode_clean` inside the VM using `sqlite3 -bail -cmd 'PRAGMA trusted_schema = OFF'`. It snapshots the database and WAL via `VACUUM INTO 'clean.db'`, drops all triggers, deletes rows from all credential/account/token/share tables (`opencodeLogins`) plus `wellknown:config:%` rows in `kv` and the `20260805200742_import_legacy_credentials` migration record, runs `PRAGMA journal_mode = DELETE; PRAGMA secure_delete = ON; VACUUM;` to purge free pages, and verifies zero rows remain before including `clean.db`. --- ## Set-id file audits (`internal/hostsafety`) Inside a VM, the default user has passwordless `sudo` and can `chmod u+s,g+s` binaries inside the mounted project folder. On macOS, `virtiofs` drops `setuid` and `setgid` bits on the host, so they have no effect. On Linux hosts, guest `setuid` and `setgid` bits persist on the host filesystem: a `setuid` binary created by the VM is owned by the host user and runs as that user if executed by anyone else on the host, while a root-owned `setuid`/`setgid` binary runs with host root privileges. ### Audit behavior and execution timing `hostsafety.SetIDFiles(dir)` (`internal/hostsafety/hostsafety.go`) walks the project directory on the host and returns up to `setIDMax = 5` regular files with `fs.ModeSetuid` or `fs.ModeSetgid` set: - **Single-filesystem and `.git` skip:** `SetIDFiles` skips `.git` directories (which hold most files in large repositories and are not executed by path) and skips subdirectories on different filesystems (`!platform.SameDevice(root, fi)`). - **Root vs. user classification:** `rootSetID(m, uid, gid)` marks `SetIDFile.Root = true` when `(m&fs.ModeSetuid != 0 && uid == 0) || (m&fs.ModeSetgid != 0 && gid == 0)`. - **When it runs:** Because walking 200,000 files takes roughly 1 second, `starter.checkSetID` (`internal/vm/start.go`) and `devvm stop` run `SetIDFiles` only on Linux hosts (`s.OS == "linux"`), only when a VM is newly created (`made == true`) or when `devvm` finds or places the VM in the `stopped` state—never when attaching to an already-running VM, and never for `--clone` VMs. - **Warnings emitted:** Root set-id files trigger `msg.SetidRootFiles` (advising `sudo rm`), and non-root set-id files trigger `msg.SetidFiles` (advising `chmod u-s,g-s FILE`). `hostsafety.SSHRemote(ctx, dir)` also checks `remote.origin.url` in `.git/config` via `hostgit.Output` when creating a new VM; if the URL starts with `git@` or `ssh://` and SSH agent forwarding is disabled, `devvm` emits `msg.SSHRemoteHint`. --- ## Secret placeholder injection and settings redaction No project VM—even on the `open` network tier—ever receives a real secret token or agent refresh token. Real credentials remain on the host in the macOS Keychain (under service `devvm`, or `DEVVM_SECRET_SERVICE` when set) or in `0600` files inside `~/.local/share/devvm/secrets/` (`0700` directory) and `~/.local/share/devvm/logins//` (excluded from Time Machine backups on macOS via `ExcludeFromBackup`). ### Known secrets and placeholder format (`internal/secrets/secrets.go`) Each VM has a random per-VM salt (`netpolicy.Salt(ctx, st)`), distinct from its proxy password. `secrets.Known` defines three managed secrets: | Secret name | `HostOnly` | Guest env var | Placeholder format (`s.Placeholder(salt)`) | Target HTTPS `:443` hosts | Header injected by proxy | | --- | --- | --- | --- | --- | --- | | `claude` | `false` | `CLAUDE_CODE_OAUTH_TOKEN` | `sk-ant-oat01-devvm-proxy-managed-` | `api.anthropic.com` | `Authorization` | | `github` | `false` | `GH_TOKEN` (also sets `DEVVM_GITHUB=1`) | `devvm-proxy-managed-github-` | `api.github.com`, `github.com`, `codeload.github.com`, `raw.githubusercontent.com` | `Authorization` | | `proxy` (`proxy.UpstreamSecret`) | `true` | Never exposed to VMs | None (never bound to a VM) | Sent only to the upstream corporate proxy configured in `config.toml` | `Proxy-Authorization` | `secrets.Token(name, given)` strips whitespace (or trims leading/trailing whitespace for `proxy`), rejects empty values or values containing `"`, `\`, or ASCII control characters (`< 0x20` or `0x7f`), and requires `claude` setup tokens to start with `sk-ant-oat01-`. ### Agent login files and OAuth capture `secrets.LoginRel(kind)` maps agent login files stored under `~/.local/share/devvm/logins//`: | Login kind | Relative path (`LoginRel`) | How the VM receives it (`syncer.stageSecrets`) | | --- | --- | --- | | `codex` | `.codex/auth.json` | Copied to `~/.codex/auth.json` (`0600`) with API keys and OAuth tokens replaced by salted placeholders (including unsigned JWT placeholders carrying account claims and a far-off `exp` so Codex does not attempt to refresh them inside the VM). | | `opencode` | `.local/share/opencode/auth.json` | Copied to `~/.local/share/opencode/auth.json` (`0600`) plus `opencode-import.json` for opencode 2 database import, with supported provider credentials (`anthropic`, `openai`, `openrouter`) replaced by salted placeholders. | | `claude` | `.claude.json` | Staged as `claude.json` (`0600`) and merged into `~/.claude.json` by `guest/sync.sh`, with MCP server secrets replaced by salted placeholders. | | `claude-oauth` | `.claude/.credentials.json` | Copied to `~/.claude/.credentials.json` (`0600`) when Claude Code uses an OAuth login instead of a `claude setup-token` secret. | When an agent logs in inside a VM (`devvm agent login`), the VM's host proxy intercepts the token endpoint response (`Store.CaptureRequest` and `Store.Capture`), writes the real credentials to `~/.local/share/devvm/logins//` on the host, and returns a response containing only salted placeholders to the VM. When the proxy needs a credential for an outbound request (`Store.Get`), `agentlogin.File.Get` reads the host login file and refreshes expired OAuth tokens on the host so the VM never holds a refresh token. ### Redacting and restoring agent settings (`internal/agentlogin/redact.go`) On every launch, `syncer.stage` (`internal/vm/sync.go`) stages `.claude/settings.json`, `.codex/config.toml`, and `.config/opencode/*.{json,jsonc,toml}` in memory and runs `agentlogin.Redact(format, data)`: 1. **Whole-file parsing:** Files are decoded completely as JSON/JSONC or TOML. Any file that fails to parse is dropped (`config_not_copied`) so unparsed lines never reach the VM. In `.config/opencode`, non-settings files other than `.md` (such as `.env` files) are also withheld (`withheld(rel)`). 2. **Key and table matching:** `Redact` replaces string values with `"devvm-redacted"` when: - The key matches `(?i)(key|token|secret|passw|^authorization$|^proxy-authorization$|^cookie$)` (excluding `keybind`/`keymap` tables). - The value sits inside an `env`, `environment`, `headers`, `http_headers`, or `shell_environment_policy.set` map (`wholeTable`), unless it is a boolean/port literal (`^(?i:true|false|[0-9]{1,5})$`), a non-credential URL, or a known non-credential Claude Code setting (`claudeEnv`, such as `ANTHROPIC_MODEL`, `CLAUDE_CODE_*`, `DISABLE_*`, `MAX_*`, `*_TIMEOUT_MS` not matching `credName`). - The string matches a known secret prefix/shape (`secretShape`), or contains credentials in URLs (`scrubURL`), command arguments (`scrubArgs`), or free text (`scrubText`). 3. **Preserved indirection (`redactKeep`):** Empty strings, uppercase environment variable names (`^[A-Z][A-Z0-9_]*$`), variable/file references (`{env:X}`, `{file:X}`, `$X`, `${X}`), and file paths (`/`, `~/`, `./`) are kept unchanged. 4. **Restoring on `devvm agent save`:** When `devvm agent save` reads a settings file or `.claude.json` subset back from a VM, `agentlogin.Restore(format, vmData, givenToVM, oldStoreData)` parses the file and replaces `"devvm-redacted"` markers and the VM's salted placeholders with the original secret values from `oldStoreData`. If a file contains a placeholder that cannot be matched back to the store, `devvm agent save` refuses to save that file (`agent_save_placeholder`). ```sh title="Manage host-stored secrets" # List known secrets (claude, github, proxy), whether saved, and bound VMs devvm secret devvm secret --json # Save a GitHub token from stdin and bind it to the current folder's VM gh auth token | devvm secret set github --stdin devvm secret use github # Unbind or delete the token devvm secret unuse github devvm secret rm github ``` --- ## Isolated repository clones (`internal/vm/clone.go`) Passing `--clone` when creating a VM (`devvm --clone`) or switching an existing VM (`devvm clone on`) eliminates the read-write project mount altogether (`projectMount(st)` returns `""`, and `gitprotect.Mode` is set to `"clone"`). Instead, `devvm` creates an independent Git repository inside the VM at the exact same filesystem path (`t.f.Dir`) and transfers commits exclusively through hardened host-initiated `git push` and `git fetch` calls over pinned SSH. ```mermaid sequenceDiagram autonumber participant Host as Host Repo (internal/vm/clone.go) participant HostGit as Hardened Git + gitSSHCommand participant Guest as VM Isolated Clone (same path) Note over Host: checkCloneRepo & gitprotect.OwnRepo
Verify HEAD, branch, no alternates/symlinks Host->>Guest: exec: sudo mkdir -p "$dir" && chown && git init -q
receive.denyCurrentBranch=updateInstead Host->>HostGit: git push --no-verify --no-recurse-submodules --no-signed HostGit->>Guest: ssh (pinned HostKeyAlias, BatchMode, no agent/forwards)
exec git-receive-pack '$dir' Host->>Host: git remote add devvm :
Warn if host worktree has uncommitted changes Note over Guest: Agent or user edits files and commits
Host working tree and .git are untouched Host->>HostGit: devvm rm / devvm clone off (cloneCheck) HostGit->>Guest: Check unfetched refs/heads/* & worktree status (up to 100) HostGit->>Guest: git fetch --no-tags --no-recurse-submodules --refmap=
into refs/heads/saved/devvm/ ``` ### Repository ownership verification (`gitprotect.OwnRepo`) Before creating or seeding a `--clone` VM (`checkCloneRepo` and `cloneSource`), `devvm` verifies that the host directory is the top level of a Git repository (`git rev-parse --show-toplevel == dir`) with at least one commit and an attached branch (`symbolic-ref -q HEAD`), and calls `gitprotect.OwnRepo(dir)` (`internal/gitprotect/own.go`) to prevent pushing objects borrowed from another repository on the host: 1. **`.git` directory or bidirectional linked worktree:** - If `.git` is a directory, `commonOf(.git)` must resolve to `.git` itself (no `.git/commondir` pointing elsewhere). - If `.git` is a regular file (`linkedWorktree`), its `gitdir:` must resolve to `/.git/worktrees/`, that worktree directory's `gitdir` file must resolve back to the project folder's `.git` file, and both `/commondir` and `/.git` must resolve to `/.git` with no further `commondir`. 2. **No alternates or symlinked objects/refs (`ownObjects`):** Neither `gitDir` nor `commonDir` may contain `objects/info/alternates`, nor may `objects`, `packed-refs`, `reftable`, `refs`, or any entry inside `objects/**` or `refs/**` be a symlink. 3. **Agreement with `git rev-parse`:** `cloneSource` verifies that `git rev-parse --path-format=absolute --absolute-git-dir --git-common-dir` resolves to the exact same `gitDir` and `commonDir` returned by `OwnRepo`. ### Seeding, pinned SSH transport, and remote setup `clones.seedClone` (`internal/vm/clone.go`) initializes the repository in the running VM and pushes the host's branches: - **What gets pushed (`seedRefs`):** For a standard repository, `seedRefs` pushes `--all` (all local branches under `refs/heads/*`). For a linked worktree, `seedRefs` pushes only the checked-out branch (`refs/heads/:refs/heads/`) plus its single upstream ref (`%(upstream)`) if valid and not a wildcard pattern. Tags, untracked files (such as `.env`), and uncommitted changes are never copied; if the host worktree is dirty, `seedClone` warns with `uncommitted_not_copied`. - **Tamper-proof `GIT_SSH_COMMAND` (`gitSSHCommand`):** Both `seedClone` pushes and `cloneCheck` fetches run `hostgit.Command` with a custom shell function in `GIT_SSH_COMMAND`: ```go title="internal/vm/clone.go" func gitSSHCommand(s vmssh.SSH, dir string) string { run := func(verb string) string { return "exec " + shell.QuoteJoin(append([]string{"ssh"}, s.CommandArgs(verb+" "+shell.Quote(dir))...)) } return `f() { for c; do :; done; case $c in "git-receive-pack "*) ` + run("git-receive-pack") + ` ;; "git-upload-pack "*) ` + run("git-upload-pack") + ` ;; esac; exit 1; }; f` } ``` This function ignores every hostname, port, path, and option that `git` passes to `ssh` (defeating any `url.*.insteadOf`, `pushInsteadOf`, or modified remote URL in `.git/config`). It connects strictly to the VM's loopback port using `vmssh.SSH.CommandArgs` (`-F none`, pinned `UserKnownHostsFile` and `HostKeyAlias`, `StrictHostKeyChecking=yes`, `BatchMode=yes`, `IdentitiesOnly=yes`, `IdentityAgent=none`, `ForwardAgent=no`, `ClearAllForwardings=yes`) and executes only `git-receive-pack ` or `git-upload-pack `. - **Host remote registration (`freeRemote`):** Adds the first available remote name among `devvm`, `devvm-`, or `` pointing to `:`. ### Pre-deletion and remake safety checks (`cloneCheck`) Before `devvm rm` deletes a `--clone` VM or `devvm clone off` remakes it to mount the host folder, `cloneCheck.run` starts the VM (if stopped) and inspects the guest clone (reading at most `cloneOutMax = 1 MiB` per command): 1. **Unfetched commits (`unfetchedRefs` & `fetchUnfetched`):** Reads `git for-each-ref --format=%(objectname) %(refname) refs/heads` from the VM. For each 40–64 hex SHA not already reachable from any ref in the host repository (`git rev-list -n1 --not --all`), `devvm` validates the branch name with `git check-ref-format` and offers to fetch the branches into `refs/heads/saved//` (or `saved/-2` through `-99` via `savedPrefix`, never overwriting existing local branches) using `git fetch --no-tags --no-recurse-submodules --refmap=`. 2. **Uncommitted changes across worktrees (`changes`):** Runs `git worktree list --porcelain -z` in the VM, filters to clean absolute paths inside the clone directory (`insideClone`), and checks `git status --porcelain` in the main clone and up to `cloneWorktreesMax = 100` linked worktrees. Displays up to `cloneShown = 20` modified/untracked lines (sanitized through `msg.OneLine`) and prompts for confirmation before deleting or remaking. If the VM has more than 100 linked worktrees, it always counts as dirty and warns with `clone_worktrees_unchecked`. 3. **Preserving fetched remote-tracking branches (`removeRemote` & `saveFetched`):** When removing the host's `devvm` remote, `saveFetched` checks `refs/remotes//*` for any commits not reachable from other refs (`--not --exclude=refs/remotes//* --all`) and atomically creates local `saved//` refs via `git update-ref --stdin` before running `git remote remove`. Both `fetchUnfetched` and `changes` require an interactive terminal confirmation and are never auto-answered by `--force` (if run with `--no-input` or without a TTY, `devvm` refuses and keeps the VM). Passing `--no-clone-check` to `devvm rm` or `devvm clone off` bypasses the guest inspection checks with a warning while still preserving already-fetched `refs/remotes//*` commits into `saved//*`. ## Related pages Create VMs with independent Git clones using devvm --clone, switch existing VMs between mounted folders and clones, and fetch VM branches to the host. Traffic confinement across allowlist, open, and offline tiers, host-side TLS SNI and HTTP inspection via devvm _proxy, and per-VM CA rotation. Run Claude Code, Codex, and opencode inside VMs, import and save agent settings, complete OAuth or API-key logins, and bind host secrets to allowed hosts. Syntax, subcommands, and flags for devvm network, secret, agent, git-protect, clone, ssh-agent, link, and group. --- ## 08. Backend contract and Lima driver > The four-role backend.Backend interface (Host, Images, Machines, Housekeeping), capability requirement checks (Caps), and the Lima driver implementation. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/08-backend-contract-and-lima-driver.md - Generated: 2026-11-07T22:06:18.068Z ### Source Files - `internal/backend/backend.go` - `internal/backend/caps.go` - `internal/backend/guard.go` - `internal/backend/lima/lima.go` - `internal/backend/lima/start.go` - `internal/backend/lima/seatbelt.go` --- title: "Backend contract and Lima driver" description: "The four-role backend.Backend interface (Host, Images, Machines, Housekeeping), capability requirement checks (Caps), and the Lima driver implementation." --- `internal/backend` defines the hypervisor abstraction that separates VM lifecycle orchestration, image recipes, and network policy from underlying virtualization tools. Every backend implements the `backend.Backend` interface—composed of four required role interfaces (`Host`, `Images`, `Machines`, and `Housekeeping`) plus an explicit capability map (`Caps`)—and is wrapped at registration time by a `guarded` decorator that enforces feature requirements and specification invariants before any driver method executes. The default `lima` driver in `internal/backend/lima` implements this contract over `limactl`, using Apple Virtualization.framework (`vz`) with Seatbelt (`sandbox-exec`) confinement on macOS and QEMU/KVM with `restrict=on` user-mode network confinement (`devvm _qemu`) on Linux. ## Four-role backend contract `backend.Backend` (`internal/backend/backend.go`) combines identification and capability reporting (`Name() string`, `Caps() Caps`) with four role interfaces. Backends that store per-VM host state outside the VM instance itself can also implement the optional `Forgetter` interface (`internal/backend/leftovers.go`), which callers discover through `backend.As[backend.Forgetter](b)` so the registry's `guarded` wrapper does not hide optional interfaces. ```mermaid classDiagram class Backend { <> +Name() string +Caps() Caps } class Host { <> +Problems(ctx) []Problem +NameMax() int +Location() (where string, isDefault bool) +DiskDirs() []string +Floors() Floors } class Images { <> +BuildImage(ctx, ImageSpec) (Built, error) +Images(ctx) ([]string, error) +ImageSize(ctx, key string) (int64, error) +DeleteImage(ctx, key string) error +BuildTools(ctx, ToolsSpec) error +ToolFormat() string +Origin(f *From) ([]Step, error) +PinOrigin(ctx, f *From, how Pin) (string, error) } class Machines { <> +CreateVM(ctx, image string, spec VMSpec) error +Start(ctx, vm string, w Want, c Confinement) error +Running(ctx, vm string) (Run, error) +ProxyURL(c Confinement) string +Stop(ctx, vm string) error +Delete(ctx, vm string) error +Rename(ctx, vm, to string) error +List(ctx) ([]Instance, error) +Inspect(ctx, vm string) (Instance, error) +Exec(ctx, vm string, argv []string, o ExecOpts) error +Shell(ctx, vm string, o ShellOpts) (Command, error) +SSH(ctx, vm string) (SSHTarget, error) } class Housekeeping { <> +Usage(ctx) (Usage, error) +Strays(ctx) ([]Stray, error) +DevvmCache(ctx) ([]CacheEntry, error) +DeleteCache(ctx, entries []CacheEntry) error } class Forgetter { <> +Forget(ctx, vm string) error } class guarded { -Backend -r *Registry +BuildImage(ctx, ImageSpec) (Built, error) +BuildTools(ctx, ToolsSpec) error +CreateVM(ctx, string, VMSpec) error +Start(ctx, string, Want, Confinement) error } class Lima { +host +images +machines +housekeeping +Dirs paths.Dirs +Runner Runner +OS string +Arch string } Backend --|> Host Backend --|> Images Backend --|> Machines Backend --|> Housekeeping guarded ..|> Backend : wraps & validates Lima ..|> Backend : implements via embedded roles ``` ### Role breakdown | Role | Responsibility | Key Methods and Return Types | | --- | --- | --- | | `Host` | Reports missing host dependencies (`devvm doctor`), maximum VM name length, storage root location, and minimum free disk thresholds (`backend.Space`). | `Problems(ctx) []Problem`, `NameMax() int`, `Location() (string, bool)`, `DiskDirs() []string`, `Floors() Floors` | | `Images` | Pins upstream base origins, emits backend-specific recipe steps for image key hashing, builds base/layer/shared-tool images, and manages stored images. | `Origin(*From) ([]image.Step, error)`, `PinOrigin(ctx, *From, Pin) (string, error)`, `BuildImage(ctx, ImageSpec) (Built, error)`, `BuildTools(ctx, ToolsSpec) error`, `ToolFormat() string`, `Images(ctx) ([]string, error)`, `ImageSize(ctx, string) (int64, error)`, `DeleteImage(ctx, string) error` | | `Machines` | Clones VMs from images, applies `Want` and `Confinement` deltas on stopped VMs at `Start`, inspects live confinement lifelines, and executes guest commands or shells. | `CreateVM(ctx, string, VMSpec) error`, `Start(ctx, string, Want, Confinement) error`, `Running(ctx, string) (Run, error)`, `ProxyURL(Confinement) string`, `Stop`, `Delete`, `Rename`, `List`, `Inspect`, `Exec`, `Shell`, `SSH` | | `Housekeeping` | Calculates VM and cache disk footprints for `devvm space`, finds unlabeled/orphaned `Stray` VMs, and prunes devvm-owned entries in backend caches for `devvm gc --caches`. | `Usage(ctx) (Usage, error)`, `Strays(ctx) ([]Stray, error)`, `DevvmCache(ctx) ([]CacheEntry, error)`, `DeleteCache(ctx, []CacheEntry) error` | | `Forgetter` *(optional)* | Removes auxiliary per-VM backend files after a VM is already gone (`devvm gc --orphans`). Discovered with `backend.As[backend.Forgetter](b)`. | `Forget(ctx, vm string) error` (not implemented by `lima`, which stores all VM state inside the Lima instance folder) | ### Core specification and state types Callers pass declarative specifications to `Images` and `Machines` methods rather than mutating backend configuration files directly: Mutable VM properties applied at `CreateVM` and re-evaluated on every `Start` of a stopped VM: - `Size` (`CPUs int`, `MemoryGiB float64`, `DiskGiB float64`): zero fields leave existing values unchanged. `backend.Validate(w, in)` rejects any `DiskGiB` smaller than `in.Size.DiskGiB` with `disk_shrink`. - `Project` (`string`) and `ProjectID` (`string`): host path of the read-write project mount and its recorded device/inode identity (`platform.FolderID`). Empty for `--clone` VMs. - `AgentDirs` (`[]string`): host paths mounted read-only at identical paths in the guest. - `AgentForwarding` (`bool`): whether SSH agent forwarding is enabled in the guest configuration. Embeds `Want` and adds creation-time parameters for `CreateVM`: `Name`, `CloneAt` (guest repo path for `--clone` VMs), `Tools` and `ToolsAt` (read-only shared tools mount at `/mnt/devvm-tools`), `AgentToggle` (`bool`), `SSHPort` (`int`), and base guest `Env` (`map[string]string`). Runtime network and credential confinement applied at `Start`: - `Net` (`backend.Net`): one of `NetDirect` (0, open tier using the backend's user-mode network), `NetProxy` (1, confined tiers reaching only `devvm _proxy` on host loopback), `NetInternet` (2, image/tool builds reaching the internet but blocked from host loopback where supported), or `NetNone` (3, offline builds/VMs). - `ProxyPort` (`int`) and `ProxyPassword` (`string`): host loopback port (`127.0.0.1`) and 64-hex-character random password (`backend.NewProxyPassword`) for `devvm _proxy`. `Confinement.GuestProxy()` returns `true` when `Net == NetProxy` or when `Net == NetDirect && ProxyPort != 0`. - `Env` (`map[string]string`): guest environment variables written on boot (proxy CA variables, `NO_PROXY`). - `SSHPort` (`int`), `SSH` (`bool`), `AgentSocket` (`string`), and `NoAgent` (`bool`). Returned by `Machines.Running(ctx, vm)`: - `ID` (`string`): unique identifier for the current boot session (`" "` on Lima, or `""` when stopped). - `Confined` (`bool`): `true` only if the VM is currently running under the exact host process started by `Start` with `NetProxy`. - `Lifeline` (`string`): path to the host hypervisor/hostagent PID file that `devvm _proxy` and `vmssh.Forwards` watch to terminate automatically when the VM exits. --- ## Capability matrix and requirement checks Every feature that `devvm` can ask of a backend is represented by a `backend.Feature` constant (`internal/backend/caps.go`). A backend declares its support level for all 19 features in `Caps() map[Feature]Support`: - `backend.Native` (`rank = 3`): supported directly by the backend. - `backend.Emulated(true)` (`rank = 2`, string `"emulated"`): built by `devvm` on top of other backend primitives and approved after a design note in `docs/content/docs/internals/design.md`, a threat review, and end-to-end tests. - `backend.Emulated(false)` (`rank = 1`, string `"unreviewed"`): implemented as an emulation but refused at runtime unless explicitly whitelisted for testing via `DEVVM_UNREVIEWED=:`. - `backend.None` (`rank = 0`, string `"none"`): unsupported. ### Feature dependencies and Lima support Three composite features depend on a base feature via `Feature.BuiltOn()`. `Caps.Check()` rejects any backend registration where a dependent feature declares a higher support rank than its base feature, or where `len(c) != len(Features)`. | Feature | `BuiltOn` Base | Required By | Lima Support | Lima Implementation Details | | --- | --- | --- | --- | --- | | `build-image` | — | `ImageSpec.Features()`, `ToolsSpec.Features()` | `Native` | Builds `devvm-img-` instances from `base.yaml` + recipe scripts | | `build-confinement` | — | `NetInternet`, `NetNone` | `Native` | macOS Seatbelt denies `localhost:*` or all outbound; Linux QEMU `restrict=on` confines `NetNone` | | `from` | — | `ImageSpec` with `From != nil` | `Native` | Converts OCI/Docker image into raw disk + kernel/initrd in a conversion VM | | `clone` | — | `VMSpec.Features()` | `Native` | `limactl clone`: APFS CoW clone on macOS (`vz`), `qcow2` backing overlay on Linux (`qemu`) | | `project-mount` | — | `VMSpec` with `Project != ""` | `Native` | Read-write mount (`virtiofs` via `/.vol//` on macOS `vz`; `9p` by path on Linux) | | `agent-folders` | — | `Want` with `len(AgentDirs) > 0` | `Native` | Read-only host folder mounts (`writable: false`) | | `tools-share` | — | `VMSpec` with `Tools != ""` | `Native` | Read-only mount at `/mnt/devvm-tools` (`writable: false`) | | `egress-proxy` | — | `NetProxy` | `Native` | macOS Seatbelt profile; Linux QEMU `restrict=on` + `guestfwd` (`192.168.5.4:3128`) | | `ssh` | — | `devvm code`, `devvm ssh-agent`, links | `Native` | Lima's driver SSH forward on `127.0.0.1` + `_config/user` identity | | `ports` | — | `Ports` with `len(Ranges) > 0` or `All` | `Emulated(true)` | `vmssh.Forwards` over `SSH()` with Lima's own `portForwards` set to `ignore: true` | | `ports-all` | `ports` | `Ports` with `All: true` | `Emulated(true)` | `vmssh.Forwards` polling guest listeners `>= 1024` via `assets/guest/ports.sh` | | `agent-forwarding` | — | `Want` with `AgentForwarding: true` | `Native` | Sets `.ssh.forwardAgent=true` and passes `SSH_AUTH_SOCK` to `limactl` | | `agent-toggle` | `agent-forwarding` | `VMSpec` with `AgentToggle: true`, `devvm ssh-agent` | `Native` | Updates `.ssh.forwardAgent` via `limactl edit` before `Start` | | `cpus` | — | `Size` with `CPUs > 0` | `Native` | `--cpus` in `limactl create` / `limactl edit` | | `memory` | — | `Size` / `ImageSpec` with `MemoryGiB > 0` | `Native` | `--memory` in `limactl create` / `limactl edit` | | `disk` | — | `Size` with `DiskGiB > 0` | `Native` | `--disk` in `limactl create` / `limactl edit` (grow-only) | | `docker-layer` | — | `ImageFeatures("...docker...")` | `Native` | Runs Docker Engine inside the guest | | `nix-layer` | — | `ImageFeatures("...nix...")`, `ImageSpec` with `nix` | `Native` | Runs Nix daemon and sandboxed builds inside the guest | | `link` | `ssh` | `devvm link`, `devvm group` | `Native` | Host-mediated SSH tunnel binding `127.77.0.0/16` in client VMs | ### Dual-layer guard enforcement Feature checks happen at two boundaries (`internal/backend/caps.go` and `internal/backend/guard.go`): 1. **Caller pre-check (`Review.Require`):** Before building an image, creating or starting a VM, or mutating settings, `internal/vm` calls `env.Backends.Require(b, vmName, features...)`. If a feature is `None`, `Require` returns `msg.BackendLacks(b.Name(), feature, vm)`. If it is `Emulated(false)` and not listed in `Review.Unreviewed` (parsed from `DEVVM_UNREVIEWED`), `Require` returns `msg.BackendUnreviewed(b.Name(), feature, vm)`. If `DEVVM_UNREVIEWED` lets an unreviewed feature through, `Require` emits a warning via `msg.BackendUnreviewedUsed`. 2. **Registry guard (`guarded`):** `Registry.Register` wraps every registered backend in `*guarded`. Calling `BuildImage`, `BuildTools`, `CreateVM`, or `Start` on a registered backend runs `spec.Validate()` and `r.refusal(g.Backend, name, nil, features)` before delegating to the driver. A caller that omits `Require` still cannot invoke a backend operation for an unsupported or unreviewed feature. ```go title="internal/backend/guard.go" func (g *guarded) BuildImage(ctx context.Context, spec ImageSpec) (Built, error) { if err := spec.Validate(); err != nil { return Built{}, err } if err := g.r.refusal(g.Backend, spec.Image, nil, spec.Features()); err != nil { return Built{}, err } return g.Backend.BuildImage(ctx, spec) } func (g *guarded) Start(ctx context.Context, vm string, w Want, c Confinement) error { if err := g.r.refusal(g.Backend, vm, nil, slices.Concat(w.Features(), c.Features())); err != nil { return err } return g.Backend.Start(ctx, vm, w, c) } ``` The `open` network tier uses `NetDirect` and requires no `egress-proxy` capability (`NetDirect.Features()` returns `nil`). Every backend must still route `NetDirect` guest traffic to `127.0.0.1:` on the host when `Confinement.GuestProxy()` is true so `devvm _proxy` can inject bound secrets and log requests. --- ## Lima driver implementation `lima.Lima` (`internal/backend/lima/lima.go`) embeds four single-field role structs—`host{l}`, `images{l}`, `machines{l}`, and `housekeeping{l}`—wired together by `withRoles`. All external commands execute through `Lima.Runner` (`logRunner` wrapping the UI's `LoggedCmd`). ### Isolated `LIMA_HOME` and `limactl` invocation Lima merges `_config/default.yaml` and `_config/override.yaml` from `LIMA_HOME` into every instance it creates or starts. To prevent user-configured Lima networks or mounts from leaking into `devvm` VMs, `Lima.limactl` (`internal/backend/lima/limactl.go`) enforces strict process isolation on every `limactl` call: - **Dedicated home (`Dirs.Lima`):** Sets `LIMA_HOME` to `~/.local/share/devvm/lima` (or `DEVVM_LIMA_HOME`), ignoring any ambient `LIMA_HOME` in the user's shell. - **Global config check:** `checkGlobalConfigs()` checks for `_config/default.yaml` and `_config/override.yaml` inside `Dirs.Lima` using `os.Lstat` and refuses to start VMs (`lima_global_config`) if either exists. `Start` also inspects `in.Config.Networks` and fails with `vm_has_networks` if any extra network is configured on the instance. - **Working directory (`Dirs.Data`):** Runs `limactl` with `cmd.Dir = l.Dirs.Data` (`~/.local/share/devvm`). When `limactl` rejects an invalid YAML config, it writes `lima.REJECTED.yaml` into its current working directory; running inside `Dirs.Data` prevents a compromised guest from planting a symlink named `lima.REJECTED.yaml` inside a writable project folder. - **Environment scrubbing:** Strips all ambient proxy variables (`HTTP_PROXY`, `HTTPS_PROXY`, `FTP_PROXY`, `NO_PROXY`, `ALL_PROXY` in upper and lower case) and `SSH_AUTH_SOCK` from `limactl`'s environment. Because Lima shares a single OpenSSH `ControlMaster` connection per VM, an ambient `SSH_AUTH_SOCK` inherited by the first `limactl shell` would expose the host's SSH agent to subsequent sessions even when the VM is configured for no agent or `devvm`'s isolated agent. - **Socket path length limit (`NameMax`):** Lima creates `//ssh.sock`, and OpenSSH appends a 17-byte temporary suffix (`sshSocketExtra = 17`) while binding. Because macOS caps `sockaddr_un.sun_path` at 104 bytes (`socketMax = 104`), `NameMax()` returns: $$\text{NameMax} = \max\bigl(0,\; 104 - 17 - \text{len}(\text{Dirs.Lima}) - \text{len}(\text{"/"}) - \text{len}(\text{"/ssh.sock"})\bigr)$$ ### Image origins, builds, and disk overlays In `internal/backend/lima`, an image is a stopped Lima instance named `devvm-img-` (`ImagePrefix = "devvm-img-"`). `imageKey(name)` distinguishes image instances from project VMs for folders named `img-...` by verifying `!vmname.IsVM(name)` (project VMs end with a `-<14-hex-hash>` suffix). Before computing `base`'s image key, the builder calls `PinOrigin(ctx, f, how)` (`internal/backend/lima/pin.go`) under `Dirs.UbuntuPinLock()`. For default Ubuntu builds, `PinOrigin` queries `ubuntu.Finder` for the newest dated Ubuntu 24.04 cloud image (`PinFirst` or `PinNewest`) or the newest release prior to the current pin (`PinOlder` fallback on `ErrBoot`), writing `~/.local/share/devvm/images/ubuntu.json`. `Origin(f)` then returns the initial `image.Step` slice hashed into `base`'s key: 1. `ubuntu-image` with the pinned release and SHA-256 argument. 2. For `--from` builds (`f != nil`), the `oci/*` conversion scripts (`convertSteps`). 3. `disk` with `backend.BaseDiskGiB` (`"100"`). 4. `base.yaml` containing the embedded Lima template from `assets/base.yaml`. `buildSize` fits `BaseCPUs` (`4`) and `BaseMemoryGiB` (`8` GiB) to the host via `backend.FitCPUs` and `backend.FitMemory` (capping memory at `hostGiB / 2` if `8 GiB` would leave the host with less than `2 GiB`). `create` invokes `limactl create --tty=false --mount-none --name=devvm-img-` and pipes `base.yaml` over `stdin` (`-`) so no shared template file is written to disk. Layer builds instead clone their stopped parent instance with `limactl clone --tty=false --mount-none `. `BuildImage` starts `devvm-img-` confined to `spec.Net` (`NetInternet` or `NetNone`). If a corporate proxy is active on a `NetInternet` build (`buildProxy` in `internal/backend/lima/buildproxy.go`), `devvm` spawns a per-build `devvm _proxy` forwarding to the upstream proxy and exports temporary proxy variables on each script's `stdin` (preserving them across `sudo` via `/etc/sudoers.d/devvm-build-proxy` and removing the sudoers file afterward) so corporate proxy credentials are never written to `/etc/environment` inside the image. If any step exits with code `75` (`backend.ExitUnreachable` from `assets/layers/apt.sh`), `runStep` returns `msg.BuildSourcesUnreachable`. After all recipe steps complete and `spec.Trust.Remove` removes temporary build CAs, `BuildImage` runs `sync` inside the guest, stops the instance, deletes its runtime directory (`Dirs.VM(inst).Dir`), and on Linux removes write bits (`mode &^ 0o222`) from `/devvm-img-/disk`. #### OS-specific VM cloning (`CreateVM`) When `CreateVM` clones `devvm-img-` into a project VM: - **macOS (`vz`):** `limactl clone` creates an APFS copy-on-write clone of the raw disk image, sharing underlying blocks automatically. - **Linux (`qemu`):** `limactl clone` copies the entire `qcow2` file. To avoid duplicating multi-gigabyte disks, `Lima.overlay` (`internal/backend/lima/lima.go`) verifies the cloned disk is `qcow2` via `qemu-img info -U --output=json`, deletes the full copy, and replaces it with a thin `qcow2` overlay backed by the read-only image disk (`qemu-img create -q -f qcow2 -b -F qcow2 `), setting mode `0600`. `DeleteImage` checks `overlayUsers` via `qemu-img info -U` and refuses to delete an image whose disk backs any existing VM (`image_in_use`). ### Idempotent configuration diffing on `Start` Any invocation of `limactl edit` causes Lima's cloud-init iso generator to assign a new instance ID on the next boot, which regenerates the guest's SSH host keys. Consequently, `Start` (`internal/backend/lima/start.go`) inspects the stopped VM via `limactl list --json` and computes the minimal set of `--set` flags through `wantEdits` and `startSets`. If `len(opts) == 0`, `Start` skips `limactl edit` altogether. | Lima YAML Path / Flag | Condition | Value Set by `configure` | Purpose | | --- | --- | --- | --- | | `--cpus`, `--memory`, `--disk` | `w.Size` differs from `in.size()` (`DiskGiB` only when larger) | Requested CPU count, GiB memory, or GiB disk | Resizes stopped VM (`Validate` blocks disk shrink first) | | `.mounts` | Project path/inode or `w.AgentDirs` changed | Project mount + read-only `AgentDirs` + existing tools share | On macOS `vz`, mounts project as `location: /.vol//` (`platform.VolPath`) with `mountPoint: ` so host symlinks replacing the project path are never followed; on Linux `9p`, mounts by path | | `.ssh.forwardAgent` | `w.AgentForwarding != in.AgentForwarding` | `true` or `false` | Toggles SSH agent forwarding | | `.ssh.localPort` | `sshPort != 0 && cfg.SSH.LocalPort != sshPort` | Allocated host loopback port | Pins the VM's SSH port across restarts | | `.env` | `guestEnv(c)` differs from `cfg.Env` | JSON map of `c.Env` + `HTTP_PROXY`/`HTTPS_PROXY` (upper & lower) | Written to `/etc/environment` by Lima; uses `devvm-proxy-password` stand-in during `limactl edit` | | `.propagateProxyEnv` | `(c.Net == NetDirect && !c.GuestProxy()) != propagates` | `false` whenever `devvm _proxy` is used or during builds; `true` otherwise | Prevents Lima from injecting host proxy credentials into images or proxy-confined VMs | | `.hostResolver.enabled` | `c.Net == NetProxy` vs. other modes | `false` for `NetProxy`; `del(.hostResolver)` otherwise | Disables Lima's built-in DNS resolver (which binds ephemeral loopback ports denied by Seatbelt) when names resolve in `devvm _proxy` | | `.dns` | `c.Net == NetProxy` vs. other modes | `["127.0.0.1"]` for `NetProxy`; `del(.dns)` otherwise | Makes direct guest DNS lookups fail fast on confined tiers | | `.portForwards` | `!forwardsNone(cfg.PortForwards)` | 3 ignore rules (`127.0.0.1`, `0.0.0.0`, `::` for ports `1..65535`, `proto: "any"`, `ignore: true`) | Disables Lima's native port forwarder (`SO_REUSEADDR`) so `vmssh.Forwards` owns all port forwarding | #### Proxy password stand-in and log redaction Because process arguments to `limactl edit` are visible to other local processes in `ps`, `startSets` substitutes `const passwordStandIn = "devvm-proxy-password"` into the proxy URL passed to `--set .env = ...`. Immediately after `limactl edit` returns, `fillPassword` reads `//lima.yaml`, validates that the real proxy URL contains only safe YAML scalar characters (`a-zA-Z0-9-._~%:/@`), replaces `devvm-proxy-password` with `c.ProxyPassword`, and writes `lima.yaml` atomically with mode `0600` (`state.WriteFile`). When `limactl start` runs, Lima's `hostagent` logs parsed environment variables to `//ha.stderr.log`. `Start` registers `defer l.redactLog(vm)`, which calls `redact.File` to overwrite any proxy password in `ha.stderr.log` with `*` bytes of the exact same length in-place so the running `hostagent`'s file offset remains valid. --- ## Host process confinement: Seatbelt and QEMU wrapper For confined VMs (`NetProxy`, `NetInternet`, `NetNone`), `Lima.confine` (`internal/backend/lima/start.go`) restricts the hypervisor/network process at the OS level so even `root` inside the guest cannot bypass `devvm _proxy` or reach arbitrary host loopback services. ```mermaid flowchart LR subgraph macOS["macOS (darwin) — internal/backend/lima/seatbelt.go"] SB["/usr/bin/sandbox-exec\n-f vms/<hash>/run/sandbox.sb"] --> HA["Lima hostagent\n(vz driver + usernet)"] HA -- "allow localhost:<SSHPort>" --> SSHD1["VM sshd forward"] HA -- "allow localhost:<ProxyPort>\n(192.168.5.2 in guest)" --> Proxy1["devvm _proxy\n(127.0.0.1:<ProxyPort>)"] HA -- "allow subpath LimaDir\n& AgentSocket" --> Socks["ssh.sock / vz sock\n/ SSH_AUTH_SOCK"] HA -. "deny network-outbound\ndeny network-bind" .-> Blocked1["Host LAN, internet,\nDocker socket, other ports"] end subgraph Linux["Linux (linux) — internal/backend/lima/qemu.go"] Wrap["QEMU_SYSTEM_<ARCH> wrapper\ndevvm _qemu exec"] --> QEMU["qemu-system-<arch>\n-netdev user,id=net0,...,restrict=on"] QEMU -- "guestfwd=tcp:192.168.5.4:3128\n-cmd: devvm _qemu connect" --> Proxy2["devvm _proxy\n(127.0.0.1:<ProxyPort>)"] QEMU -. "restrict=on drops\nall other traffic" .-> Blocked2["Host loopback,\nLAN & direct internet"] end ``` On macOS, `confine` renders a Seatbelt profile (`~/.local/share/devvm/vms//run/sandbox.sb`) and wraps `limactl start` with `/usr/bin/sandbox-exec -f `. Because the Lima `hostagent` owns the `vz` user-mode network stack, confining `limactl start` confines all guest network traffic: ```scheme title="internal/backend/lima/seatbelt.go (profileTmpl)" (version 1) (allow default) {{- if .Internet}} (deny network-outbound (remote ip "localhost:*")) {{- else}} (deny network-outbound) (allow network-outbound (remote unix-socket (subpath {{sb .LimaDir}}))) {{- with .AgentSocket}} (allow network-outbound (remote unix-socket (path-literal {{sb .}}))) {{- end}} (deny mach-lookup (global-name "com.apple.dnssd.service")) {{- end}} (allow network-outbound (remote ip "localhost:{{.SSHPort}}")) {{- with .ProxyPort}} (allow network-outbound (remote ip "localhost:{{.}}")) {{- end}} {{- if .NoBinds}} (deny network-bind) (allow network-bind (local tcp "localhost:{{.SSHPort}}")) (allow network-bind (local unix-socket (subpath {{sb .LimaDir}}))) {{- end}} ``` - **`NetProxy`:** Sets `ProxyPort = c.ProxyPort` and `NoBinds = true`. Denies all outbound IP traffic except `localhost:` and `localhost:`, denies `com.apple.dnssd.service` Mach lookups, restricts Unix domain socket connections to `LimaDir` (resolved with `paths.PhysicalParent`) and `AgentSocket` (only when `forwarding` is enabled), and denies all TCP/Unix binds except `localhost:` and sockets inside `LimaDir`. Guest processes reach `devvm _proxy` at `http://devvm:@192.168.5.2:`. - **`NetInternet` (builds):** Sets `Internet = true` (and `ProxyPort` when behind a corporate build proxy). Allows outbound internet connections while denying `localhost:*` except `SSHPort` and `ProxyPort`, preventing build scripts from probing host loopback services. - **`NetNone`:** Leaves `Internet = false` and `ProxyPort = 0`, blocking all outbound IP traffic except `localhost:`. On Linux, `confine` writes an executable wrapper script at `~/.local/share/devvm/run/qemu/qemu-system-` (mode `0700`) and sets `QEMU_SYSTEM_AARCH64` or `QEMU_SYSTEM_X86_64` in `limactl start`'s environment: ```sh title="~/.local/share/devvm/run/qemu/qemu-system-" #!/bin/sh # Written by devvm: QEMU with devvm's network restrictions. exec '/path/to/devvm' _qemu exec "$0" "$@" ``` When Lima spawns QEMU, `devvm _qemu exec` runs `lima.RunQEMU`, which rewrites the argument list with `RestrictNetdev(args, os.Getenv("DEVVM_GUESTFWD"))` before `exec`-ing the real `qemu-system-` binary on `PATH` (skipping relative `PATH` entries and the wrapper's own directory via `paths.Physical`): - **Single netdev enforcement:** `RestrictNetdev` scans for `-netdev` (or `--netdev`). It appends `,restrict=on` to the single `user,id=net0,...` argument. If `DEVVM_GUESTFWD` is set (`NetProxy`), it also appends `,guestfwd=tcp:192.168.5.4:3128-cmd:` (escaping commas as `,,`), where `` is `'/path/to/devvm' _qemu connect 127.0.0.1:`.Guest processes reach the proxy at `http://devvm:@192.168.5.4:3128`. - **Fail-closed argument validation:** `RestrictNetdev` leaves non-network probe calls (such as `-accel help` or `-netdev help` lacking `id=`) untouched, but returns `msg.QEMURefused()` if there is more than one `-netdev`, if `-netdev` does not start with `user,id=net0,` or already contains `restrict=`, if `-netdev`'s value starts with `-`, if `-nic` or `-net` is anything other than `none`, or if `-readconfig` is passed. - **`NetInternet` on Linux:** Because QEMU SLIRP `restrict=on` blocks all external internet access alongside host loopback, `NetInternet` builds on Linux run without the QEMU wrapper (`NetProxy` and `NetNone` use the wrapper). ### Lifeline tracking, degraded boots, and broken-state recovery - **Confinement verification (`recordStart` and `Running`):** Once `limactl start` succeeds, `recordStart` reads the hostagent PID from `//ha.pid` and writes `" \n"` to `~/.local/share/devvm/vms//run/started`. `Running()` compares the live PID inside `ha.pid` (and checks `daemon.Alive(pid)`) against `started`: `Run.Confined` is `true` only if the PID matches and was started with `NetProxy`. If a user starts the VM manually via `limactl start`, `ha.pid` receives a new PID, `Run.Confined` evaluates to `false`, and `devvm ls` reports the VM as `unenforced`. - **Degraded boot check (`degraded`):** `limactl start` exits non-zero when a guest boots and answers SSH but a cloud-init or guest readiness script reports a warning. When `limactl start` fails, `degraded` inspects the VM: if Lima reports `Status == "Running"`, `devvm` runs `limactl shell --workdir / sh -c "cd '/' && exec 'true'"` with a `60s` timeout (`sshTimeout`). If SSH succeeds, `Start` emits `msg.StartedWithWarnings(vm)` and continues; if SSH times out, `Start` stops the VM with `limactl stop --force` and returns `msg.StartNoSSH`. - **Settling interrupted QEMU starts (`settled`):** On slow or nested-virtualization Linux hosts, an interrupted start can leave `hostagent` running while QEMU has not yet written its PID file (`Status == "Broken"`, `HostAgentPID != 0`, `DriverPID == 0`). Killing `hostagent` with `--force` in that window orphans the starting QEMU process, holding the disk lock. `Stop` calls `settled`, polling every `100ms` up to `30s` (`driverWait`) until `DriverPID != 0` before running `limactl stop --force`. Conversely, if `Start` finds `Status == "Broken"` with `DriverPID != 0 && HostAgentPID == 0`, it runs `limactl stop --force` first to release the orphaned QEMU lock. --- ## Host diagnostics and cache housekeeping ### Host problem detection (`Problems`) `lima.Problems(ctx)` (`internal/backend/lima/check.go`) implements `Host.Problems` for `devvm doctor` and pre-launch checks (`backend.Check`): | Problem Key | Condition Checked | Auto-Fix Command (`Problem.Fix`) | | --- | --- | --- | | `os` | `l.OS != "darwin" && l.OS != "linux"` | None (`lima_host_unsupported`) | | `arch` | `l.Arch != "arm64" && l.Arch != "amd64"` | None (`lima_arch_unsupported`) | | `macos` | macOS major version `< 13` (`minMacOS = "13"`) | None (`program_old`) | | `lima` | `limactl` missing, fails `--version`, or `< 2.2` (`minLima = "2.2"`) | `brew install lima`, `brew reinstall lima`, or `brew upgrade lima` (when `brew` is on `PATH`) | | `qemu` | On Linux, `qemu-system-` or `qemu-img` missing from `PATH` | First detected package manager (`apt-get`, `dnf`, `pacman`, `zypper`) command | | `kvm` | On Linux, `os.Stat("/dev/kvm")` fails | None (`no_kvm`) | | `kvm-access` | On Linux, `/dev/kvm` exists but `os.OpenFile(l.KVM, os.O_RDWR, 0)` fails | `sudo usermod -aG kvm ` | | `lima-config` | `/_config/default.yaml` or `override.yaml` exists | None (`lima_global_config`) | ### Disk space floors and cache pruning `host.Floors()` (`internal/backend/lima/lima.go`) defines the minimum free space required on `Dirs.Lima` before disk-writing operations proceed (`backend.Space.Check` rounds each threshold up to whole GiB): | Operation | macOS (`vz` / APFS) | Linux (`qemu` / `qcow2` overlays) | | --- | --- | --- | | `Floors.Base` (building `base`) | `8 GiB` | `8 GiB` | | `Floors.Image` (building a layer image) | `2 GiB` | `8 GiB` (`+6 GiB` for full parent `qcow2` copy during build) | | `Floors.VM` (creating and starting a VM) | `2 GiB` | `2 GiB` | | `Floors.ForTools(n)` (building `n` shared tools) | `3 GiB + n * 512 MiB` | `9 GiB + n * 512 MiB` (`+6 GiB` for throwaway `devvm-tools-build` VM disk) | For `devvm gc --caches`, `housekeeping.DevvmCache` (`internal/backend/lima/caches.go`) inspects Lima's shared download cache at `/lima/download/by-url-sha256/`. Because that cache is shared with the user's own Lima instances, `devvmEntry` only selects entries that meet all four conditions: 1. The entry is a directory whose `ModTime` is older than 1 hour (`cacheRecent = time.Hour`), avoiding active downloads. 2. `/url` hashes (SHA-256 hex) to the directory's basename. 3. The URL matches `Lima`'s Ubuntu cloud image regex (`^/release-[0-9]{8}/ubuntu-24\.04-server-cloudimg-(arm64|amd64)\.img$`). 4. The URL is not referenced by any existing image record in `state.Images(dirs)` (`recordedURLs`). `DeleteCache` re-verifies all four conditions on each entry before calling `os.RemoveAll`. ## Related pages How `internal/vm` maps project folders to VM names, persists `vm.json`, and orchestrates `Start` and remakes. How `Origin` steps, layer scripts, and shared tool formats combine into deterministic SHA-256 image keys. How `devvm _proxy` enforces allowlist, internet, open, and offline policies for confined VMs. How `vmssh.Forwards` and VM links operate over `Machines.SSH` and follow the backend `Lifeline`. Troubleshooting `devvm doctor` checks, `PinOlder` cloud image fallbacks, and SSH host-key pinning. --- ## 09. Configure coding agents and credentials > Run Claude Code, Codex, and opencode inside VMs, import and save agent settings, complete OAuth or API-key logins, and bind host secrets to allowed hosts. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/09-configure-coding-agents-and-credentials.md - Generated: 2026-11-07T22:10:04.744Z ### Source Files - `internal/cli/agent.go` - `internal/cli/agentlogin.go` - `internal/vm/agents.go` - `internal/vm/agentimport.go` - `internal/vm/agentlogin.go` - `internal/vm/agentsave.go` - `internal/agentlogin/restore.go` - `internal/proxy/secrets.go` --- title: "Configure coding agents and credentials" description: "Run Claude Code, Codex, and opencode inside VMs, import and save agent settings, complete OAuth or API-key logins, and bind host secrets to allowed hosts." --- `devvm` isolates coding agents (`claude`, `codex`, and `opencode`) inside per-project virtual machines while keeping all real credentials—OAuth tokens, refresh tokens, API keys, MCP server headers, and GitHub tokens—exclusively on the host. Each VM receives read-only mounts of non-secret agent directories, redacted copies of agent configuration files, and per-VM salted placeholders in place of tokens. When a guest process makes an HTTPS request to a bound host on port `443`, the host-side proxy (`devvm _proxy`) terminates TLS using a per-VM certificate authority, substitutes the placeholder with the live credential from the host store or macOS Keychain, and forwards the request upstream. ## Supported coding agents and guest environment Agents are packaged as read-only EROFS shared tool images mounted into VMs alongside language toolchains. Because the shared tool mounts are immutable, `devvm` disables each agent's built-in self-updater and telemetry during guest synchronization. | Agent ID | Binary | Version track & guest overrides | Built-in policy (`shared-tools/.policy`) | | --- | --- | --- | --- | | `claude` | `claude` | Latest release; sets `DISABLE_AUTOUPDATER=1` and disables telemetry | `api.anthropic.com`, `claude.ai`, `downloads.claude.ai`, `platform.claude.com`, `mcp-proxy.anthropic.com` | | `codex` | `codex` | Latest release; sets `cli_auth_credentials_store = "file"` in `~/.codex/config.toml` and disables analytics/metrics unless explicitly configured | `api.openai.com`, `chatgpt.com`, `auth.openai.com` | | `opencode` | `opencode` | Latest `2.x` release; sets `OPENCODE_DISABLE_AUTOUPDATE=1` | `opencode.ai`, `models.opencode.ai` | `opencode` connects to whichever model provider you configure (`anthropic`, `openai`, `openrouter`, or custom providers). On the `allowlist` network tier, allow your model provider's API host explicitly (for example, `devvm network allow api.anthropic.com`), or bind a provider API key with `devvm agent login opencode --api-key --provider `, which automatically adds that provider's host on port `443` to `secrets.policy`. ### Adding agents to new and existing VMs Configure default agents for new VMs or attach agents to an existing project VM using the CLI: ```bash # Configure default agents selected for all new VMs devvm setup --agents claude,codex # Create a new VM with specific agents enabled devvm --agents claude,opencode # Interactively select or add agents to an existing VM devvm tools devvm tools add claude codex opencode ``` ### In-VM agent guidance On every launch, `syncer.stage` installs `/usr/share/doc/devvm/README.md` (owned by `root`, read-only) and exposes the `devvm-help` command inside the guest. This guide documents the sandbox boundaries, network tiers, and host commands (`devvm network allow`, `devvm agent login`) that an agent should ask the user to run on the host. When a VM runs with a proxy address, guest setup also maintains a `` block inside `~/.claude/CLAUDE.md` and `~/.codex/AGENTS.md` instructing agents to run `devvm-why` whenever a connection fails or returns `403`. ## Host agent store and VM synchronization `devvm` never mounts or reads `~/.claude`, `~/.claude.json`, `~/.codex`, `~/.config/opencode`, or `~/.local/share/opencode` directly from your host home directory when launching a VM. Instead, all VMs read from `devvm`'s isolated host store, ensuring VMs receive identical configurations whether or not the agents are installed on the host machine, and preventing host session transcripts from leaking into guests. ```mermaid flowchart TD subgraph HostHome["Host Home Directory (~)"] HClaude["~/.claude & ~/.claude.json"] HCodex["~/.codex"] HOpencode["~/.config/opencode & ~/.local/share/opencode"] end subgraph HostStore["Host Store & Secret Backend"] SettingsStore["~/.config/devvm/agents/<agent>/
(Settings files & skill/plugin folders)"] LoginsStore["~/.local/share/devvm/logins/<agent>/
(OAuth credentials & API keys, mode 0600)"] SecretStore["macOS Keychain / ~/.local/share/devvm/secrets/
(claude, github, proxy)"] end subgraph HostProxy["Host Proxy Process (devvm _proxy)"] Bindings["vms/<hash>/run/secrets.json
(BindingList: hosts, header, placeholder)"] Capturer["OAuth Token Capturer
(auth.openai.com, platform.claude.com)"] end subgraph GuestVM["Guest VM (~)"] ROLinks["Symlinks in ~ to read-only mounts
(skills, plugins, prompts, commands)"] RedactedCopies["Redacted copies & salted placeholders
(settings.json, config.toml, auth.json, env)"] GuestHistory["Per-VM session history & SQLite DBs"] end HostHome -- "devvm agent import
(excludes history)" --> SettingsStore HostHome -- "devvm agent import
(excludes history)" --> LoginsStore SettingsStore -- "Read-only virtiofs / 9p mount" --> ROLinks SettingsStore -- "agentlogin.Redact on launch" --> RedactedCopies LoginsStore -- "File.VMCopy (salted placeholders)" --> RedactedCopies SecretStore -- "Lookup on demand (IO.Secret)" --> HostProxy LoginsStore -- "Lookup & OAuth refresh (File.Get)" --> HostProxy RedactedCopies -- "devvm agent save
(diff + agentlogin.Restore)" --> SettingsStore GuestVM -- "POST OAuth token exchange" --> Capturer Capturer -- "Save real tokens" --> LoginsStore Capturer -- "Return salted placeholders" --> GuestVM ``` ### Store directory layout `devvm` separates non-secret agent settings from login files and host secrets across XDG config and data directories: :::files ~/.config/devvm/ └── agents/ ├── claude/ │ └── .claude/ │ ├── settings.json │ ├── CLAUDE.md │ ├── skills/ │ ├── agents/ │ ├── commands/ │ ├── output-styles/ │ └── plugins/ ├── codex/ │ └── .codex/ │ ├── config.toml │ ├── AGENTS.md │ ├── prompts/ │ └── skills/ └── opencode/ └── .config/opencode/ ├── opencode.json ├── AGENTS.md └── / ~/.local/share/devvm/ ├── logins/ │ ├── claude/ │ │ ├── .claude.json │ │ └── .claude/.credentials.json │ ├── codex/ │ │ └── .codex/auth.json │ └── opencode/ │ └── .local/share/opencode/auth.json ├── secrets/ │ ├── claude │ ├── github │ └── proxy └── vms// └── run/ ├── secrets.json ├── secrets.policy └── ca/ ├── ca.pem ├── ca.key ├── next.pem └── next.key ::: ### Mounted directories vs. copied files On every VM launch, `Launch.AgentMounts` and `syncer.stage` provision each enabled agent's files using two mechanisms: | Agent | Read-only host mounts (symlinked into guest `~`) | Copied into guest `~` on every launch (secrets redacted/replaced) | | --- | --- | --- | | `claude` | `.claude/skills`, `.claude/agents`, `.claude/commands`, `.claude/output-styles`, `.claude/plugins` | `.claude/settings.json`, `.claude/CLAUDE.md`, and `.claude.json` (merged from `logins/claude/.claude.json`) | | `codex` | `.codex/prompts`, `.codex/skills` | `.codex/config.toml`, `.codex/AGENTS.md`, and `.codex/auth.json` (from `logins/codex/.codex/auth.json`) | | `opencode` | All subdirectories of `.config/opencode/` except `node_modules` | All regular files in `.config/opencode/` (except `*.lock`, `package-lock.json`, `.gitignore`, `.DS_Store`, and withheld non-settings/non-Markdown files), plus `.local/share/opencode/auth.json` | #### Read-only folder mounts - **Host-enforced immutability:** Settings folders that exist in `~/.config/devvm/agents//` are mounted read-only into the guest at their exact host store path and symlinked into the VM's home directory (`~`). Because the mount path in the guest matches the host store path, absolute paths recorded in Claude Code plugin metadata continue to resolve. - **Unredacted content:** Mounted directories contain executable scripts, plugins, and skill definitions, so `devvm` mounts them as-is without scrubbing secrets. Do not store `.env` files, private keys, or embedded git credentials inside skill or plugin folders. - **Comma restriction:** Because Lima splits mount options on commas, `agentMounts` skips any directory whose path contains `,` and emits a warning (`agent_dir_comma`). - **Dynamic mount updates:** If you add or remove a settings folder in the store after a VM is created, `updateAgentDirs` detects the difference on the next launch. Stopped VMs pick up the new mounts immediately on start; running VMs prompt to restart so the mount change can take effect. #### Launch-time copies and secret redaction Files copied into the guest on launch are parsed and scrubbed in memory (`syncer.stageSecrets`) before being streamed over `tar` to `guest/sync.sh`: 1. **Settings file parsing and scrubbing (`agentlogin.Redact`):** - Applies to `.claude/settings.json`, `.codex/config.toml`, and `.json`, `.jsonc`, or `.toml` files in `.config/opencode/`. - Every file is decoded completely (`jsonc.Strip` + `json.Decoder` or `toml.Decode`). If a settings file fails to parse, `devvm` refuses to copy it into the VM and warns (`config_not_copied`). - Any string under a secret-matching key (`*key*`, `*token*`, `*secret*`, `*passw*`, `Authorization`, `Proxy-Authorization`, `Cookie`, excluding `keybind`/`keymap` tables) is replaced with `devvm-redacted`, unless the value is a variable reference (`OPENAI_API_KEY`, `{env:VAR}`, `{file:PATH}`, `$VAR`, `${VAR}`) or a file path (`/`, `~/`, `./`). - All string values inside `env`, `environment`, `headers`, `http_headers`, and Codex's `shell_environment_policy.set` tables are replaced with `devvm-redacted`, except booleans (`true`/`false`), up to 5-digit numbers, scrubbed non-credential URLs, and allowed top-level Claude Code configuration variables (`ANTHROPIC_MODEL`, `ANTHROPIC_SMALL_FAST_MODEL`, `ANTHROPIC_DEFAULT_*_MODEL`, `ANTHROPIC_CUSTOM_MODEL_OPTION*`, `CLAUDE_CODE_*`, `DISABLE_*`, `MAX_*`, and `*_TIMEOUT_MS` whose names do not match credential keywords). - Known token prefixes (`sk-`, `sk_`, `pk_`, `rk_`, `ghp_`, `gho_`, `ghs_`, `ghu_`, `github_pat_`, `glpat-`, `xox[abpr]-`, `AKIA`, `ASIA`, `AIza`, `ya29.`, `eyJ`), `Bearer`/`Basic` tokens in shell commands or hooks, secret-named CLI flags (`--api-key VAL`, `--token=VAL`), and URL userinfo or secret query parameters are scrubbed to `devvm-redacted`. - Non-settings, non-Markdown files in `.config/opencode/` (such as `.env` files) are withheld completely (`withheld`). 2. **Claude Code `.claude.json` merging and MCP server bindings:** - Only allowed top-level keys (`claudeTopKeys`, such as `mcpServers`, `oauthAccount`, `theme`, `editorMode`, and onboarding flags) are kept. `primaryApiKey`, `customApiKeyResponses`, and per-project `.projects` state are stripped. - For each server in `mcpServers`, `env` values are replaced with `devvm-redacted`, and `command`, `headersHelper`, `args`, and `url` are scrubbed. - If an MCP server specifies an `https://` URL on default port `443` with a valid DNS hostname, each entry in its `headers` map is replaced with `devvm-proxy-managed-claude-mcp-<12-hex-sha256>-` and registered as a `login` binding in the proxy's `secrets.json`. Plain HTTP MCP server headers are replaced with `devvm-redacted`. ## Importing and saving agent settings Use `devvm agent import` to seed the host store from your existing host configurations, and `devvm agent save` to persist configuration changes made inside a VM back to the host store. ### Importing from the host home directory ```bash # Import settings and logins for all configured agents in ~ devvm agent import # Import only specific agents devvm agent import claude codex # Overwrite existing store entries without interactive confirmation devvm agent import --force ``` `vm.AgentImport.Run` enforces the following rules when reading from `~`: - **Symlink containment:** Each agent's home folder (`~/.claude`, `~/.codex`, `~/.config/opencode`) is opened via `os.OpenRoot`. Symlinks that resolve to regular files inside that root are copied as regular files (`0600` or `0700` if owner-executable). Symlinks pointing outside the agent root, symlinks to directories, and symlinked login files are skipped with a warning (`agent_import_link_left_out`). - **Claude Code plugins:** When importing `.json` files inside `.claude/plugins`, `renamePrefix` rewrites host paths (`~/.claude/plugins/...`) to their corresponding store paths (`~/.config/devvm/agents/claude/.claude/plugins/...`) so mounted plugins resolve inside VMs. - **Claude Code credentials:** On macOS, Claude Code stores OAuth credentials in the system Keychain rather than `~/.claude/.credentials.json`; `devvm agent import` does not read the macOS Keychain for Claude Code. From `~/.claude.json`, it imports only `claudeTopKeys` via `agentlogin.ClaudeCopy`. - **opencode 2 SQLite export:** When the host `opencode --version` reports major version `2` or higher, `devvm agent import` executes `opencode auth export` in your home directory and converts active credentials via `agentlogin.OpencodeFromExport` into the store's `auth.json` format. Active API keys (`"type": "key"`) and OpenAI Codex OAuth logins (`chatgpt-browser`, `chatgpt-headless`, or legacy imported entries) are saved; unsupported OAuth methods (such as `chatgpt-token-sharing`) are skipped with a warning (`agent_import_opencode_skipped`). ### Saving settings from a VM to the host store ```bash # Inspect diffs and save settings for all agents in the current folder's VM devvm agent save # Inspect diffs and save settings only for Claude Code devvm agent save claude ``` Because files inside a VM may have been modified by an untrusted or compromised agent, `vm.AgentSave` validates every byte read from the guest before writing to `~/.config/devvm/agents/`: `devvm` starts the VM if stopped and runs `agentSaveScript` to stream agent settings over `tar` ( plus `.devvm-save/claude.json` filtered by `jq` to `ClaudeTopKeys`). `readSent` enforces hard limits: at most `128 MiB` total stream size (`saveMaxBytes`), `16 MiB` per file (`saveMaxFile`), `20,000` total entries (`saveMaxEntries`), and at most `8` relative symlink hops within the same settings folder (`saveLinkHops`). Any absolute path, `..` traversal, duplicate entry, or path containing non-graphic Unicode characters (`!unicode.IsGraphic`) aborts the entire save immediately (`agent_save_unsafe`). Guest settings files and `.claude.json` contain `devvm-redacted` and `devvm-proxy-managed-*` placeholders. `agentlogin.Restore` parses the VM's file, the redacted copy originally given to the VM, and the host store's original file. It restores the host store's real secret at a JSON/TOML path **only** if: - The VM's file has the exact same placeholder string at that path as the copy originally given to the VM, and - The immediate parent table/list and grandparent table/list (below the file's top level) are `reflect.DeepEqual` to what the VM was given. If a guest modifies the URL or arguments of an MCP server or model provider while keeping the placeholder, `Restore` returns a `*PlaceholderError`, and `devvm agent save` skips the entire file with a warning (`agent_save_placeholder`) so a VM cannot redirect a host secret to a different endpoint. Any unparsed file or non-settings file (such as a skill script) containing a placeholder is also skipped. `showPlans` prints added (`A`), modified (`M`), and deleted (`D`) folder entries and a unified line diff (capped at `20,000` lines, `1 MiB`, or `10,000` lines per file). Both sides of settings file diffs are passed through `agentlogin.Redact` so host secrets are never printed to the terminal, and non-printable/bidi characters are escaped as `\u{...}`. `devvm agent save` then prompts for confirmation (`y/N`) and refuses to run with `--no-input` or without a TTY. ## Authenticating coding agents `devvm agent login` runs an agent's login flow inside the current folder's VM (or saves an API key directly on the host) and stores the resulting credentials in `~/.local/share/devvm/logins//` (or the host secret store for `claude --setup-token`). The host machine does not need `claude`, `codex`, or `opencode` installed. ### Command syntax and flags ```bash devvm agent login AGENT [flags] ``` Target coding agent to authenticate: `claude`, `codex`, or `opencode`. The current folder's VM must include the agent in its shared tools. Valid only with `claude`. Runs `claude setup-token` inside the VM to mint a 1-year `user:inference` token and saves it as the `claude` host secret instead of an OAuth login in `.claude/.credentials.json`. Valid with `codex` and `opencode` (mutually exclusive with `--api-key`). Uses headless device-code authentication (`codex login --device-auth` or `opencode auth login openai --method chatgpt-headless`) so no localhost port forwarding is opened. Valid with `codex` and `opencode` (mutually exclusive with `--device-code`). Prompts for an API key on the host without echoing (or reads one line from standard input with `--stdin`) and writes it directly to the host store without running the agent's login binary in the VM. Required with `devvm agent login opencode --api-key` (invalid otherwise). Specifies the model provider whose API key is saved and bound by the proxy: `anthropic`, `openai`, or `openrouter`. This keeps BYOK workflows provider-neutral across supported upstream APIs. Valid only with `--api-key`. Reads the API key from a single line of standard input without prompting. ### In-VM OAuth token capture and proxy injection flow Every VM's proxy configuration registers the agents' OAuth token endpoints in `Config.Captures`. When an agent logs in inside a VM, `devvm _proxy` intercepts the token exchange on port `443`, captures the upstream response, writes the real tokens to the host store under an `flock` lock, and returns a sanitized response containing only salted placeholders to the VM. ```mermaid sequenceDiagram autonumber participant Guest as Agent in Guest VM participant Proxy as Host Proxy (devvm _proxy) participant Store as Host Store (internal/secrets & agentlogin) participant Auth as OAuth / API Upstream (:443) Note over Guest,Auth: Phase 1: In-VM OAuth Login Capture Guest->>Proxy: POST /oauth/token or /v1/oauth/token (TLS with VM CA) Proxy->>Store: Capturer.Request(Exchange) Store-->>Proxy: Verified request body Proxy->>Auth: Forward POST over upstream TLS Auth-->>Proxy: 200 OK {access_token, refresh_token, id_token} Proxy->>Store: Capturer.Response(Exchange, reply) Store->>Store: Lock & write real tokens to logins//
Regenerate vms//run/secrets.json Store-->>Proxy: Sanitized reply {access_token: placeholder-, refresh_token: "devvm-proxy-managed", expires_in: ~2099} Proxy-->>Guest: 200 OK (Sanitized reply, stripped headers) Note over Guest,Auth: Phase 2: Outbound API Request & Token Refresh Guest->>Proxy: POST https://api.anthropic.com or chatgpt.com
Authorization: Bearer placeholder- Proxy->>Store: Lookup(Binding) -> Store.Get alt Access token within 10m of expiry Store->>Auth: POST refresh_token grant (via company proxy if set) Auth-->>Store: New access_token & refresh_token Store->>Store: Atomic writeRefreshed to logins// end Store-->>Proxy: Real access_token / API key (cached 30s for login, 5m for secret) Proxy->>Auth: Forward request with real Authorization / x-api-key header Auth-->>Proxy: API response Proxy-->>Guest: API response ``` ### Agent-specific login procedures `devvm` supports two mutually exclusive authentication modes for Claude Code—a Claude subscription OAuth login or a long-lived setup token—and retains whichever credential was saved most recently: ```bash # Option 1: Log in to a Claude subscription (OAuth with automatic host refresh) devvm agent login claude # Option 2: Mint a 1-year setup token via claude setup-token devvm agent login claude --setup-token # Option 3: Save an existing sk-ant-oat01-... setup token directly on the host devvm secret set claude ``` - **Execution and code paste:** `devvm agent login claude` runs `env -u CLAUDE_CODE_OAUTH_TOKEN claude auth login --claudeai` (or `claude setup-token`) inside the VM. Open the printed URL in any browser, authorize, and paste the callback code into the terminal. No localhost ports are forwarded. - **Token endpoint capture:** The proxy intercepts `POST` requests to `platform.claude.com/v1/oauth/token` and `console.anthropic.com/v1/oauth/token`. - **Scope enforcement:** `claudeScopesOK` requires `user:inference` on new logins and rejects any login or refresh containing `org:create_api_key` with HTTP `502`. This blocks Anthropic Console logins (which only mint API keys) and prevents a guest VM from calling `api.anthropic.com/api/oauth/claude_cli/create_api_key` to generate a persistent API key using your OAuth token. - **Single active login & transition:** Capturing a subscription OAuth login writes `~/.local/share/devvm/logins/claude/.claude/.credentials.json` (`claudeAiOauth`) and deletes any saved `claude` secret. Conversely, capturing a `setup-token` request (identified by `expires_in` in the authorization code request) or running `devvm secret set claude` saves the `claude` secret and removes `.credentials.json`. Both operations synchronize on `~/.local/share/devvm/logins/claude/lock`. - **Guest files and placeholders:** A VM receives either `CLAUDE_CODE_OAUTH_TOKEN=sk-ant-oat01-devvm-proxy-managed-` in `~/.config/devvm/env` **or** `~/.claude/.credentials.json` with `"accessToken": "sk-ant-oat01-devvm-proxy-managed-oauth-"`, `"refreshToken": "devvm-proxy-managed"`, and `"expiresAt": 4070908800000` (year 2099), never both. Both bindings (`claude` and `claude-oauth`) are written to `secrets.json` and resolve to whichever credential `devvm` currently holds, so running VMs continue working across credential switches without a restart. If `CLAUDE_CODE_OAUTH_TOKEN` is set in the host environment where `devvm` runs, it overrides both stored credentials. `codex` supports ChatGPT browser OAuth, headless device-code OAuth, and OpenAI API keys: ```bash # Browser OAuth on this machine (forwards localhost:1455 and 1457 during login) devvm agent login codex # Device-code OAuth on any machine (no port forwarding) devvm agent login codex --device-code # Save an OpenAI API key from prompt or stdin devvm agent login codex --api-key printf '%s\n' "$OPENAI_API_KEY" | devvm agent login codex --api-key --stdin ``` - **Loopback callback forwarding:** Without `--device-code`, `AgentLogin.login` opens a temporary SSH loopback forward (`reach.SSH.ForwardLoopback`) from host `127.0.0.1:1455` (and fallback port `1457` when available) to the VM while `codex -c cli_auth_credentials_store="file" login` runs. If host port `1455` is already bound, `devvm` aborts and instructs you to pass `--device-code`. Pressing `Ctrl+C` closes the port forward and kills the in-VM login PID recorded in `~/.cache/devvm-login.pid`. - **Token capture and API-key exchange:** The proxy intercepts `POST https://auth.openai.com/oauth/token`. When Codex completes the `authorization_code` grant, `codexCapture` saves `id_token`, `access_token`, `refresh_token`, and `account_id` to `~/.local/share/devvm/logins/codex/.codex/auth.json` and returns JWT-shaped placeholders (`alg: "none"`, `exp: 4070908800`, signature `devvm-placeholder`) preserving non-sensitive account claims (`email`, `chatgpt_account_id`, `https://api.openai.com/auth`, `https://api.openai.com/profile`). When Codex immediately follows up with a `urn:ietf:params:oauth:grant-type:token-exchange` request (`requested_token=openai-api-key`) carrying the `id_token` placeholder, `CaptureRequest` swaps in the store's real `id_token` upstream, captures the returned API key into `OPENAI_API_KEY`, and gives the VM `sk-devvm-proxy-managed-codex-`. - **Bindings:** `codexBindings` binds `codex-key` (`Authorization` on `api.openai.com`) and `codex` (`Authorization` on `chatgpt.com` and `api.openai.com`). `opencode` supports OpenAI OAuth (via its Codex login methods) and provider-scoped API keys for `anthropic`, `openai`, and `openrouter`: ```bash # Browser OAuth to OpenAI using opencode's Codex browser method devvm agent login opencode # Device-code OAuth to OpenAI using opencode's Codex headless method devvm agent login opencode --device-code # Save an API key for Anthropic, OpenAI, or OpenRouter devvm agent login opencode --api-key --provider anthropic devvm agent login opencode --api-key --provider openai devvm agent login opencode --api-key --provider openrouter ``` - **Supported OpenAI OAuth methods:** `devvm agent login opencode` invokes `opencode auth login openai --method chatgpt-browser` (or `chatgpt-headless` with `--device-code`), which exchanges tokens at `auth.openai.com/oauth/token`. Do not select `opencode`'s interactive "Sign in with ChatGPT" (`chatgpt-token-sharing`) inside a VM: that flow requests tokens from `auth.openai.com/api/accounts/oauth/token` and verifies OpenAI's JWT signature inside the guest, so the proxy's `guard` filter blocks its token response with `502`. - **User-Agent disambiguation:** Because Codex and `opencode` share `auth.openai.com/oauth/token`, `agentlogin.Pick` inspects the HTTP `User-Agent` header in VMs that have both agents installed: requests starting with `Bun/` or `opencode/` are routed to `opencode`'s login store, and all others to Codex's. - **Provider bindings:** `opencodeBindings` creates proxy bindings for API keys under `anthropic` (`x-api-key` on `api.anthropic.com`), `openai` (`Authorization` on `api.openai.com`), and `openrouter` (`Authorization` on `openrouter.ai`), plus OAuth logins under `openai` (`Authorization` on `chatgpt.com`) and `github-copilot` (`Authorization` on `api.githubcopilot.com`, when `enterpriseUrl` is empty). API keys for other providers (such as local model servers) and `"type": "wellknown"` entries are left intact in the VM's copy with a warning (`opencode_login_kept`); unbound OAuth logins (such as Copilot Enterprise) have their `access` and `refresh` tokens replaced with `devvm-redacted` (`opencode_login_withheld`). - **SQLite credential synchronization (`guest/opencode.sh`):** `opencode` 2 imports `~/.local/share/opencode/auth.json` only once when creating `~/.local/share/opencode/opencode.db`. On subsequent VM launches—and immediately after `devvm agent login opencode --api-key`—`devvm` generates `opencode auth export` entries containing placeholders under deterministic IDs (`cred_devvm_`) via `agentlogin.OpencodeImport` and pipes them to `guest/opencode.sh`. Once `opencode.db` exists, `opencode.sh` compares the payload against `~/.local/state/devvm/opencode-import.json` and verifies all `cred_devvm_*` IDs exist in `opencode auth export`; when changed or missing, it logs out previous `cred_devvm_*` entries, imports each new entry via `opencode auth import`, and activates it via `opencode auth switch`. Credentials added manually inside the VM under non-`devvm` IDs are preserved. ## Managing host secrets and proxy bindings In addition to agent login files, `devvm` manages three named secrets in `internal/secrets`: | Secret name | Target hosts (`:443` only) | Injected HTTP header | Guest environment variable | Placeholder format | Binding scope | | --- | --- | --- | --- | --- | --- | | `claude` | `api.anthropic.com` | `Authorization` | `CLAUDE_CODE_OAUTH_TOKEN` | `sk-ant-oat01-devvm-proxy-managed-` | Automatically bound to every VM with the `claude` tool; can also be bound via `devvm secret use claude` | | `github` | `api.github.com`, `github.com`, `codeload.github.com`, `raw.githubusercontent.com` | `Authorization` | `GH_TOKEN` (plus git HTTPS credential helper when `DEVVM_GITHUB=1`) | `devvm-proxy-managed-github-` | Bound per-VM via `devvm secret use github` | | `proxy` | Host-only (`HostOnly: true`) | `Proxy-Authorization` (to upstream corporate proxy) | None (never exposed to VMs) | None | Used by host `devvm` and `devvm _proxy` when `config.toml` `[proxy]` specifies a username | ### Secret CLI operations ```bash # List known secrets, whether each is saved, and which VMs use it devvm secret devvm secret --json # Save a secret interactively or via stdin devvm secret set github gh auth token | devvm secret set github --stdin devvm secret set claude devvm secret set proxy # Bind or unbind a secret for the current folder's VM devvm secret use github devvm secret unuse github # Delete a saved secret from the host devvm secret rm github devvm secret rm github --force ``` ```bash devvm secret --json ``` ```json { "version": 1, "secrets": [ { "name": "claude", "saved": true, "used_by": [ "devvm-some-project-71a99b1076d92c" ] }, { "name": "github", "saved": true, "used_by": [ "devvm-some-project-71a99b1076d92c" ] }, { "name": "proxy", "saved": false, "used_by": [] } ] } ``` ### Storage, validation, and runtime injection - **Host storage backend:** On macOS (`platform.HasKeychain`), `secrets.Store` stores `claude`, `github`, and `proxy` in the login Keychain under service `devvm` (overridable via `DEVVM_SECRET_SERVICE` matching `^[A-Za-z0-9._-]+$`) using `security -i` with the secret passed over standard input. On Linux, secrets are stored in `~/.local/share/devvm/secrets/` with directory mode `0700` and file mode `0600`. - **Input validation (`secrets.Token`):** All whitespace is stripped from `claude` and `github` tokens (while `proxy` passwords only trim leading/trailing whitespace). Tokens containing quotes (`"`), backslashes (`\`), or ASCII control characters (`< 0x20` or `0x7f`) are rejected (`token_quotes`). A `claude` secret must start with `sk-ant-oat01-` (`not_setup_token`). - **Immediate proxy reload:** Running `devvm secret set`, `rm`, `use`, or `unuse` calls `Proxy.RefreshSecrets`, which rewrites `~/.local/share/devvm/vms//run/secrets.json` and `secrets.policy`. Because `devvm _proxy` watches `secrets.json` on every request (`watched[[]Binding]`) and clears its in-memory value cache whenever the file changes, secret additions and revocations apply immediately to open connections without restarting the VM. - **Header and Basic auth replacement (`secrets.inject`):** When `devvm _proxy` intercepts a TLS connection on port `443` to a bound host, it checks that the HTTP `Host` header matches the `CONNECT` tunnel's hostname (returning `421 Misdirected Request` otherwise). It then replaces the binding's placeholder inside the configured header—including inside base64-encoded `Authorization: Basic ...` headers used by `git` over HTTPS—and records the secret name in the `secrets` array of `network.jsonl`. - **Lookup serialization and caching:** `secrets.value` serializes lookups under `valueMu` with a `20s` timeout (`lookupTimeout`). `KindSecret` values are cached in memory for `5 minutes` (`valueTTL`); `KindLogin` values are cached for `30 seconds` (`loginTTL`). When an OAuth login (`codex`, `opencode`, or `claude-oauth`) is within `10 minutes` of expiration (`refreshMargin`), `File.Get` acquires an exclusive `flock` on the login directory and refreshes the token upstream (routing through the corporate proxy and trusting `~/.config/devvm/ca` if configured). - **Blocked GitHub credential-minting calls (`refuseMinting`):** Even when `github` is bound to a VM, `devvm _proxy` inspects every HTTPS request to `api.github.com` and blocks requests that would mint new persistent credentials capable of outliving `devvm secret unuse`: - `POST /user/keys`, `POST /user/ssh_signing_keys`, `POST /user/gpg_keys` (account keys) - `POST /repos/{owner}/{repo}/keys`, `POST /repositories/{id}/keys` (repository deploy keys) - `POST /.../{repos,repositories,orgs,organizations,enterprises}/.../actions/runners/{registration-token,generate-jitconfig}` (self-hosted Actions runners) - `POST` or `PATCH` to `/.../{repos,repositories,orgs,organizations}/.../hooks...` (webhooks) ## Exporting and importing agent history Session transcripts and databases remain inside each VM and are never placed in `~/.config/devvm/agents/` or `~/.local/share/devvm/logins/`. When you delete a VM with `devvm rm` (unless `--no-export` is passed) or run `devvm history export`, `devvm` archives the VM's agent history to a `.tar.gz` file on the host. ```bash # List saved exports in ~/.local/share/devvm/exports (oldest first) devvm history devvm history --json # Export the current VM's agent history to ~/.local/share/devvm/exports or a custom path devvm history export devvm history export ~/backups/project-history.tar.gz # Import the newest export for this folder (or a specific archive) into the current VM devvm history import devvm history import ~/backups/project-history.tar.gz ``` ### Sanitization and safety boundaries in history archives - **Included and excluded paths:** `historyExport` archives `.claude`, `.claude.json`, `.codex`, `.local/share/opencode`, and `.local/state/opencode`, capped at `2 GiB`. It explicitly excludes `.claude/.credentials.json`, `.codex/auth.json`, `.local/share/opencode/auth.json`, `.local/share/opencode/account.json`, `.local/share/opencode/mcp-auth.json`, `.local/share/opencode/bin`, and raw `opencode*.db*` files (`-wal`, `-shm`, `-journal`). - **opencode SQLite credential scrubbing (`devvm_opencode_clean`):** Because `opencode` 2 stores session history and credentials in the same SQLite database (`~/.local/share/opencode/opencode.db` in WAL mode), both `historyExport` and `historyImport` run `devvm_opencode_clean` inside the guest using `sqlite3 -bail -cmd 'PRAGMA trusted_schema = OFF'`. The function snapshots the database via `VACUUM INTO 'clean.db'`, drops all triggers, deletes all rows from any table whose name or column names match credential patterns (`opencodeLogins`: `credential`, `account`, `control_account`, `account_state`, `session_share`, or columns like `access_token`, `refresh_token`, `api_key`, `secret`), deletes `wellknown:config:%` rows from `kv`, removes the `20260805200742_import_legacy_credentials` migration entry so `opencode` re-imports the VM's `auth.json` placeholders on next launch, and runs `PRAGMA journal_mode = DELETE; PRAGMA secure_delete = ON; VACUUM;` to purge deleted pages before verifying zero rows remain. - **Untrusted archive unpacking (`historyImport`):** `devvm` streams the archive into the guest and never unpacks it on the host. Inside the VM, GNU `tar` lists all archive entries first and aborts with `devvm-history-unsafe` if any entry falls outside the five allowed history paths, starts with `/`, or contains `..`. It extracts into a temporary directory with `--no-same-owner --no-same-permissions` and excludes `importSkips()` (all login files, `.claude.json`, and all store-mounted/copied agent paths), deletes any symlink targeting an absolute path or `..`, cleans any `opencode*.db` files via `devvm_opencode_clean`, and finally copies the remaining files into `~` with `--skip-old-files` so existing VM files are never overwritten. ## Troubleshooting and failure modes Browser OAuth for `codex` and `opencode` forwards host `127.0.0.1:1455` to the VM so the browser redirect can reach the agent's local callback server. If another process on your host is listening on port `1455`, `devvm agent login` fails with `agent_login_port_taken`. Run `devvm agent login codex --device-code` or `devvm agent login opencode --device-code` to authenticate with a one-time device code without port forwarding. `devvm _proxy` returns `502` (`devvm couldn't read the login's token request or reply, so it went no further`) in three situations: 1. You authenticated Claude Code with an Anthropic Console (API billing) account that lacks the `user:inference` scope or includes `org:create_api_key`. Log in with a Claude subscription account or use `devvm agent login claude --setup-token`. 2. Inside `opencode`, you selected `Sign in with ChatGPT` (`chatgpt-token-sharing`), which uses `/api/accounts/oauth/token`. The proxy guard blocks token responses from uncaptured paths so raw tokens never enter the VM. Use `devvm agent login opencode` on the host (which selects `chatgpt-browser` or `chatgpt-headless`). 3. You ran `claude /login` inside a VM whose image does not include the `claude` shared tool. Add the tool first with `devvm tools add claude`. Each VM has a dedicated ECDSA P-256 CA (`vms//run/ca/ca.pem`) and a pre-installed successor CA (`next.pem`), each valid for 30 days. If a VM runs continuously past `next.pem`'s expiration without `devvm` being run in its project folder, `caEndedRefusal` rejects `CONNECT` tunnels to bound secret hosts on port `443` with `403` and logs `devvm's CA has ended: open the VM again (devvm in its folder) for a new one`. Run `devvm` in the project directory on the host to rotate the VM's CA certificates and restart its proxy. When `devvm agent save` warns that a file was left out because of a placeholder (`agent_save_placeholder`), the agent or user inside the VM modified the table containing a `devvm-redacted` or `devvm-proxy-managed-*` placeholder (or its parent table), or placed a placeholder into a non-settings file. Edit the file directly in `~/.config/devvm/agents//` or `~/.local/share/devvm/logins//` on the host instead. When an OAuth refresh token expires or is revoked upstream, `File.Get` logs `claude_refresh_failed`, `codex_refresh_failed`, or `opencode_refresh_failed` to `~/.local/share/devvm/vms//logs/proxy.log` andwithholds the header rather than sending a stale token. Re-authenticate by running `devvm agent login ` in the project folder. ## Related pages Traffic confinement across allowlist, internet, none, and open tiers, TLS SNI inspection, and per-VM CA rotation. Complete syntax, subcommands, and flags for devvm agent, devvm secret, devvm network, and devvm git-protect. Attach shared toolchains and coding agents to VMs with devvm tools and build custom image layers. Configure automatic retention policies and garbage collection for agent history exports and shared tool images. --- ## 10. Manage toolchains and custom layers > Add, remove, and update shared mise toolchains (node, python, rust, go, dotnet) and author custom image layers with network policies via devvm layer new. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/10-manage-toolchains-and-custom-layers.md - Generated: 2026-11-07T22:13:01.520Z ### Source Files - `internal/cli/tools.go` - `internal/cli/layer.go` - `internal/tools/tools.go` - `internal/tools/share.go` - `assets/shared-tools/mise.toml` - `assets/shared-tools/build.sh` - `internal/vm/tools.go` --- title: "Manage toolchains and custom layers" description: "Add, remove, and update shared mise toolchains (node, python, rust, go, dotnet) and author custom image layers with network policies via devvm layer new." --- `devvm` separates language toolchains from disk image layers. Shared toolchains (`node`, `python`, `rust`, `go`, `dotnet`) and coding agents (`claude`, `codex`, `opencode`) are built once per version into compressed, read-only EROFS images in `~/.local/share/devvm/tools` and loop-mounted into project VMs at `/opt/devvm/tools//`, where `mise` activates them without modifying the VM disk. Image layers (`desktop`, `docker`, `nix`, and user scripts in `~/.config/devvm/layers/.sh`) execute during image builds to modify the root filesystem on top of `base`; changing a VM's layers via `devvm layer add` or `devvm layer rm` remakes the VM disk from the new image chain while preserving its home directory and settings. ## Architecture: shared toolchains vs. image layers Shared tools and image layers follow distinct build, storage, and attachment paths. Modifying a VM's shared tools updates `vm.json` and remounts EROFS images on the next shell launch, whereas modifying layers requires cloning a new VM disk from the target image chain. ```mermaid flowchart TD subgraph HostConfig["Host configuration & lock (~/.config/devvm & ~/.local/share/devvm)"] MiseTOML["assets/shared-tools/mise.toml"] HostMise["Pinned host mise
~/.local/share/devvm/cache/mise"] ToolLock["~/.local/share/devvm/mise.lock"] LayerScripts["Built-in & custom layers
~/.config/devvm/layers/NAME.sh + NAME.policy"] end subgraph SharedStore["Read-only tool share (~/.local/share/devvm/tools)"] EROFS["--.erofs
+ .erofs.json record"] end subgraph ImageStore["Backend disk images"] BaseImg["base image
(Ubuntu 26.04 or --from)"] LayerImg["Layered image
(e.g. docker+mailpit)"] end subgraph GuestVM["Project VM"] VMDisk["Cloned VM root disk"] ShareMount["/mnt/devvm-tools
(read-only host share)"] LoopMount["/opt/devvm/tools//
(EROFS loop mount via devvm-tools)"] MiseCfg["/etc/mise/config.toml
& /etc/profile.d/devvm-tools.sh"] end MiseTOML --> HostMise --> ToolLock ToolLock -->|"build.sh in throwaway base VM"| EROFS BaseImg --> LayerScripts --> LayerImg LayerImg -. "CoW clone / overlay" .-> VMDisk EROFS --> ShareMount --> LoopMount --> MiseCfg ``` | Dimension | Shared toolchains (`--tools`, `devvm tools`) | Image layers (`--layers`, `devvm layer`) | | --- | --- | --- | | **Available names** | `node`, `python`, `rust`, `go`, `dotnet` (plus agents `claude`, `codex`, `opencode`) | Built-in `desktop`, `docker`, `nix`, plus custom `~/.config/devvm/layers/.sh` | | **Storage format** | Read-only `lz4hc`-compressed EROFS image (`--.erofs`) | Full VM disk image keyed by 12-hex-digit SHA-256 chain hash | | **Guest path** | Loop-mounted at `/opt/devvm/tools//` from `/mnt/devvm-tools` | Baked directly into the VM's root filesystem (`/`, `/usr`, `/etc`, `/nix`) | | **Default selection** | Auto-detected from top-level project files and `devcontainer.json` features | Chosen via `devvm setup --layers`, `config.toml`, or `devcontainer.json` features | | **Adding/removing on existing VM** | In-place state update (`devvm tools add` / `rm`); VM disk is untouched | Full VM disk remake (`devvm layer add` / `rm`); home folder and settings kept | | **Updating versions** | `devvm tools update` or `devvm update` swaps mounted EROFS files | `devvm rebuild ` followed by `devvm update` remakes the VM disk | --- ## Shared toolchains Every toolchain is defined in `internal/image/tool.go` (`image.SharedTools`) and configured in `assets/shared-tools/mise.toml`. When a new VM is created without `--tools`, `devvm` scans the top-level project directory for marker files (`detectFiles` in `internal/vm/detect.go`) in a fixed order and selects every matching toolchain. ### Supported toolchains and project markers | Toolchain | Lock members | Auto-detection files (top-level) | Project version file | Writable global install path | Allowlist hosts (`assets/shared-tools/.policy`) | | --- | --- | --- | --- | --- | --- | | `node` | `node` (held at `24`), `pnpm`, `bun`, plus `yarn` enabled via `corepack` | `package.json`, `bun.lock`, `bun.lockb` | `.nvmrc`, `.node-version` | `~/.npm-global/bin` (`NPM_CONFIG_PREFIX=~/.npm-global`) | `registry.npmjs.org`, `registry.yarnpkg.com`, `repo.yarnpkg.com`, `nodejs.org`, `bun.sh` | | `python` | `python` (prebuilt `python-build-standalone`), `uv` | `pyproject.toml`, `requirements.txt`, `uv.lock`, `setup.py`, `Pipfile` | `.python-version` | `~/.local/bin` (or project virtualenv) | `pypi.org`, `files.pythonhosted.org`, `releases.astral.sh` | | `rust` | `rust` (via pinned `rustup-init` with `rust-src` and `rust-analyzer`) | `Cargo.toml` | `rust-toolchain.toml` | `~/.cargo/bin` | `crates.io`, `index.crates.io`, `static.crates.io`, `static.rust-lang.org` | | `go` | `go` (with `go telemetry off`) | `go.mod` | `toolchain` directive in `go.mod` | `~/go/bin` (`go_set_gobin = false`) | `proxy.golang.org`, `sum.golang.org`, `go.dev`, `dl.google.com` | | `dotnet` | `dotnet` (SDK and ASP.NET Core / .NET runtimes) | `*.sln`, `*.slnx`, `*.csproj`, `*.fsproj`, `global.json` | `sdk.version` in `global.json` | `~/.dotnet/tools` (workloads in `~/.dotnet`) | `api.nuget.org`, `crl3.digicert.com:80`, `dot.net`, `builds.dotnet.microsoft.com` | `node` is held on release line `24` in `assets/shared-tools/mise.toml` because Node 25+ drops Corepack. All other toolchains track `"latest"`. ### How toolchains are resolved, built, and mounted 1. **Lock resolution on the host:** `devvm` downloads a pinned `mise` binary to `~/.local/share/devvm/cache/mise//mise` (verified against `miseSHA256` in `internal/tools/mise.go`) and resolves `assets/shared-tools/mise.toml` into `~/.local/share/devvm/mise.lock` for both `linux-` (`gnu`) and `linux--musl` (`musl`). - For `dotnet`, `mise` locks only the version number; `devvm` fetches Microsoft's channel metadata (`releases.json`) and injects each platform's `dotnet-sdk-.tar.gz` URL and `sha512:` checksum into the lock (`withSDKDownloads` in `internal/tools/dotnet.go`). - If the newest upstream release of a tool lacks a binary download for either `gnu` or `musl` (for example, while musl builds are still publishing), `stepBack` inspects up to 5 previous releases and locks the newest version that provides prebuilt binaries for all target platforms. - Once a day, launching a VM spawns `devvm _refresh-tools` in the background (`RefreshDaily`) to refresh `mise.lock` unless `DEVVM_NO_TOOL_CHECK=1` is set in the environment. 2. **Isolated EROFS build:** For each unbuilt tool image `--.erofs`, `Share.Build` (`internal/tools/share.go`) boots a throwaway VM cloned from `base` with a dedicated temporary output directory mounted at `/mnt/devvm-tools`. Inside the build VM, `assets/shared-tools/build.sh` installs the locked version into `/opt/devvm/tools//`, scans ELF binaries with `readelf -W -V` on `gnu` builds to record the maximum `GLIBC_2.x` symbol requirement in `.glibc`, and packs the directory using `mkfs.erofs -zlz4hc --all-root`. 3. **Guest mounting and `mise` activation:** On every VM launch, `guest/sync.sh` writes the VM's active tool list to `/etc/devvm/tools` and invokes `/usr/local/sbin/devvm-tools` (`assets/guest/tools.sh`). `devvm-tools` attaches each `.erofs` file from `/mnt/devvm-tools` via `losetup --find --show --read-only --direct-io=on` and mounts it read-only at `/opt/devvm/tools//`. - `sync.sh` generates `/etc/mise/config.toml` pointing `shared_install_dirs` to `["/opt/devvm/tools"]` and enabling idiomatic version files (`["node", "python", "bun", "pnpm", "dotnet"]`). - Non-interactive shells (`bash -c` from coding agents or SSH editors) load `/etc/profile.d/devvm-tools.sh` via `BASH_ENV`, while interactive shells also run `eval "$(mise activate bash)"` from `~/.bashrc.d/mise.sh`. - If any listed tool image is missing from `/mnt/devvm-tools` or fails to mount, `devvm-tools` writes the failed filenames to `/run/devvm/tools-unmounted` and exits `1`; `devvm` aborts the launch rather than allowing the guest to download an unverified replacement. ### Base libc and glibc compatibility Every toolchain builds separately for `gnu` (built in `devvm`'s Ubuntu `base` and shared across all `gnu` VMs) and `musl` (built in the specific `--from` Alpine base image and keyed to its digest). Before attaching a `gnu` tool image to a `--from` Debian or Ubuntu container base, `Share.Runs` compares the base image's recorded glibc version against the tool image's recorded `Glibc` requirement and refuses the tool if the base's glibc is older: | Shared tool | Required glibc (`arm64`) | Required glibc (`amd64`) | | --- | --- | --- | | `node` | `2.28` | `2.28` | | `pnpm` / `bun` (bundled with `node`) | `2.18` / `2.17` | `2.18` / `2.17` | | `python` | `2.17` | `2.17` | | `uv` (bundled with `python`) | `2.28` | `2.17` | | `rust` (with `rustup`) | `2.17` | `2.17` | | `go` | none (statically linked / no glibc symbol needs) | none | | `dotnet` | `2.27` (plus `libicu` / `icu-libs` in `base`) | `2.27` | --- ## Managing a VM's shared tools (`devvm tools`) The `--tools` and `--agents` flags on `devvm` apply only when creating a new VM. On an existing VM, use `devvm tools` and its subcommands to inspect, add, remove, or update shared toolchains and coding agents in place. ```bash title="Interactive picker or table" # In a terminal: open the interactive checkbox picker for toolchains and agents devvm tools # Without a terminal, with --no-input, or with --json: inspect current and newer versions offline devvm tools --no-input devvm tools --json ``` ```bash title="Add, remove, and update tools" # Add toolchains or agents (separate words; builds missing images, then opens a shell) devvm tools add python claude # Remove shared tools from the VM (software installed inside the VM's disk stays) devvm tools rm python # Check upstream, build newer tool images, switch the VM to them, and open a shell devvm tools update ``` ### Subcommands and behavior - **`devvm tools` (interactive):** In a terminal, opens the three-section picker (`Toolchains`, `Coding agents`, `Layers`) with the VM's current tools checked and its layers greyed out (`Fixed: true`). Confirming applies additions and removals to `vm.json` and opens a shell in the VM. If nothing changed, `devvm` prints `Nothing changed.` and exits without starting the VM. - **`devvm tools` (non-interactive or `--json`):** Reads `vm.json` and the local `~/.local/share/devvm/mise.lock` without network access or starting the VM. Prints a table with columns `TOOL`, `VERSION`, `NEWER`, and `BUILT` (with `pnpm`, `bun`, and `uv` listed on their own rows). - **`devvm tools add TOOL...`:** Validates that each argument is a known toolchain or agent name (refusing names joined with `+` or `,`), builds any missing EROFS images, updates the `tools` section of `vm.json`, and opens a shell. If you add a coding agent whose host settings folders (`~/.claude`, `~/.codex`, etc.) are not yet mounted in a running VM, `devvm` prompts to restart the VM so the read-only host mounts take effect. - **`devvm tools rm TOOL...`:** Removes the specified shared tools from `vm.json` and unmounts their EROFS images on the next sync. Refuses layer names (pointing to `devvm layer rm`) and tools the VM does not have. Any packages or alternate versions installed into the VM's home directory remain untouched. - **`devvm tools update` vs. `devvm update`:** `devvm tools update` refreshes `mise.lock` online, builds any missing or newer EROFS images for the VM's current tools, updates `vm.json`, and opens a shell without touching the VM's root disk. `devvm update` runs this exact shared-tools update first, and then—if the VM's base or layer image chain or `devcontainer.json` has changed—prompts to remake the VM disk. ### `devvm tools --json` output schema ```bash devvm tools --json ``` ```json { "version": 1, "tools": [ { "tool": "node", "version": "24.10.0", "newer": "24.11.0", "newer_built": false }, { "tool": "pnpm", "version": "10.18.0" }, { "tool": "bun", "version": "1.3.1" } ] } ``` JSON schema version (`1`). Ordered list of individual lock tools mounted in the VM (`node` expands to `node`, `pnpm`, and `bun`; `python` expands to `python` and `uv`). Tool name in `mise.lock`. Version currently mounted in the VM. Version in `mise.lock` when the lock's current image file differs from the VM's mounted file. Omitted when the VM already uses the lock's current build. Present only when `newer` is set. `true` if the newer `.erofs` image is already built in `~/.local/share/devvm/tools`; `false` if `devvm tools update` must build it first. --- ## Built-in layers and Nix projects Unlike shared tools, layers run as bash build scripts inside an image VM on top of `base` and persist in the image's disk. Image names join layers bottom-up with `+` (for example, `docker+nix` runs `nix` on top of `docker` on top of `base`). | Built-in layer | Source script & policy | What it installs into the image | | --- | --- | --- | | `desktop` | `assets/layers/desktop.sh`, `assets/layers/desktop.policy` | Xfce desktop environment and TigerVNC server (~370 MB) for `devvm desktop`. Needs no extra allowlist hosts. | | `docker` | `assets/layers/docker.sh`, `assets/layers/docker.policy` | Docker Engine, CLI, BuildKit, and Compose from Docker's apt repository (or Alpine packages on `--from alpine`), adds the guest user to the `docker` group, and enables `docker.service`. | | `nix` | `assets/layers/nix.sh`, `assets/layers/nix.policy` | Multi-user Nix daemon with `flakes` and `nix-command` enabled, plus `direnv` and `nix-direnv`. | ### Working with Nix projects When a project contains `flake.nix`, `shell.nix`, `devenv.nix`, or a `default.nix` containing `mkShell` (up to the first 64 KiB, checked by `nixShell` in `internal/vm/detect.go`), `devvm` annotates the `nix` row in the interactive new-VM picker with that filename. Because layers are not enabled automatically from project files, pass `--layers nix` in non-interactive runs or check `nix` in the picker: ```bash devvm --layers nix ``` - ** Automatic dev shells:** Add `use flake` (or `use nix`) to `.envrc` and run `direnv allow` once inside the VM. `nix-direnv` caches the evaluated profile under `~/.cache/nix-direnv` inside the VM home directory so no `.direnv` directory is written into the host project folder. While inside the project directory, flake binaries precede `/opt/devvm/tools` shims on `PATH`. - **Daemon trust and sandboxing:** Only `root` is a trusted Nix user in the VM (`nix` ignores `--option substituters` from the unprivileged guest user), and Nix package builds always run inside the Nix sandbox. - **Shared host binary cache (`http://devvm.proxy/nix`):** On every network tier with a host proxy, `guest/sync.sh` configures `/etc/nix/devvm-cache.conf` (`substituters = http://devvm.proxy/nix https://cache.nixos.org`) and sets the proxy environment on `nix-daemon`. When any VM requests a store path from `cache.nixos.org`, the host proxy verifies the `cache.nixos.org-1` signature and caches the NAR in `~/.local/share/devvm/cache/nix` (capped at 10 GiB by default, configurable with `devvm retention nix-cache.size=20G`, and cleared with `devvm gc --nix-cache`). --- ## Authoring custom layers (`devvm layer new`) Custom layers let you bake system packages, daemons, or prebuilt Nix environments into reusable images. User layers are stored in `~/.config/devvm/layers` (`$XDG_CONFIG_HOME/devvm/layers`). :::files ~/.config/devvm/ ├── config.toml └── layers/ ├── mailpit.sh ├── mailpit.policy ├── myteam.sh └── myteam.policy ::: ### Naming rules and constraints `build.CheckLayerName` (`internal/build/layer.go`) enforces strict naming rules before creating or building a layer: - Must match `^[a-z0-9][a-z0-9-]*$` (lowercase ASCII letters, digits, and hyphens, starting with a letter or digit). - Cannot be `base`. - Cannot be a built-in layer name (`desktop`, `docker`, `nix`). - Cannot be a shared tool name (`node`, `python`, `rust`, `go`, `dotnet`, `claude`, `codex`, `opencode`). Do not run `devvm` inside `~/.config/devvm` to edit layer scripts with a coding agent. `devvm` refuses to create a project VM for any directory inside `~/.config/devvm` or `~/.local/share/devvm` to prevent a guest VM from modifying host configuration or layer scripts. Keep custom layers in a separate Git repository and copy them into `~/.config/devvm/layers/`. ### Creating and building a custom layer Run `devvm layer new NAME` to atomically create `~/.config/devvm/layers/NAME.sh` and `~/.config/devvm/layers/NAME.policy` (if the policy file does not already exist). `devvm` refuses to overwrite an existing `NAME.sh`. ```bash devvm layer new mailpit ``` This writes the starter script and policy template: ```bash title="~/.config/devvm/layers/mailpit.sh" #!/bin/bash # mailpit: a devvm layer. devvm runs it once, with bash as the VM's user (who has sudo), in an image # on top of base: devvm build mailpit builds it, and a new VM gets it with --layers mailpit. Editing it rebuilds that image, # and the ones built on it, on next use. # - Put PATH changes in ~/.profile. # - Download with devvm-fetch URL SHA256 OUT: it fails when the checksum doesn't match. # - On the allowlist tier, mailpit.policy next to this file lists the hosts VMs with this layer reach. set -euo pipefail # sudo apt-get install -y postgresql-client ``` The script runs once via `bash` as the unprivileged VM user (with passwordless `sudo`) during `devvm build`. - Use `devvm-fetch URL SHA256 OUT` (installed in `/usr/local/bin/devvm-fetch` on `base`) for all binary downloads so the build aborts on a SHA-256 mismatch. - Put interactive shell customizations in `~/.bashrc.d/.sh` and `PATH` additions in `~/.profile`. ```bash title="~/.config/devvm/layers/mailpit.sh" #!/bin/bash set -euo pipefail mailpit_version=1.32.0 mailpit_sha256_amd64=4b5993c978e8604c70d43927e2ad103b311bba3b1822fa0cf1d4ee4ec890a93c mailpit_sha256_arm64=e1e45d1dc72bd497fb262a1e4100178eee057c42c5fc36d143eb8621769e1137 case "$(uname -m)" in x86_64) arch=amd64 sha256=$mailpit_sha256_amd64 ;; aarch64) arch=arm64 sha256=$mailpit_sha256_arm64 ;; *) echo "mailpit: no build for $(uname -m)" >&2; exit 1 ;; esac devvm-fetch "https://github.com/axllent/mailpit/releases/download/v$mailpit_version/mailpit-linux-$arch.tar.gz" \ "$sha256" /tmp/mailpit.tar.gz sudo tar -xzf /tmp/mailpit.tar.gz -C /usr/local/bin mailpit rm /tmp/mailpit.tar.gz printf '%s\n' '[Unit]' 'Description=Mailpit' '[Service]' 'ExecStart=/usr/local/bin/mailpit' \ 'DynamicUser=yes' '[Install]' 'WantedBy=multi-user.target' | sudo tee /etc/systemd/system/mailpit.service >/dev/null sudo systemctl enable --now mailpit ``` Edit `~/.config/devvm/layers/NAME.policy` to specify host rules for VMs using this layer (`netpolicy.Policies` in `internal/netpolicy/netpolicy.go`). Put one rule per line (`#` comments are ignored): - `example.com` or `*.example.com` (any subdomain on default HTTP/HTTPS/SSH ports) - `host:port` (specific port, such as `crl3.digicert.com:80`) - IPv4/IPv6 address or CIDR block - `!rule` (explicit deny rule; enforced on `allowlist`, `internet`, and `none` tiers) ```text title="~/.config/devvm/layers/mailpit.policy" # Hosts a VM with the mailpit layer can reach on the allowlist tier, one rule per line: api.sendgrid.com *.mailgun.net ``` Build the image directly with `devvm build`, pass `--layers` when creating a new VM, or set it as a default for all new VMs: ```bash # Pre-build the image (or stacked image docker+mailpit) devvm build mailpit devvm build docker+mailpit # Create a new project VM with docker at the bottom and mailpit on top devvm --layers docker,mailpit # Set mailpit as a default layer for all new VMs devvm setup --layers mailpit ``` Because an image's 12-hex-digit content key (`image.Input.Key` in `internal/image/key.go`) hashes the parent image's key and the exact bytes of `NAME.sh`, editing `~/.config/devvm/layers/NAME.sh` automatically marks `NAME` and any layers stacked above it as `stale` and rebuilds them on next use. To re-run an unchanged script (for example, to pull newer unpinned `apt` packages), run `devvm rebuild NAME`. ### Baking a team Nix flake into a custom layer To pre-populate a team's Nix dev shell inside an image so VMs start with all store paths already on their CoW-cloned disk, create a custom layer and stack it on top of `nix` (`devvm --layers nix,myteam` -> image `nix+myteam`): ```bash title="~/.config/devvm/layers/myteam.sh" #!/bin/bash set -euo pipefail # Pin the team flake to an exact commit so the image key corresponds to that dev shell. flake=github:myorg/devshell/0123456789abcdef0123456789abcdef01234567 bin=/nix/var/nix/profiles/default/bin "$bin/nix" develop "$flake" --profile /tmp/myteam-shell --command true # Register a GC root in the VM's home directory so nix-collect-garbage keeps the store paths. mkdir -p ~/.cache/myteam "$bin/nix-store" --add-root ~/.cache/myteam/dev-shell --realise "$(readlink -f /tmp/myteam-shell)" rm -f /tmp/myteam-shell /tmp/myteam-shell-*-link ``` --- ## Adding and removing layers on an existing VM Because layers modify the root filesystem during image build, changing an existing VM's layers (`devvm layer add` or `devvm layer rm`) remakes the VM from the new image chain (`Lifecycle.AddLayers` and `Lifecycle.RemoveLayers` in `internal/vm/remake.go`). ```bash # Add layers on top of the VM's current image (e.g., docker -> docker+desktop) devvm layer add desktop devvm layer add docker nix # Remove layers while keeping remaining layers in their original order (e.g., docker+nix+desktop -> docker+desktop) devvm layer rm nix # Run non-interactively in scripts or start with a fresh home directory from the image devvm layer add desktop --force devvm layer rm docker --clean-home ``` ### Flags for `devvm layer add` and `devvm layer rm` Skip the interactive confirmation prompt (`Make again from ? [y/N]`). Required when running without a terminal or with `--no-input`. Does not bypass the parent-folder prompt when invoked from a subdirectory. Skip exporting coding agent session history (`~/.local/share/devvm/exports`) before replacing the VM disk. Start the remade VM with the new image's clean home folder instead of saving and restoring the existing VM's home folder. Run as if `devvm` was started in the specified project directory. ### Remake lifecycle, preservation, and crash recovery Before changing anything, `remaking.check` verifies that: - All requested layer names exist (for `layer rm`, you can still remove a custom layer whose `.sh` script you deleted from `~/.config/devvm/layers/`, which is how you recover an outdated VM after deleting a layer script). - The VM is not a `--clone` VM (to change layers on a `--clone` VM, run `devvm clone off`, then `devvm layer add`/`rm`, then `devvm clone on`). - The backend supports all required features of the target image (`docker-layer`, `nix-layer`) and the host volume has sufficient free disk space. Once confirmed, `remaking.remake` executes five atomic phases: 1. **Build target image:** Builds the new image chain (`base` through the top layer) so a build failure leaves the existing VM untouched. 2. **Export history and archive home:** If the VM has coding agents, exports their transcripts to `~/.local/share/devvm/exports/` (unless `--no-export`). Ensures the VM is running and archives its home directory to `~/.local/share/devvm/vms//home.tar` (unless `--clean-home`), excluding `~/.cache`, `~/.config/devvm`, `~/.ssh/authorized_keys`, setuid files, and host-mounted agent config links. 3. **Set old machine aside:** Records the pending `remake` section in `vm.json`, stops the VM, proxy, port forwarders, and link sessions, and renames the backend VM to `-old` (`vmname.Aside`). 4. **Create and boot new machine:** Clones the new VM from the target image with the preserved settings (size, network tier, allowlist, secret bindings, SSH agent mode, links, groups, and `devcontainer.json` state), starts it, and restores the saved home archive before `guest/sync.sh` runs. 5. **Commit and drop old machine:** Marks `done: true` in `vm.json`, deletes the `-old` machine, clears the `remake` section and temporary `home.tar`, and opens a shell in the VM. If any step fails or is interrupted with `Ctrl+C` before the home directory is restored in Step 4, the cleanup handler (`remaking.undo`) deletes the new machine and renames `-old` back to ``. If the `devvm` process is killed (for example, `SIGKILL` or power loss), `vm.json` retains the `remake` state; the next `devvm`, `devvm layer add`, `devvm layer rm`, or `devvm update` in the project directory automatically resumes and finishes the remake (`finishPending`). ## Related pages Deterministic SHA-256 key computation for base images, built-in and custom layers, and read-only EROFS shared tool images. Run Claude Code, Codex, and opencode inside VMs, import and save agent settings, and bind host secrets. Translate .devcontainer/devcontainer.json properties into VM plans and build VM base images from container references with --from. Complete flag and syntax reference for devvm build, rebuild, images, layer, tools, space, gc, retention, and setup. --- ## 11. Use devcontainer.json and OCI base images > Translate .devcontainer/devcontainer.json properties into VM plans and build bootable VM base images from OCI/Docker container references with --from. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/11-use-devcontainer.json-and-oci-base-images.md - Generated: 2026-11-07T22:15:44.748Z ### Source Files - `internal/devcontainer/devcontainer.go` - `internal/devcontainer/reread.go` - `internal/vm/devcontainer.go` - `internal/from/from.go` - `internal/from/registry.go` - `internal/backend/lima/oci.go` - `assets/oci/build.sh` --- title: "Use devcontainer.json and OCI base images" description: "Translate .devcontainer/devcontainer.json properties into VM plans and build bootable VM base images from OCI/Docker container references with --from." --- `devvm` reads project `.devcontainer/devcontainer.json` or `.devcontainer.json` files through `internal/devcontainer` to configure VM base images, shared toolchains, layers, port forwards, environment variables, and lifecycle commands, and converts OCI/Docker container images specified via `--from`, `DEVVM_FROM`, or `devcontainer.json` `image` into bootable VM disks inside an isolated `devvm-oci-build` throwaway VM. ## Configuration precedence and discovery When creating a project VM (`devvm`) or re-evaluating a project on `devvm update`, `internal/vm.devcontainers` opens the project directory using `os.OpenRoot` (`O_RDONLY|platform.NoBlock`) so symlinks outside the project folder are never traversed and FIFOs never block. `devcontainer.Find` inspects `.devcontainer/devcontainer.json` first and falls back to `.devcontainer.json` at the project root, accepting only regular files up to `1 MiB` (`1048576` bytes) and stripping JSON comments and trailing commas via `jsonc.Strip`. Command-line flags always override `devcontainer.json` properties, which in turn override global defaults and environment variables: ```mermaid flowchart LR subgraph CLI["CLI Flags (Highest Precedence)"] FromFlag["--from IMAGE"] PortsFlag["--ports LIST"] ToolsLayersFlags["--tools / --layers"] NoDC["--no-devcontainer"] end subgraph ProjectFile["Project devcontainer.json"] DCImage["image (public registry only)"] DCPorts["forwardPorts (localhost / 127.0.0.1)"] DCFeatures["features (toolchains + docker)"] DCEnvCmds["containerEnv / remoteEnv / commands"] end subgraph Defaults["Host Defaults & Fallbacks"] EnvFrom["$DEVVM_FROM"] UbuntuCloud["Ubuntu 26.04 Cloud Image"] DefaultPorts["settings.Defaults.Ports"] DetectedParts["detectFiles + setup defaults"] end subgraph PlanOut["Resolved VM Plan (newVM)"] BaseOrigin["Base Image Origin"] PortForwards["Forwarded Host Ports"] VMParts["Image Layers & Shared Tools"] end NoDC -. "disables" .-> ProjectFile FromFlag --> BaseOrigin DCImage --> BaseOrigin EnvFrom --> BaseOrigin UbuntuCloud --> BaseOrigin PortsFlag --> PortForwards DCPorts --> PortForwards DefaultPorts --> PortForwards ToolsLayersFlags --> VMParts DCFeatures --> VMParts DetectedParts --> VMParts ``` ### CLI flags and environment variables Starts the `base` image from an OCI or Docker container reference (for example, `node:24`, `alpine:3`, `debian:13`, or `node:24@sha256:...`). Supported on `devvm` (new VMs), `devvm build`, `devvm rebuild`, and `devvm images rm`. Overrides `devcontainer.json` `image` (emitting a `devcontainer_flag_wins` warning) and `DEVVM_FROM`. Skips reading `.devcontainer/devcontainer.json` and `.devcontainer.json` when creating a new VM, and records `"off": true` in the VM state's `devcontainer` section so subsequent launches and `devvm update` ignore any devcontainer file in the project folder. Environment variable (`from.Env`) setting the default `--from` container image reference when neither `--from` nor a `devcontainer.json` `image` property is present. --- ## `devcontainer.json` translation reference `devcontainer.Parse` decodes the top-level JSON object into a `devcontainer.Plan` and classifies every property key via `kind(key)`. Properties that attempt to execute code on the host, mount host paths, or build Dockerfiles are either ignored with a warning or refused outright before any VM or image build starts. ### Property handling matrix | Property | Classification | `devvm` behavior | | --- | --- | --- | | `image` | Mapped (`keyUsed`) | Sets `Plan.Image`. Must resolve anonymously on a public internet registry (`from.Lookup.Public`) and is pinned to `name@sha256:`. Private images, local Docker daemon images, `localhost`, and private/link-local IP addresses are refused with `devcontainer_image_not_public`. | | `features` | Mapped (`keyUsed`) | Maps supported Dev Container feature IDs to `devvm` built-in layers or shared toolchains. Feature options or version strings are ignored with a `devcontainer_feature_options` warning. Scripts from feature registries are never fetched or executed. | | `forwardPorts` | Mapped (`keyUsed`) | Accepts integer ports (`1`–`65535`) or `"localhost:N"` / `"127.0.0.1:N"` strings and appends them to the VM's forwarded port list on `127.0.0.1`. Entries for other hosts or out-of-range ports are ignored with `devcontainer_ports_ignored`. | | `containerEnv`, `remoteEnv` | Mapped (`keyUsed`) | Merged into `Plan.Env` (sorted by variable name) and written as `export NAME=VALUE` lines in the guest environment. `remoteEnv` overrides `containerEnv`. A `null` value in `remoteEnv` unsets the entry from `Plan.Env`, whereas `null` in `containerEnv` is refused (`devcontainer_bad_value`). Variable names must match POSIX `[A-Za-z_][A-Za-z0-9_]*` (`shell.EnvName`). | | `onCreateCommand`, `updateContentCommand`, `postCreateCommand` | Mapped (`keyUsed`) | Appended in order to `Plan.Create`. Executed once inside the VM after initial setup completes. | | `postStartCommand` | Mapped (`keyUsed`) | Stored in `Plan.Start`. Executed inside the VM once per VM boot (`b.Running` boot ID hash). | | `initializeCommand` | Warned (`keyInitialize`) | Never executed because `initializeCommand` targets the host machine. Emits `devcontainer_initialize`. | | `build`, `dockerFile`, `dockerfile`, `dockerComposeFile` | Refused (`keyBuild`) | Fails parsing immediately with `devcontainer_build`. `devvm` only starts VMs from prebuilt images. | | `name`, `customizations`, `$schema` | Silent (`keySilent`) | Ignored silently without warnings. | | `postAttachCommand`, `privileged`, `capAdd`, `securityOpt`, `mounts`, `runArgs`, `workspaceMount`, `workspaceFolder`, `remoteUser`, `containerUser`, `updateRemoteUserUID`, `init`, `shutdownAction`, `overrideCommand`, `hostRequirements`, `appPort`, `portsAttributes`, `otherPortsAttributes`, `userEnvProbe`, `waitFor`, `overrideFeatureInstallOrder`, `context`, `service`, `runServices` | Ignored (`keyIgnored`) | Ignored and reported in a single `devcontainer_ignored` warning line listing the keys. | | Any other property | Unknown (`keyUnknown`) | Ignored and reported in a single `devcontainer_unknown` warning line listing the keys. | ### Feature ID mapping `devcontainer.feature` strips any `:tag` or `@sha256:...` suffix (`featureName`), lowercases the identifier, and requires the prefix `ghcr.io/devcontainers/features/`: | Feature ID (after stripping tag/digest) | Mapped `devvm` target | Kind | | --- | --- | --- | | `ghcr.io/devcontainers/features/docker-in-docker` | `docker` (`image.DockerLayer`) | Built-in image layer | | `ghcr.io/devcontainers/features/node` | `node` | Shared toolchain | | `ghcr.io/devcontainers/features/python` | `python` | Shared toolchain | | `ghcr.io/devcontainers/features/rust` | `rust` | Shared toolchain | | `ghcr.io/devcontainers/features/go` | `go` | Shared toolchain | | `ghcr.io/devcontainers/features/dotnet` | `dotnet` | Shared toolchain | | `ghcr.io/devcontainers/features/docker-outside-of-docker` | Refused (`devcontainer_docker_outside`) | Error (exposes host Docker) | | Any other feature ID or publisher | Refused (`devcontainer_feature`) | Error | ### Variable substitution rules `parser.subst` processes `${...}` expressions across `image`, `forwardPorts`, `containerEnv`, `remoteEnv`, and lifecycle command strings or arrays. Strings containing a NUL byte (`\x00`) are refused with `devcontainer_bad_value`. | Variable syntax | Allowed contexts | Substitution result | | --- | --- | --- | | `${containerWorkspaceFolder}` | All contexts (`modeLiteral`, `modeShell`, `modeEnv`) | Replaced with the project folder path (`workspace`), which is identical on host and guest. | | `${containerWorkspaceFolderBasename}` | All contexts (`modeLiteral`, `modeShell`, `modeEnv`) | Replaced with `path.Base(workspace)`. | | `${containerEnv:VAR}` | String commands (`modeShell`) and `containerEnv` / `remoteEnv` (`modeEnv`) | In `modeShell`, becomes `${VAR}`. In `modeEnv`, surrounding literals are single-quoted via `shell.Quote` and the reference is emitted as `"${VAR}"`. Refused in array commands, `image`, `forwardPorts`, or if `VAR` has a default suffix (`${containerEnv:VAR:default}`). | | `${localEnv:...}`, `${env:...}` | None | Refused in all contexts with `devcontainer_local_env` to prevent leaking host environment variables into the VM. | | `${localWorkspaceFolder}`, `${localWorkspaceFolderBasename}`, `${devcontainerId}` | None | Refused in all contexts with `devcontainer_variable`. | | Unrecognized `${OTHER}` (e.g., `${HOME}`) | All contexts | Left untouched as literal text for the guest shell to evaluate (single-quoted in `modeEnv`). | ### Lifecycle command execution Each lifecycle command property (`onCreateCommand`, `updateContentCommand`, `postCreateCommand`, `postStartCommand`) accepts three JSON shapes: - **String:** Executed via `sh -c ""` (`Command{Name: key, Argv: ["sh", "-c", s]}`). Whitespace-only strings are skipped. - **String array:** Executed directly without a shell wrapper (`Command{Name: key, Argv: argv}`). Empty arrays or arrays with `argv[0] == ""` are skipped. - **Object of named commands:** Keys are sorted lexicographically, and each non-empty entry becomes `Command{Name: key + " " + name, Argv: argv}`. At runtime, `devcontainers.run` executes each command inside the VM in the project directory (`backend.ExecOpts{Dir: dir, Agent: agent}`) using a login shell wrapper: ```sh bash -lc 'exec "$@"' devvm ``` 1. **Create phase (`Plan.Create`):** Before running `onCreateCommand`, `updateContentCommand`, and `postCreateCommand`, `devvm` persists `"made": {"created": true}` to `vm.json` so an interrupted or failed command never re-runs on subsequent launches. 2. **Start phase (`Plan.Start`):** On every launch, `devvm` queries `b.Running(ctx, vm)` for the VM's current boot instance ID, hashes it with SHA-256 (`hex.EncodeToString(sum[:16])`), and compares it against `made.started`. If different, `devvm` updates `made.started` first and runs `postStartCommand`. 3. **Failure handling:** Command stdout and stderr stream to the run log. If a command exits non-zero, `devvm` logs a `devcontainer_command_failed` warning, skips the remaining commands in that phase, and continues opening the VM shell. --- ## Re-reading `devcontainer.json` on `devvm update` Modifying `.devcontainer/devcontainer.json` does not alter an existing VM automatically. On launch, `devcontainers.check` compares the file's current path and SHA-256 against the `devcontainer` section recorded in `~/.local/share/devvm/vms//vm.json` and warns once if the file was added (`devcontainer_new`), modified (`devcontainer_changed`), or deleted. When you run `devvm update`, `devcontainers.reread` reconciles the previous state (`was`) with the newly parsed `Plan` (`now`) while preserving user-chosen overrides: ```go title="internal/devcontainer/reread.go" func Reread(was, gave, has, now []string) (add, drop, gaveNow []string) { for _, p := range gave { switch { case !slices.Contains(has, p): case !slices.Contains(now, p): drop = append(drop, p) default: gaveNow = append(gaveNow, p) } } for _, p := range now { if slices.Contains(was, p) || slices.Contains(has, p) || slices.Contains(add, p) { continue } add = append(add, p) gaveNow = append(gaveNow, p) } return add, drop, gaveNow } ``` - **Toolchains and layers (`rereadParts`):** `gave` tracks only the parts the VM received exclusively from `devcontainer.json` (excluding parts explicitly set via `--tools`/`--layers`, configured in `devvm setup` default layers, or auto-detected from project files by `detectFiles`). `Reread` drops items in `gave` that are no longer in `now`, adds items newly introduced in `now` (that weren't in `was` and aren't already in `has`), and never re-adds a feature the user previously removed or deselected. - **Ports (`filePorts`):** Applies the same `Reread(was.Ports, was.PortsGave, has, listed)` logic via `RemovePorts` and `AddPorts`. Ports added manually with `devvm ports add` remain, and file ports removed with `devvm ports rm` are not re-added. - **Base image (`Image` / `From`):** If the VM was originally created with `--from` (`was.FromFlag == true`), the CLI `--from` image is kept and `devcontainer_flag_wins` is emitted if the file specifies `image`. Otherwise, a changed `image` is pinned via `PublicImage`, or resets to `defaultFrom` (`$DEVVM_FROM` or Ubuntu cloud image) if `image` was removed. - **Remake:** `devvm update` rebuilds the VM from the updated image and tool set, preserving the guest home directory, project folder, and VM settings, and runs the new `Plan.Create` and `Plan.Start` commands on the fresh root disk. --- ## OCI reference resolution and digest pinning `internal/from` parses container references using Docker's rules (`from.ParseRef`) and resolves them to immutable SHA-256 digests so `base` image keys change automatically whenever an upstream tag moves. ### Reference syntax and parsing All references must match `^[A-Za-z0-9][A-Za-z0-9._/:@+-]*$` (`from.Valid`). `from.ParseRef` splits a reference into `Host`, `Repo`, and `Tag`: - If the first `/` segment contains `.` or `:`, or equals `localhost`, it is treated as the registry host (`docker.io` and `index.docker.io` normalize to `registry-1.docker.io`). Otherwise `Host` defaults to `registry-1.docker.io`. - Single-segment repositories on Docker Hub receive the `library/` prefix (`node:24` becomes `Host: "registry-1.docker.io"`, `Repo: "library/node"`, `Tag: "24"`). - Omitting a `:tag` defaults `Tag` to `"latest"`. - References containing `@sha256:<64-hex>` (`from.Pinned`) bypass remote lookup for `--from`. ### CLI `--from` vs. `devcontainer.json` `image` lookup policies `from.Lookup` maintains two separate registry clients and distinct security rules depending on whether you passed `--from` or the project's `devcontainer.json` specified `image`: | Aspect | CLI `--from` / `DEVVM_FROM` (`Lookup.Refresh`) | `devcontainer.json` `image` (`Lookup.Public`) | | --- | --- | --- | | Registry client | `NewRegistry`: HTTPS only, `30s` timeout, `2` retries (`1s` initial backoff). | `NewPublicRegistry`: HTTPS only, plus custom `net.Dialer.Control` (`publicOnly`) that blocks loopback, private, link-local, multicast, CGNAT (`100.64.0.0/10`), and reserved IPv4/IPv6 ranges after DNS resolution (except for a configured company proxy). | | Manifest request | `HEAD /v2//manifests/` with OCI and Docker v2 manifest/index `Accept` headers (does not consume Docker Hub pull rate limits). | Same `HEAD` request; if the reference was already `@sha256:` pinned, verifies the registry's `Docker-Content-Digest` matches. | | Authentication | Anonymous `Bearer` token exchange via `WWW-Authenticate` challenge (`https://` realm only, `1 MiB` response cap). | Anonymous `Bearer` token exchange only. Private images fail with `devcontainer_image_not_public`. | | Local Docker daemon fallback | If the registry returns not-found/unauthorized (`!errors.Is(err, ErrUnreachable)`), queries `docker image inspect --format '{{.Id}}' ` and records the local `sha256:`. Also falls back to `DockerID` if unreachable and no cached digest exists. | **Never** queries the local Docker daemon, preventing an untrusted repo from loading private or local-only host Docker images into its VM. | | Offline / cached fallback | If unreachable or not found and neither registry nor Docker resolves it, reuses the last recorded digest from `~/.local/share/devvm/cache/from-digests.json` (or the newest built `base` image's `record.json`) with a `from_cached_unreachable` or `from_cached_not_found` warning. | If unreachable and unpinned, reuses a previously cached registry digest (`name@sha256:...`) with a `from_cached_unreachable` warning. Never falls back to a local Docker ID. | --- ## Converting OCI images into bootable VM disks Lima VMs boot a virtual disk with a kernel and initramfs rather than running a container share of the host kernel. When building `base` with `--from` (or a `devcontainer.json` `image`), `internal/backend/lima/oci.go` converts the container image inside an isolated, throwaway Lima VM named `devvm-oci-build` booted from the host's pinned Ubuntu 26.04 cloud image. ```mermaid sequenceDiagram autonumber participant Host as devvm Host (lima.images.convert) participant Docker as Host Docker Daemon (optional) participant BuildVM as Throwaway VM (devvm-oci-build) participant Chroot as Image Rootfs Chroot (/var/tmp/devvm-oci/root) Host->>Host: Acquire ~/.local/share/devvm/oci-build.lock Host->>Host: Create temp dir devvm-oci./{vm,oci,guest} alt Digest is local Docker ID (sha256:... without @) Host->>Docker: docker save -o vm/image.tar Host->>Host: Verify manifest.json / index.json matches Digest (exportIs) end Host->>BuildVM: Create & start confined Ubuntu VM (mounts: oci RO, guest RO, vm RW) Host->>BuildVM: Run sudo bash oci/build.sh alt Registry digest (name@sha256:...) BuildVM->>BuildVM: Download & verify pinned crane v0.22.1 BuildVM->>BuildVM: crane export --platform linux/$devvm_arch rootfs.tar else Local Docker export (-) BuildVM->>BuildVM: crane export - rootfs.tar < vm/image.tar end BuildVM->>Chroot: tar -xpf rootfs.tar --numeric-owner BuildVM->>Chroot: Inspect /etc/os-release (gnu vs musl) & getconf GNU_LIBC_VERSION BuildVM->>Chroot: chroot_run sh /tmp/devvm/bootify.sh (install kernel, init, cloud-init, sshd, sudo, user) BuildVM->>BuildVM: Run oci/kernel.sh & mke2fs -t ext4 -d root vm/disk.raw (20G) Host->>BuildVM: Delete devvm-oci-build VM Host->>Host: Validate vm/{disk.raw,kernel,initrd,cmdline,libc,glibc} (checkConverted) ``` ### Step-by-step conversion pipeline `lima.images.convert` acquires an exclusive OS file lock on `~/.local/share/devvm/oci-build.lock` (`lockOCI`, printing `oci_waiting` after 3 seconds if another conversion is running) so only one `devvm-oci-build` VM exists at a time. It creates a temporary directory `devvm-oci.` containing: - `oci/` (mounted read-only): `build.sh` (prepended with `assets/prelude.sh`), `bootify.sh`, and `kernel.sh`. - `guest/` (mounted read-only when custom CAs exist in `~/.config/devvm/ca`): `ca.sh`. - `vm/` (mounted read-write, `0700`): the output directory where the converted artifacts are written. If `f.Digest` is a raw Docker image ID (`sha256:...` without `@`), `exportDocker` checks `docker info`, runs `docker save -o /image.tar `, and inspects `manifest.json` and `index.json` inside the archive (`exportIs`, capped at `16 MiB`) to verify the exported tarball still matches `f.Digest`. If the tag moved during the build, `devvm` aborts with `docker_image_changed`. Inside `devvm-oci-build`, `assets/oci/build.sh` downloads `crane` `v0.22.1` (`go-containerregistry`), verifies its SHA-256 checksum (`898c0cff975f898a33e8c4580bdafb0e7c02c7faa33374e946762f97c4ab7110` on `arm64`, `0ab7a1d6932a213aed964ce97666c3077fe691c8606413674a8b3e0b9ec4cda0` on `amd64`), and runs `crane export --platform linux/$devvm_arch` (or reads `vm/image.tar` for `-`) to flatten the container filesystem into `/var/tmp/devvm-oci/root` with `--numeric-owner`. If the image lacks a manifest for `linux/$devvm_arch`, `build.sh` queries `crane manifest` and fails listing the available architectures. `build.sh` reads `/etc/os-release` inside the extracted rootfs (`ID` and `ID_LIKE`): - `debian` or `ubuntu` maps to `libc=gnu`, and `chroot root getconf GNU_LIBC_VERSION` extracts the `glibc` version (for example, `2.36`). - `alpine` maps to `libc=musl`. - Any other distribution or an image missing `/bin/sh` or `/etc/os-release` is refused. `chroot_run` mounts `/proc`, `/sys`, `/dev`, and `/dev/pts`, copies `/etc/resolv.conf`, installs a temporary `/usr/sbin/policy-rc.d` (`exit 101` on Debian/Ubuntu so `apt-get` does not start daemons in the chroot), optionally installs the user's custom CAs (`ca.sh`), and executes `assets/oci/bootify.sh`: - **Debian / Ubuntu:** Installs `initramfs-tools`, kernel (`linux-image-virtual` + `netplan.io` on Ubuntu; `linux-image-$arch` on Debian because Debian's cloud kernel lacks `9p` and `gpio_keys`), `systemd`, `systemd-sysv`, `systemd-timesyncd`, `systemd-resolved`, `cloud-init`, `openssh-server`, `sudo`, `bash`, and `rsync`. Adds `virtio_pci`, `virtio_mmio`, `virtio_blk`, `virtio_net`, `virtio_console`, and `ext4` to `/etc/initramfs-tools/modules`, regenerates the initramfs, enables `en_US.UTF-8`, disables LLMNR/mDNS, preserves the container's apt sources (`preserve_sources_list: true`), and sets `cmdline` to `root=/dev/vda rw console=hvc0` (plus `net.ifnames=0` on Debian). - **Alpine:** Installs `linux-virt`, `mkinitfs`, `alpine-base`, `openrc`, `cloud-init`, `e2fsprogs`, `openssh`, `sudo`, `bash`, `shadow`, `rsync`, and full `blkid` (required to read Lima's ISO 9660 `cidata` label), enables `isofs` in `/etc/modules`, registers OpenRC `sysinit`/`boot`/`default` services (`sshd`, `cloud-init`), and sets `cmdline` to `root=/dev/vda rootfstype=ext4 rw console=hvc0 modules=virtio_blk,ext4`. - **Guest user:** Removes any existing container user sharing `USER_UID` (such as Ubuntu's `ubuntu` at UID `1000`), creates `USER_NAME` with `USER_UID` and home `/home/.guest`, removes `/etc/ssh/ssh_host_*`, and grants passwordless `sudo` in `/etc/sudoers.d/90-devvm`. `make_disk` removes `/.dockerenv` (which would otherwise cause `systemd` and `cloud-init` to detect a container environment and skip Lima's `cidata` mount), ensures `/dev/console` (`c 5 1`) exists, runs `assets/oci/kernel.sh` (which on `aarch64` decompresses EFI-stub zstd or gzip streams until it locates the uncompressed `ARMd` magic header at offset `0x38` required by macOS `Virtualization.framework`), and formats a `20G` raw ext4 image (`disk.raw`, label `devvm-root`) with `mke2fs -q -t ext4 -L devvm-root -d "$r"`. When Lima creates `base` from this disk, it expands the image to `base`'s `100 GiB` size (`backend.BaseDiskGiB`). ### Post-conversion untrusted output verification Because container packages and post-install scripts execute as `root` inside `devvm-oci-build`, `devvm` deletes `devvm-oci-build` first so nothing can mutate `vm/` during inspection, then runs `checkConverted` on the host before Lima touches the output: | Output file in `vm/` | Host verification rule in `checkConverted` | Security rationale | | --- | --- | --- | | `disk.raw`, `kernel`, `initrd`, `cmdline`, `libc` | `os.Lstat` mode must be `IsRegular()` (no symlinks, FIFOs, or device nodes). | Prevents a symlink from pointing Lima or `devvm` at arbitrary host files or blocking on a host FIFO. | | `disk.raw` | First `1024` bytes must be all `0x00`, followed by the ext4 superblock magic `0x53 0xef` (`0xEF53` little-endian) at byte offset `1080` (`ext4()`). | Rejects `qcow2` or `VMDK` headers in the first 1024 bytes, which could specify a host file as a backing image or extent. | | `cmdline` | Read capped at `4096` bytes; must match `^[A-Za-z0-9=,./:_ -]+$`. | Prevents injection into Lima's YAML configuration. | | `libc` | Must equal `"gnu"` or `"musl"`. | Selects the shared tools target ABI. | | `glibc` (when `libc == "gnu"`) | Checked with `os.Lstat` (`IsRegular()`) before opening, and must pass `image.ValidGlibc` (for example, `2.36`). | Ensures valid glibc version comparison before mounting shared tool EROFS images. | --- ## Shared toolchains, `libc`, and `glibc` compatibility Base images built with `--from` (or `devcontainer.json` `image`) sit alongside the default Ubuntu `base` in `~/.local/share/devvm/images//`. Their image keys hash the `--from` reference, its resolved digest, the conversion scripts (`assets/prelude.sh` + `assets/oci/build.sh`, `bootify.sh`, `kernel.sh`), the guest user metadata (`Username`, `Uid`, `/home/.guest`), and the converted disk size (`20G`). Shared tool images (`--.erofs`) are built and verified according to the `libc` and `glibc` fields recorded in `base`'s `record.json` (`build.Env.ToolTarget`): - **Debian and Ubuntu container bases (`libc: "gnu"`):** VMs share the exact same `gnu` EROFS tool images as VMs on `devvm`'s default Ubuntu cloud image. To prevent untrusted container code from tampering with shared tool builds mounted by other VMs, `gnu` tool images are always built inside `devvm`'s own Ubuntu cloud `base` image (`tools_need_own_base`), never inside the `--from` container image. During tool image creation, `build.sh` runs `readelf` across all ELF binaries and libraries in the tool (skipping `testdata` directories) to record the highest required `GLIBC_` symbol version in `.glibc`. Before attaching a tool to a `--from` VM (on `devvm`, `devvm tools add`, `devvm tools update`, or `devvm update`), `devvm` compares the tool's required glibc version against the container base's `glibc` version and refuses incompatible tools with `tool_needs_glibc`. - **Alpine container bases (`libc: "musl"`):** Alpine VMs require `musl` tool builds. `devvm` builds `musl` tool images inside the VM's own `--from` Alpine `base` image and includes that `base` image's `Digest` in the tool image key (`tg.Base = rec.Digest`), isolating those tool images to VMs sharing that exact Alpine image digest. --- ## Related pages Understand deterministic SHA-256 key computation for base images, built-in and custom layers, and read-only EROFS shared tool images. Add, remove, and update shared mise toolchains and author custom image layers with network policies. Inspect how state sections like devcontainer and ports persist in vm.json and how in-place vs. remake updates work. Full flag and argument reference for devvm (up), devvm update, and devvm ports. --- ## 12. Connect VMs with links, groups, and port forwards > Forward guest ports to host 127.0.0.1, publish ports and link VMs under *.devvm hostnames, and join VMs into mutually reachable groups. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/12-connect-vms-with-links-groups-and-port-forwards.md - Generated: 2026-11-07T22:20:40.413Z ### Source Files - `internal/cli/ports.go` - `internal/cli/link.go` - `internal/cli/group.go` - `internal/ports/ports.go` - `internal/vm/links.go` - `internal/vm/groups.go` - `internal/vmssh/forwards.go` - `internal/proxy/links.go` --- title: "Connect VMs with links, groups, and port forwards" description: "Forward guest ports to host 127.0.0.1, publish ports and link VMs under *.devvm hostnames, and join VMs into mutually reachable groups." --- `devvm` isolates each project VM's network stack while providing two independent SSH-backed transport mechanisms managed from the host: **host port forwarding** (`devvm ports` and `devvm _ports`) exposes guest TCP listeners on host loopback (`127.0.0.1` and `::1`), and **inter-VM links and groups** (`devvm link`, `devvm group`, and `devvm _links`) route TCP connections from a client VM to published ports on target VMs over host UNIX sockets without bridging guest networks or weakening network tier confinement. ## Transport architecture and isolation boundaries Both port forwarding and inter-VM links run as detached host processes (`devvm _ports` and `devvm _links`) that authenticate to guest `sshd` instances over host loopback using per-VM SSH keys and start-pinned `known_hosts` records (`~/.local/share/devvm/vms//run/known_hosts`). Neither mechanism uses Lima's built-in port forwarder or alters guest firewall confinement. ```mermaid flowchart LR subgraph Host["Host Machine (127.0.0.1 & ~/.local/share/devvm)"] Browser["Host Client
127.0.0.1:5173 / [::1]:5173"] FwdProc["devvm _ports
(vmssh.Forwarder)"] LinkProc["devvm _links
(vmssh.Links)"] ProxyProc["devvm _proxy
(proxy.links)"] LinkSock[("link-.sock
0600 UNIX socket")] LinkList[("links.json
proxy.LinkList v1")] end subgraph ClientVM["Client VM (e.g. devvm-app-5e1d7c08)"] ClientApp["Guest Process"] EtcHosts["/etc/hosts
127.77.0.1 db db.devvm.internal"] ClientSSHD["Guest sshd
PermitListen 127.77.*:*"] end subgraph TargetVM["Target VM (e.g. devvm-db-71a99b10)"] TargetSSHD["Target sshd"] TargetSvc["Target TCP Server
127.0.0.1:5432"] end Browser -->|"TCP 127.0.0.1:5173"| FwdProc FwdProc -->|"ssh -W 127.0.0.1:5173
(port-.ctl)"| ClientSSHD ClientSSHD -.->|"polls /proc/net/tcp{,6}
(assets/guest/ports.sh)"| FwdProc ClientApp -->|"Direct TCP to 127.77.0.1:5432"| ClientSSHD ClientApp -.->|"Resolves db / db.devvm.internal"| EtcHosts ClientSSHD -->|"ssh -R 127.77.0.1:5432:sock"| LinkSock ClientApp -->|"HTTP / CONNECT db:5432"| ProxyProc ProxyProc -.->|"Watches"| LinkList LinkProc -->|"Writes"| LinkList ProxyProc -->|"Dials UNIX socket"| LinkSock LinkSock --> LinkProc LinkProc -->|"CheckLink + policyEnforced
ssh -W 127.0.0.1:5432"| TargetSSHD TargetSSHD -->|"127.0.0.1:5432"| TargetSvc ``` | Capability | Host Port Forwarding (`devvm ports`) | Inter-VM Links (`devvm link`) | VM Groups (`devvm group`) | | --- | --- | --- | --- | | **Direction** | Host `127.0.0.1` / `::1` $\rightarrow$ Guest loopback | Client VM $\rightarrow$ Host socket $\rightarrow$ Target VM `127.0.0.1` | Full mesh of links across group members' published ports | | **Worker daemon** | `devvm _ports -vm ...` | `devvm _links ` | Uses `devvm _links ` per member | | **Guest address space** | `127.0.0.1`, `::1`, or `0.0.0.0`/`::` (excluding `127.77.0.0/16` and `127.0.77.1`) | `127.77.0.0/16` (`127.77.0.1` .. `127.77.255.254`), one IPv4 per target VM | Same as links (`Link.Group = true`) | | **Hostnames in guest** | `localhost` (on host) | `:` and `.devvm.internal:` | `:` and `.devvm.internal:` | | **Default behavior** | `all` (every TCP port $\ge 1024$) | None; requires `devvm link publish` on target and `devvm link add` on client | None; requires `devvm group join` and `devvm link publish` | | **Protocol support** | TCP (IPv4 and IPv6 on host) | TCP over IPv4 (plus HTTP/CONNECT via `devvm _proxy`) | TCP over IPv4 (plus HTTP/CONNECT via `devvm _proxy`) | --- ## Host port forwarding A project VM forwards listening guest TCP ports to the identical port number on the host's loopback interfaces (`127.0.0.1` and `::1`). Port forwarding is active on all network tiers (`allowlist`, `open`, `internet`, and `none`) and is disabled for image build VMs. ### Configuring forwarded ports You can set a VM's port forwarding policy at creation time with `devvm --ports `, or inspect and modify it afterward with `devvm ports`. ```bash title="Inspect and replace forwarded ports" # Show what the current folder's VM forwards (all, none, or a range list) devvm ports # Forward only specific ports and ranges (including privileged ports < 1024) devvm ports 5173,8000-8010 # Stop forwarding all ports devvm ports none # Restore default forwarding of all guest ports >= 1024 devvm ports all ``` ```bash title="Incrementally add or remove ports" # Add a single port or range to the VM's list devvm ports add 5432 devvm ports add 8000-8010 # Remove a listed port or range (must match an entry in the list) devvm ports rm 5173 ``` ```bash title="Configure global defaults for new VMs" # Show the default for new VMs and its configuration source devvm ports -g # Set, add, or remove default ports for new VMs in settings.json devvm ports -g none devvm ports -g 3000,5173 devvm ports add -g 8080 devvm ports rm -g 8080 ``` ### Precedence and `devcontainer.json` synchronization 1. **Creation precedence:** When creating a VM, `--ports` overrides `ports` in `~/.config/devvm/config.toml`, which in turn overrides the value saved in `~/.local/share/devvm/settings.json` by `devvm ports -g`. When none is set, the default is `all`. When `config.toml` sets `ports`, `devvm ports -g` refuses to modify the global default and points to `config.toml`. 2. **`devcontainer.json` (`forwardPorts`):** In `internal/vm/ports.go` (`filePorts`), `devvm` tracks which ports came from `.devcontainer/devcontainer.json` (`ports` and `ports_gave` in the `devcontainer` state section). Running `devvm update` removes ports dropped from `forwardPorts` and adds newly listed ports while preserving any ports added or removed manually with `devvm ports`. 3. **Live updates:** On a stopped VM, `devvm ports` updates `vm.json` (`ports.forward`) for the next boot. On a running VM, `devvm` updates `vm.json` and immediately restarts `devvm _ports` (`vmssh.Forwards.Start`), closing connections open through the previous forwarder process. ### How `devvm _ports` and `ports.sh` work `devvm` never uses Lima's native port forwarding because Lima binds with `SO_REUSEADDR`, which can intercept `localhost` traffic from existing host listeners. Instead, `vmssh.Forwards` spawns `devvm _ports` as a detached daemon per running VM: 1. **Listener discovery (`assets/guest/ports.sh`):** `devvm _ports` opens a control-master SSH session (`~/.local/share/devvm/run/port-.ctl`) and runs `ports.sh` inside the VM. Every second, `ports.sh` reads `/proc/net/tcp` and `/proc/net/tcp6` for sockets in state `0A` (`TCP_LISTEN`), emitting changed hex `ADDRESS:PORT` pairs (or a heartbeat every 10 seconds) terminated by `.`. 2. **Address filtering (`vmssh.dialAddr`):** - Guest listeners bound to `0.0.0.0` (`IPv4Unspecified`) or `127.0.0.1` are forwarded to `127.0.0.1:` in the VM. - Guest listeners bound to `::` (`IPv6Unspecified`) or `::1` are forwarded to `[::1]:` in the VM. - Listeners on other `127.0.0.0/8` loopback addresses are forwarded to that specific loopback address, **except**: - `127.77.0.0/16` (`proxy.IsLinkAddr`): reserved for inter-VM links so links are never exposed on the host's `localhost`. - `127.0.77.1` (`proxy.IsLocalProxyAddr`, port `3128`): reserved for `devvm-local-proxy`, which injects the VM's proxy password for guest browsers and must never be reachable from the host. - Listeners bound exclusively to non-loopback guest network interfaces are ignored. 3. **Port allocation and collision avoidance (`internal/ports/ports.go`):** - `ports.Allocator` allocates host ports for each VM's SSH, host proxy, and VNC desktop under the global claims lock (`state.VM.Claim`), storing them in `vm.json` under `ports.host` (`ssh`, `proxy`, `desktop`). During image or tool builds, temporary SSH ports are recorded and locked in `~/.local/share/devvm/build-ports/` (`staleAfter = 15m`). - `ports.Used(dirs, self)` collects every VM's `ssh`, `proxy`, and `desktop` host ports, active build ports, and all explicitly listed `ports.forward` ranges from other VMs. - `devvm _ports` binds `127.0.0.1:` and `[::1]:` using `platform.ListenLoopback` (without `SO_REUSEADDR`). If a host process already holds a port on any address, `devvm _ports` skips it; if the port was explicitly listed on the VM, `devvm` emits a `ports_taken` warning and continues running the VM. 4. **Operational limits (`internal/vmssh/forward.go`):** | Constant | Value | Behavior | | --- | --- | --- | | `minForwardPort` | `1024` | Minimum guest port forwarded when mode is `all`. Explicitly listed ports may be `1`–`65535` (including port `22`). | | `maxForwards` | `256` | Maximum concurrent ports forwarded per VM. | | `maxListeners` | `4096` | Maximum listening socket lines accepted in a single `ports.sh` report. | | `maxRelays` | `256` | Maximum simultaneous active forwarded TCP connections (`ssh -W`) per VM. | | `logMax` | `1 MiB` | Maximum size of `~/.local/share/devvm/vms//logs/ports.log` before truncation on forwarder start; runtime writes are rate-limited (`logBurst = 200`, `1` token per `10s`, `512` bytes/write). | **UDP traffic is not forwarded.** To tunnel low-rate UDP request/response traffic (such as DNS, StatsD, or syslog) over a forwarded TCP port, run `socat` on both sides: ```bash # Inside the VM (for a UDP server listening on 127.0.0.1:8125, with TCP 9125 forwarded) socat TCP-LISTEN:9125,fork,reuseaddr UDP:127.0.0.1:8125 # On the host socat UDP-LISTEN:8125,fork TCP:127.0.0.1:9125 ``` --- ## Inter-VM links (`devvm link`) An inter-VM link connects a **client VM** to a specific TCP port on a **target VM**. The target VM must explicitly offer the port with `devvm link publish`, and the client VM must add the link with `devvm link add`. **Links join the trust boundaries of connected VMs.** A service exposed over a link (such as PostgreSQL, Redis, or an unauthenticated HTTP API) can often execute commands or access data in the target VM, giving the client VM indirect access to the target VM's filesystem, allowed network hosts, and bound secrets. Only link VMs that you trust to share capabilities. ### Publishing and unpublishing target ports Run `devvm link publish` in the target project's folder to expose ports or port ranges to other VMs: ```bash title="Publish and unpublish ports in the target VM folder" # Show currently published ports (prints "none" or a range list like "5432,8000-8010") devvm link publish # Publish port 5432 and range 8000-8010 devvm link publish 5432,8000-8010 # Withdraw port 5432 from links (the server inside the VM keeps running) devvm link unpublish 5432 ``` - `devvm link publish` accepts individual ports (`1`–`65535`) and ranges (`from-to`), never `all` or `none`. - Publishing a port writes to `links.publish` in the target VM's `vm.json`. It does not forward the port to the host machine. - Target services must listen on `127.0.0.1` or `0.0.0.0` (all addresses) inside the target VM; `devvm _links` dials `127.0.0.1:` in the target, so servers bound exclusively to `::1` are not reached. - When you unpublish a port that existing client VMs link to, `devvm` prints a note naming those client VMs (`unpublished_linked`). Existing open connections stay up until closed, and subsequent connection attempts are refused with reason `unpublished`. ### Adding, listing, and removing links Run `devvm link add TARGET PORT` in the client project's folder: ```bash title="Manage links in the client VM folder" # Link to port 5432 of the VM for folder "db" (reachable at db:5432 and db.devvm.internal:5432) devvm link add db 5432 # Override the hostname in the client VM with --as devvm link add db 5432 --as pg # Reference the target by relative/absolute folder path, full VM name, or SSH alias devvm link add ../api 8080 devvm link add devvm-api-71a99b10 8080 devvm link add devvm-api 8080 # List links and their current applicability status devvm link # Remove a link by link name, target folder, or target VM name devvm link rm db 5432 devvm link rm pg 5432 ``` #### Target resolution and naming rules - **Target resolution (`Links.resolve`):** `TARGET` can be: 1. A path starting with `.` or containing `/` (resolved relative to the working directory and symlink-evaluated via `filepath.EvalSymlinks`). 2. A full VM name (`devvm--`). 3. An SSH alias (`devvm-`). 4. A project folder name (``). If multiple VMs share the same folder name, `devvm` fails with `link_ambiguous_target` and lists every matching ` in `. - **Single name per target VM (`linkNameFor`):** A client VM assigns a single hostname to a given target VM (`Target` + `TargetID`). If you add `devvm link add db 5432 --as pg`, a subsequent `devvm link add db 8000` automatically uses `pg` (`pg:8000`); passing a different `--as` flag for the same target fails with `link_target_named`. - **Hostname validation (`CheckLinkName`):** Link names must match `^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$`, cannot be all digits (`strings.Trim(name, "0123456789") != ""`), and cannot be `localhost`, the client VM's own hostname slug, or a name already used by a link to another target VM. - **Target identity pinning (`TargetID`):** Each `Link` stores the target VM's `state.VM.ID` at creation time. In-place remakes (`devvm layer add`, `devvm layer rm`, `devvm update`, `devvm clone`) preserve `state.VM.ID`, so links continue working. Deleting a VM (`devvm rm`) and creating a new VM in the same folder generates a new `ID`, marking any stale link as `replaced`. ### Guest addressing, `/etc/hosts`, and dual ingress When a client VM has links, `devvm` configures two ways for guest processes to reach `:` and `.devvm.internal:`: 1. **Direct TCP via `127.77.0.0/16` (`vmssh.Links`):** - `linkAddr` assigns each distinct target VM in a client its own IPv4 loopback address in `127.77.0.0/16`, starting at `127.77.0.1` (`proxy.LinkAddr`). Multiple ports on the same target share that target's `127.77.x.y` address; two different targets can publish the same port number because their `127.77.x.y` addresses differ. - On launch and whenever links change on a running VM, `syncer.sync` streams `/etc/hosts` entries to `assets/guest/sync.sh`, which updates the block between `# devvm links begin` and `# devvm links end` in place: ```text # devvm links begin 127.77.0.1 db db.devvm.internal 127.77.0.2 api api.devvm.internal # devvm links end ``` - On Alpine (`musl`) guests where `/proc/sys/net/ipv4/ip_unprivileged_port_start` defaults to `1024`, `sync.sh` writes `0` so guest `sshd` (running as the unprivileged VM user) can bind remote forwards on ports below `1024` (such as port `80` or `443`). - Every image configures `sshd` with `AllowTcpForwarding all`, `GatewayPorts clientspecified`, and `PermitListen 127.77.*:*`. `devvm _links` runs `ssh -O forward -R 127.77.x.y:PORT:/path/to/link-.sock` on its dedicated master connection. 2. **HTTP and HTTPS `CONNECT` via `devvm _proxy` (`internal/proxy/links.go`):** - Link names are intentionally excluded from `NO_PROXY` inside the guest (`DEVVM_NO_PROXY` only covers loopback and local addresses). HTTP clients that honor `HTTP_PROXY` / `HTTPS_PROXY` (such as `curl` or `fetch`) send `CONNECT db:5432` or `GET http://api:8080/` to the host proxy (`devvm _proxy`). - `devvm _links` atomically writes `~/.local/share/devvm/vms//run/links.json` (`proxy.LinkList`, `version: 1`) containing each link's `hosts` (`["db", "db.devvm.internal"]`), `addr`, `port`, `target`, and expected host socket path. - Before evaluating any network policy rule or injecting secrets, `proxy.links.find` checks `links.json` (re-reading on file change) and verifies that `e.Socket == proxy.LinkSocket(k.sockets, k.vm, addr, port)`. Matching requests are spliced (`CONNECT`) or reverse-proxied (`HTTP`) directly to `link-.sock` with `Proxy-Authorization` and `Proxy-Connection` stripped and no TLS interception. - Any request to `*.devvm.internal` (or `devvm.internal`) that does not match an active link of the client VM is immediately rejected by `proxy.links.refuse` with `403 Forbidden` (`not one of this VM's links`) so it never leaks to an upstream DNS resolver or corporate proxy. **Docker containers inside a client VM:** Containers started via the built-in `docker` layer use an isolated bridge network by default, which does not inherit the VM's `127.77.0.0/16` loopback listeners or `/etc/hosts` block. Run containers with `--network host` (`docker run --network host ...`) so they share the VM's network namespace and reach links directly. ### Network tier rules and per-connection verification Links enforce a strict monotonic network-tier rule (`internal/vm/links.go`, `policyRule`): **a client VM can only link to a target VM on the same network tier or a stricter one** (`target.tier.AtLeast(client.tier)`). | Client VM Tier | Allowed Target VM Tiers | Refused Target VM Tiers | | --- | --- | --- | | `none` (strictest) | `none` | `allowlist`, `internet`, `open` | | `allowlist` | `none`, `allowlist` | `internet`, `open` | | `internet` | `none`, `allowlist`, `internet` | `open` | | `open` (least strict) | `none`, `allowlist`, `internet`, `open` | — | - **State mutation guards (`tierLinked`):** While an active link exists, `devvm network` refuses to lower a target VM's tier below any linked client's tier (`tier_linked_clients`) and refuses to raise a client VM's tier above any linked target's tier (`tier_linked_targets`). - **Per-connection enforcement (`vm.Links.linkConn`):** `devvm _links` re-verifies every accepted connection on `link-.sock` before opening an SSH channel to the target: ```mermaid sequenceDiagram autonumber participant App as Client Guest Process participant Sock as devvm _links (link-.sock) participant State as State & Policy Check (vm.Links.linkConn) participant Backend as Target Backend (targetRuns 1s cache) participant Target as Target VM sshd (127.0.0.1:PORT) App->>Sock: Connect via 127.77.x.y:PORT or devvm _proxy Note over Sock: Batch accept loop (5ms settle / 50ms max / 256 cap) alt Active connections >= 256 (maxLinkConns) Sock-->>App: Close connection (logs event "link_limit") else Slot available Sock->>State: Check(ctx, link) State->>State: Read client & target vm.json
Verify link exists, TargetID matches,
Published(port), and linkPolicy(client, target) alt CheckLink fails (removed, gone, replaced, unpublished, policy, ungrouped) State-->>Sock: Event "link_refused" + reason Sock-->>App: Close connection else State valid State->>Backend: targetRuns.get(target + ID) [reused for 1s] Backend-->>State: Running status, start ID, pinned SSH config alt Target not running State-->>Sock: Event "link_target_stopped" Sock-->>App: Close connection (never auto-starts target) else Target running State->>Backend: policyEnforced(client, target) alt Client confined & target tier "unenforced" State-->>Sock: Event "link_refused" (reason: "unenforced") Sock-->>App: Close connection else Tier enforced State-->>Sock: LinkTarget{SSH, Start} Sock->>Target: Ensure targetMaster (.ctl) for Start ID
Relay via ssh -W 127.0.0.1:PORT Sock->>State: Log event "link" Target-->>App: Bidirectional TCP stream end end end end ``` ### Link applicability states and event log reasons `devvm link` checks each link against current VM states (`CheckLink`), and `devvm network log` records every connection attempt from `devvm _links` (`~/.local/share/devvm/vms//logs/events.jsonl`): | Status / Reason Code | Displayed in `devvm link` (`APPLIES`) | Logged in `devvm network log` (`event` / `reason`) | Meaning | | --- | --- | --- | --- | | `yes` / `""` | `yes` | `event: "link"` | Link is valid and connection was carried to target. | | `gone` | `gone` | `event: "link_refused"`, `reason: "gone"` | Target VM state no longer exists (`devvm gc --orphans` cleans these up). | | `replaced` | `replaced` | `event: "link_refused"`, `reason: "replaced"` | Target VM was recreated with a new `state.VM.ID`. | | `unpublished` | `unpublished` | `event: "link_refused"`, `reason: "unpublished"` | Target VM no longer publishes the port. | | `policy` | `policy` | `event: "link_refused"`, `reason: "policy"` | Target VM's network tier is less strict than the client VM's tier. | | `ungrouped` | `ungrouped` | `event: "link_refused"`, `reason: "ungrouped"` | Link was created by a group (`Link.Group == true`), and the VMs are no longer in the same group. | | `unenforced` | `yes` (warns on add/launch) | `event: "link_refused"`, `reason: "unenforced"` | Client is on a confined tier, and the running target switched to a confined tier without restarting (`devvm ls` shows `unenforced`) or failed its proxy self-check. Restart the target VM with `devvm restart`. | | `removed` | *(not listed)* | `event: "link_refused"`, `reason: "removed"` | Link was removed from client state before the connection was checked. | | `not carried` | *(any)* | `event: "link_refused"`, `reason: "not carried"` | Logged by `devvm _proxy` when `link-.sock` does not answer (e.g., `devvm _links` stopped). | | `ssh` | `yes` | `event: "link_refused"`, `reason: "ssh"` | Target VM's SSH endpoint or pinned host key could not be verified. | | *(target stopped)* | `yes` (`STATUS: Stopped`) | `event: "link_target_stopped"` | Target VM is not running (`devvm` never auto-starts a target VM for a link). | | *(concurrency cap)* | `yes` | `event: "link_limit"`, `reason: "256"` | Client VM already has `256` (`maxLinkConns`) active link connections open. | --- ## Mutually reachable VM groups (`devvm group`) A group connects multiple project VMs so that every member automatically gets links to every port published by every other member in the group (`internal/vm/groups.go`). A VM can belong to at most one group at a time. In each project folder that runs a service, publish its TCP ports: ```bash cd ~/code/db && devvm link publish 5432 cd ~/code/api && devvm link publish 8080 ``` Join each VM to the same group name (`^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$`, not all digits). The first VM to join creates the group: ```bash cd ~/code/db && devvm group join web cd ~/code/api && devvm group join web cd ~/code/app && devvm group join web ``` To override the hostname that other group members use to reach a VM (for example, when two project folders share the same basename), pass `--as`: ```bash cd ~/work/api && devvm group join web --as api1 cd ~/old/api && devvm group join web --as api2 ``` Inspect the group table with `devvm group` and verify the synthesized links with `devvm link`: ```bash devvm group ``` ```text NAME VM FOLDER PUBLISHED api devvm-api-0f3c2a91b7d4e6 /Users/you/code/api 8080 app devvm-app-5e1d7c08a2b9f4 /Users/you/code/app none db devvm-db-71a99b1076d92c /Users/you/code/db 5432 ``` ### Group invariants, locking, and link precedence - **Global group lock (`paths.Dirs.GroupsLock`):** Every `devvm group join`, `devvm group leave`, `devvm rm` (via `Groups.forget`), and `devvm link publish`/`unpublish` (via `Groups.syncOthers`) acquires an OS file lock on `~/.local/share/devvm/groups.lock` (`lockGroups`). If another `devvm` process holds the lock for more than `200ms` (`groupsNote`), `devvm` prints a waiting notice and re-reads VM state once acquired. - **Uniform network tier requirement (`checkJoin`):** Because group links are bidirectional (`policyRule(self, other) && policyRule(other, self)`), **all VMs in a group must be on the exact same network tier**. `devvm group join` fails with `group_tier_mismatch` if the joining VM's tier differs from existing members, and `devvm network` refuses to change a VM's tier while it is in a group (` Leave` first). - **Hostname collision rules (`checkJoinName`):** - A member's group name defaults to its folder slug unless `--as NAME` is passed (`links.group_as` in `vm.json`). - `devvm group join` refuses `localhost`, refuses a name that matches another member's group name or folder slug (`group_name_taken`), and refuses a join if an existing member's group name matches the joining VM's own hostname slug (`group_name_hostname`), which would shadow the joining VM's own hostname in `/etc/hosts`. - **Precedence between manual links (`ownLinks`) and group links (`groupLinks`):** - Links created via `devvm link add` have `Group: false`; links synthesized by a group have `Group: true`. - If a client VM already has a manual link to a group member, all group-generated links to that member adopt the manual link's `Name` and `127.77.x.y` address, and any port already linked manually is not duplicated. - If a manual link in a client VM uses the same `Name` for a *different* target VM than a group member (`groupClash`), the manual link wins: the client gets no group links to that colliding member, and `devvm` prints a `group_link_name_clash` warning. - Removing a manual link with `devvm link rm` automatically restores the group's link for that member and port (`regained`). Attempting to run `devvm link rm` on an active group-generated link fails with `link_from_group` (use `devvm group leave` or `devvm link unpublish` instead). - **Leaving a group (`devvm group leave`):** Clears `links.group` and `links.group_as` on the VM, removes all `Group: true` links from the leaving VM and to the leaving VM across all remaining group members, and preserves manual `devvm link add` links. When the last member leaves (or is deleted with `devvm rm`), the group ceases to exist. --- ## State files, runtime sockets, and `--json` schemas ### On-disk layout `devvm` stores port, link, and group state inside `~/.local/share/devvm`: :::files ~/.local/share/devvm/ ├── groups.lock # Global file lock serializing group joins, leaves, and syncs ├── build-ports/ │ └── # Locked file holding temporary SSH port during image/tool builds ├── run/ │ ├── port-.ctl # SSH control socket for devvm _ports │ ├── link-.ctl # SSH control sockets for devvm _links (session & target masters) │ └── link-.sock # Mode 0600 UNIX domain socket per active client link (addr + port) └── vms// ├── vm.json # Persistent VM state ("ports" and "links" sections) ├── logs/ │ ├── ports.log # devvm _ports output log (capped at 1 MiB + token bucket) │ ├── links.log # devvm _links output log (ssh lines prefixed with "ssh: ") │ └── events.jsonl # Proxy and link connection events read by devvm network log └── run/ ├── known_hosts # Pinned SSH host keys for the VM's current boot ├── ports.pid # PID file for devvm _ports ├── ports.status # Readiness status file for devvm _ports ├── ports.lock # Per-VM lock serializing forwarder stop/start ├── links.pid # PID file for devvm _links └── links.json # proxy.LinkList (version 1) watched by devvm _proxy ::: ### `vm.json` state sections and CLI `--json` output In `~/.local/share/devvm/vms//vm.json`, `ports` (`internal/ports/ports.go`) and `links` (`internal/vm/state.go`) are stored as top-level sections: ```json title="~/.local/share/devvm/vms//vm.json (excerpt)" { "ports": { "forward": "5173,8000-8010", "host": { "ssh": 52104, "proxy": 52105, "desktop": 52106 } }, "links": { "publish": "5432", "group": "web", "group_as": "db1", "links": [ { "name": "api", "port": 8080, "target": "devvm-api-0f3c2a91b7d4e6", "target_id": "9f8e7d6c5b4a3210", "addr": "127.77.0.1", "group": true } ] } } ``` `devvm ports --json` and `devvm ports -g --json` return version 1 JSON payloads (`internal/cli/ports.go`): ```json title="devvm ports --json" { "version": 1, "ports": "5173,8000-8010" } ``` ```json title="devvm ports -g --json" { "version": 1, "ports": "all", "source": "built-in" } ``` `source` is one of `"config.toml"`, `"settings.json"`, or `"built-in"`. `devvm link --json` and `devvm link publish --json` return version 1 JSON payloads (`internal/cli/link.go`): ```json title="devvm link --json" { "version": 1, "links": [ { "name": "db", "port": 5432, "target": "devvm-db-71a99b1076d92c", "target_folder": "/Users/you/code/db", "target_status": "Running", "applies": true, "reason": "" } ] } ``` ```json title="devvm link publish --json" { "version": 1, "ports": "5432,8000-8010" } ``` `devvm group --json` returns the group name (`""` if not in a group) and sorted member array (`internal/cli/group.go`): ```json title="devvm group --json" { "version": 1, "group": "web", "members": [ { "name": "api", "vm": "devvm-api-0f3c2a91b7d4e6", "folder": "/Users/you/code/api", "published": "8080" }, { "name": "db", "vm": "devvm-db-71a99b1076d92c", "folder": "/Users/you/code/db", "published": "5432" } ] } ``` --- ## Troubleshooting If a process inside the client VM is already bound to `0.0.0.0:` with an exclusive socket option that prevents `sshd` from binding `127.77.x.y:`, `ssh -O forward -R` fails for that specific link. `devvm _links` logs `:: refused by the VM` in `links.log` and `devvm` warns with `link_port_taken` while keeping all other links active. Stop the conflicting listener inside the client VM (or bind it to `127.0.0.1:` instead of `0.0.0.0:`) and run `devvm restart`. When a listed port is already bound by a host program (on `127.0.0.1`, `::1`, `0.0.0.0`, or `::`) or reserved in `ports.Used` by another VM, `devvm` warns (`ports_taken`) and starts the forwarder for the remaining ports. In `all` mode, ports in use on the host are silently skipped until the guest listener restarts after the host port becomes free. Inspect `~/.local/share/devvm/vms//logs/ports.log` to see per-port bind results. When a client VM is on a confined tier (`allowlist`, `internet`, or `none`) and the target VM's tier was changed from `open` to a confined tier while the target VM was already running, the target VM is not yet running behind its confined proxy (`devvm ls` shows `unenforced`). `devvm _links` refuses connections from confined clients with reason `unenforced` until you run `devvm restart` in the target VM's folder. If a target VM's state directory was removed outside `devvm rm` or a `devvm _links` process was killed with `SIGKILL`, run `devvm gc --orphans`. This removes dead links (`DeadLink` entries pointing to non-existent target states) and deletes stale `link-*.sock` and `link-*.ctl` sockets in `~/.local/share/devvm/run/` that no longer accept connections (`vm.LeftoverSockets`). ## Related pages Understand traffic confinement across allowlist, internet, open, and none tiers and how devvm _proxy inspects requests and logs events. Complete command syntax, flags, and examples for devvm link, devvm group, and devvm network. Syntax and flags for devvm ports, devvm size, devvm ls, and devvm restart. Default ports settings in config.toml and settings.json, state file layout, and version 1 --json schemas. --- ## 13. Connect SSH, VS Code, and VNC desktop > Use generated ~/.ssh/config entries, configure per-VM SSH agents, launch VS Code Remote-SSH via devvm code, and connect to the VNC desktop relay. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/13-connect-ssh-vs-code-and-vnc-desktop.md - Generated: 2026-11-07T22:24:13.098Z ### Source Files - `internal/cli/code.go` - `internal/cli/desktop.go` - `internal/cli/sshagent.go` - `internal/sshconfig/sshconfig.go` - `internal/sshagent/sshagent.go` - `internal/desktop/desktop.go` - `internal/desktop/relay.go` --- title: "Connect SSH, VS Code, and VNC desktop" description: "Use generated ~/.ssh/config entries, configure per-VM SSH agents, launch VS Code Remote-SSH via devvm code, and connect to the VNC desktop relay." --- `devvm` exposes running project VMs to host SSH clients, editors, and VNC viewers over host loopback (`127.0.0.1`) while enforcing host-key pinning, per-VM SSH agent isolation, and host-side VNC authentication. On every VM launch or stop, `internal/sshconfig` rewrites `~/.local/share/devvm/ssh_config` with hardened `Host` blocks bound to the VM's current boot session. `devvm host` and `devvm code` bring the target VM up, synchronize guest configuration, ensure `~/.ssh/config` includes the generated file, and connect `ssh` or VS Code Remote-SSH. For graphical workflows in VMs built with the `desktop` layer, `devvm desktop` starts an in-VM Xfce and TigerVNC session on a Unix domain socket (`/run/devvm-desktop/vnc.sock`) and runs a host-side RFB relay (`internal/desktop.Relay`) that verifies an 8-character per-VM password and filters client clipboard events before tunneling over `ssh -W`. ```mermaid flowchart LR subgraph Host["Host Machine (127.0.0.1)"] SSHClient["ssh / VS Code Remote-SSH"] SSHConfig["~/.local/share/devvm/ssh_config"] KnownHosts["vms/<hash>/run/known_hosts"] UserAgent["User SSH_AUTH_SOCK"] DevvmAgent["devvm Agent
~/.local/share/devvm/agent.sock"] VNCViewer["Screen Sharing / VNC Viewer"] Relay["desktop.Relay
127.0.0.1:<port>
DES Auth + RFB Filter"] end subgraph Guest["Project VM"] SSHD["sshd"] VSCServer["~/.vscode-server/data/Machine/settings.json"] VNCUnix["Xvnc (:1, 1600x1000)
/run/devvm-desktop/vnc.sock"] LocalProxy["devvm-local-proxy
127.0.77.1:3128"] end SSHClient -->|reads| SSHConfig SSHConfig -->|verifies| KnownHosts SSHConfig -.->|mode: on| UserAgent SSHConfig -.->|mode: devvm| DevvmAgent SSHClient -->|127.0.0.1:<ssh_port>| SSHD SSHD --> VSCServer VNCViewer -->|RFB 3.3 / 3.7 / 3.8| Relay Relay -->|ssh -W /run/devvm-desktop/vnc.sock| SSHD SSHD -->|Unix domain socket| VNCUnix VNCUnix -->|HTTP_PROXY| LocalProxy ``` ## SSH access and generated `ssh_config` `devvm` generates SSH `Host` blocks in `~/.local/share/devvm/ssh_config` (`paths.Dirs.SSHConfig`) whenever a project VM starts, launches, or stops. Only VMs currently in `backend.StatusRunning` with usable loopback SSH parameters and pinned host keys receive an entry. A stopped VM's loopback port is released back to the host OS and could be claimed by another local process. `devvm` omits stopped VMs from `~/.local/share/devvm/ssh_config` so `ssh` and editors never connect or forward an SSH agent to an unverified local listener. ### Host aliases and VM naming Every project VM has a canonical name in the format `devvm--`, where `` is the lowercase alphanumeric slug of the project directory and `` is a 14-hex-digit (56-bit) path digest (`vmname.HashLen = 14`). When `devvm` creates a new VM (`creation.create`), it claims a permanent SSH alias inside `vm.json` (`state.VM.Alias`) under the global claims lock (`~/.local/share/devvm/run/claims.lock`) via `sshconfig.ChooseAlias`: 1. If no existing VM state records the short name `devvm-`, the new VM claims `devvm-`. 2. If another VM already holds `devvm-`, the new VM's alias falls back to its full name `devvm--`. 3. Once recorded in `vm.json`, a VM's alias never changes even if the first VM is later deleted. Both the alias and the full name (`Host `) are written to `ssh_config`. ### Including `ssh_config` in `~/.ssh/config` To resolve `devvm-` hostnames with `ssh` or editors, `~/.ssh/config` must contain the `Include` directive before any `Host` or `Match` block: ```sshconfig title="~/.ssh/config" Include /Users/you/.local/share/devvm/ssh_config ``` When you run `devvm host`, `devvm code`, or create a `--clone` VM, `sshconfig.Config.EnsureInclude` checks whether `~/.ssh/config` contains this line: - In an interactive terminal, `devvm` prompts `Add it to ~/.ssh/config now? [Y/n]`. - Passing `--ssh-config` to `devvm host` or `devvm code` prepends the `Include` line to `~/.ssh/config` (preserving symlinks if `~/.ssh/config` links into a dotfiles repository and setting `~/.ssh` mode to `0700`) without prompting. - You can also target the generated file directly without modifying `~/.ssh/config`: ```bash ssh -F ~/.local/share/devvm/ssh_config devvm-website ``` ### Per-start host key pinning Lima disables SSH host-key verification in its own generated configurations. Instead of trusting network-supplied host keys or inheriting Lima's permissive options, `vmssh.Checked` and `vmssh.PinHostKeys` pin each running VM's public keys out-of-band: 1. Before starting a stopped VM, `starter.start` removes any prior `~/.local/share/devvm/vms//run/known_hosts` via `vmssh.ForgetHostKeys`. 2. Once the VM is running, `vmssh.PinHostKeys` queries the backend's current boot identifier (`backend.Run.ID`) and executes `cat /etc/ssh/ssh_host_*_key.pub` inside the VM through the backend execution channel (capped at 64 KiB with a 30-second timeout). 3. `vmssh.ParseHostKeys` decodes each base64 key blob, verifies that the wire-format key header matches the line's key type (`^[A-Za-z0-9][A-Za-z0-9@._+-]{0,63}$`), rejects SSH certificates (`-cert-`), and drops `@cert-authority` or `@revoked` markers and host patterns. 4. `PinHostKeys` re-verifies that `backend.Run.ID` did not change during the read, then writes `~/.local/share/devvm/vms//run/known_hosts` starting with `# devvm: host keys of start `. ### Generated `Host` block directives For each running VM, `sshconfig.Config.entry` writes a hardened block that overrides any broad `Host *` settings in your personal `~/.ssh/config`: ```sshconfig title="~/.local/share/devvm/ssh_config" # Written by devvm on every launch; edits are overwritten. Host devvm-website devvm-website-0123456789abcd HostKeyAlias devvm-website-0123456789abcd UserKnownHostsFile "/Users/you/.local/share/devvm/vms/0123456789abcd/run/known_hosts" StrictHostKeyChecking yes NoHostAuthenticationForLocalhost no ForwardAgent no ControlMaster no ControlPath none ForwardX11 no ForwardX11Trusted no Tunnel no PermitRemoteOpen none AddKeysToAgent no ProxyJump none ProxyCommand none UpdateHostKeys no HashKnownHosts no Hostname 127.0.0.1 Port 54321 User devvm IdentityFile "/Users/you/.local/share/devvm/lima/_config/user" ``` | Directive | Enforced Value | Security and Operational Purpose | | --- | --- | --- | | `HostKeyAlias` & `UserKnownHostsFile` | `` & `.../run/known_hosts` | Binds host key verification to the VM's per-start pinned key file instead of `~/.ssh/known_hosts`. | | `StrictHostKeyChecking` & `NoHostAuthenticationForLocalhost` | `yes` & `no` | Forces strict verification even though `Hostname` is `127.0.0.1`. | | `ForwardAgent` & `IdentityAgent` | `no`, `yes`, or `""` | Locks agent forwarding to the mode the VM's current start was launched with (`off`, `on`, or `devvm`). | | `ControlMaster` & `ControlPath` | `no` & `none` | Prevents connection multiplexing so a session never inherits the forwarded agent of an earlier connection. | | `ForwardX11` & `ForwardX11Trusted` | `no` | Blocks guest processes from accessing the host X11 display server. | | `Tunnel` & `PermitRemoteOpen` | `no` & `none` | Disables layer-2/3 tun/tap devices and remote SOCKS/port-open requests (requires OpenSSH 8.7+). | | `AddKeysToAgent`, `ProxyJump`, `ProxyCommand` | `no` / `none` | Prevents `Host *` jump hosts, proxy commands, or key auto-loading from applying to VM connections. | OpenSSH applies the first value it sees for most options, but `RemoteForward` directives accumulate unless `ClearAllForwardings yes` is set (which would also break explicit `ssh -R` flags on the command line). Do not place `RemoteForward` under `Host *` in `~/.ssh/config`. Inside the VM, `sshd` restricts remote forwards to `127.77.0.0/16` (used for inter-VM links) and refuses remote forwards on `localhost`. --- ## Prepare a VM and open VS Code (`devvm host` and `devvm code`) Both `devvm host` and `devvm code` call `Launch.reach` (`internal/vm/ssh.go`), which executes the full existing-VM startup and setup pipeline (`readyConf`) without opening an interactive terminal shell: Finds the made VM for the current directory (or parent project directory), verifies the backend supports `backend.FeatureSSH`, checks `.git` protection and host safety, and starts the VM if stopped. Runs `Proxy.SelfCheck` to ensure the VM's network tier holds, executes `sync.sh` inside the VM to mount shared tools and update guest settings, pins the VM's SSH host keys, and rewrites `~/.local/share/devvm/ssh_config`. Calls `EnsureInclude` to confirm `~/.ssh/config` includes `~/.local/share/devvm/ssh_config` (prompting if interactive, or adding immediately with `--ssh-config`). `devvm host` prints the resolved SSH host alias and target directory inside the VM. `devvm code` locates `code` on `PATH` and replaces the `devvm` process via `exec` with `code --remote ssh-remote+ `. ### Command reference: `devvm host` and `devvm code` ```bash title="Prepare VM and print SSH host alias" devvm host devvm host --ssh-config ``` ```bash title="Open VM in VS Code over Remote-SSH" devvm code devvm code --ssh-config ``` Prepend `Include ~/.local/share/devvm/ssh_config` to `~/.ssh/config` automatically without prompting. Target this directory's own VM rather than a parent directory's VM. Mutually exclusive with `--parent`. Target the nearest parent directory's VM, even across a nested Git repository boundary. Mutually exclusive with `--here`. When `devvm code` invokes `code --remote ssh-remote+ `, it passes `e.Environ` (the environment `devvm` was started with) so VS Code uses the host's environment rather than internal agent overrides. The target directory `` (`target.shellDir`) resolves to: - The project root `f.Dir` when run at the project root. - The current subdirectory `t.sub` when run inside a subdirectory of a parent VM's project. - For a `--clone` VM run from a host subdirectory, `t.sub` if that subdirectory exists inside the guest clone, or `f.Dir` otherwise. If `code` is not on your host `PATH`, `devvm code` exits with an error naming the SSH host alias (for example, `devvm-website`) and instructs you to run *Shell Command: Install 'code' command in PATH* in VS Code or connect manually via *Remote-SSH: Connect to Host*. ### VS Code Remote-SSH guest hardening By default, VS Code Remote-SSH automatically forwards ports opened by processes in its integrated terminal to the host's `localhost` (bypassing `devvm ports` rules) and answers `git` credential prompts in the VM using the host's GitHub sign-in. On every launch, `assets/guest/sync.sh` updates `~/.vscode-server/data/Machine/settings.json` inside the VM using `jq` to disable these behaviors while preserving other machine settings: ```json title="~/.vscode-server/data/Machine/settings.json (in VM)" { "remote.autoForwardPorts": false, "git.terminalAuthentication": false, "github.gitAuthentication": false } ``` If `~/.vscode-server/data/Machine/settings.json` contains comments (invalid strict JSON), `sync.sh` leaves the file untouched and writes a notice to stderr in the run log. Because root or the user inside the VM can modify `~/.vscode-server/data/Machine/settings.json` after launch, set those three keys to `false` in your host VS Code user settings when working with untrusted code. --- ## Per-VM SSH agent forwarding (`devvm ssh-agent`) SSH agent forwarding is `off` by default for every VM. When enabled, any process inside the VM—including coding agents—can request signatures from the forwarded agent while you are connected, though private key material never enters the VM. `devvm` supports three per-VM forwarding modes (`internal/sshagent`): | Mode | Forwarded Socket | Behavior | | --- | --- | --- | | `off` | None (`ForwardAgent no`) | Default. No SSH agent is exposed to the VM in `devvm` shells or `ssh_config`. Git over SSH fails inside the VM. | | `on` | Host `$SSH_AUTH_SOCK` (`ForwardAgent yes`) | Forwards your host SSH agent. If `$SSH_AUTH_SOCK` holds no keys when launching and `~/.ssh` contains `id_ed25519`, `id_ecdsa`, or `id_rsa`, `devvm` offers to run `ssh-add` (`ssh-add --apple-use-keychain` on macOS). | | `devvm` | `~/.local/share/devvm/agent.sock` | Forwards `devvm`'s dedicated, isolated `ssh-agent` process holding only the keys explicitly added via `devvm ssh-agent add`. | ### Configuring a VM's agent mode You can enable your personal agent at VM creation time with `devvm --ssh-agent`, or inspect and change the mode on an existing VM with `devvm ssh-agent`: ```bash # Show the current VM's forwarding mode and keys in devvm's own agent devvm ssh-agent # Forward devvm's isolated agent (prompts to restart if the VM is running) devvm ssh-agent devvm # Forward your host SSH_AUTH_SOCK devvm ssh-agent on # Disable SSH agent forwarding devvm ssh-agent off ``` #### Next-start consistency (`sshagent.Running`) A VM's backend agent socket mount is configured when the VM boots. `starter.start` records the active mode and a 128-bit SHA-256 prefix of the backend boot ID (`_`) in `vm.json` under `agent.made.started`. Both `devvm`'s own shells and `sshconfig.Config.entry` evaluate `sshagent.Running(st, start)` against the running VM's boot ID: - If you run `devvm ssh-agent on`, `devvm`, or `off` while the VM is running, `devvm` prompts to restart the VM immediately (`Restart now to apply this change? [Y/n]`). - If you decline (or run non-interactively), the running VM continues using the mode it booted with across both `devvm` shells and `ssh`/`code` connections until the next `devvm restart` or boot. - When `devvm ssh-agent on` or `devvm` is run from a subdirectory targeting a parent directory's VM, `target.widen` requires explicit `[y/N]` confirmation before widening access on the parent VM. ### Managing keys in `devvm`'s isolated agent `devvm`'s dedicated SSH agent runs at `~/.local/share/devvm/agent.sock` (`paths.Dirs.AgentSock`) and tracks registered keys in `~/.local/share/devvm/agent-keys.json` (`paths.Dirs.AgentKeys`). When a VM configured with `devvm ssh-agent devvm` writes its `ssh_config` block, `devvm` sets both `ForwardAgent` and `IdentityAgent` to `"/Users/you/.local/share/devvm/agent.sock"`, ensuring GUI editors like VS Code forward `devvm`'s agent regardless of the environment variables the editor started with. ```bash # Load a deploy key with the default 12h lifetime devvm ssh-agent add ~/.ssh/deploy_key # Load a key with a custom lifetime and per-use ssh-askpass confirmation devvm ssh-agent add ~/.ssh/deploy_key --lifetime 30m --confirm # Remove a key from devvm's agent and ~/.local/share/devvm/agent-keys.json devvm ssh-agent rm ~/.ssh/deploy_key ``` Path to the private key file to add or remove. Resolved to an absolute path with parent symlinks evaluated (`paths.PhysicalParent`). Duration the agent retains the key in memory before dropping it (`30m`, `8h`, `1d`, `2w`, or `forever`). Validated by `sshagent.CheckLifetime` and passed to `ssh-add -t`. Require confirmation via an `ssh-askpass` program before each signature (`ssh-add -c`). Requires `$SSH_ASKPASS` to be set or `ssh-askpass` to exist on `PATH`. #### Key security checks (`Agent.Add`, `Agent.Ensure`, and `Agent.load`) Whenever `devvm ssh-agent add` loads a key or `Agent.Ensure` reloads expired keys on VM launch, `internal/sshagent` enforces three invariants: 1. **No keys inside VM mounts (`Agent.mountedBy`):** `devvm` resolves the physical path (`paths.Physical`) of the key file and compares it against every directory mounted by any existing project VM (project folders, agent config directories, and shared tools). If the key resides inside a mounted folder, `devvm` refuses to add or load it because the VM could read the private key file directly. 2. **Atomic copy and fingerprint pinning (`Agent.load`):** `devvm` reads the private key into an isolated temporary file (`0600`) with no `.pub` file alongside it, derives the `SHA256:` fingerprint directly from the private key via `ssh-keygen -lf`, and verifies that it matches the `fingerprint` recorded in `agent-keys.json` when the key was added. If the file was replaced or a symlink was retargeted to a different key, `devvm` refuses to load it (`AgentKeyChanged`). (Old PEM-encrypted keys whose public fingerprint cannot be extracted without a passphrase must be converted first with `ssh-keygen -p -f KEY`). 3. **Fingerprint-based removal (`Agent.unload`):** `devvm ssh-agent rm` matches the recorded fingerprint against `ssh-add -L` output and removes the key from the running agent via `ssh-add -d` even if the original key file on disk was modified or deleted. ### JSON output (`devvm ssh-agent --json`) Running `devvm ssh-agent --json` (without positional arguments) ensures any registered keys are loaded into `devvm`'s agent and returns a version 1 JSON object: ```bash title="Command" devvm ssh-agent --json ``` ```json title="Output" { "version": 1, "mode": "devvm", "keys": [ { "file": "/Users/you/.ssh/deploy_key", "fingerprint": "SHA256:mVPwvezndPv/ARoIadVY98vAC0g+P/5633yTC4d/wXE", "confirm": false, "lifetime": "12h", "loaded": true } ] } ``` Schema version (`1`). The folder's VM forwarding mode configured in `vm.json`: `"off"`, `"on"`, or `"devvm"`. Keys registered in `~/.local/share/devvm/agent-keys.json` (`[]` when none are added). - `file` (`string`): Resolved absolute path to the private key file. - `fingerprint` (`string`, optional): Pinned `SHA256:...` fingerprint recorded when the key was added. - `confirm` (`boolean`): Whether `ssh-add -c` confirmation is enabled. - `lifetime` (`string`): Configured key lifetime (such as `"12h"` or `"forever"`). - `loaded` (`boolean`): Whether `devvm`'s running `ssh-agent` currently holds the key in memory. --- ## VNC desktop and host relay (`devvm desktop`) Project VMs built with the built-in `desktop` layer include an Xfce desktop environment and TigerVNC (`Xvnc`). `devvm desktop` starts the guest desktop on a Unix domain socket inside the VM and exposes it on a dedicated `127.0.0.1` port on the host through `internal/desktop.Relay`. ### Adding the `desktop` layer The `desktop` layer (~370 MB on disk) is opt-in so base VMs stay minimal. Add it when creating a new VM, set it as a default for new VMs, or remake an existing VM with it: ```bash # Create a new VM with the desktop layer devvm --layers desktop # Set desktop as a default layer for future VMs devvm setup --layers desktop # Remake an existing VM to add or remove the desktop layer devvm layer add desktop devvm layer rm desktop ``` ### Launching `devvm desktop` ```bash # Start the VM's desktop, serve on 127.0.0.1, and open macOS Screen Sharing devvm desktop # Share the clipboard bidirectionally for this session devvm desktop --clipboard # Print the vnc://127.0.0.1: address and password without launching Screen Sharing devvm desktop --no-open ``` Enable bidirectional clipboard sharing between the host VNC viewer and the VM for the current `devvm desktop` run. Without this flag, `desktop.Relay` strips all `ClientCutText` messages from the viewer and `devvm-desktop` disables `Xvnc` clipboard options. Print the loopback VNC URL and password without automatically opening macOS Screen Sharing. When run, `devvm desktop` prints the persistent loopback endpoint and 8-character password: ```text devvm-website-0123456789abcd's desktop is at vnc://127.0.0.1:20145 Password: k3mxq7tw It stays there until you press Ctrl+C. ``` On macOS (unless `--no-open` is passed), `devvm` opens `vnc://127.0.0.1:` in Screen Sharing. On Linux, connect any RFB 3.3/3.7/3.8 VNC client (such as TigerVNC `vncviewer` or Remmina) to `127.0.0.1:`. The relay runs until you press `Ctrl+C` or the VM stops (polled every 5 seconds via `b.Inspect`). Stopping `devvm desktop` closes the host listener while leaving the Xfce session running inside the VM. ### Desktop relay and guest architecture ```mermaid sequenceDiagram autonumber participant V as VNC Viewer (Host) participant R as desktop.Relay (Host 127.0.0.1:) participant S as ssh -W Channel (Host -> VM) participant X as Xvnc (/run/devvm-desktop/vnc.sock) V->>R: TCP connect (max 8 concurrent connections) R->>V: "RFB 003.008\n" + SecurityType 2 (VNC Auth) + 16-byte challenge V->>R: 16-byte DES-encrypted challenge response Note over R: Mutex-serialized constant-time check
against vm.json desktop.made.password
(1s FailDelay on mismatch) alt Wrong password R->>V: SecurityResult = 1 (Failed) & close else Valid password R->>S: Spawn ssh -F ssh_config -W /run/devvm-desktop/vnc.sock S->>X: Connect to Unix socket (0600) X->>R: "RFB 003.008\n" + SecurityType 1 (None) R->>X: Select SecurityType 1 (None) X->>R: SecurityResult = 0 (OK) R->>V: SecurityResult = 0 (OK) par Viewer to VM V->>R: ClientInit + RFB client messages Note over R: If !Clipboard: filter() drops ClientCutText (type 6)
and disconnects on unknown RFB message types R->>X: Forwarded RFB messages and VM to Viewer X->>R: ServerInit + FramebufferUpdates R->>V: Unmodified server stream end end ``` #### 1. Guest `Xvnc` and `devvm-desktop` (`/usr/local/bin/devvm-desktop`) In `assets/layers/desktop.sh`, `devvm-desktop start on|off` launches TigerVNC `Xvnc` on display `:1` (`1600x1000`, 24-bit depth, `-AlwaysShared`) inside `/run/devvm-desktop` (mode `0700`, owned by the VM user): - **Unix socket only:** `Xvnc` listens exclusively on `/run/devvm-desktop/vnc.sock` (`-rfbunixpath /run/devvm-desktop/vnc.sock -rfbunixmode 0600 -rfbport -1 -nolisten tcp`). Because it binds no TCP port, `devvm`'s automatic port forwarder never exposes it even when a VM forwards all ports, and no X11 TCP connections are accepted (`ForwardX11 no` is also enforced in `ssh_config` and `sshd_config`). - **In-place clipboard toggling:** When `devvm-desktop start on|off` runs against an already-running `Xvnc` process, it updates `AcceptCutText`, `SendCutText`, `SetPrimary`, and `SendPrimary` in place using `vncconfig -set`. - **Guest log:** Output from `Xvnc` and `dbus-launch --exit-with-session startxfce4` is written to `~/.cache/devvm-desktop.log` inside the VM. #### 2. Guest `devvm-local-proxy` for desktop browsers (`127.0.77.1:3128`) Every request to a VM's host proxy requires the VM's proxy password in `Proxy-Authorization`, which `devvm` provides in `HTTP_PROXY=http://devvm:KEY@192.168.5.2:PORT`. Because Chromium, Chrome, and Firefox ignore credentials embedded in proxy environment variables and prompt interactively on every site, `sync.sh` installs `/usr/local/bin/devvm-local-proxy` (mode `0700`): - When `devvm-desktop` runs, it invokes `devvm-local-proxy start`, which starts a `socat` listener on `127.0.77.1:3128` (`proxy.LocalProxyAddr:proxy.LocalProxyPort`, restricted to `range=127.0.0.0/8`, `max-children=256`) and exports `http_proxy` and `https_proxy` as `http://127.0.77.1:3128` for the Xfce session. - `devvm-local-proxy` accepts only `CONNECT host:port` tunnels or absolute `http://` / `https://` proxy requests (up to 100 headers, 30s timeout per line), strips any client-supplied `Proxy-Authorization`, injects the VM's `Proxy-Authorization: Basic ...` header, and forwards traffic to the host proxy. Origin-form requests (`GET /`) receive `400 Bad Request` so web pages in the browser cannot talk to the local proxy as a web server. - `devvm`'s port forwarder never forwards listeners bound to `127.0.77.1`. #### 3. Host loopback port allocation and password generation - **Persistent loopback port:** `Desktop.listener` binds `127.0.0.1:` using `ports.Allocator` under the claims lock and saves the port in `vm.json` under `ports.host.desktop` (`20000` up to the host OS ephemeral port boundary). If the saved port is free on later runs, `devvm desktop` reuses it so macOS Screen Sharing can match saved Keychain credentials. - **Host-only password:** On first run, `desktopPassword` generates an 8-character password from the 31-character unambiguous alphabet `abcdefghjkmnpqrstuvwxyz23456789` (~40 bits of entropy, omitting `0`, `o`, `1`, `l`, `i`) and stores it in `vm.json` under `desktop.made.password`. The password is never sent to the VM and never written to the run log. #### 4. Host RFB authentication, SSH socket channel, and clipboard filtering Because root inside the VM can reconfigure `Xvnc` or replace `/run/devvm-desktop/vnc.sock` with a different server, `internal/desktop` enforces security guarantees on the host: - **Host-side DES challenge-response (`Relay.authenticate`):** `Relay` accepts up to 8 concurrent connections (`maxConns = 8`) with a 15-second handshake timeout (`desktopTimeout`). It speaks RFB 3.3, 3.7, and 3.8 (treating Apple Screen Sharing's `3.889` as `3.8`), offers only `secVncAuth` (`2`), and verifies the 16-byte DES challenge response (`desktop.Response`, bit-reversed 8-byte key per RFC 6143 §7.2.2) in constant time under a mutex (`r.mu`). A failed attempt sleeps for `1s` (`desktopFailDelay`) while holding the mutex, limiting brute-force guessing across all connections to 1 attempt per second. - **Isolated `ssh -W` channel (`desktopSSHArgs`):** Only after authentication succeeds does the relay spawn `ssh` to connect to `/run/devvm-desktop/vnc.sock`: ```text ssh -F -T -o BatchMode=yes -o ControlMaster=no -o ControlPath=none \ -o IdentityAgent=none -o ForwardAgent=no -o ForwardX11=no \ -o ClearAllForwardings=yes -o PermitLocalCommand=no \ -o ConnectTimeout=10 -o LogLevel=ERROR -W /run/devvm-desktop/vnc.sock ``` - **Strict RFB client message parser (`internal/desktop/filter.go`):** When `--clipboard` is not set, `filter` parses every client-to-server RFB message after `ClientInit`: - `msgClientCutText` (`6`), including negative-length extended clipboard payloads, is discarded before reaching the VM. - Known message types (`SetPixelFormat=0`, `SetEncodings=2`, `FramebufferUpdateRequest=3`, `KeyEvent=4`, `PointerEvent=5`, `ContinuousUpdates=150`, `Fence=248` with max 64-byte payload, `Xvp=250`, `SetDesktopSize=251`, `GII=253`, and `QEMU=255` subtypes `0` and `1`) are framed and forwarded. - Any unknown message type immediately terminates the viewer connection so unparsed client extensions cannot smuggle clipboard data into the VM. --- ## Troubleshooting `PermitRemoteOpen none` requires OpenSSH 8.7 or newer. Check your client version with `ssh -V`. On macOS or Linux systems with an older OpenSSH binary early on `PATH`, update OpenSSH or ensure `/usr/bin/ssh` (macOS system OpenSSH) is used. 1. Verify that `~/.ssh/config` contains `Include ~/.local/share/devvm/ssh_config` at the very top of the file, above any `Host` or `Match` block. Run `devvm host --ssh-config` in the project directory to add it automatically and start the VM. 2. Confirm the VM is running (`devvm ls`). Stopped VMs are removed from `ssh_config` until started with `devvm`, `devvm host`, or `devvm code`. - **Key inside a mounted folder (`AgentKeyMounted`):** The key file (or symlink target) is inside a directory mounted by a project VM. Move the private key to `~/.ssh/` or another directory outside all project folders. - **Key fingerprint unreadable (`AgentKeyNoFingerprint`):** The key is not a valid private key or uses legacy PEM encryption that hides the public key until decrypted. Convert legacy PEM keys in place with `ssh-keygen -p -f `. - **Key changed since added (`AgentKeyChanged`):** The file's current `SHA256:` fingerprint no longer matches the fingerprint pinned in `~/.local/share/devvm/agent-keys.json`. Run `devvm ssh-agent add ` again to pin the new key. - **`--confirm` fails (`AgentConfirmNeedsAskpass`):** Install an `ssh-askpass` binary on `PATH` or set the `SSH_ASKPASS` environment variable before using `--confirm`. - **VM has no desktop (`DesktopNoLayer`):** `/usr/local/bin/devvm-desktop` is missing because the VM was created without the `desktop` layer. Run `devvm layer add desktop` to remake the VM with the layer. - **Xvnc failed to start (`DesktopStartFailed`):** Inspect `~/.cache/devvm-desktop.log` inside the VM for `Xvnc` or `startxfce4` errors. - **Browser shows ERR_TUNNEL_CONNECTION_FAILED on HTTPS URLs:** The target domain is blocked by the VM's network tier. Open a terminal inside the desktop and run `devvm-why` (or `devvm network log` on the host) to see the blocked domain and the `devvm network allow ` command to permit it. ## Related pages Complete flag and exit-code reference for devvm host, devvm code, and devvm desktop. Command syntax and subcommands for devvm ssh-agent, devvm network, and devvm secret. How the built-in desktop, docker, and nix layers are built, keyed, and added to VMs. Troubleshoot SSH host-key verification errors, moved project folders, and run logs. --- ## 14. Isolate work with repository clones > Create VMs with independent Git clones using devvm --clone, switch existing VMs between mounted folders and clones, and fetch VM branches to the host. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/14-isolate-work-with-repository-clones.md - Generated: 2026-11-07T22:27:23.519Z ### Source Files - `internal/cli/clone.go` - `internal/vm/clone.go` - `internal/hostgit/hostgit.go` - `internal/cli/rm.go` - `internal/vm/up.go` --- title: "Isolate work with repository clones" description: "Create VMs with independent Git clones using devvm --clone, switch existing VMs between mounted folders and clones, and fetch VM branches to the host." --- `devvm --clone` provisions a project VM with an independent Git repository clone on the guest disk instead of mounting the host project folder read-write. The VM mounts only the read-only shared tools directory (`/mnt/devvm-tools`) and configured agent settings directories from the host store, preventing processes inside the guest from modifying the host working tree or `.git` directory. Existing VMs can inspect or toggle clone mode using `devvm clone [on|off]`, which remakes the VM from its image while preserving its home folder and VM settings. ## Clone lifecycle and host-guest sync When a VM runs in clone mode (`clone.on = true` in `vm.json` and `git.mode = "clone"`), `devvm` seeds the guest repository over SSH on launch, configures a host Git remote pointing to the VM, and verifies unfetched commits and uncommitted worktree changes before deleting or remaking the VM. ```mermaid sequenceDiagram autonumber participant CLI as devvm CLI (internal/vm) participant HostGit as Host Git (internal/hostgit) participant SSH as Pinned VM SSH (internal/vmssh) participant Guest as Guest VM Clone CLI->>HostGit: Validate repo root, HEAD commit, symbolic ref, OwnRepo CLI->>Guest: Create VM with Project="", CloneAt= (no folder mount) CLI->>Guest: sudo mkdir -p && chown && git init -q && receive.denyCurrentBranch=updateInstead CLI->>HostGit: git push --no-verify --no-recurse-submodules --no-signed ( or worktree ref) HostGit->>SSH: GIT_SSH_COMMAND (pinned HostKeyAlias, BatchMode=yes, ForwardAgent=no) SSH->>Guest: git-receive-pack '' CLI->>HostGit: git remote add : Note over HostGit,Guest: User or coding agent commits on branches inside Guest CLI->>Guest: Pre-delete / clone off check (for-each-ref, worktree list, status --porcelain) CLI->>HostGit: Offer fetch into refs/heads/saved// (--refmap="") CLI->>HostGit: Save unique remote-tracking commits & git remote remove ``` ## Create a VM with `--clone` Run `devvm --clone` from the top-level directory of a Git repository that has at least one commit and an attached branch (`HEAD` must be a symbolic reference, not detached). Create the VM and seed its guest clone: ```bash devvm --clone ``` During creation, `devvm` sets `spec.Project = ""` and `spec.CloneAt = f.Dir`, requiring `backend.FeatureSSH` on the selected backend. Because the host folder is not mounted, host `.git` immutable flag locking (`chflags uchg`) and set-id file scans are skipped. Once the VM is running and `~/.ssh/config` includes the `devvm` SSH configuration, fetch branches from the guest repository into the host repository using the added remote: ```bash git fetch devvm git log -p main..devvm/main ``` Once a VM is created with `--clone`, subsequent `devvm` launches in that folder automatically open the clone VM without needing `--clone` again. Passing `--clone` to an existing VM that mounts the host folder fails with `clone_needs_new_vm`; use `devvm clone on` to switch an existing VM. ### Repository validation rules Before building images or creating a VM (`checkCloneRepo` and `gitprotect.OwnRepo`), and again immediately before seeding (`seedClone`), `devvm` validates the host directory so that pushing into the VM cannot leak objects or branches from an unrelated repository on the host: | Check | Implementation rule | Failure message ID | | --- | --- | --- | | Host `git` binary | `exec.LookPath("git")` must succeed on the host | `clone_needs_git` | | Repository top-level | `git rev-parse --show-toplevel` must equal the physical project directory | `clone_not_top` | | Initial commit exists | `git rev-parse -q --verify HEAD` must succeed | `clone_no_commits` | | Attached branch | `git symbolic-ref -q HEAD` must succeed (detached `HEAD` is refused) | `clone_detached` | | Self-contained `.git` or valid linked worktree | `.git` must be a directory with no `commondir` pointing elsewhere, or a regular `.git` file for a linked worktree with a bidirectional link (`/.git/worktrees//gitdir` resolves back to `/.git` and `commondir` resolves to `/.git`), and `git rev-parse --path-format=absolute --absolute-git-dir --git-common-dir` must match | `clone_git_elsewhere` | | No alternate object stores | Neither the git dir nor common dir may contain `objects/info/alternates` | `clone_alternates` | | No symlinked refs or objects | Neither `objects`, `packed-refs`, `reftable`, `refs`, nor any entry inside `objects/` or `refs/` may be a symbolic link | `clone_git_elsewhere` | If a repository uses `objects/info/alternates`, run `git repack -a -d` inside the repository and remove `.git/objects/info/alternates` before running `devvm --clone`. ### Seeding and branch copying When `setUpVM` runs on a `--clone` VM where `clone.made.seeded` is `false`, `seedClone` initializes the guest repository at the exact host folder path (`f.Dir`) so shells and editors open at identical paths: ```sh sudo mkdir -p -- "$1" && sudo chown -- "$(id -u):$(id -g)" "$1" && cd -- "$1" && git init -q && git config receive.denyCurrentBranch updateInstead && git symbolic-ref HEAD "refs/heads/$2" ``` After initializing the guest repository and pointing its `HEAD` to the host's current branch, `devvm` pushes refs from the host into the guest: - **Standard repository:** Pushes `--all` (every local branch under `refs/heads/`). - **Linked worktree (`gitDir != commonDir`):** Pushes only `refs/heads/:refs/heads/` and, if configured and valid as a single non-wildcard ref (`git check-ref-format`), its upstream ref `:`. Other branches in the common repository remain on the host. - **Excluded items:** Tags, uncommitted staged or unstaged changes, untracked files, and ignored files (such as `.env` or `node_modules`) are not copied. If `git status --porcelain --untracked-files=no` is non-empty on the host, `devvm` emits the `uncommitted_not_copied` warning. - **Retry on failure:** If seeding fails (for example, if SSH cannot reach the guest), `clone.made.seeded` remains `false` and the next `devvm` launch retries `seedClone`. ### Host remote and SSH config During `seedClone`, `devvm` selects a remote name (`freeRemote`) and records it in `vm.json` under `clone.made.remote` before adding the remote to the host repository: 1. `devvm` (if unused or already pointing to `:`) 2. `devvm-` 3. `` (full VM name, such as `devvm-web-71a99b1076d92c`) If all three remote names are already taken by other URLs, `seedClone` fails with `remotes_taken`. The remote URL is set to `:`. For standard `git fetch` or `git push` commands on the host to resolve ``: - The VM must be running (`devvm stop` removes the VM from `~/.local/share/devvm/ssh_config`). - `~/.ssh/config` must contain the `Include ~/.local/share/devvm/ssh_config` directive. If `devvm` cannot add the `Include` line (for example, when run without an interactive terminal), it warns with `fetch_until` and you can fetch explicitly with: ```bash GIT_SSH_COMMAND='ssh -F ~/.local/share/devvm/ssh_config' git fetch devvm ``` ## Hardened host Git and SSH transport Because a mounted VM or a nested repository could write malicious configuration into `.git/config`, every Git subprocess that `devvm` executes on the host goes through `internal/hostgit.Command`, and every clone push or fetch uses a locked-down `GIT_SSH_COMMAND` in `internal/vm/clone.go`. ### Host Git environment and config overrides `hostgit.Command` strips all `GIT_*` variables from the inherited environment, removes relative entries from `PATH`, and injects `GIT_CONFIG_COUNT` key-value pairs that take precedence over repository, worktree, and global config files: | Setting / Variable | Value | Purpose | | --- | --- | --- | | `GIT_TERMINAL_PROMPT` | `0` | Never prompt interactively on the terminal | | `GIT_NO_LAZY_FETCH` | `1` | Prevent partial-clone lazy object fetching from arbitrary remotes | | `GIT_ALLOW_PROTOCOL` | `ssh` | Restrict transport protocol strictly to `ssh`, overriding any `protocol..allow` in `.git/config` | | `GIT_SSH_COMMAND` | `false` (or pinned SSH function) | Neutralize `core.sshCommand` unless explicitly overridden by `clones.gitSSH` | | `GIT_SSH_VARIANT` | `ssh` | Ignore `ssh.variant` overrides in repository config | | `core.fsmonitor` | `false` | Disable `fsmonitor` background daemons or commands | | `core.hooksPath` | `/dev/null` (`os.DevNull`) | Disable directory-based Git hooks | | `core.alternateRefsCommand` | `true` | Prevent executing arbitrary commands to list alternate refs | | `protocol.allow` / `protocol.ssh.allow` | `never` / `always` | Allow only SSH transport | | `maintenance.auto` / `gc.auto` | `false` / `0` | Prevent background `git maintenance` or `git gc` | | `hook..enabled` | `false` | Disable all 28 `githooks(5)` events and any `hook..enabled` defined in the repo config | | `filter..clean` / `smudge` / `process` / `required` | `""` / `""` / `""` / `false` | Disable all clean, smudge, and process filter drivers named in the repo config | When checking dirty status on the host (`hostgit.Status`), `devvm` runs `git -C --no-optional-locks status --ignore-submodules=all --porcelain --untracked-files=no` so Git neither writes index locks nor enters submodules. ### Pinned SSH wrapper for push and fetch When `seedClone` pushes branches (`git push -q --no-verify --no-recurse-submodules --no-signed`) or `fetchUnfetched` fetches branches (`git fetch --no-tags --no-recurse-submodules --refmap=`), `gitSSHCommand` constructs a POSIX shell function for `GIT_SSH_COMMAND`: ```go // internal/vm/clone.go func gitSSHCommand(s vmssh.SSH, dir string) string { run := func(verb string) string { return "exec " + shell.QuoteJoin(append([]string{"ssh"}, s.CommandArgs(verb+" "+shell.Quote(dir))...)) } return `f() { for c; do :; done; case $c in "git-receive-pack "*) ` + run("git-receive-pack") + ` ;; "git-upload-pack "*) ` + run("git-upload-pack") + ` ;; esac; exit 1; }; f` } ``` This wrapper ignores the destination host, port, and flags that `git` passes to `ssh` (defeating `url..insteadOf`, `url..pushInsteadOf`, or modified remote URLs in `.git/config`) and connects directly to `127.0.0.1:` with: - `-F none` (ignores `~/.ssh/config`) - `-o BatchMode=yes` - `-o UserKnownHostsFile=~/.local/share/devvm/vms//run/known_hosts` and `-o GlobalKnownHostsFile=/dev/null` - `-o HostKeyAlias=` and `-o StrictHostKeyChecking=yes` - `-o ForwardAgent=no` and `-o ClearAllForwardings=yes` - Command restricted strictly to `git-receive-pack ''` or `git-upload-pack ''` ## Subfolders and multiple agents in one clone VM ### Subfolder resolution - Running `devvm`, `devvm host`, or `devvm code` inside a subdirectory of a `--clone` VM's project folder uses the parent folder's `--clone` VM (unless a nested `.git` repository sits in between). `target.shellDir` tests whether the subdirectory exists inside the guest clone (`test -d `): if present, the shell or editor opens in that subdirectory; if absent (for example, an untracked host directory), it falls back to the clone's top-level directory (`t.f.Dir`). - Running `devvm --clone` inside a subdirectory always targets the subdirectory's own VM (`need{own: o.Clone}`), identical to `--here`, and requires the subdirectory itself to be the root of a valid Git repository. ### Running multiple coding agents with Git worktrees Multiple coding agents can work concurrently inside a single `--clone` VM by creating linked Git worktrees inside the clone directory (the parent directory above the clone is owned by `root`, so worktrees must be placed inside the clone): ```bash echo .worktrees/ >> .git/info/exclude git worktree add -b agent/api .worktrees/api cd .worktrees/api ``` All worktrees inside the clone share the guest repository object store, so a single `git fetch devvm` on the host retrieves every worktree branch (such as `devvm/agent/api`). When the VM is deleted or switched off clone mode, `cloneCheck` inspects up to 100 linked worktrees inside the clone directory for uncommitted changes. ## Switch clone mode on an existing VM (`devvm clone`) `devvm clone` inspects or switches whether the current folder's VM uses an isolated clone (`on`) or mounts the host folder (`off`). ```bash title="Show current mode" devvm clone # Output: on (or off) devvm clone --json ``` ```bash title="Switch mounted VM to clone mode" devvm clone on # Or non-interactively in scripts: devvm clone on --force ``` ```bash title="Switch clone VM to mounted folder" devvm clone off # Skip clone unfetched/dirty checks (loses unfetched guest work): devvm clone off --force --no-clone-check ``` ### Command syntax and flags ```text devvm clone [on|off] [flags] ``` Target clone mode. Omit the positional argument to print the current mode (`on` or `off`). Passing `on` or `off` remakes the VM from its current image in the requested mode and opens a shell in it. Skip the remake confirmation prompt (`remake_confirm`), which is required when running without an interactive terminal or with `--no-input`. Does **not** bypass the interactive prompts for unfetched commits or uncommitted changes during `devvm clone off`. Skip checking a `--clone` VM's clone for unfetched commits and uncommitted changes before `devvm clone off` replaces the VM disk. Unfetched commits and uncommitted changes in the guest are lost. Skip exporting coding agent session history before remaking the VM. Initialize the remade VM with the image's clean home folder instead of saving and restoring the existing VM's home folder (`home.tar.gz`). Print the current clone mode as a version 1 JSON object (`{"version": 1, "clone": "on"}`). Cannot be combined with `on`, `off`, `--force`, `--no-export`, `--clean-home`, or `--no-clone-check` (`json_shows_only`). Use this folder's own VM, never a parent folder's VM. Mutually exclusive with `--parent`. Use the nearest parent folder's VM, even across a nested Git repository. Mutually exclusive with `--here`. ### How `devvm clone on` and `devvm clone off` remake the VM Switching clone mode uses the transactional remake pipeline (`Lifecycle.SetClone` -> `remaking.remake`): | Transition | Pre-remake checks | State & host mutations during remake | Rollback (`putBack`) if interrupted before home restore | | --- | --- | --- | --- | | `devvm clone on` | Runs `checkCloneRepo` on the host folder and `remaking.check` (backend `FeatureSSH`, image layers, disk space). Prompts `remake_confirm` unless `--force`. | Exports agent history (unless `--no-export`), saves `/home/` to `home.tar.gz` (unless `--clean-home`), renames old VM to `devvm--old`, creates new VM with `CloneAt = dir`, sets `clone.on = true`, starts VM, restores home, deletes `devvm--old`, runs `gitprotect.Check` + unlocks host `.git` (`nouchg`), sets `git.mode = "clone"`, and runs `seedClone`. | Deletes new VM, renames `devvm--old` back, resets `clone` section to `cloneInVM{}`, preserves host `.git` locks. | | `devvm clone off` | Runs `remaking.check`, prompts `remake_confirm` (unless `--force`), then runs `cloneCheck` (unless `--no-clone-check`) to offer fetching unfetched branches and warn on dirty worktrees. | Exports agent history, saves home, renames old VM to `devvm--old`, creates new VM mounting `Project = dir`, sets `clone.on = false` and `git.mode` (`on` on macOS, `off` on Linux), saves unique remote-tracking branches under `saved//*`, removes host Git remote, locks host `.git` (`uchg` on macOS), starts VM, restores home, and deletes `devvm--old`. | Deletes new VM, renames `devvm--old` back, unlocks host `.git`, re-adds host Git remote ``, sets `git.mode = "clone"` and `clone.on = true`. | Because a `--clone` VM stores its repository clone on the guest disk, `devvm layer add`, `devvm layer rm`, `devvm update`, and `devvm adopt ` refuse to run on a `--clone` VM (`remake_clone` / `adopt_clone`). To change layers or update the base image of a `--clone` VM, run `devvm clone off`, run the layer or update command, and then run `devvm clone on`. ## Pre-delete checks and `saved/` branches (`devvm rm`) When `devvm rm` (or `devvm clone off`) runs on a seeded `--clone` VM (`clone.made.seeded == true`), `cloneCheck.run` starts the VM if stopped (enforcing its network confinement self-check) and inspects the guest clone before any disk state is destroyed. ### 1. Unfetched commits check `devvm` runs `git -C for-each-ref --format="%(objectname) %(refname)" refs/heads` inside the VM (capped at `cloneOutMax = 1 MiB`). For each branch SHA matching `^[0-9a-f]{40,64}$`, `devvm` checks on the host whether the commit is already reachable from any ref in the host repository: ```bash git -C rev-list -n1 --not --all ``` If any guest branches contain commits missing from the host repository: - Branch names rejected by `git check-ref-format refs/heads/` are warned (`branch_names_invalid`) and skipped. - `devvm` prompts `fetch_before_delete` (`[Y/n]`) offering to fetch the branches into `saved//`. - Without a terminal or with `--no-input`, `devvm` refuses with `no_terminal_fetch` (even if `--force` was passed) and keeps the VM. ### 2. Uncommitted worktree changes check Next, `cloneCheck.changes` inspects the main clone directory and linked worktrees inside it: - Runs `git -C worktree list --porcelain -z` in the guest. - Filters linked worktrees (`cloneWorktrees`) to clean absolute paths strictly below ``, skipping `bare` and `prunable` entries, up to `cloneWorktreesMax = 100` worktrees. - Runs `git status --porcelain` in `` and in each existing linked worktree directory, printing up to `cloneShown = 20` status lines total across all worktrees. - If any worktree has uncommitted changes—or if the clone has more than 100 linked worktrees (`clone_worktrees_unchecked`)—`devvm` prompts `delete_anyway` (for `devvm rm`) or `remake_anyway` (for `devvm clone off`). - Without a terminal or with `--no-input`, `devvm` refuses with `no_terminal_delete` (even if `--force` was passed) and keeps the VM. ### 3. Remote removal and `saved//` preservation When the VM is deleted (or switched to `clone off`) and the host remote's URL still matches `:`, `removeRemote` removes the remote with `git remote remove `. Because removing a remote deletes its remote-tracking refs (`refs/remotes//*`), `saveFetched` first checks whether any remote-tracking branch contains commits not reachable from any other ref in the host repo: ```bash git -C rev-list -n1 --not --exclude=refs/remotes//* --all ``` Any remote-tracking branch with unique commits is atomically created as a local branch via `git update-ref --stdin` under `saved//` before `git remote remove ` runs. - **Collision-free prefix selection (`savedPrefix`):** `devvm` checks `refs/heads/saved/`. If a local branch already exists at, inside, or above that prefix, it tries `saved/-2`, `saved/-3`, up to `saved/-99`. If all 99 prefixes are taken, `fetchUnfetched` fails with `saved_names_taken` and `removeRemote` warns `remote_kept` and leaves the remote intact. - **Deleting a clone VM whose host folder is gone (`devvm rm `):** If the host project folder was deleted, there is no host repository to receive unfetched commits, so `devvm rm ` refuses a `--clone` VM unless `--no-clone-check` is passed (`rm_gone_clone`). ## State schema in `vm.json` A `--clone` VM records its clone state in the `clone` section and sets the `git` section's `mode` to `"clone"` in `~/.local/share/devvm/vms//vm.json`: ```json { "clone": { "on": true, "made": { "remote": "devvm", "seeded": true } }, "git": { "mode": "clone" } } ``` `true` when the VM uses an isolated guest clone instead of mounting the host project folder. Name of the Git remote added to the host repository (`devvm`, `devvm-`, or ``). Recorded before `git remote add` runs so interrupted creations or `devvm rm` clean it up. `true` once `seedClone` has initialized the guest repository, pushed the host branches, and added the host remote. Set to `"clone"` while `clone.on` is `true`, indicating that host `.git` protection locks do not apply. ## Related pages Read-only .git mount protection, symlink and hardlink containment, set-id file audits, and secret placeholder injection. How project paths map to VM names and hashes, state persistence in vm.json, and in-place vs. remake transitions. Configure ~/.ssh/config Include entries, per-VM SSH agents, and VS Code Remote-SSH connections. Complete command syntax and flags for devvm clone, git-protect, network, secret, and agent. --- ## 15. CLI reference: VM and workspace commands > Syntax, flags, exit codes, and examples for devvm (up), ls, restart, stop, update, rm, adopt, size, ports, host, code, desktop, and history. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/15-cli-reference-vm-and-workspace-commands.md - Generated: 2026-11-07T22:31:27.092Z ### Source Files - `internal/cli/root.go` - `internal/cli/up.go` - `internal/cli/flags.go` - `internal/cli/args.go` - `internal/cli/ls.go` - `internal/cli/size.go` - `internal/cli/update.go` - `internal/cli/restart.go` --- title: "CLI reference: VM and workspace commands" description: "Syntax, flags, exit codes, and examples for devvm (up), ls, restart, stop, update, rm, adopt, size, ports, host, code, desktop, and history." --- The `devvm` root command and its VM and workspace subcommands (`ls`, `restart`, `stop`, `update`, `rm`, `adopt`, `size`, `ports`, `host`, `code`, `desktop`, and `history`) manage the lifecycle, resource allocation, port forwarding, editor/desktop connections, and agent history exports for per-project virtual machines. Commands execute through `cli.Execute` and `ui.Main`, validating all flags and positional arguments before running layout version checks or inspecting backend state. ## Global flags, folder resolution, and exit codes Every command inherits persistent flags from the root command, validates arguments and flag values prior to execution, and resolves the target VM from the working directory unless a command operates globally across all VMs (`ls`, `history`) or targets a specific VM by name (`rm VM`, `adopt VM`). ### Persistent and shared flags | Flag | Shorthand | Applies to | Description | | --- | --- | --- | --- | | `--verbose` | `-v` | All commands | Stream external command output (`LoggedCmd`) directly to `stderr` instead of displaying a status spinner. | | `--no-input` | — | All commands | Never prompt interactively; fail immediately at any prompt and print the flag or command that answers the question. | | `--help` | `-h` | All commands | Print command synopsis, examples, flags, and documentation URL. | | `--here` | — | Folder VM commands | Target only the current directory's own VM, ignoring any parent folder VM. Mutually exclusive with `--parent` and `--global` (`-g`). | | `--parent` | — | Folder VM commands | Target the nearest parent directory's VM, crossing nested Git repositories and bypassing the current directory's own VM. Mutually exclusive with `--here`, `--clone`, and `--global` (`-g`). | | `--json` | — | `ls`, `size`, `ports`, `history` | Emit a single indented JSON object with `"version": 1` to `stdout` and route informational messages to `stderr`. | ### Target VM folder resolution Commands that act on a project folder resolve their target VM through `finder.target` and `resolveFolder` using one of three `FolderRule` modes: ```mermaid flowchart TD Start["Command invoked in working directory (dir)"] --> CheckOwn["Read state for project.Hash(dir) or moved folder ID"] CheckOwn --> Rule{"FolderRule (--here / --parent)"} Rule -->|"FolderHere (--here)"| OwnOnly{"Own VM state exists?"} OwnOnly -->|Yes| UseOwn["Target current folder's VM"] OwnOnly -->|No| NoVM["New VM (devvm) or error: no_vm"] Rule -->|"FolderAuto (default)"| HasOwn{"Own VM state exists?"} HasOwn -->|Yes| UseOwn HasOwn -->|No| SearchParent["Walk project.Parents(dir, HOME)"] SearchParent --> FoundParent{"Parent VM exists?"} FoundParent -->|No| NoVM FoundParent -->|Yes| NestedGit{"Nested .git between dir and parent?"} NestedGit -->|Yes| NoVM NestedGit -->|No| UseParent["Target parent VM (prints UsingParentVM)"] Rule -->|"FolderParent (--parent)"| ForceParent["Walk project.Parents(dir, HOME) ignoring nested .git"] ForceParent --> HasParent{"Parent VM exists?"} HasParent -->|Yes| UseParent HasParent -->|No| ErrNoParent["Error: no_parent_vm"] ``` - **Default (`FolderAuto`)**: Uses the current folder's own VM if `vm.json` exists or the backend holds the instance. If the folder has no VM and was moved or renamed from another path on the same filesystem (`platform.FolderID` match), `devvm` prompts to move the VM record to the new path. Otherwise, it walks parent directories up to (but excluding) `$HOME` and `/`, selecting the nearest parent VM unless a nested `.git` directory or file sits between the current directory and that parent. - **`--here` (`FolderHere`)**: Restricts lookup to the current directory only. - **`--parent` (`FolderParent`)**: Forces lookup of the nearest parent folder with a VM, even if the current folder has its own VM or a nested `.git` repository lies in between. Fails with `no_parent_vm` if no parent folder has a VM. ### Exit codes and pre-execution checks | Exit code | Meaning | Trigger conditions | | --- | --- | --- | | `0` | Success | Command completed, or the user declined an interactive confirmation prompt (`[y/N]`) that leaves the VM unchanged. | | `1` | Runtime failure | `RunE` returned an error, `state.CheckLayout` detected an incompatible state directory version, a prompt was refused under `--no-input` or non-TTY `stdin`, or a backend command failed. | | `2` | Usage error | Unknown command or flag, blank flag argument (`--flag ""`), invalid flag value (`--cpus`, `--memory`, `--disk`, `--ports`, `--network`, `--tools`, `--agents`, `--layers`), mutually exclusive flags, or wrong positional argument count. Prints the error and points to ` --help`. | | `130` | Interrupted | Process received `SIGINT`, `SIGTERM`, or `SIGHUP`. Cancels root context, waits for active logged commands and up to `2s` (`returnGrace`) when cleanups are registered, runs the cleanup stack, and suppresses post-interrupt error output. | Before any command logic or VM lookup runs, `checkCommand` rejects empty string arguments or blank string flag values (`noBlanks`), validates mutually exclusive flag groups, and parses typed flag values. Passing a toolchain, agent, or layer name as a bare positional argument to `devvm` (such as `devvm node` or `devvm node+claude`) exits with code `2` and prints the exact `--tools`, `--agents`, or `--layers` flags to use instead. --- ## `devvm` (up) Opens an interactive shell in the current folder's VM at the matching guest path, creating and provisioning the VM first if the folder does not yet have one. ```bash devvm [flags] ``` ### Behavior 1. **New VM creation**: - Checks whether the folder is safe to mount (`checkUnsafe` refuses `$HOME`, directories holding `$HOME` or `devvm` state/config/Lima directories, or directories inside `devvm`'s own folders unless `--unsafe-folder` is passed). - Evaluates `.devcontainer/devcontainer.json` or `.devcontainer.json` unless `--no-devcontainer` is set. - On first run (when base features or default agents are unconfigured in `settings.json` / `config.toml`), prompts in the terminal for default VM size, default coding agents, and base image features. - Displays a summary of the planned VM (image layers, detected and selected toolchains, coding agents, network tier, forwarded ports, and CPU/memory/disk size) and offers an interactive picker (`Customize? [y/N]`) when running in a terminal with unflagged options. - Builds the base and layer image chain and any missing read-only EROFS/ext4 shared tool images, holding a creation lock (`holdCreate`) and tool-making lock (`tools.Making.Hold`) so concurrent `gc` or retention passes do not delete in-progress artifacts. - Protects `.git` read-only (unless `--no-git-protect` or `--clone`), starts the VM and host proxy (`devvm _proxy`), offers to import the newest saved agent history export for this folder if the VM includes a coding agent, syncs guest environment scripts, updates `~/.local/share/devvm/ssh/config`, runs `devcontainer.json` lifecycle commands, and replaces the `devvm` process (`e.Exec`) with the guest SSH shell. - If interrupted (`Ctrl+C`) before the first shell opens, the registered cleanup deletes the partially created VM, removes its state and proxy, and unlocks `.git`. 2. **Existing VM launch**: - If a prior remake (`devvm update`, `devvm layer add/rm`, `devvm clone`) was interrupted before completion, `devvm` resumes and finishes the pending remake first. - Warns if the VM's image chain or shared tools are outdated compared to current definitions or locks. - Warns and ignores creation-only flags (`--tools`, `--agents`, `--layers`, `--network`, `--no-git-protect`, `--cpus`, `--memory`, `--disk`, `--ports`, `--ssh-agent`, `--from`, `--no-devcontainer`), printing the corresponding subcommand (`devvm tools`, `devvm size`, etc.) to modify the existing VM. Passing `--clone` on an existing folder-mounted VM fails and directs you to `devvm clone on`. ### Flags Comma-separated `LIST` of shared toolchains for a new VM (`node`, `python`, `rust`, `go`, `dotnet`), or `none` alone. Passing an agent or layer name to `--tools` fails with a usage error naming `--agents` or `--layers`. Comma-separated `LIST` of coding agents for a new VM (`claude`, `codex`, `opencode`), or `none` alone. Comma-separated `LIST` of image layers for a new VM, ordered bottom-first (built-ins: `docker`, `nix`, `desktop`, plus custom layers in `~/.config/devvm/layers/`), or `none` alone. OCI/Docker container `IMAGE` reference (such as `node:24`) to build the base image from instead of Ubuntu's cloud image. Network confinement `TIER` for a new VM: `open`, `internet`, `allowlist`, or `none`. Whole number `N` of virtual CPUs (`>= 1`, up to host CPU count). Memory allocation in `GiB` (such as `16` or `16G`; decimals allowed, `>= 0.5`, capped at `backend.MaxMemory` of host RAM). Disk size in `GiB` (such as `100` or `100G`; at least the base image size). Guest ports to forward to host `127.0.0.1`: `all`, `none`, or a comma-separated list of ports and ranges (`1`–`65535`) such as `5173,8000-8010`. Create the VM with an isolated Git clone of the current repository inside the VM disk instead of mounting the host project directory. Mutually exclusive with `--parent`; implies `--here`. Allow mounting a directory that `devvm` normally refuses (such as `$HOME` or a parent of `$HOME`). Create the VM without read-only `.git` mount protection. Create the VM with host `SSH_AUTH_SOCK` agent forwarding enabled (`on`). Ignore `.devcontainer/devcontainer.json` and `.devcontainer.json` when creating the VM. Add `Include ~/.local/share/devvm/ssh/config` to `~/.ssh/config` without prompting. ### Examples ```bash # Open this folder's VM, interactively choosing parts if creating a new VM devvm # Create a VM with Claude Code and auto-detected project toolchains devvm --agents claude # Create a VM with Node.js, no coding agents, and the docker layer only devvm --tools node --agents none --layers docker # Create a VM with an isolated repo clone and 16 GiB RAM devvm --clone --memory 16 ``` --- ## `devvm ls` Lists all project VMs across all registered backends, sorted alphabetically by VM name. ```bash devvm ls [--json] ``` ### Columns and JSON schema In terminal/table output, `devvm ls` prints 8 columns (`-` when empty or unset): | Column | JSON field | Type | Values and semantics | | --- | --- | --- | --- | | `NAME` | `vms[].name` | `string` | Full VM identifier (`devvm--<14-hex>`). | | `STATUS` | `vms[].status` | `string` | `Running` (green), `Stopped` (yellow), or `Broken` / other backend status (red). | | `IMAGE` | `vms[].image`, `vms[].from` | `string` | Layer chain name (e.g., `base`, `docker`, `docker+nix`). In table output, formats as ` (from )` when `from` is non-empty. | | `TOOLS` | `vms[].tools` | `[]string` | Shared toolchains and coding agents mounted in the VM (comma-separated in table output; `-` if empty). | | `GIT` | `vms[].git` | `string` | `.git` protection mode: `on`, `strict`, `clone` (green), `off`, or `""`. | | `NET` | `vms[].net` | `string` | Network policy tier: `allowlist`, `internet`, `none` (green), `open`, or `unenforced` (red). | | `LINKS` | `vms[].links` | `number` | Count of outbound links configured from this VM to other VMs (`-` in table output when `0`). | | `FOLDER` | `vms[].folder`, `vms[].folder_state` | `string` | Host project path. In table output, appends `(gone)` or `(not connected)` in yellow when `folder_state` is `"gone"` or `"not_connected"`; normal when `"there"`. | | — | `vms[].backend` | `string` | Backend driver name (e.g., `"lima"`). Included in `--json` output. | If any `vm.json` state file under `~/.local/share/devvm/vms/` is unreadable or corrupt, `devvm ls` emits a warning to `stderr` (`vm_state_unreadable`) for each unreadable file and lists the remaining VMs. ```bash title="Table output" devvm ls # NAME STATUS IMAGE TOOLS GIT NET LINKS FOLDER # devvm-api-0123456789abcd Running docker go,claude on allowlist 1 /Users/alex/src/api # devvm-web-71a99b1076d92c Stopped base node,claude on allowlist - /Users/alex/src/web (gone) ``` ```json title="devvm ls --json" { "version": 1, "vms": [ { "name": "devvm-api-0123456789abcd", "status": "Running", "image": "docker", "tools": [ "go", "claude" ], "git": "on", "net": "allowlist", "backend": "lima", "folder": "/Users/alex/src/api", "folder_state": "there", "links": 1 } ] } ``` --- ## `devvm restart` and `devvm stop` Control the running state of the current folder's VM. Both commands accept `--here` and `--parent` and take no positional arguments. ```bash devvm restart [--here | --parent] devvm stop [--here | --parent] ``` ### `devvm restart` 1. Resolves the target VM (must be a completed VM; fails if no VM exists or if a remake is pending). 2. Stops the VM instance via the backend (`t.stop`). 3. Runs the full existing-VM launch sequence (`open`): checks `devcontainer.json`, brings the VM up with any pending configuration changes (new CPU/memory/disk size, updated agent directory mounts, SSH agent forwarding mode, network tier confinement, and port forwards), syncs guest scripts, updates SSH config and links, and replaces the process with an interactive shell in the VM. ### `devvm stop` 1. Resolves the target VM. If the VM is already stopped, prints `==> is already stopped` (`already_stopped`); otherwise stops the backend instance and prints `==> Stopped `. 2. Stops the host egress proxy (`e.Proxy.Stop`), port forwarders (`e.Forwarders.Stop`), and inter-VM link sessions (`e.linkRuns.stop`). 3. Applies any `.git` protection mode queued while the VM was running, re-locks `.git` on the host, runs host safety checks (including set-id file checks on Linux), and removes the VM's entry from `~/.local/share/devvm/ssh/config`. --- ## `devvm update` Updates the current folder's VM to the newest shared tool versions and, if its image chain or `devcontainer.json` has changed since creation, remakes the VM from the updated image while preserving its home directory and settings, then opens a shell in it. ```bash devvm update [--force] [--no-export] [--clean-home] [--here | --parent] ``` ### Execution flow 1. **Pending remake recovery**: If a prior remake on the VM was interrupted, finishes the pending remake first while holding `RemakeLock`. 2. **Clone restriction**: Refuses `--clone` VMs (`remake_clone`) because the repository clone resides on the VM disk; only `devvm clone` can remake a clone VM. 3. **Outdated check (`updatePlan`)**: - Checks whether the VM's base or layer image build chain (`st.Chain`) differs from current image definitions (`Builds.Outdated`). - Re-reads `.devcontainer/devcontainer.json` or `.devcontainer.json` (`devcontainers.reread`) to detect changes to the container base (`From`, falling back to `$DEVVM_FROM` or Ubuntu cloud image if removed), toolchains/agents, forwarded ports, or lifecycle commands. 4. **Confirmation**: If an image or `devcontainer.json` remake is needed, validates backend capabilities and disk space, then prompts in the terminal (`[y/N]`) before modifying anything. `--force` answers yes automatically. 5. **Shared tools update**: Updates mounted shared tools to their newest lock versions (`e.vmTools.update`), which takes effect on launch without remaking the VM disk. 6. **Remake and launch**: - If a remake was agreed to: builds the new image and any reread tools, exports agent history to `~/.local/share/devvm/exports/` (unless `--no-export`), archives `/home/` from the running VM (unless `--clean-home`), stops the VM and renames the old machine aside (`-old`), creates and starts the new machine, restores `/home/`, deletes the old machine, and opens a shell. Until `/home/` is restored and committed, any failure or interrupt rolls back to the old machine (`putBack`). - If only shared tools were updated (or the user declined the image remake after tools updated), launches the existing VM with the newer tool mounts. - If neither tools nor images changed, prints that the VM is current (`vm_current`) and exits `0`. ### Flags (`-f`) Remake the VM without prompting for confirmation. Required when running non-interactively or with `--no-input` if an image or `devcontainer.json` change requires a remake. Does not bypass the parent-VM confirmation prompt when run from a subfolder. Skip exporting coding agents' session history to `~/.local/share/devvm/exports/` before replacing the VM disk. Start the remade VM with a fresh home directory from the image instead of copying the existing VM's home directory. --- ## `devvm rm` Deletes the current folder's VM, or deletes a named VM whose project folder no longer exists on the host. ```bash devvm rm [VM] [--force] [--no-export] [--no-clone-check] [--here | --parent] ``` ### Target modes and safeguards - **Current folder mode (`devvm rm`)**: - Acquires the creation lock (`holdForRm`) so it never deletes a VM that another `devvm` process is actively creating. - Prompts for confirmation (`[y/N]`), which `--force` answers for the current folder's own VM. - **Non-overridable prompts**: If `devvm rm` resolves to a parent folder's VM from a subdirectory (`t.sub != ""`), or if the VM at this path was created for an earlier deleted folder with a different inode/FolderID (`t.other`), `devvm` always prompts interactively and `--force` does not bypass the prompt. - **Clone check**: If the VM is a `--clone` VM, inspects the guest clone for uncommitted changes and commits not yet fetched into the host repository, offering to save them to `saved/devvm/` on the host before deletion unless `--no-clone-check` is passed. - **Agent history export**: If the VM has coding agents installed and finished creation, starts the VM (if stopped) and exports agent transcripts/sessions to `~/.local/share/devvm/exports/` unless `--no-export` is set. If the export fails, the VM is kept (`export_failed_kept`); if the export exceeds `2 GiB` (`historyMax`), `devvm` warns and proceeds without saving the oversized archive. - **Teardown**: Deletes the backend VM instance, state directory (`~/.local/share/devvm/vms//`), proxy, port forwarders, link sessions, `.git` protection locks or `devvm` git remote, inbound links from other VMs, pinned SSH host keys, and SSH config entry. - **Named VM mode (`devvm rm VM`)**: - Accepts a VM name (e.g., `devvm-web-71a99b1076d92c`) or SSH alias (e.g., `devvm-web`). Cannot be combined with `--here` or `--parent` (`rm_vm_folder_flag`). - If `VM` is the current directory's own VM, behaves identically to `devvm rm`. - If `VM`'s recorded project folder still exists (`FolderThere`), refuses (`rm_vm_folder_there`) and instructs you to run `devvm rm` inside that folder. - If `VM`'s folder is on an unmounted external volume (`FolderNotConnected`), prompts interactively; `--force` never answers this prompt (`rm_vm_not_connected`). - If `VM` is a `--clone` VM whose host repo is gone, requires `--no-clone-check` (`rm_gone_clone`). - If `VM` has coding agents and is stopped, refuses unless `--no-export` is given (`rm_gone_stopped`), because starting a stopped folder-mounted VM would cause the backend to recreate the missing host folder. ### Flags (`-f`) Skip confirmation when deleting the current folder's own VM or a named VM whose folder is permanently gone (`FolderGone`). Never skips confirmation for a parent folder's VM, an earlier folder's VM at the same path, or a VM whose folder is on a disconnected disk (`FolderNotConnected`). Do not export coding agents' history before deleting the VM. Required when deleting a stopped named VM (`devvm rm VM`) that has coding agents and a missing host folder. Do not check a `--clone` VM for uncommitted changes or unfetched branches before deletion (unfetched work is permanently lost). Required when deleting a `--clone` VM whose host repository is gone. ### Examples ```bash # Delete this folder's VM interactively devvm rm # Delete this folder's VM in a script without prompting devvm rm --force # Delete a stopped VM whose project folder was deleted, skipping history export devvm rm devvm-web-71a99b1076d92c --no-export --force ``` --- ## `devvm adopt` Associates the current folder with a VM that was created for an earlier folder at the same path, a folder that was moved or renamed, or a named VM whose original folder is gone (such as a folder copied to another disk). ```bash devvm adopt [VM] [--force] ``` ### Behavior `devvm adopt` always acts on the current directory (`own: true`) and does not accept `--here` or `--parent`. 1. **Without `VM` (`devvm adopt`)**: - Checks if a VM's folder was moved or renamed to the current path on the same filesystem (`moveHere` matches `st.Folder == platform.FolderID(dir)` when the old path is gone or is a symlink to the new path). If found, stops the VM if running, updates `st.Dir`, records the path transition in `movedSection` so agent state paths update on next launch, and preserves `.git` protection state. - Otherwise, if a VM exists for the current path's hash (`project.Hash(dir)`) but its recorded `FolderID` belongs to an earlier deleted directory (`t.other`), prompts to adopt the VM (`[y/N]`, answered by `--force`), updates `st.Folder` to the new directory's ID, and resets `.git` protection state (`gitprotect.Made{}`) so `.git` is locked fresh on next launch. - If the current folder already owns the VM, prints `==> is already this folder's VM` (`vm_already_here`) and exits `0`. 2. **With `VM` (`devvm adopt VM`)**: - Looks up `VM` by full name or SSH alias. - Refuses if `VM` is a `--clone` VM (`adopt_clone`), if the backend lost the instance (`rm_vm_lost`), if `VM` creation is unfinished (`vm_unfinished`), if `VM`'s recorded folder still exists on disk (`adopt_folder_there`), or if the current folder already has its own VM (`adopt_has_vm`). - Checks that the current directory is not an unsafe folder (`$HOME` or `devvm` directories). - Prompts for confirmation (`[y/N]`, answered by `--force`), stops `VM` if running, updates `st.Dir` and `st.Folder` to the current folder, and resets `.git` protection state so the new folder's `.git` is locked fresh on next start. --- ## `devvm size` Displays or updates the CPU, memory, and disk allocation for the current folder's VM, or with `-g` (`--global`) for new VMs. ```bash devvm size [--cpus N] [--memory GiB] [--disk GiB] [-g | --here | --parent] [--json] ``` ### Behavior and constraints - **Showing size (`devvm size` with no size flags)**: - For a folder VM: prints active `cpus= memory=GiB disk=GiB` to `stdout`. If changes are queued in `vm.json` for the next boot (`pendingSize`), emits a note on `stderr` showing the pending values. - With `-g`: prints three lines (`cpus=`, `memory=`, `disk=`) with `# ` comments indicating whether each default comes from `~/.config/devvm/config.toml`, `devvm: settings.json`, or `devvm: default` (image default). - **Changing size (`--cpus`, `--memory`, `--disk`)**: - Validates against host capacity (`ThisHost`): `--cpus` cannot exceed host logical CPUs (`too_many_cpus`), and `--memory` cannot exceed `backend.MaxMemory` (`too_much_memory`). - **Disk growth only**: For an existing VM, `--disk` must be greater than or equal to the VM's current disk size (`disk_shrink`); for `-g`, `--disk` must be at least `backend.BaseDiskGiB`. - For a running VM, size changes take effect on next start; `devvm` prompts in the terminal (`Restart now? [Y/n]`) and runs `devvm restart` immediately if confirmed, or warns that the new size applies on next start. - With `-g`, writes to `~/.local/share/devvm/settings.json`. Fails with `set_in_config` if the target key (`size.cpus`, `size.memory`, or `size.disk`) is pinned in `~/.config/devvm/config.toml`. - `--json` cannot be combined with `--cpus`, `--memory`, or `--disk` (`showsOnly` exits with code `2`). ### Flags Whole number `N` of CPUs (`>= 1`). Memory in `GiB`, such as `16` or `16G` (`>= 0.5`). Disk size in `GiB`, such as `200` or `200G` (cannot shrink an existing disk). (`-g`) Show or change default size settings for new VMs in `settings.json` instead of this folder's VM. Mutually exclusive with `--here` and `--parent`. Print the current or global size as a version 1 JSON object. Cannot be combined with `--cpus`, `--memory`, or `--disk`. ### Output examples ```bash title="Folder VM and global text output" devvm size # cpus=4 memory=8GiB disk=100GiB devvm size -g # cpus= # devvm: default # memory=16 # /Users/alex/.config/devvm/config.toml # disk=100 # devvm: settings.json ``` ```json title="devvm size --json (with pending restart changes)" { "version": 1, "cpus": 4, "memory_gib": 8, "disk_gib": 100, "pending": { "cpus": 6, "memory_gib": 16 } } ``` ```json title="devvm size -g --json" { "version": 1, "memory_gib": 16, "disk_gib": 100, "sources": { "cpus": "built-in", "disk": "settings.json", "memory": "config.toml" } } ``` --- ## `devvm ports` Displays or updates which TCP ports the current folder's VM forwards from the guest to `127.0.0.1` on the host, or with `-g` (`--global`) what new VMs forward by default. Changes to a running VM restart the host SSH port forwarder (`vmssh.Forwards.Start`) immediately without restarting the VM. ```bash devvm ports [all|none|LIST] [-g | --here | --parent] [--json] devvm ports add LIST [-g | --here | --parent] devvm ports rm LIST [-g | --here | --parent] ``` ### Subcommands and arguments | Command | Argument | Behavior | | --- | --- | --- | | `devvm ports` | None | Prints `all`, `none`, or the normalized comma-separated list of ports/ranges (e.g., `5173,8000-8010`). With `-g`, appends `# `. | | `devvm ports [all\|none\|LIST]` | `all`, `none`, or `LIST` | Replaces the entire port forwarding configuration at once. Cannot be combined with `--json`. | | `devvm ports add LIST` | `LIST` (e.g., `5432,8000-8010`) | Merges ports or ranges into the current list. Rejects `all` or `none` as `LIST` (`ports_list_needed`). If the VM already forwards `all`, warns (`ports_all_already`) and keeps `all`. | | `devvm ports rm LIST` | `LIST` (e.g., `5173`) | Removes exact matching port or range entries from the current list; transitions to `none` when the last entry is removed. Fails if the VM currently forwards `all` (`ports_rm_all`) or if an entry in `LIST` is not present as listed (`port_not_listed`). | ```bash title="Examples" # Show forwarded ports for this folder's VM devvm ports # Restrict forwarding to specific ports and ranges immediately devvm ports 5173,8000-8010 # Add PostgreSQL port forwarding while the VM runs devvm ports add 5432 # Change the default for new VMs to forward no ports devvm ports -g none ``` ```json title="devvm ports --json & devvm ports -g --json" // devvm ports --json { "version": 1, "ports": "5173,8000-8010" } // devvm ports -g --json { "version": 1, "ports": "all", "source": "built-in" } ``` --- ## `devvm host`, `devvm code`, and `devvm desktop` Prepare the current folder's VM (starting it if stopped, syncing guest scripts, and writing `~/.local/share/devvm/ssh/config`) and connect external tools over SSH or VNC. | Command | Syntax | Description | | --- | --- | --- | | `devvm host` | `devvm host [--ssh-config] [--here \| --parent]` | Starts the VM if stopped, ensures `~/.ssh/config` includes `devvm`'s SSH config (prompting unless `--ssh-config` is passed), and prints the SSH host alias (e.g., `devvm-website`) and guest working directory (`shellDir`). | | `devvm code` | `devvm code [--ssh-config] [--here \| --parent]` | Runs the same preparation as `devvm host`, locates the `code` binary on `$PATH` (failing with `no_code_command` if missing), and replaces the `devvm` process with `code --remote ssh-remote+ `. | | `devvm desktop` | `devvm desktop [--clipboard] [--no-open] [--here \| --parent]` | Starts the VM, verifies the `desktop` layer is installed (`/usr/local/bin/devvm-desktop`), starts the guest desktop service, and runs a host VNC relay on `127.0.0.1:` until `Ctrl+C` or VM shutdown. | ### `devvm desktop` relay architecture and flags `devvm desktop` requires a VM built with the `desktop` layer (`devvm --layers desktop` or `devvm layer add desktop`). It never exposes the guest VNC socket directly on the network: 1. **Per-VM password**: Generates a random password on first run (`desktop.NewPassword`), stores it in the `desktop` section of `vm.json` on the host, and validates VNC client handshakes (`15s` timeout, `1s` failure delay) in the host relay (`internal/desktop.Relay`) before opening a channel to the VM. 2. **Isolated SSH socket channel**: Upon successful password verification, the host relay dials the guest Unix socket (`/run/devvm-desktop.sock`) via `ssh -W` using `BatchMode=yes`, `IdentityAgent=none`, `ForwardAgent=no`, and `ClearAllForwardings=yes`. 3. **Sticky loopback port**: Allocates and persists a dedicated loopback port (`127.0.0.1:`) in `vm.json` (`ports.Host.Desktop`) so VNC viewers reconnect to the same address across runs. 4. **Health polling**: Polls backend status every `5s` (`desktopEvery`) and terminates cleanly if the VM stops. Share the clipboard bidirectionally between the host and the VM desktop. By default (`false`), the host VNC relay strips `ClientCutText` and `ServerCutText` messages and starts `devvm-desktop` with clipboard sync `off`. Print the `127.0.0.1:` address and VNC password without automatically launching macOS Screen Sharing (`vnc://127.0.0.1:`). --- ## `devvm history` Lists, exports, and imports tarballs of coding agents' session transcripts and history without including agent credentials or login tokens. ```bash devvm history [--json] devvm history export [FILE] [--here | --parent] devvm history import [FILE] [--here | --parent] ``` ### Storage layout and behavior Exports managed by `devvm` live in `~/.local/share/devvm/exports/` (permissions `0700`) with filenames matching `-YYYYMMDD-HHMMSS.tar.gz` (in host local time, mode `0600`) paired with a sidecar metadata file `.folder` containing two lines: the host project directory path and the VM's network tier at export time. - **`devvm history`**: - Lists saved exports in `~/.local/share/devvm/exports/` sorted oldest-first (`FILE`, `VM`, `FOLDER`, `DATE`, `SIZE`). Starts no VM and modifies no state. - **`devvm history export [FILE]`**: - Starts the folder's VM if stopped (`ensureRunning`). - Streams a `.tar.gz` of the guest's agent history directories (excluding login/credential files) to a temporary file `.devvm-export-*` capped at `2 GiB` (`historyMax`), then atomically renames it to `FILE` (if specified) or to `~/.local/share/devvm/exports/-.tar.gz` along with its `.folder` sidecar. - **`devvm history import [FILE]`**: - Selects `FILE` if given, or locates the newest export in `~/.local/share/devvm/exports/` whose `.folder` sidecar matches the current VM's folder (`Exports.Newest`). Fails with `history_no_export` if none exists. - **Network tier downgrade warning (`warnTier`)**: Compares the export's recorded source network tier against the target VM's tier (`netpolicy.Tier.AtLeast`). If the export came from a less strict tier (e.g., `open` or `internet` imported into an `allowlist` or `none` VM) or from an external `FILE` without a trusted sidecar, warns on `stderr` before importing. - Starts the VM if stopped and streams the tarball into guest `tar` extraction without reading the archive on the host, preserving existing files in the VM and rejecting unsafe archive paths (`historyUnsafe`). ```bash title="Export and import commands" # List all saved agent history exports devvm history # Export this folder's VM agent history to the default exports store devvm history export # Export to a custom archive path devvm history export ~/backups/api-agents.tar.gz # Import the newest export saved for this folder's VM devvm history import ``` ```json title="devvm history --json" { "version": 1, "exports": [ { "file": "/Users/alex/.local/share/devvm/exports/devvm-api-0123456789abcd-20261001-120000.tar.gz", "vm": "devvm-api-0123456789abcd", "folder": "/Users/alex/src/api", "date": "2026-10-01T12:00:00Z", "size": 148290 } ] } ``` ## Related pages How project paths map to VM names and 8-hex hashes, state persistence in vm.json, layout version checks, and in-place vs. remake transitions. Syntax, subcommands, and flags for devvm network, secret, agent, git-protect, clone, ssh-agent, link, and group. Syntax, subcommands, and flags for devvm build, rebuild, images, layer, tools, space, gc, retention, setup, doctor, upgrade, completion, and limactl. Complete reference for ~/.config/devvm/config.toml keys, settings.json precedence, environment variables, state directory layout, and version 1 --json payloads. Use generated ~/.ssh/config entries, configure per-VM SSH agents, launch VS Code Remote-SSH via devvm code, and connect to the VNC desktop relay. Create VMs with independent Git clones using devvm --clone, switch existing VMs between mounted folders and clones, and fetch VM branches to the host. --- ## 16. CLI reference: network, security, and agent commands > Syntax, subcommands, and flags for devvm network, secret, agent, git-protect, clone, ssh-agent, link, and group. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/16-cli-reference-network-security-and-agent-commands.md - Generated: 2026-11-07T22:34:27.553Z ### Source Files - `internal/cli/network.go` - `internal/cli/secret.go` - `internal/cli/agent.go` - `internal/cli/agentlogin.go` - `internal/cli/gitprotect.go` - `internal/cli/sshagent.go` - `internal/cli/link.go` - `internal/cli/group.go` --- title: "CLI reference: network, security, and agent commands" description: "Syntax, subcommands, and flags for devvm network, secret, agent, git-protect, clone, ssh-agent, link, and group." --- The network, security, and agent commands in `internal/cli` configure per-VM network confinement tiers, host-injected secrets, coding agent credentials and settings stores, `.git` hook and config immutability locks, isolated repository clones, SSH agent forwarding, and inter-VM port links and groups. Commands that target the current project folder's VM resolve the VM through `vm.Env` and update either the VM's state file (`~/.local/share/devvm/vms//vm.json`) or host-wide stores under `~/.config/devvm` and `~/.local/share/devvm`. ## Command inventory and shared conventions All commands in this group share global persistent flags, exit codes, and—for commands that operate on a project folder's VM—folder-resolution flags. | Command | Subcommands | Scope | Supports `--json` | | --- | --- | --- | --- | | `devvm network` | `allow`, `deny`, `rm`, `log` | Project VM or host default (`-g`) | `network`, `network -g`, `network log` | | `devvm secret` | `set`, `rm`, `use`, `unuse` | Host secret store and project VM bindings | `secret` (root only) | | `devvm agent` | `login`, `import`, `save` | Host agent store and project VM | No | | `devvm git-protect` | None | Project VM (`.git` locks on host) | `git-protect` (when showing) | | `devvm clone` | None | Project VM (remake with/without repo clone) | `clone` (when showing) | | `devvm ssh-agent` | `add`, `rm` | Project VM forwarding mode and `devvm` SSH agent | `ssh-agent` (when showing) | | `devvm link` | `add`, `rm`, `publish`, `unpublish` | Project VM links and published ports | `link`, `link publish` (when showing) | | `devvm group` | `join`, `leave` | Project VM group membership and auto-links | `group` (root only) | ### Shared flags and exit codes Inherited global flag on all commands. Stream the full output of external commands `devvm` runs instead of displaying a single status line. Inherited global flag on all commands. Never prompt interactively; fail immediately where `devvm` would ask a question and name the flag that answers it (such as `--force` or `--stdin`). Available on commands that act on the current folder's VM. Use this folder's own VM, never inheriting the VM of an enclosing parent folder. Mutually exclusive with `--parent` and `--global` (`-g`). Available on commands that act on the current folder's VM. Use the VM of the nearest enclosing parent folder that has one, even across a nested Git repository boundary. Mutually exclusive with `--here` and `--global` (`-g`). Available on inspection and listing commands. Print output as a single indented JSON object with `"version": 1` as the first key. On commands that both show a setting when run bare and change it when given an argument or mutating flag (`network`, `git-protect`, `clone`, `ssh-agent`, `link publish`), passing `--json` together with a positional value or mutating flag is refused as a usage error (`showsOnly`). | Exit code | Meaning | | --- | --- | | `0` | Command succeeded, or an interactive confirmation prompt was declined cleanly. | | `1` | Runtime failure, backend error, layout version check failure, or non-interactive prompt refusal (`--no-input`). | | `2` | Usage error: unknown command or flag, invalid argument or flag value, blank string argument/flag (`noBlanks`), or mutually exclusive flags combined. | | `130` | Interrupted by `SIGINT`, `SIGTERM`, or `SIGHUP`. | ```mermaid flowchart LR subgraph CLI["internal/cli Commands"] NetCmd["network / secret"] AgentCmd["agent (login, import, save)"] IsoCmd["git-protect / clone / ssh-agent"] LinkCmd["link / group"] end subgraph Services["internal/vm Services"] VMNet["vm.Net"] VMAgent["vm.AgentLogin / AgentImport / AgentSave"] VMSettings["vm.Settings / vm.Lifecycle"] VMLinks["vm.Links / vm.Groups"] end subgraph Storage["Host Stores & Per-VM State"] PolicyFiles["~/.config/devvm/policy\n~/.local/share/devvm/vms//policy"] SecretStore["macOS Keychain / ~/.local/share/devvm/secrets\n~/.local/share/devvm/logins/"] AgentStore["~/.config/devvm/agents/\n~/.local/share/devvm/agent-keys.json"] VMState["~/.local/share/devvm/vms//vm.json\n(net, secrets, git, clone, agent, links)"] end NetCmd --> VMNet AgentCmd --> VMAgent IsoCmd --> VMSettings LinkCmd --> VMLinks VMNet --> PolicyFiles VMNet --> SecretStore VMNet --> VMState VMAgent --> SecretStore VMAgent --> AgentStore VMSettings --> AgentStore VMSettings --> VMState VMLinks --> VMState ``` --- ## `devvm network` Show or change the network confinement tier of the current folder's VM, manage allow and deny policy rules, or inspect the proxy connection log. ```bash devvm network [open|internet|allowlist|none] [-g] [--json] [--here | --parent] ``` ### Network tiers and transitions | Tier | Traffic behavior | Transition behavior on a running VM | | --- | --- | --- | | `allowlist` | Default for new VMs. Allows built-in base, shared-tool, and layer policies plus user and per-VM rules. Denies beat allows. | Switching between `allowlist`, `internet`, and `none` reloads the host proxy (`devvm _proxy`) immediately without restarting the VM. | | `internet` | Allows any public host on any port (`network/internet.policy`) plus secret hosts; only deny rules (`!rule`) from user and layer policy files apply. | Immediate proxy reload when switching to or from `allowlist` or `none`. | | `none` | Allows only the token endpoints and API hosts required by the VM's installed coding agents and bound secrets; only deny rules from user and layer policy files apply. | Immediate proxy reload when switching to or from `allowlist` or `internet`. | | `open` | Unconfined direct networking without forcing traffic through the host policy proxy. | Switching into or out of `open` takes effect on the VM's next start; in an interactive terminal, `devvm` offers to restart a running VM immediately. | When `devvm network` (or `devvm ls`) inspects a running VM whose confined tier failed its post-boot confinement self-check, it reports the policy as `unenforced` instead of the tier name. Changing a VM's tier is refused if the VM is currently in a group (`devvm group`) or if the new tier would violate the tier invariant of an active link (`target.tier >= client.tier`). ### Root flags Show or change the default network tier saved in `settings.json` for newly created VMs instead of acting on the current folder's VM. Fails if `network` is pinned in `~/.config/devvm/config.toml`. Mutually exclusive with `--here` and `--parent`. Output the current VM's policy and ordered rule files (or, with `-g`, the default tier and its config source) as version 1 JSON. Refused when a tier argument is provided. Target this folder's own VM. Target the nearest enclosing parent folder's VM. ### Subcommands #### `devvm network allow` ```bash devvm network allow RULE... [-g] [--here | --parent] ``` Add one or more allow `RULE` entries to this folder's VM policy file (`~/.local/share/devvm/vms//policy`), or with `-g` (`--global`) to the user-wide policy file (`~/.config/devvm/policy`). Changes apply on the proxy's next connection. - **Rule syntax (`RULE`):** - Exact hostname: `example.com` (matches ports `443` and `80` by default). - Subdomain wildcard: `'*.example.com'`. - Any host wildcard: `'**'`. - IPv4/IPv6 address or CIDR block: `10.0.0.5` or `10.0.0.0/8`. - Port-qualified rule: `example.com:8443` or `10.0.0.5:5432` (matches only that port instead of `443` and `80`). - Passing a rule prefixed with `!` to `devvm network allow` is refused with a usage error. #### `devvm network deny` ```bash devvm network deny RULE... [-g] [--here | --parent] ``` Add one or more deny rules to the VM's policy file, or with `-g` (`--global`) to `~/.config/devvm/policy`. `devvm` automatically prepends `!` if not already present. Deny rules override any allow rule across all confined tiers (`allowlist`, `internet`, and `none`) from the next connection onward. #### `devvm network rm` ```bash devvm network rm RULE... [-g] [--here | --parent] ``` Remove matching rules from the VM's own policy file, or with `-g` (`--global`) from `~/.config/devvm/policy`. To remove a deny rule, include its leading `!` (for example, `devvm network rm '!example.com'`). #### `devvm network log` ```bash devvm network log [-n N] [--follow] [--blocked] [--json] [--here | --parent] ``` Display connection and request events recorded by the VM's host proxy and inter-VM link sessions, ordered oldest first. Number of most recent connection and request events to show. Mutually exclusive with `--blocked`. Stream new log events as they occur until interrupted. Mutually exclusive with `--blocked` and `--json`. Aggregate blocked connections by host and port, showing the last timestamp, block count, and the `devvm network allow` command that permits each host. Mutually exclusive with `--follow` and `--lines`. Print log events (`{"version": 1, "events": [...]}`) or blocked hosts (`{"version": 1, "blocked": [...]}`) as JSON. Mutually exclusive with `--follow`. ### Examples and `--json` payloads ```bash title="Manage tiers and rules" devvm network devvm network internet devvm network -g none devvm network allow example.com '*.example.com' 10.0.0.5:5432 devvm network deny -g '*.telemetry.example.com' devvm network rm '!*.telemetry.example.com' devvm network log --blocked ``` ```json title="devvm network --json" { "version": 1, "policy": "allowlist", "allowlist": [ { "builtin": "layers/base.policy", "rules": [ "archive.ubuntu.com", "security.ubuntu.com" ], "not_rules": [] }, { "file": "/Users/alice/.config/devvm/policy", "denies_only": false, "rules": [ "example.com" ], "not_rules": [ { "line": 2, "text": "invalid rule line" } ] } ] } ``` ```json title="devvm network log --blocked --json" { "version": 1, "blocked": [ { "last": "2026-10-01T12:00:00Z", "count": 3, "host": "registry.npmjs.org", "port": 443, "allow": "devvm network allow registry.npmjs.org" } ] } ``` --- ## `devvm secret` Manage host-stored tokens and bind them to project VMs. A bound VM receives a per-VM salted placeholder in its environment, and its host proxy substitutes the real secret into HTTPS (`443`) requests addressed to the secret's allowed hosts. ```bash devvm secret [--json] ``` ### Known secret names Every `devvm secret` subcommand validates `NAME` against `secrets.Known`: | `NAME` | Purpose | Guest environment variable | Allowed target hosts (`443` only) | VM binding behavior | | --- | --- | --- | --- | --- | | `claude` | Long-lived token printed by `claude setup-token` (must start with `sk-ant-oat01-`). | `CLAUDE_CODE_OAUTH_TOKEN` (placeholder prefix `sk-ant-oat01-devvm-proxy-managed-`) | `api.anthropic.com` | Automatically bound on any VM whose shared tools include `claude`. Can also be bound with `secret use claude`. | | `github` | GitHub personal access token or `gh auth token`. | `GH_TOKEN` (placeholder prefix `devvm-proxy-managed-github-`) | `api.github.com`, `github.com`, `codeload.github.com`, `raw.githubusercontent.com` | Opt-in per VM via `devvm secret use github`. | | `proxy` | Password for the upstream corporate proxy configured in `config.toml`. | None (`HostOnly: true`) | Sent only by `devvm _proxy` to the upstream corporate proxy | Never bound to or exposed inside any VM; `secret use proxy` and `secret unuse proxy` are refused. | Secret values are stored in the macOS Keychain under service `devvm` (or `$DEVVM_SECRET_SERVICE`), or in `0600` files under `~/.local/share/devvm/secrets/` on hosts without Keychain support. Saving a `claude` token removes any stored Claude Code OAuth login so `devvm` retains only the newest Claude credential. ### Subcommands #### `devvm secret set` ```bash devvm secret set NAME [--stdin] ``` Save a secret value on the host. Running VM proxies pick up `claude` and `github` token updates immediately; `proxy` is read when a VM's proxy starts. Tokens cannot be empty or contain double quotes (`"`), backslashes (`\`), or control characters. One of `claude`, `github`, or `proxy`. Read the secret from a single line on standard input instead of prompting interactively. Required when `--no-input` is set or standard input is not a terminal. #### `devvm secret rm` ```bash devvm secret rm NAME [-f | --force] ``` Delete a saved secret from the host store. Running VM proxies stop injecting the secret immediately. One of `claude`, `github`, or `proxy`. Skip the interactive confirmation prompt (required in scripts or with `--no-input`). #### `devvm secret use` and `devvm secret unuse` ```bash devvm secret use NAME [--here | --parent] devvm secret unuse NAME [--here | --parent] ``` Bind (`use`) or unbind (`unuse`) `claude` or `github` on the current folder's VM. - The VM's proxy begins or stops injecting the secret immediately, and updates the VM's `secrets.policy` allowlist. - The guest environment variable placeholder (`GH_TOKEN` or `CLAUDE_CODE_OAUTH_TOKEN`) is populated on the VM's next shell launch. - Running `devvm secret unuse claude` on a VM whose image includes the `claude` shared tool is refused because VMs with Claude Code always bind `claude`. ### Examples and `--json` payload ```bash title="Configure and bind a GitHub token" gh auth token | devvm secret set github --stdin devvm secret use github devvm secret devvm secret unuse github devvm secret rm github --force ``` ```json title="devvm secret --json" { "version": 1, "secrets": [ { "name": "claude", "saved": false, "used_by": [] }, { "name": "github", "saved": true, "used_by": [ "devvm-repo-0123456789abcdef" ] }, { "name": "proxy", "saved": false, "used_by": [] } ] } ``` --- ## `devvm agent` Manage `devvm`'s isolated host store of coding agent settings (`~/.config/devvm/agents/`) and logins (`~/.local/share/devvm/logins/`). VMs never mount or read the agent configuration folders directly from your host home directory (`~/.claude`, `~/.codex`, `~/.config/opencode`). Running `devvm agent` with no subcommand prints its help text. ```bash devvm agent [login | import | save] ``` ### Supported `AGENT` values All `devvm agent` subcommands validate agent names against `claude`, `codex`, and `opencode`. | Agent identifier | Settings store path (`~/.config/devvm/agents//...`) | Login store file (`~/.local/share/devvm/logins//...`) | | --- | --- | --- | | `claude` | `.claude/` (settings, skills, plugins) | `.claude.json` (MCP server secrets and subset) and `.claude/.credentials.json` (OAuth login) | | `codex` | `.codex/` | `.codex/auth.json` | | `opencode` | `.config/opencode/` | `.local/share/opencode/auth.json` | ### Subcommands #### `devvm agent login` ```bash devvm agent login AGENT [--setup-token] [--device-code | --api-key] [--provider PROVIDER] [--stdin] [--here | --parent] ``` Authenticate `AGENT` inside the current folder's VM (starting the VM first if stopped) and capture the resulting credential into `devvm`'s host login store. Every VM with that agent receives salted placeholders instead of raw tokens. The target VM must have `AGENT` installed in its shared tools. - **`claude` login flows:** - Default (`devvm agent login claude`): runs `claude auth login --claudeai` inside the VM (with `CLAUDE_CODE_OAUTH_TOKEN` unset). Displays a URL to open in a browser on any machine; paste the code back into the terminal. The VM proxy captures the OAuth token response into `.claude/.credentials.json` and deletes any saved `claude` setup-token. - `--setup-token`: runs `claude setup-token` inside the VM, captures the long-lived token as the `claude` secret, and deletes any stored Claude OAuth login. - **`codex` and `opencode` login flows:** - Default browser login: temporarily forwards host loopback port `1455` (and fallback `1457`) into the VM while running `codex -c cli_auth_credentials_store="file" login` or `opencode auth login openai --method chatgpt-browser`. The VM's proxy intercepts the token exchange and stores the credentials on the host. - `--device-code`: runs `codex ... login --device-auth` or `opencode auth login openai --method chatgpt-headless` with no loopback port forwarding. - `--api-key`: prompts for an API key on the host (or reads one line with `--stdin`), saves it to the host login store without running the agent's interactive login CLI, writes the placeholder copy into the VM, and—for `opencode`—runs the guest `opencode.sh` import script. `devvm agent login` enforces strict flag combinations before execution: - `--device-code` and `--api-key` are mutually exclusive. - `--setup-token` is only valid with `claude`. - `--device-code`, `--api-key`, and `--stdin` are refused when `AGENT` is `claude`. - `--stdin` is only valid when `--api-key` is also set. - `--provider` is required when `AGENT` is `opencode` with `--api-key`, must be one of `anthropic`, `openai`, or `openrouter`, and is refused without `--api-key` or for other agents. Agent to log in: `claude`, `codex`, or `opencode`. With `claude`, obtain a long-lived Claude Code token via `claude setup-token` instead of an OAuth subscription login. With `codex` or `opencode`, authenticate using a one-time device code in any browser without forwarding local ports. With `codex` or `opencode`, save an API key directly into the host store instead of running a browser OAuth flow. Required with `devvm agent login opencode --api-key`. Target provider: `anthropic`, `openai`, or `openrouter`. With `--api-key`, read the API key from a single line on standard input without prompting. #### `devvm agent import` ```bash devvm agent import [AGENT...] [-f | --force] ``` Copy agent settings and logins from your host home directory into `devvm`'s host store (`~/.config/devvm/agents` and `~/.local/share/devvm/logins`). With no `AGENT` arguments, imports all found agents (`claude`, `codex`, `opencode`). - Never imports conversation history or project state. - Opens each host agent folder via `os.OpenRoot` so symlinks pointing outside the agent's folder (or symlinks to directories) are never followed and are skipped with a warning. - Rewrites host plugin paths in `.claude/plugins/*.json` to point to the `devvm` agent store path. - For `opencode` v2+, runs `opencode auth export` in the host home directory to extract database-stored logins into the store's `auth.json` format. Zero or more of `claude`, `codex`, `opencode`. Defaults to all three. Overwrite existing settings or logins in `devvm`'s store without prompting for confirmation. #### `devvm agent save` ```bash devvm agent save [AGENT...] [--here | --parent] ``` Copy modified agent settings (and for `claude`, the non-history settings, account state, and MCP server subset of `~/.claude.json`) from the current folder's VM back into `devvm`'s host store so all VMs with that agent receive them. - Starts the VM first if it is stopped. - Reads the VM's untrusted stream with a hard byte cap, parses settings files whole, and restores host secret values in place of VM placeholders (`agentlogin.Restore`). Any file containing an unrecognized placeholder or failing to parse is skipped with a warning. - Prints a redacted unified line diff (`diffShowMax = 20000` lines) of all added, modified, and deleted files and asks for confirmation. - Because `devvm agent save` has no `--force` flag, it always requires an interactive terminal without `--no-input`. ```bash title="Agent login, import, and save" devvm agent login claude devvm agent login codex --device-code devvm agent login opencode --api-key --provider anthropic devvm agent import claude codex devvm agent save claude ``` --- ## `devvm git-protect` Show or change `.git` protection for the current folder's VM. On macOS, `.git` protection locks `.git/hooks`, `.git/config`, `.git/config.worktree`, linked worktree `commondir` files, and submodule Git directories using the user-immutable file flag (`chflags uchg`) so processes inside a VM mounting the folder read-write cannot plant hooks or config directives that execute code on the host. ```bash devvm git-protect [on|strict|off] [--json] [--here | --parent] ``` ### Modes | Mode | Description | | --- | --- | | `on` | Default on macOS for mounted-folder VMs. Locks `.git/hooks`, `.git/config`, worktree configs, and submodule Git directories with `uchg` while the VM is stopped, and audits `.git` for symlinks, `commondir` redirections, and config `include.path` delegations on every launch. | | `strict` | Applies all locks from `on` and sets `diff.ignoreSubmodules = all` in `.git/config` so host `git status` and `git diff` do not recurse into untrusted submodule working trees. Switching from `strict` to `on` or `off` unsets `diff.ignoreSubmodules`. | | `off` | Removes `uchg` locks from `.git` and submodules. Default on Linux hosts, where unprivileged user-immutable flags are unavailable (`on` and `strict` require macOS). | | `clone` | Reported by `devvm git-protect` when the VM is a `--clone` VM (which does not mount the host folder or `.git`). Not a valid input argument to `devvm git-protect`. | Changing `.git` protection with `devvm git-protect on|strict|off` applies immediately if the VM is stopped. If the VM is running, the new mode is recorded in `vm.json` (`git.next`) to take effect when the VM next stops, and in an interactive terminal `devvm` offers to restart the VM immediately. ### Flags and `--json` output When run with no mode argument, print the active mode as `{"version": 1, "mode": "on"}`. Refused if `on`, `strict`, or `off` is passed. Target this folder's own VM. Target the nearest enclosing parent folder's VM. ```bash title="Show and change .git protection" devvm git-protect devvm git-protect strict devvm git-protect off ``` ```json title="devvm git-protect --json" { "version": 1, "mode": "on" } ``` --- ## `devvm clone` Show whether the current folder's VM works on an isolated Git clone inside the VM (`on`) or mounts the host project folder (`off`), or remake the VM from its base image to switch modes. ```bash devvm clone [on|off] [-f | --force] [--no-export] [--clean-home] [--no-clone-check] [--json] [--here | --parent] ``` ### Switching behavior (`on` vs. `off`) Switching modes with `devvm clone on` or `devvm clone off` remakes the VM from its image, preserving the VM's home directory (unless `--clean-home` is set) and its `vm.json` settings while discarding the rest of its root disk, then opens a shell in the new VM: 1. **`devvm clone on`:** - Requires the project folder to be the top-level directory of a Git repository with at least one commit and a checked-out branch (not a detached `HEAD`). - Unlocks host `.git` protection from the old machine, sets `git.mode` to `clone`, initializes a repository at the same path inside the new VM with `receive.denyCurrentBranch=updateInstead`, pushes the host repo's branches into the VM over pinned SSH (`GIT_SSH_COMMAND`), and adds a host Git remote (`devvm`, `devvm-`, or the VM name) pointing to `:`. - Warns if the host working tree has uncommitted tracked changes, which are not copied. 2. **`devvm clone off`:** - Unless `--no-clone-check` is passed, starts the VM if stopped and checks the VM's clone and up to 100 linked worktrees for unfetched commits and uncommitted changes. - If unfetched commits exist on VM branches, offers to fetch them into local host branches under `saved//`. If uncommitted changes exist, prompts before remaking. - Removes the host Git remote (preserving any previously fetched remote-tracking commits with unique history under `saved//`), remakes the VM with the host project folder mounted read-write, and enables `.git` protection (`on` on macOS). ### Flags Skip the remake confirmation prompt (required without a terminal or with `--no-input`). Only valid when `on` or `off` is given. Do not export coding agent conversation history to `~/.local/share/devvm/exports` before replacing the VM's disk. Only valid when `on` or `off` is given. Initialize the remade VM with a fresh home directory from the image instead of preserving the existing VM's home directory. Only valid when `on` or `off` is given. When switching `off`, skip checking the VM's clone for unfetched commits and uncommitted changes (any unsaved work in the clone is lost). Only valid when `on` or `off` is given. When run with no arguments, print `{"version": 1, "clone": "on"}` or `"off"`. Refused when `on`, `off`, or any remake flag is passed. ```bash title="Switch between mounted folder and repo clone" devvm clone devvm clone on devvm clone off devvm clone on --force --clean-home ``` ```json title="devvm clone --json" { "version": 1, "clone": "off" } ``` --- ## `devvm ssh-agent` Show or change SSH agent forwarding for the current folder's VM, or manage keys in `devvm`'s dedicated host SSH agent (`~/.local/share/devvm/agent.sock`). ```bash devvm ssh-agent [on|devvm|off] [--json] [--here | --parent] ``` ### Forwarding modes | Mode | Description | | --- | --- | | `off` | Default. No SSH agent socket is forwarded into the VM. | | `devvm` | Forwards `devvm`'s own isolated host `ssh-agent` (`~/.local/share/devvm/agent.sock`), which holds only the keys explicitly registered via `devvm ssh-agent add` (`~/.local/share/devvm/agent-keys.json`). | | `on` | Forwards your host's primary `SSH_AUTH_SOCK` into the VM. On launch, if your host agent holds no keys and `~/.ssh/id_ed25519`, `id_ecdsa`, or `id_rsa` exists, `devvm` offers to run `ssh-add` (`--apple-use-keychain` on macOS). | While SSH agent forwarding (`devvm` or `on`) is active and you are connected to the VM, any process running as your user inside the VM can request signatures from the forwarded agent's loaded keys. Mode changes apply from the VM's next start; in a terminal, `devvm` offers to restart a running VM immediately. ### Subcommands #### `devvm ssh-agent add` ```bash devvm ssh-agent add KEY_FILE [--lifetime DURATION] [--confirm] ``` Start `devvm`'s dedicated SSH agent if it is not running, load `KEY_FILE` into it via `ssh-add`, and record the key's physical parent path, `SHA256:...` fingerprint, and options in `~/.local/share/devvm/agent-keys.json`. - Whenever a VM with `ssh-agent devvm` starts, `devvm` automatically reloads any recorded key whose lifetime expired or after a host reboot, verifying that the file's fingerprint still matches the pinned `SHA256:...` fingerprint. - `devvm` refuses to add or reload any key located inside a directory that any project VM mounts. Path to the private SSH key file on the host. Duration the agent retains the key in memory once loaded (for example, `30m`, `8h`, `1d`, `2w`), or `forever`. Passed to `ssh-add -t`. Require confirmation via an `ssh-askpass` program (`$SSH_ASKPASS` or `ssh-askpass` on `PATH`) before each signature use (`ssh-add -c`). Fails if no askpass program is available. #### `devvm ssh-agent rm` ```bash devvm ssh-agent rm KEY_FILE ``` Unload `KEY_FILE` (matched by its recorded fingerprint) from `devvm`'s SSH agent and remove its entry from `~/.local/share/devvm/agent-keys.json`. The private key file on disk is untouched and does not need to still exist. ### Examples and `--json` payload ```bash title="Manage devvm SSH agent keys and forwarding" devvm ssh-agent add ~/.ssh/deploy_key --lifetime 8h --confirm devvm ssh-agent devvm devvm ssh-agent devvm ssh-agent rm ~/.ssh/deploy_key devvm ssh-agent off ``` ```json title="devvm ssh-agent --json" { "version": 1, "mode": "devvm", "keys": [ { "file": "/Users/alice/.ssh/deploy_key", "fingerprint": "SHA256:abc123def456...", "confirm": true, "lifetime": "8h", "loaded": true } ] } ``` --- ## `devvm link` List or manage one-way service links from the current folder's VM (the client) to TCP ports published by your other project VMs (targets), or publish and unpublish ports on the current VM. ```bash devvm link [--json] [--here | --parent] ``` ### How links resolve and enforce policy - Inside a client VM, each active link allocates a dedicated loopback address in `127.77.0.0/16` (starting at `127.77.0.1` per target VM) and resolves both `NAME` and `NAME.devvm.internal` to that address, forwarding `NAME:PORT` and `NAME.devvm.internal:PORT` (as well as HTTP proxy requests to those hostnames) to `PORT` on the target VM. - **Network tier rule (`policyRule`):** A client VM can only link to a target VM whose network tier is at least as strict as the client's tier (`none` > `allowlist` > `internet` > `open`). A confined VM cannot bypass its network tier by linking to a less-restricted VM. - **Link status (`applies` / `reason`):** When `devvm link` lists links, `APPLIES` is `yes` (in JSON: `"applies": true, "reason": ""`) or one of five non-applying reasons: - `gone`: the target VM was deleted. - `replaced`: the target VM was remade and now has a different `state.VM.ID`. - `unpublished`: the target VM does not currently publish `PORT`. - `policy`: the target VM's network tier is less strict than the client VM's tier. - `ungrouped`: the link was created by a group (`devvm group`) and the two VMs are no longer in the same group. ### Subcommands #### `devvm link add` ```bash devvm link add TARGET PORT [--as NAME] [--here | --parent] ``` Create a link from the current folder's VM to `PORT` (`1`–`65535`) on `TARGET`. Applied immediately on a running VM. - `TARGET` can be a project folder path (such as `../api`), a folder basename (such as `db`, if unambiguous across your VMs), or a full VM name (`devvm-db-...`). A VM cannot link to itself. - The target VM must already publish `PORT` via `devvm link publish`, satisfy the network tier rule, and run on a backend with `FeatureLink`. Target VM name, project folder name, or folder path. Single TCP port number (`1`–`65535`) published by `TARGET`. Hostname label (`NAME`) by which the client VM reaches `TARGET` (default: the target VM's folder slug). Must be a single lowercase DNS label (`^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$`), not all digits, and not `localhost` or the client VM's own hostname slug. All links from a client to the same target VM share one name. #### `devvm link rm` ```bash devvm link rm TARGET PORT [--here | --parent] ``` Remove the link to `PORT` on `TARGET` (where `TARGET` is the link's `NAME`, the target VM name, or folder name/path). Links actively managed by a group cannot be removed with `devvm link rm` while both VMs remain in that group (use `devvm group leave` or `devvm link unpublish` instead). #### `devvm link publish` and `devvm link unpublish` ```bash devvm link publish [PORTS] [--json] [--here | --parent] devvm link unpublish PORTS [--here | --parent] ``` - With no arguments, `devvm link publish` prints the ports and ranges currently published by this folder's VM (or `none`). Supports `--json`. - With `PORTS` (comma-separated ports and ranges such as `5432,8000-8010`), `devvm link publish PORTS` adds those ports to the VM's published list, and `devvm link unpublish PORTS` withdraws them. - If the VM belongs to a group (`devvm group`), publishing or unpublishing ports immediately synchronizes the group's links across all other group members under the host-wide groups lock (`~/.local/share/devvm/groups.lock`). ### Examples and `--json` payloads ```bash title="Publish ports and link two VMs" # In ../db devvm link publish 5432 # In ./app devvm link add ../db 5432 --as pg devvm link devvm link rm pg 5432 ``` ```json title="devvm link --json" { "version": 1, "links": [ { "name": "pg", "port": 5432, "target": "devvm-db-0123456789abcdef", "target_folder": "/Users/alice/work/db", "target_status": "Running", "applies": true, "reason": "" } ] } ``` ```json title="devvm link publish --json" { "version": 1, "ports": "5432,8000-8010" } ``` --- ## `devvm group` Show the group of VMs the current folder's VM belongs to, or join and leave a group. Every VM in a group automatically gets links (`NAME:PORT` and `NAME.devvm.internal:PORT`) to all ports published with `devvm link publish` by every other member of the group. ```bash devvm group [--json] [--here | --parent] ``` ### Group invariants and synchronization - **Single group per VM:** A VM can belong to at most one group at a time. The group is created when its first member runs `devvm group join NAME` and ceases to exist when its last member leaves or is deleted. - **Uniform network tier:** Because group links are bidirectional between all members that publish ports, every VM in a group must be on the exact same network tier. `devvm network ` is refused while a VM is in a group. - **Link precedence:** Explicit links created with `devvm link add` take precedence over group-generated links. If a user-added link in a VM uses the same `NAME` for a different target VM, `devvm` warns of the clash and omits the conflicting group member's auto-links in that VM. - **Concurrency lock:** All group joins, leaves, and `link publish`/`unpublish` updates acquire `~/.local/share/devvm/groups.lock`. ### Subcommands #### `devvm group join` ```bash devvm group join NAME [--as NAME] [--here | --parent] ``` Add the current folder's VM to group `NAME` and synchronize published-port links across all members of the group. Re-running `devvm group join NAME` for the group the VM is already in re-synchronizes all members' group links. Group name. Must be a valid lowercase DNS label (`^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$`) and not all digits. Hostname (`NAME`) by which other VMs in the group reach this VM (default: this VM's folder slug). Must be a valid DNS label, cannot be `localhost`, and cannot collide with any existing member's group name or folder hostname slug. #### `devvm group leave` ```bash devvm group leave [--here | --parent] ``` Remove the current folder's VM from its group and remove all group-generated links from both this VM and the remaining members. Explicit links created with `devvm link add` are preserved. ### Examples and `--json` payload ```bash title="Join, inspect, and leave a VM group" devvm group join web devvm group join web --as api2 devvm group devvm group leave ``` ```json title="devvm group --json" { "version": 1, "group": "web", "members": [ { "name": "api2", "vm": "devvm-api-89abcdef01234567", "folder": "/Users/alice/work/api", "published": "8080" }, { "name": "db", "vm": "devvm-db-0123456789abcdef", "folder": "/Users/alice/work/db", "published": "5432" } ] } ``` ## Related pages Traffic confinement across allowlist, internet, open, and none tiers, host-side TLS SNI and HTTP inspection via devvm _proxy, and per-VM CA rotation. Run Claude Code, Codex, and opencode inside VMs, import and save agent settings, complete OAuth or API-key logins, and bind host secrets. Read-only .git mount protection, symlink and hardlink containment, set-id file audits, secret placeholder injection, and isolated repo clones. Forward guest ports to host 127.0.0.1, publish ports and link VMs under *.devvm.internal hostnames, and join VMs into mutually reachable groups. Create VMs with independent Git clones using devvm --clone, switch existing VMs between mounted folders and clones, and fetch VM branches to the host. Complete reference for ~/.config/devvm/config.toml keys, settings.json precedence, environment variables, state directory layout, and version 1 --json payloads. --- ## 17. CLI reference: images, storage, and host commands > Syntax, subcommands, and flags for devvm build, rebuild, images, layer, tools, space, gc, retention, setup, doctor, upgrade, completion, and limactl. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/17-cli-reference-images-storage-and-host-commands.md - Generated: 2026-11-07T22:37:48.386Z ### Source Files - `internal/cli/build.go` - `internal/cli/images.go` - `internal/cli/layer.go` - `internal/cli/tools.go` - `internal/cli/space.go` - `internal/cli/gc.go` - `internal/cli/retention.go` - `internal/cli/upgrade.go` --- title: "CLI reference: images, storage, and host commands" description: "Syntax, subcommands, and flags for devvm build, rebuild, images, layer, tools, space, gc, retention, setup, doctor, upgrade, completion, and limactl." --- `devvm` groups image construction, shared tool management, disk accounting, garbage collection, and host-level administration across three help sections in `internal/cli`: **Images** (`build`, `rebuild`, `images`, `layer`), **Disk space** (`space`, `gc`, `retention`), and **This machine** (`setup`, `doctor`, `upgrade`, `completion`, `limactl`), plus `devvm tools` and `devvm layer add`/`rm` under **Change your VM**. Every command validates flags and positional arguments before running `PersistentPreRunE`, exiting with code `2` on usage errors, `1` on runtime failures, `130` on `INT`/`TERM`/`HUP`, and `0` on success. ## Command inventory and pre-run behavior Before executing a command's `RunE`, the root command enforces directory permissions (`0700` on `~/.config/devvm` and `~/.local/share/devvm`), checks layout version compatibility (`state.CheckLayout`), and runs daily automatic retention (`autoRetain`) unless the target command is explicitly exempted. | Command | Help group | Modifies VM or disk | Takes `--json` | Skips layout check | Skips auto-retention | | --- | --- | --- | --- | --- | --- | | `devvm build IMAGE` | Images | Builds image chain | No | No | No | | `devvm rebuild IMAGE` | Images | Rebuilds image and deletes unused stale images | No | No | No | | `devvm images` | Images | Read-only | Yes | No | No | | `devvm images rm IMAGE\|KEY...` | Images | Deletes built images | No | No | No | | `devvm layer new NAME` | Images | Writes layer script and policy in `~/.config/devvm/layers` | No | No | No | | `devvm layer add LAYER...` | Change your VM | Remakes project VM from new image | No | No | No | | `devvm layer rm LAYER...` | Change your VM | Remakes project VM without specified layers | No | No | No | | `devvm tools` | Change your VM | Interactive picker remounts tools; non-TTY / `--json` is read-only | Yes | No | No | | `devvm tools add TOOL...` | Change your VM | Builds missing tools and mounts into project VM | No | No | No | | `devvm tools rm TOOL...` | Change your VM | Unmounts shared tools from project VM | No | No | No | | `devvm tools update` | Change your VM | Resolves, builds, and mounts newest shared tools | No | No | No | | `devvm space` | Disk space | Read-only | Yes | No | Yes | | `devvm gc` | Disk space | Deletes stale/unused artifacts and prunes caches | No | No | Yes | | `devvm retention [KEY=VALUE...]` | Disk space | Bare call is read-only; `KEY=VALUE` writes `settings.json` | Yes (bare only) | No | Yes | | `devvm setup` | This machine | Saves base features and new-VM defaults to `settings.json` | No | No | No | | `devvm doctor` | This machine | Read-only host and backend readiness audit | Yes | No | No | | `devvm upgrade` | This machine | Replaces binary or runs `brew upgrade` (`--check`/`--json` read-only) | Yes | Yes | Yes | | `devvm completion SHELL` | This machine | Writes completion script to stdout | No | Yes | Yes | | `devvm limactl [ARGS...]` | This machine | Passes through to `limactl` on `~/.local/share/devvm/lima` | No | No | No | ```mermaid flowchart LR subgraph CLI["internal/cli command layer"] ImgCmds["build / rebuild / images / layer new"] VMCmds["layer add / layer rm / tools"] SpaceCmds["space / gc / retention"] HostCmds["setup / doctor / upgrade / completion / limactl"] end subgraph Services["Domain services"] BuildSvc["internal/build\nBuilds, Images, Choice"] VMSvc["internal/vm\nLifecycle, Tools, Housekeeping"] SettingsSvc["internal/settings\nRead, Save, ParseRetention"] UpgradeSvc["internal/upgrade\nUpgrader, Source, Detect"] end subgraph Storage["Host storage & backend"] DataDir["~/.local/share/devvm\nimages/, tools/, vms/, exports/, cache/"] ConfigDir["~/.config/devvm\nconfig.toml, layers/, ca/"] Backend["internal/backend (lima)\n~/.local/share/devvm/lima"] end ImgCmds --> BuildSvc VMCmds --> VMSvc SpaceCmds --> VMSvc SpaceCmds --> SettingsSvc HostCmds --> SettingsSvc HostCmds --> UpgradeSvc HostCmds --> Backend BuildSvc --> DataDir BuildSvc --> ConfigDir BuildSvc --> Backend VMSvc --> DataDir VMSvc --> Backend SettingsSvc --> DataDir SettingsSvc --> ConfigDir ``` ### Global flags Every command except `devvm limactl` (which sets `DisableFlagParsing: true` and forwards all flags directly to `limactl`) inherits two root persistent flags: Never prompt interactively. Commands that require confirmation or interactive selection fail immediately and report the flag that answers the prompt (such as `--force`). Stream external command output directly to stderr instead of displaying a single status line. `devvm` rejects empty string arguments and empty string flag values (such as `--from ""` or `--features ""`) across all commands via `noBlanks` before execution, exiting with code `2`. --- ## Image and layer commands ### `devvm build` Builds a target image and every parent image beneath it in the chain, skipping images that are already built and current. ```bash devvm build IMAGE [--from IMAGE] ``` #### Arguments and flags Target image name: `base`, or one or more layer names joined with `+` from bottom to top (for example, `docker` or `docker+mailpit`). Repeated layers and explicit `base+` prefixes are normalized (`base+foo+bar` and `foo+bar+foo` both resolve to `foo+bar`). Passing a shared tool name (`node`, `python`, `rust`, `go`, `dotnet`, `claude`, `codex`, `opencode`) fails with a message pointing to `devvm tools`. OCI/Docker container image reference that `base` starts from (such as `node:24` or `alpine:3.21`) instead of Ubuntu's cloud image. #### Behavior 1. Validates `--from` syntax (`from.Check`), parses `IMAGE` (`build.Names.Parse`), and verifies backend readiness (`backend.Check`). 2. If `base` features have not yet been chosen in `config.toml` or `settings.json`, prompts interactively in a terminal via `pickFeatures` (or selects all built-in features `cli-tools`, `fzf`, `omb` when non-interactive). 3. Computes the 12-hex-character SHA-256 content key for each image in the chain (`base` first, then each layer in order) and builds any missing or stale step. --- ### `devvm rebuild` Rebuilds an existing image from its parent as the parent currently stands, increments its rebuild counter in `~/.local/share/devvm/images/rebuilds.json`, and deletes stale images that no VM uses. ```bash devvm rebuild IMAGE [--from IMAGE] [-f|--force] ``` #### Arguments and flags Image to rebuild (`base`, or layers joined with `+` bottom first). Container image reference that `base` starts from instead of Ubuntu's cloud image. Skip the confirmation prompt when child images are built on top of `IMAGE`. Required when running without a terminal or with `--no-input` if dependent images exist. #### Behavior - When `IMAGE` is `base` (without `--from`), `PinOrigin` pins the newest upstream Ubuntu cloud image release (`backend.PinNewest`) and fetches fresh apt packages instead of reusing cached package sets in `~/.local/share/devvm/packages/`. - When other built images depend on `IMAGE`, `devvm rebuild` lists those dependent images and prompts for confirmation unless `--force` is passed. Those child images become `stale` until rebuilt or next used. - After building the new image, `devvm rebuild` deletes the superseded image and any stale images built on it that no VM uses. --- ### `devvm images` Lists all VM images across backends and all shared tool images, followed by summary notes on stderr (`imagesNotes`) and optional JSON output. ```bash devvm images [--json] devvm images rm IMAGE|KEY... [--from IMAGE] [-f|--force] ``` #### Flags (`devvm images`) Print the image inventory, shared tool inventory, totals, and Nix cache usage as a version 1 JSON object on stdout. #### Table output (`devvm images`) `devvm images` outputs two tables separated by a blank line: 1. **Images table** (`IMAGE`, `FROM`, `DIGEST`, `KEY`, `STATE`, `SIZE`): - `STATE` values: `current` (green), `stale` (yellow; inputs or rebuild counters changed), `unfinished` (yellow; interrupted build), or `building` (another `devvm` process holds the image lock). - `DIGEST` shows the first 12 hex characters of the origin SHA-256 (`-` if unknown). 2. **Shared tools table** (`NAME`, `TOOL`, `VERSION`, `LIBC`, `SIZE`, `VMS`, `STATE`): - `STATE` values: `current` (green; version in `mise.lock` for new VMs), `in use` (plain; older version still mounted by at least one VM), `unused` (yellow; older version mounted by `0` VMs), `not built` (yellow; current version in `mise.lock` when another version of the same tool is built), or `not installed` (no version built). #### Subcommand: `devvm images rm` Deletes one or more built images by name (such as `foo+bar`) or by 12-hex-character image key (such as `7a76021c46ef`). One or more image names or 12-hex-digit keys. Every specified target must exist and be built, or `devvm images rm` deletes nothing. Stale and unfinished images can only be targeted by their `KEY`. Resolve named images against the specified `--from` container base image. Delete without prompting for confirmation. Required without a terminal or with `--no-input`. On macOS, VMs cloned from an image continue working after the image is deleted because APFS clones share blocks independently. On Linux hosts, VMs use qcow2 backing overlays; `devvm images rm` refuses to delete an image whose disk is still backed by an existing VM. ```bash devvm images --json ``` ```json { "version": 1, "images": [ { "name": "base", "from": "ubuntu 26.04 (20261001)", "digest": "sha256:8eb876c2ccc4b6ff89e4a65d511815923b285e04503ba465809bf587f763da24", "key": "7a76021c46ef", "state": "current", "size": 2254857830, "sized": true, "backend": "lima" } ], "tools": [ { "name": "node", "tool": "node", "version": "24.11.0", "built": true, "backend": "lima", "libc": "gnu", "size": 125829120, "vms": 1, "state": "current" } ], "total": 2380686950, "unused_tools": 0, "nix_cache": { "files": 0, "size": 0, "cap": 10737418240 } } ``` --- ### `devvm layer` Creates custom layer scripts in `~/.config/devvm/layers` or remakes the current project folder's VM with layers added or removed. Running `devvm layer` with no subcommand prints its help and exits `0`. ```bash devvm layer new NAME devvm layer add LAYER... [-f|--force] [--no-export] [--clean-home] [--here | --parent] devvm layer rm LAYER... [-f|--force] [--no-export] [--clean-home] [--here | --parent] ``` #### Subcommands Creates `~/.config/devvm/layers/.sh` (with `#!/bin/bash` and `set -euo pipefail`) and `~/.config/devvm/layers/.policy` (if the policy file does not already exist). Refuses to overwrite an existing `.sh`. Layer identifier matching `^[a-z0-9][a-z0-9-]*$`. Must not be `base`, a built-in layer (`desktop`, `docker`, `nix`), or a shared tool name (`node`, `python`, `rust`, `go`, `dotnet`, `claude`, `codex`, `opencode`). Remakes the current folder's VM from an image with `LAYER...` appended on top of its existing layers in the order given, then opens an interactive shell in the VM. One or more built-in (`desktop`, `docker`, `nix`) or user-defined layer names whose scripts exist in `~/.config/devvm/layers/.sh`. Layers the VM already has are kept in their current position; if the VM already has all requested layers, `devvm` reports that and changes nothing. Remake the VM without prompting for confirmation. Skip exporting coding agent history to `~/.local/share/devvm/exports/` before replacing the VM disk. Start with the new image's `/home/` directory instead of copying the existing VM's home directory across the remake. Target this directory's own VM, ignoring any parent directory's VM. Mutually exclusive with `--parent`. Target the nearest ancestor directory's VM, even across a nested Git repository boundary. Mutually exclusive with `--here`. Remakes the current folder's VM from an image without `LAYER...`, preserving the remaining layers in their original order (or falling back to `base` if all layers are removed), then opens a shell in the VM. One or more layer names to remove. Unlike `layer add`, `layer rm` allows removing a custom layer whose `~/.config/devvm/layers/.sh` script has already been deleted. Remake the VM without prompting for confirmation. Skip exporting coding agent history before replacing the VM disk. Use the new image's home directory instead of preserving the VM's existing home directory. Control folder-to-VM resolution (mutually exclusive). Both `devvm layer add` and `devvm layer rm` refuse `--clone` VMs because the repository clone resides on the VM's root disk. Run `devvm clone off` first, then modify layers, then run `devvm clone on`. --- ## Shared tool commands (`devvm tools`) Inspects or modifies the read-only EROFS shared toolchains (`node`, `python`, `rust`, `go`, `dotnet`) and coding agents (`claude`, `codex`, `opencode`) mounted into the current folder's VM. ```bash devvm tools [--json] [--here | --parent] devvm tools add TOOL... [--here | --parent] devvm tools rm TOOL... [--here | --parent] devvm tools update [--here | --parent] ``` ### Modes and subcommands | Invocation | Behavior | Opens VM shell | | --- | --- | --- | | `devvm tools` (in a TTY without `--no-input` or `--json`) | Opens an interactive multi-select picker showing toolchains, coding agents, and the VM's read-only layers; builds any newly selected tools and applies changes | Yes | | `devvm tools` (non-TTY, `--no-input`, or `--json`) | Lists the VM's mounted tools, their versions, any newer version in `mise.lock`, and whether that newer version is built (`TOOL`, `VERSION`, `NEWER`, `BUILT`). Starts no VM and makes no network requests | No | | `devvm tools add TOOL...` | Validates tool names against `image.Toolchains` + `image.Agents`, builds any unbuilt tools, adds them to the VM, and prompts before restarting if a new coding agent is added to a running VM | Yes | | `devvm tools rm TOOL...` | Removes the specified shared tools from the VM's state and unmounts them; tools installed locally inside the VM stay untouched | Yes | | `devvm tools update` | Checks upstream for newer shared tool versions, builds them, switches the VM to the newest versions without modifying the VM disk, and opens a shell | Yes | ### Argument validation and JSON output - `devvm tools add` and `devvm tools rm` require each tool name as a separate argument (`devvm tools add python claude`). Passing names joined with `+` or `,` (such as `python,claude` or `python+claude`) fails with exit code `2` (`msg.ToolsJoined`). - `devvm tools --json` outputs a version 1 JSON object where `newer` and `newer_built` are omitted when the VM already has the version in `mise.lock`: ```json { "version": 1, "tools": [ { "tool": "node", "version": "24.10.0", "newer": "24.11.0", "newer_built": true } ] } ``` --- ## Storage, garbage collection, and retention commands ### `devvm space` Reports disk usage across all categories in `~/.local/share/devvm` and across registered backends without starting any VM or modifying state. Each backend query is bounded by a 30-second timeout (`spaceTimeout`). ```bash devvm space [--json] ``` #### Table columns (`devvm space`) | Column | Description | | --- | --- | | `WHAT` | Artifact category: `VMs`, `images`, `tools gnu` / `tools musl` / `tools -`, `history`, `network logs`, `run logs`, `mise`, `packages`, `Nix cache`, `leftover state`, `state`, and `total` | | `COUNT` | Item count in the category | | `SIZE` | Disk usage in binary powers of 1024 (`512K`, `2.1G`), or `-` when unsized | | `KEPT` | Retention classification: `remade` (rebuilt on demand), `only copy` (yellow; VMs and agent history exports), `log` (logs), `state` (metadata), or `leftover` (yellow; orphaned VM folders and dead link sockets) | | `GC` | Bytes that `devvm gc` (with no extra flags) will free immediately | | `FREED BY` | Command or policy that frees the remaining bytes (`devvm rm`, `images rm`, `gc --tool-images`, `gc --history`, `the newest 20 stay`, `rebuild base`, `gc --nix-cache`, `gc --orphans`, or `-`) | | `RETAIN` | Active retention setting for the row and the bytes it would free right now (for example, `30d (1.0M)`), or `-` when off | When Lima has entries in its own download cache (`~/Library/Caches/lima` on macOS, `~/.cache/lima` on Linux), `devvm space` prints a second table (`CACHE`, `COUNT`, `SIZE`, `PATH`) outside `total`. On stderr, `devvm space` also notes any backend that did not respond (`unknown`), VMs with `devvm`'s label that have no state in this `devvm` (`strays`), VMs whose project directory was deleted (`folder_gone`), and VMs whose project directory is on an unmounted volume (`folder_not_connected`). #### JSON schema (`devvm space --json`) Schema version (`1`). Rows from the primary table (`what`, optional `backend`, `count`, `size`, `sized`, `kept`, `gc`, `rest`, and optional `retain` and `retain_frees`). Never `null` (`[]` when empty). Sum of `size` across `devvm` rows in bytes. Sum of `gc` across `devvm` rows in bytes. Backend cache rows (`backend`, `name`, optional `path`, `count`, `size`, `sized`). Never `null` (`[]` when empty). Backends used by known VMs that failed to respond within 30 seconds (omitted when empty). Backend VMs carrying `devvm`'s label with no matching local state (`backend`, `name`, `data`; omitted when empty). Existing VMs whose host project folder no longer exists (`backend`, `name`, `folder`; omitted when empty). Existing VMs whose host project folder resides on a disconnected mount (`backend`, `name`, `folder`; omitted when empty). --- ### `devvm gc` Deletes stale and unfinished images, unused shared tool images, stale host `mise` versions, unreferenced base `.deb` packages, and least-recently-used files in the shared Nix cache exceeding its cap (`nix-cache.size`, or `10 GiB` when unset). `devvm gc` never deletes a project VM. ```bash devvm gc [--tool-images [TOOL...]] [--history [--older-than DURATION]] [--nix-cache] [--orphans] [--caches] [--retention] [-f|--force] ``` #### Flags and arguments Also delete `current` shared tool images that no VM uses. Optionally accepts positional `TOOL...` names (`node`, `python`, `rust`, `go`, `dotnet`, `claude`, `codex`, `opencode`) to restrict deletion to those specific tools. Passing positional `TOOL...` arguments without `--tool-images` fails with exit code `2`. Also delete saved agent history exports in `~/.local/share/devvm/exports/`. Requires `--history` (fails with exit code `2` if given without `--history`). Deletes only history exports at least `DURATION` old (such as `12h`, `30d`, `2w`). Delete the entire shared Nix binary cache (`~/.local/share/devvm/cache/nix-cache/`) instead of only trimming files above the cap. Forget state folders (`vms//`), SSH config entries, `devvm` Git remotes (for `--clone` VMs), links from other VMs, and dead link sockets for VMs that no longer exist in their backend. Skips VMs whose backend is unreachable, VMs created under a different `DEVVM_LIMA_HOME`, and VMs whose state changed within the last hour. After running `devvm`'s own cleanup, delete `devvm`'s share of each backend's download cache (Ubuntu cloud images downloaded by `devvm` that no image record references and that have not changed in the last hour), prompting separately per backend unless `--force` is set. Immediately apply all configured retention policies in `settings.json` / `config.toml`, including `orphans.age`. Delete planned items and backend cache entries without interactive confirmation. Required when running without a terminal or with `--no-input`. While another `devvm` process is creating a VM or updating tools (holding `making.lock`), `devvm gc --tool-images` retains all `current` shared tool images that have no VM yet, prints which tools were kept (`msg.GCToolsMaking`), and deletes the rest of the plan. --- ### `devvm retention` Displays or updates automatic retention rules in `~/.local/share/devvm/settings.json`. Every retention setting defaults to `off` until explicitly configured in `config.toml` or via `devvm retention`. ```bash devvm retention [--json] devvm retention KEY=VALUE... ``` #### Retention keys and value formats | Key | Value type | Default when `off` | Applied by daily `autoRetain` | Applied by `devvm gc --retention` | What it deletes | | --- | --- | --- | --- | --- | --- | | `logs.runs` | Integer `1`..`10000` or `off` | Keeps newest `20` run logs | Yes (every run via `KeepRunLogs`) | Yes | Oldest `.log` files in `~/.local/share/devvm/logs/runs/` beyond the count | | `history.age` | Duration (`30d`, `12w`, `24h`) or `off` | `off` (keeps all) | Yes | Yes | Agent history exports in `exports/` older than the duration | | `history.count` | Integer `1`..`10000` or `off` | `off` (keeps all) | Yes | Yes | Oldest agent history exports in `exports/` beyond the newest `N` | | `tools.unused-age` | Duration (`30d`, `2w`) or `off` | `off` (keeps all) | Yes (unless `tools.lock` is busy) | Yes | `current` shared tool images that have had `0` VMs for at least the duration | | `orphans.age` | Duration (`7d`, `30d`) or `off` | `off` (keeps all) | No | Yes | Leftover state in `vms//` for VMs missing from their backend whose state last changed at least `DURATION` ago | | `nix-cache.size` | Byte size (`5G`, `500M`, `10GiB`) or `off` | `10 GiB` cap | Yes | Yes | Least-recently-used files in `cache/nix-cache/` exceeding the size cap | When both `history.age` and `history.count` are set, an export is deleted if either rule retires it. Passing `--json` together with `KEY=VALUE` arguments is rejected with exit code `2` (`showsOnly`). If a key is set in `~/.config/devvm/config.toml` under `[retention]`, `config.toml` takes precedence over `settings.json`. ```bash Terminal output $ devvm retention logs.runs=off # built-in history.age=90d # ~/.local/share/devvm/settings.json history.count=off # built-in tools.unused-age=30d # ~/.config/devvm/config.toml orphans.age=off # built-in nix-cache.size=off # built-in ``` ```json devvm retention --json { "version": 1, "settings": { "history.age": "90d", "history.count": "off", "logs.runs": "off", "nix-cache.size": "off", "orphans.age": "off", "tools.unused-age": "30d" }, "sources": { "history.age": "settings.json", "history.count": "built-in", "logs.runs": "built-in", "nix-cache.size": "built-in", "orphans.age": "built-in", "tools.unused-age": "config.toml" } } ``` --- ## Host configuration and maintenance commands ### `devvm setup` Configures `base` image features and default settings for newly created VMs in `~/.local/share/devvm/settings.json`. Existing VMs keep the settings, agents, and layers they were created with. ```bash devvm setup [--features LIST|none] [--agents LIST|none] [--layers LIST|none] ``` #### Flags Comma-separated list of `base` image features (`cli-tools`, `fzf`, `omb`), or `none`. Changing the feature selection marks `base` and any images built on it for rebuild on next use. Comma-separated list of coding agents that new VMs receive by default (`claude`, `codex`, `opencode`), or `none`. Comma-separated list of layers that new VMs' images receive by default, ordered bottom first (such as `docker,mailpit`), or `none`. #### Interactive vs. flag execution - **When any of `--features`, `--agents`, or `--layers` is passed:** `devvm setup` applies only the specified flags without prompting for the other settings. - **When run with no flags in a terminal:** `devvm setup` walks through five prompts in order, starting from current values and skipping any setting locked in `~/.config/devvm/config.toml` (printing a note naming `config.toml`): 1. `base` image features (`cli-tools`, `fzf`, `omb`) 2. Default CPUs and memory for new VMs (`vm.SizeChoice.Ask`) 3. Default network tier for new VMs (`allowlist`, `open`, `local-only`, `offline`) 4. Default coding agents for new VMs (`claude`, `codex`, `opencode`) 5. Default image layers for new VMs (built-in layers and custom layers in `~/.config/devvm/layers`) --- ### `devvm doctor` Checks host prerequisites for the active backend, inspects `PATH` and state directory hygiene, reports whether `docker` (for `--from` builds) and corporate proxy CAs are present, and prints backend feature support. `devvm doctor` never installs software or modifies files. ```bash devvm doctor [--json] ``` #### Flags Output the diagnostic report as a version 1 JSON object and exit `0` even when backend `problems` are present. #### Checks and exit behavior 1. **Feature support table (`FEATURE`, `SUPPORT`):** Lists all 19 `backend.Features` (`build-image`, `build-confinement`, `from`, `clone`, `project-mount`, `agent-folders`, `tools-share`, `egress-proxy`, `ssh`, `ports`, `ports-all`, `agent-forwarding`, `agent-toggle`, `cpus`, `memory`, `disk`, `docker-layer`, `nix-layer`, `link`) with support status (`native`, `emulated`, `unreviewed`, or `none`). 2. **Host Docker and proxy CA notes:** Reports whether `docker` is on `PATH` (used when converting `--from` container images) and whether `HTTPS_PROXY` / `proxy.https` is configured while `~/.config/devvm/ca` is empty. 3. **Non-fatal warnings (`warnings`):** Never cause `devvm doctor` to exit non-zero: - Another `devvm` binary appears earlier on `PATH` than the running executable. - Disk free space on the backend volume is below the backend's floor. - Host hardware is below recommended minimums (`< 4` CPUs or `< 8 GiB` RAM). - Malformed lines in `.policy` files (`netpolicy.NotRules`). - Unexpected files, missing `vm.json`, or unreadable directories inside `~/.local/share/devvm/vms/` (`state.VMFolderProblems`). 4. **Fatal backend problems (`problems`):** Missing or unsupported `limactl`, `qemu-system-*`, or `/dev/kvm` access on Linux. Without `--json`, `devvm doctor` exits `1` when one or more `problems` exist; with `--json`, it prints the report and exits `0`. ```json { "version": 1, "backend": "lima", "problems": [ { "key": "lima", "fix": "brew install lima", "message": "Lima isn't installed. Run: brew install lima" } ], "warnings": [], "docker": true, "features": { "lima": { "agent-folders": "native", "agent-forwarding": "native", "agent-toggle": "native", "build-confinement": "native", "build-image": "native", "clone": "native", "cpus": "native", "disk": "native", "docker-layer": "native", "egress-proxy": "native", "from": "native", "link": "native", "memory": "native", "nix-layer": "native", "ports": "native", "ports-all": "native", "project-mount": "native", "ssh": "native", "tools-share": "native" } }, "release": { "latest": "0.2.0", "checked": "2026-11-05T00:00:00Z" } } ``` --- ### `devvm upgrade` Upgrades `devvm` using the installation method detected for the running binary (`install.sh` layout in `~/.local/lib/devvm/` or Homebrew formula `devvm` / `devvm-rc`), or checks for newer releases without installing. ```bash devvm upgrade [--check] [--to VERSION] [-f|--force] [--json] ``` #### Flags and mutual exclusions Check GitHub releases and report whether a newer version is available without installing anything. Mutually exclusive with `--force` and `--to`. Install the exact release `VERSION` (leading `v` is stripped; validated by `upgrade.Valid`), allowing downgrades or prereleases (`0.2.0-rc.1`). Only supported for `install.sh` installations. Mutually exclusive with `--check` and `--json`. Upgrade without prompting for confirmation. Mutually exclusive with `--check` and `--json`. Query release availability and print a version 1 JSON object (`current`, `latest`, `newer`, `install`, `path`) without installing anything. Mutually exclusive with `--force` and `--to`. #### Verification and installation methods - **`install.sh` (`~/.local/lib/devvm//bin/devvm`):** Fetches the release archive, `SHA256SUMS`, and `SHA256SUMS.minisig`, verifies the Minisign `ED` signature against the embedded public key in `packaging/minisign.pub`, verifies the archive SHA-256, unpacks into `~/.local/lib/devvm/`, atomically switches the `~/.local/bin/devvm` symlink (with rollback on failure), and removes old versions under `~/.local/lib/devvm/` that are not running. - **Homebrew (`homebrew` or `homebrew-rc`):** Runs `brew upgrade devvm` or `brew upgrade devvm-rc`. Passing `--to` on a Homebrew installation is refused because Homebrew manages formula versions. --- ### `devvm completion` Generates the shell completion script for `bash`, `zsh`, or `fish` on stdout. Completions dynamically resolve image names, `+`-joined layer stacks, `,`-joined `--tools`, `--agents`, `--layers`, and `--features` lists, and `--ports` keywords (`all`, `none`), while disabling default filename completion (`ShellCompDirectiveNoFileComp`). ```bash devvm completion SHELL ``` Target shell: must be `bash`, `zsh`, or `fish`. Any other value exits with code `2`. ```bash ~/.bashrc source <(devvm completion bash) ``` ```zsh ~/.zshrc source <(devvm completion zsh) ``` ```fish ~/.config/fish/config.fish devvm completion fish | source ``` --- ### `devvm limactl` Runs Lima's `limactl` binary directly against `devvm`'s isolated `LIMA_HOME` (`~/.local/share/devvm/lima`, or `$DEVVM_LIMA_HOME` when set), forwarding all arguments and flags verbatim, including `--help`. ```bash devvm limactl [ARGS...] ``` `devvm limactl` is intended for low-level debugging and cleanup (such as `devvm limactl list` or deleting a stray VM with `devvm limactl delete NAME`). Starting a VM via `devvm limactl start` boots it **unconfined** without `devvm _proxy`, regardless of the VM's configured network tier. Always start project VMs with `devvm`. ## Related pages Syntax, flags, and JSON schemas for devvm (up), ls, restart, stop, update, rm, adopt, size, ports, host, code, desktop, and history. Syntax, subcommands, and flags for devvm network, secret, agent, git-protect, clone, ssh-agent, link, and group. How devvm computes 12-hex SHA-256 content keys for base images, built-in and custom layers, and read-only EROFS shared tool images. Operational workflows for automatic retention policies, garbage collection locks, and Minisign-verified binary upgrades. --- ## 18. Configuration (config.toml) and --json schema reference > Complete reference for ~/.config/devvm/config.toml keys, settings.json precedence, environment variables, state directory layout, and version 1 --json payloads. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/18-configuration-config.toml-and---json-schema-reference.md - Generated: 2026-11-07T22:42:12.966Z ### Source Files - `internal/settings/config.go` - `internal/settings/settings.go` - `internal/settings/values.go` - `internal/settings/retention.go` - `internal/cli/json.go` - `internal/paths/dirs.go` - `internal/paths/vm.go` --- title: "Configuration (config.toml) and --json schema reference" description: "Complete reference for ~/.config/devvm/config.toml keys, settings.json precedence, environment variables, state directory layout, and version 1 --json payloads." --- `devvm` reads host configuration from two files managed by `internal/settings` (`~/.config/devvm/config.toml` and `~/.local/share/devvm/settings.json`), resolves runtime paths and overrides from host environment variables (`internal/paths` and `internal/cli`), stores all host-side state outside project directories under XDG base directories, and emits versioned machine-readable output (`"version": 1`) for every read-only CLI inspection command that accepts `--json`. ## Configuration precedence and resolution When resolving defaults for new VMs and automated retention (`settings.Read`), `devvm` evaluates four layers in order, where the first source that defines a setting wins: 1. **Per-command CLI flags** (such as `--network`, `--ports`, `--cpus`, `--memory`, `--disk`, `--features`, `--agents`, `--layers`). 2. **`~/.config/devvm/config.toml`** (`$XDG_CONFIG_HOME/devvm/config.toml`, source label ``, JSON source `"config.toml"`). Edited manually; `devvm` reads this file and never writes to it. 3. **`~/.local/share/devvm/settings.json`** (`$XDG_DATA_HOME/devvm/settings.json`, source label `devvm: settings.json`, JSON source `"settings.json"`). Written by `devvm setup`, `devvm size -g`, `devvm ports -g`, `devvm network -g`, and `devvm retention`. 4. **Built-in defaults** (source label `devvm: default`, JSON source `"built-in"`). ```mermaid flowchart LR subgraph Overrides["1. Per-Invocation CLI Flags"] Flags["--network / --ports / --cpus\n--memory / --disk / --agents / --layers"] end subgraph UserConfig["2. User Config ($XDG_CONFIG_HOME/devvm)"] TOML["config.toml\n(decodeConfig: strict TOML + Undecoded check)\nSource: \"config.toml\""] end subgraph SavedState["3. Saved CLI Defaults ($XDG_DATA_HOME/devvm)"] JSON["settings.json\n(readSaved / writeSaved)\nSource: \"settings.json\""] end subgraph Builtin["4. Built-in Fallbacks (internal/settings)"] Def["NetworkDefault = \"allowlist\"\nPortsAll = \"all\"\nNixCacheDefault = 10 GiB\nSource: \"built-in\""] end Flags -->|"Overrides for single VM"| Effective["Effective Plan / settings.Defaults"] TOML -->|"Wins over settings.json"| Effective TOML -.->|"Blocks settings.Save()\n(set_in_config error)"| JSON JSON -->|"Wins over built-in"| Effective Def -->|"Fallback when unset"| Effective ``` `settings.Save` checks `config.toml` before writing to `settings.json`. If a CLI command (`devvm setup`, `devvm size -g`, `devvm ports -g`, `devvm network -g`, or `devvm retention`) attempts to save a key that is already defined in `config.toml`, `devvm` refuses the write with `set_in_config`, names the file path and key, and leaves `settings.json` unchanged. ### Strict validation and corruption handling Both `config.toml` and `settings.json` are validated in full on every read: - **`config.toml` (`decodeConfig`)**: Decoded with `github.com/BurntSushi/toml`. Any syntax error, type mismatch, invalid value, duplicate list element, or unknown key (`md.Undecoded()`) returns a `*state.CorruptError` containing the file path and line number (`keyLine`). Note that `config.toml` has no `backend` key; adding one fails as an unknown key. - **`settings.json` (`readSaved`)**: Unmarshaled into `settings.Values`. Unknown top-level JSON fields and unknown retention keys are ignored so older `devvm` binaries tolerate newer saved keys, whereas invalid values for known keys return a `*state.CorruptError` naming the file path and offending key. --- ## `config.toml` reference `~/.config/devvm/config.toml` is optional. All keys are optional and validated on read. ```toml title="~/.config/devvm/config.toml" # Network tier for new VMs: "open", "internet", "allowlist", or "none" network = "allowlist" # Ports forwarded from new VMs to host 127.0.0.1: "all", "none", or port/range list ports = "5173,8000-8010" # Base image features in execution order, or [] for none features = ["cli-tools", "fzf", "omb"] # Coding agents mounted into new VMs, or [] for none agents = ["claude", "codex"] # Image layers applied bottom-first to new VMs, or [] for none layers = ["docker"] # Ubuntu archive mirror tried first during initial base image builds ubuntu-mirror = "http://mirror.example.com/ubuntu-ports" # Daily check for newer devvm releases when running in a terminal release-check = true [size] cpus = 4 memory = 8 disk = 100 [retention] logs.runs = 20 history.age = "90d" history.count = 10 tools.unused-age = "30d" orphans.age = "30d" nix-cache.size = "20G" [proxy] https = "http://alice@proxy.example.com:8080" http = "http://alice@proxy.example.com:8080" no-proxy = "localhost,.example.com" ``` ### Top-level and `[size]` keys | Key | TOML Type | Default | Constraints & Validation | Saved in `settings.json`? | | --- | --- | --- | --- | --- | | `network` | `string` | `"allowlist"` | One of `"open"`, `"internet"`, `"allowlist"`, `"none"` (`settings.NetworkChoices`). | Yes (`devvm network -g`, `devvm setup`) | | `ports` | `string` | `"all"` | `"all"`, `"none"`, or comma-separated ports/ranges (`1`–`65535`, e.g. `"5173,8000-8010"`). Normalized via `settings.ParsePorts`. | Yes (`devvm ports -g`) | | `features` | `string[]` | Unchosen (`nil`) | Valid feature names matching `^[a-z0-9][a-z0-9-]*$` (`image.FeatureName`, e.g. `"cli-tools"`, `"fzf"`, `"omb"`). No duplicates. `[]` selects no features. | Yes (`devvm setup --features`) | | `agents` | `string[]` | Unchosen (`nil`) | Subset of `image.Agents`: `"claude"`, `"codex"`, `"opencode"`. No duplicates. `[]` selects no agents. | Yes (`devvm setup --agents`) | | `layers` | `string[]` | Unchosen (`nil`) | Valid layer names matching `^[a-z0-9][a-z0-9-]*$` (`image.LayerName`), excluding `"base"` and shared tool names (`node`, `python`, `rust`, `go`, `dotnet`, `claude`, `codex`, `opencode`). Built-in layers are `"desktop"`, `"docker"`, `"nix"`. Ordered bottom-first. No duplicates. | Yes (`devvm setup --layers`) | | `size.cpus` | `int`, `float`, or `string` | `""` (image default: `4`) | Positive whole integer matching `^[1-9][0-9]*$` (`settings.ParseCPUs`). | Yes (`devvm size -g`, `devvm setup`) | | `size.memory` | `int`, `float`, or `string` | `""` (image default: `8`) | Positive number of GiB (`N`, `NG`, `Ng`, or `NGiB`, decimals allowed, e.g. `8`, `8.5`, `"16G"`). Normalized to numeric string `N` via `settings.ParseGiB`. | Yes (`devvm size -g`, `devvm setup`) | | `size.disk` | `int`, `float`, or `string` | `""` (image default: `100`) | Positive number of GiB (`N`, `NG`, `Ng`, or `NGiB`, decimals allowed). Normalized to numeric string `N` via `settings.ParseGiB`. | Yes (`devvm size -g`) | | `ubuntu-mirror` | `string` | `""` (Ubuntu archive) | `http://` or `https://` URL with non-empty host and no user info, query string, fragment, whitespace, or control characters (`checkUbuntuMirror`). | No (`config.toml` only) | | `release-check` | `bool` | `true` | TOML boolean (`true` or `false`). Disabled when `false` or when `DEVVM_NO_RELEASE_CHECK` is non-empty. | No (`config.toml` only) | ### `[retention]` keys All retention settings default to `off` unless explicitly configured in `config.toml` or `settings.json`. When `logs.runs` is off, `devvm` retains the newest 20 run logs; when `nix-cache.size` is off, the shared Nix cache cap defaults to 10 GiB (`settings.NixCacheDefault`). | Key in `[retention]` | CLI Key (`devvm retention`) | TOML Type | Format & Constraints | What It Prunes | | --- | --- | --- | --- | --- | | `logs.runs` | `logs.runs` | `int` or `string` | Whole integer `1`–`10000` (`count`). | Run logs in `logs/runs/*.log` beyond the newest `N` files (20 when off). | | `history.age` | `history.age` | `string` | Duration matching `^[1-9][0-9]*[mhdw]$` (`1`–`100000`, e.g. `"30m"`, `"12h"`, `"90d"`, `"2w"` via `state.ParseDuration`). | Agent history archives in `exports/*.tar.gz` (and `.folder` sidecars) at least this old. | | `history.count` | `history.count` | `int` or `string` | Whole integer `1`–`10000` (`count`). | Agent history archives in `exports/*.tar.gz` beyond the newest `N` exports. | | `tools.unused-age` | `tools.unused-age` | `string` | Duration (`m`, `h`, `d`, `w` via `state.ParseDuration`). | Current shared tool images in `tools/` unused by any VM for at least this duration. | | `orphans.age` | `orphans.age` | `string` | Duration (`m`, `h`, `d`, `w` via `state.ParseDuration`). | State directories in `vms//` for VMs deleted outside `devvm` whose state has not changed for this duration (applied only by `devvm gc --retention`). | | `nix-cache.size` | `nix-cache.size` | `string` | Whole number of `K`, `M`, `G`, or `T` (powers of 1024, `> 0`, `<= 1 PiB`, e.g. `"500M"`, `"20G"` via `state.ParseSize`). | Least-recently-used entries in `cache/nix-cache/` when total size exceeds the cap (10 GiB when off). | ### `[proxy]` keys The `[proxy]` table configures an upstream corporate proxy (`settings.HostProxy`) used by per-VM host proxies, image builds, and host-side `devvm` downloads when neither `HTTPS_PROXY`/`https_proxy` nor `HTTP_PROXY`/`http_proxy` is set in the host environment. If the environment defines `HTTPS_PROXY` or `HTTP_PROXY`, the environment's `HTTPS_PROXY`, `HTTP_PROXY`, and `NO_PROXY` override the entire `[proxy]` table. | Key | TOML Type | Format & Constraints | | --- | --- | --- | | `proxy.https` | `string` | `http://` URL or `host:port` (`backend.CheckProxyURL`). Non-`http://` schemes and line breaks are rejected. Must **not** contain a password (`proxy_password_in_config`). If a username is present (`http://user@host:port`), `devvm` reads the password from the `proxy` secret (`devvm secret set proxy`). | | `proxy.http` | `string` | Same format and password restrictions as `proxy.https`. | | `proxy.no-proxy` | `string` | Comma-separated host/domain/IP bypass list without line breaks (`backend.CheckNoProxy`). | ### `settings.json` wire format `~/.local/share/devvm/settings.json` serializes the `settings.Values` struct with 2-space indentation. Unset scalar fields are omitted; chosen empty lists serialize as `[]`: ```json title="~/.local/share/devvm/settings.json" { "network": "allowlist", "ports": "5173,8000-8010", "size": { "cpus": "4", "memory": "8", "disk": "100" }, "features": [ "cli-tools", "fzf", "omb" ], "agents": [ "claude" ], "layers": [], "retention": { "history.age": "90d", "tools.unused-age": "30d" } } ``` --- ## Environment variables `internal/structure` enforces single-owner rules on environment variable lookups across the codebase (`checkEnv`). ### User and host environment variables | Variable | Owner / Reader | Behavior | | --- | --- | --- | | `HOME` | `internal/paths/dirs.go` | Must be an absolute path (`home_not_absolute` if relative or empty). Base for default data, config, and cache directories. | | `XDG_CONFIG_HOME` | `internal/paths/dirs.go` | Base directory for user config (`$XDG_CONFIG_HOME/devvm`). Ignored if relative; defaults to `~/.config/devvm`. | | `XDG_DATA_HOME` | `internal/paths/dirs.go` | Base directory for `devvm` state (`$XDG_DATA_HOME/devvm`). Ignored if relative; defaults to `~/.local/share/devvm`. | | `XDG_CACHE_HOME` | `internal/paths/dirs.go` | On Linux, locates Lima's download cache (`$XDG_CACHE_HOME/lima`, default `~/.cache/lima`). On macOS, `UserCache` is always `~/Library/Caches`. | | `DEVVM_LIMA_HOME` | `internal/paths/dirs.go` | Moves `devvm`'s isolated Lima home directory from `~/.local/share/devvm/lima` to the specified path (resolved via `filepath.Abs`). `devvm` never reads `LIMA_HOME`. | | `DEVVM_FROM` | `internal/cli/flags.go` | Default OCI/Docker image reference for `--from` when building or launching VMs without an explicit `--from` flag. | | `DEVVM_NO_TOOL_CHECK` | `internal/tools/tools.go` | When non-empty (e.g. `1`), disables the daily background check for newer shared tool versions. | | `DEVVM_NO_RELEASE_CHECK` | `internal/settings/release.go` | When non-empty (e.g. `1`), disables the daily check for a newer `devvm` release. | | `DEVVM_SECRET_SERVICE` | `internal/secrets/secrets.go` | Overrides the macOS Keychain service name used to store secrets (defaults to `"devvm"`). | | `CLAUDE_CODE_OAUTH_TOKEN` | `internal/secrets/claude.go` | Supplies the Claude Code OAuth token from the host environment, taking precedence over any saved `claude` secret or agent login. | | `HTTPS_PROXY`, `https_proxy`, `HTTP_PROXY`, `http_proxy`, `NO_PROXY`, `no_proxy` | `internal/backend/hostproxy.go` | Upstream corporate proxy settings. Uppercase takes precedence over lowercase. When either `HTTPS_PROXY` or `HTTP_PROXY` is non-empty, overrides `[proxy]` in `config.toml`. | | `NO_COLOR`, `FORCE_COLOR`, `CLICOLOR_FORCE`, `TERM` | `internal/ui/style.go` | Controls ANSI terminal color output. Evaluated in order: `NO_COLOR != ""` (or `DEVVM_MESSAGES=names`) disables color; `FORCE_COLOR` or `CLICOLOR_FORCE` set to any non-empty value other than `"0"` forces color on non-TTYs; `TERM=dumb` disables color. | | `SSH_AUTH_SOCK`, `SSH_ASKPASS` | `internal/vm/sshagent.go`, `internal/sshagent/sshagent.go` | `SSH_AUTH_SOCK` provides the user's host SSH agent socket when `ssh-agent` mode is `on`; `SSH_ASKPASS` is checked when adding keys to `devvm`'s agent with `--confirm` on Linux. | ### Internal and test-build environment variables | Variable | Scope | Purpose | | --- | --- | --- | | `DEVVM_MESSAGES` | Any build (`internal/ui/style.go`) | When set to `"names"`, formats messages as machine-checkable `id key=value` lines and disables ANSI styling. | | `DEVVM_GUESTFWD` | Internal (`internal/backend/lima/qemu.go`) | Passed by `devvm` to its Linux QEMU wrapper (`devvm _qemu`) to configure guestfwd rules. | | `DEVVM_DAEMON_STATUS` | Internal (`internal/daemon/status.go`) | Passed to detached background processes (`_proxy`, `_ports`, `_links`) naming the readiness status file. | | `DEVVM_HOST_MISE`, `DEVVM_PIPED_ANSWERS`, `DEVVM_PICK_ON_PIPE`, `DEVVM_HISTORY_MAX`, `DEVVM_UNREVIEWED`, `DEVVM_KVM`, `DEVVM_UBUNTU_URL`, `DEVVM_UBUNTU_RELEASE`, `DEVVM_REPO_URL`, `DEVVM_RELEASE_KEY`, `DEVVM_RELEASE_CA`, `DEVVM_TEST_VERSION`, `DEVVM_TEST_CPUS`, `DEVVM_TEST_MEMORY_GIB`, `DEVVM_TEST_FREE_GIB`, `DEVVM_TEST_LIST_WAIT` | Test builds only (`internal/cli/overrides.go`) | Read exclusively by `readOverrides` when `testbuild.On()` is true (`-ldflags` in test builds). Ignored in release binaries. | --- ## State and configuration directory layout Every host path written by `devvm` is defined in `internal/paths` (`Dirs` and `VM`) and verified against layout version `1` (`state.Layout = 1`) at startup under `version.lock`. Commands fail with `layout_newer` if either `version` file contains a number greater than `1`. Only `version`, `--version`, `help`, and `completion` bypass the layout version check. :::files ~/.config/devvm/ ($XDG_CONFIG_HOME/devvm, mode 0700) ├── version # Layout version integer ("1\n") ├── config.toml # Hand-edited defaults, retention, proxy, ubuntu-mirror, release-check ├── policy # Global network allow/deny rules applied to every VM ├── ca/ # Trusted corporate root CA certificates (*.pem, *.crt; no symlinks) ├── layers/ # User-defined custom layers (.sh and optional .policy) └── agents/ # Imported agent settings mounted/copied into VMs ├── claude/ ├── codex/ └── opencode/ ::: :::files ~/.local/share/devvm/ ($XDG_DATA_HOME/devvm, mode 0700) ├── version # Layout version integer ("1\n") ├── version.lock # Flock held during layout checks and migrations ├── settings.json # CLI-saved defaults (setup, size -g, ports -g, network -g, retention) ├── ssh_config # Generated OpenSSH Host blocks for running VMs ├── agent.sock # UNIX domain socket for devvm's own SSH agent ├── agent-keys.json # Private keys registered with devvm's SSH agent ├── mise.lock # Resolved shared tool versions and SHA-256/SHA-512 checksums ├── mise.lock.toml # Copy of shared-tools/mise.toml from which mise.lock was resolved ├── mise.lock.checked # Timestamp marker of last upstream tool version check ├── retention.run # Timestamp marker of last automatic daily retention run ├── retention.lock # Flock held while retention runs ├── tools.lock # Flock held while building shared tools or updating mise.lock ├── oci-build.lock # Flock serializing OCI (--from) base image builds ├── making.lock # Shared/exclusive flock coordinating VM creation and gc --tool-images ├── groups.lock # Flock serializing group join/leave and link updates ├── logins/ # Per-agent OAuth/API credentials (/...) excluded from backups ├── secrets/ # Secret values on hosts without macOS Keychain (mode 0700) ├── exports/ # Saved agent history archives (-- :::files ~/.local/share/devvm/vms// ├── vm.json # Authoritative VM state (state.VM, updated under lock) ├── lock # Flock held across any vm.json read-modify-write cycle ├── policy # Per-VM network allow/deny rules ├── home.tar.gz # Temporary home backup present only during an active VM remake ├── logs/ # Mode 0700 log directory (10 MiB rotation with .1 backup) │ ├── network.jsonl # Proxy allow/block/forward/link event log (JSON Lines) │ ├── proxy.log # Host proxy daemon stderr/stdout log │ ├── ports.log # Port forwarder daemon log │ └── links.log # Inter-VM link session daemon log └── run/ # Mode 0700 runtime directory ├── remake.lock # Flock held across VM remake/recovery ├── started # Records hostagent PID and confinement state for current boot ├── known_hosts # Pinned guest SSH host keys for the current boot ├── sandbox.sb # Seatbelt sandbox profile on macOS hosts ├── proxy.json # Host proxy runtime configuration (proxy.Config) ├── proxy.pid # Host proxy PID and argv verification file ├── proxy.status # Host proxy readiness status file ├── ports.lock # Flock serializing port forwarder restarts ├── ports.pid # Port forwarder PID file ├── ports.status # Port forwarder readiness status file ├── links.pid # Link session daemon PID file ├── links.json # Active link targets read by the VM's proxy ├── secrets.json # Placeholder-to-secret bindings read by the VM's proxy ├── secrets.policy # Host allow rules contributed by bound secrets └── ca/ # Per-VM TLS interception CA (excluded from Time Machine on macOS) ├── lock # Flock protecting CA rotation ├── ca.pem # Current interception CA certificate (30-day validity) ├── ca.key # Current interception CA private key (never leaves host) ├── next.pem # Next CA certificate (generated 7 days before expiry) └── next.key # Next CA private key ::: --- ## Version 1 `--json` schema reference Every command that displays or lists state supports `--json` (`internal/cli/json.go`). `TestJSONShapes` and `TestShowCommandsTakeJSON` in `internal/cli/json_test.go` enforce the contract across the CLI: - **Top-level envelope**: `printJSON` writes a single 2-space indented JSON object to `stdout` whose first key is always `"version": 1` (`const jsonVersion = 1`). - **Additive compatibility**: Fields are only added to version 1. Renaming, removing, or changing the type of any field requires incrementing `jsonVersion`. - **Encoding rules**: HTML characters (`<`, `>`, `&`) are not escaped (`enc.SetEscapeHTML(false)`). Empty slices serialize as `[]`, never `null`. Sizes are integer byte counts unless the field name specifies a unit (such as `memory_gib` or `disk_gib`). Optional fields that do not apply are omitted (`omitempty`). - **Read-only enforcement (`showsOnly`)**: Commands that either show a setting when invoked bare or mutate it when given arguments/flags (`size`, `ports`, `network`, `ssh-agent`, `git-protect`, `clone`, `link publish`, `retention`) reject any mutating argument or flag when `--json` is passed (`json_shows_only`). - **Diagnostic separation**: Warnings and notes are still written to `stderr`; `stdout` receives only the JSON document without ANSI color codes. ### Single-value and default-setting payloads Commands that inspect a single VM setting (`printValue`) or a global default (`printDefault`) emit compact objects: | Command | JSON Payload Shape | Field Values & Notes | | --- | --- | --- | | `devvm ports --json` | `{"version": 1, "ports": "all"}` | `"all"`, `"none"`, or normalized port/range list (`"5173,8000-8010"`). | | `devvm ports -g --json` | `{"version": 1, "ports": "all", "source": "built-in"}` | `source` is `"config.toml"`, `"settings.json"`, or `"built-in"`. | | `devvm network -g --json` | `{"version": 1, "policy": "allowlist", "source": "config.toml"}` | `policy` is `"open"`, `"internet"`, `"allowlist"`, or `"none"`. | | `devvm git-protect --json` | `{"version": 1, "mode": "on"}` | `mode` is `"on"`, `"strict"`, `"off"`, or `"clone"` (for `--clone` VMs). | | `devvm clone --json` | `{"version": 1, "clone": "on"}` | `clone` is `"on"` or `"off"`. | | `devvm link publish --json` | `{"version": 1, "ports": "5432,8000-8010"}` | `"none"` when no ports are published, or normalized port/range list. | ### `devvm size --json` and `devvm size -g --json` Defined by `sizeOutput` and `sizeShape` (`internal/cli/size.go`). Unset fields (such as `disk_gib` when global defaults leave disk size to the image) are omitted. ```json title="devvm size --json" { "version": 1, "cpus": 4, "memory_gib": 8, "disk_gib": 100, "pending": { "cpus": 6, "memory_gib": 16, "disk_gib": 120 } } ``` ```json title="devvm size -g --json" { "version": 1, "cpus": 4, "memory_gib": 8, "sources": { "cpus": "config.toml", "disk": "built-in", "memory": "settings.json" } } ``` | Field | Type | Description | | --- | --- | --- | | `cpus` | `number` | Current CPU count (omitted in `-g` when unset). | | `memory_gib` | `number` | Current memory in GiB (omitted in `-g` when unset). | | `disk_gib` | `number` | Current disk size in GiB (omitted in `-g` when unset or on backends without disk sizing). | | `pending` | `object` | Omitted unless a running VM has staged size changes (`cpus`, `memory_gib`, `disk_gib`) awaiting restart. | | `sources` | `object` | Present only with `-g`. Maps `"cpus"`, `"memory"`, and `"disk"` to `"config.toml"`, `"settings.json"`, or `"built-in"`. | ### `devvm retention --json` Defined by `retentionOutput` (`internal/cli/retention.go`). Both `settings` and `sources` always include all six retention keys (`settings.RetentionKeys()`). ```json title="devvm retention --json" { "version": 1, "settings": { "history.age": "90d", "history.count": "off", "logs.runs": "off", "nix-cache.size": "off", "orphans.age": "off", "tools.unused-age": "30d" }, "sources": { "history.age": "config.toml", "history.count": "built-in", "logs.runs": "built-in", "nix-cache.size": "built-in", "orphans.age": "built-in", "tools.unused-age": "settings.json" } } ``` ### `devvm ls --json` Defined by `lsOutput` and `vmJSON` (`internal/cli/ls.go`). Lists all project VMs sorted by name. ```json title="devvm ls --json" { "version": 1, "vms": [ { "name": "devvm-app-0123456789abcd", "status": "Running", "image": "docker", "tools": [ "node", "claude" ], "from": "node:24", "git": "on", "net": "allowlist", "backend": "lima", "folder": "/Users/you/src/app", "folder_state": "there", "links": 1 } ] } ``` | Field | Type | Description | | --- | --- | --- | | `vms[].name` | `string` | Full VM name (`devvm--<14-hex-hash>`). | | `vms[].status` | `string` | Backend status (`"Running"`, `"Stopped"`, `"Broken"`, or `""` if unknown). | | `vms[].image` | `string` | Image name (`"base"` or `+`-joined layers such as `"docker+desktop"`). | | `vms[].tools` | `string[]` | Shared toolchains and coding agents attached to the VM (`[]` when none). | | `vms[].from` | `string` | Container image reference if built with `--from`; omitted for standard Ubuntu base images. | | `vms[].git` | `string` | `.git` protection mode: `"on"`, `"strict"`, `"off"`, `"clone"`, or `""`. | | `vms[].net` | `string` | Active network tier (`"open"`, `"internet"`, `"allowlist"`, `"none"`) or `"unenforced"`. | | `vms[].backend` | `string` | Backend driver name (`"lima"`). | | `vms[].folder` | `string` | Host project folder path. | | `vms[].folder_state` | `string` | Host folder availability: `"there"`, `"gone"`, or `"not_connected"`. | | `vms[].links` | `number` | Count of outbound links configured on this VM. | ### `devvm images --json` and `devvm tools --json` `devvm images --json` (`imagesOutput` in `internal/cli/images.go`) lists all VM images, shared tool EROFS images, aggregate byte totals, and shared Nix cache usage. `devvm tools --json` (`toolsOutput` in `internal/cli/tools.go`) lists the shared tools mounted in the current folder's VM. ```json title="devvm images --json" { "version": 1, "images": [ { "name": "base", "from": "ubuntu 26.04", "digest": "sha256:abcdef012345...", "key": "7a76021c46ef", "state": "current", "size": 1932735283, "sized": true, "backend": "lima" } ], "tools": [ { "name": "node", "tool": "node", "version": "24.1.0", "built": true, "backend": "lima", "libc": "gnu", "size": 134217728, "vms": 2, "state": "current" } ], "total": 2066953011, "unused_tools": 0, "nix_cache": { "files": 42, "size": 52428800, "cap": 10737418240 } } ``` ```json title="devvm tools --json" { "version": 1, "tools": [ { "tool": "node", "version": "24.1.0", "newer": "24.2.0", "newer_built": true } ] } ``` | Command | Field | Type | Description | | --- | --- | --- | --- | | `images` | `images[].state` | `string` | `"current"`, `"stale"`, `"unfinished"`, or `"building"`. | | `images` | `images[].digest` | `string` | `"sha256:"` or `""` for unfinished images. | | `images` | `tools[].name` / `tools[].tool` | `string` | `name` is the shared tool group (`"node"`, `"python"`); `tool` is the individual binary/runtime in `mise.lock` (`"node"`, `"pnpm"`, `"bun"`, `"python"`, `"uv"`). | | `images` | `tools[].state` | `string` | `"current"`, `"unused"`, or `"not built"`. `backend` and `libc` (`"gnu"` or `"musl"`) are omitted when `built` is `false`. | | `images` | `total`, `unused_tools`, `nix_cache` | `number`, `object` | Byte totals and `nix_cache` (`files`, `size`, `cap` in bytes). | | `tools` | `tools[].newer`, `tools[].newer_built` | `string`, `bool` | Omitted when the VM already uses the latest version in `mise.lock`. | ### Network, secrets, SSH agent, links, and groups payloads Defined in `internal/cli/network.go` (`networkOutput`, `logOutput`, `blockedOutput`). Note that `--json` and `--follow` are mutually exclusive on `devvm network log`, and `--blocked` is mutually exclusive with `--follow` and `--lines`. ```json title="devvm network --json" { "version": 1, "policy": "allowlist", "allowlist": [ { "builtin": "layers/base.policy", "rules": [ "archive.ubuntu.com", "security.ubuntu.com" ], "not_rules": [] }, { "file": "/Users/you/.local/share/devvm/vms/0123456789abcd/policy", "denies_only": true, "rules": [ "!example.com" ], "not_rules": [ { "line": 2, "text": "invalid line" } ] } ] } ``` ```json title="devvm network log --json" { "version": 1, "events": [ { "time": "2026-10-01T12:00:00Z", "event": "forward", "host": "api.github.com", "port": 443, "method": "GET", "path": "/user", "rule": "api.github.com", "agent": "claude", "secrets": [ "github" ], "upstream": "proxy.example.com:8080" } ] } ``` ```json title="devvm network log --blocked --json" { "version": 1, "blocked": [ { "last": "2026-10-01T12:05:00Z", "count": 3, "host": "registry.npmjs.org", "port": 443, "allow": "devvm network allow registry.npmjs.org" } ] } ``` - In `allowlist[]`, built-in policies set `builtin` (omitting `file`), whereas user policy files set `file` (omitting `builtin`). `denies_only` is `true` on `internet` and `none` tiers when only deny rules apply. - In `events[]`, optional fields (`method`, `path`, `rule`, `reason`, `agent`, `secrets`, `refused`, `upstream`, `target`) are omitted when empty/false. Defined in `internal/cli/secret.go` (`secretOutput`) and `internal/cli/sshagent.go` (`sshAgentOutput`). Secret values are never included in JSON output. ```json title="devvm secret --json" { "version": 1, "secrets": [ { "name": "claude", "saved": false, "used_by": [] }, { "name": "github", "saved": true, "used_by": [ "devvm-app-0123456789abcd" ] }, { "name": "proxy", "saved": false, "used_by": [] } ] } ``` ```json title="devvm ssh-agent --json" { "version": 1, "mode": "devvm", "keys": [ { "file": "/Users/you/.ssh/deploy_key", "fingerprint": "SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", "confirm": true, "lifetime": "12h", "loaded": true } ] } ``` Defined in `internal/cli/link.go` (`linkOutput`) and `internal/cli/group.go` (`groupOutput`). ```json title="devvm link --json" { "version": 1, "links": [ { "name": "db", "port": 5432, "target": "devvm-db-0123456789abcd", "target_folder": "/Users/you/src/db", "target_status": "Running", "applies": false, "reason": "unpublished" } ] } ``` ```json title="devvm group --json" { "version": 1, "group": "web", "members": [ { "name": "db", "vm": "devvm-db-0123456789abcd", "folder": "/Users/you/src/db", "published": "5432" } ] } ``` - In `links[]`, `applies` is `true` when `reason` is `""`. When `applies` is `false`, `reason` is one of `"gone"`, `"replaced"`, `"unpublished"`, `"policy"`, or `"ungrouped"`. - In `group`, when the VM belongs to no group, `group` is `""` and `members` is `[]`. ### Storage, history, diagnostics, and upgrade payloads Defined in `internal/cli/space.go` (`spaceReport`) and `internal/cli/history.go` (`historyOutput`). ```json title="devvm space --json" { "version": 1, "devvm": [ { "what": "VMs", "backend": "lima", "count": 1, "size": 5368709120, "sized": true, "kept": "only copy", "gc": 0, "rest": "devvm rm" }, { "what": "history", "count": 2, "size": 3145728, "sized": true, "kept": "only copy", "gc": 0, "rest": "gc --history", "retain": "90d", "retain_frees": 1048576 } ], "total": 5371854848, "gc": 0, "backends": [ { "backend": "lima", "name": "download cache", "path": "/Users/you/Library/Caches/lima", "count": 1, "size": 629145600, "sized": true } ], "unknown": [ "qemu" ], "strays": [ { "backend": "lima", "name": "devvm-other-0123456789abcd", "data": "/tmp/other-data/devvm" } ], "folder_gone": [ { "backend": "lima", "name": "devvm-old-0123456789abcd", "folder": "/Users/you/src/deleted" } ], "folder_not_connected": [ { "backend": "lima", "name": "devvm-ext-0123456789abcd", "folder": "/Volumes/Work/ext" } ] } ``` ```json title="devvm history --json" { "version": 1, "exports": [ { "file": "/Users/you/.local/share/devvm/exports/devvm-app-0123456789abcd-20261001-120000-000000000.tar.gz", "vm": "devvm-app-0123456789abcd", "folder": "/Users/you/src/app", "date": "2026-10-01T12:00:00Z", "size": 1572864 } ] } ``` - In `spaceReport`, `devvm[].what` values are `"VMs"`, `"images"`, `"tools gnu"`, `"tools musl"`, `"tools -"`, `"history"`, `"network logs"`, `"run logs"`, `"mise"`, `"packages"`, `"Nix cache"`, `"leftover state"`, and `"state"`. `devvm[].kept` values are `"remade"`, `"only copy"`, `"log"`, `"state"`, and `"leftover"`. - `unknown`, `strays`, `folder_gone`, and `folder_not_connected` are omitted when empty. Defined in `internal/cli/doctor.go` (`doctorOutput`) and `internal/cli/upgrade.go` (`upgradeJSON`). `devvm doctor --json` always exits `0` even when `problems` is non-empty. `devvm upgrade --json` checks for updates without installing anything and is mutually exclusive with `--force` and `--to`. ```json title="devvm doctor --json" { "version": 1, "backend": "lima", "problems": [ { "key": "lima", "fix": "brew install lima", "message": "Lima isn't installed. Run: brew install lima" } ], "warnings": [], "docker": true, "features": { "lima": { "clone-from-image": "native", "confined-network": "native", "disk-resize": "native", "isolated-networks": "native", "nested-virt": "native", "oci-import": "native", "read-only-mounts": "native", "ssh": "native", "tool-mounts": "native", "writable-mounts": "native" } }, "release": { "latest": "0.2.0", "checked": "2026-11-05T00:00:00Z" } } ``` ```json title="devvm upgrade --json" { "version": 1, "current": "0.1.0", "latest": "0.2.0", "newer": true, "install": "install.sh", "path": "/Users/you/.local/lib/devvm/0.1.0/devvm" } ``` - In `doctorOutput`, `problems[].fix` is `null` when no automated fix command exists, `features..` is `"native"`, `"emulated"`, `"unreviewed"`, or `"none"`, and `release` is `null` if no daily release check has recorded a result yet. - In `upgradeJSON`, `install` is `"install.sh"`, `"homebrew"`, `"homebrew-rc"`, or `"other"`. ## Related pages Inspect how project folders map to 14-hex-digit VM hashes, `vm.json` state sections, and layout version checks. Configure automatic pruning with `devvm retention`, reclaim disk space with `devvm gc`, and upgrade `devvm` installations. Configure corporate proxies in `[proxy]`, add custom root CAs in `~/.config/devvm/ca`, and inspect `devvm network log`. Complete command syntax and flags for `devvm setup`, `devvm space`, `devvm retention`, `devvm doctor`, and `devvm upgrade`. --- ## 19. Diagnose host, boot, and SSH failures > Resolve devvm doctor host problems, Ubuntu cloud image boot fallbacks, SSH host-key verification errors, run logs, and moved project directories with devvm adopt. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/19-diagnose-host-boot-and-ssh-failures.md - Generated: 2026-11-07T22:47:29.255Z ### Source Files - `internal/cli/doctor.go` - `internal/backend/lima/check.go` - `internal/vmssh/checked.go` - `internal/vmssh/hostkeys.go` - `internal/vm/move.go` - `internal/ui/runlog.go` --- title: "Diagnose host, boot, and SSH failures" description: "Resolve devvm doctor host problems, Ubuntu cloud image boot fallbacks, SSH host-key verification errors, run logs, and moved project directories with devvm adopt." --- `devvm` diagnoses host prerequisites, VM boot failures, SSH host-key mismatches, and project folder changes across five subsystems: host readiness checks in `devvm doctor` (`internal/cli/doctor.go` and `internal/backend/lima/check.go`), per-run command logs with credential redaction (`internal/ui/runlog.go` and `internal/redact/redact.go`), PGP-verified Ubuntu cloud image pinning and boot fallback (`internal/ubuntu`, `internal/backend/lima/pin.go`, and `internal/build`), per-boot SSH host-key pinning (`internal/vmssh`), and inode birth-time folder identity tracking (`internal/vm/move.go` and `internal/platform`). ## Host readiness and `devvm doctor` `devvm doctor` inspects the host OS, CPU architecture, Lima installation, virtualization drivers, feature support, disk and memory capacity, network policy syntax, and VM state directories without modifying host state or installing packages automatically. ```bash devvm doctor devvm doctor --json ``` When run without `--json`, `devvm doctor` exits `1` if one or more blocking host problems exist (`len(problems) > 0`) and exits `0` when only warnings or informational notices are present. When run with `--json`, it always prints a versioned JSON report to stdout and exits `0` so scripts and CI pipelines can inspect `problems` and `warnings` programmatically. ### Blocking host problems Every startup path (`starter.hostChecks` in `internal/vm/start.go` and `Builds.checkHost` in `internal/build/build.go`) runs `b.Problems(ctx)` before creating or starting a VM. On the Lima backend (`internal/backend/lima/check.go`), `Problems` evaluates the checks below in order: | Problem key | Condition checked | Minimum / requirement | Suggested `fix` command in `devvm doctor` | | --- | --- | --- | --- | | `os` | `runtime.GOOS` is neither `darwin` nor `linux` | macOS or Linux | `null` | | `arch` | `runtime.GOARCH` is neither `arm64` nor `amd64` | `arm64` (`aarch64`) or `amd64` (`x86_64`) | `null` | | `macos` | `sw_vers -productVersion` major version is below `13` (`minMacOS = "13"`) | macOS 13 (Ventura) or newer for `vz` (`Virtualization.framework`) | `null` (update macOS in System Settings) | | `lima` | `limactl` is not on `PATH` | Lima `2.2` or newer (`minLima = "2.2"`) | `brew install lima` (if `brew` is on `PATH`; otherwise `null`) | | `lima` | `limactl --version` fails or outputs an unparseable version string | Valid `limactl version X.Y.Z` output | `brew reinstall lima` (if `brew` is on `PATH`) | | `lima` | `limactl --version` is older than `2.2` | Lima `2.2` or newer (`user-v2` removal and `overwrite` DNS support) | `brew upgrade lima` (if `brew` is on `PATH`) | | `qemu` | Linux only: `qemu-system-aarch64` (on `arm64`) or `qemu-system-x86_64` (on `amd64`) or `qemu-img` is missing from `PATH` | QEMU system emulator and `qemu-img` | First package manager found on `PATH`:
• `apt-get` (`amd64`): `sudo apt-get install -y --no-install-recommends qemu-utils ipxe-qemu qemu-system-x86 ovmf`
• `apt-get` (`arm64`): `sudo apt-get install -y --no-install-recommends qemu-utils ipxe-qemu qemu-system-arm qemu-efi-aarch64`
• `dnf`: `sudo dnf install -y qemu-kvm qemu-img`
• `pacman`: `sudo pacman -S --noconfirm qemu-base`
• `zypper`: `sudo zypper --non-interactive install qemu-kvm` | | `kvm` | Linux only: `/dev/kvm` does not exist | Hardware virtualization enabled in BIOS/UEFI or nested virtualization enabled on the hypervisor | `null` | | `kvm-access` | Linux only: `/dev/kvm` exists but cannot be opened `O_RDWR` by the current user | Read/write access to `/dev/kvm` | `sudo usermod -aG kvm ` (log out and back in for the group change to take effect) | | `lima-config` | `_config/default.yaml` or `_config/override.yaml` exists inside `~/.local/share/devvm/lima` (`DEVVM_LIMA_HOME`) | No Lima global override files inside `devvm`'s isolated Lima directory | `null` (move or remove `_config/default.yaml` and `_config/override.yaml` from `~/.local/share/devvm/lima/_config/`) | `devvm` stores its VMs in `~/.local/share/devvm/lima` (`LIMA_HOME`) rather than `~/.lima`. Lima merges `_config/default.yaml` and `_config/override.yaml` from `LIMA_HOME` into every VM's configuration, which can inject unexpected mounts, networks, or provisioning scripts. Both `devvm doctor` and `Lima.Start` refuse to proceed if either file exists under `~/.local/share/devvm/lima/_config/`. ### Non-fatal warnings and notices After checking blocking problems, `devvm doctor` evaluates non-fatal host warnings (`d.warnings(b)` in `internal/cli/doctor.go`) and informational notices: - **Shadowed `devvm` binary on `PATH` (`devvm_other_first`):** Compares `os.Executable()` against `exec.LookPath("devvm")` after resolving symlinks (`filepath.EvalSymlinks`). Warns if an older or different `devvm` binary appears earlier on `PATH`. - **Low disk space (`disk_low_warning` / `disk_free_unknown`):** Checks free space on `~/.local/share/devvm/lima` against the `base` build floor (`8 GiB` via `b.Floors().Build["base"]`). Warns if free space is below `8 GiB` and suggests `devvm space` and `devvm gc`. - **Low host memory or CPU count (`host_memory_low` / `host_cpus_few`):** Warns when host RAM is below `8 GiB` (`recommendedMemoryGiB = 8`, where default VMs receive half of host RAM) or when logical CPUs are fewer than `4` (`recommendedCPUs = 4`, where default VMs share all host CPUs). Suggests `devvm size` to adjust per-VM or default sizing. - **Invalid network policy rules (`policy_not_rule` / `policy_broken_deny`):** Scans `~/.config/devvm/policy`, `~/.config/devvm/layers/*.policy`, and `~/.local/share/devvm/vms/*/policy` with `netpolicy.NotRules`. Reports exact `file:line` locations for malformed rules; if a malformed line contains `deny`, warns that the broken `deny` rule is not blocking traffic. - **Broken or stray VM state directories (`vms_stray`, `vm_folder_stateless`, `vms_unreadable`):** Calls `state.VMFolderProblems(a.dirs.VMs)` to inspect `~/.local/share/devvm/vms/`: - Non-directory files or folders whose names are not 12-character lowercase hexadecimal hashes (`vmname.HashLen = 12`) are reported as stray entries. - Directories with valid 12-character hex names that lack `vm.json` (and are not mid-creation with only a `lock` file) are reported as stateless VM folders. - **Host proxy CA notice (`doctor_proxy_ca`):** If `HTTPS_PROXY` / `https_proxy` is set in the environment or `proxy.https` is configured in `~/.config/devvm/config.toml`, and `~/.config/devvm/ca/` contains no `.crt` or `.pem` files, `devvm doctor` prints a reminder to place your TLS inspection root certificate in `~/.config/devvm/ca` so guest VMs trust the proxy. ### `devvm doctor --json` output structure ```json { "version": 1, "backend": "lima", "problems": [ { "key": "kvm-access", "problem": "/dev/kvm cannot be opened by sheing.", "fix": "sudo usermod -aG kvm sheing" } ], "warnings": [ "Only 5.2 GiB is free on /home/sheing/.local/share/devvm/lima; building the base image needs at least 8.0 GiB. Run devvm space to see what devvm is using, or devvm gc to clean up." ], "docker": true, "features": { "mounts-ro": "native", "mounts-rw": "native", "net-confinement": "native", "port-forward": "native", "snapshots": "native", "ssh": "native", "ssh-agent": "native", "vsock": "native" }, "release": { "current": "v0.4.0", "latest": "v0.4.1", "update_available": true } } ``` Schema version (`1`). Active backend name (`"lima"`). Blocking host problems. Each entry contains `key` (`os`, `arch`, `macos`, `lima`, `qemu`, `kvm`, `kvm-access`, or `lima-config`), `problem` (human-readable description), and `fix` (shell command string or `null`). Non-fatal warning strings (shadowed binary, low disk/RAM/CPUs, malformed policy lines, or stray folders in `~/.local/share/devvm/vms`). Whether the `docker` CLI binary was found on `PATH` (used only when `devvm --from` resolves a local Docker image). Map of backend feature keys (`mounts-ro`, `mounts-rw`, `net-confinement`, `port-forward`, `snapshots`, `ssh`, `ssh-agent`, `vsock`) to their capability state (`"native"`, `"emulated"`, `"unreviewed"`, or `"none"`). Present when `~/.local/share/devvm/release-check.json` contains a cached release check and the binary is not a `dev` build. Reports `current`, `latest`, and `update_available`. --- ## Run logs, verbose streaming, redaction, and exit codes Whenever `devvm` runs an external host command (`limactl`, `ssh`, `git`, `mise`, `docker`) or a provisioning script inside a VM, `ui.UI` records the command line and combined output to a per-run log file. ### Run log location, retention, and failure tails - **Log path:** `~/.local/share/devvm/logs/runs/-.log` (created with mode `0600` inside a `0700` directory). - **Automatic pruning:** Creating a new run log prunes older `.log` files in `~/.local/share/devvm/logs/runs/`, keeping the newest `20` logs by default (`keepLogs = 20`) or the count configured by `retention.logs.runs` (`devvm retention`). - **Step markers and guest output cap:** Numbered steps write `# [n/total] ` followed by `+ ` into the run log. In-VM command streams (`vmLog`) are capped at `4 MiB` per command (`vmLogMax = 4 << 20`), appending `devvm: truncated` if exceeded. - **Default failure output (`ErrorLogged`):** When a logged command fails and `-v` / `--verbose` is not enabled, `devvm` prints the failure message, the path to the run log, and the last `25` non-empty lines of the log with terminal control characters stripped: ```text --- the end of ~/.local/share/devvm/logs/runs/20260711-142210-48291.log: ``` - **Live verbose streaming (`-v` / `--verbose`):** Passing `-v` or `--verbose` to any `devvm` command streams each command's output live to `stderr` (dimmed on terminals, with ANSI C0/C1 control sequences stripped) while simultaneously writing to the run log. If the command fails under `-v`, `devvm` prints `(details: )` without repeating the 25-line tail. ### Automatic credential redaction `devvm` redacts secrets before writing lines to the run log and scrubs Lima's hostagent log on disk after every VM start: 1. **Run log stream redaction (`redact.String` in `internal/redact/redact.go`):** - Replaces proxy authentication tokens matching `devvm:<32–64 lowercase hex digits>@` with `devvm:@`. - Replaces URL credentials matching `://:@` with `://:@` (literal ``). - Replaces bare `user:password@host:port` tokens when `:port` starts with a digit. - `redactWriter` buffers up to `64 KiB` (`redactLineMax`) for incomplete lines and holds up to `4 KiB` (`redactTailMax`) of a trailing non-whitespace word across write boundaries so credentials split across `write()` calls are still redacted. 2. **Lima hostagent log redaction (`redact.File` on `ha.stderr.log`):** - During `limactl start`, Lima's hostagent writes `lima.yaml` (including `env.HTTP_PROXY` and `env.HTTPS_PROXY`) to `~/.local/share/devvm/lima//ha.stderr.log`. - Immediately after `limactl start` returns (even on failure), `Lima.Start` calls `redact.File` on `ha.stderr.log`, which overwrites the secret bytes in place with `*` characters without changing the file's byte length or offsets. If in-place redaction fails, `devvm` emits the `lima_log_unredacted` warning naming the file path. ### CLI exit codes | Exit code | Meaning | Triggered by | | --- | --- | --- | | `0` | Success | Command completed normally | | `1` | Runtime or operational failure | Host check failures (`devvm doctor`), VM boot/start errors, build failures, SSH failures, refused prompts without `--force`, or non-zero exit from a command run via `devvm -- ` (when no signal interrupted `devvm`) | | `2` | CLI usage error | Unknown commands or words (`word.go`), invalid argument counts (`args.go`), invalid flag values (`checkFlagValue`), or mutually exclusive flags (`exclusive`), all validated before any VM or host action runs | | `130` | Interrupted (`SIGINT`, `SIGTERM`, `SIGHUP`) | Signal cancels the root context, waits for any in-flight logged command, waits up to `2s` (`returnGrace`) for `RunE` to return when cleanups are registered, runs all registered cleanups with a fresh context, and exits `130`. A second signal is ignored | --- ## Ubuntu cloud image pinning, boot fallback, and Lima start failures When building the `base` image on Lima (without `--from`), `devvm` resolves an upstream Ubuntu 26.04 (`resolute`) cloud image, verifies its PGP signature, pins the release, and boots a build VM. ```mermaid flowchart TD A["Build base image (internal/build)"] --> B{"~/.local/share/devvm/images/ubuntu.json exists?"} B -- "Yes (and not Rebuild)" --> C["Use pinned release & SHA-256"] B -- "No (or Rebuild)" --> D["Fetch releases index, SHA256SUMS, SHA256SUMS.gpg"] D --> E{"PGP signature valid against ubuntu-cloud.gpg?"} E -- "No" --> F["Fail immediately: ubuntu_bad_signature (no fallback)"] E -- "Yes" --> G["Write pin to ubuntu.json (mark fresh in current run)"] C --> H["Create & start build VM devvm-img-"] G --> H H --> I{"Boot succeeded?"} I -- "Yes" --> J["Run base.sh & feature scripts"] I -- "ErrBoot on freshly pinned release (once per run)" --> K["Warn origin_no_boot & PinOlder release"] K --> H I -- "ErrBoot on previously pinned or second failure" --> L["Keep stopped devvm-img- & fail"] ``` ### PGP verification and `ubuntu.json` pinning `ubuntu.Finder` (`internal/ubuntu/ubuntu.go`) queries `https://cloud-images.ubuntu.com/releases/resolute/` for date-stamped release directories (`YYYYMMDD` or `YYYYMMDD.N`), starting from the newest release: 1. For each candidate directory, it downloads `SHA256SUMS` and `SHA256SUMS.gpg` (capped at `64 KiB`). 2. `verifyPGP` verifies the detached signature in `SHA256SUMS.gpg` against the embedded Ubuntu Cloud Image keyring (`assets/layers/ubuntu-cloud.gpg`, fingerprint `D2EB44626FDDC30B513D5BB71A5D6C4C7DB87C81`). - If the signature is invalid or made by a different key, `devvm` fails immediately with `ubuntu_bad_signature` (`Ubuntu's SHA256SUMS signature for is not valid`). It never falls back past a bad signature. 3. Once verified, it extracts the SHA-256 digest for `ubuntu-26.04-server-cloudimg-arm64.img` or `ubuntu-26.04-server-cloudimg-amd64.img`, resolves the final redirect URL via `HEAD`, and writes `~/.local/share/devvm/images/ubuntu.json` under the file lock `~/.local/share/devvm/images/locks/ubuntu`. ### Automatic boot fallback (`origin_no_boot`) Occasionally, a newly published upstream Ubuntu cloud image passes PGP verification and SHA-256 checks but fails to boot under Lima's `vz` or `qemu` driver. `Builds.fallBack` (`internal/build/build.go`) handles this automatically under strict conditions: - The image being built is `base` (with `From == ""`), - The error wraps `backend.ErrBoot` (`limactl start` failed to bring the VM up), - The Ubuntu release was **freshly pinned during the current `devvm` invocation** (`b.fresh != ""`), and - `devvm` has not already fallen back in this invocation (`!b.fellBack`). When all four conditions hold, `devvm` prints a warning (`Ubuntu cloud image did not boot; trying the previous release`), calls `PinOrigin(ctx, nil, backend.PinOlder)` to pin the newest verified release older than the failed one in `~/.local/share/devvm/images/ubuntu.json`, and retries the build once. A release that was already pinned in a prior run does not trigger fallback because it already booted when first pinned. To manually re-pin to the latest upstream Ubuntu cloud image and rebuild `base`, run: ```bash devvm rebuild base ``` ### Degraded boot, 60-second SSH check, and serial logs When `limactl start` returns a non-zero exit code during `Lima.Start` (`internal/backend/lima/start.go`), `degraded()` checks whether Lima's hostagent actually brought the VM to `Status == "Running"` (for example, when a non-critical cloud-init check timed out): 1. If the VM is not `Running`, `Start` fails immediately. 2. If the VM is `Running`, `degraded()` tests SSH readiness by running `limactl shell --workdir / true` with a **60-second timeout** (`sshTimeout = 60 * time.Second`). 3. **SSH succeeds within 60 seconds:** `devvm` emits the `started_with_warnings` warning (`VM started with warnings; see `) and continues normal startup. 4. **SSH fails or times out after 60 seconds:** `devvm` force-stops the half-started VM (`limactl stop --force `) and returns `start_no_ssh` (`wrapping backend.ErrBoot`), pointing you to the guest serial console log: ```text ~/.local/share/devvm/lima//serial*.log ``` Inspect `serial*.log` (such as `serial.log` or `serialv.log`) and `ha.stderr.log` in `~/.local/share/devvm/lima//` to see kernel panic messages, cloud-init errors, or rootfs mount failures. ### Inspecting a failed image build VM When an image build fails **after** the temporary build VM (`devvm-img-`) has been created, `devvm` stops the build VM and **keeps it on disk** so you can inspect the failure inside the guest. The error output prints the exact command to open a shell in the failed build VM: ```bash devvm limactl shell --start devvm-img- ``` The next `devvm build`, `devvm rebuild`, or `devvm gc` automatically deletes the leftover `devvm-img-` VM before starting a new build. ### Broken Lima driver states, socket path limits, and network checks - **Broken Lima state recovery (`statusBroken`):** - If an earlier start was interrupted and left the VM in Lima's `Broken` state with a live `HostAgentPID` and `DriverPID == 0`, `Lima.Stop` waits up to **30 seconds** (`driverWait = 30 * time.Second`, polling every `200ms` in `settled()`) for the hostagent to record `DriverPID` before running `limactl stop --force `, preventing orphaned `qemu-system-*` processes. - Conversely, when `Lima.Start` sees `Status == "Broken"` with `DriverPID != 0` and `HostAgentPID == 0` (an orphaned QEMU process still holding the disk write lock after a crashed hostagent), `Start` runs `limactl stop --force ` before starting the VM. - **Unix socket path length limit (`lima_home_too_long`):** - Lima creates `ssh.sock` inside `//ssh.sock` and OpenSSH appends a 17-byte suffix (`sshSocketExtra = 17`) against the 104-byte Unix domain socket path limit (`socketMax = 104`). - If `~/.local/share/devvm/lima` is on a deep directory path such that `//ssh.sock` plus 17 bytes reaches 104 bytes, `CreateVM` fails with `lima_home_too_long`. Set `DEVVM_LIMA_HOME` to a shorter path (for example, `/tmp/devvm-lima` or `~/.devvm-lima`) to shorten the socket path. - **Unauthorized Lima networks (`vm_has_networks`):** - `devvm` VMs use only Lima's user-mode networking (`user-v2: false`) confined by the host proxy. If a VM's `lima.yaml` contains any `networks:` entries, `Lima.Start` refuses to start the VM with `vm_has_networks` (`VM has Lima networks configured, which would bypass network confinement`). Run `devvm update` or `devvm rebuild` to recreate the VM from a clean template. --- ## SSH host-key verification and connection failures Every SSH connection managed by `devvm`—including `~/.local/share/devvm/ssh_config`, local port forwarders (`devvm _ports`), inter-VM links (`devvm _links`), and `--clone` git fetches—passes through `vmssh.Checked` (`internal/vmssh/checked.go`). ### How per-boot host-key pinning works (`PinHostKeys`) Instead of using `StrictHostKeyChecking=accept-new` or trusting static keys across reboots, `devvm` reads and pins the VM's public host keys directly through the backend's local control channel (`b.Exec`) on every VM start: 1. **Clear previous keys before boot (`ForgetHostKeys`):** Before calling `b.Start`, `starter.start` (`internal/vm/start.go`) removes `~/.local/share/devvm/vms//run/known_hosts`. 2. **Verify loopback SSH target (`FromTarget`):** `vmssh.Checked` verifies that `b.SSH(ctx, vm)` returns a loopback address (`127.0.0.1`), a valid TCP port (`1..65535`), a valid username, and an existing identity key file. Any non-loopback address or missing field returns `*vmssh.UnusableError`. 3. **Match against the current boot's start ID (`backend.Run.ID`):** On Lima, `Running()` constructs a unique start ID from ` `. `PinHostKeys` checks whether `~/.local/share/devvm/vms//run/known_hosts` starts with the header line: ```text # devvm: host keys of start ``` If the header matches the running VM's `run.ID` and contains valid keys, `devvm` reuses the pinned file without re-querying the guest. 4. **Read and validate keys from the guest (`readVMHostKeys`):** When `run/known_hosts` is missing or belongs to an earlier boot, `PinHostKeys` runs `sh -c "cat /etc/ssh/ssh_host_*_key.pub"` inside the VM via `b.Exec` (with a `30s` timeout and `64 KiB` output cap): - `ParseHostKeys` validates each line: the key type must match `^[A-Za-z0-9][A-Za-z0-9@._+-]{0,63}$`, must not contain `-cert-`, must base64-decode to at most `4096` bytes, and its inner SSH wire-format length-prefixed algorithm name must equal the outer key type field. - Any `@cert-authority` or `@revoked` markers, wildcard host patterns, or malformed lines are dropped. - `PinHostKeys` re-checks `b.Running(ctx, vm)` to confirm the VM did not restart while keys were being read, then writes `~/.local/share/devvm/vms//run/known_hosts` with `StrictHostKeyChecking yes` and `UserKnownHostsFile ~/.local/share/devvm/vms//run/known_hosts`. ### Diagnosing SSH and editor connection errors **Cause:** Whenever `Lima.Start` edits `lima.yaml` via `limactl edit` (for example, after changing CPU/memory size, mounts, or `forwardAgent`), Lima's cloud-init regenerates `/etc/ssh/ssh_host_*_key.pub` on the next boot. While `devvm` batches all changes into a single `limactl edit` and pins the new keys into `~/.local/share/devvm/vms//run/known_hosts`, starting a VM outside `devvm` (such as running `devvm limactl start ` directly) leaves `run/known_hosts` with the previous boot's keys and leaves network confinement unenforced (`confinement: unenforced` in `devvm ls`). **Fix:** Run `devvm` or `devvm host` in the project directory. `devvm` will detect the new boot `run.ID`, re-pin `/etc/ssh/ssh_host_*_key.pub` into `run/known_hosts`, and update `~/.local/share/devvm/ssh_config`. If your personal `~/.ssh/known_hosts` cached an old key for `[127.0.0.1]:`, ensure you connect using the VM host alias (`ssh devvm--`) rather than `ssh 127.0.0.1 -p `, because the generated `Host` entry isolates host keys to `~/.local/share/devvm/vms//run/known_hosts`. **Cause:** There are two common causes: 1. **The VM is stopped:** `devvm` removes stopped VMs from `~/.local/share/devvm/ssh_config` so another local process cannot bind the freed loopback port and impersonate the VM. 2. **Missing `Include` in `~/.ssh/config`:** OpenSSH only applies `Include` directives that appear **before** any `Host` or `Match` block in `~/.ssh/config` (unless placed at the very top of the file). **Fix:** 1. Run `devvm host` in your project directory. This starts the VM if stopped, writes its `Host` entry to `~/.local/share/devvm/ssh_config`, ensures `Include ~/.local/share/devvm/ssh_config` is at the top of `~/.ssh/config`, and prints the exact host alias (`devvm--`). 2. To refresh `~/.local/share/devvm/ssh_config` for all currently running VMs without starting a new one, run `devvm --ssh-config`. **Cause:** Every `Host` block generated in `~/.local/share/devvm/ssh_config` sets `PermitRemoteOpen none` so a compromised guest cannot open reverse tunnel destinations through your forwarded SSH client connection. OpenSSH added `PermitRemoteOpen` in **OpenSSH 8.7**. **Fix:** Upgrade the host OpenSSH client to OpenSSH `8.7` or newer. **Cause:** `PinHostKeys` ran `cat /etc/ssh/ssh_host_*_key.pub` inside the guest, and none of the output lines passed `ParseHostKeys` validation (`vmssh.ErrNoHostKeys`), or `sshd` / cloud-init failed to generate `/etc/ssh/ssh_host_*_key.pub`. **Fix:** 1. Inspect the guest state with `devvm limactl shell -- ls -la /etc/ssh`. 2. Regenerate missing host keys inside the guest (`sudo ssh-keygen -A`) or recreate the VM from its base image while preserving `/home/dev` with `devvm update`. --- ## Moved, recreated, and missing project folders (`devvm adopt` and `devvm rm`) `devvm` stores zero files inside your project directory. Instead, it hashes the project directory's resolved physical path (`paths.Physical`) into a 12-character hex ID (`vmname.HashOf(dir)`) and stores both the path (`"dir"`) and the directory's filesystem identity (`"folder"`) in `~/.local/share/devvm/vms//vm.json`. ### How `devvm` identifies project folders (`FolderID`) `platform.FolderID(path)` (`internal/platform/folder_darwin.go` and `folder_linux.go`) records a composite string of device ID, inode number, and creation (birth) timestamp: ```text ::. ``` - **macOS (`darwin`):** Reads `st.Dev`, `st.Ino`, and `st.Birthtimespec` via `os.Lstat`. - **Linux (`linux`):** Calls `unix.Statx` with `STATX_BTIME | STATX_INO`. If the kernel or filesystem sets `STATX_BTIME`, formats `:::.`; otherwise falls back to `:` without a birth time. Automatic move detection (`finder.moveHere` in `internal/vm/move.go`) only trusts `FolderID` automatically when `platform.UniqueFolderID(dir, id)` returns `true`: 1. `id` includes a birth timestamp (`strings.Count(id, ":") == 2`), and 2. `dir` resides on a local filesystem whose inode and birth time are stable and never fabricated (`apfs` or `hfs` on macOS; `ext4`, `xfs`, `btrfs`, `tmpfs`, `zfs`, `f2fs`, or `bcachefs` on Linux), and 3. The current user owns the directory (`uid == os.Getuid()`). ### Resolving folder state changes | Scenario | What `devvm` detects | How to resolve | | --- | --- | --- | | **Moved or renamed on the same disk** (`mv ~/code/app ~/code/app-v2` or symlinked old path to new path) | Current directory has no VM of its own, `UniqueFolderID` matches an existing finished non-clone VM's `st.Folder`, and the old path is gone (`FolderGone`) or is a symlink to the new path. | • **Interactive terminal:** Run `devvm` in the new folder and answer `y` to `Folder moved from to . Move VM here? [y/N]`.
• **Non-interactive / scripts:** Run `devvm adopt` (or `devvm adopt --force`) in the new folder. | | **Deleted and recreated at the same path** (`rm -rf app && mkdir app` or fresh `git clone` at the same path) | `st.Dir` matches the current directory, but `platform.FolderID(dir) != st.Folder`. Commands refuse with `This folder was recreated since VM was made for it`. | • **Keep the existing VM for the new folder:** Run `devvm adopt` (or `devvm adopt --force`). This updates `st.Folder` and resets `.git` protection state (`gitprotect.Made{}`) so `.git` is locked fresh on the next launch.
• **Discard the old VM:** Run `devvm rm` in an interactive terminal (for safety, `--force` does **not** bypass the `Delete VM from the earlier folder at ?` prompt, and agent history export is skipped). | | **Copied across disks or onto NFS/SMB/FUSE** (`cp -R` to another volume, then deleted original) | New directory has a different `::`, so automatic move detection does not match. | 1. Ensure the original folder has been deleted or its drive is disconnected (`devvm adopt ` refuses with `VM 's folder still exists at ` while the old folder is present).
2. In the new folder, run `devvm adopt ` (or `devvm adopt --force`). | | **Deleted project folder (`(gone)` in `devvm ls`)** | `FolderOf(st)` walks up to the nearest existing parent directory and sees it is on the **same** device as `st.Folder`, proving the folder was deleted rather than unmounted. | • **Attach to a replacement folder:** `cd` into the new folder and run `devvm adopt `.
• **Delete the VM:** Run `devvm rm `. If the VM is stopped and has coding agents installed, pass `--no-export` (`devvm rm --no-export `) because starting a stopped Lima VM to export agent history would cause Lima to recreate the missing host mount directory. If the VM is a `--clone` VM, pass `--no-clone-check` (`devvm rm --no-clone-check `). | | **Project folder on an unmounted drive (`(not_connected)` in `devvm ls`)** | `FolderOf(st)` walks up to the nearest existing ancestor directory (such as `/Volumes` or `/mnt`) and sees its device ID differs from the device prefix in `st.Folder`. | • Reconnect the external drive or volume and run `devvm`.
• To delete the VM without reconnecting the drive, run `devvm rm ` in an interactive terminal (`--force` never bypasses the `not connected` confirmation prompt). | | **Nested project folder replaced by a symlink or another folder** | Checked three times during startup (`folderChanged` in `internal/vm/start.go`): before host checks, immediately before `b.Start`, and immediately after `b.Start`. | `devvm` refuses to start (`project_folder_changed`) or immediately stops the VM and proxy (`project_folder_changed_stopped`) if the directory was swapped or replaced by a symlink during startup. Restore the real directory or run `devvm adopt`. | `--clone` VMs cannot be moved or adopted (`devvm adopt` refuses with `Clone VM cannot be adopted`) because their git worktree resides inside the guest disk rather than on a host mount. ### What `devvm` updates when a VM moves to a new folder When a VM is moved (either by answering `y` to the move prompt or by running `devvm adopt` / `devvm adopt `): 1. **Stops the running VM and host daemons (`stopForMove`):** If the VM is running, `devvm` stops the guest VM, stops its host proxy (`_proxy`), port forwarder (`_ports`), and inter-VM link sessions (`_links`), and refreshes `~/.local/share/devvm/ssh_config` so the VM restarts with a mount pointing to the new host path. 2. **Updates `vm.json` and records the previous path:** Updates `st.Dir` and `st.Folder` in `~/.local/share/devvm/vms//vm.json` (the VM's name and `` stay unchanged) and saves the previous path in the `moved` section (`state.MovedSection`) until the next guest sync completes. 3. **Preserves or resets `.git` protection:** - For a same-folder move (`moveHere` or `adoptVM` with an explicit `` argument), `.git` protection state is preserved across the move. - For a folder recreated in place (`devvm adopt` with no arguments), `gitprotect.Save(st, gitprotect.Made{})` clears the recorded `.git` lock state so the new directory's `.git` is locked fresh on the next `devvm` launch. 4. **Migrates coding agent session history:** - **Claude Code:** On the next launch, `sync.sh` passes the old folder path to `assets/guest/claude.sh`, which renames `~/.claude/projects/` to `~/.claude/projects/` (where `` replaces every non-alphanumeric character with `-`) and updates the key in `~/.claude.json` under `projects` so `claude --continue` and `claude --resume` find prior conversations immediately. - **Codex:** Stores session transcripts by date under `~/.codex/sessions` tagged with the working directory; after moving a folder, run `codex resume --all` inside the VM to view sessions from the previous path. - **opencode:** Indexes sessions by git root commit in `~/.local/share/opencode`, so sessions remain attached across directory moves as long as the git repository history is unchanged. ## Related pages Install devvm, Lima, QEMU/KVM on Linux, and verify host prerequisites. Understand how devvm creates, starts, stops, updates, and tracks project VMs in vm.json. Connect to VMs over SSH, VS Code Remote-SSH, SSH agent forwarding, and VNC desktop. Debug blocked proxy requests, TLS inspection CAs, disk space floors, and garbage collection. --- ## 20. Troubleshoot network policies, CAs, and disk space > Inspect blocked requests with devvm network log, configure corporate HTTP/HTTPS proxies and custom CAs in ~/.config/devvm/ca, and recover disk space. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/20-troubleshoot-network-policies-cas-and-disk-space.md - Generated: 2026-11-07T22:51:14.840Z ### Source Files - `internal/cli/network.go` - `internal/netpolicy/log.go` - `internal/trust/trust.go` - `internal/settings/companyproxy.go` - `internal/vm/housekeep.go` - `internal/cli/gc.go` --- title: "Troubleshoot network policies, CAs, and disk space" description: "Inspect blocked requests with devvm network log, configure corporate HTTP/HTTPS proxies and custom CAs in ~/.config/devvm/ca, and recover disk space." --- `devvm` confines VM network traffic through a per-VM host proxy (`devvm _proxy`), validates custom and corporate root certificates from `~/.config/devvm/ca`, and enforces pre-flight disk space floors before building images or creating VMs. When requests fail, TLS handshakes are rejected, or disk checks block a command, use `devvm network log`, `devvm doctor`, `devvm space`, and `devvm gc` to diagnose the failure and restore operation. ```mermaid flowchart LR subgraph Guest["Guest VM"] App["Process / Agent"] Why["devvm-why / Prompt Hook"] LocalProxy["devvm-local-proxy
(127.0.0.1 socat)"] GuestTrust["/etc/ssl/certs/ca-certificates.crt
/etc/ssl/devvm.pem"] end subgraph Host["Host Machine"] Proxy["devvm _proxy"] NetLog["vms/<hash>/logs/network.jsonl
(+ .1 rotation)"] PolicyFiles["Built-in policies
~/.config/devvm/policy
vms/<hash>/policy"] CADir["~/.config/devvm/ca/*.pem, *.crt"] VMCA["vms/<hash>/run/ca/
(ca.pem, next.pem)"] Config["HTTPS_PROXY / HTTP_PROXY
or config.toml [proxy]
+ secret 'proxy'"] end subgraph External["Upstream Network"] CorpProxy["Corporate HTTP Proxy"] Target["Target Host"] end App -->|"HTTP_PROXY / HTTPS_PROXY"| Proxy App -.->|"GUI / Browsers"| LocalProxy --> Proxy Why -->|"http://devvm.proxy/blocked"| Proxy PolicyFiles --> Proxy CADir -->|"Sync on launch"| GuestTrust VMCA -->|"Sync on launch"| GuestTrust CADir -->|"UpstreamCAs"| Proxy VMCA -->|"TLS interception"| Proxy Proxy -->|"Append events"| NetLog Config --> Proxy Proxy -->|"CONNECT / HTTP"| CorpProxy --> Target Proxy -->|"Direct (if no upstream / NO_PROXY)"| Target ``` ## Troubleshoot blocked requests and network policies On the `allowlist`, `internet`, and `none` tiers, a VM cannot resolve DNS names or open outbound connections directly. All outbound traffic must pass through the VM's host proxy (`devvm _proxy`), which logs every decision to `~/.local/share/devvm/vms//logs/network.jsonl` (rotating to `network.jsonl.1` at 10 MiB, keeping up to ~20 MiB per VM). ### Inspect blocked requests You can inspect blocked connections from either the host or inside the VM: Run `devvm network log --blocked` in the project folder to aggregate all blocked requests across `network.jsonl` and `network.jsonl.1` by host and port, ordered by the most recent block: ```sh devvm network log --blocked ``` ```text LAST COUNT HOST ALLOW WITH 2026-10-09T10:01:00Z 2 example.com:443 devvm network allow example.com 2026-10-09T10:02:15Z 1 10.0.0.5:5432 devvm network allow 10.0.0.5:5432 2026-10-09T10:03:00Z 1 api.github.com:443 - ``` To stream live connection decisions (allowed tunnels, decrypted secret injections, Nix cache hits, VM link connections, and blocks) every 250 ms as they occur: ```sh devvm network log --follow ``` In an interactive bash shell inside the VM, a `PROMPT_COMMAND` hook (`~/.bashrc.d/devvm-blocked.sh`) queries `http://devvm.proxy/blocked/new?after=N` before each prompt and prints newly blocked hosts and the host command to allow them to standard error. To view the last 50 blocked requests at any time from inside the VM (including from coding agents): ```sh devvm-why ``` `devvm-why` executes `curl -sS --proxy "$HTTP_PROXY" http://devvm.proxy/blocked`. Pass `--json` to inspect full event metadata (`method`, `path`, `rule`, `reason`, `agent`, `secrets`, `refused`, `upstream`, `target`) or aggregated block records: ```sh devvm network log --lines 20 --json devvm network log --blocked --json ``` `--blocked` cannot be combined with `--follow` or `--lines` (`-n`), and `--json` cannot be combined with `--follow`. Passing incompatible flags exits with code `2`. ### Understand why `ALLOW WITH` shows `-` In `devvm network log --blocked`, the `ALLOW WITH` column prints a paste-ready `devvm network allow ` command whenever an allowlist rule can permit the request (and includes `"allow": ""` in `--blocked --json`). It prints `-` (and omits `allow` in JSON) in two cases: 1. **Unconditional proxy refusals (`"refused": true`):** The request matched a security boundary that `devvm _proxy` refuses regardless of allowlist rules, such as GitHub endpoints that mint new credentials or requests blocked while a policy file contains a broken deny rule. 2. **Invalid policy target:** The requested host string is not a valid hostname or IP address that `proxy.AllowRule` can format into a policy rule. ### Allow or deny hosts immediately Add or remove rules for the current project's VM (`~/.local/share/devvm/vms//policy`) or globally for all VMs (`~/.config/devvm/policy` with `-g` / `--global`): ```sh # Allow a host on ports 80 and 443 for this folder's VM devvm network allow example.com # Allow any subdomain of example.com devvm network allow '*.example.com' # Allow a specific port or internal IP/CIDR devvm network allow 10.0.0.5:5432 # Allow a host for every VM devvm network allow -g pypi.org # Deny a host (denies override any allow rule) devvm network deny tracker.example.com # Remove an allow or deny rule devvm network rm example.com devvm network rm '!tracker.example.com' ``` `devvm _proxy` watches policy files and reloads rules on the next connection without restarting the VM. If you run `devvm network allow` while the VM is on the `internet` or `none` tier, `devvm` saves the rule to the policy file but prints a note that only `!RULE` denies apply on `internet` and `none`. Switch the VM to `allowlist` (`devvm network allowlist`) for allow rules to take effect. ### Fix malformed policy lines and broken denies If you edit `~/.config/devvm/policy`, `~/.config/devvm/layers/.policy`, or `~/.local/share/devvm/vms//policy` by hand, `devvm` validates every non-comment line: | Symptom | Cause | Effect | Resolution | | --- | --- | --- | --- | | `devvm network` prints `# not a rule, skipped (line N)` | A line contains invalid syntax (e.g., `example.com:x`, `a*.example.com`, or spaces in a host rule). | The proxy skips that line and applies the remaining valid rules. `devvm doctor` and `devvm network` emit a warning. | Edit the file named in the header (`# /path/to/policy`) and fix or delete line `N`. | | Every outbound request is blocked and `devvm doctor` warns of a broken deny | A line starts with `!` but is not a valid rule (e.g., `! example.com` with a space after `!`). | Because the proxy cannot determine what the rule intended to deny, it fails closed and **denies all new outbound connections** (except VM links) for every VM reading that file. | Fix the syntax (remove the space after `!`) or remove the line with `devvm network rm` or an editor. | ### Resolve DNS errors when tools bypass the proxy If a command inside a confined VM fails with `no such host`, `Temporary failure in name resolution`, `ENOTFOUND`, or `EAI_AGAIN` for a host that is already on the allowlist—and `devvm network log` and `devvm-why` show no blocked event—the tool attempted to resolve DNS or connect directly instead of using `HTTP_PROXY` / `HTTPS_PROXY`. Configure the tool to use the proxy environment variables set in the VM: | Tool / Runtime | Why it bypasses the proxy | How to fix inside the VM | | --- | --- | --- | | **Node.js `fetch` (older releases)** | Built-in `fetch` honors `NODE_USE_ENV_PROXY=1` (which `devvm` sets automatically) only in Node `>=24.0.0` and `>=22.21.0`. | Upgrade the project's Node version to `>=22.21.0` or `>=24.0.0`, or configure an explicit `undici.ProxyAgent` using `process.env.HTTPS_PROXY`. | | **Python `aiohttp`** | `aiohttp.ClientSession` ignores environment proxies by default. | Pass `trust_env=True` when constructing `aiohttp.ClientSession(trust_env=True)`. | | **Go custom `http.Transport`** | Custom `http.Transport{}` structs omit `Proxy` unless set. | Set `Proxy: http.ProxyFromEnvironment` on the custom `http.Transport`. | | **Java / JVM tools** | The JVM ignores `HTTP_PROXY` and `HTTPS_PROXY`. | Pass `-Dhttps.proxyHost= -Dhttps.proxyPort=` (parsed from `$HTTPS_PROXY`). | | **Hugo (`resources.GetRemote`)** | Hugo disables environment proxies unless enabled in site configuration. | Set `security.http.proxyFromEnvironment = true` in `hugo.toml` / `config.toml`. | | **Desktop browsers / GUI apps** | GUI browsers prompt interactively for proxy credentials instead of reading `http://devvm:KEY@host:port` from `$HTTP_PROXY`. | Run `devvm-local-proxy start` in the VM and point the app at `$(devvm-local-proxy url)`. `devvm-local-proxy` runs a loopback `socat` listener that injects the VM's `Proxy-Authorization` header automatically. | ### Fix `unenforced` network status `devvm ls` and `devvm network` report a VM's policy as `unenforced` when the VM is configured for a confined tier (`allowlist`, `internet`, or `none`) but is currently running without network confinement or failed `Proxy.SelfCheck`: - **VM started outside `devvm` or switched from `open` while running:** Run `devvm` in the project folder and confirm the prompt to restart the VM confined, or run `devvm restart` directly. In non-interactive mode or with `--no-input`, `devvm` stops the unconfined VM so the next `devvm` launch starts it confined. - **Direct route detected while confined (`NetDirectRoute`):** If `Proxy.SelfCheck` detects that `curl --noproxy '*' -sS -m 3 -o /dev/null https://1.1.1.1` succeeds from inside a VM that the backend started confined, `devvm` stops the VM immediately and errors because an external network interface is routing around the proxy. - **Host proxy failed to start or port conflict:** For confined VMs, if `devvm _proxy` cannot start, `devvm` stops the VM (`NetProxyDownStopped`) so it never runs unproxied. Run `devvm restart` to allocate a fresh host port and restart the proxy. For `open` VMs, if `http://devvm.proxy/blocked` does not answer during the startup check, `devvm` warns (`NetProxyUnreachable`) and leaves the VM running; run `devvm restart` so tools that use the proxy (`apt`, `docker`, `devvm-why`) work. --- ## Configure corporate HTTP/HTTPS proxies When the host machine accesses the internet through a corporate proxy, `devvm _proxy`, image/tool builds, host `mise`, and OAuth token refreshes route allowed outbound requests through that upstream proxy. ### Proxy configuration precedence `settings.HostProxy` resolves the corporate proxy using all-or-nothing precedence between the shell environment and `~/.config/devvm/config.toml`: 1. **Shell environment (`HTTPS_PROXY`, `HTTP_PROXY`, `NO_PROXY` or lowercase):** If the shell where you run `devvm` sets either `HTTPS_PROXY` (`https_proxy`) or `HTTP_PROXY` (`http_proxy`), `devvm` uses `HTTPS_PROXY`, `HTTP_PROXY`, and `NO_PROXY` from the environment and ignores `config.toml`'s `[proxy]` table completely. 2. **`~/.config/devvm/config.toml` `[proxy]` table:** When neither `HTTPS_PROXY` nor `HTTP_PROXY` is set in the environment (for example, inside an IDE terminal that does not source your shell profile), `devvm` reads `[proxy]` from `config.toml`. Add the `[proxy]` table to `~/.config/devvm/config.toml`. Use an `http://` URL (or `host:port`, which defaults to `http://` and port `80`). If the proxy requires authentication, include **only the username** in the URL—never the password: ```toml title="~/.config/devvm/config.toml" [proxy] https = "http://alice@proxy.example.com:8080" http = "http://alice@proxy.example.com:8080" no-proxy = "localhost,.example.com,10.0.0.0/8" ``` When `proxy.https` or `proxy.http` in `config.toml` includes a username, save the password in the `proxy` secret (stored in the macOS Keychain under service `devvm` or `~/.local/share/devvm/secrets/proxy` with mode `0600` on Linux): ```sh devvm secret set proxy # Or from standard input: printf '%s\n' "$CORP_PROXY_PASS" | devvm secret set proxy --stdin ``` Re-run `devvm` or `devvm restart` in each running VM's project folder so its `devvm _proxy` restarts with the upstream proxy configuration and credentials: ```sh devvm restart ``` ### Corporate proxy error cases and constraints - **Password in `config.toml` rejected:** `checkedProxyURL` refuses any `proxy.https` or `proxy.http` value in `config.toml` that contains a `user:password@...` password. Remove `:password` from `config.toml` and run `devvm secret set proxy`. - **Missing `proxy` secret (`proxy_password_missing`):** If `config.toml` names a username in `proxy.https` or `proxy.http` and the `proxy` secret has not been saved, `devvm` refuses to create VMs, start VM proxies, or run builds before any process starts. - **Unsupported proxy scheme (`https://` or `socks5://`):** `proxy.ParseUpstream` supports only `http://` upstream proxies (`ErrUpstreamScheme`). If `HTTPS_PROXY`, `HTTP_PROXY`, or `config.toml` specifies `https://` or `socks5://`, replace it with the proxy's `http://` endpoint. - **Unresolved hostnames on the `internet` tier or under `**`:** When routing through an upstream proxy, `devvm _proxy` first attempts to resolve the requested hostname on the host (with a 3-second timeout) to verify that it does not resolve to an internal or denied IP address (`floorUpstream`). If the host machine cannot resolve external DNS names (because only the corporate proxy resolves them), `devvm _proxy` forwards the unresolved name to the corporate proxy **only when the VM's policy explicitly matches the hostname** (`example.com` or `*.example.com`). If a host is allowed only by `**` (including on the `internet` tier), `devvm _proxy` blocks it with: ```text this machine can't resolve it, and only ** allows it; allow the name itself to reach it through the company proxy ``` Run `devvm network allow ` (or `devvm network allow '*.domain.com'`) to allow the name explicitly. - **Upstream proxy disappeared between launches:** If a VM's proxy previously used an upstream proxy (from shell environment variables) and you later run `devvm` from a shell without `HTTPS_PROXY`/`HTTP_PROXY` and without `[proxy]` in `config.toml`, `devvm` warns and starts the proxy with direct connections. Set `[proxy]` in `~/.config/devvm/config.toml` so every shell uses the corporate proxy consistently. --- ## Configure custom CAs in `~/.config/devvm/ca` When a corporate proxy inspects TLS traffic (such as Zscaler or Netskope) or when your organization runs internal HTTPS services signed by a private root CA, place the root CA certificates in `~/.config/devvm/ca`. ### Install custom CA certificates Create `~/.config/devvm/ca` and copy your root CA certificate files into it with a `.pem` or `.crt` extension: ```sh mkdir -p ~/.config/devvm/ca cp /path/to/corporate-root-ca.pem ~/.config/devvm/ca/corporate-root-ca.crt ``` Run `devvm doctor` to verify that `trust.Read` parses the folder without warnings: ```sh devvm doctor ``` If `HTTPS_PROXY` or `config.toml`'s `proxy.https` is set and `~/.config/devvm/ca` contains no certificates, `devvm doctor` notes that a TLS-inspecting proxy needs its CA in `~/.config/devvm/ca`. Run `devvm` in each project folder. On every launch, `devvm` sends the parsed certificates to the VM and runs `ca.sh` to update the system trust store and `/etc/ssl/devvm.pem` without rebuilding any image. ### Strict validation rules for `~/.config/devvm/ca` `trust.Read` validates `~/.config/devvm/ca` before every VM launch, image build, and shared tool resolution. If **any** file fails validation, `devvm` refuses the entire folder and names the offending path: | Rule | Refusal condition | Why `devvm` enforces it | | --- | --- | --- | | **Regular files only (no symlinks)** | `~/.config/devvm/ca` itself is a symlink (`CAFolderLink`), or any `*.pem` / `*.crt` entry is a symlink (`CALink`). | A symlink pointing into a project folder would allow untrusted code inside that VM to replace the root CA trusted by all other VMs, builds, and host proxies. Always `cp` files into `~/.config/devvm/ca`. | | **Extensions** | Only `*.pem` and `*.crt` (case-insensitive) are read; other filenames are ignored. | Prevents editor swap files or READMEs from breaking validation. | | **PEM `CERTIFICATE` blocks only** | File contains a `PRIVATE KEY` block, non-certificate PEM block, or zero certificates (`CANotCertificate`), or corrupt ASN.1 DER (`CAInvalid`). | Ensures private keys are never copied into VMs or build scripts. `trust.Read` re-encodes only parsed `x509.Certificate.Raw` bytes. | | **Must be a CA certificate** | Certificate lacks `BasicConstraintsValid` or `IsCA: true` (`CANotCA`). | Leaf server certificates cannot act as trust anchors in `update-ca-certificates` or X.509 cert pools. | ### How CAs are trusted across guests, builds, and the host Inside every project VM, `ca.sh` installs each user CA as `/usr/local/share/ca-certificates/devvm-extra-N.crt` alongside the VM's own proxy CAs (`devvm-proxy.crt` and `devvm-proxy-next.crt`), runs `update-ca-certificates --fresh` to rebuild `/etc/ssl/certs/ca-certificates.crt` (and triggers `/etc/ca-certificates/update.d` hooks such as Java's `cacerts`), and writes all `devvm-*.crt` certificates to `/etc/ssl/devvm.pem`. Every VM exports the following environment variables (`trust.Env`) so runtimes that do not read the OS trust store by default trust both the system bundle and `devvm`'s CAs: | Environment variable | Value in guest | Target tools and runtimes | | --- | --- | --- | | `NODE_EXTRA_CA_CERTS` | `/etc/ssl/devvm.pem` | Node.js, npm, pnpm, Bun | | `SSL_CERT_FILE` | `/etc/ssl/certs/ca-certificates.crt` | OpenSSL, Go `crypto/tls`, Rust `rustls-native-certs` | | `REQUESTS_CA_BUNDLE` | `/etc/ssl/certs/ca-certificates.crt` | Python `requests` | | `PIP_CERT` | `/etc/ssl/certs/ca-certificates.crt` | Python `pip` | | `UV_SYSTEM_CERTS` | `1` | `uv` | - **Image and tool builds:** During `devvm build`, `devvm rebuild`, and shared tool builds, `CAs.Build` installs `~/.config/devvm/ca` certificates into the temporary build VM and exports `trust.Env` at the start of each build script, then strips all `devvm-*.crt` certificates and `/etc/ssl/devvm.pem` before finalizing the image. Images never retain corporate CAs, and adding or removing a CA never invalidates image cache keys. - **Host `mise` (`invalid peer certificate: UnknownIssuer`):** When `devvm` resolves shared tool versions on the host using its managed `mise` binary: - On **Linux**, `CAs.Host` combines the host's CA bundle with `~/.config/devvm/ca` into a temporary bundle and sets `SSL_CERT_FILE` (and `SSL_CERT_DIR` when neither `SSL_CERT_FILE` nor `SSL_CERT_DIR` is set in the host environment). - On **macOS**, `mise`'s Sigstore/GitHub build-provenance verifier queries the macOS Keychain directly. In addition to placing the certificate in `~/.config/devvm/ca`, import the corporate CA into **Keychain Access** on macOS and set its trust setting to **Always Trust**. ### Expired per-VM proxy CA (`devvm's CA has ended`) To inject secrets (`claude`, `github`, and agent OAuth logins) into HTTPS requests, each VM's proxy uses a per-VM ECDSA P-256 CA stored in `~/.local/share/devvm/vms//run/ca/` (`ca.pem`/`ca.key` and `next.pem`/`next.key`). - Each CA is valid for 30 days (`caLife`), and `next.pem` becomes valid 7 days plus 1 hour before `ca.pem` expires (`caRenew + caSkew`). - Every time you run `devvm` in the project folder, `trust.InitCA` rotates any CA with less than 7 days remaining and installs both `ca.pem` and `next.pem` into the VM. - If a VM stays running for more than 30 days without `devvm` being run on the host (for example, when you connect exclusively via `ssh `), `next.pem` eventually expires. Requests to secret-bearing hosts then fail and `devvm network log --blocked` shows the reason `devvm's CA has ended`. - **Fix:** Run `devvm` in the project folder to generate a new CA pair, install it into the VM, and restart the VM's proxy. --- ## Audit and recover disk space Before building an image, building shared tools, or creating a VM, `backend.Space.Check` checks `statfs` available space on the target volumes and refuses the operation if free space falls below the step's floor: | Operation | macOS (Lima) floor | Data directory (`~/.local/share/devvm`) floor | | --- | --- | --- | | Build `base` image | `8 GiB` | — | | Build layer image or create VM | `2 GiB` | — | | Build `N` shared tool images | `4 GiB` (1 tool) to `8 GiB` (10 tools) | `512 MiB + (N × 256 MiB)` | ### Inspect storage usage with `devvm space` Run `devvm space` (or `devvm space --json`) to inspect every category of disk usage across `~/.local/share/devvm` and Lima without starting any VM: ```sh devvm space ``` `devvm space` prints two tables and diagnostic notes: 1. **`devvm` storage table:** Lists `VMs`, `images`, `tools gnu` / `tools musl`, `history` (agent history exports in `exports/`), `network logs` (`vms//logs/` and `logs/tools-check.log`), `run logs` (`logs/runs/*.log`), `mise` (`cache/mise/`), `packages` (`packages/*.deb`), `Nix cache` (`cache/nix-cache/`), `leftover state` (orphaned VM folders and dead link sockets), and `state`. For each row, `GC` shows how much `devvm gc` will free immediately, `FREED BY` shows the command that frees the remainder, and `RETAIN` shows any active retention policy and how much it would free now. 2. **Backend caches table:** Lists Lima's download cache (`~/Library/Caches/lima` on macOS, `~/.cache/lima` or `$XDG_CACHE_HOME/lima` on Linux), which stores downloaded Ubuntu cloud images (`600–900 MB` compressed plus `~2.4 GB` raw disk conversion on macOS). 3. **Diagnostic notes:** Lists `strays` (Lima VMs bearing `devvm`'s label that belong to a different `XDG_DATA_HOME`), `folder_gone` (VMs whose project folder was moved or deleted), and `folder_not_connected` (VMs whose project folder is on an unmounted external drive). ### Reclaim disk space with `devvm gc` `devvm gc` never deletes a project VM. By default, an interactive `devvm gc` run lists what it plans to remove, asks for confirmation (`--force` skips the prompt), and deletes: - **Stale and unfinished images** (`STATE` is `stale` or `unfinished` in `devvm images`). - **Unused shared tool images** (`STATE` is `unused` in `devvm images`—older tool versions not referenced by any VM's `vm.json`). - **Stale host `mise` binaries** in `~/.local/share/devvm/cache/mise/` left behind after `devvm` upgrades (skipping any modified within the last hour). - **Stale `.deb` package sets and files** in `~/.local/share/devvm/packages/` no longer referenced by any current or built `base` image. - **Shared Nix cache files over the cap** in `~/.local/share/devvm/cache/nix-cache/`, evicting least-recently-used files until the cache is within `nix-cache.size` (`10 GiB` when unset). Use `devvm gc` flags to recover additional space: | Command / Flag | What it deletes | Constraints and safety checks | | --- | --- | --- | | `devvm gc --tool-images [TOOL...]` | Current shared tool images (`tools/*.erofs`) that no VM uses (`VMS` is `0`), or only those matching `TOOL...` (`node`, `python`, `rust`, `go`, `dotnet`, `claude`, `codex`, `opencode`). | Acquires `making.lock` (`tools.Making`). If another `devvm` process is currently creating a VM or updating tools, `gc` keeps all current unused tool images and reports which ones it kept. | | `devvm gc --nix-cache` | The entire shared Nix cache (`~/.local/share/devvm/cache/nix-cache/`), not just files over the 10 GiB cap. | VMs with the `nix` layer re-fetch needed narinfos/nars from `cache.nixos.org` on demand. | | `devvm gc --history [--older-than DURATION]` | Saved agent history archives in `~/.local/share/devvm/exports/*.tar.gz` (and `.folder` files). With `--older-than` (e.g., `30d`, `2w`, `12h`), deletes only exports at least that old. | Exports are the `only copy` of deleted VMs' agent transcripts; `devvm gc` never touches them without `--history`. `--older-than` requires `--history`. | | `devvm gc --orphans` | State folders (`vms//`), SSH config entries, dead VM links, and unused link sockets (`run/`) for VMs deleted outside `devvm` (e.g., via `limactl delete`). | Skips VMs whose backend is unreachable, VMs created under a different `DEVVM_LIMA_HOME`, and VMs whose `vm.json` changed within the last hour. Once orphan state is removed, shared tools held only by that orphan become eligible for `devvm gc`. | | `devvm gc --caches` | `devvm`'s unused Ubuntu cloud image downloads inside Lima's cache (`~/Library/Caches/lima` or `~/.cache/lima`). | Deletes only Ubuntu cloud image cache entries from `devvm`'s download URL that no `devvm` image record references and that have not changed in the last hour. Never runs `limactl prune`. | | `devvm gc --retention` | Immediately runs all configured `[retention]` policies (`logs.runs`, `history.age`, `history.count`, `tools.unused-age`, `orphans.age`, `nix-cache.size`). | Unlike daily automatic retention at command startup, `devvm gc --retention` also queries backends to apply `orphans.age`. | | `devvm gc --force` | Runs any of the above without interactive confirmation prompts. | Required when running `devvm gc` without a terminal or with `--no-input`. | ### Delete current images, old VMs, and stray instances When `devvm gc` does not free enough disk space, inspect and remove current images or unneeded VMs: `devvm gc` keeps images whose `STATE` is `current`. To list all images with their sizes and delete a current image by name or key: ```sh devvm images devvm images rm docker+desktop ``` - On **macOS**, project VMs are APFS clones of the image disk, so deleting a current image succeeds even while VMs cloned from it exist (note that shared APFS blocks are only freed on disk once both the image and all VMs sharing those blocks are deleted). - On **Linux**, project VMs use QCOW2 overlays backed by the image's disk. `devvm images rm` and `devvm gc` refuse to delete an image while any VM's overlay depends on it, naming the VMs that use it. When you delete or move a project directory on the host, `devvm` never auto-deletes the VM. `devvm ls` marks the folder `(gone)` and `devvm space` lists it under `folder_gone`. - If the folder was moved, run `devvm adopt ` inside the new folder location. - To delete the VM and free its disk, pass the VM's name to `devvm rm` from any directory. If the VM is stopped and has a coding agent, pass `--no-export` so `devvm rm` does not start the VM (which would recreate the missing mount folder) to export history: ```sh devvm rm devvm-my-project-71a99b1076d92c --no-export ``` - **Stray Lima VMs:** If `devvm space` reports a stray VM (a Lima VM with `devvm`'s label that has no state in `~/.local/share/devvm/vms/`, such as one created under a different `XDG_DATA_HOME`), delete it with: ```sh devvm limactl delete ``` - **Incomplete folders in `vms/`:** If `devvm doctor` warns of a folder in `~/.local/share/devvm/vms/` that has no `vm.json` (left behind by a killed process) or an unrecognized entry, remove that directory directly with `rm -rf ~/.local/share/devvm/vms/`. If a shared tool image in `~/.local/share/devvm/tools/` was manually deleted while a VM's `vm.json` still lists it, launching the VM stops with an error naming the unmounted tool (`/run/devvm/tools-unmounted`). Rebuild and remount the missing tool image by running: ```sh devvm tools update ``` ## Related pages Traffic confinement across allowlist, open, and offline tiers, host-side TLS SNI and HTTP inspection via devvm _proxy, and per-VM CA rotation. Configure automatic retention policies for run logs, agent history exports, unused tools, orphaned state, and Nix caches, and upgrade devvm installations. Resolve devvm doctor host problems, Ubuntu cloud image boot fallbacks, SSH host-key verification errors, run logs, and moved project directories with devvm adopt. Syntax, subcommands, and flags for devvm network, secret, agent, git-protect, clone, ssh-agent, link, and group. Syntax, subcommands, and flags for devvm build, rebuild, images, layer, tools, space, gc, retention, setup, doctor, upgrade, completion, and limactl. Complete reference for ~/.config/devvm/config.toml keys, settings.json precedence, environment variables, state directory layout, and version 1 --json payloads. --- ## 21. Manage disk retention, garbage collection, and upgrades > Configure automatic retention policies for run logs, agent history exports, unused tools, orphaned state, and Nix caches, and upgrade devvm installations. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/21-manage-disk-retention-garbage-collection-and-upgrades.md - Generated: 2026-11-07T22:54:24.105Z ### Source Files - `internal/vm/housekeep.go` - `internal/vm/exports.go` - `internal/settings/retention.go` - `internal/upgrade/upgrade.go` - `internal/upgrade/source.go` - `internal/upgrade/install.go` --- title: "Manage disk retention, garbage collection, and upgrades" description: "Configure automatic retention policies for run logs, agent history exports, unused tools, orphaned state, and Nix caches, and upgrade devvm installations." --- `devvm` stores virtual machine disks, base and layer images, read-only shared tool images, agent history exports, run logs, and build caches under `~/.local/share/devvm` (`$XDG_DATA_HOME/devvm`). Through `vm.Housekeeping`, `devvm space` inspects disk consumption across host directories and Lima caches, `devvm retention` configures automatic daily pruning policies, and `devvm gc` executes interactive or scripted garbage collection. For the CLI binary itself, `devvm upgrade` (`upgrade.Upgrader`) performs Minisign-verified atomic binary upgrades for script installations or delegates to Homebrew for formula installations. Neither `devvm gc` nor automatic retention ever deletes a project VM (`vms//` backed by an existing backend instance) or a current base/layer image. Project VMs can contain unsaved work and are only removed explicitly with `devvm rm`. ## Inspect disk usage (`devvm space`) `devvm space` inventories everything `devvm` and its backends store on the host without starting any VM or modifying state. Each registered backend is queried with a 30-second timeout (`spaceTimeout`). ```sh devvm space devvm space --json ``` ### Reported storage categories `devvm space` outputs two tables: a primary table of `devvm`-managed assets with a `total` summary row, and a secondary `CACHE` table for backend-owned caches (such as Lima's download cache at `~/Library/Caches/lima` on macOS or `~/.cache/lima` on Linux). | `WHAT` row | Host location (relative to `~/.local/share/devvm`) | `KEPT` classification | `GC` column (`devvm gc` default) | `FREED BY` column | Applicable `RETAIN` keys | | --- | --- | --- | --- | --- | --- | | `VMs` | `lima//` | `only copy` | `0` | `devvm rm` | — | | `images` | `lima/devvm-img-/` and `images//` | `remade` | Stale and unfinished images | `images rm` | — | | `tools gnu` / `tools musl` | `tools/.erofs` and `.json` | `remade` | Tool images in `unused` state | `gc --tool-images` | `tools.unused-age` | | `history` | `exports/--.tar.gz` | `only copy` | `0` | `gc --history` | `history.age`, `history.count` | | `network logs` | `vms//logs/` and `logs/tools-check.log` | `log` | `0` | `devvm rm` | — | | `run logs` | `logs/runs/*.log` | `log` | `0` | `the newest stay` (`20` when off) | `logs.runs` | | `mise` | `cache/mise//` | `remade` | Older host `mise` versions | `-` | — | | `packages` | `packages/*.deb` and `packages/sets/` | `remade` | Unreferenced `.deb` files and sets | `rebuild base` | — | | `Nix cache` | `cache/nix-cache/` | `remade` | Bytes exceeding `nix-cache.size` cap | `gc --nix-cache` | `nix-cache.size` | | `leftover state` | `vms//` (orphans) and dead sockets in `run/` | `leftover` | `0` | `gc --orphans` | `orphans.age` | | `state` | `vms/`, `images/`, `run/`, `mise.lock*`, `cache/from-digests.json` | `state` | `0` | `-` | — | On macOS, Lima clones VM disks and child images using APFS copy-on-write (`cp -c`). Because `devvm space` measures each file's allocated blocks the way `du` does, shared blocks are counted in both the image and the cloned VM, making the reported `total` larger than actual physical disk consumption. ### VM and backend anomaly notices During inventory, `devvm space` detects four out-of-band states, printing notes to standard error and populating top-level arrays in `--json` output: - **Unreachable backends (`unknown`)**: Backends referenced by at least one VM's `vm.json` that failed `backend.Check` or timed out after 30 seconds. Their VMs and images are omitted from the totals. - **Stray VMs (`strays`)**: Backend instances carrying `devvm`'s label that have no corresponding state directory in `~/.local/share/devvm/vms/` (for example, VMs created under a different `XDG_DATA_HOME`). `devvm` never deletes strays; remove them with `devvm limactl delete `. - **Missing project folders (`folder_gone`)**: VMs that still exist in the backend whose project directory on the host was moved or deleted. They remain counted under `VMs` until re-linked with `devvm adopt` or deleted with `devvm rm `. - **Unmounted disks (`folder_not_connected`)**: VMs whose project folder resides on an unmounted external volume. `devvm rm ` requires interactive terminal confirmation and refuses `--force` when a volume is disconnected. ```sh devvm space --json ``` ```json { "version": 1, "devvm": [ { "what": "VMs", "backend": "lima", "count": 2, "size": 5368709120, "sized": true, "kept": "only copy", "gc": 0, "rest": "devvm rm" }, { "what": "history", "count": 3, "size": 3145728, "sized": true, "kept": "only copy", "gc": 0, "rest": "gc --history", "retain": "30d", "retain_frees": 1048576 } ], "total": 5371854848, "gc": 0, "backends": [ { "backend": "lima", "name": "download cache", "path": "/Users/you/Library/Caches/lima", "count": 1, "size": 629145600, "sized": true } ] } ``` --- ## Configure automatic retention (`devvm retention`) Every retention policy is `off` by default. When enabled, `devvm` applies configured retention rules automatically at command startup at most once every 24 hours (`retainEvery`), or immediately when you run `devvm gc --retention`. ### Retention keys and behavior | Key | Value format | Built-in behavior when `off` | What retention deletes | | --- | --- | --- | --- | | `logs.runs` | Integer `1`–`10000` | Keeps newest `20` run logs | Run logs in `logs/runs/*.log` beyond the newest `N` files, pruned by the UI logger on command execution. | | `history.age` | Duration (`30m`, `12h`, `90d`, `2w`) | Keeps all exports | Agent history archives (`exports/-.tar.gz` and their `.folder` metadata files) whose timestamp is at least `history.age` old. | | `history.count` | Integer `1`–`10000` | Keeps all exports | Oldest remaining agent history exports after `history.age` filtering so at most the newest `N` exports remain. | | `tools.unused-age` | Duration (`30d`, `4w`) | Keeps all current tool images | Current shared tool images in `tools/` that no VM lists in `vm.json` and that have been continuously idle for at least `tools.unused-age` (tracked in `.json`). | | `orphans.age` | Duration (`30d`, `2w`) | Keeps all orphan state | State folders (`vms//`), SSH config entries, and `--clone` git remotes for VMs missing from their backend whose `vm.json` mtime is at least `orphans.age` old. Applied **only** by `devvm gc --retention`. | | `nix-cache.size` | Binary size (`500M`, `20G`, `1T`) | `10G` (`10 GiB`) cap | Least-recently-used `.narinfo` and `nar/*` files in `cache/nix-cache/` until total size is at or below the cap. | - **Durations** (`state.ParseDuration`) require a positive integer followed by `m` (minutes), `h` (hours), `d` (days, 24h), or `w` (weeks, 7d). - **Sizes** (`state.ParseSize`) require a positive integer followed by `K`, `M`, `G`, or `T` (powers of 1024). ### Configuration precedence and CLI usage `devvm` resolves retention settings in this order: 1. `[retention]` in `~/.config/devvm/config.toml` (user-authored, read-only to `devvm`). 2. `~/.local/share/devvm/settings.json` (written by `devvm retention KEY=VALUE...`). 3. Built-in default (`off`). If a key is defined in `config.toml`, running `devvm retention =` refuses the change and names `config.toml` so the file on disk never silently overrides what you just ran. ```sh title="CLI (settings.json)" # Show current settings and their sources devvm retention # Enable automatic pruning for exports and idle current tool images devvm retention history.age=90d history.count=10 tools.unused-age=30d # Disable a specific retention setting devvm retention history.age=off # Inspect settings and sources as JSON devvm retention --json ``` ```toml title="~/.config/devvm/config.toml" [retention] logs.runs = 20 history.age = "90d" history.count = 10 tools.unused-age = "30d" orphans.age = "30d" nix-cache.size = "20G" ``` ### Daily automatic execution (`autoRetain`) At the start of normal CLI commands, `autoRetain` configures `KeepRunLogs(r.RunLogs)` and checks the modification time of `~/.local/share/devvm/retention.run`: 1. **Schedule gate**: Retention runs only if at least one setting is enabled and `retention.run` is missing, at least 24 hours old, or has a future timestamp (indicating the system clock moved backward). Commands that inspect or manage retention directly (`devvm gc`, `devvm space`, `devvm retention`, and internal plumbing commands that skip layout checks) skip `autoRetain`. 2. **Non-blocking lock**: `autoRetain` calls `platform.TryLockFile` on `~/.local/share/devvm/retention.lock`. If another `devvm` process holds the lock, `autoRetain` returns immediately without waiting. 3. **Non-blocking tool share check**: If `b.Tools.ShareBusy()` is true (an image or tool build holds `tools.lock`), `autoRetain` sets `r.ToolsAge = 0` for that run so tool pruning defers to the next day rather than blocking the command. Furthermore, if a VM is actively being created (`making.lock` is held), `retainTools` leaves all current tool images and their idle timestamps untouched. 4. **Backend-free execution**: `autoRetain` invokes `Housekeeping.Retain(ctx, b, r, false)`, passing `orphans = false` so ordinary commands never wait on backend VM listings. To apply `orphans.age`, run `devvm gc --retention`. --- ## Run garbage collection (`devvm gc`) `devvm gc` plans deletions across all reachable backends and host state directories, prints the plan, prompts for confirmation in an interactive terminal, and applies the removals in deterministic order. ```mermaid flowchart TD subgraph CLI["internal/cli (gc.go & gc_caches.go)"] GCCmd["devvm gc [flags]"] GCRet["gcRetention (--retention)"] GCCache["gcCaches (--caches)"] end subgraph Locks["Host File Locks (~/.local/share/devvm)"] ShareLock["tools.lock (LockShare)"] MakingLock["making.lock (Making.Exclude)"] RetLock["retention.lock (lockRetention)"] NixLock["cache/nix-cache/.lock (lockNix)"] end subgraph Housekeep["internal/vm (Housekeeping)"] Plan["PlanGC(ctx, b, GCOpts)"] Apply["GCPlan.Apply(ctx)"] Retain["Housekeeping.Retain(ctx, b, r, true)"] end subgraph Targets["Pruned Host & Backend Targets"] Orphans["vm.Leftovers: vms/, dead links, sockets"] Images["build.Images: stale & unfinished backend images"] Mise["tools.Versions: stale cache/mise/"] Pkgs["build.Images: stale packages/*.deb & sets"] Exports["vm.Exports: exports/-.tar.gz"] Tools["tools.Share: unused & --tool-images .erofs"] Nix["proxy.TrimNixCache: LRU narinfo & nar/*"] LimaCache["backend.DeleteCache: Lima cloud image cache"] end GCCmd --> ShareLock ShareLock -. "--tool-images" .-> MakingLock GCCmd --> Plan Plan --> Apply Apply --> Orphans Apply --> Images Apply --> Mise Apply --> Pkgs Apply --> Exports Apply --> Tools Apply --> NixLock --> Nix GCCmd -. "--retention" .-> RetLock --> GCRet --> Retain GCCmd -. "--caches" .-> GCCache --> LimaCache ``` ### Default sweep vs. opt-in flags By default, running `devvm gc` with no flags plans and deletes: 1. **Stale and unfinished images** (`build.StateStale` or `build.StateUnfinished`) on every reachable backend, skipping any image currently being built by another `devvm` process (`build.StateBuilding`). On Linux hosts, where VM disks use QCOW2 overlays backed by the image disk, the backend refuses deletion if a VM still references the stale image, and `devvm gc` warns and continues. 2. **Unused shared tool images** (`tools.StateUnused`): older tool builds in `tools/` that are neither the current version in `mise.lock` nor listed in any VM's `vm.json`. 3. **Stale host `mise` binaries**: older versions under `cache/mise/` left behind by previous `devvm` releases, excluding directories modified within the last hour. 4. **Stale base `.deb` package caches**: package set lists in `packages/sets/` for base keys that are neither current nor built, plus `.deb` files in `packages/` not referenced by any remaining set. 5. **Shared Nix cache over cap**: least-recently-used files in `cache/nix-cache/` exceeding `nix-cache.size` (or `10 GiB` when `nix-cache.size` is `off`). ### Flags and parameters Also delete `current` shared tool images in `tools/` that no VM lists in `vm.json` (across all C libraries, `gnu` and `musl`). Pass specific tool or agent names (such as `devvm gc --tool-images node claude`) to restrict deletion to those tools. If another `devvm` process holds `making.lock` (creating a VM or updating a VM's tools), `gc` keeps all current unattached tool images, reports them, and cleans the rest. Also delete saved agent history exports (`exports/--.tar.gz` and their `.folder` sidecars). In-progress export files (`.devvm-export-*`) and non-matching files are ignored. Requires `--history`. Restricts export deletion to archives whose filename timestamp (parsed in local wall-clock time) is at least `DURATION` old (for example, `30d` or `12h`). Evict the entire shared Nix cache (`Limit = 0`) instead of only files exceeding the `nix-cache.size` cap. Active in-flight downloads (`.tmp-*` modified within the last hour) are preserved. Forget state for VMs that no longer exist in their backend (such as VMs deleted directly via `limactl delete`), remove dead inter-VM links, and delete leftover link sockets in `run/` that no process listens on. After the main GC pass, inspect each reachable backend's own download cache (`be.DevvmCache`) and prompt separately to delete entries created by `devvm` builds that no image record uses and that have not changed in the last hour. Never runs `limactl prune`. After the main GC pass, immediately run `Housekeeping.Retain` with `orphans = true`, applying all configured `devvm retention` policies including `orphans.age`. Skip interactive confirmation prompts for the main GC plan and `--caches`. Required when running `devvm gc` without a terminal or with `--no-input`. ### Orphaned VM safety gates (`vm.Leftovers`) When `devvm gc --orphans` or `devvm gc --retention` (with `orphans.age`) evaluates a VM state folder `vms//`, `Leftovers.FindIn` only classifies the VM as an `Orphan` if every check below passes: 1. **Readable state**: `vm.json` exists and parses cleanly. 2. **Backend usable**: The backend named in `vm.json` was queried in the inventory, passed `backend.Check` within 30 seconds (`leftoverAsk`), and did not skip image listing. 3. **No remake in progress**: `remakePending(st)` is false (a VM mid-remake may temporarily lack a backend instance until the next `devvm` command finishes it). 4. **Confirmed missing**: Calling `b.Inspect(ctx, vmName)` explicitly returns `backend.ErrNotFound`. 5. **Matching backend home (`madeHere`)**: `st.BackendAt` matches the backend's active location in this run (or the backend is using its default `~/.local/share/devvm/lima` home when `st.BackendAt` is empty). If `DEVVM_LIMA_HOME` points elsewhere, `devvm` lists the VM under `Elsewhere` and leaves its state untouched. 6. **Minimum age (`leftoverRecent`)**: `vm.json`'s modification time is at least **1 hour** old, preventing a newly initializing VM from being mistaken for an orphan before its backend instance is created. 7. **Creation lock on teardown**: `Leftovers.Forget` acquires the per-VM creation lock (`holdCreate`), re-reads `vm.json`, and calls `b.Inspect` one final time before running `teardown` (which removes `vms//`, SSH config entries, host keys, inter-VM links, and any `--clone` git remote). ### Shared Nix cache eviction (`internal/proxy/nixcache.go`) VMs built with the `nix` layer use `http://devvm.proxy/nix/` as a binary substituter (priority 30, ahead of `cache.nixos.org` at priority 40). The host proxy verifies Ed25519 signatures on `.narinfo` files against the pinned `cache.nixos.org-1` public key and verifies SHA-256 `FileHash` and `FileSize` on `.nar` archives before storing them in `~/.local/share/devvm/cache/nix-cache/`. - **Runtime trimming**: Before writing a new file of `room` bytes (rounded up to 4 KiB disk blocks), `nixCache.reserve` acquires an exclusive `flock` on `cache/nix-cache/.lock` and checks `.size`. If `total + room > max`, `trim` sorts cached files by `mtime` (updated via `touch` on every cache hit) and deletes the least recently used files until usage drops to **90% of `max`** (`max/10*9 - room`). - **In-flight protection**: Active NAR downloads write to pre-truncated `.tmp-*` files. `trim` counts `.tmp-*` files toward total disk usage and never deletes them unless their `mtime` is over 1 hour old (`nixTmpAge`, indicating a dead proxy process). --- ## Upgrade `devvm` (`devvm upgrade`) `devvm upgrade` updates the `devvm` binary using the mechanism that originally installed it. Because all VMs, images, and shared tools live in `~/.local/share/devvm` and user configuration lives in `~/.config/devvm`, upgrading or reinstalling `devvm` leaves existing VMs and settings intact. ```sh devvm upgrade # Upgrade to the newest stable release (prompts first) devvm upgrade --check # Check if a newer release is available without installing devvm upgrade --json # Output current/latest version and install kind as JSON devvm upgrade --force # Upgrade without prompting (required in scripts) devvm upgrade --to 0.1.0 # Install a specific release (including older versions) devvm upgrade --to 0.2.0-rc.1 # Install a prerelease (install.sh installations only) ``` ### Installation detection (`upgrade.Detect`) `upgrade.Detect` resolves symlinks on `os.Executable()` and classifies the running binary without executing external commands: | `Install.Kind` | Detection rule | Upgrade mechanism | Prerelease channel | | --- | --- | --- | --- | | `install.sh` (`Script`) | Binary resolves to `/lib/devvm//devvm` and `/bin/devvm` is a symlink pointing to it (`~/.local` by default, or `DEVVM_PREFIX`). | Downloads release from GitHub, verifies Minisign signature and SHA-256 checksum, stages, smoke-tests, and atomically swaps symlink. | Stable `/releases/latest` only; use `--to ` for prereleases. | | `homebrew` (`Homebrew`) | Resolved binary path contains `/Cellar/devvm/` or `/Cellar/devvm-rc/`. | Runs `brew upgrade devvm` or `brew upgrade devvm-rc`. Rejects `--to`. | `devvm-rc` reads `releases.atom` to follow prereleases and releases. | | `other` (`Other`) | Any other path, `go install` binary (`FromSource`), or non-release version string (`(devel)`). | Refuses automatic upgrade and prints instructions for how the binary was built or installed. | N/A | ### Atomic script upgrade and rollback flow When `Install.Kind` is `install.sh`, `Upgrader.script` performs a multi-stage verification and atomic swap with automatic rollback on failure: ```mermaid sequenceDiagram autonumber participant CLI as upgrade.Upgrader participant GH as GitHub Releases (HTTPS) participant FS as /lib/devvm & bin/devvm participant Proc as Process Table (ps -A) CLI->>GH: GET SHA256SUMS (64 KiB cap) & SHA256SUMS.minisig (4 KiB cap) CLI->>CLI: Verify Minisign ED (BLAKE2b-512) signature & trusted comment "devvm " CLI->>GH: GET devvm---.tar.gz (256 MiB cap) & verify SHA-256 CLI->>FS: Unpack flat regular files into ..new. (512 MiB cap) CLI->>FS: Exec ..new./devvm --version (30s timeout) alt If /lib/devvm/ already exists CLI->>FS: Rename -> ..old. end CLI->>FS: Rename ..new. -> CLI->>FS: Symlink .devvm.new. -> /devvm & rename over /bin/devvm CLI->>FS: Exec /bin/devvm --version (30s timeout) alt Final link execution fails CLI->>FS: Restore /bin/devvm -> previous Exe & restore ..old. else Final link execution succeeds CLI->>FS: Remove ..old. CLI->>Proc: Inspect running commands via ps -A -ww -o command= CLI->>FS: Prune unused older dirs & dead ..new/.old. dirs end ``` Key security and reliability invariants in `internal/upgrade/`: - **Minisign `ED` verification (`minisign.go`)**: `PublicKey.Verify` requires a prehashed `ED` signature over the BLAKE2b-512 digest of `SHA256SUMS` using the public key embedded from `packaging/minisign.pub` (`packaging.MinisignKey()`), rejects legacy non-prehashed `Ed` signatures, verifies the global signature over the trusted comment, and requires the trusted comment to equal `"devvm " + version` so a signature from another release cannot be replayed. - **Strict archive unpacking (`unpack.go`)**: `unpack` enforces a 512 MiB uncompressed cap (`unpackCap`) and accepts only regular files directly inside the single top-level directory `devvm---/`. Any symlink, subdirectory, or path traversal entry aborts unpacking immediately. - **Downgrade refusal**: If the latest release reported by the server is older than `u.Current` (`Compare(target, u.Current) < 0`) and `--to` was not explicitly passed, `Upgrade` refuses to install it—even with `--force`—unless `u.Current` is a prerelease of an upcoming version that is not yet out. - **In-use binary preservation (`prune`)**: When cleaning old version directories under `/lib/devvm/`, `Upgrader.prune` inspects `ps -A -ww -o command=`. Any version directory referenced by a running process (such as a running VM's `devvm _proxy`, including paths with doubled commas `,,` passed to QEMU `guestfwd`) is kept until a subsequent upgrade runs after that VM stops. ### Background daily release check (`releaseNotice`) When you launch a VM interactively in a terminal (`a.u.CanPick()`), `releaseNotice` checks `~/.local/share/devvm/cache/release.json`: 1. If the cached record (`upgrade.Known`) names a release newer than `Current`, `devvm` prints a one-line upgrade notice before opening the VM. 2. If `Known.Due(now)` is true (no check attempted in the last 24 hours), `devvm` spawns a background goroutine calling `Source.Refresh` with a 5-second timeout (`noticeTimeout`), and waits at most 1 second (`releaseWait`) after the command finishes for the check to complete. 3. Disable the check by setting `release-check = false` in `~/.config/devvm/config.toml` or `DEVVM_NO_RELEASE_CHECK=1` in the environment. ### Uninstall `devvm` completely List all VMs with `devvm ls` and run `devvm rm` in each project folder (or `devvm rm `). For `--clone` VMs, this also cleans up the `devvm` git remote in the host repository. Run `devvm gc` to remove stale images, then `devvm images rm ` for each remaining image in `devvm images`. Run `devvm gc --caches` to remove `devvm`'s Ubuntu cloud image downloads from Lima's cache (or `limactl prune` if you do not use Lima outside `devvm`). Run `devvm secret rm ` for each secret listed by `devvm secret` to clear macOS Keychain items under the `devvm` service or Linux secret files under `~/.local/share/devvm/secrets/`. ```sh brew uninstall devvm # Or if using the prerelease formula: brew uninstall devvm-rc rm -rf ~/.local/share/devvm ~/.config/devvm ``` ```sh rm -rf ~/.local/lib/devvm ~/.local/bin/devvm rm -rf ~/.local/share/devvm ~/.config/devvm ``` Finally, if you allowed `devvm` to configure SSH, remove the `Include ~/.local/share/devvm/ssh_config` line from `~/.ssh/config`. --- ## Troubleshooting retention, cleanup, and upgrades If another `devvm` process is creating a VM or running `devvm tools add` / `devvm tools update`, it holds `~/.local/share/devvm/making.lock`. Because the in-progress VM may have selected shared tools that are not yet written to its `vm.json`, `LockGC` sets `Making = true` and preserves all current unattached tool images. Wait for the VM creation or tool update to finish, then re-run `devvm gc --tool-images`. `Leftovers.FindIn` leaves a VM's state directory untouched and prints a note when: - **Its state changed in the last hour**: `vm.json` must be at least 1 hour old (`leftoverRecent`) before `devvm` considers it an orphan. - **The VM was created under a different Lima home**: `vm.json` records `BackendAt`. Ensure `DEVVM_LIMA_HOME` matches the location where the VM was created. - **The backend is unreachable**: Run `devvm doctor` to diagnose Lima issues. If a retention key is defined under `[retention]` in `~/.config/devvm/config.toml`, `devvm retention KEY=VALUE` refuses to write to `settings.json` because `config.toml` takes precedence. Edit or remove the key in `~/.config/devvm/config.toml` directly. - **Homebrew installations and `--to`**: Homebrew formulas only install the version pinned in the tap formula. To install a specific historical version or prerelease via `--to`, use an `install.sh` installation, or switch formulas between `devvm` and `devvm-rc`. - **Source or `go install` builds**: Binaries outside `/lib/devvm//devvm` and Homebrew's Cellar (`Install.Kind == Other`), or binaries built without the embedded `packaging/minisign.pub` key, cannot be upgraded by `devvm upgrade`. Re-run `install.sh` or `go install github.com/devvmhq/devvm/cmd/devvm@latest`. ## Related pages Complete flag, argument, and exit code reference for `devvm space`, `devvm gc`, `devvm retention`, `devvm images`, and `devvm upgrade`. Full specification of `~/.config/devvm/config.toml` tables, `settings.json` precedence, state directory layout, and version 1 JSON payloads. How base images, custom layers, and read-only EROFS shared tool images are keyed, cached, and mounted into VMs. Diagnose host disk pressure, inspect proxy network logs, and recover from interrupted image builds or VM remakes. --- ## 22. Codebase invariants, testscript stubs, and E2E suite > Enforce single-owner and layer rules in internal/structure, write deterministic CLI testscript tests with internal/stubtest, and run real-VM E2E suites. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/22-codebase-invariants-testscript-stubs-and-e2e-suite.md - Generated: 2026-11-07T22:58:17.381Z ### Source Files - `internal/structure/structure_test.go` - `internal/structure/owners_test.go` - `internal/structure/rules_test.go` - `internal/cli/script_test.go` - `internal/stubtest/stubtest.go` - `tests/e2e/sandbox_test.go` - `mise.toml` --- title: "Codebase invariants, testscript stubs, and E2E suite" description: "Enforce single-owner and layer rules in internal/structure, write deterministic CLI testscript tests with internal/stubtest, and run real-VM E2E suites." --- `devvm` enforces architectural boundaries, single-owner concerns, CLI conventions, and runtime correctness across three verification tiers: static AST and linter gates (`internal/structure` and `.golangci.yml`), deterministic CLI integration scripts backed by a multi-call host binary stub (`internal/cli/script_test.go` and `internal/stubtest`), and a multi-slot real-VM end-to-end harness (`tests/e2e`). ## Verification gate and `mise` tasks Every change must pass `mise run fmt && mise run lint && mise run test` before and after implementation. Changes that affect VM lifecycle, image builds, networking, mounts, or guest integration also run `mise run e2e` against real Lima VMs. | Task | Command / Script | Behavior and Verification Scope | | --- | --- | --- | | `mise run fmt` | `golangci-lint fmt` | Formats Go packages with `gofumpt` and `goimports` (local prefix `github.com/devvmhq/devvm`). | | `mise run lint` | Depends on `lint:code`, `lint:docs` | Runs all code, workflow, shell, and documentation checks. | | `mise run lint:code` | `golangci-lint`, `actionlint`, `zizmor`, `shellcheck` | Runs Go linters, verifies every `forbidigo` rule fires on `testdata/lint/*.go`, checks GitHub workflows offline with `actionlint` and `zizmor --offline .`, and lints all repo shell scripts (excluding `testdata/script` and `.agents/skills/security-audit`) with `shellcheck --external-sources`. | | `mise run lint:docs` | `markdownlint-cli2`, `vale`, `lychee`, `go test`, `hugo` | Lints Markdown and prose voice, checks offline links and fragments, runs `TestDocLinks` (`./internal/msg`) and `TestCommandLines` (`./internal/cli`), and builds the Hugo site with `--panicOnWarning`. | | `mise run test` | `scripts/test.sh` | Runs unit tests, `internal/structure`, CLI `testscript` suites, and `TestHarness*` E2E runner unit tests with `-race -cover -timeout 30m` in a throwaway `HOME`, then merges unit and CLI coverage against `testdata/coverage/.txt`. | | `mise run coverage` | `scripts/test.sh --raise` | Runs the full test suite and updates `testdata/coverage/.txt` for any package whose coverage increased, adding new packages and removing deleted ones. | | `mise run vuln` | `govulncheck ./...` | Scans dependencies for known vulnerabilities. | | `mise run deadcode` | `go run golang.org/x/tools/cmd/deadcode@... ./...` | Lists unreachable functions outside test entry points (`backendtest`, `stubtest`, `upgradetest`, `msg.IDOf`, `msg.NamesOf`, `msg.Recorder`, `msg.Fake`). | | `mise run cli-docs` | `go run ./internal/cli/docgen docs/content/docs/cli` | Regenerates the CLI reference pages checked by `TestReferenceIsCurrent`. | | `mise run e2e` | `go test -tags e2e -count=1 -timeout 0 -run '^TestE2E$' -args` | Runs real-VM end-to-end sections in `tests/e2e`. | ### Merged unit and CLI coverage baselines `scripts/test.sh` sets `CGO_ENABLED=1` (required by `-race`), creates an isolated temporary `HOME` and XDG directory tree, unsets `SSH_AUTH_SOCK`, `NO_COLOR`, `FORCE_COLOR`, and `CLICOLOR_FORCE`, and exports `CLI_GOCOVERDIR=$cov/cli`. When `CLI_GOCOVERDIR` is set, `TestScript` in `internal/cli/script_test.go` compiles the `devvm` binary with `-cover -covermode=atomic -coverpkg=github.com/devvmhq/devvm/...` and assigns each parallel `.txtar` script its own subdirectory (`$CLI_GOCOVERDIR/script.*`) as `GOCOVERDIR` so concurrent `devvm` invocations never race on Go's coverage metadata emission. After `gotestsum` finishes, `scripts/test.sh` merges `$cov/unit` and all non-empty `$cov/cli/*` directories with `go tool covdata merge` and passes `go tool covdata percent` output to `scripts/coverage.sh`: - **Baseline check (`check` mode):** Compares each package's statement coverage against `testdata/coverage/darwin.txt` or `testdata/coverage/linux.txt`. Fails if any package drops more than `COVERAGE_TOLERANCE` (default `1` percentage point) below its baseline, if a new package has no baseline entry, or if a deleted package remains in the baseline. - **Baseline update (`raise` mode):** Raises baseline percentages for packages that beat their recorded line, adds new packages, drops removed packages, and never lowers an existing baseline entry within the 1-point tolerance. --- ## Codebase invariants in `internal/structure` and `.golangci.yml` `internal/structure` parses every non-generated `.go` file in the repository across all OS build tags (skipping hidden directories, `testdata`, `docs`, and `node_modules`) and enforces structural rules that standard linters cannot express. `TestStructure` checks the live repository against `devvm()`, while `TestRulesFail` in `internal/structure/rules_test.go` runs synthetic `fstest.MapFS` test cases against every rule and verifies that every layer in `devvm().layers` has a test case that breaks it. ```mermaid flowchart TD subgraph Entry["Entry & CLI Layer"] CMD["cmd/devvm"] CLI["internal/cli/..."] UI["internal/ui"] end subgraph Core["VM & Domain Services"] VM["internal/vm"] DOMAIN["internal/build, tools, netpolicy, secrets,\nproject, gitprotect, hostsafety,\nsshconfig, sshagent, trust, vmssh"] AGENTLOGIN["internal/agentlogin"] DEVCONTAINER["internal/devcontainer"] end subgraph BackendLayer["Backend Abstraction & Drivers"] REG["internal/backend/registry"] BACKEND["internal/backend"] LIMA["internal/backend/lima"] end subgraph StateAndPaths["State, Images & Messages"] STATE["internal/state"] IMAGE["internal/image"] PATHS["internal/paths"] MSG["internal/msg"] end subgraph Standalone["Stdlib-Only & Leaf Packages"] PROXY["internal/proxy/..."] DAEMON["internal/daemon"] LEAVES["assets, locales, packaging, internal/platform,\ninternal/testbuild, internal/hostgit, internal/shell,\ninternal/vmname, internal/fetch, internal/jsonc, internal/redact"] end CMD --> CLI CMD --> UI CLI --> UI CLI --> VM CLI --> REG CLI --> LIMA VM --> DOMAIN VM --> REG VM --> BACKEND REG --> LIMA REG --> BACKEND DOMAIN --> BACKEND DOMAIN --> STATE AGENTLOGIN --> STATE AGENTLOGIN --> PROXY DEVCONTAINER --> IMAGE PROXY --> DAEMON STATE --> PATHS IMAGE --> PATHS PATHS --> MSG MSG --> LEAVES ``` ### Structural rule categories `check(m, devvm())` in `internal/structure/check_test.go` evaluates five rule categories and formats violations as `: :: `: | Rule ID | Invariant Enforced | Configuration in `devvm()` | | --- | --- | --- | | `max-methods` | Caps any struct or interface at `24` methods (value and pointer receivers combined; embedded interfaces count toward their own type, not the embedding interface). | `maxMethods: 24`, with explicit caps in `largeTypes` for `internal/paths.Dirs` (`58`) and `internal/ui.UI` (`46`). If a `largeTypes` entry shrinks below its cap or drops to `<= 24`, the check fails until `largeTypes` is lowered or the entry is removed. | | `no-init` | Forbids `func init()` across all production and test files. Initialization happens in constructors or where used. | Checked unconditionally by `checkInit(m)`. | | `package-vars` | Bans mutable package-level `var` declarations in production and test files. | Permits `var _`, `//go:embed`, `Err*`/`err*` sentinels from `errors.New`/`fmt.Errorf`, and `regexp.MustCompile`, `template.Must`, or `sync.Once*` calls. All other read-only tables, fixtures, and `-ldflags` targets must be listed in `allowedVars` with a reason (`ldflags`, `table`, `fixture`, `derived`). Stale entries fail the test. | | `layers` | Enforces directional package import rules across non-test files (test files may wire packages together). | 18 `layer` rules with `from`, `exceptFrom`, `to`, and `exceptTo` package glob patterns (`...` and `pkg/...`). | | `owners` | Ensures each critical operation, command, environment variable, path builder, string literal, separator, and state reader has a single owner in non-test files. | `commands`, `execSites`, `guardOwner`, `envOwner`, `envSettings`, `envOwners`, `literals`, `separators`, `pathsOwner`, and `callers`. | ### Package layer rules The 18 layer rules in `internal/structure/structure_test.go` enforce strict upward and downward boundaries: 1. **UI ownership:** Only `cmd/devvm` and `internal/cli/...` may import `internal/ui`. All other packages emit messages, ask questions, and run logged commands through the `msg.Reporter` interface. 2. **VM orchestration:** Only `internal/cli/...` may import `internal/vm`, and only `cmd/devvm` may import `internal/cli/...`. 3. **Backend isolation:** `internal/backend/...` cannot import `internal/vm`, `internal/build`, `internal/tools`, `internal/netpolicy`, `internal/secrets`, `internal/project`, `internal/gitprotect`, `internal/hostsafety`, `internal/sshconfig`, `internal/sshagent`, `internal/trust`, or `internal/vmssh`. Only `internal/cli/...` and `internal/backend/registry` may import `internal/backend/lima`. 4. **Test-only packages:** `internal/backend/backendtest`, `internal/upgrade/upgradetest`, and `internal/stubtest/...` cannot be imported by non-test files outside their own trees. 5. **Stub isolation:** `internal/stubtest/...` may import only `internal/msg` and its own subpackages. `internal/stubtest/stub` and `internal/stubtest/stubstate` may import nothing from `devvm` except `internal/stubtest/stubstate`, because the `stub` binary is compiled as a standalone executable replacing host tools. 6. **Proxy and leaf boundaries:** - `internal/proxy/...` imports only the Go standard library and `internal/daemon` (enforced by both `structure` and `depguard`). - Leaf packages (`assets`, `locales`, `packaging`, `internal/platform`, `internal/msg/gen`, `internal/testbuild`, `internal/hostgit`, `internal/shell`, `internal/vmname`, `internal/daemon`, `internal/fetch`, `internal/jsonc`, `internal/redact`) import zero `devvm` packages. - `internal/agentlogin` imports only `internal/fetch`, `internal/jsonc`, `internal/proxy`, and `internal/state`. - `internal/msg` imports only `locales`. - `internal/paths` imports only `internal/msg` and `internal/vmname`. - `internal/state` imports only `internal/msg`, `internal/paths`, `internal/platform`, and `internal/vmname`. - `internal/image` imports only `assets`, `internal/msg`, and `internal/paths`. - `internal/devcontainer` imports only `internal/image`, `internal/jsonc`, `internal/msg`, and `internal/shell`. ### Single-owner checks (`owners` and `forbidigo`) When a second copy of a parser, path builder, or system call would introduce drift or a security bug, `internal/structure/owners_test.go` and `forbidigo` in `.golangci.yml` enforce a single owner: | Concern | Enforced By | Single Owner | Rule Details | | --- | --- | --- | --- | | Terminal output & process exit | `forbidigo` | `internal/ui` (`fmt.Print*`, `os.Stdout`, `os.Stderr`), `cmd/devvm` (`os.Exit`) | All packages return errors; only `internal/ui` writes to terminal streams and only `cmd/devvm` calls `os.Exit`. | | Root context creation | `forbidigo` | `cmd/devvm`, `internal/ui` | Bans `context.Background` and `context.TODO` elsewhere unless annotated with `//nolint:forbidigo // ` for work that must outlive the caller. | | Host hardware & free space | `forbidigo` | `internal/platform` (`platform.Host`) | Bans `runtime.NumCPU` and `unix`/`syscall` `Statfs`, `Fstatfs`, `Sysinfo`, `SysctlUint64` elsewhere so test builds can override host sizing. | | Test build detection | `forbidigo` | `internal/cli` (`readOverrides` in `internal/cli/overrides.go`) | Bans `testbuild.On` outside `internal/cli`; other packages receive overrides as struct fields. | | Process signaling by PID | `forbidigo` | `internal/daemon`, `internal/platform` | Bans `os.FindProcess`, `syscall.Kill`, `unix.Kill`, and `os.Process.Kill`/`Signal` elsewhere; `internal/daemon` verifies PID files and process arguments before signaling. | | HTTP clients & transports | `forbidigo` | `internal/fetch` (`fetch.NewClient`) | Bans `http.Client`, `http.DefaultClient`, `http.Transport`, `http.Get`, etc. outside `internal/fetch` and stdlib-only `internal/proxy`. | | Atomic file writes | `forbidigo` | `internal/state` (`state.WriteFile`, `state.WriteFileMode`) | Bans `os.WriteFile`, `os.Create`, and `os.O_TRUNC` outside `internal/state`, `internal/proxy` (`writeAtomic`), `internal/msg/gen`, and `internal/stubtest`. | | Process environment mutation | `forbidigo` | `internal/daemon/status.go` (`daemon.Ready`) | Bans `os.Setenv`, `os.Unsetenv`, `os.Clearenv` (and `syscall` equivalents) outside tests so child processes never inherit mutated global env vars. | | Host `git` execution | `structure` (`commands`) | `internal/hostgit` | Any `exec.Command` or `exec.CommandContext` invoking `"git"` (literal or constant) outside `internal/hostgit` fails. | | `os/exec` construction sites | `structure` (`execSites`) | 19 allowlisted files in `devvmExecSites()` | Outside tests, only files listed in `execSites` may call `exec.Command` or `exec.CommandContext`. Removing the call without removing the file from `execSites` also fails. | | Backend type assertions | `structure` (`guardOwner`) | `internal/backend/guard.go` (`backend.As`) | Bans type assertions and type switches on any type in `internal/backend` or `internal/backend/...` outside `guard.go` because registry backends are wrapped in a guard struct. | | `HOME` & `DEVVM_*` env reads | `structure` (`envOwner`, `envOwners`, `envSettings`) & `forbidigo` | `internal/paths/dirs.go` (`HOME`), `internal/cli/overrides.go` (`DEVVM_*`) | Matches any call or field named `Getenv` or `LookupEnv` (case-insensitive). `HOME` is read only in `internal/paths/dirs.go`; `DEVVM_*` variables are read only in `internal/cli/overrides.go` or the exact files listed in `devvmEnvSettings()`. | | Shell quoting & VM name regex | `structure` (`literals`) | `internal/shell` (`shell.Quote`), `internal/vmname` (`vmname.IsVM`) | Bans string literals containing `'\''` outside `internal/shell` and `[0-9a-f]{14}` outside `internal/vmname`. | | Image layer separator | `structure` (`separators`) | `internal/image` (`image.Name`) | Bans `strings.*` calls taking `"+"` (literal or constant) outside `internal/image`. | | Data & config folder paths | `structure` (`pathsOwner`) | `internal/paths` | Bans `filepath.Join`, `os.MkdirTemp`, or `os.CreateTemp` on a `.Data` or `.Config` selector outside `internal/paths`. | | State, finder & agent store callers | `structure` (`callers`) | `internal/vm/finder.go`, `internal/vm/inventory.go`, `internal/vm/agents.go`, `internal/secrets/secrets.go` | Restricts external calls to `state.ReadHash` and `project.Parents` (`finder.go`), `state.ReadAll` (`inventory.go`), `paths.Dirs.AgentSettings` (`agents.go`), and `paths.Dirs.AgentLogins` (`secrets.go`). | Every `forbidigo` rule in `.golangci.yml` has a corresponding violation line ending with `// want forbidigo` in `testdata/lint/forbidigo.go`. During `mise run lint:code`, `golangci-lint run --enable-only forbidigo` runs against `./testdata/lint/` and fails unless the reported lines match the `// want forbidigo` annotations 1-to-1. ### CLI rule and agreement tests In addition to `internal/structure`, unit tests across `internal/cli`, `internal/image`, and `internal/msg` verify cross-cutting invariants: - **Command tree and flag conventions (`internal/cli`):** `TestEveryCommandsHelp`, `TestPersistentFlags`, `TestFlagNames`, `TestFlagNamesHaveOneMeaning`, `TestDurationFlags`, `TestCommandNames`, `TestUseMatchesArgs`, `TestUnknownWord`, and `TestUsageErrorsEveryCommand` walk the Cobra command tree to enforce naming, help text, flag types, and exit code `2` on usage errors. - **JSON output schemas (`internal/cli/json_test.go`):** `TestShowCommandsTakeJSON` verifies that every display command accepts `--json` (unless listed in `withoutJSON`), and `TestJSONShapes` locks every `"version": 1` field path and type so fields are never renamed or removed without bumping `jsonVersion`. - **Command-line occurrences (`TestCommandLines`):** Scans all Markdown docs, `README.md`, `AGENTS.md`, `locales/en.toml`, Go comments/strings, `assets/`, `install.sh`, `.txtar` scripts, and skills to verify that every `devvm` command, subcommand, flag, and placeholder is valid. - **Interactive prompts (`TestQuestionsDeclareAnswers`):** Scans non-test ASTs to ensure every `msg.Question` passed to `Ask`, `Confirm`, `Answer`, or `Secret` declares a valid flag/command via `AnsweredBy(...)` or a refusal message via `RefusedWith(...)`. - **Shared tools and messages (`internal/image`, `internal/msg`):** `TestEveryPartHasAPolicy`, `TestSharedToolsPolicies`, `TestConfigTools`, `TestBuiltinPolicies`, `TestPreludeUsers`, `TestEveryMessageIsUsed`, `TestDocLinks`, and `TestReferenceIsCurrent` keep `image.SharedTools`, `assets/shared-tools/mise.toml`, `locales/en.toml`, `internal/msg/docs.go`, and `docs/content/docs/cli/` synchronized. --- ## Deterministic CLI tests (`TestScript` and `internal/stubtest`) CLI integration tests live in `testdata/script/*.txtar` and are executed by `TestScript` in `internal/cli/script_test.go` using `github.com/rogpeppe/go-internal/testscript`. Run a single testscript file by name: ```sh mise exec -- go test ./internal/cli -run TestScript/ ``` ### Test builds and environment overrides `TestScript` (and `tests/e2e`) compiles `./cmd/devvm` with: ```sh -ldflags "-X github.com/devvmhq/devvm/internal/testbuild.testBuild=yes" ``` Only binaries linked with `testBuild=yes` read the test override environment variables in `readOverrides` (`internal/cli/overrides.go`). Production release builds ignore all test override variables. | Override Variable | Field in `cli.Overrides` | What It Replaces in Test Builds | | --- | --- | --- | | `DEVVM_HOST_MISE` | `HostMise` | Path to `devvm`'s pinned host `mise` binary (pointed at the `mise` stub in CLI tests). | | `DEVVM_KVM` | `Overrides.KVM` | Path to Linux's `/dev/kvm` character device (pointed at `$STUB_DIR/kvm` in CLI tests). | | `DEVVM_UBUNTU_URL` | `Overrides.UbuntuURL` | Base URL for Ubuntu cloud image releases (`file://$ROOT/ubuntu` in CLI tests, copied from `internal/ubuntu/testdata/resolute`). | | `DEVVM_UBUNTU_RELEASE` | `Overrides.UbuntuRelease` | Pins a specific `release-YYYYMMDD` folder instead of selecting the newest release (`e2eUbuntuRelease` in E2E). | | `DEVVM_UNREVIEWED` | `Overrides.Unreviewed` | Comma-separated `backend:feature` pairs allowed to run before emulation review completes. | | `DEVVM_TEST_CPUS` | `Overrides.Host.CPUs` | Host CPU count read by `platform.Host` (default `8` in `TestScript`). | | `DEVVM_TEST_MEMORY_GIB` | `Overrides.Host.MemoryGiB` | Host RAM in GiB read by `platform.Host` (default `16` in `TestScript`). | | `DEVVM_TEST_FREE_GIB` | `Overrides.Host.FreeGiB` | Free disk space in GiB read by `platform.Host` (default `500` in `TestScript`). | | `DEVVM_TEST_LIST_TIMEOUT` | `Overrides.Host.ListTimeout` | Timeout for backend instance listings (`10m` in `TestScript`). | | `DEVVM_PIPED_ANSWERS` | `PipedAnswers` | When `1`, reads prompt answers line-by-line from non-TTY `stdin` instead of refusing with `no_input`. | | `DEVVM_PICK_ON_PIPE` | `PickOnPipe` | When `1`, runs the interactive picker when only `stdin` is a TTY (used with `testscript`'s `ttyin`). | | `DEVVM_HISTORY_MAX` | `HistoryMax` | Maximum byte size for agent history exports (replaces the 2 GiB cap). | | `DEVVM_REPO_URL`, `DEVVM_RELEASE_KEY`, `DEVVM_RELEASE_CA`, `DEVVM_TEST_VERSION` | `Release.*` | Points `devvm upgrade` at the local TLS `upgradetest` server, custom Minisign public key, CA PEM file, and current binary version. | ### `TestScript` sandbox environment and custom commands For each `.txtar` script, `sandbox()` creates `/tmp/dvt.*` (keeping Unix domain socket paths within `paths.SocketMax`), prepends the built `devvm` directory and the `stubtest` binary directory to `PATH`, pre-populates `$DATA/settings.json` with `{"features": []}`, and sets `DEVVM_MESSAGES=names` so assertions match message IDs and key-value parameters (`stderr '^devvm: flag_unknown command=devvm flag=--nope$'`) rather than prose wording.When a script ends, a deferred cleanup kills any background `_proxy`, `_ports`, `_links` daemons and `ssh -M` control masters tied to that sandbox root before removing `/tmp/dvt.*`. | Environment Variable in `.txtar` | Sandbox Path / Value | | --- | --- | | `HOME` | `/tmp/dvt./home` | | `XDG_DATA_HOME`, `DATA`, `VMS`, `RUN` | `/tmp/dvt./data`, `.../data/devvm`, `.../data/devvm/vms`, `.../data/devvm/run` | | `XDG_CONFIG_HOME`, `CFG` | `/tmp/dvt./config`, `.../config/devvm` | | `DEVVM_LIMA_HOME`, `LIMA_HOME` | `/tmp/dvt./lima` | | `STUB_DIR`, `STUB_LOG` | `/tmp/dvt./stub`, `/tmp/dvt./stub/calls` | | `DEVVM_SECRET_SERVICE` | `devvm-test` | | `DEVVM_NO_RELEASE_CHECK` | `1` | | `MISE_STUB_LOCK` | `/testdata/mise.lock` | | `DEVVM_BIN`, `DEVVM_REPO` | Built `devvm` directory and repository checkout root | `scriptCmds()` in `internal/cli/script_test.go` registers custom commands for `.txtar` scripts: | Command | Syntax | Behavior | | --- | --- | --- | | `project` | `project NAME [git]` | Creates `$WORK/NAME`, `chdir`s into it, and runs `git init -q` when `git` is passed. | | `vm_name` | `[!] vm_name VAR NAME` | Finds the single VM in `$VMS/*/vm.json` matching `^devvm-NAME-[0-9a-f]+$`, sets `$VAR` to the VM name, and sets `${VAR}_dir` to `$VMS/`. Negated (`!`), passes when the count is not `1`. | | `vm_state` | `vm_state GLOB` | Reads the single `vm.json` matched by `GLOB` and prints sorted, flattened `KEY=VALUE` lines (`net.on=true`, `tools.names=node claude`, `links.links.0.name=db`). | | `json_set` | `json_set GLOB KEY=VALUE...` | Modifies or creates JSON files matching `GLOB`. `KEY=VALUE` sets a string; `KEY:=JSON` sets a parsed JSON value; dotted keys (`net.on:=false`) traverse or create nested objects. | | `catg` | `[!] catg GLOB` | Writes the single file matching `GLOB` to `stdout`; fails if `0` or `>1` files match (negated: fails if `1` file matches). | | `stub_calls` | `[!] stub_calls` | Prints `$STUB_LOG` to `stdout`. Negated (`! stub_calls`), fails if any stub calls were logged. | | `stub_reset` | `stub_reset` | Truncates `$STUB_LOG` and clears `testscript`'s `stdout` and `stderr` buffers. | | `wait_file` | `wait_file FILE` | Polls every `50ms` up to `60s` (`waitFileTimeout`) for `FILE` to exist; fails immediately if all background commands have already exited. | | `cp_exe` | `cp_exe SRC DST` | Copies an executable with mode `0755` while holding `syscall.ForkLock.RLock()`, preventing `ETXTBSY` (`text file busy`) on Linux when parallel scripts fork subprocesses. | | `free_port` | `free_port VAR... \| free_port -range FIRST LAST` | Binds `:0` across all interfaces, closes the listener, and assigns unused TCP port numbers (or two consecutive ports with `-range`) to the named variables. | | `listen_any` | `listen_any VAR` | Listens on `:0` for the lifetime of the script and sets `$VAR` to the bound port. | | `release_server` | `release_server [-badsig] LATEST [VERSION...]` | Starts an in-process TLS release server (`upgradetest.Start`), signs `SHA256SUMS` with a test Minisign key (or a mismatched key with `-badsig`), and exports `DEVVM_REPO_URL`, `DEVVM_RELEASE_KEY`, and `DEVVM_RELEASE_CA`. | ### Multi-call stub binary (`internal/stubtest`) `internal/stubtest/build.go` compiles `github.com/devvmhq/devvm/internal/stubtest/stub` once into `$TMPDIR/devvm-stubs-`, where `` is the first 16 hex characters of the SHA-256 over `runtime.Version()`, `GOOS/GOARCH`, and the non-test `.go` files in `internal/stubtest/stub` and `internal/stubtest/stubstate`. In the same folder, it creates symlinks to `stub` for every command returned by `stub names`, and prunes any `devvm-stubs-*` directories older than 24 hours. ```mermaid sequenceDiagram participant TS as TestScript / Go Test participant Devvm as devvm (testBuild=yes) participant Stub as stub binary (argv[0] dispatch) participant State as $STUB_DIR/state.json (Flock) TS->>Stub: stub vm devvm-web-0123456789abcd Running Stub->>State: stubstate.Update(STUB_DIR) under state.lock TS->>TS: touch $STUB_DIR/start-fails (optional switch) TS->>Devvm: exec devvm ls Devvm->>Stub: PATH resolves limactl -> stub Stub->>Stub: Append "limactl list --json" to $STUB_LOG (O_APPEND) Stub->>State: stubstate.Read(STUB_DIR) under state.lock State-->>Stub: Instances, Keychain, Agent, Excluded Stub-->>Devvm: JSON instance list Devvm-->>TS: stdout / stderr (DEVVM_MESSAGES=names) TS->>TS: stub_calls -> verify $STUB_LOG ``` #### Stubbed commands and `stub` control subcommands The `stub` binary dispatches on `filepath.Base(os.Args[0])` across 18 host commands plus the `stub` control interface: - **Virtualization & containers:** `limactl`, `qemu-img`, `qemu-system-aarch64`, `qemu-system-x86_64`, `docker` - **SSH & credentials:** `ssh`, `ssh-add`, `ssh-agent`, `ssh-keygen`, `security` - **Toolchains, agents & editors:** `mise`, `opencode`, `code`, `open` - **Host OS utilities:** `chflags`, `tmutil`, `sw_vers` (`STUB_SW_VERS`), `uname` (`STUB_UNAME_S`, `STUB_UNAME_M`) Every stub invocation appends a single line (` `) to `$STUB_LOG` in one `O_APPEND` write so concurrent calls never interleave and sensitive arguments (such as proxy URLs with passwords passed to `limactl edit`) are never forwarded to another child process's argv. Persistent state across stub calls is stored in `$STUB_DIR/state.json`, protected by `syscall.Flock` on `$STUB_DIR/state.lock` (`internal/stubtest/stubstate`). Scripts inspect or mutate this state with `exec stub ...`: ```txtar # Pre-populate a running Lima instance and inspect stub calls exec stub vm devvm-app-0123456789abcd Running exec stub size devvm-app-0123456789abcd 4 8589934592 107374182400 exec stub vms stdout '^devvm-app-0123456789abcd Running$' ``` | `stub` Subcommand | Arguments | Effect on `$STUB_DIR/state.json` | | --- | --- | --- | | `stub names` | None | Prints all registered stub command names, sorted one per line. | | `stub vm` | `NAME STATUS` | Creates or updates Lima instance `NAME` with `STATUS` (`Running`, `Stopped`, `Broken`) and default sizing (`4` CPUs, `8 GiB` RAM, `100 GiB` disk). | | `stub vms` | None | Prints all non-empty Lima instances as sorted `NAME STATUS` lines. | | `stub size` | `NAME CPUS MEMORY DISK` | Updates CPU count and memory/disk sizes (in bytes) for existing instance `NAME`. | | `stub keychain` | `ACCOUNT` | Prints the stored secret for `ACCOUNT`, or exits `1` if missing. | | `stub keychain-set` | `ACCOUNT` | Reads a secret value from `stdin` (trimming trailing newlines) and stores it under `ACCOUNT`. | | `stub agent` | None | Prints keys held by the stubbed `ssh-agent` in `ssh-add -l` format, or exits `1` if no agent is running. | | `stub agent-stop` | None | Clears the running `ssh-agent` state (`st.Agent = nil`). | #### Fault-injection switches and guest simulation Tests inject failures, hangs, or custom payloads by writing switch files into `$STUB_DIR`: - **Switch naming conventions:** `-fails`, `-hangs`, `-fails-on`, and `-waits-on`. When a stub hits a `-hangs` or `-waits-on` switch, `c.hang()` touches `$STUB_DIR/hanging` (so `wait_file $STUB_DIR/hanging` can synchronize before sending `kill -INT`) and sleeps `30s`. - **Guest shell simulation (`internal/stubtest/stub/guest.go`):** Both `limactl shell` and `ssh` route guest commands to `guest.go`. During VM setup and builds, the guest stub records tar streams into `$STUB_DIR/sync.tar` and `$STUB_DIR/packages-in.tar`, emits fake tool images or OCI disk artifacts, and supports targeted fault switches such as `$STUB_DIR/hostkeys-fails`, `$STUB_DIR/net-direct`, and `$STUB_DIR/interrupt-on`. - **Real Lima contract and golden verification:** `TestLimactlStubContract` (`internal/backend/lima/contract_test.go`) runs an identical multi-step scenario against both the `limactl` stub and real `limactl` dummy instances, asserting identical `limactl list --json` state after every step. Golden fixtures in `testdata/golden/darwin/` and `testdata/golden/linux/` (captured via `mise run e2e -- --golden`) verify both `devvm`'s parsers and the stubs' output shapes in `internal/backend/lima/golden_test.go` and `internal/platform`. ### Using `stubtest` and `msg.Fake` in Go unit tests Go unit tests use `stubtest.New(t, only...)` to place all stubs (or only the named command symlinks, leaving host tools like `git` intact) first on `PATH` with a per-test `STUB_DIR`: ```go // Example unit test setup with stubtest.New and msg.Fake s := stubtest.New(t, "limactl", "ssh") s.VM(t, "devvm-web-0123456789abcd", "Running") s.Touch(t, "start-fails") var rec msg.Recorder rep := &msg.Fake{OnWarn: rec.Add} ``` - `stubtest.ExecRunner{}` implements `backend.Runner` (`Run`, `Output`, `Attach`) by running `exec.Cmd` directly without a UI status line or run log. - `msg.IDOf(err)` and `msg.NamesOf(err)` extract the message ID (`"no_vm"`) or `names`-mode string (`"no_vm dir=/path"`) from a returned error. --- ## Real-VM E2E test suite (`tests/e2e`) `tests/e2e` (built only when `-tags e2e` is passed) runs `devvm` against real Lima VMs. `TestE2E` in `tests/e2e/main_test.go` requires `-timeout 0` so `go test`'s default 10-minute timeout never aborts a section mid-check. ```sh title="Run via mise" # Run default sections (all except desktop and from) mise run e2e # Run specific sections mise run e2e -- --only images,gc # Run all sections including --from and all base features across 3 parallel slots mise run e2e -- --from --slots 3 # Keep reusable VMs running between iterations on a single section mise run e2e -- --only project --reuse ``` ```sh title="Run via go test directly" # Run from tests/e2e without a package argument so go test streams output live cd tests/e2e && CGO_ENABLED=0 go test -tags e2e -count=1 -timeout 0 -run '^TestE2E$' -args --only gc ``` ### E2E CLI flags Flags are registered on `flag.CommandLine` in `tests/e2e/options_test.go` and passed after `--` (with `mise run e2e`) or `-args` (with `go test`): Comma-separated list of sections to run (executed in canonical `sections()` order). Fails if an unknown section name is given. Runs every section (including `desktop` and `from`) and enables `--every-feature` so base builds with all optional features in `assets/features/*.sh`. Deletes cached `devvm-img-*` Lima instances, `$DATA/devvm/images`, and `$DATA/devvm/tools` in the target sandbox before running. Writes `config.toml` with every feature in `assets/features/*.sh` enabled (uses the `features` timing tier). Automatically enabled when `--from` or `--only=from` is selected. Runs `images`, `project`, and `tools` in a dedicated `-newest` sandbox against Ubuntu's latest live cloud image and mirror packages instead of `e2eUbuntuRelease`, reporting upstream drift on failure. Cannot be combined with `--slots` or `--slot`. Captures anonymized golden tool outputs into `testdata/golden//` from `TestLimactlStubContract` and the `images` and `project` sections. Retains VMs created via `s.Reuse(...)` at the end of the run and reuses them on subsequent runs if still `Running`. Spreads the selected sections across `1` to `9` parallel slot sandboxes, seeding slots `1..N-1` from slot `0` before launching child `TestE2E` processes. Must be run from slot `0`. Runs in slot `K`'s isolated sandbox (`0` to `9`), using that slot's cached images and port offset (`+100 * K`). ### Sandbox isolation, locks, and multi-slot seeding Each slot `K` (`0` by default) runs in an isolated sandbox (`tests/e2e/sandbox_test.go`) and refuses to start if its resolved `LIMA_HOME` matches `~/.lima` or `~/.local/share/devvm/lima`. | Resource | Slot `0` | Slot `K` (`1..9`) | `--newest-base` | | --- | --- | --- | --- | | Lima home (`LIMA_HOME`, `DEVVM_LIMA_HOME`) | `~/.lt-e2e-go` (or `$DEVVM_E2E_LIMA_HOME`) | `~/.lt-e2e-go-K` | `~/.lt-e2e-go-newest` | | State & work dir (`XDG_DATA_HOME`, `XDG_CONFIG_HOME`) | `/tmp/devvm-go-e2e` | `/tmp/devvm-go-e2e-K` | `/tmp/devvm-go-e2e-newest` | | Keychain service (`DEVVM_SECRET_SERVICE`) | `devvm-go-e2e` | `devvm-go-e2e-K` | `devvm-go-e2e-newest` | | PID symlink lock | `/tmp/devvm-go-e2e.lock` | `/tmp/devvm-go-e2e-K.lock` | `/tmp/devvm-go-e2e-newest.lock` | | Fixed port offset (`sb.ports`) | `+0` | `+100 * K` | `+1000` | | Timings directory (`sb.tdir`) | `/tmp/devvm-go-e2e/timings/` | `/tmp/devvm-go-e2e/timings/` (shared) | `/tmp/devvm-go-e2e/timings/-newest` | Before building `devvm` into `$WORK/src-go/devvm`, the sandbox setup performs three checks: 1. **Host prerequisite check (`checkTools`):** Verifies `limactl`, `pkill`, and all tools declared by the selected sections (and ensures `openssl` on macOS is OpenSSL 3 rather than `/usr/bin/openssl` LibreSSL). Missing tools fail immediately with platform-specific `brew`, `apt-get`, or `dnf` install commands. 2. **Stale-safe PID lock (`takeLock`):** Creates an atomic symlink `/tmp/devvm-go-e2e[-K].lock -> `. If the symlink exists, `takeLock` checks `syscall.Kill(pid, 0)` and reclaims the lock if the previous process is no longer alive. 3. **Pinned tool lock verification (`pinLock`):** Verifies that the first line of `tests/e2e-mise.lock` matches `# devvm e2e: shared-tools/mise.toml sha256:` for the current `assets/shared-tools/mise.toml`. If `mise.toml` changed without running `scripts/bump.sh e2e-tools`, setup aborts immediately. When `--slots N` (`N > 1`) is passed, `plan()` in `tests/e2e/slots_test.go` schedules sections longest-first (using recorded durations from `/tmp/devvm-go-e2e/timings//` or the section's fallback `guess`), grouping `from` and `link` together because `link` in the `features` tier boots an Alpine client from the image built by `from`. Before spawning slot subprocesses, `seed()` clones slot `0`'s `devvm-img-*` Lima instances, EROFS tool images, caches, and cached `.deb` packages into slots `1..N-1` using copy-on-write APFS `clonefile` on macOS or reflinks on Linux. ### Check windows, heartbeats, and the check-count guard `runner` (`tests/e2e/runner_test.go`) wraps every setup phase and `s.Check(...)` in a timed `window`: - **Time limits (`limitOf`):** Each check gets `3x` its longest duration from the section's last complete run, with a minimum floor of `2m` (`limitFloor`), or `10m` (`limitDefault`) if no prior timing exists. Checks marked with `Builds()` and inter-check setup steps get at least `1h` (`limitBuild`). - **Process-group termination:** Every command executed via `s.Run` runs in its own process group (`Setpgid: true`). When a window's deadline expires, the runner sends `SIGTERM` to the process group and follows with `SIGKILL` after `30s` (`killAfter`), recording the last heartbeat line in the failure output. - **Heartbeat:** Every `1m`, a background goroutine prints `... since the last check: `, pulling the most recently modified line across `devvm`'s run logs and `$WORK/check.log`. - **Check options (`Flaky`, `Builds`, `Needs`):** - `Flaky(reason)` retries a check once on failure, marking the output and timing record with `retried once`. - `Needs(backend.Feature...)` (or section-wide `s.Needing(...)`) queries `devvm doctor --json` once per run and records `SKIP` if the active backend reports `none` or `unreviewed` for a required feature. - **Check-count guard (`sectionEnd`):** Each section writes its check timings to `timings//
` and a SHA-256 hash of `tests/e2e/*.go` to `timings//
.sum`. If a section completes with the same `tests/e2e/*.go` hash as the previous run but executes fewer checks, `sectionEnd` flags the run as `FAILED` (`
ran M checks, but its last complete run ran N: it ended early`) and preserves the previous timings file. ### E2E sections and `*S` check API `sections()` in `tests/e2e/sections_test.go` defines the 10 E2E sections in execution order: | Section | File | Default Run | Fallback Guess | Primary Verification Scope | | --- | --- | --- | --- | --- | | `images` | `tests/e2e/images_test.go` | Yes | `600s` | Base and custom layer builds, rebuilds, failed script cleanup, `SIGINT` during build, and offline replay from recorded `.deb` package sets. | | `project` | `tests/e2e/project_test.go` | Yes | `900s` | Project VM launch, mounts, `ssh` config and host-key pinning, `.git` locking, `TestE2EExecQuoting`, `devcontainer.json`, `agent import`/`save`, and `ssh-agent` modes. | | `clone` | `tests/e2e/clone_test.go` | Yes | `120s` | `devvm --clone` isolated repositories, branch syncing, and switching between mounted folders and clones. | | `gc` | `tests/e2e/gc_test.go` | Yes | `120s` | `devvm gc` removal of stale layer images and unused shared tool images while preserving `img-*` project VMs. | | `tools` | `tests/e2e/tools_test.go` | Yes | `600s` | Upstream tool resolution with host `mise`, read-only EROFS mounts for `node` and `dotnet`, global installs into `$HOME`, and `.nvmrc`/`global.json` overrides. | | `network` | `tests/e2e/network_test.go` | Yes | `600s` | `allowlist`, `open`, `internet`, and `none` tiers, host `_proxy` TLS interception, per-VM CAs, secret placeholder injection, corporate proxy traversal, and `agent login` flows. | | `desktop` | `tests/e2e/desktop_test.go` | `--from` / `--only` | `300s` | `desktop` layer VNC relay, authentication, clipboard filtering, and `TestE2EDesktop`. | | `nix` | `tests/e2e/nix_test.go` | Yes | `900s` | `nix` layer build behind a corporate proxy, multi-user `nix-daemon` confinement, `flake.nix` + `direnv` activation, and blocked fixed-output derivations. | | `from` | `tests/e2e/from_test.go` | `--from` / `--only` | `1800s` | OCI base conversion (`node:24` Debian glibc, `alpine:3` OpenRC musl, `ubuntu:26.04`) without a running Docker daemon, musl tool images, and layer compatibility. | | `link` | `tests/e2e/link_test.go` | Yes | `900s` | Inter-VM links (`127.77.*.*`), `publish`/`unpublish`, 256-connection concurrency, `sshd` `PermitListen`/`GatewayPorts` hardening, and cross-tier enforcement. | To add a check or a new section, call `s.Check` inside `Section(s *S)`: ```go func gcSection(s *S) { s.Check("gc deletes a tool image no VM uses, with its record", func() error { if Exists(old) || Exists(old+".json") { return errors.New("the old tool image or its record is still there") } return nil }) } ``` ## Related pages Explore the four-role backend.Backend interface, capability guard checks, and the backendtest.Conformance suite. Understand how vm.json sections, layout migrations in internal/state, and paths.Dirs ownership work. Maintain SHA-256 download pins, e2e-mise.lock, e2eUbuntuRelease, and Minisign release signatures with scripts/bump.sh. Review config.toml settings, environment variables, state directory layout, and version 1 --json schemas. --- ## 23. Pinning, release signing, and supply-chain verification > Maintain SHA-256 download pins and Ubuntu PGP archive checks with scripts/bump.sh, build and sign releases with Minisign, and publish artifacts. - Page Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/pages/23-pinning-release-signing-and-supply-chain-verification.md - Generated: 2026-11-07T23:01:02.126Z ### Source Files - `scripts/bump.sh` - `scripts/release.sh` - `scripts/publish.sh` - `.goreleaser.yaml` - `internal/ubuntu/pgp.go` - `internal/ubuntu/archive.go` - `internal/fetch/fetch.go` --- title: "Pinning, release signing, and supply-chain verification" description: "Maintain SHA-256 download pins and Ubuntu PGP archive checks with scripts/bump.sh, build and sign releases with Minisign, and publish artifacts." --- `devvm` enforces cryptographic verification across every external download, Ubuntu cloud image, APT archive index, and release binary. Image build scripts pin tool versions and per-architecture SHA-256 digests managed by `scripts/bump.sh`, `internal/ubuntu` verifies OpenPGP signatures on Ubuntu cloud image checksums and APT `InRelease` files against embedded RSA keyrings, `internal/fetch` enforces redirect policies, size limits, and SHA-256 checks on host HTTP transfers, and `scripts/release.sh` alongside `scripts/publish.sh` builds GoReleaser artifacts from signed Git tags and signs `SHA256SUMS` with Minisign (`ED` Blake2b-512 prehashed Ed25519 signatures). ```mermaid flowchart TB subgraph Maintainer["Maintainer & CI workflows"] Bump["scripts/bump.sh\n(.github/workflows/pins.yml)"] Tag["Signed Git tag (vX.Y.Z)\nchecked vs packaging/release-signer"] Rel["scripts/release.sh\n+ .goreleaser.yaml"] Pub["scripts/publish.sh\n(GitHub release + homebrew-tap)"] end subgraph RepoPins["Committed repository pins & keys"] Scripts["assets/layers/*.sh\nassets/features/*.sh\nassets/shared-tools/build.sh\nassets/oci/build.sh\ninternal/tools/mise.go"] PGPKeys["assets/layers/ubuntu-cloud.gpg\nassets/layers/ubuntu-archive.gpg"] MinisignPub["packaging/minisign.pub\n(packaging.MinisignKey)"] end subgraph RuntimeVerify["Host & guest verification"] Prelude["assets/prelude.sh (devvm_sha)\n+ /usr/local/bin/devvm-fetch"] UbuntuPkg["internal/ubuntu (pgp.go, archive.go)\n+ internal/build/packages.go"] FetchPkg["internal/fetch.Client\n(HTTPSOnly / SameScheme / Download)"] UpgradePkg["install.sh & internal/upgrade\n(PublicKey.Verify + SHA256SUMS)"] end Bump -->|"stages & validates regexes"| Scripts Scripts --> Prelude PGPKeys --> UbuntuPkg FetchPkg --> UbuntuPkg Tag --> Rel Rel -->|"minisign -Sm SHA256SUMS"| Pub MinisignPub --> Rel MinisignPub --> UpgradePkg Pub -->|"dist/* + SHA256SUMS.minisig"| UpgradePkg FetchPkg --> UpgradePkg ``` ## Download pinning and `scripts/bump.sh` Every external tool fetched during image construction or host toolchain resolution is pinned by version and per-architecture checksum. No build script runs unverified `curl | sh` pipelines or resolves floating `"latest"` URLs at build time. ### Guest download verification (`prelude.sh` and `devvm-fetch`) Before `devvm` executes any image build script with `bash` (`assets/layers/base.sh`, `assets/layers/apt.sh`, built-in layers, features, `assets/shared-tools/build.sh`, and `assets/oci/build.sh`), it prepends `assets/prelude.sh` to the script input and hashes both together into the image key. `assets/prelude.sh` sets two environment variables and defines one lookup function: - `devvm_arch`: normalizes `uname -m` (`aarch64`/`arm64` to `arm64`, `x86_64`/`amd64` to `amd64`). - `devvm_libc`: sets `musl` when `apk` is present on `PATH`, or `gnu` otherwise. - `devvm_sha `: resolves `_sha256__musl` when running on `musl` (if defined in the script), or `_sha256_`. Fails immediately if the script does not define the variable. `assets/layers/base.sh` installs `/usr/local/bin/devvm-fetch`, which all layer, feature, and shared-tool scripts call to download pinned files: ```sh title="assets/layers/base.sh" #!/bin/sh set -eu url=$1 want=$2 out=$3 curl -fsSL --retry 3 -o "$out" "$url" sum=sha256sum [ "${#want}" -ne 128 ] || sum=sha512sum got=$("$sum" "$out" | cut -d' ' -f1) if [ "$got" != "$want" ]; then rm -f "$out" printf 'devvm-fetch: checksum mismatch for %s\n expected %s\n got %s\n' "$url" "$want" "$got" >&2 exit 1 fi ``` `devvm-fetch` checks 64-character hex strings with `sha256sum` and 128-character hex strings (used for Microsoft `.NET` SDK archives in the tools lock) with `sha512sum`, removing the output file before exiting on mismatch. ### Pin targets managed by `scripts/bump.sh` `scripts/bump.sh` inspects and updates all 14 pinned targets across `assets/`, `internal/tools/mise.go`, `tests/e2e/`, and `.github/workflows/e2e.yml`. Requires `curl` and `jq` on `PATH`. ```bash # Compare all current pins against upstream without modifying files scripts/bump.sh --check # Update all pins and print a commit-ready summary scripts/bump.sh # Update specific targets only scripts/bump.sh mise rustup docker ``` | Target | Pinned file(s) | Variable(s) updated | Upstream source and checksum verification | | --- | --- | --- | --- | | `docker` | `assets/layers/docker.sh` | `docker_version`, `containerd_version`, `buildx_version`, `compose_version`, `model_version`, `docker_gpg_sha256` | Reads `download.docker.com/linux/ubuntu/dists/resolute/stable/binary-arm64/Packages` for `docker-ce`, `containerd.io`, `docker-buildx-plugin`, `docker-compose-plugin`, and `docker-model-plugin`; hashes `https://download.docker.com/linux/ubuntu/gpg`. | | `nix` | `assets/layers/nix.sh` | `nix_version`, `nix_sha256_arm64`, `nix_sha256_amd64` | Parses the tarball version from `https://nixos.org/nix/install` and reads published `.tar.xz.sha256` files from `releases.nixos.org`. | | `direnv` | `assets/layers/nix.sh` | `direnv_version`, `direnv_sha256_arm64`, `direnv_sha256_amd64` | Resolves latest GitHub release of `direnv/direnv` and computes SHA-256 from `direnv.linux-arm64` and `direnv.linux-amd64` binaries. | | `nix-direnv` | `assets/layers/nix.sh` | `nix_direnv_version`, `nix_direnv_sha256` | Resolves latest GitHub release of `nix-community/nix-direnv` and computes SHA-256 of `direnvrc`. | | `mise` | `assets/layers/base.sh`, `internal/tools/mise.go` | `mise_version`, `mise_sha256_arm64`, `mise_sha256_amd64`, `mise_sha256_arm64_musl`, `mise_sha256_amd64_musl`, `miseVersion`, `macos-arm64`, `macos-x64`, `linux-arm64`, `linux-x64` | Reads `SHASUMS256.txt` from the latest `jdx/mise` release. Updates both guest `base.sh` and host `internal/tools/mise.go` together (`TestMisePins` enforces equality). | | `rustup` | `assets/shared-tools/build.sh` | `rustup_version`, `rustup_sha256_arm64`, `rustup_sha256_amd64`, `rustup_sha256_arm64_musl`, `rustup_sha256_amd64_musl` | Reads `https://static.rust-lang.org/rustup/release-stable.toml` and fetches published `rustup-init.sha256` files for glibc and musl targets. | | `crane` | `assets/oci/build.sh` | `crane_version`, `crane_sha256_arm64`, `crane_sha256_amd64` | Reads `checksums.txt` from the latest `google/go-containerregistry` GitHub release. | | `omb` | `assets/features/omb.sh` | `omb_commit` | Resolves the `master` branch HEAD commit SHA of `ohmybash/oh-my-bash`. | | `eza` | `assets/features/cli-tools.sh` | `eza_version`, `eza_sha256_arm64`, `eza_sha256_amd64` | Resolves latest `eza-community/eza` release and computes SHA-256 of the `aarch64-unknown-linux-gnu` and `x86_64-unknown-linux-musl` tarballs. | | `tealdeer` | `assets/features/cli-tools.sh` | `tealdeer_version`, `tealdeer_sha256_arm64`, `tealdeer_sha256_amd64` | Resolves latest `tealdeer-rs/tealdeer` release and reads published `.sha256` files for `aarch64-musl` and `x86_64-musl`. | | `e2e-tools` | `tests/e2e-mise.lock` | Header `# devvm e2e: shared-tools/mise.toml sha256:` and all `[[tools.*]]` entries | Downloads the host `mise` binary pinned in `internal/tools/mise.go`, verifies its SHA-256, runs `mise lock --platform linux-arm64,linux-arm64-musl,linux-x64,linux-x64-musl` on `assets/shared-tools/mise.toml`, injects `.NET` SDK URLs and SHA-512 sums from `builds.dotnet.microsoft.com`, and verifies every non-`rust` tool has a platform checksum. | | `e2e-from` | `tests/e2e/from_test.go` | `fromNodeDigest`, `fromAlpineDigest`, `fromUbuntuDigest` | Fetches manifest list indexes from `registry-1.docker.io` for `library/node:24`, `library/alpine:3`, and `library/ubuntu:26.04` and computes their `sha256:` digests. | | `e2e-ubuntu` | `tests/e2e/sandbox_test.go` | `e2eUbuntuRelease` | Reads `https://cloud-images.ubuntu.com/releases/resolute/` and selects the newest `release-YYYYMMDD` directory. | | `e2e-lima` | `.github/workflows/e2e.yml` | `LIMA_VERSION`, `LIMA_SHA256` | Fetches the latest `lima-vm/lima` release, extracts the `SHA256SUMS` file's own SHA-256 from the release notes body, verifies the downloaded `SHA256SUMS` against that digest, and extracts the checksum for `lima--Linux-x86_64.tar.gz`. | ### Atomic staging and validation rules in `scripts/bump.sh` Because several files modified by `scripts/bump.sh` execute on the host or inside build VMs, `scripts/bump.sh` validates every upstream value against a strict regular expression in `valid()` before staging any edit: - `*_sha256*`: `^[0-9a-f]{64}$` - `*_commit`: `^[0-9a-f]{40}$` - `*_digest` or `*Digest`: `^sha256:[0-9a-f]{64}$` - `*_release`: `^release-[0-9]{8}$` - `*_built`: `^[0-9]{8}T[0-9]{6}Z$` - All other version pins: `^[0-9A-Za-z][0-9A-Za-z.+~_-]*$` In `report()`, control characters (`\000-\037\177`) are stripped before printing upstream strings to the terminal, and if an upstream release marker lags behind the currently pinned version (`sort -V`), `bump.sh` reports `newer than upstream's latest` and refuses to downgrade the pin. All edits are queued in the `staged` array by `set_var()` and applied in `write_staged()` only after every target succeeds, leaving the working tree untouched if any download or validation fails. Every Monday at `06:47 UTC`, `.github/workflows/pins.yml` runs `scripts/bump.sh --check` and opens or updates a single GitHub issue titled `"Pinned versions to bump"` (or closes it when all pins are current). The workflow has `contents: read` permission and never commits changes automatically. ## Host HTTP client (`internal/fetch`) All host-side HTTP requests in `devvm` go through `internal/fetch.Client`. Constructing a zero `&fetch.Client{}` literal fails every request with `fetch.ErrNoClient`; callers must call `fetch.NewClient(timeout, redirects)`. | Function / Type | Signature | Behavior and verification | | --- | --- | --- | | `NewClient` | `NewClient(timeout time.Duration, r Redirects) *Client` | Clones `http.DefaultTransport`, sets the request timeout (covering connection and full body read), and attaches a redirect policy (`HTTPSOnly` or `SameScheme`, capped at `maxRedirects = 10`). | | `HTTPSOnly` | `Redirects = iota` | Rejects any redirect target whose URL scheme is not `https` with `fetch.ErrRedirect`. Used by `upgrade.NewSource`. | | `SameScheme` | `Redirects` | Rejects any redirect that changes the initial request's URL scheme (`from to `), preventing an HTTP server from redirecting a `file://`-capable client to a local file. Used by `ubuntu.NewClient`. | | `ReadCapped` | `ReadCapped(r io.Reader, limit int64) ([]byte, error)` | Reads up to `limit + 1` bytes and returns `fetch.ErrTooLarge` if the stream exceeds `limit`. | | `Get` | `Get(ctx context.Context, c *Client, url string, limit int64) ([]byte, error)` | Requires `200 OK` (returning `*fetch.StatusError` otherwise) and reads the body through `ReadCapped(body, limit)`. | | `Download` | `Download(ctx context.Context, c *Client, url, path, sum string, limit int64) error` | Streams `url` to `path` (`0600`) while hashing with `crypto/sha256` and limiting size to `limit`. Deletes `path` and returns `fetch.ErrTooLarge` or `fetch.ErrChecksum` if the byte count exceeds `limit` or the hex digest does not match `sum`. | | `Retry` / `Transient` | `Retry(ctx context.Context, retries int, delay time.Duration, try func() (bool, error)) error` | Retries with exponential backoff (`delay *= 2`) only when `Transient(err)` is true: network I/O errors, HTTP `408`, `429`, or `5xx`. Non-transient errors (`ErrRedirect`, `ErrNoClient`, `ErrProxy`, `ErrTooLarge`, `ErrChecksum`) stop immediately. | ## Ubuntu OpenPGP and archive verification `devvm` does not shell out to `gpg` on the host. Instead, `internal/ubuntu/pgp.go` implements a minimal RFC 4880 packet reader backed by Go's standard library (`crypto/rsa`, `crypto/sha256`, `crypto/sha512`) that verifies Ubuntu cloud image checksums and APT archive `InRelease` files against embedded RSA keyrings. ### Embedded Ubuntu keyrings | Constant | Embedded path | Key identity and fingerprint | Verification method | | --- | --- | --- | --- | | `ubuntu.Keyring` | `assets/layers/ubuntu-cloud.gpg` | Ubuntu Cloud Image Builder (`D2EB44626FDDC30B513D5BB71A5D6C4C7DB87C81`) | `verifyDetached` (`sigBinary = 0x00`) over `/SHA256SUMS` and `/SHA256SUMS.gpg` | | `ubuntu.ArchiveKeyring` | `assets/layers/ubuntu-archive.gpg` | Ubuntu Archive Automatic Signing Key (2018) `` (`F6ECB3762474EDA9D21B7022871920D1991BC93C`) | `VerifyRelease` / `verifyClearsigned` (`sigText = 0x01`) over suite `InRelease` files | ### OpenPGP packet verification rules (`internal/ubuntu/pgp.go`) `verify()` and `verifyClearsigned()` enforce strict packet constraints and reject any other input with `errBadSignature`: 1. **Keyring parsing (`rsaKeys`)**: Accepts only version 4 `tagPublicKey` (`6`) and `tagSubkey` (`14`) packets using algorithm `pubRSA` (`1`) or `pubRSASignOnly` (`3`), with an exponent between 1 and 4 bytes. Streamed partial-length packets are refused. 2. **Armor decoding (`dearmor`, `decodeArmor`)**: Accepts binary packets or ASCII-armored `-----BEGIN PGP SIGNATURE-----` blocks, validating the 24-bit OpenPGP CRC-24 checksum (`init = 0xB704CE`, `poly = 0x1864CFB`) when present. 3. **Signature packet constraints (`readSignature`, `checkSubpackets`)**: - At most `maxSignatures = 4` signature packets (`tagSignature = 2`) are tried per file. - Must be version 4, matching `sigType` (`0x00` binary or `0x01` canonical text), signed with RSA (`1` or `3`), and hashed with SHA-256 (`8`), SHA-384 (`9`), or SHA-512 (`10`). - Hashed subpackets are inspected for creation time (`subCreated = 2`) and expiration offset (`subExpires = 3`). If `now.Unix() > int64(created)+int64(expires)`, the signature is rejected. Any critical subpacket (`typ & 0x80 != 0`) other than `subCreated`, `subExpires`, `subIssuer` (`16`), or `subIssuerFP` (`33`) is rejected. 4. **Clearsigned text framing (`verifyClearsigned`)**: - Must begin with `-----BEGIN PGP SIGNED MESSAGE-----`, followed only by `Hash: ` headers, a blank line, dash-escaped text (`- ` prefix stripped; any unescaped leading `-` fails), and `-----BEGIN PGP SIGNATURE-----`. - Trailing spaces/tabs are stripped per RFC 4880, lines are joined with `\r\n` for signature verification (`sigText`), and returned normalized with `\n` line endings. ### Cloud image discovery and pinning (`internal/ubuntu/ubuntu.go`) On the first build of `base` (Ubuntu `26.04`, series `resolute`), `ubuntu.Finder.Newest` fetches `https://cloud-images.ubuntu.com/releases/resolute/` (up to `maxBody = 4 MiB`, `timeout = 60s`, 3 retries) and iterates `release-YYYYMMDD` directories from newest to oldest: 1. Downloads `/SHA256SUMS` and `/SHA256SUMS.gpg`. 2. Verifies `SHA256SUMS.gpg` against `assets/layers/ubuntu-cloud.gpg` via `verifyDetached`. If a signature fails verification, `Finder.Newest` stops immediately with `ubuntu_bad_signature`—it never falls back to an older release on a bad signature. 3. Extracts the 64-hex digest for `ubuntu-26.04-server-cloudimg-.img`. Releases that do not list an image for the host architecture are skipped (`errNoImage`). 4. Saves `state.UbuntuPin{Release, File, Digest, Pinned}` on the host and hashes ` ` (`ubuntu.PinArg`) into `base`'s image key. ### APT package recording and host verification (`internal/build/packages.go`) When building `base` for a new image key, `assets/layers/apt.sh` runs in `record` mode inside the build VM and streams `/var/cache/devvm/packages/out` back to the host (`taken.take`). Because the build VM is untrusted, the host re-verifies every file in `internal/build/packages.go`: 1. **Stream caps**: Enforces limits on `Packages` (`64 MiB`), individual `.deb` files (`2 GiB`), total `.deb` stream (`16 GiB`, max `20,000` tar entries), individual `.InRelease` files (`4 MiB`, `32 MiB` total), and `.Packages` lists (`512 MiB`, `2 GiB` total). 2. **`InRelease` signature and staleness check (`taken.verified`)**: - Verifies each `.InRelease` against `assets/layers/ubuntu-archive.gpg` via `ubuntu.VerifyRelease`. - Rejects any suite not in `{resolute, resolute-updates, resolute-security, resolute-backports}`. - Checks `Release.Stale(pin.Release)`: Because Ubuntu's `InRelease` files omit `Valid-Until`, `Stale` rejects the `resolute` release pocket if its `Date` is before `ubuntu.Released` (`20260423`), and rejects `-updates`, `-security`, or `-backports` pockets if their `Date` is more than `pocketLag` (`24h`) older than the pinned cloud image's `release-YYYYMMDD` date (`packages_stale_mirror` or `packages_stale`). 3. **Suite completeness (`signed.complete`)**: Requires verified `InRelease` files and all non-empty `Packages` lists for both `resolute-updates` and `resolute-security` (`ubuntu.RequiredSuites()`) across every architecture used by the recorded packages (`packages_lists_missing`). 4. **List and `.deb` digest binding**: Hashes each `.Packages` list on the host and matches its SHA-256 and byte size against a verified `InRelease`. Hashes each `.deb` as it streams to disk and verifies its metadata and SHA-256 match the signed `Packages` list entry (`archiveEntry`). 5. **No downgrade check (`signed.checkNewest`)**: Compares every recorded `.deb` version against the newest version listed across `resolute`, `resolute-updates`, and `resolute-security` (`ubuntu.CompareVersions`), failing with `packages_not_newest` if an older package was substituted. Once verified, the `.deb` files and `state.PackageSet` are committed to the host cache under `PackagesLock()`. Subsequent builds of the same `base` key run `apt.sh` in `replay` mode using `deb [trusted=yes] file:/var/cache/devvm/packages/in ./` with no network access, after the host re-verifies each cached `.deb`'s SHA-256 and size (`checkCached` and `sendCached`). ### Testing against Ubuntu's newest cloud image End-to-end tests normally pin `base` to `e2eUbuntuRelease` (`tests/e2e/sandbox_test.go`). To verify that a user's unpinned first build of `base` works against Ubuntu's latest cloud image and live mirror packages, run: ```bash mise run e2e -- --newest-base ``` `--newest-base` runs in an isolated sandbox (`~/.lt-e2e-go-newest`, `/tmp/devvm-go-e2e-newest`, port offset `+1000`), deletes cached images and package sets first, runs the `images`, `project`, and `tools` sections, and fails with `UPSTREAM DRIFT` if `base` fails to build or falls back to an older release (`origin_no_boot`). ## Building and signing releases (`scripts/release.sh`) Releases are built locally from a signed Git tag using GoReleaser (`.goreleaser.yaml`) and signed with Minisign. The maintainer's secret Minisign key never leaves the local machine and is never stored in GitHub Actions secrets. ### Key setup and rotation `scripts/release.sh` enforces two independent cryptographic keys: 1. **Git tag-signing key (`packaging/release-signer`)**: - Tags must be created with `git tag -s v`. - `scripts/release.sh` runs `git verify-tag` and `git verify-tag --raw`, extracting the GPG fingerprint (`[GNUPG:] VALIDSIG`) or SSH key fingerprint (`SHA256:<43-base64-chars>`) and matching it case-insensitively against `packaging/release-signer` in the current working tree (never reading `packaging/release-signer` from the tagged commit itself). - To rotate the tag-signing key, add the new fingerprint to `packaging/release-signer`, sign the tag with the new key, and remove the old fingerprint. 2. **Minisign release key (`packaging/minisign.pub`)**: - Generated once with `minisign -G -p packaging/minisign.pub -s ~/.minisign/devvm.key`. - `packaging/minisign.pub` is committed to the repository and embedded into the `devvm` binary by `packaging/key.go` (`packaging.MinisignKey()`). - The secret key defaults to `~/.minisign/devvm.key` (overridable via `DEVVM_MINISIGN_KEY`). ### `scripts/release.sh` modes and environment variables Run `scripts/release.sh` through `mise` so the pinned `goreleaser` and `minisign` binaries from `mise.toml` are on `PATH`: ```bash # Build and sign dist/ from the signed tag v mise exec -- scripts/release.sh # Validate tag signature, VERSION, and build prerequisites without building mise exec -- scripts/release.sh --check # Build a snapshot from HEAD for local testing mise exec -- scripts/release.sh --head # Regenerate Homebrew formulas from an already-published release's SHA256SUMS + SHA256SUMS.minisig mise exec -- scripts/release.sh --formula-from /path/to/downloaded/dir ``` Path to the Minisign secret key used to sign `dist/SHA256SUMS`. When running `scripts/release.sh --head` without `DEVVM_MINISIGN_KEY`, the build succeeds unsigned and sets `minisign_key=""` in `dist/install.sh`. Public key file used only with `scripts/release.sh --head` to test signing with a throwaway key. Tag builds always use the `packaging/minisign.pub` committed in the tag. Base URL written into `dist/devvm.rb` and `dist/devvm-rc.rb` for archive downloads. ### Build pipeline and `dist/` outputs When invoked for a release, `scripts/release.sh` executes the following sequence: 1. **Version and tag validation**: Reads `VERSION`, verifying it matches `^[0-9]+\.[0-9]+\.[0-9]+(-[a-z]+\.[0-9]+)?$` (such as `0.2.0` or `0.2.0-rc.1`). Verifies `refs/tags/v` exists, passes `git verify-tag` against `packaging/release-signer`, and has an identical `VERSION` file inside the tagged commit. 2. **Clean detached clone build**: Clones the repository into a temporary directory (`$TMPDIR/devvm-release.XXXXXX/src`), checks out the tagged commit detached, and runs `DEVVM_VERSION=$v goreleaser release --clean --skip=publish`. - `.goreleaser.yaml` builds `./cmd/devvm` with `CGO_ENABLED=0`, `-trimpath`, `mod_timestamp: "{{ .CommitTimestamp }}"`, and `-ldflags "-s -w -X github.com/devvmhq/devvm/internal/cli.version={{ .Version }}"` across `darwin` and `linux` on `arm64` and `amd64`. 3. **Host binary version check**: Runs the built `devvm --version` binary matching the host OS and architecture and verifies its output equals `VERSION`. 4. **Installer and Homebrew formula generation**: - Injects `release_version=""` and `minisign_key=""` into the tagged commit's `install.sh` and writes `dist/install.sh`. - Renders `packaging/homebrew/devvm.rb` into `dist/devvm-rc.rb` (class `DevvmRc`, `conflicts_with "devvmhq/tap/devvm"`) for all releases, plus `dist/devvm.rb` (class `Devvm`, `conflicts_with "devvmhq/tap/devvm-rc"`) for non-prereleases, substituting `@URL__@` and `@SHA256__@` from `dist/SHA256SUMS` and failing if any `@...@` placeholder remains. 5. **Checksum and Minisign signing**: - Appends the SHA-256 of `dist/install.sh` to `dist/SHA256SUMS` so users can verify `install.sh` against the release signature before executing it. - Signs `dist/SHA256SUMS` with `minisign -Sm "$out/SHA256SUMS" -s "$key" -t "devvm $v"`, binding the version in the signed trusted comment (`devvm `). - Verifies the generated `dist/SHA256SUMS.minisig` immediately against `packaging/minisign.pub` (`minisign -Vm "$out/SHA256SUMS" -P "$pubkey"`). :::files dist/ ├── devvm--darwin-arm64.tar.gz ├── devvm--darwin-amd64.tar.gz ├── devvm--linux-arm64.tar.gz ├── devvm--linux-amd64.tar.gz ├── devvm-.tar.gz ├── SHA256SUMS ├── SHA256SUMS.minisig ├── install.sh ├── devvm-rc.rb └── devvm.rb ::: ## Publishing releases (`scripts/publish.sh`) `scripts/publish.sh` orchestrates the end-to-end release publication to `devvmhq/devvm` and `devvmhq/homebrew-tap`. Every step is idempotent and resumable if a network failure interrupts publication. Never run `scripts/publish.sh` without `--dry-run` unless explicitly publishing a release. The script pushes to `origin` and `devvmhq/homebrew-tap` and creates public GitHub releases. Update `VERSION` and add a matching `## ` section to `CHANGELOG.md` on `main`. Commit the changes and create a signed tag at `main`'s `HEAD`: ```bash git tag -s "v$(tr -d '[:space:]' < VERSION)" -m "devvm $(tr -d '[:space:]' < VERSION)" ``` Run `scripts/publish.sh --dry-run` via `mise` to run all repository, tag, GitHub, and Homebrew tap checks, build and sign `dist/`, and inspect the release notes and formula diffs without pushing: ```bash mise exec -- scripts/publish.sh --dry-run ``` Before prompting for the Minisign password, `scripts/publish.sh` checks that: - The working tree is clean (`git status --porcelain`), `gh` is authenticated (`gh auth status`), and `minisign` is installed. - `scripts/release.sh --check` succeeds, the signed tag points to `refs/heads/main`, `origin/main` is an ancestor of `main`, and `origin` does not have a conflicting `refs/tags/v`. - The `CHANGELOG.md` file in the tagged commit has a non-empty `## ` section. - The Homebrew tap checkout at `DEVVM_TAP_DIR` (default `../homebrew-tap`) exists, is clean, is on branch `main`, and is not behind `origin/main`. Run `scripts/publish.sh` and confirm the terminal prompt (`Publish devvm ? [y/N]`): ```bash mise exec -- scripts/publish.sh ``` `scripts/publish.sh` executes four steps in order, skipping any step already completed on a prior run: 1. `git push origin main` 2. `git push origin refs/tags/v` 3. Creates the GitHub release `v` as a **draft** first (`gh release create --verify-tag --draft`, plus `--prerelease` when `VERSION` contains `-`), uploads all 4 platform archives, the source tarball, `SHA256SUMS`, `SHA256SUMS.minisig`, and `install.sh`, and then publishes the release with `gh release edit --draft=false` so a partial upload is never visible. 4. Updates `devvmhq/homebrew-tap`: writes `Formula/devvm.rb` (for stable releases) and `Formula/devvm-rc.rb` (whenever the new version is newer than the tap's current `Formula/devvm-rc.rb`, evaluated by `newer()`), commits with message `devvm ` (or `devvm-rc ` for prereleases), and pushes `main`. ## Client-side signature verification (`install.sh` and `devvm upgrade`) Both `install.sh` and `devvm upgrade` (`internal/upgrade`) verify the Minisign signature on `SHA256SUMS` before unpacking any release archive. | Surface | Public key source | Verification implementation | Rollback and safety guarantees | | --- | --- | --- | --- | | `install.sh` | `minisign_key` injected by `scripts/release.sh` | Runs `minisign -Vqm "$tmp/SHA256SUMS" -x "$tmp/SHA256SUMS.minisig" -P "$minisign_key"`, then matches `sha256sum` / `shasum -a 256` of `devvm---.tar.gz` against `SHA256SUMS`. | Extracts to `~/.local/lib/devvm/..new.$$`, atomically updates `~/.local/bin/devvm`, moves to `~/.local/lib/devvm/`, and retains older versions still referenced by running VM processes (`ps -A -ww -o command=`). | | `devvm upgrade` | `packaging.MinisignKey()` embedded from `packaging/minisign.pub` | `upgrade.Source.Fetch` (`internal/upgrade/source.go`) calls `PublicKey.Verify` (`internal/upgrade/minisign.go`), verifying: (1) algorithm is prehashed `ED` (rejecting legacy unhashed `Ed`), (2) 8-byte key ID matches, (3) Ed25519 signature over `blake2b.Sum512(SHA256SUMS)` is valid, (4) global Ed25519 signature over `signature \|\| trusted_comment` is valid, and (5) trusted comment equals `"devvm " + version`. | Downloads archive via `fetch.Download` (enforcing `archiveCap = 256 MiB` and the `SHA256SUMS` digest), unpacks to `.version.new.`, runs `devvm --version` before and after switching the symlink (`switchLink`), and rolls back the symlink and directory on failure. Refuses downgrades unless `--to` (`Options.Version`) is specified. | ## Related pages Install devvm via Homebrew or the signed release installer and verify Minisign signatures. How pinned scripts, prelude.sh, and Ubuntu cloud image pins hash into deterministic image keys. Upgrade devvm installations with devvm upgrade and clean up unused images and cached packages. Run unit, structure, and real-VM E2E tests including --newest-base drift checks. ---