# Overview > Public entry points, per-project VM sandbox architecture, runtime assumptions, and primary documentation routes. - Repository: devvmhq/devvm - GitHub: https://github.com/devvmhq/devvm - Human docs: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853 - Complete Markdown: https://grok-wiki.com/public/docs/devvmhq-devvm-4242986d2853/llms-full.txt ## Source Files - `cmd/devvm/main.go` - `internal/cli/root.go` - `internal/cli/app.go` - `internal/vm/env.go` - `README.md` --- --- title: "Overview" description: "Public entry points, per-project VM sandbox architecture, runtime assumptions, and primary documentation routes." --- `devvm` is a Go CLI (`cmd/devvm`) that provisions and manages one isolated virtual machine per project directory on macOS and Linux using Lima (`internal/backend/lima`). Each project VM clones a content-addressed Ubuntu or OCI base image, mounts shared toolchains and coding agents read-only from host EROFS images, confines outbound network traffic through a per-VM host proxy (`devvm _proxy`), and stores all runtime state under `~/.local/share/devvm` and `~/.config/devvm` without writing files into the project workspace. ## System architecture `devvm` separates CLI command dispatch and terminal interaction (`internal/cli`, `internal/ui`) from VM lifecycle orchestration (`internal/vm`), host-side security boundaries (`internal/netpolicy`, `internal/proxy`, `internal/gitprotect`, `internal/secrets`), and the hypervisor driver contract (`internal/backend`). ```mermaid flowchart TB subgraph CLI["CLI & UI Layer (cmd/devvm, internal/cli, internal/ui)"] Main["cmd/devvm/main.go\nplatform.CloseInheritedOnExec() + ui.Main"] Root["internal/cli.NewRoot\nCobra command tree + checkCommands"] App["internal/cli.app\nLazy singletons (made[T])"] end subgraph Core["VM & Policy Orchestration (internal/vm, internal/netpolicy)"] VMEnv["vm.Env\nfinder, starter, Launch, Lifecycle, Tools, Links"] Starter["starter.bringUp\nhostChecks, requirements, confinementFor"] ProxyMgr["netpolicy.Proxy\ndevvm _proxy daemon + SelfCheck"] Forwarder["vmssh.Forwards\ndevvm _ports SSH loopback forwarder"] end subgraph State["Host State & Config (internal/paths, internal/state, internal/settings)"] ConfigDir["~/.config/devvm\nconfig.toml, layers/, policy, ca/, agents/"] DataDir["~/.local/share/devvm\nsettings.json, vms//vm.json, images/, tools/, logins/"] end subgraph BackendLayer["Backend Contract (internal/backend)"] Registry["backend.Registry & Review.Require\nCapability guard (Caps)"] Lima["lima.Backend\nHost, Images, Machines, Housekeeping"] end subgraph Guest["Project VM Sandbox (Lima Guest)"] MountRW["Project Folder Mount (read-write)\nor Independent Git Clone (--clone)"] ToolsRO["Shared Tools Mount (read-only)\nEROFS images at /opt/devvm/tools"] AgentRO["Agent Settings Mounts (read-only)\nPlaceholder credentials"] end Main --> Root --> App App --> VMEnv VMEnv --> Starter Starter --> ProxyMgr Starter --> Forwarder Starter --> Registry --> Lima VMEnv --> ConfigDir VMEnv --> DataDir Lima --> Guest Guest -- "HTTP/TLS via 127.0.0.1 proxy port" --> ProxyMgr ``` ## Public entry points and execution flow ### Binary entry point (`cmd/devvm/main.go`) The `devvm` binary serves both user-facing commands and internal background daemons. Execution begins in `cmd/devvm/main.go`: ```go title="cmd/devvm/main.go" func main() { platform.CloseInheritedOnExec() os.Exit(ui.Main(func(ctx context.Context, u *ui.UI) int { return cli.Execute(ctx, u, os.Args[1:]) })) } ``` 1. **Descriptor hygiene:** `platform.CloseInheritedOnExec()` marks any file descriptors above standard input (`0`), standard output (`1`), and standard error (`2`) close-on-exec so child processes (`limactl`, `ssh`, `git`, or background daemons) never inherit leaked descriptors. 2. **Terminal & signal lifecycle:** `ui.Main` constructs the run-scoped `*ui.UI` instance (`internal/ui`), traps `SIGINT`, `SIGTERM`, and `SIGHUP` to cancel the root context, waits for any active `LoggedCmd` subprocess to finish, executes the LIFO cleanup stack (`AddCleanup`), and returns the process exit code to `main`, which holds the sole `os.Exit` call in the repository. 3. **Command execution:** `cli.Execute` builds the Cobra command tree via `NewRoot(u)` and runs pre-execution validation before dispatching `RunE`. ### Service initialization (`internal/cli/app.go`) `NewRoot` creates a single `app` struct that holds lazy-initialized services wrapped in `made[T]`. Each service is constructed at most once per CLI invocation when requested by a command: | Service accessor | Underlying type | Responsibility | | --- | --- | --- | | `a.dirs()` | `paths.Dirs` | Resolves `$HOME`, `$XDG_DATA_HOME/devvm`, `$XDG_CONFIG_HOME/devvm`, `$DEVVM_LIMA_HOME`, and `$XDG_CACHE_HOME` | | `a.secrets()` | `*secrets.Store` | Manages host-stored API tokens, proxy passwords, and OAuth credentials (macOS Keychain or `~/.local/share/devvm/secrets`) | | `a.registry()` | `*backend.Registry` | Registers VM backends (`lima`), applies corporate proxy settings, and enforces capability checks | | `a.tools()` | `*tools.Env` | Manages `mise.lock` resolution and read-only shared tool image builds | | `a.vmEnv()` | `*vm.Env` | Wires `finder`, `starter`, `Launch`, `Lifecycle`, `Tools`, `Groups`, `Links`, `Proxy`, and `Forwarders` for project VM operations | | `a.builder()` | `*build.Env` | Builds and rebuilds base and layer VM images on the default backend | ### Pre-run invariants and exit codes Every command registered under `NewRoot` passes through two gates before its `RunE` handler executes: 1. **Static argument and flag validation (`checkCommands` in `internal/cli/args.go`):** Wraps every command's `Args` validator to reject blank arguments, enforce mutually exclusive flag groups (`exclusive`), and validate flag formats (`--cpus`, `--memory`, `--disk`, `--ports`, `--network`, `--tools`, `--agents`, `--layers`, `--lifetime`, `--older-than`, `--to`) prior to touching host state or looking up a VM. 2. **Workspace privacy and layout version check (`PersistentPreRunE` in `internal/cli/root.go`):** - Calls `u.DataOnly()` when `--json` is passed so informational messages route to `stderr` and `stdout` emits only versioned JSON (`"version": 1`). - Enforces `0700` permissions on the config and data directories via `state.MakePrivate`. - Verifies `~/.local/share/devvm/version` and `~/.config/devvm/version` via `state.CheckLayout` (skipped only for `help`, `version`, `completion`, `upgrade`, and hidden `_` commands). - Runs background retention cleanup (`autoRetain`) when due. | Exit code | Meaning | Trigger condition | | --- | --- | --- | | `0` | Success | Command completed without error | | `1` | Runtime failure | `RunE` or `checkLayout` returned an error (`ranError`), or an interactive prompt was refused under `--no-input` / non-TTY stdin (`msg.Refusal`) | | `2` | Usage error | Unknown command/flag, invalid flag value, mutually exclusive flags, or wrong argument count (`usageCode`) | | `130` | Interrupted | Process received `SIGINT`, `SIGTERM`, or `SIGHUP`, ran registered cleanups, and exited | ### Command surface organization Running `devvm` with no subcommand executes `up` (`internal/cli/up.go`), opening an interactive shell at the same path inside the current folder's VM after creating and starting the VM if needed. Subcommands are organized into six functional groups in `internal/cli/help.go`, alongside internal daemon entry points: | Category | Commands | Purpose | | --- | --- | --- | | **Your VM** | `devvm` (root `up`), `restart`, `stop`, `update`, `rm`, `adopt`, `ls`, `code`, `desktop`, `host`, `history` | Create, enter, stop, delete, adopt moved folders, list VMs, connect editors/VNC, and export agent history | | **Change your VM** | `tools`, `size`, `ports`, `network`, `ssh-agent`, `git-protect`, `clone`, `secret`, `layer add`, `layer rm` | Reconfigure a VM's mounted toolchains/agents, CPU/RAM/disk, port forwards, network tier, `.git` protection, clone mode, secrets, and image layers | | **Links between VMs** | `link`, `group` | Publish guest ports and connect VMs across `*.devvm` hostnames or mutually reachable groups | | **Images** | `images`, `build`, `rebuild`, `layer`, `layer new` | Inspect, build, and rebuild base and layer images, or scaffold custom layers in `~/.config/devvm/layers` | | **Disk space** | `space`, `gc`, `retention` | Report disk usage across VMs/images/tools/caches, garbage-collect unused artifacts, and configure automatic retention policies | | **This machine** | `setup`, `agent`, `doctor`, `upgrade`, `completion`, `limactl` | Configure global defaults, import/login/save coding agent state, verify host dependencies, self-upgrade, and pass commands to Lima | | **Internal daemons** | `_proxy`, `_ports`, `_links`, `_qemu`, `_refresh-tools`, `_msg`, `version` | Hidden subprocess entry points spawned by `devvm` for per-VM HTTP/TLS proxying, SSH port forwarding, inter-VM links, Linux QEMU netns wrapping, and daily tool checks | ## Per-project VM sandbox architecture ### Project path to VM identity mapping `devvm` maps each project folder to a deterministic VM name and state directory (`internal/project/project.go`, `internal/vmname/vmname.go`): - **Physical path resolution:** The project directory is resolved to its physical path (`pwd -P` / `paths.Physical`). Paths containing commas or control characters are rejected (`project.BadPathError`) because Lima splits mount options on commas. - **VM naming grammar (`devvm--`):** - `` is derived from `filepath.Base(dir)` converted to lowercase `a-z0-9` with non-alphanumeric runs replaced by `-`, trimmed to at most 20 characters (`slugMax = 20`, minimum 4 characters, falling back to `root`). - `` is the first 14 lowercase hexadecimal digits (56 bits) of the SHA-1 digest of the physical directory path (`vmname.HashLen = 14`). This 14-hex length distinguishes project VMs from image build VMs (`devvm-img-<12-hex>`). - **Parent folder resolution:** When run inside a subdirectory that does not have its own VM, `finder` searches parent directories up to (but excluding) `$HOME` and `/` unless `--here` is passed (`--parent` forces using an ancestor VM). - **Moved folder detection:** Each VM's `vm.json` records both the physical path (`Dir`) and the filesystem device/inode identity (`Folder` via `platform.FolderID`). If a project folder is moved or replaced on the host, `starter.bringUp` detects the mismatch via `folderChanged` and refuses to mount an untrusted replacement path; `devvm adopt` rebinds a VM to a moved directory. ### Layered images and read-only shared tools Project VMs do not install toolchains or agents into their writable root disks on creation: - **Base and layer images (`internal/image`, `internal/build`):** Every VM clones an image identified by a deterministic SHA-256 content key (`12` hex chars). The `base` image starts from a pinned Ubuntu cloud image (or an OCI container image via `--from`) and optionally stacks built-in layers (`docker`, `nix`, `desktop`) or custom user layers (`~/.config/devvm/layers/.sh`). Adding or removing layers (`devvm layer add`, `devvm layer rm`), switching `--from`, or running `devvm update` performs a remake (`internal/vm/remake.go`), preserving the VM's `/home` directory across the new image while keeping the old machine aside as `devvm--old` until the remake succeeds. - **Shared toolchains and coding agents (`internal/tools`, `assets/shared-tools`):** - **Supported toolchains:** `node`, `python`, `rust`, `go`, `dotnet` (auto-detected from project files such as `package.json`, `pyproject.toml`, `Cargo.toml`, `go.mod`, `*.csproj` / `*.sln`, or `.devcontainer/devcontainer.json`). - **Supported coding agents:** `claude` (Claude Code), `codex` (Codex), `opencode`. - Toolchains and agents are built inside throwaway VMs into compressed EROFS images stored in `~/.local/share/devvm/tools` and mounted read-only into project VMs. Writes such as `npm install -g` or `cargo install` are redirected to the guest's `$HOME` (`~/.npm-global`, `go_set_gobin = false`). ### Filesystem, Git, network, and credential isolation `starter.bringUp` (`internal/vm/start.go`) enforces four isolation boundaries before any shell or command runs inside a VM: 1. **Workspace mount vs. repository clone:** - **Mounted folder (default):** Only the project directory is mounted read-write at the identical path inside the VM. `project.Unsafe` refuses to create or start a VM whose project path is, contains, or resides inside `$HOME`, `~/.config/devvm`, `~/.local/share/devvm`, or `$DEVVM_LIMA_HOME`. - **`.git` protection (`internal/gitprotect`):** On mounted folders, `devvm` locks `.git` hooks, config, and attributes on the host so guest processes cannot plant malicious Git hooks or filters that execute on the host. On Linux hosts, `checkSetID` audits the project folder for setuid/setgid files when starting or stopping a VM. - **Isolated clone (`devvm --clone`):** The VM mounts no project folder from the host; instead, `devvm` clones the repository into the VM's disk over SSH and configures a `devvm` Git remote on the host so changes are pulled only via explicit `git fetch devvm`. 2. **Network confinement tiers (`internal/netpolicy/tier.go`):** - `allowlist` (default): Confines the VM to `backend.NetProxy`. Guest traffic can reach only the host-side `devvm _proxy` listener, which enforces built-in base/tool/layer policies, global rules (`~/.config/devvm/policy`), per-VM rules (`~/.local/share/devvm/vms//policy`), and secret target hosts. - `none`: Confines the VM to `backend.NetProxy` and permits only traffic to configured coding agent endpoints, bound secret hosts, and linked VMs (`*.devvm`). - `internet`: Confines the VM to `backend.NetProxy` and permits public internet hosts while blocking private/loopback ranges and applying user `deny` rules. - `open`: Starts the VM on `backend.NetDirect` with `HTTP_PROXY`/`HTTPS_PROXY` pointed at `devvm _proxy` for credential injection without rule filtering. - After starting a VM, `Proxy.SelfCheck` verifies from inside the guest that direct connections bypassing the proxy fail on confined tiers, stopping the VM immediately if confinement does not hold. 3. **Host-side secret and login substitution (`internal/secrets`, `internal/agentlogin`):** - Coding agent settings in `~/.config/devvm/agents/` are mounted read-only or copied into the guest. - OAuth logins (`~/.local/share/devvm/logins/`) and user secrets (`devvm secret`) never enter the VM in plaintext. Guest files receive random per-VM salted placeholders (`netpolicy.Salt`), and the host-side `devvm _proxy` performs TLS termination using a per-VM CA (`~/.local/share/devvm/vms//run/ca`) to inject the real credentials only on requests to authorized destination hosts. 4. **SSH host-key pinning and agent isolation (`internal/vmssh`, `internal/sshagent`):** - `vmssh.FromTarget` pins each VM's SSH host keys per boot in `vms//run/known_hosts` and verifies all host-to-guest SSH connections, port forwarders (`devvm _ports`), and inter-VM links (`devvm _links`). - By default, host SSH agent forwarding is disabled; users can enable `devvm`'s isolated SSH agent (`devvm ssh-agent`) or forward their host `SSH_AUTH_SOCK`. ## Runtime assumptions and host layout ### Backend contract (`internal/backend/backend.go`) All hypervisor interactions go through `backend.Backend`, which comprises `Name()`, `Caps()`, and four role interfaces: - `Host`: checks host readiness (`Problems`), VM name length limits (`NameMax`), storage locations (`Location`), and minimum free disk space (`DiskDirs`, `Floors`). - `Images`: builds and deletes base, layer, and shared-tool EROFS images (`BuildImage`, `BuildTools`, `PinOrigin`). - `Machines`: creates, starts, stops, deletes, renames, lists, inspects, and opens shells/SSH sessions in VMs (`CreateVM`, `Start`, `Running`, `Stop`, `Delete`, `Exec`, `Shell`, `SSH`). - `Housekeeping`: reports VM/cache disk usage (`Usage`), identifies untracked VMs (`Strays`), and cleans build caches (`DevvmCache`, `DeleteCache`). Callers verify required features (`backend.Feature`) via `Registry.Require(b, vm, features...)` before invoking backend methods. If a backend lacks a required capability or relies on an unreviewed emulation, `devvm` fails immediately with an actionable message rather than silently degrading isolation. ### Host requirements - **Operating system and architecture:** macOS 13 or newer (`arm64` or `x86_64`) or Linux (`arm64` or `x86_64` with `/dev/kvm` access and QEMU). - **Lima runtime:** Lima 2.2 or newer (`limactl` on `PATH`). `devvm` isolates its Lima instances under `$DEVVM_LIMA_HOME` (default `~/.local/share/devvm/lima`) and ignores the ambient `LIMA_HOME` environment variable so external Lima configurations cannot alter `devvm` guests. - **Host readiness check:** `devvm doctor` inspects the host through `Backend.Problems` and reports missing dependencies or permission issues without installing software automatically. ### Host directory layout and configuration precedence `devvm` reads and writes state exclusively under the XDG config and data directories (`internal/paths/dirs.go`): :::files ~/.config/devvm/ # $XDG_CONFIG_HOME/devvm (mode 0700) ├── version # Config layout version ├── config.toml # User-authored global defaults (read-only to devvm) ├── policy # Global network allow/deny rules for all VMs ├── ca/ # Custom/corporate root CAs trusted by VMs and builds ├── layers/ # User-defined layer scripts (.sh and .policy) └── agents/ # Imported coding agent settings, skills, and MCP configs ~/.local/share/devvm/ # $XDG_DATA_HOME/devvm (mode 0700) ├── version # Data layout version ├── settings.json # CLI-managed global defaults (devvm setup, -g flags) ├── ssh_config # Generated SSH config included from ~/.ssh/config ├── mise.lock # Resolved versions of shared toolchains and agents ├── tools/ # Read-only EROFS images (-.erofs) and records ├── images/ # Image build records, locks, rebuild counters, ubuntu.json ├── packages/ # Content-addressed .deb cache for base image builds ├── logins/ # Host-only coding agent OAuth tokens and auth files ├── secrets/ # Fallback secret files on hosts without OS Keychain ├── lima/ # Isolated Lima home ($DEVVM_LIMA_HOME) ├── vms/ │ └── <14-hex-hash>/ # Per-project VM directory │ ├── vm.json # VM state (folder path/ID, image, tools, net, ports, size) │ ├── policy # Per-VM network policy rules │ ├── secrets.policy # Auto-generated rules for bound secret hosts │ └── run/ # Per-boot CA keys, known_hosts, proxy/forwarder PIDs & logs ├── run/ # Global locks (claims.lock, create-.lock), sockets, QEMU wrapper ├── exports/ # Agent history archives (.tar.gz and .folder metadata) ├── cache/ # Host mise binaries, shared nix-cache, --from digests, release.json └── logs/ # Command run logs (runs/) and daily tools-check.log ::: Global defaults for new VMs (`network`, `ports`, `size.cpus`, `size.memory`, `size.disk`, `features`, `agents`, `layers`, and `retention.*`) follow strict precedence in `internal/settings/settings.go`: 1. **CLI flags** (`--network`, `--ports`, `--cpus`, `--memory`, `--disk`, `--tools`, `--agents`, `--layers`) 2. **`~/.config/devvm/config.toml`** (edited manually; `devvm` never overwrites it and refuses `-g` commands that conflict with keys defined in `config.toml`) 3. **`~/.local/share/devvm/settings.json`** (updated by `devvm setup`, `devvm size -g`, `devvm ports -g`, `devvm network -g`, and `devvm retention`) 4. **Built-in defaults** (`network = "allowlist"`, `ports = "all"`, image default CPU/RAM/disk) ## Next Install devvm on macOS and Linux via Homebrew or the signed release installer, verify Minisign signatures, and validate host readiness with devvm doctor. Configure initial defaults with devvm setup, launch a project VM with auto-detected toolchains and agents, and verify isolation in the guest shell. How project paths map to VM names and hashes, state persistence in vm.json, layout version checks, and in-place vs. remake transitions. Deterministic SHA-256 key computation for base images, built-in and custom layers (docker, nix, desktop), and read-only EROFS shared tool images. Traffic confinement across allowlist, open, and offline tiers, host-side TLS SNI and HTTP inspection via devvm _proxy, and per-VM CA rotation. Read-only .git mount protection, symlink and hardlink containment, set-id file audits, secret placeholder injection, and isolated repo clones. The four-role backend.Backend interface (Host, Images, Machines, Housekeeping), capability requirement checks (Caps), and the Lima driver implementation. Complete reference for ~/.config/devvm/config.toml keys, settings.json precedence, environment variables, state directory layout, and version 1 --json payloads.