# Workspaces and isolation

> Per-sandbox bind mounts under SANDBOXED_DATA_DIR/workspaces, skeleton seeding, read-only rootfs and caps, memory/PID limits, userns=host default, and v1 storage trade-offs.

- Repository: tastyeffectco/sandboxes
- GitHub: https://github.com/tastyeffectco/sandboxes
- Human docs: https://grok-wiki.com/public/docs/tastyeffectco-sandboxes-f551c1a2e9a0
- Complete Markdown: https://grok-wiki.com/public/docs/tastyeffectco-sandboxes-f551c1a2e9a0/llms-full.txt

## Source Files

- `control-plane/internal/loopback/loopback.go`
- `image/HOME_LAYOUT.md`
- `image/skel/.profile`
- `control-plane/internal/docker/docker.go`
- `ARCHITECTURE.md`
- `control-plane/migrations/0001_init.sql`

---

---
title: "Workspaces and isolation"
description: "Per-sandbox bind mounts under SANDBOXED_DATA_DIR/workspaces, skeleton seeding, read-only rootfs and caps, memory/PID limits, userns=host default, and v1 storage trade-offs."
---

Each sandbox’s durable state lives in a host directory under `SANDBOXED_DATA_DIR/workspaces/<id>/`, provisioned once by `sandboxd` (`internal/loopback`), bind-mounted into the container at `/home/sandbox`, while the container itself runs with a hardened `docker run` flag set (read-only rootfs, dropped capabilities, memory and PID ceilings). SQLite still records `workspace_img` and `workspace_mnt` columns for compatibility; in the OSS directory-storage build both resolve to the same path.

## On-disk layout

The data root defaults to `/var/lib/sandboxed` (`SANDBOXED_DATA_DIR`). Per-sandbox trees and control-plane state sit beside each other:

```text
${SANDBOXED_DATA_DIR}/
├── workspaces/
│   └── <ulid>/          # bind-mounted → /home/sandbox in container s-<ulid>
├── state/
│   └── sandboxd.db    # SQLite WAL — sandbox rows, ports, tasks
├── _snapshots/        # optional hourly zstd archives (legacy path naming)
├── templates/         # golden templates for fast spin-up
└── library/           # v1 snapshot images when configured
```

<Note>
`docker-compose.yml` bind-mounts `${SANDBOXED_DATA_DIR}` **symmetrically** host→`sandboxd` container. The path `sandboxd` writes must be the same absolute path the host Docker daemon uses for `docker run -v <path>:/home/sandbox`. Do not mount only one side to a different path.
</Note>

| Path | Owner | Survives `docker stop`? | Survives host reboot? |
|------|-------|-------------------------|----------------------|
| `workspaces/<id>/` | Host filesystem | Yes | Yes |
| `state/sandboxd.db` | Host filesystem | Yes | Yes |
| Container writable layer | — | No (`--read-only`) | No |
| `/tmp`, `/var/tmp` inside sandbox | tmpfs | No | No |

## Provisioning and skeleton seeding

`loopback.Manager.Provision` is **idempotent**:

1. `mkdir` `workspaces/<id>/` if missing.
2. If the directory is **empty** (ignoring `lost+found` from older loopback workspaces), run a one-shot seed container from `SANDBOXD_IMAGE` (default `sandboxed-base:1.0.0`) that copies `/opt/sandbox-skel/` into the workspace and `chown`s to `sandbox:sandbox`.
3. If the directory already has content, seeding is skipped — safe for id reuse and reconciler boot passes.

Seeding uses `--user 0` and, by default, `--userns host` on the seed container so root inside the seed maps predictably on hosts with `userns-remap` enabled.

**Template path:** `ProvisionFromTemplate` clones a populated directory with `cp -a` when the workspace is empty (templates under `templates/`, or library snapshots via the v1 API).

**Id reuse:** A workspace directory may exist without a SQLite row. `POST /sandbox` can attach to that directory if no row exists; Phase 8 checks `workspace_owner` so another `external_user_id` cannot resurrect the id.

## In-container home contract

Inside the running container, user state is entirely under `/home/sandbox` (the bind mount). Project code is expected at **`/home/sandbox/workspace`** (and often `workspace/app/` for agent flows).

```text
/home/sandbox/
├── workspace/       # user project — dev servers, git, builds
├── .bashrc, .profile
├── .config/         # agent/tool configs
├── .cache/          # pnpm, pip, uv, bun caches (persistent)
├── .local/, .bun/
└── .runtimed/       # runtimed Unix socket (platform-reserved)
```

`image/skel/.profile` sources `/etc/profile.d/sandbox-env.sh` and `~/.bashrc`. The runtime entrypoint does **not** re-seed; users restore dotfiles manually from `/opt/sandbox-skel/` if needed.

`runtimed` binds its control socket at `/home/sandbox/.runtimed/sock`, visible on the host at `workspaces/<id>/.runtimed/sock`. The v1 files API refuses writes under `.runtimed/` and `lost+found/`.

## Persistence operations

| API | Effect on container | Effect on `workspaces/<id>/` | SQLite row |
|-----|---------------------|------------------------------|------------|
| `POST /v1/sandboxes/{id}/stop` | `docker stop` | Kept | Kept (`stopped`) |
| `DELETE /sandbox/{id}` | `docker rm` | **Kept** | Deleted |
| `POST /sandbox/{id}/purge` | Stop + remove | **`RemoveAll`** | Purged (+ `_snapshots/<id>/`) |

Backup a workspace by copying its directory; backup control-plane state with `state/sandboxd.db` (WAL files alongside it if present).

<Warning>
`DELETE` removes the DB row but leaves disk. A later create with the same ULID can reattach only if ownership rules allow it. Use `purge` when you intend to free disk and erase tenant data.
</Warning>

## Container isolation flag set

`sandboxd` passes an explicit `docker.RunSpec` on create — no hidden defaults inside `internal/docker`:

| Docker flag | Value at create | Role |
|-------------|-----------------|------|
| `--read-only` | `true` | No persistent writes outside mounts/tmpfs |
| `--cap-drop` | `ALL` | Drop all capabilities |
| `--security-opt` | `no-new-privileges` | Block privilege escalation |
| `--memory` / `--memory-swap` | `10g` / `10g` | Hard RSS ceiling per sandbox |
| `--pids-limit` | `1024` | Process count cap |
| `--cpu-shares` | `100` | Relative CPU weight |
| `--ulimit` | `nofile=65536:65536` | FD limit |
| `--tmpfs` | `/tmp:size=512m`, `/var/tmp:size=128m` | Ephemeral writable dirs |
| `-v` | `<workspace>:/home/sandbox` | Only durable writable tree |

<Info>
**Threat model:** Isolation targets **authenticated, accountable users running their own code**, not anonymous multi-tenant adversaries. Hardened containers mitigate misconfiguration and casual abuse; kernel escape against a determined attacker is a host-patch and trust-boundary problem — use VM-per-tenant or stronger runtimes if you need that bar.
</Info>

## Memory policy: hard ceiling vs soft throttle

Two layers apply:

| Layer | Default | Config | Behavior |
|-------|---------|--------|----------|
| Hard limit | `--memory=10g` | Fixed in create handler | OOM kill at ceiling |
| Soft throttle | `memory_high` = `4G` on row | Request field `memory_high`; DB default `4G` | cgroup v2 `memory.high` write |

`SANDBOXED_SET_MEMORY_HIGH=false` by default. When false, create/wake/reconcile skip writing `memory.high`; the 10g hard limit still applies. When true, `cgroup.SetMemoryHigh` discovers the path via `/proc/<pid>/cgroup` — failures are logged but do not fail create/wake.

Host memory pressure and wake admission are separate concerns (idle/pressure reapers, `SANDBOXD_MEM_*` knobs).

## User namespaces (`SANDBOXED_USERNS`)

| Component | Default | Purpose |
|-----------|---------|---------|
| `sandboxd`, Traefik | `userns_mode: host` in compose | Infrastructure can access Docker socket and data dir on remapped daemons |
| Sandboxes + seed | `SANDBOXED_USERNS=host` → `--userns host` | Deterministic ownership on workspace bind mounts |

Set `SANDBOXED_USERNS=` to empty to omit `--userns` on sandbox containers and seed runs, opting back into the Docker daemon’s default user namespace mode.

On `userns-remap` hosts, keeping `host` avoids seed containers mapping root to a high subuid that cannot write host-owned workspace directories — a common source of seed permission errors (see troubleshooting).

## v1 storage trade-offs

sandboxed v1 optimizes for **single-host, Docker-only install** with no extra host modules:

| Area | v1 choice | Trade-off | Hardening direction |
|------|-----------|-----------|---------------------|
| Workspace storage | Plain directory per sandbox | **No per-workspace disk quota** — shared host filesystem | Filesystem quotas, dedicated volumes, sharding |
| Snapshots (hourly) | zstd of path `workspaces/<id>.img` in `internal/snapshot` | Legacy `.img` naming; expects a **file** at that path — mismatched with directory workspaces in OSS | Use directory-aware snapshot backend or v1 library snapshots |
| v1 `POST /v1/snapshots` | `cp --reflink=auto` workspace → `library/<snapId>.img` | Works with directory trees on reflink-capable FS; `.img` suffix is naming legacy | Prefer this path for templates; verify `SANDBOXD_LIBRARY_DIR` |
| Templates | `cp -a` clone into empty workspace | Fast cold start; golden template git untouched | — |
| Egress | Default allow (OSS) | No per-sandbox egress logging | Host firewall / proxy |
| Control plane | Docker socket access | Root-equivalent on host | Dedicated VM, auth, network isolation |

ARCHITECTURE.md documents these as conscious v1 compromises, not oversights.

## Writing into workspaces from outside

Integrators can inject files without `exec`:

:::endpoint PUT /v1/sandboxes/{id}/files?path=<relative>
Atomic write under the workspace mount root. Body: `{"path","content","append"}`. 25 MiB cap; path traversal and `.runtimed/` blocked; atomic rename; optional chown to mount owner uid/gid.
:::

List/read use `GET /v1/sandboxes/{id}/files` and `.../files/content?path=`.

## Verification

<Steps>
<Step title="Inspect workspace on host">
```bash
ls -la "${SANDBOXED_DATA_DIR:-/var/lib/sandboxed}/workspaces/"
```
After create, a new ULID directory should exist with `workspace/`, dotfiles, and caches after first use.
</Step>
<Step title="Confirm bind mount in container">
```bash
API="${SANDBOXED_API_BIND:-http://127.0.0.1:9090}"
ID=<your-ulid>
curl -s -XPOST "$API/sandbox/$ID/exec" -H 'content-type: application/json' \
  -d '{"cmd":["bash","-lc","df -h /home/sandbox; mount | grep sandbox"]}'
```
`/home/sandbox` should show the host-bound filesystem, not the read-only root layer.
</Step>
<Step title="Confirm destroy keeps data">
```bash
curl -s -XDELETE "$API/sandbox/$ID"
test -d "${SANDBOXED_DATA_DIR:-/var/lib/sandboxed}/workspaces/$ID" && echo workspace retained
```
</Step>
</Steps>

## Related pages

<CardGroup>
<Card title="Sandbox lifecycle" href="/sandbox-lifecycle">
SQLite status machine, container naming, destroy vs purge semantics, and reconcile-on-boot.
</Card>
<Card title="Wake, idle, and pressure" href="/wake-idle-reapers">
Stop-on-idle preserves workspaces; wake admission uses host memory headroom.
</Card>
<Card title="Configuration reference" href="/configuration-reference">
`SANDBOXED_DATA_DIR`, `SANDBOXED_USERNS`, `SANDBOXED_SET_MEMORY_HIGH`, and related compose env keys.
</Card>
<Card title="v1 API reference" href="/v1-api-reference">
Files CRUD, snapshots-as-templates, tasks, and error envelopes.
</Card>
<Card title="Troubleshooting" href="/troubleshooting">
userns-remap seed errors, warming-page stalls, and workspace permission failures.
</Card>
<Card title="Uninstall and maintenance" href="/uninstall-maintenance">
`--data` / `--all` flags, workspace retention defaults, and backup paths.
</Card>
</CardGroup>