# Production deployment
> Wildcard DNS, traefik websecure + cert resolver, PREVIEW_TLS=true, enable API auth, hardening checklist (isolation, egress, disk), and scaling boundaries from README.
- Repository: tastyeffectco/sandboxes
- GitHub: https://github.com/tastyeffectco/sandboxes
- Human docs: https://grok-wiki.com/public/docs/tastyeffectco-sandboxes-f551c1a2e9a0
- Complete Markdown: https://grok-wiki.com/public/docs/tastyeffectco-sandboxes-f551c1a2e9a0/llms-full.txt
## Source Files
- `README.md`
- `traefik/traefik.yml`
- `.env.example`
- `docker-compose.yml`
- `ARCHITECTURE.md`
- `control-plane/internal/egress/nftables.go`
---
---
title: "Production deployment"
description: "Wildcard DNS, traefik websecure + cert resolver, PREVIEW_TLS=true, enable API auth, hardening checklist (isolation, egress, disk), and scaling boundaries from README."
---
Production moves the stack off `*.localhost` plain HTTP to a real wildcard preview domain: Traefik terminates TLS on `websecure`, `sandboxd` emits `tls=true` preview routers from `PREVIEW_ENTRYPOINT` / `PREVIEW_TLS`, and the control-plane API must run with service-token auth before you widen `SANDBOXED_API_BIND` beyond loopback.
```mermaid
flowchart TB
subgraph edge["Host — Traefik v3"]
DNS["*.preview.yourdomain.com"]
WEB["entrypoint web :80"]
SEC["entrypoint websecure :443"]
WAKE["file: sandbox-wake priority 1"]
APIR["file: sandbox-api optional"]
end
subgraph cp["sandboxd container"]
SQLITE[(SQLite WAL)]
AUTH[auth middleware]
WAKEH[wake handler]
end
subgraph sandboxes["Per-sandbox containers s-{ulid}"]
RT[runtimed + dev server]
LBL[Traefik Docker labels priority 100]
end
Browser --> DNS
DNS --> SEC
SEC --> LBL
SEC --> WAKE
WAKE --> WAKEH
WAKEH --> RT
APIR --> AUTH
AUTH --> SQLITE
LBL --> RT
```
## Wildcard DNS
Preview hostnames are literal per sandbox and port:
`s-{ulid}-{port}.preview.{PREVIEW_DOMAIN}`
For `PREVIEW_DOMAIN=yourdomain.com` and port `3000`, browsers hit `https://s-01HXANYZ-3000.preview.yourdomain.com`. Point a **wildcard** record at the host running Traefik:
| Record | Target |
|--------|--------|
| `*.preview.yourdomain.com` | Host public IP or load balancer in front of Traefik |
The wake catch-all in `traefik/dynamic/wake.yml` matches `HostRegexp(\`^s-[0-9A-Za-z]+-[0-9]+\\.preview\\..+$\`)`, so you do not edit that file when changing `PREVIEW_DOMAIN` — `sandboxd` validates the exact domain on wake.
Local dev uses `PREVIEW_DOMAIN=localhost`; browsers resolve `*.localhost` to `127.0.0.1` with no DNS or certificates. Production replaces only the domain and TLS path.
## Traefik: websecure, certificates, and dynamic routes
The shipped `traefik/traefik.yml` exposes plain HTTP on entrypoint `web` (`:80`). For production:
1. Uncomment the `websecure` entrypoint (`:443`) in `traefik/traefik.yml`.
2. Add a certificate resolver (README recommends **Let's Encrypt DNS-01** so one wildcard cert covers every preview host and you avoid per-host ACME rate limits).
3. In `docker-compose.yml`, publish `443` (uncomment `"${HTTPS_PORT:-443}:443"`).
4. Align **file-provider** routers with TLS: `traefik/dynamic/wake.yml` and `traefik/dynamic/api.yml` default to `entryPoints: [web]` — change them to `[websecure]` when previews and optional API routing use HTTPS only.
`sandboxd` does **not** set a per-router `certresolver`. Preview routers get `traefik.http.routers..tls=true` when `PREVIEW_TLS=true`; Traefik serves them from the **default TLS store** using one shared wildcard `*.preview.` certificate (no per-sandbox ACME orders). Supply that cert via your resolver output or a file under `traefik/dynamic/` (Traefik watches that directory).
Traefik's Docker provider is constrained to `Label(\`sandboxed.managed\`,\`true\`)` so only stack-owned sandboxes are routed. Running sandboxes register priority **100** routers; the wake catch-all stays at priority **1**.
## sandboxd preview and compose env
Set these in `.env` (see `.env.example`) before `docker compose up -d`:
| Variable | Production value | Effect |
|----------|------------------|--------|
| `PREVIEW_DOMAIN` | `yourdomain.com` | Host rule suffix `*.preview.yourdomain.com` |
| `PREVIEW_ENTRYPOINT` | `websecure` | Docker label `entrypoints=websecure` |
| `PREVIEW_TLS` | `true` | Docker label `tls=true` on preview routers |
| `HTTP_PORT` | `80` (optional) | Host map for Traefik `web` |
| `HTTPS_PORT` | `443` | Host map for Traefik `websecure` when uncommented in compose |
| `SANDBOXED_API_BIND` | `127.0.0.1:9090` default | Loopback API; use `0.0.0.0:9090` only with auth enabled |
| `SANDBOXD_API_AUTH_DISABLED` | `false` | Require bearer tokens on external API paths |
| `SANDBOXD_API_TOKENS` | `name1:secret1,name2:secret2` | Service-token pairs |
Compose passes `PREVIEW_DOMAIN`, `PREVIEW_ENTRYPOINT`, `PREVIEW_TLS`, and auth vars into the `sandboxd` service environment.
Point `*.preview.yourdomain.com` at the host. Enable `websecure`, add a cert resolver or mount a wildcard cert into `traefik/dynamic/`, publish port 443, and set file-provider `entryPoints` to `websecure` where needed.
```bash
PREVIEW_DOMAIN=yourdomain.com
PREVIEW_ENTRYPOINT=websecure
PREVIEW_TLS=true
SANDBOXD_API_AUTH_DISABLED=false
SANDBOXD_API_TOKENS=prod:your-long-random-secret
SANDBOXED_API_BIND=127.0.0.1:9090
```
```bash
docker compose up -d
curl -s https://s--3000.preview.yourdomain.com/
```
```bash
curl -s -XPOST http://127.0.0.1:9090/sandbox \
-H 'Authorization: Bearer your-long-random-secret' \
-H 'content-type: application/json' \
-d '{"ports":[3000]}'
```
Loopback requests without `X-Forwarded-For` bypass auth (operator path). Traefik-forwarded calls must send `Authorization: Bearer `.
## Enable API authentication
Default install keeps the API open (`SANDBOXD_API_AUTH_DISABLED=true` in `.env.example`) for local integration. Production should set `SANDBOXD_API_AUTH_DISABLED=false` and non-empty `SANDBOXD_API_TOKENS`.
Comma-separated `name:secret` pairs. The middleware matches the bearer token against secrets; the token **name** is recorded for audit.
When not `false` / `0` / `no`, every external request is treated as unauthenticated (`auth-disabled` actor). Intended as an emergency rollback only.
Exempt paths (no bearer required): `/healthz`, `/readyz`, `/preview-auth`, `/forward-auth`, `/llm.txt`. `/metrics` is **loopback-only** and returns 404 externally.
Optional edge exposure: `traefik/dynamic/api.yml` routes `api.preview.` to `sandboxd`; the same token rules apply. Delete that file to keep the API off Traefik.
Send tokens on every integrator call:
```http
Authorization: Bearer
```
If `SANDBOXD_API_AUTH_DISABLED=false` but `SANDBOXD_API_TOKENS` is empty, `sandboxd` logs a startup warning and every external API call receives `401`.
Send `SIGHUP` to `sandboxd` to reload auth config from `SANDBOXD_ENV_FILE` (default `/etc/sandboxed/sandboxd.env` in non-compose deployments) without restarting the process — token rotation path.
## Hardening checklist
v1 optimizes for a single Docker host in one command. The README and `ARCHITECTURE.md` call out what to tighten for real users and revenue.
### Isolation
Each sandbox is created with hardened `docker run` flags:
| Control | Value |
|---------|--------|
| Capabilities | `--cap-drop=ALL` |
| Privilege escalation | `--security-opt=no-new-privileges` |
| Root filesystem | `--read-only` + `tmpfs` on `/tmp` and `/var/tmp` |
| Memory | hard `--memory=10g`, `--memory-swap=10g` |
| PIDs | `--pids-limit=1024` |
| Writable disk | bind-mounted workspace only (`/home/sandbox`) |
Default `memory_high` on create is **4G** (cgroup soft throttle); applying it requires `SANDBOXED_SET_MEMORY_HIGH=true` and host cgroup visibility from the control-plane container.
Threat model: **authenticated, accountable users running their own code** — not anonymous hostile multi-tenancy. For untrusted strangers' code, README recommends **VM-per-tenant** or stronger runtimes (gVisor, Kata, Firecracker).
`SANDBOXED_USERNS` defaults to `host` so workspace ownership stays deterministic with or without daemon `userns-remap`. Clear it to use the daemon default for sandboxes.
Idle and **host memory pressure** reapers `docker stop` sandboxes to free RAM; wake admission refuses starts when host memory is low.
### Egress
OSS `sandboxd` sets `egress.Manager` to **nil** — default-allow outbound network, no connection logging. The `control-plane/internal/egress` package implements nftables `sandbox_sources_v4` membership for a future host-level policy (metadata blocks, SMTP, abuse lists, RFC1918, cross-sandbox rules per rule comments). That path requires host `nft`, journald, and systemd timers not present in plain `docker compose`.
To harden egress today: add host firewall rules or an egress proxy; treat open egress as a known v1 trade-off in `ARCHITECTURE.md`.
### Disk and data
| Asset | Location | Quota |
|-------|----------|-------|
| Workspaces | `SANDBOXED_DATA_DIR/workspaces//` | **No** per-workspace hard quota (plain directories) |
| State | `SANDBOXED_DATA_DIR/state/sandboxd.db` | Host filesystem shared |
| Logs | `SANDBOXED_LOG_DIR/traefik-access.log` | Shared access log for activity tailing |
Back up workspaces by copying directories; back up SQLite for control-plane truth. Plan filesystem or volume quotas and multi-host sharding before heavy multi-tenant load.
### Previews and host trust
| Risk | v1 behavior | Mitigation |
|------|-------------|------------|
| Public preview URLs | Anyone with the link can load the app | Create with `visibility=private` + forward-auth (`traefik/dynamic/auth.yml`) |
| Control plane power | `sandboxd` mounts the Docker socket (root-equivalent on host) | Dedicated host, patching, no unrelated secrets co-located |
| API exposure | Open by default | Auth + bind API to loopback unless Traefik + tokens protect the edge |
## Scaling boundaries
| Dimension | v1 limit | Beyond v1 |
|-----------|----------|-----------|
| Hosts | **One server**, one Docker socket | Shard by tenant or region; control plane is a thin `docker` CLI boundary (K8s noted as interface swap in README, not shipped) |
| Density | Many stopped sandboxes, fewer running; idle stop + wake | Tune `SANDBOXD_IDLE_THRESHOLD_SECONDS` (default 2100s); monitor pressure reaper |
| State | Single SQLite (WAL) on disk | Backup `sandboxd.db`; no built-in HA |
| Previews | Traefik on same host as sandboxes | Wildcard cert + DNS; optional `api.preview.*` router |
| Snapshots/templates | API present; directory storage experimental | Prefer workspace copies until snapshot backend matures |
The README's scaling summary for fast growth: (1) stronger isolation if code is untrusted, (2) **API auth + host lockdown**, (3) plan **more than one machine** — other items are configuration or operational layers, not a rewrite of the create → build → preview loop.
## Verification
| Check | Command / signal |
|-------|------------------|
| Control plane live | `curl -s http://127.0.0.1:9090/healthz` → `ok` |
| Ready for sandboxes | `curl -s http://127.0.0.1:9090/readyz` → `ready` |
| TLS preview | Browser or `curl` to `https://s--.preview./` |
| Auth enforced | API without bearer → `401` JSON `{"error":"unauthorized"}` |
| Managed sandboxes only | `docker ps --filter label=sandboxed.managed=true` |
## Related pages
Prerequisites, `install.sh`, and first health checks before going public.
Host rules, router priorities, and `PREVIEW_DOMAIN` / entrypoint behavior.
Bearer tokens, loopback exemptions, SIGHUP reload, and LAN bind guidance.
`visibility=private`, forward-auth, and preview tokens for sensitive apps.
Full `.env` keys including idle, memory, and advanced toggles.
Bind mounts, seeding, caps, and storage trade-offs.