# The Seven Backend Primitives — What Each One Does > InsForge exposes seven backend services to agents: Auth, Database (Postgres + PostgREST), Storage (S3-compatible), Model Gateway (OpenAI-compatible AI routing), Edge Functions (Deno serverless), Compute (long-running containers), and Site Deployment. This page explains what each primitive is for and how it is wired in the codebase. - Repository: InsForge/InsForge - GitHub: https://github.com/InsForge/InsForge - Human wiki: https://grok-wiki.com/public/wiki/insforge-insforge-357039661319 - Complete Markdown: https://grok-wiki.com/public/wiki/insforge-insforge-357039661319/llms-full.txt ## Source Files - `backend/src/server.ts` - `backend/src/api/routes/auth/index.routes.ts` - `backend/src/api/routes/database/tables.routes.ts` - `backend/src/api/routes/storage/index.routes.ts` - `backend/src/api/routes/ai/index.routes.ts` - `backend/src/api/routes/functions/index.routes.ts` - `backend/src/api/routes/compute/services.routes.ts` - `backend/src/api/routes/deployments/index.routes.ts` ---
Relevant source files The following files were used as context for generating this wiki page: - [backend/src/server.ts](backend/src/server.ts) - [backend/src/api/routes/auth/index.routes.ts](backend/src/api/routes/auth/index.routes.ts) - [backend/src/api/routes/database/index.routes.ts](backend/src/api/routes/database/index.routes.ts) - [backend/src/api/routes/database/tables.routes.ts](backend/src/api/routes/database/tables.routes.ts) - [backend/src/api/routes/storage/index.routes.ts](backend/src/api/routes/storage/index.routes.ts) - [backend/src/api/routes/ai/index.routes.ts](backend/src/api/routes/ai/index.routes.ts) - [backend/src/api/routes/functions/index.routes.ts](backend/src/api/routes/functions/index.routes.ts) - [backend/src/api/routes/compute/services.routes.ts](backend/src/api/routes/compute/services.routes.ts) - [backend/src/api/routes/deployments/index.routes.ts](backend/src/api/routes/deployments/index.routes.ts) - [backend/src/services/functions/function.service.ts](backend/src/services/functions/function.service.ts) - [backend/src/services/compute/services.service.ts](backend/src/services/compute/services.service.ts)
# The Seven Backend Primitives — What Each One Does InsForge is a self-hosted application platform that gives agents and developers a set of reusable backend services: identity management, a relational database with a REST query layer, S3-compatible file storage, a model-neutral AI gateway, Deno-based serverless functions, long-running containerized compute, and static site deployment. All seven are wired into a single Express server and exposed under the `/api` prefix. This page explains what each primitive does, where it lives in the codebase, and what shapes the API it exposes. It is written for developers who are new to the repo and want to understand the system before editing any of it. --- ## How the Primitives Fit Together At startup the server mounts every primitive as an Express sub-router. The ordering matters in one place: the S3 protocol gateway and the Stripe webhooks are mounted *before* the JSON body-parser so their raw bytes are never consumed. ```text ┌───────────────────── Express app (port 7130) ─────────────────────┐ │ /storage/v1/s3 ← S3 gateway (raw stream, no JSON parse) │ │ /api/webhooks ← Stripe webhooks (raw body) │ │ /api/auth ← Auth primitive │ │ /api/database ← Database primitive │ │ /api/storage ← Storage primitive │ │ /api/ai ← Model Gateway primitive │ │ /api/functions ← Edge Functions primitive │ │ /api/compute/services ← Compute primitive │ │ /api/deployments ← Site Deployment primitive │ │ /functions/:slug ← Backwards-compat proxy to Deno runtime │ └───────────────────────────────────────────────────────────────────┘ ``` Sources: [backend/src/server.ts:174-221]() --- ## 1. Auth **What it does.** Auth manages users, sessions, OAuth, email verification, and password resets. Every other primitive checks tokens minted here before allowing writes. **How it works.** The service keeps user records in a PostgreSQL database. When someone logs in, it issues a short-lived JWT *access token* and a longer-lived *refresh token*. Web clients receive the refresh token in an `httpOnly` cookie plus a CSRF nonce; mobile, desktop, and server clients receive it in the response body — the code branches explicitly on a `client_type` query parameter. ``` POST /api/auth/sessions → login (email + password → access + refresh tokens) POST /api/auth/users → registration POST /api/auth/refresh → rotate refresh token → new access token POST /api/auth/logout → clear cookie (web) / discard token (non-web) GET /api/auth/sessions/current → who am I? POST /api/auth/email/send-verification → send OTP or magic link POST /api/auth/email/verify → redeem code GET /api/auth/email/verify-link → browser click → redirect POST /api/auth/email/reset-password → apply new password POST /api/auth/id-token → sign in with Google One Tap ID token GET /api/auth/oauth/* → OAuth 2.0 flows (GitHub, Google, custom providers) ``` Admin-only routes (guarded by `verifyAdmin`) let you list, create, or delete users and update auth config, SMTP settings, and email templates. Sources: [backend/src/api/routes/auth/index.routes.ts:428-465](), [backend/src/api/routes/auth/index.routes.ts:522-619]() ### Token flavors | Token | Transport | Audience | |-------|-----------|----------| | Access JWT | `Authorization: Bearer` header | All clients | | Refresh token + CSRF nonce | `httpOnly` cookie + `X-CSRF-Token` header | Web | | Refresh token | Response body | Mobile / desktop / server | | Anonymous JWT (never expires) | Response body | Public read-only callers | Sources: [backend/src/api/routes/auth/index.routes.ts:393-423](), [backend/src/api/routes/auth/index.routes.ts:771-783]() --- ## 2. Database (Postgres + PostgREST-style layer) **What it does.** The Database primitive provides a managed PostgreSQL database with a REST query API. Agents and application code can read and write rows, call stored procedures, and manage schema — all over HTTP, without writing raw SQL. **How it works.** The backend holds a connection pool (`DatabaseManager`) to the project's Postgres instance. The `/api/database` router has six sub-routers: | Sub-route | Handles | |-----------|---------| | `/tables` | Create / inspect / patch / delete table schemas | | `/records` | CRUD on individual rows (PostgREST-style query strings) | | `/rpc` | Call Postgres stored procedures | | `/advance` | Raw SQL execution for power users | | `/migrations` | Apply and track migrations | | `/admin` | Extensions, role grants, etc. | Top-level paths on the router expose schemas, SQL functions, indexes, RLS policies, and triggers as read endpoints. ``` GET /api/database/schemas → list schemas GET /api/database/tables?schema=public → list tables POST /api/database/tables → create table GET /api/database/tables/:name/schema → introspect columns PATCH /api/database/tables/:name/schema → add/rename/drop columns POST /api/database/records/:table → insert row GET /api/database/records/:table → filtered select POST /api/database/rpc/:function → call stored proc ``` Every write flushes a `clearColumnTypeCache` call so the type-coercion layer inside `DatabaseManager` picks up schema changes immediately. Sources: [backend/src/api/routes/database/index.routes.ts:17-23](), [backend/src/api/routes/database/tables.routes.ts:31-67](), [backend/src/api/routes/database/tables.routes.ts:107-126]() ### Row-Level Security Tables can be created with `rlsEnabled: true`. When a JWT caller queries `/api/database/records`, the backend uses their token's role claim to evaluate RLS policies. API-key callers use the backend's privileged pool and bypass user-level RLS. Sources: [backend/src/api/routes/database/tables.routes.ts:44-45]() --- ## 3. Storage (S3-compatible) **What it does.** Storage gives agents a place to put files — images, PDFs, user uploads, build artifacts — organized into named *buckets*. It presents an InsForge-native REST API **and** a full AWS S3 protocol endpoint at `/storage/v1/s3`, so any S3 SDK works out of the box. **How it works.** Buckets are either public (no auth needed to download) or private. The `conditionalDownloadAuth` middleware checks bucket visibility before deciding whether to require a token on GET requests. When an external S3 provider is configured, large uploads can bypass the backend entirely: the client calls `POST /upload-strategy` first, receives a presigned URL, uploads directly to S3, then confirms with `POST /confirm-upload`. For local/default storage, the backend streams the bytes itself. ``` POST /api/storage/buckets → create bucket GET /api/storage/buckets/:name/objects → list objects PUT /api/storage/buckets/:name/objects/* → upload (specific key) POST /api/storage/buckets/:name/objects → upload (server-generated key) GET /api/storage/buckets/:name/objects/* → download (auth conditional) DELETE /api/storage/buckets/:name/objects/* → delete object POST /api/storage/buckets/:name/upload-strategy → get presigned PUT URL POST /api/storage/buckets/:name/objects/:key/confirm-upload → confirm presigned upload POST /api/storage/buckets/:name/objects/:key/download-strategy → get presigned GET URL GET /api/storage/s3/config → S3 endpoint + region for SDK config POST /api/storage/s3/access-keys → create long-lived S3 credential GET /api/storage/s3/access-keys → list keys (no secrets) DELETE /api/storage/s3/access-keys/:id → revoke key ``` The S3 gateway is mounted before the JSON middleware so chunked `STREAMING-AWS4-HMAC-SHA256-PAYLOAD` requests stream through untouched. Sources: [backend/src/server.ts:178-179](), [backend/src/api/routes/storage/index.routes.ts:29-46](), [backend/src/api/routes/storage/index.routes.ts:666-680](), [backend/src/api/routes/storage/index.routes.ts:682-710]() --- ## 4. Model Gateway (OpenAI-compatible AI routing) **What it does.** The Model Gateway lets agents send chat, embedding, and image-generation requests to any supported model. The gateway presents a single stable API surface regardless of which underlying provider is used — today that is OpenRouter, but the provider is isolated behind an interface. **How it works.** `ChatCompletionService`, `EmbeddingService`, and `ImageGenerationService` are singletons that delegate to provider implementations. For streaming chat completions, the router opens a Server-Sent Events (SSE) response and forwards chunks, token usage, tool calls, and annotations one frame at a time. ``` GET /api/ai/models → list available models (admin) GET /api/ai/overview → key-level usage overview from OpenRouter (admin) GET /api/ai/:provider/api-key → masked API key (admin) POST /api/ai/chat/completion → chat (streaming SSE or single JSON) POST /api/ai/image/generation → image generation POST /api/ai/embeddings → text embeddings ``` `verifyUser` (not `verifyAdmin`) guards the inference endpoints, so any authenticated user in the project can call them. ### Streaming protocol ``` POST /api/ai/chat/completion { stream: true, messages: [...] } SSE frames: data: {"chunk": "Hello"} data: {"chunk": " world"} data: {"tokenUsage": {"promptTokens": 10, "completionTokens": 4}} data: {"done": true} ``` Sources: [backend/src/api/routes/ai/index.routes.ts:19-20](), [backend/src/api/routes/ai/index.routes.ts:127-170](), [backend/src/api/routes/ai/index.routes.ts:195-264]() --- ## 5. Edge Functions (Deno serverless) **What it does.** Edge Functions let you deploy short-lived Deno TypeScript code that runs on a request. Think of them as HTTP endpoints you write and deploy without provisioning a server. **How it works.** `FunctionService` stores function metadata and source code in the `functions.definitions` Postgres table. At runtime, function invocations are proxied to whichever Deno runtime is available: 1. **Deno Subhosting** — if `DENO_SUBHOSTING_ORG_ID` and related vars are set, functions are synced to Deno's managed cloud runtime and invoked there. 2. **Local Deno runtime** — otherwise, the backend proxies to `http://localhost:7133` (configurable via `DENO_RUNTIME_URL`). The management API lives at `/api/functions`; execution calls reach the function directly at `/functions/:slug` (a backwards-compatible catch-all on the Express root). ``` GET /api/functions → list functions + runtime health GET /api/functions/:slug → get function code + metadata POST /api/functions → create function (deploys immediately) PUT /api/functions/:slug → update code (redeploys) DELETE /api/functions/:slug → delete + undeploy ALL /functions/:slug → proxy execution to Deno runtime ``` On startup, `functionService.syncDeployment()` runs in the background (non-blocking) to push any locally stored functions to Subhosting if configured. Sources: [backend/src/server.ts:228-286](), [backend/src/server.ts:339-345](), [backend/src/services/functions/function.service.ts:52-80](), [backend/src/api/routes/functions/index.routes.ts:55-105]() --- ## 6. Compute (long-running containers) **What it does.** Compute lets you run Docker containers that stay alive between requests — background workers, databases, long-running ML jobs, or any process that doesn't fit in a serverless function's execution window. **How it works.** `ComputeServicesService` wraps a `ComputeProvider` interface. In self-hosted mode the backing provider is `FlyProvider` (Fly.io machines); in cloud-managed mode `CloudComputeProvider` is used. This abstraction means the route layer stays the same regardless of where containers actually run. Services are identified by `projectId`, which is read from the JWT claim (`req.projectId`) in cloud mode or from the `PROJECT_ID` environment variable in self-hosted mode. ``` GET /api/compute/services → list services GET /api/compute/services/:id → get service details POST /api/compute/services → create + immediately launch container POST /api/compute/services/deploy → prepare (create app, no machine yet) PATCH /api/compute/services/:id → update image / env vars / resources DELETE /api/compute/services/:id → delete + destroy container POST /api/compute/services/:id/start → start a stopped service POST /api/compute/services/:id/stop → stop a running service GET /api/compute/services/:id/events → lifecycle events (start/stop/exit) POST /api/compute/services/:id/deploy-token → issue a Fly deploy token for the CLI ``` The `deploy-token` endpoint exists for the two-step CLI deploy path: the CLI issues `flyctl` commands using a short-lived token minted here rather than requiring the user to hold their own `FLY_API_TOKEN`. Env vars are stored encrypted in the database. The audit log records only the *keys* that changed, never the values. Sources: [backend/src/api/routes/compute/services.routes.ts:17-21](), [backend/src/api/routes/compute/services.routes.ts:107-141](), [backend/src/api/routes/compute/services.routes.ts:143-165](), [backend/src/services/compute/services.service.ts:1-10]() --- ## 7. Site Deployment **What it does.** Site Deployment publishes static frontends (HTML/JS/CSS) to a CDN-backed hosting provider, manages custom domains, and tracks deployment history. The result is a live URL for each deployment. **How it works.** `DeploymentService` wraps a provider (currently Vercel). Deployments follow one of two flows: **Legacy zip flow** 1. `POST /api/deployments` — create a deployment record (status: `WAITING`), receive presigned S3 upload info. 2. Upload source zip to S3. 3. `POST /api/deployments/:id/start` — trigger the build. **Direct-upload flow** (used by the dashboard and CLI) 1. `POST /api/deployments/direct` — create a record with a manifest of files. 2. `PUT /api/deployments/:id/files/:fileId/content` — stream each file's bytes through the backend to Vercel. This endpoint is intentionally not rate-limited because one `POST /direct` call may fan out to many file uploads. 3. `POST /api/deployments/:id/start` — trigger the deployment. ``` POST /api/deployments → create deployment (zip flow) POST /api/deployments/direct → create deployment (direct flow) PUT /api/deployments/:id/files/:fileId/content → upload one file POST /api/deployments/:id/start → trigger build GET /api/deployments → list deployments GET /api/deployments/:id → get deployment POST /api/deployments/:id/sync → pull latest status from Vercel POST /api/deployments/:id/cancel → cancel in-progress build GET /api/deployments/metadata → current live deployment + domain URLs PUT /api/deployments/slug → set/change project slug GET /api/deployments/domains → list custom domains POST /api/deployments/domains → add custom domain POST /api/deployments/domains/:domain/verify → verify DNS config DELETE /api/deployments/domains/:domain → remove domain ``` Sources: [backend/src/api/routes/deployments/index.routes.ts:30-90](), [backend/src/api/routes/deployments/index.routes.ts:96-150](), [backend/src/api/routes/deployments/index.routes.ts:156-193]() --- ## Putting it All Together ```mermaid flowchart TD Client["Agent / Browser / CLI"] subgraph Express["Express Server (port 7130)"] Auth["/api/auth\nAuth"] DB["/api/database\nDatabase"] Stor["/api/storage\nStorage"] AI["/api/ai\nModel Gateway"] Fn["/api/functions\nEdge Functions"] Compute["/api/compute/services\nCompute"] Deploy["/api/deployments\nSite Deployment"] end subgraph Infra["Infrastructure / Providers"] PG["PostgreSQL"] DenoRT["Deno Runtime\n(local or Subhosting)"] Fly["Fly.io / CloudComputeProvider"] Vercel["Vercel"] S3["S3 / Local filesystem"] OR["OpenRouter (AI models)"] end Client --> Auth Client --> DB Client --> Stor Client --> AI Client --> Fn Client --> Compute Client --> Deploy Auth --> PG DB --> PG Stor --> S3 AI --> OR Fn --> DenoRT Compute --> Fly Deploy --> Vercel ``` All seven primitives share a single authentication layer — the `verifyToken` / `verifyAdmin` middleware from `backend/src/api/middlewares/auth.ts` — so a token issued by Auth is accepted by every other primitive without additional integration work. Every write operation emits an audit log entry and, for admin-facing resources, a Socket.IO broadcast to update connected dashboards in real time. Sources: [backend/src/server.ts:204-222](), [backend/src/api/routes/compute/services.routes.ts:23-41]()