# BYOK / BYOC: What Does Provider Neutrality Actually Cost?

> Pi promises that users bring their own keys and providers. This page probes what that claim requires in practice: how env-api-keys.ts discovers credentials, how the OAuth flow is handled, how the add-llm-provider skill teaches the agent to register new providers at runtime, and what invariants must hold for every provider adapter. The skill file .pi/skills/add-llm-provider.md is the primary non-README evidence.

- Repository: earendil-works/pi
- GitHub: https://github.com/earendil-works/pi
- Human wiki: https://grok-wiki.com/public/wiki/earendil-works-pi-8b87608fc234
- Complete Markdown: https://grok-wiki.com/public/wiki/earendil-works-pi-8b87608fc234/llms-full.txt

## Source Files

- `packages/ai/src/env-api-keys.ts`
- `packages/ai/src/oauth.ts`
- `.pi/skills/add-llm-provider.md`
- `packages/coding-agent/src/core/auth-storage.ts`
- `packages/coding-agent/src/core/auth-guidance.ts`

---

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:

- [packages/ai/src/env-api-keys.ts](packages/ai/src/env-api-keys.ts)
- [packages/ai/src/oauth.ts](packages/ai/src/oauth.ts)
- [packages/ai/src/utils/oauth/index.ts](packages/ai/src/utils/oauth/index.ts)
- [packages/ai/src/utils/oauth/types.ts](packages/ai/src/utils/oauth/types.ts)
- [packages/ai/src/api-registry.ts](packages/ai/src/api-registry.ts)
- [packages/ai/src/providers/register-builtins.ts](packages/ai/src/providers/register-builtins.ts)
- [packages/coding-agent/src/core/auth-storage.ts](packages/coding-agent/src/core/auth-storage.ts)
- [packages/coding-agent/src/core/auth-guidance.ts](packages/coding-agent/src/core/auth-guidance.ts)
- [packages/coding-agent/src/core/resolve-config-value.ts](packages/coding-agent/src/core/resolve-config-value.ts)
- [.pi/skills/add-llm-provider.md](.pi/skills/add-llm-provider.md)
</details>

# BYOK / BYOC: What Does Provider Neutrality Actually Cost?

Pi makes a strong architectural promise: you bring your own keys and your own provider. No vendor lock-in, no proxied credentials, no forced model choice. But a claim like that only holds if every part of the stack — credential discovery, token lifecycle, provider dispatch, and the extension contract — actually enforces it uniformly. This page probes what provider neutrality costs in practice: the exact credential-resolution chain, how OAuth fits into it, what the `add-llm-provider` skill requires of any new adapter, and which invariants must hold at each boundary.

This matters because BYOK/BYOC is not a UI toggle; it is an engineering commitment that runs from a handful of environment-variable lookups all the way to a locked JSON file on disk. Understanding each layer reveals where the system is genuinely open and where hidden constraints encode provider-specific assumptions.

---

## What is the simplest version of BYOK?

The floor-level design is just `process.env`. Every API-key provider has one canonical environment variable (e.g. `OPENAI_API_KEY`, `GEMINI_API_KEY`), and if that variable is set the system uses it. No login flow, no stored file, no agent aware of it.

`env-api-keys.ts` encodes this mapping as a plain string dictionary:

```ts
// packages/ai/src/env-api-keys.ts:101-131
const envMap: Record<string, string> = {
  openai: "OPENAI_API_KEY",
  "azure-openai-responses": "AZURE_OPENAI_API_KEY",
  deepseek: "DEEPSEEK_API_KEY",
  google: "GEMINI_API_KEY",
  "google-vertex": "GOOGLE_CLOUD_API_KEY",
  groq: "GROQ_API_KEY",
  // ...26 providers total
};
```

`findEnvKeys(provider)` returns the variable names that are actually populated; `getEnvApiKey(provider)` returns the first live value. Both are pure reads — no side effects, no caching between calls for standard providers.

Sources: [packages/ai/src/env-api-keys.ts:91-151]()

---

## Where does the simple version break down?

Two categories of providers cannot be reduced to a single environment variable:

**1. Ambient credential providers (Vertex AI, Amazon Bedrock)**

Neither expects a bare API key. Vertex requires `GOOGLE_APPLICATION_CREDENTIALS` (or the `~/.config/gcloud/application_default_credentials.json` fallback) *plus* `GOOGLE_CLOUD_PROJECT` and `GOOGLE_CLOUD_LOCATION`. Bedrock accepts six distinct AWS credential forms (profile, IAM keys, bearer token, ECS task roles, IRSA). Both return the sentinel string `"<authenticated>"` when all preconditions are met, signalling "auth exists" without exposing a real secret.

```ts
// packages/ai/src/env-api-keys.ts:167-207
if (provider === "google-vertex") {
  const hasCredentials = hasVertexAdcCredentials();
  const hasProject = !!( process.env.GOOGLE_CLOUD_PROJECT || ... );
  const hasLocation = !!( process.env.GOOGLE_CLOUD_LOCATION || ... );
  if (hasCredentials && hasProject && hasLocation) {
    return "<authenticated>";
  }
}
```

**2. OAuth providers (Anthropic, GitHub Copilot, OpenAI Codex)**

These cannot be satisfied by any environment variable. They require an interactive login that writes `refresh`/`access`/`expires` credentials to `auth.json`. The OAuth credential surface is materially different from a static API key.

Sources: [packages/ai/src/env-api-keys.ts:61-210]()

---

## The five-source credential priority chain

`AuthStorage.getApiKey()` in `auth-storage.ts` defines the full resolution order. Reading it from top to bottom is the only way to understand which credential wins when multiple sources are present:

```
1. Runtime override     --api-key CLI flag (in-memory, not persisted)
2. Stored api_key       auth.json, type: "api_key"
3. Stored oauth         auth.json, type: "oauth" (auto-refreshed with file lock)
4. Environment variable process.env / /proc/self/environ fallback
5. Fallback resolver    models.json custom providers, injected by setFallbackResolver()
```

```ts
// packages/coding-agent/src/core/auth-storage.ts:462-522
async getApiKey(providerId: string, options?: { includeFallback?: boolean }): Promise<string | undefined> {
  const runtimeKey = this.runtimeOverrides.get(providerId);
  if (runtimeKey) return runtimeKey;                    // 1

  const cred = this.data[providerId];
  if (cred?.type === "api_key") return resolveConfigValue(cred.key);  // 2

  if (cred?.type === "oauth") { /* refresh logic */ }   // 3

  const envKey = getEnvApiKey(providerId);
  if (envKey) return envKey;                            // 4

  return this.fallbackResolver?.(providerId) ?? undefined;  // 5
}
```

The `source` field on `AuthStatus` mirrors this: `"stored"`, `"runtime"`, `"environment"`, `"fallback"`, or `"models_json_key"` / `"models_json_command"`.

Sources: [packages/coding-agent/src/core/auth-storage.ts:349-368, 462-522]()

---

## Secret resolution beyond raw strings

Step 2 above passes `cred.key` through `resolveConfigValue()` before returning it. This is non-obvious: an API key stored in `auth.json` is not necessarily a literal string. `resolveConfigValue` supports three forms:

| Value prefix | Resolution |
|---|---|
| `!` | Execute the rest as a shell command, return trimmed stdout (cached per process) |
| `$VAR` (no prefix) | Check `process.env[config]` first, fall back to literal |
| Anything else | Literal string |

This means `auth.json` can store `"!op read op://vault/key"` or `"MY_CUSTOM_ENV_VAR"` as a key, and the agent will resolve them at call time. The same logic applies to custom HTTP headers via `resolveHeaders()`.

Sources: [packages/coding-agent/src/core/resolve-config-value.ts:17-23]()

---

## The OAuth flow: a deeper contract

Three providers (Anthropic, GitHub Copilot, OpenAI Codex) go through an OAuth dance rather than a bare API key exchange. The flow is defined by the `OAuthProviderInterface`:

```ts
// packages/ai/src/utils/oauth/types.ts:54-72
export interface OAuthProviderInterface {
  readonly id: OAuthProviderId;
  readonly name: string;

  login(callbacks: OAuthLoginCallbacks): Promise<OAuthCredentials>;

  refreshToken(credentials: OAuthCredentials): Promise<OAuthCredentials>;

  getApiKey(credentials: OAuthCredentials): string;

  modifyModels?(models: Model<Api>[], credentials: OAuthCredentials): Model<Api>[];
}
```

The `OAuthLoginCallbacks` interface is the boundary between the auth library and the CLI/UI layer. It provides hooks for browser-redirect (`onAuth`), device-code display (`onDeviceCode`), text prompts (`onPrompt`), and interactive selectors (`onSelect`). The underlying transport — PKCE, device-code flow — is an implementation detail of each provider module.

After login, credentials are stored in `auth.json` as `{ type: "oauth", refresh, access, expires, ...provider-specific }`. When the access token expires, `AuthStorage.refreshOAuthTokenWithLock()` re-enters the file with a `proper-lockfile` advisory lock to prevent race conditions when multiple Pi processes run concurrently:

```ts
// packages/coding-agent/src/core/auth-storage.ts:407-450
private async refreshOAuthTokenWithLock(providerId) {
  return await this.storage.withLockAsync(async (current) => {
    // re-read file under lock
    // if already refreshed by another process, use those creds
    // otherwise call provider.refreshToken(), persist, return
  });
}
```

Sources: [packages/ai/src/utils/oauth/types.ts:43-72](), [packages/coding-agent/src/core/auth-storage.ts:407-450]()

---

## The provider dispatch layer: `Api` vs `Provider`

A key design tension: credentials are keyed by `provider` string, but the stream dispatch is keyed by `api` string. A single `provider` (e.g. `"openai"`) may map to multiple `api` protocols (`"openai-completions"`, `"openai-responses"`). The `api-registry.ts` stores a map from `Api` to stream functions:

```ts
// packages/ai/src/api-registry.ts:40
const apiProviderRegistry = new Map<string, RegisteredApiProvider>();
```

Registration happens at module load time in `register-builtins.ts`:

```ts
// packages/ai/src/providers/register-builtins.ts:406
registerBuiltInApiProviders();
```

Every registration wraps the stream function with a type guard that throws on `api` mismatch, so a Gemini model cannot accidentally be dispatched through the Anthropic adapter. Provider modules are loaded lazily via dynamic `import()` to keep the browser/Vite bundle clean and avoid loading AWS SDKs in a browser context:

```ts
// packages/ai/src/providers/register-builtins.ts:89-92
const importNodeOnlyProvider = (specifier: string): Promise<unknown> => {
  const runtimeSpecifier = import.meta.url.endsWith(".js") ? specifier.replace(/\.ts$/, ".js") : specifier;
  return import(runtimeSpecifier);
};
```

Sources: [packages/ai/src/api-registry.ts:40-98](), [packages/ai/src/providers/register-builtins.ts:89-92, 345-406]()

---

## Architecture: how the layers compose

```text
┌─────────────────────────────────────────────────────────────┐
│  CLI / UI layer                                             │
│  --api-key, /login, /model                                  │
└──────────────────┬──────────────────────────────────────────┘
                   │  AuthStorage.getApiKey(providerId)
┌──────────────────▼──────────────────────────────────────────┐
│  auth-storage.ts  (coding-agent)                            │
│  Priority chain: runtime → stored api_key → stored oauth    │
│                → env var → fallback resolver                 │
│  auth.json  ← FileAuthStorageBackend (proper-lockfile)      │
└──────────────────┬──────────────────────────────────────────┘
                   │  getEnvApiKey / findEnvKeys
┌──────────────────▼──────────────────────────────────────────┐
│  env-api-keys.ts  (packages/ai)                             │
│  envMap[provider] → env var name                            │
│  Vertex ADC: file-based credential detection                │
│  Bedrock: six AWS credential forms                          │
└──────────────────┬──────────────────────────────────────────┘
                   │  resolveConfigValue(key)
┌──────────────────▼──────────────────────────────────────────┐
│  resolve-config-value.ts  (coding-agent)                    │
│  "!cmd" → shell exec (cached)                               │
│  "VAR" → process.env first, then literal                    │
└─────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────┐
│  OAuth layer  (packages/ai/src/utils/oauth/)                │
│  OAuthProviderInterface: login / refreshToken / getApiKey   │
│  Built-ins: anthropic, github-copilot, openai-codex         │
│  registerOAuthProvider() / unregisterOAuthProvider()        │
└─────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────┐
│  API dispatch layer  (packages/ai/src/)                     │
│  api-registry: Map<Api, {stream, streamSimple}>             │
│  register-builtins: lazy dynamic import per provider        │
│  Keyed by Api protocol, not by provider name                │
└─────────────────────────────────────────────────────────────┘
```

---

## What the `add-llm-provider` skill requires: the real cost

The `.pi/skills/add-llm-provider.md` skill file is the clearest statement of what provider neutrality actually demands from each new adapter. It specifies seven distinct touch-points across the codebase, not one or two:

| Step | File(s) | What must be done |
|---|---|---|
| 1 | `packages/ai/src/types.ts` | Add to `Api` union, create options interface, add to `ApiOptionsMap`, add to `KnownProvider` |
| 2 | `packages/ai/src/providers/<name>.ts` | Implement `stream()`, `streamSimple()`, message/tool conversion, standardized event emission |
| 3 | `packages/ai/package.json`, `src/index.ts`, `register-builtins.ts`, `env-api-keys.ts` | Subpath export, lazy registration, credential detection |
| 4 | `packages/ai/scripts/generate-models.ts` | Fetch/parse models, map to `Model` interface |
| 5 | `packages/ai/test/` | Full test matrix: stream, tokens, abort, empty, context overflow, unicode, tool calls, cross-provider handoff |
| 6 | `packages/coding-agent/` | Default model ID, display name, CLI arg docs, README, providers.md |
| 7 | `packages/ai/README.md`, `CHANGELOG.md` | Provider table entry, auth docs, env vars |

The test requirement in step 5 is deliberately exhaustive: the `cross-provider-handoff.test.ts` requirement means every new provider must prove it can receive a conversation that started with a different provider. This is the mechanical guarantee that the "bring your own" claim holds at the protocol level, not just the auth level.

Sources: [.pi/skills/add-llm-provider.md:1-58]()

---

## Invariants every adapter must uphold

Synthesizing what the code enforces:

1. **Standardized event stream.** Every `stream()` implementation must emit typed events: `text`, `tool_call`, `thinking`, `usage`, `stop`, `error`. The registry dispatches blindly; callers never know the underlying SDK.

2. **`api` key identity.** A provider module is registered under exactly one `api` string and dispatched only for models whose `model.api` matches. Mismatches throw at the `wrapStream` boundary.

3. **Credential isolation.** Auth is keyed by `provider` string; stream dispatch is keyed by `api` string. A provider can share an `api` protocol with another (e.g., multiple providers using `"openai-completions"`) without credential bleed, because `getApiKey` receives the `provider` string, not the `api` string.

4. **No top-level Node imports in `env-api-keys.ts`.** The comment at line 1 is explicit: `// NEVER convert to top-level imports - breaks browser/Vite builds`. This makes the credential-detection layer safe in browser bundles.

5. **Locked writes for OAuth refresh.** Any provider using OAuth must tolerate concurrent process refresh attempts. The `proper-lockfile` advisory lock in `FileAuthStorageBackend.withLockAsync` serializes these; a provider that bypasses `AuthStorage` and writes `auth.json` directly would break this invariant.

Sources: [packages/ai/src/env-api-keys.ts:1-4](), [packages/ai/src/api-registry.ts:42-78](), [packages/coding-agent/src/core/auth-storage.ts:122-170]()

---

## What is closed: the static `KnownProvider` type

One place where BYOK/BYOC has a real seam: `KnownProvider` in `types.ts` is a static TypeScript union. Adding a new provider means a code change and rebuild; there is no dynamic registration path that allows a fully external adapter to be loaded without touching the core package. The `Provider` type is `KnownProvider | string`, so the credential and dispatch layers will accept arbitrary strings at runtime, but the type system treats unknown providers as opaque. The `add-llm-provider` skill exists precisely because there is no plugin manifest that bypasses the type union — extensibility is a code contribution, not a configuration contribution.

The fallback resolver (`setFallbackResolver`) and `models.json` provide a partial escape hatch for custom API endpoints that share an existing `api` protocol (e.g., an OpenAI-compatible endpoint at a different base URL), but they do not add new protocol implementations.

Sources: [.pi/skills/add-llm-provider.md:10-15](), [packages/coding-agent/src/core/auth-storage.ts:238-244]()

---

## Summary

Provider neutrality in Pi is real but structural: it holds because each layer — env-var discovery, OAuth token lifecycle, file-locked credential storage, lazy API dispatch, and a mandatory test matrix — enforces the same interface regardless of which provider sits behind it. The actual cost is the seven-step checklist in `.pi/skills/add-llm-provider.md`, where the heaviest line items are the standardized event stream, the `KnownProvider`/`Api` type additions, and the cross-provider-handoff test. The system's openness has one clear limit: there is no runtime plugin protocol; new providers require source changes to `packages/ai`. Everything else — auth sources, credential resolution (including shell commands via `!`-prefix), OAuth provider registration, and API dispatch — is genuinely pluggable at runtime via the registries in `api-registry.ts` and `oauth/index.ts`.

Sources: [packages/ai/src/utils/oauth/index.ts:35-89]()