# The Registry: How Does a New Provider Become Callable? > pi-ai uses a runtime registry pattern: providers are registered by API string key, and the agent loop calls them through that registry without knowing which concrete module it will hit. This page traces registerApiProvider, registerBuiltInApiProviders, and the lazy-load pattern in register-builtins.ts that defers heavy provider modules until first use—exposing the tradeoff between startup time and call overhead. - Repository: earendil-works/pi - GitHub: https://github.com/earendil-works/pi - Human wiki: https://grok-wiki.com/public/wiki/earendil-works-pi-8b87608fc234 - Complete Markdown: https://grok-wiki.com/public/wiki/earendil-works-pi-8b87608fc234/llms-full.txt ## Source Files - `packages/ai/src/api-registry.ts` - `packages/ai/src/providers/register-builtins.ts` - `packages/ai/src/types.ts` - `packages/ai/src/models.ts` - `packages/ai/src/index.ts` ---
Relevant source files The following files were used as context for generating this wiki page: - [packages/ai/src/api-registry.ts](packages/ai/src/api-registry.ts) - [packages/ai/src/providers/register-builtins.ts](packages/ai/src/providers/register-builtins.ts) - [packages/ai/src/types.ts](packages/ai/src/types.ts) - [packages/ai/src/models.ts](packages/ai/src/models.ts) - [packages/ai/src/index.ts](packages/ai/src/index.ts) - [packages/ai/src/stream.ts](packages/ai/src/stream.ts) - [packages/ai/src/providers/faux.ts](packages/ai/src/providers/faux.ts) - [packages/ai/src/utils/event-stream.ts](packages/ai/src/utils/event-stream.ts) - [packages/coding-agent/src/core/model-registry.ts](packages/coding-agent/src/core/model-registry.ts) - [packages/coding-agent/examples/extensions/custom-provider-gitlab-duo/test.ts](packages/coding-agent/examples/extensions/custom-provider-gitlab-duo/test.ts)
# The Registry: How Does a New Provider Become Callable? The `@earendil-works/pi-ai` package separates *what model to call* from *how to call it*. Models carry a plain string `api` field (e.g. `"anthropic-messages"`, `"openai-responses"`). At call time, the `stream()` and `streamSimple()` entry points look that string up in a runtime `Map` to find the concrete streaming implementation. Nothing in the call path imports provider modules directly — every provider reaches the agent loop through one registry lookup. This indirection is what makes the library provider-neutral. A custom provider (GitLab Duo, a local proxy, a test double) becomes callable by calling `registerApiProvider` with a matching `api` string. The agent loop does not need to change. This page traces the full lifecycle: how the registry is structured, how built-in providers self-register lazily, the precise cost of that laziness, and how third-party and test providers plug in via the same path. --- ## The Registry Data Structure The registry is a module-level `Map` in `api-registry.ts`: ```ts // packages/ai/src/api-registry.ts:40 const apiProviderRegistry = new Map(); ``` Each entry stores the provider's internal representation and an optional `sourceId` used for bulk removal: ```ts // packages/ai/src/api-registry.ts:35-38 type RegisteredApiProvider = { provider: ApiProviderInternal; sourceId?: string; }; ``` `ApiProviderInternal` normalises the two stream functions to a common signature `(model, context, options?) => AssistantMessageEventStream`, erasing provider-specific option types behind the registry boundary. The Map key is the `api` string — a plain `string` value from the `Api` type union. This is the entire "address" of a provider at runtime. A `Model` carries `api: TApi`; that value is the lookup key. Sources: [packages/ai/src/api-registry.ts:35-45]() --- ## `registerApiProvider`: The One Entry Point ```ts // packages/ai/src/api-registry.ts:66-78 export function registerApiProvider( provider: ApiProvider, sourceId?: string, ): void { apiProviderRegistry.set(provider.api, { provider: { api: provider.api, stream: wrapStream(provider.api, provider.stream), streamSimple: wrapStreamSimple(provider.api, provider.streamSimple), }, sourceId, }); } ``` Two things happen here that are easy to overlook: **1. Type erasure via `wrapStream`.** Each `ApiProvider` carries strongly-typed stream functions. The registry stores `ApiStreamFunction`, a uniformly-typed wrapper. `wrapStream` closes over the provider's `api` key and validates that the model handed to the function at call time has a matching `api`; if not, it throws immediately rather than making a confused upstream call. ```ts // packages/ai/src/api-registry.ts:42-52 function wrapStream( api: TApi, stream: StreamFunction, ): ApiStreamFunction { return (model, context, options) => { if (model.api !== api) { throw new Error(`Mismatched api: ${model.api} expected ${api}`); } return stream(model as Model, context, options as TOptions); }; } ``` **2. Map semantics imply last-write-wins.** Registering a provider for an already-registered `api` key silently replaces the old entry. This is how `resetApiProviders` works: it calls `clearApiProviders()` then re-registers all built-ins, returning the registry to a known state. Sources: [packages/ai/src/api-registry.ts:42-78]() --- ## How the Agent Loop Calls Providers `stream.ts` is the single file callers interact with. It imports `register-builtins.ts` as a side effect, then delegates every call through the registry: ```ts // packages/ai/src/stream.ts:1-3 import "./providers/register-builtins.ts"; import { getApiProvider } from "./api-registry.ts"; ``` ```ts // packages/ai/src/stream.ts:17-32 function resolveApiProvider(api: Api) { const provider = getApiProvider(api); if (!provider) { throw new Error(`No API provider registered for api: ${api}`); } return provider; } export function stream( model: Model, context: Context, options?: ProviderStreamOptions, ): AssistantMessageEventStream { const provider = resolveApiProvider(model.api); return provider.stream(model, context, options as StreamOptions); } ``` `resolveApiProvider` does one `Map.get`. It never imports Anthropic, OpenAI, Google, or any other SDK module. That decoupling is intentional: callers import `stream` from `@earendil-works/pi-ai`; the concrete provider code only loads when the lazy `import()` inside `register-builtins.ts` resolves. Sources: [packages/ai/src/stream.ts:1-59]() --- ## The Lazy-Load Pattern in `register-builtins.ts` ### What problem does it solve? A naive implementation would `import` every provider SDK at module load time. Bedrock alone pulls in `@aws-sdk/client-bedrock-runtime`. If the caller only ever uses Anthropic, loading the AWS SDK wastes startup time and memory. The lazy pattern defers each provider module's import until the first call that actually needs it. ### How it works Each provider gets a module-level `Promise` slot, initialised to `undefined`: ```ts // packages/ai/src/providers/register-builtins.ts:94-123 let anthropicProviderModulePromise: | Promise> | undefined; // ... (one per provider) ``` A `load*` function populates that slot on first call, using the `||=` idiom to ensure the import fires exactly once regardless of concurrent calls: ```ts // packages/ai/src/providers/register-builtins.ts:206-217 function loadAnthropicProviderModule() { anthropicProviderModulePromise ||= import("./anthropic.ts").then((module) => { const provider = module as AnthropicProviderModule; return { stream: provider.streamAnthropic, streamSimple: provider.streamSimpleAnthropic, }; }); return anthropicProviderModulePromise; } ``` `createLazyStream` wraps a `loadModule` function into a `StreamFunction`. It returns an `AssistantMessageEventStream` immediately — the caller can start iterating right away — and asynchronously resolves the module promise, then forwards events from the inner provider stream: ```ts // packages/ai/src/providers/register-builtins.ts:162-181 function createLazyStream( loadModule: () => Promise>, ): StreamFunction { return (model, context, options) => { const outer = new AssistantMessageEventStream(); loadModule() .then((module) => { const inner = module.stream(model, context, options); forwardStream(outer, inner); }) .catch((error) => { const message = createLazyLoadErrorMessage(model, error); outer.push({ type: "error", reason: "error", error: message }); outer.end(message); }); return outer; }; } ``` `forwardStream` drives the async iteration of `inner` and pushes each event into `outer`: ```ts // packages/ai/src/providers/register-builtins.ts:132-139 function forwardStream(target: AssistantMessageEventStream, source: AsyncIterable): void { (async () => { for await (const event of source) { target.push(event); } target.end(); })(); } ``` These lazy stream functions are then registered directly: ```ts // packages/ai/src/providers/register-builtins.ts:326-327 export const streamAnthropic = createLazyStream(loadAnthropicProviderModule); export const streamSimpleAnthropic = createLazySimpleStream(loadAnthropicProviderModule); ``` ### The Bedrock special case AWS Bedrock requires Node.js-specific modules. To support environments where the AWS SDK may not be available (e.g., browsers, Deno without Node compat), `register-builtins.ts` introduces a secondary escape hatch: ```ts // packages/ai/src/providers/register-builtins.ts:89-92 const importNodeOnlyProvider = (specifier: string): Promise => { const runtimeSpecifier = import.meta.url.endsWith(".js") ? specifier.replace(/\.ts$/, ".js") : specifier; return import(runtimeSpecifier); }; ``` An additional `setBedrockProviderModule` export lets the caller inject a pre-loaded Bedrock module, bypassing the dynamic import entirely: ```ts // packages/ai/src/providers/register-builtins.ts:125-130 export function setBedrockProviderModule(module: BedrockProviderModule): void { bedrockProviderModuleOverride = { stream: module.streamBedrock, streamSimple: module.streamSimpleBedrock, }; } ``` Sources: [packages/ai/src/providers/register-builtins.ts:89-130, 162-217, 310-406]() --- ## `registerBuiltInApiProviders` and Module-Load Side Effect `registerBuiltInApiProviders` registers all nine built-in providers. The lazy stream functions are already bound closures at this point — the actual provider modules have not loaded yet: ```ts // packages/ai/src/providers/register-builtins.ts:345-399 export function registerBuiltInApiProviders(): void { registerApiProvider({ api: "anthropic-messages", stream: streamAnthropic, streamSimple: streamSimpleAnthropic }); registerApiProvider({ api: "openai-completions", stream: streamOpenAICompletions, streamSimple: streamSimpleOpenAICompletions }); registerApiProvider({ api: "mistral-conversations", stream: streamMistral, streamSimple: streamSimpleMistral }); registerApiProvider({ api: "openai-responses", stream: streamOpenAIResponses, streamSimple: streamSimpleOpenAIResponses }); registerApiProvider({ api: "azure-openai-responses", stream: streamAzureOpenAIResponses, streamSimple: streamSimpleAzureOpenAIResponses }); registerApiProvider({ api: "openai-codex-responses", stream: streamOpenAICodexResponses, streamSimple: streamSimpleOpenAICodexResponses }); registerApiProvider({ api: "google-generative-ai", stream: streamGoogle, streamSimple: streamSimpleGoogle }); registerApiProvider({ api: "google-vertex", stream: streamGoogleVertex, streamSimple: streamSimpleGoogleVertex }); registerApiProvider({ api: "bedrock-converse-stream", stream: streamBedrockLazy, streamSimple: streamSimpleBedrockLazy }); } ``` Critically, the file ends with an unconditional call: ```ts // packages/ai/src/providers/register-builtins.ts:406 registerBuiltInApiProviders(); ``` This means that merely importing `register-builtins.ts` — which `stream.ts` does as a side-effect import — populates the registry. Any code that imports from `@earendil-works/pi-ai` transitively triggers this. The registry is ready before any user code runs, but no provider SDK has loaded yet. Sources: [packages/ai/src/providers/register-builtins.ts:345-406]() --- ## Tradeoff: Startup Time vs. First-Call Overhead ```text Module load (import register-builtins.ts) │ ├─ registerBuiltInApiProviders() runs immediately │ └─ 9 × registerApiProvider(lazyStream, ...) │ └─ registry Map populated; provider modules NOT loaded │ First call to stream(model, context) ← model.api = "anthropic-messages" │ ├─ resolveApiProvider("anthropic-messages") → registry hit ├─ provider.stream(model, context) → createLazyStream closure fires │ ├─ outer = new AssistantMessageEventStream() ← returned immediately │ └─ loadAnthropicProviderModule() starts │ ├─ import("./anthropic.ts") resolves (network/disk) │ └─ forwardStream(outer, inner) begins piping events │ └─ caller starts iterating outer — first event arrives after module resolves Subsequent calls (same api key) └─ loadAnthropicProviderModule() returns cached Promise immediately └─ module already loaded; forwardStream fires without import delay ``` | Phase | Cost | Notes | |---|---|---| | Module import | Low — 9 `Map.set` calls | No SDK loaded | | First call per `api` | `dynamic import` round-trip | One-time per provider per process | | Subsequent calls | Effectively zero | `||=` guard returns resolved Promise | | Error during load | Encoded in stream | `createLazyLoadErrorMessage` wraps the error as an `AssistantMessage` with `stopReason: "error"`, keeping the caller's async-iteration contract intact | Sources: [packages/ai/src/providers/register-builtins.ts:162-204](), [packages/ai/src/stream.ts:17-23]() --- ## Registering a Custom Provider The `Api` type is open-ended: ```ts // packages/ai/src/types.ts:17 export type Api = KnownApi | (string & {}); ``` This means any string can be a valid API key. A custom provider registers itself by calling the same function the built-ins use: ```ts // packages/coding-agent/examples/extensions/custom-provider-gitlab-duo/test.ts:40-44 registerApiProvider({ api: "gitlab-duo-api" as Api, stream: streamGitLabDuo, streamSimple: streamGitLabDuo, }); ``` Then a `Model` is constructed with `api: "gitlab-duo-api"`, and `stream(model, context, options)` routes to the GitLab Duo implementation without any change to `stream.ts`. The `sourceId` parameter supports lifecycle management: plugins pass a stable ID, and `unregisterApiProviders(sourceId)` removes all entries associated with that ID when the plugin is torn down. ```ts // packages/ai/src/api-registry.ts:88-94 export function unregisterApiProviders(sourceId: string): void { for (const [api, entry] of apiProviderRegistry.entries()) { if (entry.sourceId === sourceId) { apiProviderRegistry.delete(api); } } } ``` Sources: [packages/ai/src/api-registry.ts:88-94](), [packages/coding-agent/examples/extensions/custom-provider-gitlab-duo/test.ts:40-44]() --- ## The Faux Provider: Same Path for Tests Testing uses the same registration path. `registerFauxProvider` in `faux.ts` creates a controllable `stream` function that pops scripted responses from a queue, then calls `registerApiProvider` with an auto-generated `api` key and a `sourceId`: ```ts // packages/ai/src/providers/faux.ts:470 registerApiProvider({ api, stream, streamSimple }, sourceId); ``` The test harness calls `fauxProvider.unregister()` to clean up, which calls `unregisterApiProviders(sourceId)`. Tests never touch the real provider modules; they go through exactly the same registry lookup as production code. Sources: [packages/ai/src/providers/faux.ts:391-499]() --- ## Architecture Summary ```text ┌─────────────────────────────────────────────────────────┐ │ @earendil-works/pi-ai │ │ │ │ stream.ts api-registry.ts │ │ ┌──────────────┐ ┌──────────────────────────┐ │ │ │ stream() │──────▶│ Map │ │ │ │ streamSimple()│ │ │ │ │ │ complete() │ │ "anthropic-messages" → │ │ │ └──────────────┘ │ lazyStream(loadAnthr.) │ │ │ ▲ │ "openai-responses" → │ │ │ │ │ lazyStream(loadOAI) │ │ │ registerApiProvider() │ "gitlab-duo-api" → │ │ │ (public API) │ streamGitLabDuo │ │ │ │ │ "faux:abc123" → │ │ │ ┌─────┴─────────┐ │ fauxStream │ │ │ │Built-ins │ └──────────────────────────┘ │ │ │register- │ │ │ │builtins.ts │ provider modules (lazy) │ │ │(side effect │ ┌──────────────────────────┐ │ │ │ on import) │ │ anthropic.ts (SDK) │ │ │ └───────────────┘ │ openai-responses.ts │ │ │ │ google.ts │ │ │ ┌───────────────┐ │ amazon-bedrock.ts │ │ │ │Custom/Test │ │ ... (loaded on 1st use) │ │ │ │Providers │ └──────────────────────────┘ │ │ │(registerApi │ │ │ │ Provider) │ │ │ └───────────────┘ │ └─────────────────────────────────────────────────────────┘ ``` The registry is a thin `Map`. It carries no provider logic. Every provider — built-in, third-party, or test — becomes callable by writing one `api` string key into that map. The lazy-load pattern in `register-builtins.ts` ensures that startup cost is fixed (nine `Map.set` calls) regardless of how many providers are registered, while deferring SDK import costs to first actual use per provider. The `AssistantMessageEventStream` returned immediately by every lazy-wrapped call preserves the streaming contract even before the backing module has resolved. Sources: [packages/ai/src/api-registry.ts:40](), [packages/ai/src/providers/register-builtins.ts:162-181, 406]()