# Configure LLM providers

> Wire OpenAI, Anthropic, Gemini, Bedrock, DeepSeek, GLM, Kimi, Qwen, and Ollama via env keys and server URLs; UI provider profiles for per-agent models; test providers before flows.

- Repository: vxcontrol/pentagi
- GitHub: https://github.com/vxcontrol/pentagi
- Human docs: https://grok-wiki.com/public/docs/vxcontrol-pentagi-eca0cf4de5f5
- Complete Markdown: https://grok-wiki.com/public/docs/vxcontrol-pentagi-eca0cf4de5f5/llms-full.txt

## Source Files

- `backend/pkg/config/config.go`
- `backend/pkg/providers/providers.go`
- `backend/pkg/providers/provider/provider.go`
- `.env.example`
- `frontend/src/pages/settings/settings-providers.tsx`
- `backend/pkg/server/services/providers.go`
- `backend/docs/config.md`

---

---
title: "Configure LLM providers"
description: "Wire OpenAI, Anthropic, Gemini, Bedrock, DeepSeek, GLM, Kimi, Qwen, and Ollama via env keys and server URLs; UI provider profiles for per-agent models; test providers before flows."
---

PentAGI wires LLMs as a **BYOK (bring your own key)** stack: environment variables enable default providers of type `openai`, `anthropic`, `gemini`, `bedrock`, `deepseek`, `glm`, `kimi`, `qwen`, `ollama`, and `custom`; the web UI and GraphQL store optional user-defined provider profiles (per-agent models and runtime options) that reuse those same credentials. At least one provider type must become available before flows can call models.

## Two configuration layers

| Layer | What it configures | Where it lives | Editable at runtime |
| --- | --- | --- | --- |
| Connection credentials | API keys, base URLs, auth modes, optional config paths | `.env` / process env via `Config` | No — restart required after changes |
| Provider profiles | Per-agent `model`, temperature, reasoning, prices, etc. | Default embedded `config.yml` per type, or DB rows from Settings → Providers | Yes — UI / GraphQL CRUD |

<Info>
Web Settings manage **profiles** (agent model options). They do **not** replace `OPEN_AI_KEY`, `ANTHROPIC_API_KEY`, `BEDROCK_*`, `OLLAMA_SERVER_URL`, or other connection env vars. Without credentials for a type, that type stays disabled in `settingsProviders.enabled`.
</Info>

```mermaid
flowchart LR
  subgraph env [Environment]
    Keys["API keys / URLs<br/>Config struct"]
  end
  subgraph ctrl [ProviderController]
    Defaults["defaultConfigs + built-in Providers"]
    UserDB["user-defined providers table"]
  end
  subgraph surfaces [Surfaces]
    REST["GET /api/v1/providers/"]
    GQL["settingsProviders / test* / createProvider"]
    UI["Settings → Providers"]
    Flow["NewFlowProvider / NewAssistantProvider"]
  end
  Keys --> Defaults
  Defaults --> Flow
  UserDB --> Flow
  Defaults --> REST
  UserDB --> REST
  Defaults --> GQL
  UserDB --> GQL
  GQL --> UI
```

Lookup order for a named provider: **user-defined profile by name for the current user first**, then built-in defaults (`openai`, `anthropic`, …). User profiles of a type only work when that type is already enabled by env credentials.

## Prerequisites

- Stack deployed with a valid `DATABASE_URL` (see [Installation](/installation)).
- `.env` created from `.env.example` with **at least one** LLM credential set that satisfies an activation gate below.
- Process restart (or `docker compose up -d` recreate) after env changes so `NewProviderController` re-registers providers.
- Browser access to the UI (typically `https://localhost:8443`) or API auth with `providers.view` / `settings.providers.*` permissions.

## Enable default providers via environment

Copy `.env.example` keys into `.env`. Empty keys leave the provider unregistered. Defaults match `backend/pkg/config/config.go`.

### Activation gates

`NewProviderController` only registers a built-in provider when:

| Type | Register when |
| --- | --- |
| `openai` | `OPEN_AI_KEY` non-empty |
| `anthropic` | `ANTHROPIC_API_KEY` non-empty |
| `gemini` | `GEMINI_API_KEY` non-empty |
| `bedrock` | `BEDROCK_DEFAULT_AUTH=true`, **or** `BEDROCK_BEARER_TOKEN` set, **or** both `BEDROCK_ACCESS_KEY_ID` and `BEDROCK_SECRET_ACCESS_KEY` set |
| `ollama` | `OLLAMA_SERVER_URL` non-empty |
| `custom` | `LLM_SERVER_URL` non-empty **and** (`LLM_SERVER_MODEL` **or** `LLM_SERVER_CONFIG_PATH` set) |
| `deepseek` | `DEEPSEEK_API_KEY` non-empty |
| `glm` | `GLM_API_KEY` non-empty |
| `kimi` | `KIMI_API_KEY` non-empty |
| `qwen` | `QWEN_API_KEY` non-empty |

Failed construction of a registered provider returns an error from controller startup (hard fail for that process).

### Provider env reference

<Tabs>
  <Tab title="OpenAI">
    | Variable | Default | Role |
    | --- | --- | --- |
    | `OPEN_AI_KEY` | *(empty)* | API key; required to enable |
    | `OPEN_AI_SERVER_URL` | `https://api.openai.com/v1` | Chat Completions base URL |

    ```bash
    OPEN_AI_KEY=sk-...
    OPEN_AI_SERVER_URL=https://api.openai.com/v1
    ```
  </Tab>
  <Tab title="Anthropic">
    | Variable | Default | Role |
    | --- | --- | --- |
    | `ANTHROPIC_API_KEY` | *(empty)* | API key; required to enable |
    | `ANTHROPIC_SERVER_URL` | `https://api.anthropic.com/v1` | Anthropic API base URL |

    No dedicated Vertex AI env path exists for Claude. Use Bedrock or an OpenAI-compatible gateway as `custom` if you need non-Anthropic hosting.
  </Tab>
  <Tab title="Gemini">
    | Variable | Default | Role |
    | --- | --- | --- |
    | `GEMINI_API_KEY` | *(empty)* | Google AI Studio key; required to enable |
    | `GEMINI_SERVER_URL` | `https://generativelanguage.googleapis.com` | Gemini REST endpoint |
  </Tab>
  <Tab title="Bedrock">
    | Variable | Default | Role |
    | --- | --- | --- |
    | `BEDROCK_REGION` | `us-east-1` | AWS region |
    | `BEDROCK_DEFAULT_AUTH` | `false` | Use AWS SDK default credential chain |
    | `BEDROCK_BEARER_TOKEN` | *(empty)* | Bearer auth |
    | `BEDROCK_ACCESS_KEY_ID` | *(empty)* | Static access key |
    | `BEDROCK_SECRET_ACCESS_KEY` | *(empty)* | Static secret |
    | `BEDROCK_SESSION_TOKEN` | *(empty)* | Optional STS/session token |
    | `BEDROCK_SERVER_URL` | *(empty)* | Optional VPC/custom endpoint |

    Auth priority: `BEDROCK_DEFAULT_AUTH` → `BEDROCK_BEARER_TOKEN` → static key pair.
  </Tab>
  <Tab title="DeepSeek / GLM / Kimi / Qwen">
    | Provider | Key | Server URL default | Optional LiteLLM prefix |
    | --- | --- | --- | --- |
    | DeepSeek | `DEEPSEEK_API_KEY` | `https://api.deepseek.com` | `DEEPSEEK_PROVIDER` (e.g. `deepseek`) |
    | GLM | `GLM_API_KEY` | `https://api.z.ai/api/paas/v4` | `GLM_PROVIDER` (e.g. `zai`) |
    | Kimi | `KIMI_API_KEY` | `https://api.moonshot.ai/v1` | `KIMI_PROVIDER` (e.g. `moonshot`) |
    | Qwen | `QWEN_API_KEY` | `https://dashscope-us.aliyuncs.com/compatible-mode/v1` | `QWEN_PROVIDER` (e.g. `dashscope`) |

    Regional URL alternatives (China / Singapore / coding endpoints) are documented in the config guide; override `*_SERVER_URL` only.
  </Tab>
  <Tab title="Ollama / custom">
    **Ollama** (local or cloud): enable with `OLLAMA_SERVER_URL`. Cloud (`https://ollama.com`) needs `OLLAMA_SERVER_API_KEY`. Optional: `OLLAMA_SERVER_MODEL`, `OLLAMA_SERVER_CONFIG_PATH`, pull/load flags.

    **Custom** OpenAI-compatible endpoint: `LLM_SERVER_URL` plus `LLM_SERVER_MODEL` and/or `LLM_SERVER_CONFIG_PATH`. Optional: `LLM_SERVER_KEY`, `LLM_SERVER_PROVIDER`, reasoning flags.

    Full wiring, YAML config paths, aggregators (OpenRouter, DeepInfra), and vLLM are covered on [Local and custom providers](/local-and-custom-providers) and [Deploy with vLLM and Qwen](/vllm-qwen-deployment).
  </Tab>
</Tabs>

### Shared HTTP settings that affect all providers

| Variable | Default | Role |
| --- | --- | --- |
| `PROXY_URL` | *(empty)* | HTTP(S) proxy for outbound calls (including LLM clients) |
| `EXTERNAL_SSL_CA_PATH` | *(empty)* | Extra CA bundle for TLS to LLM backends |
| `EXTERNAL_SSL_INSECURE` | `false` | Skip TLS verify for external LLM HTTP (not recommended) |
| `HTTP_CLIENT_TIMEOUT` | `600` | Seconds for external API HTTP clients (0 falls back to default) |

## Default agent baselines

Each built-in type ships an embedded `config.yml` (for example `backend/pkg/providers/openai/config.yml`) with per-agent models. Agent option keys:

| Key | Role |
| --- | --- |
| `simple` | Lightweight completion (image chooser, titles, language) |
| `simple_json` | Structured JSON completion |
| `primary_agent` | Main flow agent loop |
| `assistant` | Assistant mode |
| `generator` / `refiner` | Subtask plan generation and refinement |
| `adviser` / `reflector` | Advice and reflection |
| `searcher` / `enricher` | Search and context enrichment |
| `coder` / `installer` / `pentester` | Specialist tool-using agents |

User profiles and UI edits override these per agent without changing env keys. Field-level schema (temperature, reasoning, `extra_body`, prices) is on [Provider configuration schema](/provider-config-schema).

## UI provider profiles

**Settings → Providers** (`settings-providers.tsx` / `settings-provider.tsx`) loads GraphQL `settingsProviders`:

- `enabled` — readiness booleans per type (credential gates above)
- `default` — server baselines for each type
- `userDefined` — your saved profiles
- `models` — catalogs from each type’s `models.yml` (where present)

Supported profile types in the UI: Anthropic, Bedrock, Custom, DeepSeek, Gemini, GLM, Kimi, Ollama, OpenAI, Qwen.

### Create or edit a profile

<Steps>
  <Step title="Confirm the type is enabled">
    Open Settings → Providers. If a type is missing from create options or readiness is false, set its env credentials and restart the backend.
  </Step>
  <Step title="Create or clone">
    Use **New** for a type, or clone an existing profile. Profiles are stored per user in the `providers` table (`name`, `type`, JSON `config`).
  </Step>
  <Step title="Set per-agent models">
    Assign models and options for each agent slot. Defaults are prefilled from the type’s embedded config when you start from a baseline.
  </Step>
  <Step title="Test before save (recommended)">
    Run **Test** (full provider) or per-agent **Test**. Mutations call live backends with your env credentials and the draft agent config.
  </Step>
  <Step title="Save">
    Persist with `createProvider` / `updateProvider`. Soft-delete with `deleteProvider`.
  </Step>
</Steps>

Permissions:

| Operation | Permission |
| --- | --- |
| List for flow picker / REST | `providers.view` |
| Settings list, test | `settings.providers.view` |
| Create / update / delete | `settings.providers.edit` |

### GraphQL surface (profiles and tests)

| Operation | Kind | Purpose |
| --- | --- | --- |
| `settingsProviders` | Query | Readiness, defaults, user profiles, model lists |
| `providers` | Query | Short list for flow/assistant selection |
| `testAgent` | Mutation | Run tests for one agent config slot |
| `testProvider` | Mutation | Run tests for all agent slots (parallel workers) |
| `createProvider` / `updateProvider` / `deleteProvider` | Mutation | CRUD user profiles |
| `providerCreated` / `providerUpdated` / `providerDeleted` | Subscription | Real-time UI updates |

### REST list for automation

:::endpoint GET /api/v1/providers/
Returns registered providers for the authenticated user (built-in + user-defined). Requires Bearer (or session) auth and `providers.view`.

Each item includes `name`, `type`, `default_model` (primary agent), and `models` with agent-type bindings and optional price info.
:::

## Test providers before flows

`TestProvider` / `TestAgent` build a temporary provider from the requested type and agent config, then run the internal tester suite (`tester.TestProvider`, default 16 parallel workers). Results expose per-agent suites: `simple`, `simpleJson`, `primaryAgent`, `assistant`, `generator`, `refiner`, `adviser`, `reflector`, `searcher`, `enricher`, `coder`, `installer`, `pentester`.

Each `TestResult` reports `name`, `type`, `result`, `reasoning`, `streaming`, optional `latency`, and `error`.

<Warning>
Tests consume real tokens against the live endpoint configured in env. A failed type construction (missing key for that type) returns an API error rather than a soft skip.
</Warning>

Recommended order: enable env → restart → UI readiness true → **Test** on defaults or a draft profile → create a flow selecting that provider name.

## End-to-end setup

<Steps>
  <Step title="Set credentials">
    Edit `.env` with at least one complete provider block (key and optional URL override).
  </Step>
  <Step title="Restart the stack">
    ```bash
    docker compose up -d
    # or restart the pentagi backend process after godotenv reload
    ```
  </Step>
  <Step title="Verify readiness">
    Open Settings → Providers, or query `settingsProviders { enabled { openai anthropic ... } }`. The types you enabled should be `true`.
  </Step>
  <Step title="Optional: tune profiles">
    Create a user profile if default models or temperatures need adjustment. Test the profile.
  </Step>
  <Step title="Create a flow">
    Choose the provider name (`openai`, a custom profile name, etc.) when starting a flow. The controller resolves user profile first, then built-in.
  </Step>
</Steps>

### Minimal `.env` examples

<CodeGroup>
```bash title="OpenAI only"
OPEN_AI_KEY=sk-...
OPEN_AI_SERVER_URL=https://api.openai.com/v1
```

```bash title="Anthropic only"
ANTHROPIC_API_KEY=sk-ant-...
ANTHROPIC_SERVER_URL=https://api.anthropic.com/v1
```

```bash title="Bedrock static credentials"
BEDROCK_REGION=us-east-1
BEDROCK_ACCESS_KEY_ID=AKIA...
BEDROCK_SECRET_ACCESS_KEY=...
# optional:
# BEDROCK_SESSION_TOKEN=...
# BEDROCK_SERVER_URL=https://bedrock-runtime.us-east-1.amazonaws.com
```

```bash title="DeepSeek"
DEEPSEEK_API_KEY=...
DEEPSEEK_SERVER_URL=https://api.deepseek.com
```
</CodeGroup>

## Failure modes and troubleshooting

| Symptom | Likely cause | What to check |
| --- | --- | --- |
| Type missing from UI / `enabled` false | Activation gate not met | Env var non-empty; restart process |
| Bedrock not registered | Incomplete auth | Default auth **or** bearer **or** both access+secret keys |
| Custom type missing | Incomplete custom gate | Need `LLM_SERVER_URL` **and** model or config path |
| User profile create fails / type unavailable | Type not in `ListTypes()` | Enable built-in credentials for that type first |
| REST `422` / invalid provider type | Unknown type string | Valid types only: `openai`, `anthropic`, `gemini`, `bedrock`, `ollama`, `custom`, `deepseek`, `glm`, `kimi`, `qwen` |
| Tests fail with TLS errors | Corporate proxy / private CA | `PROXY_URL`, `EXTERNAL_SSL_CA_PATH`, avoid `EXTERNAL_SSL_INSECURE` in production |
| Tests timeout | Slow or blocked egress | `HTTP_CLIENT_TIMEOUT`, network/firewall to provider URLs |
| Flow fails at first LLM call | Bad key, wrong base URL, or quota | Re-run provider test; confirm server URL path includes `/v1` where required |
| Profile name not used | Typo or wrong user | User profiles are scoped by `user_id`; name must match exactly |

## Embeddings note

Flow/tool embeddings use separate `EMBEDDING_*` env vars (`EMBEDDING_PROVIDER` defaults to `openai`). Configuring chat LLM providers does not automatically configure embeddings. See [Memory and knowledge](/memory-and-knowledge) and [Environment variables](/environment-variables).

## Scope boundaries

This page covers **credential env wiring**, **default registration**, **UI/GraphQL user profiles**, and **pre-flow testing** for the built-in provider types.

| Topic | Page |
| --- | --- |
| Custom `LLM_SERVER_*`, Ollama pull/load, OpenRouter/DeepInfra | [Local and custom providers](/local-and-custom-providers) |
| Per-agent YAML fields, `testAgent` field shapes | [Provider configuration schema](/provider-config-schema) |
| Copy-paste YAML examples | [Example provider configs](/example-provider-configs) |
| Full env inventory | [Environment variables](/environment-variables) |

## Related pages

<CardGroup>
  <Card title="Local and custom providers" href="/local-and-custom-providers">
    OpenAI-compatible endpoints, Ollama, reasoning flags, and aggregators.
  </Card>
  <Card title="Provider configuration schema" href="/provider-config-schema">
    AgentsConfig fields, built-in baselines, and testAgent/testProvider detail.
  </Card>
  <Card title="Example provider configs" href="/example-provider-configs">
    Ready YAML for vLLM, Ollama, OpenRouter, DeepInfra, and Azure-style endpoints.
  </Card>
  <Card title="Environment variables" href="/environment-variables">
    Authoritative Config keys and defaults across the stack.
  </Card>
  <Card title="Quickstart" href="/quickstart">
    First flow after setting one LLM key and bringing the stack up.
  </Card>
  <Card title="REST API" href="/rest-api">
    `GET /api/v1/providers/` and other automation endpoints.
  </Card>
</CardGroup>