# LLM providers reference

> LLMProvider enum: Groq, Gemini, OpenAI, Anthropic endpoints, default models, rewriteTimeout ceilings, validationRequest probes, keyURL/billingURL, OpenAI-compatible vs Anthropic Messages API shapes, and RewriteFailure reason codes.

- Repository: cartesia-ai/InkIt
- GitHub: https://github.com/cartesia-ai/InkIt
- Human docs: https://grok-wiki.com/public/docs/cartesia-ai-inkit-18975554254b
- Complete Markdown: https://grok-wiki.com/public/docs/cartesia-ai-inkit-18975554254b/llms-full.txt

## Source Files

- `InkIt/LLMProvider.swift`
- `InkIt/TranscriptRewriter.swift`
- `InkIt/CartesiaKeyValidator.swift`
- `InkIt/SettingsStore.swift`
- `InkIt/TranscriptHistoryStore.swift`

---

---
title: "LLM providers reference"
description: "LLMProvider enum: Groq, Gemini, OpenAI, Anthropic endpoints, default models, rewriteTimeout ceilings, validationRequest probes, keyURL/billingURL, OpenAI-compatible vs Anthropic Messages API shapes, and RewriteFailure reason codes."
---

InkIt’s optional Polish step routes raw STT transcripts through `TranscriptRewriter`, which posts to whichever `LLMProvider` is selected in `SettingsStore.rewriteProvider`. The `LLMProvider` enum in `LLMProvider.swift` centralizes chat endpoints, curated models, per-provider timeouts, key/billing URLs, and credit-free validation probes; `TranscriptRewriter` handles the HTTP shapes and classifies failures into `RewriteFailure` cases that downstream code maps to history tooltips and persisted service-issue flags.

## Provider enum

`LLMProvider` is a `String`-backed enum with four cases: `groq`, `gemini`, `openai`, and `anthropic`. It conforms to `CaseIterable`, `Identifiable`, and `Hashable`; `id` is the `rawValue`.

| Case | `displayName` | `keyPlaceholder` | `isRecommended` | `isOpenAICompatible` |
|------|---------------|------------------|-----------------|----------------------|
| `groq` | Groq | `gsk_…` | `true` | `true` |
| `gemini` | Google Gemini | `AIza…` | `false` | `true` |
| `openai` | OpenAI | `sk-…` | `false` | `true` |
| `anthropic` | Anthropic | `sk-ant-…` | `false` | `false` |

Groq is the default `rewriteProvider` when no value is stored. Settings shows a “(Recommended)” badge on Groq via `isRecommended`.

<Info>
Each provider exposes one curated model in `models`, ordered with the recommended default first. `defaultModel` returns `models.first!`. If the stored `rewriteModel` is not in the provider’s list, Settings resets it to `defaultModel`.
</Info>

### Default models

| Provider | `defaultModel` |
|----------|----------------|
| Groq | `llama-3.3-70b-versatile` |
| Gemini | `gemini-2.5-flash-lite` |
| OpenAI | `gpt-4.1-nano` |
| Anthropic | `claude-haiku-4-5-20251001` |

## Chat endpoints

Polish uses provider-specific chat URLs. Anthropic speaks the native Messages API; all others use OpenAI-compatible `/chat/completions`.

| Provider | `endpoint` |
|----------|------------|
| Groq | `https://api.groq.com/openai/v1/chat/completions` |
| Gemini | `https://generativelanguage.googleapis.com/v1beta/openai/chat/completions` |
| OpenAI | `https://api.openai.com/v1/chat/completions` |
| Anthropic | `https://api.anthropic.com/v1/messages` |

:::endpoint POST /chat/completions (Groq, Gemini, OpenAI)
OpenAI-compatible polish request issued by `TranscriptRewriter.call`.

**Headers**
- `Content-Type: application/json`
- `Authorization: Bearer {apiKey}`

**Body fields**

<ParamField body="model" type="string" required>
Provider model ID from `rewriteModel` (e.g. `llama-3.3-70b-versatile`).
</ParamField>

<ParamField body="max_tokens" type="integer" required>
Computed as `min(1500, estimatedInputTokens * 3 + 80)` where `estimatedInputTokens = max(48, transcript.count / 3)`.
</ParamField>

<ParamField body="temperature" type="number" required>
Always `0`.
</ParamField>

<ParamField body="messages" type="array" required>
Two messages: a `system` role with flattened instruction text, and a `user` role wrapping the transcript in `<transcript>…</transcript>`.
</ParamField>

**Response extraction**
`choices[0].message.content` as a string.

:::
:::endpoint POST /v1/messages (Anthropic)
Native Messages API polish request.

**Headers**
- `Content-Type: application/json`
- `x-api-key: {apiKey}`
- `anthropic-version: 2023-06-01`

**Body fields**

<ParamField body="model" type="string" required>
Anthropic model ID (e.g. `claude-haiku-4-5-20251001`).
</ParamField>

<ParamField body="max_tokens" type="integer" required>
Same formula as OpenAI-compatible providers.
</ParamField>

<ParamField body="temperature" type="number" required>
Always `0`.
</ParamField>

<ParamField body="system" type="array" required>
Anthropic-style system blocks: `[{ "type": "text", "text": "…" }]`.
</ParamField>

<ParamField body="messages" type="array" required>
Single `user` message with `<transcript>…</transcript>` content.
</ParamField>

**Response extraction**
Concatenates all `content` blocks where `type == "text"`.

:::

## Rewrite timeouts

`rewriteTimeout` is the hard ceiling before `TranscriptRewriter` abandons polish and the caller falls back to the raw transcript. The value is applied to both `timeoutIntervalForRequest` and `timeoutIntervalForResource` on the rewriter’s `URLSession`, and to each request’s `timeoutInterval`.

| Provider | `rewriteTimeout` |
|----------|------------------|
| Groq | `1.0` s |
| Gemini, OpenAI, Anthropic | `2.0` s |

Groq’s 1.0 s ceiling is sized above observed p99 latency for the default Llama model; other providers get more headroom. On timeout, `RewriteFailure.timedOut` is returned and the raw transcript is pasted.

<Note>
`TranscriptRewriter.prewarm()` issues a `HEAD` to `provider.endpoint` with a separate 2.5 s timeout. The response is discarded; the goal is to warm DNS/TCP/TLS on the same `URLSession` before the hot-path `POST`.
</Note>

## Key and billing URLs

| Provider | `keyURL` | `billingURL` | `keyHint` |
|----------|----------|--------------|-----------|
| Groq | `https://console.groq.com/keys` | `https://console.groq.com/settings/billing` | Free tier, no card needed. |
| Gemini | `https://aistudio.google.com/apikey` | `https://aistudio.google.com/` | Free tier from Google AI Studio. |
| OpenAI | `https://platform.openai.com/api-keys` | `https://platform.openai.com/settings/organization/billing` | Uses your existing OpenAI account. |
| Anthropic | `https://console.anthropic.com/settings/keys` | `https://console.anthropic.com/settings/billing` | Uses your existing Anthropic account. |

Settings links the key field to `keyURL`. When Home surfaces a polish out-of-credits issue, the card action opens `billingURL` for the active provider.

## Key storage and settings keys

Per-provider API keys live in the macOS Keychain under account `llm.{provider.rawValue}` (e.g. `llm.groq`). `SettingsStore.llmAPIKeys` is a `[String: String]` dictionary keyed by `LLMProvider.rawValue`.

| Setting | UserDefaults key | Default |
|---------|------------------|---------|
| Active provider | `rewriteProvider` | `groq` |
| Active model | `rewriteModel` | Groq default model |
| Polish enabled | `correctionEnabled` | `false` |
| Key rejected (401/403) | `polishKeyInvalid` | `false` |
| Billing exhausted (402) | `polishOutOfCredits` | `false` |

Switching `rewriteProvider` clears `polishKeyInvalid` and `polishOutOfCredits` so verdicts do not carry across providers.

## Validation probes

InkIt validates polish keys with credit-free `GET /models` probes — no tokens spent.

### `validationRequest(key:)`

`LLMProvider.validationRequest` builds the probe for the selected provider:

| Provider | Validation URL | Auth |
|----------|----------------|------|
| Groq | `https://api.groq.com/openai/v1/models` | `Authorization: Bearer {key}` |
| Gemini | `https://generativelanguage.googleapis.com/v1beta/openai/models` | `Authorization: Bearer {key}` |
| OpenAI | `https://api.openai.com/v1/models` | `Authorization: Bearer {key}` |
| Anthropic | `https://api.anthropic.com/v1/models` | `x-api-key: {key}`, `anthropic-version: 2023-06-01` |

All probes use `GET`, `timeoutInterval = 8`, and are advisory — they never block dictation.

### Validator classes

`APIKeyValidator` debounces keystrokes by 0.6 s, ignores stale completions, and caches verdicts per key:

| HTTP status | `APIKeyValidator.State` |
|-------------|-------------------------|
| 2xx | `verified` |
| 400, 401, 403 | `invalidKey` (Gemini returns 400 for `API_KEY_INVALID`) |
| Other / transport error | `couldNotVerify` |

- **`LLMKeyValidator`** — used in Settings `PolishKeyField`; calls `provider.validationRequest` and repoints when the user switches providers.
- **`GroqKeyValidator`** — Groq-only subclass hard-wired to `GET https://api.groq.com/openai/v1/models`; intended for a focused onboarding polish path while Settings uses the provider-aware validator.

A verified key in setup or key-broken state triggers `SettingsStore.enablePolish(provider:)`.

## Rewrite pipeline

```mermaid
sequenceDiagram
    participant AC as AppCoordinator
    participant TR as TranscriptRewriter
    participant LP as LLMProvider endpoint
    participant HS as TranscriptHistoryStore

    AC->>TR: prewarm() on hotkey press
    TR->>LP: HEAD (discarded)
    AC->>TR: rewriteWithoutContext(transcript)
    TR->>LP: POST chat/messages
    alt success
        LP-->>TR: model text
        TR-->>AC: .success(cleaned)
        AC->>HS: polish=.polished, original=raw
    else failure
        LP-->>TR: HTTP/URLError
        TR-->>AC: .failure(RewriteFailure)
        AC->>HS: polish=.failed, text=raw, failure=PolishFailure
    end
```

`rewriteWithoutContext` guards against empty API keys (`.invalidKey`) and empty transcripts (`.unknown`), wraps input in `<transcript>…</transcript>`, and runs a length sanity check: responses longer than `max(120, transcript.count * 2.5 + 40)` are rejected as `.unknown`.

## `RewriteFailure` reason codes

`RewriteFailure` is the error enum returned by `TranscriptRewriter`. On any failure, `AppCoordinator` pastes the raw transcript and records a `TranscriptHistoryStore.PolishFailure` for the history row tooltip.

| `RewriteFailure` | Trigger | HTTP / network source |
|------------------|---------|----------------------|
| `rateLimited(retryAt:)` | Provider throttling | HTTP 429; `retryAt` from `Retry-After` header (seconds or HTTP date) |
| `offline` | No reachable network | `URLError`: `.notConnectedToInternet`, `.cannotConnectToHost`, `.cannotFindHost`, `.networkConnectionLost`, `.dataNotAllowed`, `.dnsLookupFailed` |
| `timedOut` | Request exceeded timeout | HTTP 408, 504; `URLError.timedOut`; also URLSession timeout |
| `invalidKey` | Auth rejected | HTTP 401, 403; empty API key before request |
| `outOfCredits` | Billing limit | HTTP 402 |
| `serverError` | Provider fault | HTTP 5xx |
| `unknown` | Unclassified | JSON parse failure, empty model output, length sanity reject, other `URLError` codes |

### Persistence side effects

Only two failure reasons set persisted Home-card flags via `AppCoordinator.correctedTranscript`:

| `RewriteFailure` | UserDefaults flag | Home `ServiceIssue` |
|------------------|-------------------|---------------------|
| `invalidKey` | `polishKeyInvalid = true` | `.keyInvalid` |
| `outOfCredits` | `polishOutOfCredits = true` | `.outOfCredits` |

A successful rewrite clears both flags. Transient failures (rate limit, offline, timeout, server error) surface in the notch moment and history tooltip but do not set persisted flags.

### History mapping

`AppCoordinator.polishResult` maps each `RewriteFailure` 1:1 to `PolishFailureReason` (same case names). `TranscriptHistoryStore.PolishFailure` adds `provider` (`displayName`) and optional `retryAt` for rate limits. History rows show an amber warning pill; hover text comes from `HistoryRow.failureMessage`, e.g.:

- Rate limit: `"{provider} rate limit — raw text pasted. {retry hint}"`
- Invalid key: `"Invalid {provider} API key — raw text pasted. Fix it in Settings."`
- Out of credits: `"Out of {provider} credits — raw text pasted. Review your {provider} plan to re-enable Polish."`

<Warning>
Polish failures never block paste. The user always receives at least the raw STT transcript; `PolishUIState.keyBroken` only reflects persisted auth failures, not transient errors.
</Warning>

## Request examples

<RequestExample>

```json title="OpenAI-compatible (Groq)"
POST https://api.groq.com/openai/v1/chat/completions
Authorization: Bearer gsk_…
Content-Type: application/json

{
  "model": "llama-3.3-70b-versatile",
  "max_tokens": 230,
  "temperature": 0,
  "messages": [
    { "role": "system", "content": "You are a transcription cleaner…" },
    { "role": "user", "content": "<transcript>\nhey um can you send the draft by friday\n</transcript>" }
  ]
}
```

</RequestExample>

<RequestExample>

```json title="Anthropic Messages API"
POST https://api.anthropic.com/v1/messages
x-api-key: sk-ant-…
anthropic-version: 2023-06-01
Content-Type: application/json

{
  "model": "claude-haiku-4-5-20251001",
  "max_tokens": 230,
  "temperature": 0,
  "system": [{ "type": "text", "text": "You are a transcription cleaner…" }],
  "messages": [
    { "role": "user", "content": "<transcript>\nhey um can you send the draft by friday\n</transcript>" }
  ]
}
```

</RequestExample>

<RequestExample>

```http title="Validation probe (any OpenAI-compatible provider)"
GET https://api.groq.com/openai/v1/models
Authorization: Bearer gsk_…
```

</RequestExample>

## Related pages

<CardGroup>
<Card title="Configure Polish" href="/configure-polish">
Enable polish, pick a provider, enter keys, and interpret PolishUIState (setup, on, paused, keyBroken).
</Card>
<Card title="Dictation pipeline" href="/dictation-pipeline">
End-to-end flow from hotkey through STT, optional TranscriptRewriter polish, and paste.
</Card>
<Card title="Settings reference" href="/settings-reference">
All persisted SettingsStore keys including rewriteProvider, rewriteModel, and Keychain accounts.
</Card>
<Card title="Polish troubleshooting" href="/polish-troubleshooting">
Failure modes, rate-limit Retry-After handling, timeout fallbacks, and prewarm behavior.
</Card>
</CardGroup>