# Build Setup & Code Generation Pipeline

> How .proto files become Rust service traits: connectrpc-build in build.rs, the protoc-gen-connect-rust buf plugin, the two generation modes (unified vs. service-stubs-only), checked-in generated directories, and the task generate:all workflow.

- Repository: anthropics/connect-rust
- GitHub: https://github.com/anthropics/connect-rust
- Human wiki: https://grok-wiki.com/public/wiki/anthropics-connect-rust-abe117693c52
- Complete Markdown: https://grok-wiki.com/public/wiki/anthropics-connect-rust-abe117693c52/llms-full.txt

## Source Files

- `connectrpc-build/src/lib.rs`
- `connectrpc-codegen/src/lib.rs`
- `connectrpc-codegen/src/codegen.rs`
- `connectrpc-codegen/src/plugin.rs`
- `Taskfile.yaml`
- `conformance/buf.gen.yaml`
- `examples/eliza/src/generated`

---

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:

- [connectrpc-build/src/lib.rs](connectrpc-build/src/lib.rs)
- [connectrpc-codegen/src/lib.rs](connectrpc-codegen/src/lib.rs)
- [connectrpc-codegen/src/codegen.rs](connectrpc-codegen/src/codegen.rs)
- [connectrpc-codegen/src/plugin.rs](connectrpc-codegen/src/plugin.rs)
- [Taskfile.yaml](Taskfile.yaml)
- [conformance/buf.gen.yaml](conformance/buf.gen.yaml)
- [examples/eliza/buf.gen.yaml](examples/eliza/buf.gen.yaml)
- [examples/eliza/src/generated/connect/connectrpc.eliza.v1.eliza.__connect.rs](examples/eliza/src/generated/connect/connectrpc.eliza.v1.eliza.__connect.rs)
</details>

# Build Setup & Code Generation Pipeline

This page explains how `.proto` service definitions become Rust service traits and client structs in connect-rust. The pipeline has two distinct entry points — a `build.rs` integration crate (`connectrpc-build`) and a `protoc`/`buf` plugin binary (`protoc-gen-connect-rust`) — each suited to a different workflow. Understanding the difference between unified and split generation, and knowing which checked-in directories exist, will save you from mysterious compile errors and regeneration confusion.

The core library (`connectrpc-codegen`) sits underneath both entry points. It owns all `quote!`-based Rust emission logic and exposes two public functions: `generate_files` (unified, used by `connectrpc-build`) and `generate_services` (service-stubs-only, used by the plugin). Both are built on `buffa-codegen` for protobuf message types; connect-rust adds only the service traits, clients, and `Encodable` impls.

---

## Prerequisites

| Tool | Minimum | Purpose |
|---|---|---|
| Rust | 1.88 (MSRV) | Compiles the workspace; requires let-chains |
| `protoc` | v27+ | Default descriptor source (editions syntax) |
| `buf` | any current | BSR module resolution, checked-in regeneration |
| `task` (go-task) | any | Command runner — `task --list` shows all targets |

`protoc` can be overridden with the `PROTOC` environment variable. When using `buf` mode, `protoc` is not needed at all. Sources: [connectrpc-build/src/lib.rs:377](), [conformance/buf.gen.yaml:1-33]()

---

## Two Entry Points, Two Generation Modes

```text
.proto files
     │
     ├─── connectrpc-build (build.rs) ──► generate_files()  ─► $OUT_DIR/
     │    Unified: messages + stubs      (buffa types + service traits
     │    in one module tree)             in one $OUT_DIR tree)
     │
     └─── protoc-gen-connect-rust ────► generate_services() ─► out/connect/
          (buf plugin, checked-in code)   Service stubs only; message types
                                          referenced via extern_paths
```

### Unified mode (`connectrpc-build` / `generate_files`)

Used in `build.rs`. Emits buffa message types **and** ConnectRPC service stubs together into `$OUT_DIR`. Service stubs reference message types with `super::`-relative paths, so both must live in the same module tree. This is the simplest workflow for projects that don't check in generated code.

```rust
// build.rs
fn main() {
    connectrpc_build::Config::new()
        .files(&["proto/my_service.proto"])
        .includes(&["proto/"])
        .include_file("_connectrpc.rs")
        .compile()
        .unwrap();
}

// lib.rs
connectrpc::include_generated!();
```

Sources: [connectrpc-build/src/lib.rs:1-27](), [connectrpc-codegen/src/lib.rs:16-20]()

### Split mode (`protoc-gen-connect-rust` / `generate_services`)

Used as a `buf` plugin to regenerate checked-in code. Emits **service stubs only** (`<stem>.__connect.rs` files) into a separate output directory. Message types are referenced via absolute Rust paths configured with `buffa_module=<rust_path>`. This is the workflow for checked-in generated code and BSR cargo SDKs.

Sources: [connectrpc-codegen/src/lib.rs:22-27](), [connectrpc-codegen/src/codegen.rs:274-285]()

---

## `connectrpc-build`: The `build.rs` Integration

`connectrpc-build` is a builder-pattern crate you add to `build-dependencies`. Its `Config` struct collects options and then drives the full pipeline on `compile()`.

### Descriptor acquisition

`Config` supports three ways to get a `FileDescriptorSet`:

| Mode | Trigger | Tool required |
|---|---|---|
| `Protoc` (default) | — | `protoc` on `PATH` or `$PROTOC` |
| `Buf` | `.use_buf()` | `buf` on `PATH` |
| `Precompiled` | `.descriptor_set("path.bin")` | none at build time |

Under `Protoc` mode, includes are sorted longest-first before stripping so that nested prefix directories (`proto/vendor/`) beat shallower ones (`proto/`). Under `Buf` and `Precompiled` modes, `.files()` takes proto-relative names (e.g. `"my/service.proto"`) not filesystem paths — the include stripping does not apply. Sources: [connectrpc-build/src/lib.rs:44-54](), [connectrpc-build/src/lib.rs:270-294]()

### The `compile()` pipeline

```
1. Acquire FileDescriptorSet bytes
   (run_protoc / run_buf / read precompiled file)
2. Decode FileDescriptorSet via buffa
3. codegen::generate_files(fds, files_to_generate, options)
4. Write per-file .rs outputs (write_if_changed)
5. Optionally emit include file (_connectrpc.rs)
6. Emit cargo:rerun-if-changed directives
```

`write_if_changed` skips writing identical content to avoid bumping mtimes and forcing downstream recompilation on every proto touch. Sources: [connectrpc-build/src/lib.rs:249-353](), [connectrpc-build/src/lib.rs:366-373]()

### Key builder methods

| Method | Default | Effect |
|---|---|---|
| `.files(&[...])` | required | Proto files to compile |
| `.includes(&[...])` | `[]` | Import search dirs (protoc only) |
| `.use_buf()` | — | Use `buf build` instead of `protoc` |
| `.descriptor_set("x.bin")` | — | Read precompiled FDS |
| `.include_file("_inc.rs")` | none | Emit module-tree include file |
| `.file_per_package(true)` | `false` | Emit one `.rs` per package (unified layout) |
| `.generate_json(false)` | `true` | Disable serde derives |
| `.emit_rerun_directives(false)` | `true` | For non-Cargo build systems |
| `.buffa_config(cfg)` | — | Replace buffa CodeGenConfig wholesale |

Sources: [connectrpc-build/src/lib.rs:83-190]()

---

## `protoc-gen-connect-rust`: The buf Plugin

The plugin binary lives in `connectrpc-codegen/src/main.rs` (installed via `cargo install --path connectrpc-codegen`). It implements the protoc plugin protocol: reads a `CodeGeneratorRequest` from stdin, calls `codegen::generate()`, and writes a `CodeGeneratorResponse` to stdout.

`generate()` parses comma-separated plugin `opt` parameters then delegates to `generate_services`. Sources: [connectrpc-codegen/src/codegen.rs:426-483]()

### Plugin options

| Option | Effect |
|---|---|
| `buffa_module=crate::proto` | Set the catch-all extern path (most common) |
| `extern_path=.pkg=::rust::path` | Map a specific proto package prefix (repeatable) |
| `file_per_package` | Emit one `<dotted.pkg>.rs` per package (no stitchers) |
| `strict_utf8_mapping` | Emit `Vec<u8>` for non-utf8 string fields |
| `no_json` | Accepted for compatibility; ignored (no messages emitted) |
| `no_register_fn` | Accepted for compatibility; ignored |

At least one catch-all extern path (`buffa_module` or `extern_path=.=...`) is required so every message type resolves. Sources: [connectrpc-codegen/src/codegen.rs:398-476]()

---

## Output File Layout

### Per-proto split (default)

For each `.proto` with at least one `service`, both modes emit:

```text
out/
├── <stem>.rs                  # buffa message types (unified only)
├── <stem>.__view.rs           # buffa view types (unified only)
├── <stem>.__connect.rs        # ConnectRPC service traits + clients
└── <pkg>.mod.rs               # Package stitcher (include!s siblings)
```

The `.__connect.rs` suffix is intentional — it avoids colliding with buffa's own per-proto filenames (`<stem>.rs`, `<stem>.__view.rs`, etc.). The package stitcher (`<pkg>.mod.rs`) is wired by `apply_companions` in the unified path, or emitted directly by `generate_services` in the split path. Sources: [connectrpc-codegen/src/codegen.rs:109-125](), [connectrpc-codegen/src/codegen.rs:182-184]()

### `file_per_package` layout

When `file_per_package` is enabled, everything collapses to one file per proto package:

```text
out/
└── <dotted.pkg>.rs   # messages + service stubs inlined (no companion files, no stitcher)
```

This mirrors the BSR/tonic filename convention. No `__connect.rs` siblings exist; no `protoc-gen-buffa-packaging` invocations are needed. Sources: [connectrpc-codegen/src/codegen.rs:168-178](), [connectrpc-build/src/lib.rs:151-174]()

### The include file

When `.include_file("_inc.rs")` is set, `connectrpc-build` emits a module-tree include file with nested `pub mod` blocks matching the proto package hierarchy. Include paths use either `env!("OUT_DIR")` (default `$OUT_DIR` workflow) or sibling-relative strings (when `out_dir()` is set explicitly).

```rust
// Generated _inc.rs (env!("OUT_DIR") form)
#[allow(impl_trait_redundant_captures, ...)]
pub mod connectrpc {
    use super::*;
    pub mod eliza {
        use super::*;
        pub mod v1 {
            use super::*;
            include!(concat!(env!("OUT_DIR"), "/connectrpc.eliza.v1.mod.rs"));
        }
    }
}
```

Sources: [connectrpc-build/src/lib.rs:463-545]()

---

## Checked-In Generated Directories

Four directories contain `buf generate` output that must be regenerated whenever `connectrpc-codegen` changes or the `buffa` dependency is bumped:

| Directory | Proto source | Notes |
|---|---|---|
| `conformance/src/generated/` | `buf.build/connectrpc/conformance` (BSR) | Separate `buffa/` and `connect/` subdirs |
| `examples/eliza/src/generated/` | local `proto/` | Same split layout |
| `examples/multiservice/src/generated/` | local `proto/` | Same split layout |
| `benches/rpc/src/generated/` | local `proto/` | Same split layout |

All four use the split layout: `protoc-gen-buffa` into `src/generated/buffa/`, `protoc-gen-connect-rust` into `src/generated/connect/`. Each directory has its own `buf.gen.yaml`.

### Example: eliza `buf.gen.yaml`

```yaml
# examples/eliza/buf.gen.yaml
version: v2
plugins:
  - local: ../../../buffa/target/release/protoc-gen-buffa
    out: src/generated/buffa
    opt: [views=true, json=true]
  - local: ../../../buffa/target/release/protoc-gen-buffa-packaging
    out: src/generated/buffa
    strategy: all
  - local: ../../target/release/protoc-gen-connect-rust
    out: src/generated/connect
    opt: [buffa_module=crate::proto]
  - local: ../../../buffa/target/release/protoc-gen-buffa-packaging
    out: src/generated/connect
    strategy: all
    opt: [filter=services]
```

The second `protoc-gen-buffa-packaging` invocation with `filter=services` emits the `<pkg>.mod.rs` stitcher for the connect output tree, covering only packages with service-declaring protos. Sources: [examples/eliza/buf.gen.yaml:1-16](), [conformance/buf.gen.yaml:1-33]()

### What the generated connect file looks like

The `.__connect.rs` files use fully-qualified paths (`::connectrpc::`, `::buffa::`) throughout — no top-level `use` statements — so multiple generated files can be `include!`d into the same module without E0252 collisions. Message types are referenced through the `buffa_module` path (e.g. `crate::proto::connectrpc::eliza::v1::SayResponse`). Sources: [examples/eliza/src/generated/connect/connectrpc.eliza.v1.eliza.__connect.rs:1-60]()

---

## The `task generate:all` Workflow

`task generate:all` is the single command to regenerate all four checked-in directories:

```bash
task generate:all
```

It runs the following steps in order:

```
1. cargo build --release -p connectrpc-codegen --bin protoc-gen-connect-rust
   (build the plugin from the current source)
2. cargo build --release --manifest-path ../buffa/Cargo.toml \
       -p protoc-gen-buffa -p protoc-gen-buffa-packaging
   (build sibling buffa plugins)
3. task example:eliza:generate    → cd examples/eliza && buf generate
4. task example:multiservice:generate → cd examples/multiservice && buf generate
5. task conformance:generate      → cd conformance && BUF_TOKEN="" buf generate
6. task bench:generate            → cd benches/rpc && buf generate
```

The build steps use release mode because `buf.gen.yaml` references the plugin binaries at `../../target/release/protoc-gen-connect-rust`. If you're iterating on codegen, run `task generate:all` after every change. CI detects stale generated code as compile errors in the `Check`/`Test` jobs, not as a dedicated stale-check. Sources: [Taskfile.yaml:368-376](), [Taskfile.yaml:119-123]()

### Local buffa override

When testing combined speculative changes across both repos:

```bash
task buffa:link    # writes .cargo/config.toml with path overrides to ../buffa
task buffa:unlink  # removes the override
```

The `.cargo/config.toml` override is gitignored. Sources: [Taskfile.yaml:98-109]()

---

## Sequence: How a `.proto` Becomes a Rust Trait

```mermaid
sequenceDiagram
    participant BR as build.rs / buf generate
    participant CB as connectrpc-build Config
    participant PC as protoc / buf
    participant CG as codegen::generate_files
    participant BC as buffa-codegen
    participant FS as $OUT_DIR / src/generated/

    BR->>CB: Config::new().files(...).compile()
    CB->>PC: run_protoc() / run_buf()
    PC-->>CB: FileDescriptorSet bytes
    CB->>CG: generate_files(fds, files, options)
    CG->>BC: buffa_codegen::generate(...)
    BC-->>CG: per-proto GeneratedFiles (Owned, View, Oneof, PackageMod)
    CG->>CG: emit_service_files() → __connect.rs Companion files
    CG->>CG: apply_companions() wires companions into PackageMod stitchers
    CG-->>CB: Vec<GeneratedFile>
    CB->>FS: write_if_changed() for each file
    CB->>FS: generate_include_file() → _inc.rs (if configured)
    CB->>BR: cargo:rerun-if-changed directives
```

The key invariant enforced in debug builds: every `Companion` (`.__connect.rs`) file must be referenced by an `include!` inside a matching `PackageMod` stitcher. If `apply_companions` orphans a companion, the service trait silently vanishes at use-site. Sources: [connectrpc-codegen/src/codegen.rs:155-200]()

---

## Summary

The connect-rust build pipeline revolves around two modes sharing one codegen core. The `connectrpc-build` crate handles the `build.rs` path end-to-end: it shells out to `protoc` or `buf`, decodes the descriptor set, calls `generate_files` for unified output, and writes everything to `$OUT_DIR`. The `protoc-gen-connect-rust` plugin handles checked-in regeneration via `buf generate`: it emits service stubs only (`__connect.rs` files) into a separate directory alongside buffa's message output, using `buffa_module=crate::proto` to resolve all type paths. The single `task generate:all` command rebuilds all plugin binaries from source and reruns `buf generate` in all four checked-in directories, ensuring the generated code stays synchronized with the codegen logic. Sources: [connectrpc-codegen/src/lib.rs:1-43]()