# Inside the PoC: Six Python Modules

> Line-by-line reality of the working code: TinyCausalModel, SovereignTrainingNode local training loop and artifact retention, ConsortiumCoordinator.run_round delta application, ContributionPolicy iterative capping, the exact messages, and the one test file that locks the invariants.

- Repository: The-AI-Alliance/tapestry
- GitHub: https://github.com/The-AI-Alliance/tapestry
- Human wiki: https://grok-wiki.com/public/wiki/the-ai-alliance-tapestry-4a40b7cf9eb6
- Complete Markdown: https://grok-wiki.com/public/wiki/the-ai-alliance-tapestry-4a40b7cf9eb6/llms-full.txt

## Source Files

- `src/tapestry/training/consortium/coordinator.py`
- `src/tapestry/training/consortium/node.py`
- `src/tapestry/training/consortium/policy.py`
- `src/tapestry/training/consortium/model.py`
- `src/tapestry/training/consortium/messages.py`
- `src/tapestry/training/consortium/types.py`
- `src/tests/tapestry/training/consortium/test_consortium_training.py`
- `examples/consortium_training_demo.py`

---

<details>
<summary>Relevant source files</summary>
The following files were used as context for generating this wiki page:
- [src/tapestry/training/consortium/model.py](src/tapestry/training/consortium/model.py)
- [src/tapestry/training/consortium/node.py](src/tapestry/training/consortium/node.py)
- [src/tapestry/training/consortium/coordinator.py](src/tapestry/training/consortium/coordinator.py)
- [src/tapestry/training/consortium/policy.py](src/tapestry/training/consortium/policy.py)
- [src/tapestry/training/consortium/messages.py](src/tapestry/training/consortium/messages.py)
- [src/tapestry/training/consortium/types.py](src/tapestry/training/consortium/types.py)
- [src/tapestry/training/consortium/__init__.py](src/tapestry/training/consortium/__init__.py)
- [src/tapestry/training/consortium/README.md](src/tapestry/training/consortium/README.md)
- [src/tests/tapestry/training/consortium/test_consortium_training.py](src/tests/tapestry/training/consortium/test_consortium_training.py)
- [examples/consortium_training_demo.py](examples/consortium_training_demo.py)
</details>

# Inside the PoC: Six Python Modules

This page provides a line-by-line examination of the working consortium-training proof-of-concept (PoC) in `src/tapestry/training/consortium/`. It is the only implemented training slice in the repository and directly embodies the N+1 outcome described in the ADRs: one governed shared base model plus N participant-owned sovereign model artifacts. Nodes train locally on sovereign corpora and return only weight deltas; the coordinator applies governed weights before integration.

The PoC is deliberately minimal—`torch` only, no distributed runtime, no production data pipelines—so that the core invariants (sovereign retention, delta-only sharing, quality floor, anti-capture cap, and unchanged base on total rejection) can be read, executed, and tested in minutes.

## The Six Modules at a Glance

| File | Primary Class / Type | Responsibility |
|------|----------------------|----------------|
| `model.py` | `TinyCausalModel` | Minimal next-token `nn.Module` used by every test and demo. |
| `node.py` | `SovereignTrainingNode` | Local continued-pretraining loop; retains full sovereign artifact; emits delta only. |
| `coordinator.py` | `ConsortiumCoordinator` | Orchestrates one round, calls nodes, invokes policy, applies weighted deltas. |
| `policy.py` | `ContributionPolicy` | Quality-floor filter + iterative per-node weight capping (anti-capture). |
| `messages.py` | Dataclasses + `ModelState` alias | Frozen records for artifacts, contributions, cycle results, round results. |
| `types.py` | Re-exports | Backward-compatibility shim; new code imports from `messages`. |

All public symbols are re-exported by `src/tapestry/training/consortium/__init__.py`.

## TinyCausalModel

```python:12:19:src/tapestry/training/consortium/model.py
class TinyCausalModel(nn.Module):
    """Small next-token model used to keep the PoC fast and testable."""

    def __init__(self, vocab_size: int = 256, hidden_size: int = 32) -> None:
        super().__init__()
        self.embedding = nn.Embedding(vocab_size, hidden_size)
        self.proj = nn.Linear(hidden_size, vocab_size)

    def forward(self, input_ids: torch.Tensor) -> torch.Tensor:
        """Return next-token logits for ``(batch, sequence)`` token ids."""
        return self.proj(self.embedding(input_ids))
```

The model is intentionally tiny (default hidden size 32) so that a full local epoch on a handful of sequences completes in milliseconds. Every node and the coordinator start from an identical copy of this architecture.

Sources: [src/tapestry/training/consortium/model.py:9-20]()

## SovereignTrainingNode – Local Training Loop and Artifact Retention

`SovereignTrainingNode` owns the sovereign side of the contract. On each round it receives the current base state, trains locally, keeps the resulting full model, and returns a delta plus metadata.

Key excerpts:

```python:51:78:src/tapestry/training/consortium/node.py
    def run_sovereign_cycle(self, round_num: int, base_state: ModelState) -> SovereignCycleResult:
        """Run local continued pretraining and return artifact plus delta."""
        starting_state = {name: tensor.clone() for name, tensor in base_state.items()}
        self.model.load_state_dict(starting_state)

        avg_loss = self._train_locally()
        sovereign_state = {name: tensor.clone() for name, tensor in self.model.state_dict().items()}
        delta = {name: sovereign_state[name] - starting_state[name] for name in starting_state}

        ...
        artifact = SovereignModelArtifact(...)
        self.latest_artifact = artifact
        ...
        return SovereignCycleResult(artifact=artifact, contribution=contribution)
```

The private training loop (`_train_locally`) builds an in-memory `DataLoader` from the caller's sovereign corpus (padded byte or token sequences), then runs AdamW + CrossEntropy for `local_epochs`. The node stores the complete post-training state in `latest_artifact` and in the returned `SovereignModelArtifact.model_state`; only the difference tensor is placed in the `SovereignContribution.weight_delta`.

Sources: [src/tapestry/training/consortium/node.py:51-78](), [src/tapestry/training/consortium/node.py:92-112]()

## ConsortiumCoordinator.run_round and Delta Application

The coordinator drives the N+1 cycle:

```python:38:66:src/tapestry/training/consortium/coordinator.py
    def run_round(self, nodes: Sequence[SovereignTrainingNode]) -> ConsortiumRoundResult:
        self._round += 1
        previous_state = self.shared_base_state

        cycle_results = [node.run_sovereign_cycle(self._round, previous_state) for node in nodes]
        for result in cycle_results:
            self.sovereign_artifacts[result.artifact.node_id] = result.artifact

        contributions = [result.contribution for result in cycle_results]
        weights = self.contribution_policy.weights(...)
        ...
        if weights:
            ...
            integrated_state = self._apply_weighted_deltas(previous_state, deltas_by_node, weights)
            self.base_model.load_state_dict(integrated_state)
        ...
```

The static applicator performs a weighted sum of deltas before adding them to the prior base:

```python:68:80:src/tapestry/training/consortium/coordinator.py
    @staticmethod
    def _apply_weighted_deltas(
        base_state: ModelState,
        deltas_by_node: dict[str, ModelState],
        weights: dict[str, float],
    ) -> ModelState:
        next_state: ModelState = {}
        for name, base_tensor in base_state.items():
            delta = torch.zeros_like(base_tensor)
            for node_id, weight in weights.items():
                delta = delta + deltas_by_node[node_id][name] * weight
            next_state[name] = base_tensor + delta
        return next_state
```

After a successful round the coordinator holds exactly one shared base plus one `SovereignModelArtifact` per participating node.

Sources: [src/tapestry/training/consortium/coordinator.py:38-66](), [src/tapestry/training/consortium/coordinator.py:68-80]()

## ContributionPolicy – Iterative Capping

The policy first drops any score below `quality_floor` (or ≤ 0). It then normalizes the survivors and repeatedly caps any node whose share would exceed `max_node_weight`:

```python:30:58:src/tapestry/training/consortium/policy.py
        weights = self._normalize(accepted)
        capped: dict[str, float] = {}
        remaining = dict(weights)
        remaining_mass = 1.0

        while remaining:
            over_cap = {
                node_id: weight
                for node_id, weight in remaining.items()
                if weight * remaining_mass > self.max_node_weight
            }
            if not over_cap:
                ...
                break

            for node_id in over_cap:
                capped[node_id] = self.max_node_weight
                remaining_mass -= self.max_node_weight
                remaining.pop(node_id)

            ...
            remaining = self._normalize(remaining)
```

A final renormalization absorbs any floating-point residue. The algorithm guarantees that no single node ever receives more than `max_node_weight` after the round and that accepted weights sum to 1.0.

Sources: [src/tapestry/training/consortium/policy.py:22-59]()

## Exact Messages and Types

All communication uses frozen dataclasses defined in `messages.py`:

- `ModelState = dict[str, torch.Tensor]` – canonical snapshot type.
- `SovereignModelArtifact` – full post-training state + jurisdiction + metrics; retained by the node and by the coordinator.
- `SovereignContribution` – the delta, quality score, token count, and round number sent to the coordinator.
- `SovereignCycleResult` – pairs the artifact (kept) with the contribution (shared).
- `ConsortiumRoundResult` – round number, before/after base states, accepted/rejected lists, and the final weights.

`types.py` exists solely for import-path compatibility; its `__all__` simply re-exports the five symbols from `messages`.

Sources: [src/tapestry/training/consortium/messages.py:9-52]()

## The Test File That Locks the Invariants

`src/tests/tapestry/training/consortium/test_consortium_training.py` contains exactly four tests that together enforce the PoC contract:

1. `test_sovereign_node_returns_artifact_and_weight_delta` – verifies a node keeps a full `model_state` in its artifact while the contribution carries a non-zero delta of the same keys.
2. `test_contribution_policy_applies_quality_floor_and_capture_cap` – exercises the floor filter and the iterative cap; asserts rejected nodes are absent and the sum of accepted weights ≈ 1.0.
3. `test_coordinator_maintains_n_plus_one_model_outcome` – after one round with two nodes, `coordinator.sovereign_artifacts` has length 2 and the shared base has changed.
4. `test_low_quality_contribution_does_not_update_shared_base` – when the policy rejects every contribution, `previous_base_state` and `shared_base_state` remain bitwise identical.

These tests are the executable specification of the N+1 guarantees.

Sources: [src/tests/tapestry/training/consortium/test_consortium_training.py:82-123](), [src/tests/tapestry/training/consortium/test_consortium_training.py:126-153]()

## Runnable Entry Point

`examples/consortium_training_demo.py` wires three synthetic jurisdictions (Vietnam, Switzerland, India), each with its own corpus and quality score, then runs a configurable number of rounds under a `max_node_weight=0.5` policy. It prints exactly the information a reader needs to see the governance decisions in action: accepted vs. rejected lists, final normalized weights, and the growing set of retained sovereign artifacts.

## Summary

The six modules implement a complete, self-contained, and test-locked demonstration of governed consortium training. A reader who starts at `TinyCausalModel`, follows `SovereignTrainingNode.run_sovereign_cycle`, watches `ConsortiumCoordinator.run_round` call the policy and `_apply_weighted_deltas`, and then runs the four tests will have verified every architectural claim the PoC makes. All subsequent engineering work on Tapestry's training subsystem begins from this concrete baseline.

Sources: [src/tapestry/training/consortium/README.md:1-35]()