# Develop and test

> uv/pytest workflow, coverage omit rules, validate and security CI, optional search-quality eval, and contributor constraints from AGENTS.md.

- Repository: mvanhorn/last30days-skill
- GitHub: https://github.com/mvanhorn/last30days-skill
- Human docs: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca
- Complete Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/llms-full.txt

## Source Files

- `AGENTS.md`
- `pyproject.toml`
- `tests/test_pipeline_v3.py`
- `.github/workflows/validate.yml`
- `.github/workflows/security.yml`
- `docs/search-quality-eval.md`
- `skills/last30days/scripts/evaluate_search_quality.py`

---

---
title: "Develop and test"
description: "uv/pytest workflow, coverage omit rules, validate and security CI, optional search-quality eval, and contributor constraints from AGENTS.md."
---

Local development centers on **Python 3.12+**, the **uv** toolchain (`pyproject.toml`, `uv.lock`), and a **pytest** suite under `tests/` that imports engine modules from `skills/last30days/scripts` via `tests/conftest.py`. Default CI on pull requests and `main` runs `uv run pytest` only; advisory security scans and optional retrieval-quality evaluation sit outside that blocking path.

## Prerequisites

| Requirement | Source |
|-------------|--------|
| Python ≥ 3.12 | `pyproject.toml` `requires-python` |
| [uv](https://docs.astral.sh/uv/) | `AGENTS.md`, `.github/workflows/validate.yml` |
| Virtualenv at `.venv/` | Created by `uv` on first `uv run` |

Runtime dependencies for the skill engine are stdlib-first (`[project] dependencies = []`). Dev tools live in `[dependency-groups] dev`: `pytest` and `pytest-cov`.

<Steps>
<Step title="Install toolchain and sync dev deps">

```bash
uv python install 3.12
uv sync --group dev
```

</Step>
<Step title="Run the full test suite">

```bash
uv run pytest
```

Default pytest options (`pyproject.toml`): quiet mode (`-q`), short tracebacks (`--tb=short`), `testpaths = ["tests"]`.

</Step>
<Step title="Optional: coverage report">

```bash
uv run pytest --cov
```

Vendor code under `skills/last30days/scripts/lib/vendor/` is omitted from coverage (see below).

</Step>
</Steps>

## Test layout and imports

`tests/conftest.py` prepends `skills/last30days/scripts` to `sys.path`, so tests import engine modules as `from lib import pipeline` (same layout as the engine at runtime).

The suite spans **94** `test_*.py` modules and **1600+** collected cases (pipeline, sources, CLI, rendering, env, competitors, watchlist, evaluator, and workflow contract tests). Representative offline pipeline coverage lives in `tests/test_pipeline_v3.py`, which exercises `pipeline.run(..., mock=True)` without live credentials.

### Targeted runs

```bash
uv run pytest tests/test_pipeline_v3.py
uv run pytest tests/test_dedupe_v3.py -k test_some_case
```

### Direct engine smoke test (dev only)

`AGENTS.md` treats the slash command as the product path; direct CLI invocation is for scripting, cron, and engine debugging:

```bash
python3 skills/last30days/scripts/last30days.py "test query" --emit=compact
```

## Coverage configuration

Coverage is configured under `[tool.coverage.run]` and `[tool.coverage.report]` in `pyproject.toml`:

| Setting | Value |
|---------|--------|
| `branch` | `true` |
| `source` | `skills/last30days/scripts`, `tests` |
| `omit` | `skills/last30days/scripts/lib/vendor/*`, `dist/*` |
| `skip_empty` | `true` (report) |
| `show_missing` | `true` (report) |

Vendored X client code (`lib/vendor/bird-search/`) is excluded so coverage reflects project-owned logic, not third-party trees.

## CI workflows

```mermaid
flowchart LR
  subgraph pr_main["PR / push to main"]
    V[validate.yml\nuv run pytest]
    S[security.yml\nadvisory scans]
  end
  subgraph tag["Tag push v*"]
    R[release.yml\nbuild-skill.sh → .skill]
  end
  pr_main --> V
  pr_main --> S
  tag --> R
```

### Validate (blocking)

`.github/workflows/validate.yml` runs on every pull request and push to `main`:

1. Checkout
2. `astral-sh/setup-uv@v5`
3. `uv python install 3.12`
4. `uv run pytest`

No search-quality eval, lint gate, or release build in this workflow.

### Security (advisory)

`.github/workflows/security.yml` runs on PR, push to `main`, and `workflow_dispatch`. Both jobs use **`continue-on-error: true`** until maintainers confirm a clean baseline; comments in the workflow describe flipping to blocking enforcement.

| Job | Tool | Purpose |
|-----|------|---------|
| `dependency-audit` | `uv export --locked --all-groups` → `pip-audit` | CVE visibility on locked dev deps |
| `secret-scan` | TruffleHog OSS `--only-verified` | Verified secrets in PR/push ranges |

`tests/test_security_workflow.py` locks the advisory policy (pip-audit and TruffleHog present, `continue-on-error: true`, contributor guidance in workflow comments). `AGENTS.md` requires not weakening this workflow without an explained PR rationale.

### Release (tags only)

`.github/workflows/release.yml` triggers on tags `v*`, runs `skills/last30days/scripts/build-skill.sh`, uploads `dist/last30days.skill` via `softprops/action-gh-release@v2`.

## Contract and policy tests

Beyond behavioral unit tests, several modules enforce repository invariants:

| Module | Enforces |
|--------|----------|
| `tests/test_plugin_contract.py` | Version alignment across `pyproject.toml`, `SKILL.md`, `.claude-plugin/plugin.json`, `gemini-extension.json`, marketplace shape; no removed `.codex-plugin/`; workflows must not reference legacy root `scripts/` paths |
| `tests/test_version_consistency.py` | `SKILL.md` double-quoted version, header badge, memory-dir env conventions |
| `tests/test_security_workflow.py` | Security workflow structure and `AGENTS.md` secret-hygiene mentions |

<Warning>
Avoid cross-file version pin tests that compare a bumped `SKILL.md` to a stale hardcoded path in another artifact — that pattern caused cascade CI failures across open PRs after releases (`docs/solutions/workflow-issues/release-consistency-test-cascade-2026-05-16.md`). Prefer single-source version reads or generated pins.
</Warning>

## Optional search-quality evaluation

`skills/last30days/scripts/evaluate_search_quality.py` compares two git revisions on ranked-candidate output. It is **not** part of default CI (`docs/solutions/architecture/search-quality-eval-manual-by-default-2026-05-10.md`): live API cost, latency, and judge non-determinism make it a maintainer-triggered check, not a per-PR gate.

### What it measures

**Deterministic stability** (baseline vs candidate):

- Jaccard overlap on ranked item keys
- Retention vs baseline
- Per-source counts and overlap

**Optional LLM judge** (when a Google/Gemini API key is available):

- `Precision@5`, `nDCG@5`, source-coverage recall over a judged union pool
- Judgments cached under `--output-dir` (default `tmp/search-quality`)

### Default topics

`fixtures/eval_topics.json` supplies eight reviewer topics (comparison, how_to, breaking_news, product, opinion, prediction, concept, factual). Override with `--topics-file`.

### CLI flags (script)

| Flag | Default | Role |
|------|---------|------|
| `--baseline` | `HEAD~1` | Git ref or label; materialized via `git worktree` unless `WORKTREE` |
| `--candidate` | `WORKTREE` | Same; `WORKTREE` uses the current checkout |
| `--search` | `""` | Passed to engine `--search` when set |
| `--output-dir` | `tmp/search-quality` | Metrics and judgment cache |
| `--judge-model` | `gemini-3.1-flash-lite` | Gemini judge model (`lib.providers.GEMINI_FLASH_LITE`) |
| `--timeout` | `240` | Per-topic subprocess timeout (seconds) |
| `--limit` | `20` | Ranked items considered per report |
| `--mock` | off | Forwards `--mock` to engine |
| `--quick` | off | Forwards `--quick` to engine |
| `--topics-file` | — | JSON list of `{topic, query_type}` |

Example:

```bash
uv run python skills/last30days/scripts/evaluate_search_quality.py \
  --baseline origin/main \
  --candidate WORKTREE \
  --quick
```

`create_eval_env()` clears `LAST30DAYS_CONFIG_DIR` and passes selected API keys from env/config (`GOOGLE_API_KEY`, `OPENAI_API_KEY`, `XAI_API_KEY`, `SCRAPECREATORS_API_KEY`, Bluesky/Truth Social tokens, etc.) so eval runs use a controlled config surface. See `docs/search-quality-eval.md` for judge key names and operational notes.

<Info>
Jaccard and retention are regression guards, not ground-truth quality. Judge metrics help compare revisions but do not replace a larger labeled benchmark.
</Info>

Unit tests in `tests/test_evaluator_v3.py` cover ranked-item extraction, worktree resolution, env construction, and summary writers without requiring a full live eval pass.

## Contributor constraints (`AGENTS.md`)

| Area | Rule |
|------|------|
| Product boundary | Agent Skills package; new engine flags need `SKILL.md` integration so harness models see them |
| `lib/__init__.py` | Bare package marker only — no eager imports |
| Install sync | `npx skills add . -g -y` copies a frozen skill tree; re-run after edits, or symlink `skills/last30days` → `~/.agents/skills/last30days` for live dev |
| Slash vs shell | Harness slash commands cannot pipe flags; use direct `python3 .../last30days.py` for shell mechanics |
| Secrets | Never commit real keys, cookies, or `.env`; use `lib/env.py` patterns; dummy values in tests/fixtures |
| Security CI | Do not disable `.github/workflows/security.yml` without PR explanation |
| `CONFIGURATION.md` | Update when adding env vars, CLI config flags, install patterns, or config precedence |
| Beta | Experimental work on private repo → `/last30days-beta`; public promotion via review PR |

## Skill install during development

To test harness behavior after skill edits:

```bash
npx skills add . -g -y
```

For iterative skill work without reinstalling:

```bash
ln -sfn "$PWD/skills/last30days" ~/.agents/skills/last30days
```

(run from repo root)

## Related pages

<CardGroup>
<Card title="Skill, engine, and harness" href="/skill-engine-harness">
Boundaries between SKILL.md, `last30days.py`, and multi-harness runtimes — context for why tests import `lib` from `scripts/`.
</Card>
<Card title="Research pipeline" href="/research-pipeline">
v3 orchestration exercised heavily by `test_pipeline_v3.py` and mock runs.
</Card>
<Card title="CLI reference" href="/cli-reference">
Direct engine flags (`--emit`, `--quick`, `--mock`) used in dev smoke tests and eval subprocesses.
</Card>
<Card title="Beta channel" href="/beta-channel">
Parallel `/last30days-beta` install path for pre-public experiments.
</Card>
<Card title="Troubleshooting" href="/troubleshooting">
`--diagnose` and runtime failure modes outside pytest.
</Card>
</CardGroup>