Agent-readable docs
last30days Documentation

Technical reference for the multi-source Agent Skills research package: slash-command and engine entry points, v3 retrieval pipeline, configuration layers, synthesis output contract, and operator workflows across 50+ agent harnesses.
Complete Markdown

# last30days Documentation

> Technical reference for the multi-source Agent Skills research package: slash-command and engine entry points, v3 retrieval pipeline, configuration layers, synthesis output contract, and operator workflows across 50+ agent harnesses.

## Context Links

- [Agent index](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/llms.txt)
- [Human interactive docs](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca)
- [GitHub repository](https://github.com/mvanhorn/last30days-skill)

## Repository Metadata

- Repository: mvanhorn/last30days-skill

- Generated: 2026-06-04T23:26:24.258Z
- Updated: 2026-06-05T19:21:29.664Z
- Runtime: Grok CLI
- Format: Documentation
- Pages: 22

## Page Index

- 01. [Overview](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/01-overview.md) - What the Skill exposes, primary slash-command vs engine paths, zero-config sources, and the shortest successful research path.
- 02. [Installation](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/02-installation.md) - Install via Claude Code marketplace, npx skills add for Agent Skills hosts, Hermes, live-edit symlinks, and version sync expectations.
- 03. [Quickstart](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/03-quickstart.md) - First /last30days invocation, first-run setup wizard signals, saved output location, and --diagnose verification.
- 04. [Skill, engine, and harness](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/04-skill-engine-and-harness.md) - Project-specific boundaries between the Agent Skills package, Python engine, and multi-harness runtimes that load the skill.
- 05. [Query types](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/05-query-types.md) - QUERY_TYPE classification (GENERAL, NEWS, PROMPTING, RECOMMENDATIONS, COMPARISON), intent parsing, and engine pre-flight refusal paths.
- 06. [Research pipeline](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/06-research-pipeline.md) - v3 orchestration: planner sub-queries, parallel source fan-out, fusion, clustering, dedupe, rerank, and depth profiles (--quick / default / --deep).
- 07. [Output contract](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/07-output-contract.md) - Badge line, LAWs voice contract, --emit modes (compact, json, context, md, html), evidence blocks, and footer pass-through rules.
- 08. [Configure sources](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/08-configure-sources.md) - Credential layers (.env paths, keychain), per-source API keys, INCLUDE_SOURCES, setup wizard auto-actions, and --diagnose availability checks.
- 09. [Model clients](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/09-model-clients.md) - Per-harness install patterns: Claude Code marketplace, npx skills -a targets, Codex/Cursor/Gemini, Hermes, OpenClaw, and stale-install avoidance.
- 10. [Comparison mode](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/10-comparison-mode.md) - --competitors discovery fan-out, --competitors-list and --competitors-plan JSON schema, per-entity Step 0.55 targeting, and comparison synthesis template.
- 11. [HTML briefs](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/11-html-briefs.md) - Shareable offline HTML artifacts via --emit=html, --synthesis-file, save paths under LAST30DAYS_MEMORY_DIR, and SKILL.md-driven save flow.
- 12. [Trend monitoring](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/12-trend-monitoring.md) - SQLite --store persistence, watchlist.py scheduling and delivery, briefing.py digests, and recommended baseline cadence.
- 13. [Per-client setup](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/13-per-client-setup.md) - Project-scoped .claude/last30days.env, save-dir and --save-suffix isolation, category peer subreddits, and recurring --competitors-plan templates.
- 14. [Configuration reference](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/14-configuration-reference.md) - Environment variables, .env lookup order, reasoning provider priority, web-backend priority, output paths, and trend-monitoring toggles.
- 15. [CLI reference](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/15-cli-reference.md) - last30days.py positional topic, all argparse flags, --emit choices, targeting flags, and direct CLI vs slash-command constraints.
- 16. [Data sources](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/16-data-sources.md) - Per-platform retrieval modules, keyless vs keyed sources, --search alias map, and engagement scoring inputs by source.
- 17. [Skill contract](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/17-skill-contract.md) - SKILL.md runtime authority: STEP 0 stale-clone guard, pre-flight protocol, engine invocation rules, and SKILL_DIR substitution.
- 18. [Comparison recipes](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/18-comparison-recipes.md) - Copy-paste comparison workflows: vs-topic phrasing, --competitors fan-out, and multi-entity Head-to-Head table expectations.
- 19. [Prompting recipes](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/19-prompting-recipes.md) - PROMPTING query workflows: community technique extraction, KEY PATTERNS output shape, and copy-paste prompt generation patterns.
- 20. [Troubleshooting](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/20-troubleshooting.md) - --diagnose probes, missing-source recovery, stale Claude marketplace clones, thin results, and documented failure modes.
- 21. [Develop and test](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/21-develop-and-test.md) - uv/pytest workflow, coverage omit rules, validate and security CI, optional search-quality eval, and contributor constraints from AGENTS.md.
- 22. [Beta channel](https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/22-beta-channel.md) - Parallel /last30days-beta install from the private repo, promotion workflow, and where client-specific customizations belong.

## Source File Index

- `.agents/plugins/marketplace.json`
- `.claude-plugin/marketplace.json`
- `.claude-plugin/plugin.json`
- `.github/workflows/security.yml`
- `.github/workflows/validate.yml`
- `AGENTS.md`
- `CHANGELOG.md`
- `CONCEPTS.md`
- `CONFIGURATION.md`
- `docs/how-search-works.md`
- `docs/releases/v3.0.9.md`
- `docs/search-quality-eval.md`
- `docs/solutions/architecture/search-quality-eval-manual-by-default-2026-05-10.md`
- `docs/solutions/workflow-issues/release-consistency-test-cascade-2026-05-16.md`
- `fixtures/eval_topics.json`
- `gemini-extension.json`
- `HERMES_SETUP.md`
- `pyproject.toml`
- `README.md`
- `skills/last30days/agents/openai.yaml`
- `skills/last30days/references/save-html-brief.md`
- `skills/last30days/scripts/briefing.py`
- `skills/last30days/scripts/evaluate_search_quality.py`
- `skills/last30days/scripts/last30days.py`
- `skills/last30days/scripts/lib/bird_x.py`
- `skills/last30days/scripts/lib/categories.py`
- `skills/last30days/scripts/lib/cluster.py`
- `skills/last30days/scripts/lib/competitors.py`
- `skills/last30days/scripts/lib/dedupe.py`
- `skills/last30days/scripts/lib/env.py`
- `skills/last30days/scripts/lib/fanout.py`
- `skills/last30days/scripts/lib/fusion.py`
- `skills/last30days/scripts/lib/github.py`
- `skills/last30days/scripts/lib/html_render.py`
- `skills/last30days/scripts/lib/pipeline.py`
- `skills/last30days/scripts/lib/planner.py`
- `skills/last30days/scripts/lib/polymarket.py`
- `skills/last30days/scripts/lib/preflight.py`
- `skills/last30days/scripts/lib/providers.py`
- `skills/last30days/scripts/lib/query.py`
- `skills/last30days/scripts/lib/reddit_public.py`
- `skills/last30days/scripts/lib/render.py`
- `skills/last30days/scripts/lib/rerank.py`
- `skills/last30days/scripts/lib/schema.py`
- `skills/last30days/scripts/lib/setup_wizard.py`
- `skills/last30days/scripts/lib/skill_meta.py`
- `skills/last30days/scripts/lib/youtube_yt.py`
- `skills/last30days/scripts/setup-keychain.sh`
- `skills/last30days/scripts/store.py`
- `skills/last30days/scripts/verify_v3.py`
- `skills/last30days/scripts/watchlist.py`
- `skills/last30days/SKILL.md`
- `tests/test_cli_competitors.py`
- `tests/test_competitor_fanout.py`
- `tests/test_pipeline_v3.py`
- `tests/test_regression.py`
- `tests/test_render_comparison_multi.py`
- `tests/test_watchlist_commands.py`

---

## 01. Overview

> What the Skill exposes, primary slash-command vs engine paths, zero-config sources, and the shortest successful research path.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/01-overview.md
- Generated: 2026-06-04T23:18:54.834Z

### Source Files

- `README.md`
- `CONCEPTS.md`
- `skills/last30days/SKILL.md`
- `skills/last30days/scripts/last30days.py`
- `AGENTS.md`
- `pyproject.toml`

---
title: "Overview"
description: "What the Skill exposes, primary slash-command vs engine paths, zero-config sources, and the shortest successful research path."
---

`/last30days` is an [Agent Skills](https://agentskills.io) package (`skills/last30days/`) whose runtime contract lives in `SKILL.md` (v3.3.1) and whose Python engine is `scripts/last30days.py`. The product surface is a harness-invoked slash command; the engine aggregates Reddit, Hacker News, Polymarket, GitHub, YouTube, optional social sources, and grounded web retrieval into ranked evidence the hosting model synthesizes under a fixed output contract (badge, LAWs voice rules, pass-through footer).

## Skill, engine, and harness

Three boundaries define every run:

| Layer | Artifact | Role |
|-------|----------|------|
| **Skill** | `skills/last30days/SKILL.md` + sibling `scripts/` | Tells the hosting model how to parse intent, run pre-flight (handles, subreddits, plans), invoke the engine, and format the user-facing brief |
| **Engine** | `skills/last30days/scripts/last30days.py` → `lib/pipeline.py` | Plans sub-queries, fans out parallel source retrieval, clusters/dedupes/reranks, emits `--emit` payloads |
| **Harness** | Claude Code, Codex, Cursor, Gemini CLI, Copilot, Hermes, OpenClaw, 50+ Agent Skills hosts | Loads the skill, exposes `/last30days`, and (on reasoning-model paths) acts as planner and synthesizer |

```mermaid
flowchart TB
  subgraph harness["Harness (agent runtime)"]
    SC["/last30days topic"]
    WS["WebSearch supplement"]
  end
  subgraph skill["Skill package skills/last30days/"]
    SM["SKILL.md contract"]
  end
  subgraph engine["Engine scripts/last30days.py"]
    PL["lib/pipeline.run"]
    REN["lib/render compact/md/html"]
  end
  subgraph sources["Retrieval modules lib/*"]
    RD["reddit_keyless"]
    HN["hackernews"]
    PM["polymarket"]
    GH["github"]
    YT["youtube_yt"]
    MORE["x, tiktok, grounding, …"]
  end
  SC --> SM
  SM -->|"Bash: SKILL_DIR/scripts/last30days.py"| PL
  PL --> RD & HN & PM & GH & YT & MORE
  PL --> REN
  REN -->|"stdout --emit=compact"| SM
  SM --> WS
  SM -->|"synthesis + footer pass-through"| OUT["User brief"]
```

<Note>
On a reasoning-model path, the harness model **is** the planner: named-entity topics require `--plan` written to a tmpfile (LAW 7). The engine’s internal LLM planner is for headless/cron runs without a hosting model.
</Note>

## What the skill exposes

| Surface | Entry | Typical use |
|---------|-------|-------------|
| **Slash command** | `/last30days <topic>` | Primary UX in Claude Code (`/plugin install last30days`) and Agent Skills hosts (`npx skills add mvanhorn/last30days-skill -g`) |
| **Direct CLI** | `python3 <SKILL_DIR>/scripts/last30days.py "<topic>"` | Scripting, cron, engine testing, `--diagnose`, HTML export with `--emit=html` |
| **Setup** | First run via SKILL.md Step 0 → `nux-wizard.md`, or `last30days.py setup` | Writes `~/.config/last30days/.env` with `SETUP_COMPLETE=true` |
| **Trend stack** (optional) | `--store`, `watchlist.py`, `briefing.py` | SQLite persistence and scheduled digests |

The skill frontmatter declares `user-invocable: true`, optional env keys (`SCRAPECREATORS_API_KEY`, `OPENROUTER_API_KEY`, X cookies, Bluesky app password, etc.), and required bins `node` + `python3` (Python **3.12+** enforced at engine startup).

### Slash command vs direct CLI

| Constraint | Slash command | Direct CLI |
|------------|---------------|------------|
| Shell flags/pipes | Invalid — harness does not pass `|`, `>`, or arbitrary flags through | Full `argparse` surface (`--emit`, `--plan`, `--competitors`, …) |
| Who plans | Hosting model generates `--plan` JSON for named entities | Optional `--plan` file; otherwise engine `providers.resolve_runtime` |
| Who synthesizes | Model follows SKILL.md LAWs on engine stdout | `--emit=compact` still targets model synthesis in skill flows; `html`/`json` can be consumed headlessly |
| Engine path | `SKILL_DIR` = directory of the `SKILL.md` the harness loaded | Caller supplies path (repo: `skills/last30days/scripts/last30days.py`) |

<Warning>
Treating `/last30days` as a generic “research the last 30 days” prompt and answering from WebSearch alone violates the skill contract: valid output always includes an engine run and the `✅ All agents reported back!` footer block.
</Warning>

## Zero-config and baseline sources

With no `.env` file, the engine still activates keyless or CLI-backed sources. `lib/pipeline.available_sources` always includes **Reddit** (public/keyless pipeline), **Hacker News**, and **Polymarket**. Additional sources join when tools or keys are present:

| Source | Activation without API keys in `.env` |
|--------|----------------------------------------|
| Reddit | Always (`reddit` — no `SCRAPECREATORS` required) |
| Hacker News | Always |
| Polymarket | Always |
| GitHub | `gh` on `PATH` (uses GitHub CLI auth) |
| YouTube | `yt-dlp` on `PATH` |
| Digg | `digg-pp-cli` on `PATH` |
| X / Twitter | `AUTH_TOKEN`+`CT0`, `XAI_API_KEY`, `SCRAPECREATORS_API_KEY`, `FROM_BROWSER`, or authenticated Bird/xurl |
| TikTok, Instagram, Threads | `SCRAPECREATORS_API_KEY` (+ `INCLUDE_SOURCES` where applicable) |
| Bluesky | `BSKY_HANDLE` + `BSKY_APP_PASSWORD` |
| Web grounding | `BRAVE_API_KEY`, `EXA_API_KEY`, `SERPER_API_KEY`, or `PARALLEL_API_KEY` |
| Perplexity | `OPENROUTER_API_KEY` and `INCLUDE_SOURCES` containing `perplexity` |

README’s “zero config” line matches this: Reddit, HN, Polymarket, and GitHub (with `gh`) work on first install; the first-run wizard optionally unlocks X, YouTube enrichment, TikTok, and ScrapeCreators-backed sources in one pass.

### Active source announcement

Before research, SKILL.md instructs the model to build `ACTIVE_SOURCES_LIST` from live checks (`which gh`, `which yt-dlp`, env keys, `EXCLUDE_SOURCES`) and emit a single branded line, for example:

```
/last30days - searching Reddit, Hacker News, Polymarket, and GitHub for what people are saying about {TOPIC}.
```

Only configured sources are named — the skill must not promise platforms that `--diagnose` would omit.

## Shortest successful research path

<Steps>
<Step title="Install the skill package">

<Tabs>
<Tab title="Claude Code (marketplace)">

```text
/plugin marketplace add mvanhorn/last30days-skill
/plugin install last30days
```

</Tab>
<Tab title="Agent Skills hosts">

```bash
npx skills add mvanhorn/last30days-skill -g
```

</Tab>
</Tabs>

</Step>

<Step title="Invoke with a concrete topic">

```text
/last30days OpenClaw
```

If `~/.config/last30days/.env` is missing, Step 0 runs `nux-wizard.md` once, then continues.

</Step>

<Step title="Let the contract run (automatic in skill mode)">

The hosting model should: load WebSearch (`ToolSearch select:WebSearch`), parse `QUERY_TYPE`, run Step 0.45 pre-flight, resolve handles/subreddits (Step 0.5 / 0.55), write `--plan` for named entities, and execute:

```bash
"${SKILL_DIR}/scripts/last30days.py" "<topic>" --emit=compact --plan "$QUERY_PLAN_FILE" ...
```

Stdout supplies badge + evidence blocks + footer; the model synthesizes `What I learned:` prose and passes the footer through verbatim.

</Step>

<Step title="Verify configuration (optional)">

```bash
python3 skills/last30days/scripts/last30days.py --diagnose
```

Prints JSON: `providers`, `available_sources`, `x_backend`, `native_web_backend`, `has_scrapecreators`, etc., without running a full search.

</Step>
</Steps>

### Saved artifacts

Default raw dumps land under `LAST30DAYS_MEMORY_DIR` (default `~/Documents/Last30Days/`) as `<slug>-raw.md`. The engine footer points to the saved path; HTML briefs use `--emit=html` or skill-driven save flow under the same directory.

### Query types (intent routing)

SKILL.md classifies input before retrieval shape and synthesis template:

| QUERY_TYPE | Trigger examples | Synthesis shape |
|------------|------------------|-----------------|
| GENERAL | Default topic research | Badge → `What I learned:` → KEY PATTERNS |
| NEWS | “latest on X”, “X news” | Same as GENERAL |
| RECOMMENDATIONS | “best X”, “top X” | Signal-weighted picks |
| PROMPTING | “X prompts”, “prompting for X” | Techniques + copy-paste prompts |
| COMPARISON | “X vs Y”, `--competitors` | Required `##` comparison sections + Head-to-Head table |

Depth profiles on the engine: `--quick`, default, `--deep` (maps to `DEPTH_SETTINGS` in `lib/pipeline.py`).

## Repository layout (skill product)

:::files
last30days-skill/
├── skills/last30days/
│   ├── SKILL.md          # Runtime authority for slash-command runs
│   ├── nux-wizard.md     # First-run setup (Step 0)
│   └── scripts/
│       ├── last30days.py # CLI entry
│       ├── lib/          # pipeline, sources, render, env
│       ├── watchlist.py
│       └── briefing.py
├── CONFIGURATION.md      # User-facing knobs (env, flags, paths)
├── CONCEPTS.md           # Skill / Engine / Harness vocabulary
└── pyproject.toml        # Python 3.12+, pytest suite
:::

<Info>
`AGENTS.md` states the distribution unit is the Agent Skills package, not a standalone CLI product. Engine flags without matching `SKILL.md` integration are considered incomplete for multi-harness behavior.
</Info>

## Engine output modes

<ParamField body="--emit" type="string" default="compact">
Choices: `compact`, `md`, `json`, `context`, `html`. Skill runs normally use `compact` for model synthesis; `html` produces shareable offline briefs; `json` exposes the structured `schema.Report`.
</ParamField>

<ParamField body="topic" type="string" required>
Positional words joined into the research topic. Empty topic with no `--diagnose` prints usage and exits 2.
</ParamField>

The compact emitter places ranked evidence inside `<!-- EVIDENCE FOR SYNTHESIS -->` comments; LAW 6 forbids dumping those clusters verbatim to the user.

## Related pages

<CardGroup>
<Card title="Installation" href="/installation">
Claude Code marketplace, `npx skills add`, Hermes, live-edit symlinks, and version sync.
</Card>
<Card title="Quickstart" href="/quickstart">
First invocation, setup wizard signals, saved output location, and `--diagnose`.
</Card>
<Card title="Skill, engine, and harness" href="/skill-engine-harness">
Boundaries between SKILL.md, `last30days.py`, and multi-harness runtimes.
</Card>
<Card title="Research pipeline" href="/research-pipeline">
v3 planner fan-out, fusion, clustering, dedupe, rerank, depth profiles.
</Card>
<Card title="Output contract" href="/output-contract">
Badge, LAWs, `--emit` modes, evidence blocks, footer pass-through.
</Card>
<Card title="Configure sources" href="/configure-sources">
`.env` layers, API keys, `INCLUDE_SOURCES`, setup wizard actions.
</Card>
</CardGroup>

---

## 02. Installation

> Install via Claude Code marketplace, npx skills add for Agent Skills hosts, Hermes, live-edit symlinks, and version sync expectations.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/02-installation.md
- Generated: 2026-06-04T23:19:06.798Z

### Source Files

- `README.md`
- `HERMES_SETUP.md`
- `.claude-plugin/plugin.json`
- `.claude-plugin/marketplace.json`
- `gemini-extension.json`
- `AGENTS.md`

---
title: "Installation"
description: "Install via Claude Code marketplace, npx skills add for Agent Skills hosts, Hermes, live-edit symlinks, and version sync expectations."
---

The distributable unit is the Agent Skills package at `skills/last30days/` (`SKILL.md` plus `scripts/last30days.py` and `scripts/lib/`). Harnesses load that package through marketplace plugins, `npx skills`, harness-native installers, or a manual symlink; the engine always runs from whichever directory contains the `SKILL.md` the model read (`SKILL_DIR`).

## Prerequisites

| Requirement | Used for | Notes |
|---|---|---|
| Python **3.12+** | Engine (`scripts/last30days.py`) | `requires-python = ">=3.12"` in `pyproject.toml`; Hermes docs recommend `python3.12` explicitly |
| **Node.js** | Vendored Bird client for X search | Listed under `metadata.openclaw.requires.bins` in `SKILL.md` |
| **yt-dlp** (optional) | YouTube search and transcripts | `brew install yt-dlp` or `pip install yt-dlp` |
| **gh** CLI (optional) | GitHub source | Uses your existing GitHub auth when present |

Reddit (public JSON), Hacker News, Polymarket, and GitHub (with `gh`) work with zero API configuration after install. Additional sources unlock via the first-run setup wizard or keys documented on [Configure sources](/configure-sources).

## Install surfaces

| Surface | Command | Update | On-disk layout (typical) |
|---|---|---|---|
| **Claude Code** (recommended) | `/plugin marketplace add mvanhorn/last30days-skill` then `/plugin install last30days` | Marketplace auto-refresh; `claude plugin update last30days@last30days-skill` | `~/.claude/plugins/cache/last30days-skill/last30days/{version}/skills/last30days/` |
| **Agent Skills hosts** (Codex, Cursor, Copilot, Gemini CLI, 50+ others) | `npx skills add mvanhorn/last30days-skill -g` | `npx skills update last30days -g` | `~/.agents/skills/last30days/` (copy; see sync below) |
| **Claude Code via Agent Skills CLI** | `npx skills add mvanhorn/last30days-skill -g -a claude-code` | Same as above | Per-host symlink under `~/.claude/skills/` pointing at `~/.agents/skills/last30days` when supported |
| **claude.ai** (web) | Download `last30days.skill` from latest release; upload in Settings | Re-download and re-upload | Hosted skill bundle (not local engine path) |
| **Hermes** | `hermes skills install mvanhorn/last30days-skill --force` | Re-run install, or `git pull` if symlinked | `~/.hermes/skills/research/last30days/` |
| **OpenClaw** | `clawhub install last30days-official` | `clawhub update last30days-official` | OpenClaw skill dir (see OpenClaw metadata in `SKILL.md`) |
| **Developer / live-edit** | `git clone` + symlink into a harness skill dir | `git pull` in the clone | Symlink targets `skills/last30days/` in your checkout |

Current release version is **3.3.1**, pinned consistently in `pyproject.toml`, `skills/last30days/SKILL.md` frontmatter, `.claude-plugin/plugin.json`, `.claude-plugin/marketplace.json`, and `gemini-extension.json` (enforced by `tests/test_plugin_contract.py`).

```text
skills/last30days/                 # canonical package in the repo
├── SKILL.md                       # runtime contract (slash command reads this)
└── scripts/
    ├── last30days.py              # Python engine
    └── lib/                       # search, render, env modules

Harness install → copy or symlink of skills/last30days/
Model sets SKILL_DIR = directory containing the SKILL.md it Read
Engine invoked as:  ${SKILL_DIR}/scripts/last30days.py
```

## Claude Code marketplace

<Steps>
<Step title="Add the marketplace">
In Claude Code:

```
/plugin marketplace add mvanhorn/last30days-skill
```

The marketplace manifest (`.claude-plugin/marketplace.json`) registers plugin name `last30days` with `source: "./"` from the public repo.
</Step>
<Step title="Install the plugin">
```
/plugin install last30days
```

Claude Code auto-discovers `skills/*/SKILL.md` when the plugin manifest omits a `"skills"` key (current `.claude-plugin/plugin.json` shape).
</Step>
<Step title="Verify version in cache">
```bash
cat ~/.claude/plugins/cache/last30days-skill/last30days/*/.claude-plugin/plugin.json | grep version
```

Expect `"version": "3.3.1"` (or whatever release you installed).
</Step>
</Steps>

<Tip>
Force an update check with `claude plugin update last30days@last30days-skill`. After upgrades, run `/plugin update last30days` then `/reload-plugins` if slash-command registration or `/doctor` errors persist.
</Tip>

### Stale marketplace clone (Claude Code only)

Claude Code maintains two plugin paths:

- **Versioned cache** — `~/.claude/plugins/cache/last30days-skill/last30days/{version}/` (preferred)
- **Marketplaces git clone** — `~/.claude/plugins/marketplaces/last30days-skill/` (can lag `origin/main`)

`SKILL.md` **STEP 0** tells the model: if the loaded path contains `/.claude/plugins/marketplaces/` and a newer cache `SKILL.md` exists, stop and re-read from the cache. Symptoms of a stale marketplace load include missing CLI flags (for example `--competitors`) and improvised comparison flows.

<Warning>
Do not run engine `--help` or plan against `marketplaces/` when the versioned cache is available. Users cannot fix this with `git pull` alone — rely on marketplace update or the STEP 0 re-read behavior baked into the skill contract.
</Warning>

### One install method per machine

The native marketplace plugin and `npx skills add … -a claude-code` can coexist. Claude Code does **not** dedupe slash commands across methods — both active installs surface two `/last30days` entries. Pick one path per machine.

## Agent Skills CLI (`npx skills`)

Default install for Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, Cline, Continue, Roo, OpenCode, Goose, and other [Agent Skills](https://agentskills.io) hosts:

```bash
npx skills add mvanhorn/last30days-skill -g
```

| Flag | Effect |
|---|---|
| `-g` | Global user install (`~/.agents/skills/last30days/`) — available in all projects |
| (omit `-g`) | Project-local install into `./.skills/` (committed with the repo) |
| `-a <harness>` | Target specific harness(es), e.g. `-a codex`, `-a cursor`, `-a gemini-cli` |
| `-y` | Non-interactive (used in contributor docs: `npx skills add . -g -y` from repo root) |

<Tabs>
<Tab title="Single harness">
```bash
npx skills add mvanhorn/last30days-skill -g -a codex
```
</Tab>
<Tab title="Multiple harnesses">
```bash
npx skills add mvanhorn/last30days-skill -g -a codex -a cursor
```
</Tab>
<Tab title="Claude Code via skills CLI">
```bash
npx skills add mvanhorn/last30days-skill -g -a claude-code
```
</Tab>
</Tabs>

Maintenance commands:

```bash
npx skills update last30days -g    # update this skill globally
npx skills update -g               # update all globally installed skills
npx skills list -g
npx skills remove last30days -g
```

<Note>
`.codex-plugin/` was removed from the repo; Codex should use `npx skills add` or a copy/symlink under `~/.codex/skills/last30days/`. `tests/test_plugin_contract.py` asserts the Codex plugin scaffold stays removed to avoid a forked install surface.
</Note>

## Hermes

Prerequisites: Hermes installed, Python 3.12+, optional `yt-dlp` for YouTube. See `HERMES_SETUP.md`.

<Steps>
<Step title="Install from GitHub">
```bash
hermes skills install mvanhorn/last30days-skill --force
```

Deploys to `~/.hermes/skills/research/last30days/`. `--force` reinstalls over an existing copy.
</Step>
<Step title="Invoke in Hermes">
```
last30days "your research topic"
```

Options pass through to the engine, e.g. `last30days "AI news" --days=7 --deep`.
</Step>
<Step title="Diagnose from install dir">
```bash
cd ~/.hermes/skills/research/last30days
python3.12 scripts/last30days.py --diagnose
```
</Step>
</Steps>

### Hermes live-edit symlink

```bash
git clone https://github.com/mvanhorn/last30days-skill.git
mkdir -p ~/.hermes/skills/research
ln -s "$(pwd)/last30days-skill/skills/last30days" ~/.hermes/skills/research/last30days
```

Edits in the clone propagate immediately; update with `git pull` instead of re-running `hermes skills install`.

## claude.ai (web)

<Steps>
<Step title="Enable code execution">
In [claude.ai Settings → Capabilities](https://claude.ai/settings/capabilities), enable **Code execution and file creation** — skills do not run without it.
</Step>
<Step title="Download the bundle">
[Download `last30days.skill`](https://github.com/mvanhorn/last30days-skill/releases/latest/download/last30days.skill) from the latest GitHub release.
</Step>
<Step title="Upload">
Settings → Capabilities → Skills → `+`, drop the file.
</Step>
</Steps>

Maintainers rebuild the upload artifact from a clean tree:

```bash
bash skills/last30days/scripts/build-skill.sh
```

Produces `dist/last30days.skill` (zip with top-level `last30days/`, capped at 200 files for claude.ai). Requires a clean git working tree.

## OpenClaw

```bash
clawhub install last30days-official
clawhub update last30days-official
```

`SKILL.md` includes `metadata.openclaw` (optional env keys, required bins `node` and `python3`, `primaryEnv: SCRAPECREATORS_API_KEY`).

## Developer install and live-edit symlinks

For contributors hacking the repo, two patterns matter: **frozen copy** vs **live symlink**.

### Frozen copy (`npx skills` default)

From the repository root:

```bash
npx skills add . -g -y
```

This copies `skills/last30days` into `~/.agents/skills/last30days/`. **Working-tree edits do not propagate** until you re-run `npx skills add . -g -y`.

### Live symlink (recommended for local dev)

From the repository root:

```bash
ln -sfn "$PWD/skills/last30days" ~/.agents/skills/last30days
```

Equivalent manual paths from the README:

```bash
git clone https://github.com/mvanhorn/last30days-skill.git
ln -s "$(pwd)/last30days-skill/skills/last30days" ~/.claude/skills/last30days
```

Symlinked installs track `git pull` in the clone with no re-copy step.

<Check>
After either pattern, confirm the engine exists:

```bash
test -f ~/.agents/skills/last30days/scripts/last30days.py && echo OK
```

For Claude marketplace cache, substitute the resolved `SKILL_DIR` under `~/.claude/plugins/cache/.../skills/last30days/`.
</Check>

## Version sync expectations

| Install model | When you get new releases | What to run |
|---|---|---|
| Claude Code marketplace | Auto-refresh on new GitHub releases; versioned cache directory per release | `claude plugin update last30days@last30days-skill` or `/plugin update last30days` |
| `npx skills add` (copy) | **Not** automatic — install is a snapshot | `npx skills update last30days -g` or re-run `npx skills add mvanhorn/last30days-skill -g` |
| Git symlink / repo checkout | Immediate for local edits; upstream fixes via git | `git pull` in the clone |
| claude.ai `.skill` upload | Manual | Re-download release artifact and re-upload |
| Hermes `hermes skills install` | On demand | `hermes skills install mvanhorn/last30days-skill --force` |
| OpenClaw ClawHub | On demand | `clawhub update last30days-official` |

Manifest version drift historically broke Claude marketplace resolution (marketplace.json lagging plugin.json). CI now requires `pyproject.toml`, `SKILL.md`, `plugin.json`, `marketplace.json` plugins[0].version, and `gemini-extension.json` to match.

The synthesizing model reports installed version in the mandatory badge (`🌐 last30days v{VERSION} · synced {YYYY-MM-DD}`), resolved from `SKILL_DIR/../../.claude-plugin/plugin.json` or `SKILL.md` frontmatter.

## Verify installation

| Check | Command / signal |
|---|---|
| Slash command registered | `/last30days` appears once (not duplicated) in your harness |
| Engine reachable | `python3 skills/last30days/scripts/last30days.py --diagnose` from `SKILL_DIR` |
| Sources available | `--diagnose` prints per-source keys, CLIs, and backends without a full search |
| Version alignment | Marketplace cache `plugin.json` version matches `SKILL.md` `version:` frontmatter |

First successful `/last30days` invocation triggers the setup wizard (browser cookie scan for X, `yt-dlp` check, optional ScrapeCreators signup). See [Quickstart](/quickstart).

## Beta channel (parallel install)

Experimental work ships from private repo `mvanhorn/last30days-skill-private` as slash command `/last30days-beta`. Install and promotion workflow live in `BETA.md` (private repo). Do not ship beta-only changes to the public marketplace without a review PR here.

## Related pages

<CardGroup>
<Card title="Quickstart" href="/quickstart">
First `/last30days` run, setup wizard signals, saved output location, and `--diagnose`.
</Card>
<Card title="Model clients" href="/model-clients">
Per-harness install targets, `-a` flags, and stale-install avoidance in depth.
</Card>
<Card title="Configure sources" href="/configure-sources">
API keys, `.env` lookup order, and source enablement after install.
</Card>
<Card title="Troubleshooting" href="/troubleshooting">
`--diagnose`, missing sources, stale marketplace clones, and thin results.
</Card>
<Card title="Skill, engine, and harness" href="/skill-engine-harness">
Boundaries between the Agent Skills package, Python engine, and harness runtimes.
</Card>
<Card title="Beta channel" href="/beta-channel">
Parallel `/last30days-beta` install and promotion workflow.
</Card>
</CardGroup>

---

## 03. Quickstart

> First /last30days invocation, first-run setup wizard signals, saved output location, and --diagnose verification.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/03-quickstart.md
- Generated: 2026-06-04T23:19:11.649Z

### Source Files

- `skills/last30days/SKILL.md`
- `skills/last30days/scripts/lib/setup_wizard.py`
- `skills/last30days/scripts/last30days.py`
- `CONFIGURATION.md`
- `README.md`

---
title: "Quickstart"
description: "First /last30days invocation, first-run setup wizard signals, saved output location, and --diagnose verification."
---

The `/last30days` slash command loads `skills/last30days/SKILL.md`, runs Python 3.12+ preflight, optionally executes Step 0 first-run setup, then invokes `scripts/last30days.py` with `--emit=compact` and `--save-dir` pointed at `LAST30DAYS_MEMORY_DIR` (default `~/Documents/Last30Days/`). Reddit, Hacker News, Polymarket, and GitHub (when `gh` is installed) work with no API keys on the first run.

## Prerequisites

| Requirement | Detail |
|---|---|
| Skill installed | Claude Code marketplace, `npx skills add`, Hermes, or another Agent Skills host — see [Installation](/installation) |
| Python | 3.12+ (`python3.12`, `python3.13`, or `python3.14`); the skill resolves `LAST30DAYS_PYTHON` before any engine call |
| Optional keys | None required for the zero-config sources above |

<Note>
The product path is the slash command in your harness. Direct CLI (`python3 …/last30days.py`) is for scripting, cron, and verification — not a substitute for `/last30days` in chat.
</Note>

## First invocation

<Steps>
<Step title="Install the skill">
Install once per machine (marketplace plugin or `npx skills add mvanhorn/last30days-skill -g`). Re-run install or update when syncing a dev checkout.
</Step>
<Step title="Run a topic">
In Claude Code, Codex, Cursor, Gemini CLI, or any host that exposes the skill:

```
/last30days OpenClaw
```

The hosting model reads `SKILL.md`, parses intent (GENERAL by default), builds `ACTIVE_SOURCES_LIST` from your config, and prints a one-line confirmation such as:

```
/last30days - searching Reddit, Hacker News, Polymarket, and GitHub for what people are saying about OpenClaw.
```

It then runs pre-research (when WebSearch is available), plans sub-queries, and calls the engine with `--emit=compact --save-dir="${LAST30DAYS_MEMORY_DIR}" --save-suffix=v3`.
</Step>
<Step title="Read the response">
Valid output starts with the engine badge (`🌐 last30days v{VERSION} · synced {YYYY-MM-DD}`), then `What I learned:` (or the comparison title for `vs` topics), KEY PATTERNS, the pass-through emoji-tree footer, and an invitation — no trailing `Sources:` block.
</Step>
</Steps>

```mermaid
sequenceDiagram
  participant User
  participant Harness as Agent harness
  participant Skill as SKILL.md contract
  participant Engine as last30days.py

  User->>Harness: /last30days topic
  Harness->>Skill: Read + Step 0 check
  alt First run
    Skill->>Harness: Setup wizard (nux-wizard flow)
    Harness->>Engine: optional setup subcommand
  end
  Harness->>Engine: --emit=compact --save-dir=MEMORY_DIR
  Engine-->>Harness: compact stdout + footer save path
  Harness-->>User: Synthesis + saved raw path
```

## First-run setup wizard

Setup splits into **detection** (Python), **UI** (hosting model via `SKILL.md`), and **auto-actions** (engine `setup` subcommand).

### Detection signals

| Signal | Meaning |
|---|---|
| `~/.config/last30days/.env` missing | Treated as first run (`SKILL.md` Step 0) |
| `SETUP_COMPLETE` absent or empty in config | `setup_wizard.is_first_run()` returns true |
| `SETUP_COMPLETE=true` present | Step 0 skipped silently — no setup banner on every run |

Config merge order: process environment → `.claude/last30days.env` (walked up from cwd) → `~/.config/last30days/.env` (override with `LAST30DAYS_CONFIG_DIR`).

### What the hosting model does (Step 0)

On first run the agent must:

1. Read `skills/last30days/nux-wizard.md` (relative to the skill root) and follow platform-specific flow (Claude Code vs OpenClaw, ScrapeCreators opt-in, topic picker).
2. After setup, ensure `SETUP_COMPLETE=true` is written to `~/.config/last30days/.env`.
3. Proceed to intent parsing and research without repeating setup on later runs.

<Warning>
`nux-wizard.md` is referenced in `SKILL.md` Step 0 as the wizard script. If that file is missing from your install tree, first-run UX falls back to manual configuration — use the engine `setup` subcommand and [Configure sources](/configure-sources) instead.
</Warning>

### Engine auto-setup (`setup` subcommand)

The model or you can run auto-setup directly (dev/fallback path):

```bash
# From the directory that contains scripts/last30days.py
python3 scripts/last30days.py setup
```

`run_auto_setup()` then:

- Extracts browser cookies for registered domains (X/Twitter and related cookie-jar sources) via `FROM_BROWSER=auto`
- Checks `yt-dlp`; on macOS with Homebrew, may run `brew install yt-dlp`
- Writes `SETUP_COMPLETE=true` and `FROM_BROWSER={browser}` to `~/.config/last30days/.env` (append-only, no overwrite of existing keys)

Optional flags after `setup` (parsed as extra argv): `--openclaw`, `--github`, `--device-auth` for platform-specific auth flows.

### Status text you should recognize

stderr/stdout from `setup` ends with a summary from `get_setup_status_text()`, for example:

<RequestExample>

```text
Setup complete! Here's what I found:

  - X cookies found in chrome
  - yt-dlp already installed

Configuration saved. Future runs will auto-detect your browsers.
```

</RequestExample>

Other branches include `No browser cookies found for X/Twitter`, `Installed yt-dlp via Homebrew`, `yt-dlp install failed — run brew install yt-dlp manually`, and `yt-dlp not found. Install Homebrew first…`.

<Info>
README describes “run it once and the setup wizard unlocks X, YouTube, TikTok, and more in 30 seconds” — that unlock is cookie extraction + `yt-dlp` plus optional ScrapeCreators keys, not a separate product install.
</Info>

## Where output is saved

| Artifact | Path pattern | When |
|---|---|---|
| Raw research dump | `{LAST30DAYS_MEMORY_DIR}/{slug}-raw[-suffix].md` | Every agent-mode run (`--save-suffix=v3` in `SKILL.md`) |
| Same-day collision | `{slug}-raw[-suffix]-{YYYY-MM-DD}.md` | If the target file already exists |
| HTML brief (optional) | `{LAST30DAYS_MEMORY_DIR}/{slug}-brief.html` | User asks for shareable HTML |
| Engine pointer | Footer line `📎 Raw results saved to …` | Confirms resolved path in chat |

<ParamField body="LAST30DAYS_MEMORY_DIR" type="string">
Default `~/Documents/Last30Days/` on Linux/macOS; `C:\Users\<you>\Documents\Last30Days\` on Windows. Set before invoking the skill to redirect all saves.
</ParamField>

<ParamField body="--save-dir" type="path">
Per-run override passed by the skill or CLI. Creates the directory if needed.
</ParamField>

<ParamField body="--save-suffix" type="string">
Distinguishes parallel runs of the same topic (e.g. `v3`, client slug).
</ParamField>

Slug rules: topic lowercased, non-alphanumerics folded to hyphens (`save_output` / `slugify` in `last30days.py`).

```text
~/Documents/Last30Days/
├── openclaw-raw-v3.md          # default agent run
├── openclaw-brief.html         # optional HTML export
└── peter-steinberger-raw.md    # person-topic example
```

Step 2.5 in `SKILL.md` requires the hosting model to append WebSearch supplements into the saved raw file under `## WebSearch Supplemental Results` — the on-disk file is the durable citation surface, not only chat output.

## Verify with `--diagnose`

`--diagnose` runs availability probes only — no topic search, no synthesis.

```bash
python3 skills/last30days/scripts/last30days.py --diagnose
```

<ResponseExample>

```json
{
  "available_sources": ["reddit", "hackernews", "polymarket", "github", "grounding"],
  "bird_authenticated": false,
  "bird_installed": true,
  "bird_username": null,
  "has_github": true,
  "has_scrapecreators": false,
  "local_mode": true,
  "native_web_backend": null,
  "providers": {
    "google": false,
    "openai": false,
    "xai": false,
    "openrouter": false
  },
  "reasoning_provider": "auto",
  "x_backend": null
}
```

</ResponseExample>

| Field | Use when verifying |
|---|---|
| `available_sources` | Which platforms the next run can query |
| `providers` / `local_mode` | Whether headless planning needs API keys (`local_mode: true` → deterministic fallback unless the host passes `--plan`) |
| `x_backend`, `bird_*` | X/Twitter auth path (cookies, Bird CLI, xAI, ScrapeCreators) |
| `has_scrapecreators` | TikTok, Instagram, Threads unlock |
| `native_web_backend` | First configured Brave → Exa → Serper → Parallel key |

<Check>
After setup, confirm `youtube` appears in `available_sources` when `yt-dlp` is on PATH, and `x` when cookies or `XAI_API_KEY` / Bird auth is configured. Missing sources with no keys is expected — add credentials per [Configure sources](/configure-sources).
</Check>

`verify_v3.py` in the repo uses the same `--diagnose` call as a release gate.

## Direct CLI smoke test (optional)

For engine testing without the skill contract:

```bash
cd skills/last30days/scripts
python3 last30days.py "test query" --emit=compact --mock
```

Full research requires the skill’s pre-flight flags (`--plan`, handles, subreddits). A bare `python3 last30days.py "Topic"` from cron is supported but uses the engine’s internal planner, not the hosting model’s Step 0.55 / 0.75 resolution.

## Common first-run outcomes

| Outcome | Cause | Next step |
|---|---|---|
| Thin social coverage | No X / TikTok / YouTube keys yet | Run `setup` or add keys; re-check `--diagnose` |
| `## Pre-Research Status` in output | Model skipped Step 0.55 on a named entity | Re-run via slash command with WebSearch; see [Skill contract](/skill-contract) |
| Stale behavior / missing flags | Claude marketplace clone lagging cache | `SKILL.md` STEP 0 stale-clone guard — see [Troubleshooting](/troubleshooting) |
| Permission warning on `.env` | File mode looser than `600` | `chmod 600 ~/.config/last30days/.env` |

## Related pages

<CardGroup>
<Card title="Installation" href="/installation">
Marketplace, `npx skills add`, Hermes, and live-edit symlinks.
</Card>
<Card title="Overview" href="/overview">
Slash command vs engine paths and zero-config sources.
</Card>
<Card title="Configure sources" href="/configure-sources">
API keys, `.env` layers, and wizard auto-actions in depth.
</Card>
<Card title="CLI reference" href="/cli-reference">
All `last30days.py` flags including `--emit` and `--save-*`.
</Card>
<Card title="Troubleshooting" href="/troubleshooting">
`--diagnose` recovery, thin results, and stale installs.
</Card>
</CardGroup>

---

## 04. Skill, engine, and harness

> Project-specific boundaries between the Agent Skills package, Python engine, and multi-harness runtimes that load the skill.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/04-skill-engine-and-harness.md
- Generated: 2026-06-04T23:19:02.027Z

### Source Files

- `CONCEPTS.md`
- `skills/last30days/SKILL.md`
- `skills/last30days/scripts/last30days.py`
- `AGENTS.md`
- `skills/last30days/scripts/lib/skill_meta.py`

---
title: "Skill, engine, and harness"
description: "Project-specific boundaries between the Agent Skills package, Python engine, and multi-harness runtimes that load the skill."
---

`mvanhorn/last30days-skill` ships as an [Agent Skills](https://agentskills.io) package under `skills/last30days/`: `SKILL.md` is the runtime contract the hosting model follows, and `scripts/last30days.py` plus `scripts/lib/` perform retrieval, fusion, and rendering. The product surface is harness-invoked research (`/last30days <topic>` on most hosts); direct `python3 …/last30days.py` is a dev, cron, and scripting fallback only.

## Three layers

| Layer | Location | Owns | Does not own |
|-------|----------|------|----------------|
| **Skill** | `skills/last30days/SKILL.md` + sibling `scripts/`, `references/`, `assets/` | Slash-command UX, pre-flight (Step 0–0.75), query planning on reasoning hosts, synthesis LAWs, which engine flags to pass | Platform HTTP clients, clustering, dedupe, rerank |
| **Engine** | `scripts/last30days.py`, `scripts/lib/*` | v3 pipeline (`pipeline.run`), `--emit` output shapes, `--diagnose`, SQLite `--store`, headless `--auto-resolve` / internal planner | User-facing prose contract, WebSearch supplements, invitation copy |
| **Harness** | Claude Code, Codex, Cursor, Copilot, Gemini CLI, Hermes, OpenClaw, 50+ Agent Skills hosts | Loads `SKILL.md`, exposes tools (Bash, Read, WebSearch), routes `/last30days` | Research algorithms, source scoring |

<Note>
Project vocabulary is fixed in `CONCEPTS.md`: **Skill** = distribution unit; **Engine** = Python implementation; **Harness** = agent runtime that loads the Skill. “Multi-harness” means a feature must work on every install path the Skill supports—not only Claude Code.
</Note>

```mermaid
flowchart TB
  subgraph harness["Harness (Claude Code, Codex, Cursor, Hermes, …)"]
    SC["/last30days slash command"]
    M["Reasoning model"]
    WS["WebSearch / Read / Bash tools"]
    SC --> M
    M --> WS
  end

  subgraph skill["Skill package skills/last30days/"]
    SM["SKILL.md contract"]
    SM -->|"SKILL_DIR/scripts/last30days.py"| ENG
  end

  subgraph engine["Engine scripts/"]
    ENG["last30days.py"]
    LIB["lib/ pipeline · render · env · …"]
    ENG --> LIB
    LIB --> SRC["Reddit · X · YouTube · …"]
  end

  M -->|"Read SKILL.md"| SM
  WS -->|"Bash invoke"| ENG
  ENG -->|"--emit=compact stdout"| M
  M -->|"synthesis + LAW pass-through"| U["User response"]
```

## Skill package layout

Every install layout keeps **`SKILL.md` and `scripts/` as siblings** under one directory (`SKILL_DIR`). The engine entrypoint is always `${SKILL_DIR}/scripts/last30days.py`.

:::files
skills/last30days/
├── SKILL.md              # Runtime spec (~1400+ lines); harness reads this on invoke
├── scripts/
│   ├── last30days.py     # CLI entry (Python 3.12+)
│   ├── lib/              # pipeline, render, env, per-source modules
│   ├── briefing.py       # Trend-monitoring digests (optional)
│   ├── watchlist.py      # Scheduled runs (optional)
│   └── store.py          # SQLite persistence (optional)
├── references/           # Supplemental skill docs (e.g. HTML brief flow)
└── assets/               # Static assets referenced from SKILL.md
:::

The Skill conforms to the Agent Skills open format (`name`, `version`, `description`, `allowed-tools`, `metadata` in YAML frontmatter). Version strings in output come from `SKILL.md` frontmatter via `lib/skill_meta.py`, with an optional `.claude-plugin/plugin.json` walk in `lib/render.py` for marketplace cache installs.

Distribution paths include:

- **Claude Code marketplace** — plugin cache under `~/.claude/plugins/cache/last30days-skill/…` (versioned) plus a known-stale `~/.claude/plugins/marketplaces/` clone (STEP 0 guard).
- **`npx skills add`** — copies into `~/.agents/skills/last30days/` (frozen until re-run); per-host symlinks when supported.
- **Per-harness dirs** — e.g. `~/.codex/skills/last30days/`, `~/.hermes/skills/research/last30days/`.
- **Repo checkout** — `skills/last30days/` in the working tree for development.

<Warning>
Working-tree edits do **not** propagate to `~/.agents/skills/` automatically. Re-run `npx skills add . -g -y` or symlink the working tree for live dev (`ln -sfn "$PWD/skills/last30days" ~/.agents/skills/last30days`).
</Warning>

## Engine responsibilities

`last30days.py` is the sole supported engine entry. It requires **Python 3.12+**, inserts `SCRIPT_DIR` on `sys.path`, and delegates to `lib.pipeline` for orchestration (planner input, parallel source fan-out, fusion, clustering, dedupe, rerank, depth profiles via `--quick` / default / `--deep`).

| Concern | Engine module / surface |
|---------|-------------------------|
| Config & keys | `lib/env.py`, `.env` / keychain patterns |
| Retrieval | `lib/reddit.py`, `lib/bird_x.py`, `lib/youtube_yt.py`, … |
| Output wire format | `lib/render.py` (`--emit=compact|md|json|context|html`) |
| Badge line | `render._render_badge()` — first line of compact/md stdout |
| Class-1 refusal | `lib/preflight.py` before pipeline |
| Headless planning | `lib/planner.py` + `--auto-resolve` when no host WebSearch |
| Version string | `lib/skill_meta.read_skill_version` + `render._skill_version()` |

`lib/__init__.py` is intentionally a bare package marker (no eager imports)—contributor constraint from `AGENTS.md`.

Direct CLI invocation (fallback only):

```bash
python3 skills/last30days/scripts/last30days.py "topic" --emit=compact --diagnose
```

Headless/cron paths may rely on engine `--plan` omission (internal/deterministic planner) or `--auto-resolve` when the harness lacks WebSearch—documented in-engine as the OpenClaw / raw CLI equivalent of SKILL.md Steps 0.55 and 0.75.

## Harness responsibilities

The harness loads `SKILL.md` when the user runs `/last30days <topic>` (or host-specific equivalent). It does **not** forward raw shell syntax from the slash line: pipes, inline flags, and `| pbcopy` on `/last30days …` are invalid; the model maps intent to engine flags, or the operator uses the direct CLI with a real shell.

On reasoning harnesses (Claude Code, Codex with WebSearch, Gemini, Hermes, etc.):

1. **Read** the Skill contract (including STEP 0 stale-clone check for Claude marketplace paths).
2. **Plan** — generate `QUERY_PLAN_JSON` and Step 0.55 targeting; pass `--plan` via a tmpfile (never inline quoted JSON).
3. **Invoke** — Bash `${LAST30DAYS_PYTHON} "${SKILL_DIR}/scripts/last30days.py" … --emit=compact`.
4. **Supplement** — WebSearch steps SKILL.md defines post-engine.
5. **Synthesize** — transform evidence blocks; pass through badge and emoji-tree footer per LAWs.

`SKILL_DIR` is the directory of the **SKILL.md the harness actually loaded**—not a searched precedence list. That keeps spec and code aligned across install layouts without enumerating every host path.

```mermaid
sequenceDiagram
  participant U as User
  participant H as Harness
  participant M as Model
  participant S as SKILL.md
  participant E as last30days.py

  U->>H: /last30days topic
  H->>M: Load skill + tools
  M->>S: Read (STEP 0 if Claude marketplace)
  M->>M: Step 0.55 / 0.75 plan + resolve
  M->>E: Bash SKILL_DIR/scripts/last30days.py --plan … --emit=compact
  E-->>M: Badge + evidence + footer stdout
  M->>M: WebSearch supplements (if available)
  M-->>U: Synthesis + pass-through footer
```

### Platform branch

| Harness capability | Skill path | Engine flags |
|--------------------|------------|--------------|
| WebSearch available | Steps 0.55 + 0.75 mandatory; model is planner (LAW 7) | `--plan "$QUERY_PLAN_FILE"`, targeting flags, `--emit=compact` |
| No WebSearch (OpenClaw, some Codex) | Skip 0.55/0.75 | `--auto-resolve` |
| Agent mode (`--agent` in args) | Skip intro, AskUserQuestion, invitation | Same engine run; structured report template |

Resolve once per session:

- `LAST30DAYS_PYTHON` — Python 3.12+ interpreter for all engine calls.
- `LAST30DAYS_MEMORY_DIR` — raw save root (default `~/Documents/Last30Days`); passed as `--save-dir`.

## Skill ↔ engine contract

The contract is bidirectional: **SKILL.md → flags**, **engine → output shape**.

### SKILL.md → engine

The model must translate user intent into explicit flags the engine understands, including:

| Flag family | Purpose |
|-------------|---------|
| `--plan` | External JSON query plan (reasoning-model path) |
| `--x-handle`, `--subreddits`, `--github-user`, `--github-repo`, TikTok/IG flags | Step 0.55 targeting |
| `--competitors`, `--competitors-plan` | Comparison / vs-mode fan-out |
| `--emit` | Wire format (`compact` is the primary user-facing mode) |
| `--save-dir`, `--save-suffix` | Artifact paths |
| `--quick` / `--deep` | Depth profiles |
| `--diagnose` | Source availability probe (verification) |

<Info>
A new engine flag without a matching `SKILL.md` section is considered **incomplete**: the invoking model will not know the flag exists. Feature design starts from slash-command UX, per `AGENTS.md`.
</Info>

### Engine → model

Default `--emit=compact` stdout includes:

- **Badge** — `🌐 last30days v{version} · synced {date}` (engine-emitted since v3.0.8).
- **Evidence for synthesis** — `<!-- EVIDENCE FOR SYNTHESIS -->` bounded blocks (read, do not emit verbatim — LAW 6).
- **Pass-through footer** — `<!-- PASS-THROUGH FOOTER -->` emoji-tree ending in raw save path (LAW 5).

The model adds narrative (`What I learned:`, KEY PATTERNS, comparison template sections) and must not improvise alternate titles, `Sources:` blocks, or bare `python3 scripts/last30days.py "$TOPIC"` on named entities without `--plan`.

## Install surfaces vs runtime

| Install mechanism | Typical `SKILL_DIR` | Sync model |
|-------------------|---------------------|------------|
| Claude marketplace | `~/.claude/plugins/cache/…/skills/last30days` | Marketplace/plugin update; watch STEP 0 |
| `npx skills add -g` | `~/.agents/skills/last30days` | Re-run add or dev symlink |
| Codex / Cursor skills dir | `~/.codex/skills/last30days` | Host-specific refresh |
| Hermes | `~/.hermes/skills/research/last30days` | `hermes skills install … --force` or symlink |
| Repo dev | `$REPO/skills/last30days` | Immediate on save |

Beta channel (`/last30days-beta` from private repo) is a **parallel Skill install**, not a different engine—see beta docs for promotion boundaries.

## Contributor boundaries

When extending the project:

1. **User-facing behavior** — update `SKILL.md` first (steps, flags, LAWs, examples).
2. **Implementation** — `scripts/last30days.py` + `scripts/lib/`.
3. **Operator knobs** — mirror env vars and CLI flags in `CONFIGURATION.md`.
4. **Tests** — `uv run pytest` (~89 files); version consistency via `tests/test_plugin_contract.py` and `skill_meta.read_skill_version`.

Do not treat the repo as a standalone CLI product in docs or UX: README and PR examples lead with `/last30days <topic>`; direct Python invocation is labeled fallback only.

## Related pages

<CardGroup>
  <Card title="Overview" href="/overview">
    Slash-command vs engine paths, zero-config sources, shortest successful research path.
  </Card>
  <Card title="Skill contract" href="/skill-contract">
    STEP 0 stale-clone guard, pre-flight protocol, SKILL_DIR substitution, invocation rules.
  </Card>
  <Card title="Research pipeline" href="/research-pipeline">
    v3 orchestration inside the engine: planner, fan-out, fusion, clustering, dedupe, rerank.
  </Card>
  <Card title="Output contract" href="/output-contract">
    Badge, LAWs voice rules, --emit modes, evidence blocks, footer pass-through.
  </Card>
  <Card title="Installation" href="/installation">
    Marketplace, npx skills add, Hermes, live-edit symlinks, version sync.
  </Card>
  <Card title="Model clients" href="/model-clients">
    Per-harness install patterns and stale-install avoidance.
  </Card>
  <Card title="CLI reference" href="/cli-reference">
    Direct last30days.py flags and constraints vs slash-command usage.
  </Card>
</CardGroup>

---

## 05. Query types

> QUERY_TYPE classification (GENERAL, NEWS, PROMPTING, RECOMMENDATIONS, COMPARISON), intent parsing, and engine pre-flight refusal paths.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/05-query-types.md
- Generated: 2026-06-04T23:19:52.899Z

### Source Files

- `skills/last30days/SKILL.md`
- `skills/last30days/scripts/lib/preflight.py`
- `skills/last30days/scripts/lib/planner.py`
- `skills/last30days/scripts/lib/categories.py`
- `skills/last30days/scripts/lib/query.py`

---
title: "Query types"
description: "QUERY_TYPE classification (GENERAL, NEWS, PROMPTING, RECOMMENDATIONS, COMPARISON), intent parsing, and engine pre-flight refusal paths."
---

`/last30days` classifies each user topic twice: the hosting model assigns a **QUERY_TYPE** for synthesis shape and WebSearch supplements (`skills/last30days/SKILL.md`), while the Python engine assigns a planner **intent** for sub-queries, source weights, freshness, and clustering (`skills/last30days/scripts/lib/planner.py`). A separate engine refuse-gate in `preflight.py` blocks Class 1 demographic-shopping topics before any pipeline work runs.

## Two classification layers

| Layer | Where it runs | Identifier set | Primary effect |
|-------|---------------|----------------|----------------|
| Skill / harness | Model before engine (Step “Parse User Intent”) | `GENERAL`, `NEWS`, `PROMPTING`, `RECOMMENDATIONS`, `COMPARISON` | Branded status line, WebSearch supplement queries, synthesis template (LAWs 2/4), invitation copy |
| Engine planner | `plan_query()` in `planner.py` (or `--plan` JSON from Step 0.75) | `factual`, `product`, `concept`, `opinion`, `how_to`, `comparison`, `breaking_news`, `prediction` | Sub-query fan-out, `freshness_mode`, `cluster_mode`, per-source weights and quick-depth caps |

The skill layer and engine layer are related but not identical. A `RECOMMENDATIONS` slash query often becomes a `product` or `opinion` plan intent; `PROMPTING` queries align with `how_to` planner behavior (YouTube/tutorial weighting). `COMPARISON` is the one type where both layers share the same name and force a deterministic per-entity plan when two or more entities parse from `vs` / `versus` / `/` splits.

```mermaid
flowchart LR
  subgraph harness["Harness (SKILL.md)"]
    U[User topic] --> PI[Parse intent → QUERY_TYPE]
    PI --> PF45[Step 0.45 keyword-trap check]
    PF45 --> PF5[Step 0.5 / 0.55 flags]
    PF5 --> P75[Step 0.75 → QueryPlan JSON]
  end
  subgraph engine["Engine (last30days.py)"]
    P75 -->|"--plan"| RUN[Pipeline]
    PI -->|positional topic| PRE[preflight.check_class_1_trap]
    PRE -->|REFUSE exit 2| STOP[No pipeline]
    PRE -->|pass| RUN
    RUN --> PLAN[plan_query / _infer_intent]
    PLAN --> SRC[Source fan-out + fusion]
  end
  SRC --> SYN[Synthesis by QUERY_TYPE template]
```

## QUERY_TYPE (skill contract)

The model parses the user message **before** engine invocation and stores `TOPIC`, optional `TARGET_TOOL`, `QUERY_TYPE`, and for comparisons `TOPIC_A` / `TOPIC_B`. Do not emit a multi-line “Parsed intent” block; show the branded one-liner instead.

| QUERY_TYPE | Trigger patterns (from user text) | Status line shape | Synthesis shape |
|------------|-----------------------------------|-------------------|-----------------|
| **PROMPTING** | “X prompts”, “prompting for X”, “X best practices” | `searching … for what people are saying about {TOPIC}` | Badge → `What I learned:` → KEY PATTERNS; invitation offers copy-paste prompt help for `TARGET_TOOL` |
| **RECOMMENDATIONS** | “best X”, “top X”, “what X should I use”, “recommended X” | Same as non-comparison types | Signal-weighted picks (not raw mention counts); invitation suggests compare/explain picks |
| **NEWS** | “what’s happening with X”, “X news”, “latest on X” | Same | Current-events narrative; WebSearch targets news/announcements |
| **COMPARISON** | “X vs Y”, “X versus Y”, “compare X and Y” (split on spaced ` vs ` / ` versus `) | `comparing {TOPIC_A} vs {TOPIC_B} across …` | Required `# {A} vs {B}` title and fixed `##` sections (`Quick Verdict`, per-entity, `Head-to-Head`, etc.); skips `What I learned:` |
| **GENERAL** | Default when no other pattern matches | Same as non-comparison types | Badge → `What I learned:` → KEY PATTERNS; broad understanding |

<Note>
The stored-variable line in SKILL.md lists `HOW-TO` among allowed `QUERY_TYPE` values, but classification rules and LAWs throughout the file use **PROMPTING** for technique-and-prompt workflows. Treat **PROMPTING** as the operative label for those queries.
</Note>

**TARGET_TOOL** is parsed from `[topic] for [tool]` or `[topic] prompts for [tool]`. The skill explicitly does **not** ask for a missing tool before research; it runs first, then may ask after results.

**Common pattern shortcuts:**

- `best [topic]` / `top [topic]` → `RECOMMENDATIONS`
- `X vs Y` (with spaces) → `COMPARISON` with entity split
- `[topic] for [tool]` → tool captured, type otherwise inferred

## Engine planner intents

When no `--plan` JSON is supplied, `plan_query()` calls `_infer_intent()` with ordered regex guards, then builds a `QueryPlan` (`schema.QueryPlan`: `intent`, `freshness_mode`, `cluster_mode`, `raw_topic`, `subqueries`, `source_weights`, `notes`).

| Intent | Detection signals (topic, lowercased) | Default `freshness_mode` | Default `cluster_mode` | Source bias (examples) |
|--------|--------------------------------------|--------------------------|------------------------|-------------------------|
| `comparison` | `vs`, `versus`, `compare`, slash-separated proper nouns (`React/Vue`) | `balanced_recent` | `debate` | Reddit, X, HN, YouTube |
| `prediction` | odds, predict, forecast, probability | `strict_recent` | `market` | Polymarket +2.5 weight |
| `how_to` | how to, tutorial, guide, setup, deploy, install | `evergreen_ok` | `workflow` | YouTube +2.0 |
| `factual` | what is/are, who is, release date, parameter count | `balanced_recent` | `none` | HN, Reddit, X |
| `opinion` | thoughts on, worth it, should I, review | `balanced_recent` | `debate` | Reddit, X, YouTube |
| `breaking_news` | latest, news, announced, launched; sports/ceremony keywords; trending/this week | `strict_recent` | `story` | X +1.5, Reddit +1.3 |
| `product` | pricing, feature(s), best/top … for | `balanced_recent` | `none` | YouTube, Reddit, TikTok |
| `concept` | explain, protocol, architecture; **default fallback** | `evergreen_ok` | `none` | HN, Reddit, X |

Comparison topics with ≥2 extracted entities skip LLM planning and use `_fallback_plan()` with per-entity sub-queries (`_should_force_deterministic_plan`).

Allowed intents are validated in `ALLOWED_INTENTS`; invalid LLM output is coerced or replaced by the deterministic fallback. Quick depth applies `SOURCE_LIMITS` caps (typically two sources per intent); default depth searches all configured sources.

## Query preprocessing (`query.py`)

`query.py` does not assign QUERY_TYPE. It supplies shared preprocessing for search modules:

- **`PREFIXES` / `SUFFIXES` / `NOISE_WORDS`** — strip meta-phrasing (“what are the best”, “prompting techniques”, “vs”, “news”, etc.) from platform queries.
- **`extract_core_subject()`** — compact keyword core after prefix/suffix/noise removal.
- **`extract_compound_terms()`** — hyphenated and Title Case multi-word phrases for quoting in planner `_keyword_query()`.

Planner and Reddit modules call these helpers so literal user phrasing does not get echoed verbatim into APIs (the documented “Hermes Agent use cases” failure mode).

## Category peers (`categories.py`)

`detect_category()` and `peer_subs_for()` are orthogonal to QUERY_TYPE. They map product-shaped topics to curated category-peer subreddits (AI image gen, coding agents, prediction markets, etc.) for Step 0.55 Reddit targeting. First pattern match wins; this is code-reviewed data, not user-editable config.

## Pre-flight: skill vs engine

Pre-flight runs at two boundaries with different coverage.

### Step 0.45 (skill — model-side)

Mandatory before Step 0.5. The model classifies keyword-trap **classes** and may stop for a clarifying question:

| Class | Pattern | Action |
|-------|---------|--------|
| **1** Demographic shopping | gift/present for age/gender/relationship; best X for men/women/kids | Ask hobbies / relationship / budget **or** reframe + gift subreddits; do not run engine on literal phrase without user OK |
| **2** Numeric / age trap | Bare numbers that collide (42, 40, 50) | Strip number from engine query unless semantically load-bearing |
| **3** Overly-literal tutorial | “how to use X”, “tutorial for Z” | Reframe to discussion vocabulary (“Docker tips workflows”) |
| **4** Generic single noun | `bread`, `sneakers` without hook | Ask which facet before running |

Emit `Pre-Flight: …` before the Resolved block. If the action is a clarifying question, **stop** until the user responds. Inline qualifiers on Class 1 (budget, hobbies, “cooking-obsessed”) skip the question and proceed with reframe.

### Engine refuse-gate (`preflight.py`)

Only **Class 1** is enforced in Python at `main()` before `ProgressDisplay` starts:

```text
topic → check_class_1_trap()
  ├─ no Class 1 match → continue pipeline
  ├─ Class 1 + qualifier (budget, hobbies, $, activity noun) → continue
  └─ Class 1 bare demographic → stderr REFUSE, exit code 2
```

<ParamField body="LAST30DAYS_SKIP_PREFLIGHT" type="env (optional)">
Set to any non-empty value to bypass the engine gate. Use only when the user insists on running the literal demographic-shopping phrase anyway.
</ParamField>

The REFUSE message includes `Class 1`, the echoed topic, and instructions to ask for hobbies, relationship, or budget, then re-run. Tests in `tests/test_preflight.py` lock match/skip behavior (e.g. `birthday gift for 40 year old` refuses; `gift for my cooking-obsessed husband` passes).

Classes 2–4 exist only in SKILL.md; the engine does not regex-block them. The hosting model must reframe before invocation.

## WebSearch supplements by QUERY_TYPE

After the engine returns (Step 2), supplement queries differ by skill QUERY_TYPE (2–3 searches, exclude reddit.com/x.com):

| QUERY_TYPE | Supplement focus |
|------------|------------------|
| RECOMMENDATIONS | `best {TOPIC} recommendations`, lists, popular examples |
| NEWS | `{TOPIC} news 2026`, announcements |
| PROMPTING | `{TOPIC} prompts examples`, techniques/tips |
| GENERAL | `{TOPIC} 2026`, discussion |
| COMPARISON | Follow comparison-mode workflow; per-entity engine runs may apply |

Use the user’s exact terminology; do not substitute model-known aliases unless disambiguation is required.

## Output contract differences

LAWs 2 and 4 split on QUERY_TYPE:

- **GENERAL / NEWS / PROMPTING / RECOMMENDATIONS:** No invented title; no `##` body headers; body opens with `What I learned:` after the version badge.
- **COMPARISON:** Required markdown title and mandated `##` sections; engine may emit a `## Head-to-Head` table scaffold to fill verbatim.

RECOMMENDATIONS synthesis must rank by signal quality, not mention frequency. PROMPTING invitations reference concrete techniques from research and offer paste-ready prompts when `TARGET_TOOL` is known.

## Passing intent into the engine

On WebSearch-capable harnesses, Step 0.75 expects the model to emit JSON aligned with `QueryPlan` and pass `--plan`. Intent in that JSON should reflect the topic (often mirroring QUERY_TYPE, e.g. `breaking_news` for NEWS, `how_to` for PROMPTING). Without `--plan`, stderr warns (LAW 7) and `_infer_intent()` supplies a deterministic plan—acceptable for headless/cron, weaker for agent-hosted runs.

Engine-internal competitor fan-out sets `internal_subrun=True` on `plan_query()` to suppress the false-positive “No --plan passed” warning.

## Verification signals

| Check | Expected signal |
|-------|-----------------|
| Class 1 trap (bare) | `[last30days] REFUSE: … Class 1` on stderr; exit `2`; no emoji-tree footer |
| Class 1 with hobby/budget | Engine runs normally |
| Comparison topic | Status line uses “comparing A vs B”; synthesis uses comparison `##` template |
| Skill pre-flight only | `Pre-Flight: topic matches Class N` in model output before bash |
| Planner fallback | `[Planner] No --plan passed` on stderr when hosting model skipped Step 0.75 |

## Related pages

<CardGroup>
  <Card title="Skill contract" href="/skill-contract">
    STEP 0 stale-clone guard, pre-flight protocol, engine invocation, and LAW anchors that depend on QUERY_TYPE.
  </Card>
  <Card title="Research pipeline" href="/research-pipeline">
    How planner intents drive sub-query fan-out, fusion, clustering, and depth profiles after classification.
  </Card>
  <Card title="Comparison mode" href="/comparison-mode">
    Competitor discovery, per-entity targeting, and comparison synthesis when QUERY_TYPE is COMPARISON.
  </Card>
  <Card title="Prompting recipes" href="/prompting-recipes">
    PROMPTING workflows, KEY PATTERNS shape, and copy-paste prompt patterns.
  </Card>
  <Card title="Output contract" href="/output-contract">
    Badge line, LAWs voice rules, emit modes, and footer pass-through by query type.
  </Card>
</CardGroup>

---

## 06. Research pipeline

> v3 orchestration: planner sub-queries, parallel source fan-out, fusion, clustering, dedupe, rerank, and depth profiles (--quick / default / --deep).

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/06-research-pipeline.md
- Generated: 2026-06-04T23:19:52.528Z

### Source Files

- `skills/last30days/scripts/lib/pipeline.py`
- `skills/last30days/scripts/lib/schema.py`
- `skills/last30days/scripts/lib/fusion.py`
- `skills/last30days/scripts/lib/cluster.py`
- `skills/last30days/scripts/lib/dedupe.py`
- `skills/last30days/scripts/lib/rerank.py`
- `docs/how-search-works.md`

---
title: "Research pipeline"
description: "v3 orchestration: planner sub-queries, parallel source fan-out, fusion, clustering, dedupe, rerank, and depth profiles (--quick / default / --deep)."
---

The v3 research engine in `skills/last30days/scripts/lib/pipeline.py` turns a topic string into a `schema.Report`: a `QueryPlan`, per-source evidence, fused `Candidate` rows, optional `Cluster` groups, and warnings. `last30days.py` resolves depth from `--quick` / `--deep`, calls `pipeline.run()`, then hands the report to renderers (`--emit=compact`, `json`, `context`, `md`, `html`). Comparison and vs-topic flows invoke the same pipeline once per entity via `fanout.run_competitor_fanout`.

## Pipeline overview

```mermaid
flowchart TB
  subgraph entry [CLI entry]
    CLI[last30days.py]
  end
  subgraph plan [Planning]
    PL[planner.plan_query / --plan]
    RT[providers.resolve_runtime]
  end
  subgraph phase1 [Phase 1 — parallel retrieval]
    TP[ThreadPoolExecutor]
    RS[_retrieve_stream per subquery × source]
    NSD[_normalize_score_dedupe]
  end
  subgraph phase2 [Phase 2 — recovery]
    SUP[_run_supplemental_searches]
    RET[_retry_thin_sources]
  end
  subgraph rank [Global ranking]
    FUS[weighted_rrf]
    RR[rerank.rerank_candidates]
    FUN[rerank.score_fun]
    GH[github.enrich_candidates_with_stars]
    CL[cluster_candidates]
  end
  CLI --> PL
  CLI --> RT
  PL --> TP
  RT --> PL
  TP --> RS --> NSD
  NSD --> SUP
  SUP --> RET
  RET --> FUS --> RR --> FUN --> GH --> CL
  CL --> REP[schema.Report]
```

| Stage | Module | Output |
| --- | --- | --- |
| Plan | `planner.py` | `QueryPlan` with 1–5 `SubQuery` rows |
| Fan-out | `pipeline.py` | `RetrievalBundle` keyed by `(subquery_label, source)` |
| Per-stream cleanup | `normalize`, `signals`, `dedupe`, `snippet` | `SourceItem` lists capped by depth |
| Fusion | `fusion.py` | `Candidate` pool (RRF + diversity) |
| Rerank | `rerank.py` | `rerank_score`, `final_score`, `fun_score` |
| Cluster | `cluster.py` | `Cluster` list with MMR representatives |

## Depth profiles

Depth is a single string: `"quick"`, `"default"`, or `"deep"`. The CLI sets it as `deep` if `--deep`, else `quick` if `--quick`, else `default`.

### Global pool limits (`DEPTH_SETTINGS`)

These caps apply after each retrieval stream is normalized; they do not replace per-source `DEPTH_CONFIG` inside Reddit, X, YouTube, and similar adapters.

| Depth | `per_stream_limit` | `pool_limit` (fusion) | `rerank_limit` (LLM shortlist) |
| --- | ---: | ---: | ---: |
| `quick` | 6 | 15 | 12 |
| `default` | 12 | 40 | 40 |
| `deep` | 20 | 60 | 60 |

### Planner behavior by depth

| Depth | Subqueries | Sources per subquery |
| --- | --- | --- |
| `quick` | At most **1** after sanitization | Up to **2** per intent via `SOURCE_LIMITS` and `QUICK_SOURCE_PRIORITY` |
| `default` | LLM or fallback (typically 2–5) | **All** eligible sources expanded per intent (`_trim_subqueries_for_depth` overrides narrow LLM source lists) |
| `deep` | Same as default | Same as default; higher retrieval caps downstream |

### Phases skipped at `quick`

- **Phase 2 supplemental** (`_run_supplemental_searches`): entity-derived X handle searches from Phase 1 Reddit/X text — skipped when `depth == "quick"` or `mock`.
- **Phase 2b thin retry** (`_retry_thin_sources`): simplified core-subject re-query for sources with fewer than three items — skipped when `depth == "quick"`.

### Reasoning models

`providers.resolve_runtime(config, depth)` picks planner and rerank models. With `LAST30DAYS_REASONING_PROVIDER=auto`, the first configured key wins: Gemini → OpenAI → xAI → OpenRouter; otherwise `reasoning_provider="local"` and planner/rerank fall back to deterministic scoring.

<ParamField body="--quick" type="flag">
Lower-latency profile: one planner subquery, tight source budget, no supplemental or thin-source retry, smaller fusion/rerank pools.
</ParamField>

<ParamField body="--deep" type="flag">
Higher-recall profile: larger per-stream and pool limits. When the reasoning provider is Gemini, default rerank model upgrades to Gemini Pro (`providers._resolve_model_pins`).
</ParamField>

<ParamField body="LAST30DAYS_PLANNER_MODEL" type="string">
Overrides default planner model for the resolved reasoning provider.
</ParamField>

<ParamField body="LAST30DAYS_RERANK_MODEL" type="string">
Overrides default rerank model for the resolved reasoning provider.
</ParamField>

## Query planning

`planner.plan_query()` returns a `QueryPlan`:

- **`intent`**: `factual`, `product`, `concept`, `opinion`, `how_to`, `comparison`, `breaking_news`, or `prediction`
- **`freshness_mode`**: `strict_recent`, `balanced_recent`, or `evergreen_ok`
- **`cluster_mode`**: `none`, `story`, `workflow`, `market`, or `debate` (drives downstream clustering)
- **`subqueries`**: each `SubQuery` has `label`, `search_query`, `ranking_query`, `sources`, and `weight`
- **`source_weights`**: per-source multipliers applied during fusion

### Plan sources (priority order)

1. **`--plan` JSON** (hosting agent or file path): sanitized through `_sanitize_plan`; `plan_source="external"`.
2. **LLM planner** when a reasoning provider and `planner_model` are configured: `provider.generate_json`; `plan_source="llm"`.
3. **Deterministic fallback** when LLM fails or no provider keys exist: comparison entities, intent heuristics, or generic primary subquery; `plan_source="deterministic"`.

<Warning>
When no `--plan` is passed and the engine has no internal reasoning credentials, stderr emits a LAW 7 reminder: the **hosting** agent (Claude Code, Codex, Hermes, etc.) should generate the plan JSON and pass `--plan`. The deterministic fallback is intended for headless/cron runs.
</Warning>

Planner traces always go to stderr (not user stdout):

```
[Planner] Plan: intent=..., freshness=..., cluster_mode=..., subqueries=N, source=llm|deterministic|external
[Planner]   sq1 label=... search="..." sources=[reddit,x,...]
```

If `grounding` is available and `--web-backend` is not `none`, every subquery gets `grounding` appended as a safety net even when the planner omits it.

## Phase 1: parallel source fan-out

For each `(subquery, source)` in the plan where `source` is in `available_sources(config)`:

1. Submit `_retrieve_stream` to a `ThreadPoolExecutor` (`max_workers` between 4 and 16, scaled to stream count).
2. Enforce **`MAX_SOURCE_FETCHES`**: X is capped at **2** submissions per run to limit 429 cascades.
3. On **429**, mark the source rate-limited (thread-safe set) so sibling futures skip it.
4. On **5xx**, sleep 3s and retry the stream once.
5. Run **`_normalize_score_dedupe`** on raw dicts, then truncate to `per_stream_limit`.

### GitHub pre-pass

Before the main loop, optional **`--github-repo`** (project mode) or **`--github-user`** (person mode) runs a dedicated GitHub search and injects results under the primary subquery label, skipping redundant keyword GitHub streams later.

### `available_sources`

Sources are included only when credentials or binaries exist (for example `reddit` always, `x` when Bird/xAI/xurl is configured, `grounding` when Brave/Exa/Serper/Parallel keys exist). `EXCLUDE_SOURCES` and `INCLUDE_SOURCES` further filter the set. `pipeline.diagnose()` reports the resolved list for `--diagnose`.

### Per-stream normalization (`_normalize_score_dedupe`)

| Step | Module | Role |
| --- | --- | --- |
| Normalize | `normalize.normalize_source_items` | Dates, URLs, engagement fields |
| Annotate | `signals.annotate_stream` | `local_relevance`, freshness, engagement, `local_rank_score` |
| Prune | `signals.prune_low_relevance` | Drops items below relevance floor (default 0.15) |
| Dedupe | `dedupe.dedupe_items` | Hybrid n-gram + token Jaccard (threshold 0.7) |
| Snippet | `snippet.extract_best_snippet` | Best excerpt for rerank prompts |

Reddit and YouTube often use **`raw_topic`** (original user string) inside `_retrieve_stream` so query expansion helpers see the full topic, not only the planner’s narrowed `search_query`.

## Phase 2: supplemental and thin-source retry

**Supplemental** (default/deep only): `entity_extract` pulls X handles and subreddits from Phase 1 Reddit/X payloads (plus `--x-handle` / `--x-related`). With Bird as the X backend, `bird_x.search_handles` adds targeted posts; related handles use label `supplemental-related` at weight **0.3**.

**Thin retry**: Sources with fewer than three items (and no error) get a second fetch using `query.extract_core_subject(topic, max_words=3)` as `search_query`, weight **0.3**, merged without duplicate URLs. GitHub is skipped when project/person mode already ran.

## Fusion (`weighted_rrf`)

`fusion.weighted_rrf` merges all `(label, source)` streams:

- **Score**: `subquery.weight × plan.source_weights[source] / (RRF_K + rank)` with `RRF_K = 60`
- **Dedup key**: normalized URL, or `source:item_id`
- **Merge**: accumulates `rrf_score`, max relevance/freshness/engagement, provenance metadata
- **Per-author cap**: at most **3** items per author/handle in the fused list
- **Diversity pool**: reserves up to **2** slots per source whose best item has `local_relevance ≥ 0.25`, then fills to `pool_limit`

## Rerank and fun scoring

`rerank.rerank_candidates` scores the fused shortlist (size `rerank_limit`):

- **LLM path**: JSON relevance 0–100 per `candidate_id`, with intent-specific hints and entity-grounding rules for the primary entity extracted from the topic
- **Fallback**: weighted mix of `local_relevance`, freshness, and `source_quality`; **−25** `ENTITY_MISS_PENALTY` when the primary entity is absent from title/snippet/transcript/comments
- **Tail**: candidates beyond the shortlist get fallback scores only

`rerank.score_fun` adds optional `fun_score` / `fun_explanation` (humor/virality judge) on up to 60 candidates.

Post-rerank, **`github.enrich_candidates_with_stars`** fills star counts for GitHub survivors when not in mock mode.

## Clustering (`cluster_candidates`)

| `plan.intent` / `cluster_mode` | Behavior |
| --- | --- |
| Intent not in `CLUSTERABLE_INTENTS` or `cluster_mode == "none"` | One cluster per candidate (no merging) |
| `breaking_news`, `opinion`, `comparison`, `prediction` + story/workflow/market/debate | Greedy text-similarity groups (threshold 0.42 news / 0.48 otherwise), then **entity overlap merge** across sources for small clusters |
| Representatives | MMR (`diversity_lambda=0.75`), up to 3 per cluster |

Uncertainty flags: `single-source` or `thin-evidence` when group scores or source diversity are weak.

## Final report and warnings

`pipeline.run` returns `schema.Report` with:

- `query_plan`, `ranked_candidates`, `clusters`, `items_by_source`
- `errors_by_source` (cleared when that source still returned items)
- `warnings` (thin evidence, single-source concentration, partial source failures)
- `artifacts` (grounding bundles, `plan_source` for degraded-run banners)

`_finalize_items_by_source` re-sorts, re-dedupes per source, applies Polymarket topic/keyword filters, and runs Digg X-post enrichment on survivors only.

## Invocation examples

<RequestExample>
```bash
# Default depth — full planner expansion, supplemental + thin retry
python3 skills/last30days/scripts/last30days.py "Hermes agent workflows" --emit=compact
```
</RequestExample>

<RequestExample>
```bash
# Quick profile — latency-biased
python3 skills/last30days/scripts/last30days.py "OpenClaw plugins" --quick --search=reddit,hackernews --emit=json
```
</RequestExample>

<RequestExample>
```bash
# Hosting agent supplies plan (headless-quality planning without engine API keys)
python3 skills/last30days/scripts/last30days.py "Topic" --plan /path/to/plan.json --emit=context
```
</RequestExample>

<RequestExample>
```bash
# Mock pipeline (no live credentials) — used in tests
python3 skills/last30days/scripts/last30days.py "test" --mock --quick --emit=compact
```
</RequestExample>

## Operational signals

| Signal | Where | Meaning |
| --- | --- | --- |
| `[Planner] Plan: ... source=deterministic` | stderr | No LLM plan; consider passing `--plan` from the hosting agent |
| `DEGRADED RUN` banner | compact output | `plan_source=deterministic` on named-entity topics without pre-research flags |
| `Some sources failed: ...` | `Report.warnings` | Partial retrieval; check keys via `--diagnose` |
| `Evidence is thin` | `Report.warnings` | Fewer than five ranked candidates |

<Note>
Slash-command invocations (`/last30days topic`) do not pass shell flags. The hosting model maps user intent (quick vs deep, source subset, comparison) into engine flags or a `--plan` file before calling `last30days.py`.
</Note>

## Key modules

| File | Responsibility |
| --- | --- |
| `lib/pipeline.py` | Orchestration, depth settings, phases, `run()` |
| `lib/planner.py` | LLM/deterministic `QueryPlan` |
| `lib/schema.py` | `SubQuery`, `QueryPlan`, `SourceItem`, `Candidate`, `Cluster`, `Report` |
| `lib/fusion.py` | Weighted RRF and pool diversification |
| `lib/dedupe.py` | Within-stream near-duplicate removal |
| `lib/rerank.py` | LLM relevance + fun judge + entity demotion |
| `lib/cluster.py` | Story grouping and cross-source entity merge |
| `lib/providers.py` | Reasoning provider and model pins |

Per-platform retrieval limits and auth live in each source module; see the data-sources page for adapter-specific `DEPTH_CONFIG` tables.

## Related pages

<CardGroup>
  <Card title="Skill, engine, and harness" href="/skill-engine-harness">
    Boundaries between SKILL.md, the Python engine, and harness runtimes that invoke `/last30days`.
  </Card>
  <Card title="Query types" href="/query-types">
    Intent classification, pre-flight refusal, and how intent shapes planner defaults.
  </Card>
  <Card title="Data sources" href="/data-sources">
    Per-platform retrieval modules, keys, and source-level depth configs.
  </Card>
  <Card title="Output contract" href="/output-contract">
    How `Report` fields map to compact, JSON, and synthesis output.
  </Card>
  <Card title="CLI reference" href="/cli-reference">
    All flags including `--plan`, `--search`, `--quick`, and `--deep`.
  </Card>
  <Card title="Comparison mode" href="/comparison-mode">
    Multi-entity fan-out that reuses `pipeline.run` per competitor.
  </Card>
</CardGroup>

---

## 07. Output contract

> Badge line, LAWs voice contract, --emit modes (compact, json, context, md, html), evidence blocks, and footer pass-through rules.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/07-output-contract.md
- Generated: 2026-06-04T23:20:12.838Z

### Source Files

- `skills/last30days/SKILL.md`
- `skills/last30days/scripts/lib/render.py`
- `skills/last30days/scripts/lib/html_render.py`
- `skills/last30days/scripts/last30days.py`
- `skills/last30days/scripts/lib/skill_meta.py`

---
title: "Output contract"
description: "Badge line, LAWs voice contract, --emit modes (compact, json, context, md, html), evidence blocks, and footer pass-through rules."
---

The last30days output contract spans three layers: Python rendering in `skills/last30days/scripts/lib/render.py` and `html_render.py`, the `--emit` router in `last30days.py`, and the hosting model’s synthesis rules in `SKILL.md` (badge, eight LAWs, evidence envelope, footer pass-through, invitation). Slash-command runs should use `--emit=compact` on stdout; saved artifacts and wire formats diverge by emit mode.

## Contract layers

```text
┌─────────────────────────────────────────────────────────────────┐
│ SKILL.md OUTPUT CONTRACT (LAWs 1–8, badge, synthesis shape)     │
│  → governs what the hosting model emits to the user             │
└────────────────────────────┬────────────────────────────────────┘
                             │ reads stdout / saves files
┌────────────────────────────▼────────────────────────────────────┐
│ last30days.py: emit_output() / emit_comparison_output()         │
│  --emit compact|md|json|context|html                            │
└────────────────────────────┬────────────────────────────────────┘
                             │
        ┌────────────────────┼────────────────────┐
        ▼                    ▼                    ▼
  render_compact      render_context      html_render.render_html
  (model scratchpad)  (abbreviated)       (shareable artifact)
```

| Layer | Owner | Primary consumer |
| --- | --- | --- |
| Badge + evidence + footer envelopes | `render.py` | Hosting model (compact/md stdout) |
| LAWs + invitation | `SKILL.md` | Hosting model (final chat response) |
| `--emit` routing | `last30days.py` | CLI, cron, skill Bash step |
| Saved `*-raw.md` | `render_full()` via `save_output()` | Durable debug / WebSearch appendix target |

## Badge line

The engine emits the mandatory first line in every compact, md, and HTML-prep path via `_render_badge()`:

```text
🌐 last30days v{VERSION} · synced {YYYY-MM-DD}

```

<ParamField body="VERSION" type="string">
Resolved from the nearest ancestor `.claude-plugin/plugin.json` `version` field; if missing (typical on per-harness skill installs), falls back to `SKILL.md` frontmatter via `skill_meta.read_skill_version()`. Returns `?` only when neither source yields a version.
</ParamField>

<ParamField body="synced date" type="string">
`date.today()` at render time (`YYYY-MM-DD`), not the research date range.
</ParamField>

<Note>
`SKILL.md` requires the hosting model to pass through the engine badge verbatim on line 1. Manual badge emission is a fallback when synthesizing without reusing engine stdout.
</Note>

**Placement by query type** (from `SKILL.md`):

| Query types | Line 1 | Line 3 (after blank line 2) |
| --- | --- | --- |
| GENERAL, NEWS, PROMPTING, RECOMMENDATIONS | Badge | `What I learned:` (prose label, not a title) |
| COMPARISON (`vs` / `versus`) | Badge | `# {A} vs {B} [vs {C}]: What the Community Says (/Last30Days)` |

## Voice contract (LAWs)

The eight LAWs in `SKILL.md` § OUTPUT CONTRACT override global formatting preferences inside `/last30days`. They apply at synthesis time; the engine enforces structure via HTML comments and warning blocks.

| LAW | Rule | Engine support |
| --- | --- | --- |
| **1** | No trailing `Sources:` / `References:` / link dumps; visible citation is the `🌐 Web:` footer line | Canonical boundary text; saved raw file holds WebSearch appendix |
| **2** | No invented title; body starts with `What I learned:` (COMPARISON uses required `# … vs …` title) | Badge as structural anchor |
| **3** | No em-dashes or en-dashes; use ` - ` | — |
| **4** | No `##` body headers except COMPARISON template sections and Python-pass-through blocks | Comparison scaffold emits allowed `## Head-to-Head` |
| **5** | Pass through emoji-tree footer verbatim | `<!-- PASS-THROUGH FOOTER -->` envelope |
| **6** | Do not emit raw ranked evidence clusters in user output | `<!-- EVIDENCE FOR SYNTHESIS -->` envelope |
| **7** | Reasoning-model hosts must pass `--plan` on named-entity topics | `## DEGRADED RUN WARNING` when `plan_source=deterministic` and flags missing |
| **8** | Inline `[name](url)` citations in narrative; footer links untouched | Evidence items carry URLs in compact output |

<LAW 6 failure signal>: User output containing `### 1.` followed by `(score N, M items, sources: …)` or `- Uncertainty: single-source` means evidence was dumped instead of synthesized.

## `--emit` modes

Default: `compact`. Choices are registered in `last30days.py` and routed through `emit_output()` / `emit_comparison_output()`.

| Mode | Stdout renderer | Typical use | Saved file when `--save-dir` set |
| --- | --- | --- | --- |
| `compact` | `render_compact()` | Primary skill path; model reads evidence + footer | `render_full()` → `{slug}-raw{suffix}.md` |
| `md` | `render_compact()` (identical to compact) | Debug/inspection only; **disallowed** as primary user-facing flow per `SKILL.md` | Same as compact: `render_full()` |
| `json` | `json.dumps(schema.to_dict(report))` | Scripting, watchlist, verify | `{slug}-raw{suffix}.json` (wire JSON) |
| `context` | `render_context()` or `render_comparison_multi_context()` | Token-efficient agent context | `render_full()` if saved as `.md` |
| `html` | `html_render.render_html()` | Shareable offline brief | `{slug}-raw-html{suffix}.html`; accepts `--synthesis-file` |

<Warning>
`--emit=md` and `--emit=compact` produce the same stdout (including evidence and footer envelopes). Do not treat `md` as a different user-facing format. The distinction matters for saved files: non-JSON/HTML saves always use `render_full()`, not compact stdout.
</Warning>

### `compact` / `md` stdout structure

`render_compact()` emits blocks in this order:

1. Badge + blank line
2. Debug header `# last30days vX: {topic}`
3. Model-facing safety blockquote (untrusted internet content)
4. Date range and active source list
5. Optional `## Freshness`, `## Warnings`
6. Optional `## DEGRADED RUN WARNING` (before evidence; LAW 7 backstop)
7. **Evidence envelope** (see below)
8. Optional `## Pre-Research Status` (after evidence closes)
9. Optional `## Head-to-Head` table scaffold (comparison topics)
10. **Pass-through footer** envelope
11. **Canonical boundary** (`# END OF last30days CANONICAL OUTPUT`)

`fun_level` (`low` / `medium` / `high`, from `FUN_LEVEL` env) adjusts Best Takes threshold and count inside the evidence envelope. Default cluster cap is 8 (4 per entity in comparison multi-render).

### `json`

Single-entity: full `Report` as dict via `schema.to_dict()`. Comparison: wrapper `{"comparison": true, "entities": [...], "reports": [{"entity", "report"}, ...]}`.

### `context`

Abbreviated cluster titles, representative items, safety note, optional freshness warning. No badge, evidence envelope, or emoji footer. Comparison context adds entity sections and resolved-entity summary when artifacts exist.

### `html`

Pipeline: `render_for_html()` → strip evidence / invitation / canonical boundary → promote prose labels → markdown-to-HTML → wrap engine footer → embed in styled template.

<Info>
HTML artifacts omit the evidence scratchpad, debug header, safety note, and invitation. Data-quality warnings go to **stderr** via `collect_html_warnings()`, not into the shareable file. Pass `--synthesis-file` with model-authored markdown (post-chat synthesis) for a populated brief body.
</Info>

## Evidence blocks

Raw research for the hosting model lives inside HTML comment boundaries:

```text
<!-- EVIDENCE FOR SYNTHESIS: read this, do not emit verbatim. ... -->
## Ranked Evidence Clusters
  ### N. {title} (score X, M items, sources: ...)
  ...
## Stats
## Best Takes          (fun_level-gated)
## Source Coverage
<!-- END EVIDENCE FOR SYNTHESIS -->
```

<Check>
LAW 6 scope: synthesize from this block into `What I learned:` prose (or COMPARISON template sections). Do **not** pass the envelope through to the user. Only the PASS-THROUGH FOOTER is verbatim in the final response (plus Python-emitted warning/scaffold blocks when present).
</Check>

Comparison multi-entity runs use one envelope with per-entity `## {label}` subsections, `#### Ranked Evidence Clusters`, and a shared `## Head-to-Head` scaffold after the envelope closes.

## Footer pass-through

The emoji-tree footer is wrapped for model copy-paste:

```text
<!-- PASS-THROUGH FOOTER: emit verbatim in the model response per LAW 5. -->
---
✅ All agents reported back!
├─ 🟠 Reddit: … │ …
├─ 🔵 X: … │ …
└─ 📎 Raw results saved to ~/…/slug-raw.md   (when --save-dir set)
---
<!-- END PASS-THROUGH FOOTER -->
```

Per-source lines are built from `_FOOTER_SOURCES` (Reddit, X, YouTube, TikTok, Instagram, Threads, Pinterest, HN, Bluesky, Truth Social, GitHub, Digg), plus Polymarket odds summaries and `🌐 Web:` publication names. Optional `🗣️ Top voices:` aggregates handles and subreddits.

<ParamField body="save_path" type="string | null">
Passed into `render_compact()` when `--save-dir` is set; `compute_save_path_display()` shows a `~/{relative}` path in the `📎 Raw results saved to` line.
</ParamField>

The canonical boundary after the footer explicitly scopes verbatim pass-through to the footer only and repeats LAW 1 / LAW 6 self-check strings.

## Final user-facing response shape

The engine does **not** emit the invitation block. The hosting model composes the chat response on top of compact stdout:

```text
[Badge — pass through line 1 from engine]

What I learned:                    ← model synthesis (LAW 2/4/8)
**Lead-in** - paragraph …
KEY PATTERNS from the research:    ← numbered list
1. …

[PASS-THROUGH FOOTER — verbatim]   ← LAW 5

---
I'm now an expert on {TOPIC}…       ← model invitation (SKILL.md templates)
```

| Block | Source | Pass through? |
| --- | --- | --- |
| Badge | Engine | Yes (line 1) |
| Evidence envelope | Engine | No (read only) |
| Head-to-Head scaffold | Engine (comparison) | Yes, fill cells |
| Emoji-tree footer | Engine | Yes (verbatim) |
| Invitation | Model | N/A (not in engine stdout) |
| Trailing `Sources:` | Model must omit | LAW 1 |

## Comparison and JSON/HTML notes

- **Comparison stdout**: `render_comparison_multi()` shares badge, one evidence envelope, Head-to-Head scaffold, single footer from the main report, and the same canonical boundary.
- **HTML footer**: `_wrap_engine_footer()` preserves the tree through markdown conversion; other HTML comments are stripped except `<!-- META: … -->` (date range and sources).
- **Zero-source runs**: If `_render_emoji_footer()` returns empty, `SKILL.md` allows skipping the footer and proceeding from KEY PATTERNS to the invitation.

## Verification signals

| Check | Command / test |
| --- | --- |
| Envelope order | `tests/test_render_v3.py` (`OutputEnvelopeTests`) |
| compact ≡ md stdout | `test_envelopes_appear_in_md_emit_mode` |
| HTML strips evidence | `tests/test_html_render.py` |
| Emit choices | `last30days.py` `--emit` argparse `choices=[...]` |

<Steps>
<Step title="Run compact research">
<code>python3 skills/last30days/scripts/last30days.py "topic" --emit=compact --save-dir="${LAST30DAYS_MEMORY_DIR:-$HOME/Documents/Last30Days}"</code>
</Step>
<Step title="Confirm stdout markers">
Stdout should start with the badge, contain exactly one `<!-- EVIDENCE FOR SYNTHESIS` / `<!-- END EVIDENCE FOR SYNTHESIS -->` pair, and one `<!-- PASS-THROUGH FOOTER` / `<!-- END PASS-THROUGH FOOTER -->` pair when sources returned data.
</Step>
<Step title="Synthesize per LAWs">
Transform evidence into `What I learned:` prose; copy the footer verbatim; end at the invitation with no trailing `Sources:` block.
</Step>
</Steps>

## Related pages

<CardGroup>
<Card title="Skill contract" href="/skill-contract">
SKILL.md runtime authority, engine invocation, and STEP 0 guards that precede output emission.
</Card>
<Card title="CLI reference" href="/cli-reference">
Full `--emit` flag listing, `--synthesis-file`, and direct CLI vs slash-command constraints.
</Card>
<Card title="HTML briefs" href="/html-briefs">
Shareable `--emit=html` artifacts, synthesis embedding, and save paths.
</Card>
<Card title="Comparison mode" href="/comparison-mode">
Multi-entity evidence layout, Head-to-Head scaffold, and comparison synthesis template.
</Card>
<Card title="Query types" href="/query-types">
QUERY_TYPE-specific placement rules for badge, titles, and invitation variants.
</Card>
</CardGroup>

---

## 08. Configure sources

> Credential layers (.env paths, keychain), per-source API keys, INCLUDE_SOURCES, setup wizard auto-actions, and --diagnose availability checks.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/08-configure-sources.md
- Generated: 2026-06-04T23:20:24.498Z

### Source Files

- `CONFIGURATION.md`
- `skills/last30days/scripts/lib/env.py`
- `skills/last30days/scripts/lib/setup_wizard.py`
- `skills/last30days/scripts/lib/providers.py`
- `skills/last30days/scripts/setup-keychain.sh`
- `skills/last30days/scripts/last30days.py`

---
title: "Configure sources"
description: "Credential layers (.env paths, keychain), per-source API keys, INCLUDE_SOURCES, setup wizard auto-actions, and --diagnose availability checks."
---

The last30days engine loads credentials through `lib/env.py::get_config()`, merges project and global `.env` files with optional macOS Keychain entries, and computes which retrieval backends are runnable via `lib/pipeline.py::available_sources()` and `diagnose()`. Slash-command hosts read the same keys when they invoke `scripts/last30days.py`; direct CLI users can verify wiring with `--diagnose` before spending a full research run.

## Credential resolution order

Configuration is assembled once per engine invocation. Effective value for each key follows this precedence (highest wins):

| Priority | Source | Path or mechanism |
| --- | --- | --- |
| 1 | Process environment | `export KEY=value` or harness-injected env |
| 2 | Project-scoped file | `.claude/last30days.env` (walks up from `cwd`) |
| 3 | Global file | `~/.config/last30days/.env` (override dir with `LAST30DAYS_CONFIG_DIR`) |
| 4 | macOS Keychain | Generic password service `last30days-<KEY>` (Darwin only) |

<Note>
`LAST30DAYS_CONFIG_DIR=""` disables global file loading entirely (clean/no-config mode). `LAST30DAYS_CONFIG_DIR=/path` points the global file at `/path/.env`.
</Note>

```text
  os.environ ──────────────┐
                           ▼
  .claude/last30days.env ──┼──► merged config dict
                           │         │
  ~/.config/last30days/.env┘         │ _CONFIG_SOURCE label
                                     ▼
  Keychain (last30days-*) ──► lowest file priority
```

POSIX hosts warn on stderr when a loaded secrets file is group- or world-readable; target mode `600`.

### macOS Keychain

`scripts/setup-keychain.sh` stores generic passwords for the keys listed in `lib/env.py::KEYCHAIN_KEYS` (kept in sync via `tests/test_env_keychain.py`). Each item uses service name `last30days-<KEY>` for the current user.

<CodeGroup>
```bash title="Interactive store all keys"
./skills/last30days/scripts/setup-keychain.sh
```

```bash title="List stored items"
./skills/last30days/scripts/setup-keychain.sh --list
```

```bash title="Store one key"
security add-generic-password -a "$USER" -s last30days-XAI_API_KEY -w "xai-..."
```
</CodeGroup>

Keychain values never override `.env` or process env; they only fill gaps.

## Environment file locations

| File | When used |
| --- | --- |
| `.claude/last30days.env` | Present in `cwd` or any parent up to `$HOME` |
| `$LAST30DAYS_CONFIG_DIR/.env` | Custom global path |
| `~/.config/last30days/.env` | Default global fallback |

Per-client workflows typically place keys and `INCLUDE_SOURCES` in `.claude/last30days.env`, then `cd` into that project so the engine picks up the file without extra flags.

## Per-source credentials

### Always-on (no API key)

| Source | Requirement |
| --- | --- |
| Reddit | Public Algolia API |
| Hacker News | Free Algolia API |
| Polymarket | Free Gamma API |

### CLI-backed (no skill API key)

| Source | Requirement |
| --- | --- |
| GitHub | `gh` CLI installed and authenticated |
| YouTube | `yt-dlp` on `PATH`, or `SCRAPECREATORS_API_KEY` as SC fallback |
| Digg | `digg-pp-cli` on `PATH` |

### Keyed social and enrichment

| Source | Key(s) | Engine gate |
| --- | --- | --- |
| X / Twitter | `XAI_API_KEY`, or `AUTH_TOKEN` + `CT0`, or `FROM_BROWSER`, or `xurl` OAuth CLI | `env.get_x_source()` |
| TikTok | `SCRAPECREATORS_API_KEY` or `APIFY_API_TOKEN` | On when SC/Apify key set; opt out with `EXCLUDE_SOURCES` |
| Instagram | `SCRAPECREATORS_API_KEY` | Same as TikTok |
| Threads | `SCRAPECREATORS_API_KEY` | On with SC key; opt out with `EXCLUDE_SOURCES=threads` |
| Pinterest | `SCRAPECREATORS_API_KEY` | Only when `--search=pinterest` (or planner requests `pinterest`) |
| Bluesky | `BSKY_HANDLE` + `BSKY_APP_PASSWORD` | `env.is_bluesky_available()` |
| Truth Social | `TRUTHSOCIAL_TOKEN` or browser cookie via `FROM_BROWSER` | `env.is_truthsocial_available()` |
| Xiaohongshu | `XIAOHONGSHU_API_BASE` (HTTP API health + login probe) | `--search=xiaohongshu` when reachable |
| Xquik | `XQUIK_API_KEY` | `env.is_xquik_available()` |

### Web grounding and deep research

| Capability | Key(s) | Notes |
| --- | --- | --- |
| Native web search (Step 2 / `--auto-resolve`) | `BRAVE_API_KEY`, `EXA_API_KEY`, `SERPER_API_KEY`, or `PARALLEL_API_KEY` | Auto priority: Brave → Exa → Serper → Parallel; override with `--web-backend` |
| Perplexity Sonar | `OPENROUTER_API_KEY` + `INCLUDE_SOURCES=perplexity` or `--search=perplexity` | Also auto-added when `--deep-research` runs |
| Deep Research (~$0.90/query) | `OPENROUTER_API_KEY` | `--deep-research` flag |

### Reasoning (headless runs only)

When the host model is not doing plan/rerank, `lib/providers.py::resolve_runtime()` auto-picks: Gemini (`GOOGLE_API_KEY` / `GEMINI_API_KEY` / `GOOGLE_GENAI_API_KEY`) → OpenAI (`OPENAI_API_KEY`) → xAI → OpenRouter → local deterministic fallback. Pin with `LAST30DAYS_REASONING_PROVIDER`.

<Warning>
Do not commit real API keys, cookies, or `.env` contents. Tests use obvious dummy values only.
</Warning>

### Legacy and rotation notes

- `SCRAPE_CREATORS_API_KEY` (underscore spelling) is accepted as an alias when the canonical `SCRAPECREATORS_API_KEY` is unset.
- Comma-separated `SCRAPECREATORS_API_KEY` values rotate per run via `random.choice`.
- Browser cookie extraction (`FROM_BROWSER`): default tries Firefox and Safari silently; `FROM_BROWSER=auto` adds Chrome; `FROM_BROWSER=off` disables extraction.

## INCLUDE_SOURCES and EXCLUDE_SOURCES

Both are comma-separated, case-insensitive lists in `.env` or process env.

### EXCLUDE_SOURCES (runtime denylist)

`pipeline.available_sources()` removes any listed source after computing availability. This is the supported way to disable TikTok, Instagram, or Threads while keeping `SCRAPECREATORS_API_KEY` set.

```bash
EXCLUDE_SOURCES=tiktok,instagram,threads
```

### INCLUDE_SOURCES (opt-in extensions)

Empty `INCLUDE_SOURCES` means no allowlist filter in the pipeline. Non-empty values gate specific add-ons:

| Token | Effect |
| --- | --- |
| `perplexity` | Enables Perplexity Sonar when `OPENROUTER_API_KEY` is set |
| `youtube_comments` | Comment enrichment on YouTube items (requires SC key) |
| `tiktok_comments` | Comment enrichment on TikTok items (requires SC key) |
| `threads`, `pinterest` | Mentioned in quality nudges; Threads is already default-on with SC key; Pinterest still needs `--search=pinterest` |

`--deep-research` temporarily appends `perplexity` to `INCLUDE_SOURCES` for that run.

<Info>
Quality scoring (`lib/quality_nudge.py`) treats a non-empty `INCLUDE_SOURCES` as an intentional allowlist when detecting “silent” Instagram failures. That does not change which sources the pipeline fetches unless combined with `EXCLUDE_SOURCES` or `--search`.
</Info>

### Per-run source selection (`--search`)

`--search` (alias map: `hn`, `bsky`, `truth`, `web`, `xhs`, `xquik`) intersects the availability list. Explicit `--search=perplexity` or `--search=threads` can enable those sources without editing `INCLUDE_SOURCES` (see `tests/test_cli_v3.py`).

## Example `.env` skeleton

```bash
# Reasoning (headless / cron)
GOOGLE_API_KEY=<your-gemini-key>

# Web grounding
BRAVE_API_KEY=<your-brave-key>

# ScrapeCreators family (TikTok, Instagram, Threads default-on)
SCRAPECREATORS_API_KEY=<your-sc-key>
EXCLUDE_SOURCES=threads          # optional denylist

# Opt-in enrichments
INCLUDE_SOURCES=youtube_comments,perplexity

# X (pick one path)
XAI_API_KEY=<your-xai-key>
# FROM_BROWSER=firefox

# Bluesky
BSKY_HANDLE=<handle>.bsky.social
BSKY_APP_PASSWORD=xxxx-xxxx-xxxx-xxxx
BSKY_SEARCH_HOST=api.bsky.app

SETUP_COMPLETE=true
```

After editing: `chmod 600` on the secrets file.

## First-run setup wizard

Setup spans the SKILL.md Step 0 UX (host model reads `nux-wizard.md` when referenced) and engine-side automation in `lib/setup_wizard.py`.

### Detection

- **SKILL path:** missing `~/.config/last30days/.env`, or file without `SETUP_COMPLETE=true` → run wizard; otherwise skip silently.
- **Engine path:** `setup_wizard.is_first_run(config)` when `SETUP_COMPLETE` is unset.

### CLI auto-setup (`setup` subcommand)

From the skill scripts directory (or with full path):

```bash
python3 skills/last30days/scripts/last30days.py setup
```

<Steps>
<Step title="Auto actions">
`run_auto_setup()` probes browsers in `auto` mode for X and Truth Social cookies (`COOKIE_DOMAINS`), checks `yt-dlp`, and runs `brew install yt-dlp` when Homebrew exists.
</Step>
<Step title="Persist flags">
`write_setup_config()` appends `SETUP_COMPLETE=true` and `FROM_BROWSER=<browser>` to the global `.env` without overwriting existing keys.
</Step>
<Step title="Optional auth flows">
`setup --github` — PAT via `gh auth token`, else GitHub device flow through ScrapeCreators (returns JSON API key). `setup --device-auth` — device flow only. `setup --openclaw` — JSON probe of CLI + key presence (no cookies).
</Step>
</Steps>

Status text for auto-setup is returned by `get_setup_status_text()` on stderr.

## Verify with `--diagnose`

`--diagnose` loads config, calls `pipeline.diagnose()`, prints JSON, and exits without running research. No topic required.

```bash
python3 skills/last30days/scripts/last30days.py --diagnose
```

<ParamField body="--search" type="string">
Optional comma-separated list; narrows `available_sources` the same way as a real run (e.g. `--diagnose --search=threads,pinterest`).
</ParamField>

### Response shape

| Field | Meaning |
| --- | --- |
| `providers` | `google`, `openai`, `xai`, `openrouter` booleans |
| `local_mode` | `true` when no reasoning API keys resolve |
| `reasoning_provider` | `LAST30DAYS_REASONING_PROVIDER` or `auto` |
| `x_backend` | Active X backend: `bird`, `xai`, `xurl`, or `null` |
| `bird_installed` / `bird_authenticated` / `bird_username` | Bird CLI state |
| `native_web_backend` | `brave`, `exa`, `serper`, `parallel`, or `null` |
| `has_scrapecreators` | SC key detected |
| `has_github` | `GITHUB_TOKEN` or `gh` on PATH |
| `available_sources` | Sources that would run for this config (and `--search` filter) |

During normal runs, `ui.show_diagnostic_banner()` and post-run promos consume the same `diag` object when core sources are missing.

### Quick interpretation

| Symptom in `available_sources` | Typical fix |
| --- | --- |
| No `x` | Add `XAI_API_KEY`, Bird cookies, or install/authenticate `xurl` |
| No `youtube` | `brew install yt-dlp` or set SC key |
| No `grounding` | Add a web search API key or use a host with native WebSearch |
| No `tiktok` / `instagram` | Set `SCRAPECREATORS_API_KEY`; check `EXCLUDE_SOURCES` |
| No `perplexity` | `OPENROUTER_API_KEY` + `INCLUDE_SOURCES=perplexity` or `--search=perplexity` |
| No `bluesky` | `BSKY_HANDLE` + `BSKY_APP_PASSWORD` |

## Configuration vs skill contract

- **Engine truth:** `get_config()`, `available_sources()`, `diagnose()` in `scripts/lib/`.
- **Slash-command truth:** `SKILL.md` Step 0 (wizard), intent parsing, and `ACTIVE_SOURCES_LIST` messaging must match keys you actually set.
- **User reference:** `CONFIGURATION.md` mirrors knobs for operators; if it diverges from code, code wins.

New env vars or CLI flags that affect sources should update `CONFIGURATION.md`, `SKILL.md`, and this page together.

## Related pages

<CardGroup>
<Card title="Configuration reference" href="/configuration-reference">
Full env var catalog, web-backend priority, output paths, and trend-monitoring toggles.
</Card>
<Card title="Data sources" href="/data-sources">
Per-platform retrieval modules, scoring inputs, and `--search` alias map.
</Card>
<Card title="Quickstart" href="/quickstart">
First `/last30days` run, wizard signals, and initial `--diagnose` check.
</Card>
<Card title="Troubleshooting" href="/troubleshooting">
Missing-source recovery, thin results, and documented failure modes.
</Card>
<Card title="Per-client setup" href="/per-client-setup">
Project-scoped `.claude/last30days.env` and save-dir isolation patterns.
</Card>
</CardGroup>

---

## 09. Model clients

> Per-harness install patterns: Claude Code marketplace, npx skills -a targets, Codex/Cursor/Gemini, Hermes, OpenClaw, and stale-install avoidance.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/09-model-clients.md
- Generated: 2026-06-04T23:20:54.008Z

### Source Files

- `README.md`
- `CONFIGURATION.md`
- `HERMES_SETUP.md`
- `.claude-plugin/marketplace.json`
- `.agents/plugins/marketplace.json`
- `gemini-extension.json`
- `skills/last30days/agents/openai.yaml`

---
title: "Model clients"
description: "Per-harness install patterns: Claude Code marketplace, npx skills -a targets, Codex/Cursor/Gemini, Hermes, OpenClaw, and stale-install avoidance."
---

The last30days Agent Skills package (`skills/last30days/`) installs into harness-specific skill directories; each host loads `SKILL.md` and invokes `scripts/last30days.py` from that install via `SKILL_DIR` (the directory containing the `SKILL.md` the model read). Distribution is provider-neutral: one skill tree, multiple install surfaces, no requirement for a single model vendor.

## Harness vs engine

| Layer | Role | Canonical artifact |
| --- | --- | --- |
| **Harness** | Agent runtime that exposes the slash command or skill invocation | Claude Code, Codex, Cursor, Gemini CLI, Hermes, OpenClaw, 50+ Agent Skills hosts |
| **Skill** | Prose contract + flags the model must pass | `skills/last30days/SKILL.md` |
| **Engine** | Python research pipeline | `skills/last30days/scripts/last30days.py` |

```mermaid
flowchart LR
  subgraph hosts [Harnesses]
    CC[Claude Code marketplace]
    AS[npx skills add -a targets]
    HM[Hermes skills install]
    OC[OpenClaw clawhub]
  end
  subgraph install [Install tree]
    SM[SKILL.md]
    PY[scripts/last30days.py]
  end
  hosts --> SM
  SM --> PY
```

When a reasoning harness runs `/last30days`, the host model is the planner and synthesizer; external reasoning API keys are only required for headless runs (cron, watchlist, direct CLI without a host). See the configuration reference for provider priority.

## Install matrix

Current release version is **3.3.1** (aligned across `skills/last30days/SKILL.md`, `.claude-plugin/plugin.json`, `.claude-plugin/marketplace.json`, and `gemini-extension.json`).

| Harness / surface | Install command | Update | Typical on-disk path |
| --- | --- | --- | --- |
| **Claude Code** (recommended) | `/plugin marketplace add mvanhorn/last30days-skill` then `/plugin install last30days@last30days-skill` | Auto via marketplace; `claude plugin update last30days@last30days-skill` | `~/.claude/plugins/cache/last30days-skill/last30days/{version}/skills/last30days/` |
| **Agent Skills hosts** (Codex, Cursor, Copilot, Gemini CLI, Windsurf, Cline, Continue, Roo, OpenCode, Goose, …) | `npx skills add mvanhorn/last30days-skill -g` | `npx skills update last30days -g` | `~/.codex/skills/last30days`, `~/.agents/skills/last30days`, or host-specific dir |
| **claude.ai** (web) | Download `last30days.skill` from [latest release](https://github.com/mvanhorn/last30days-skill/releases/latest/download/last30days.skill); upload in Settings → Capabilities → Skills | Re-download and re-upload | Hosted skill bundle (not a local tree) |
| **Hermes** | `hermes skills install mvanhorn/last30days-skill --force` | Same command | `~/.hermes/skills/research/last30days/` |
| **OpenClaw** | `clawhub install last30days-official` | `clawhub update last30days-official` | `~/.openclaw/skills/last30days/` (per OpenClaw layout) |
| **Developer live-edit** | `ln -sfn "$PWD/skills/last30days" ~/.agents/skills/last30days` (from repo root) | `git pull` in the repo | Symlink to working tree |

<Note>
Claude Code does **not** dedupe across install methods. If both the marketplace plugin and an `npx skills` copy are active, `/last30days` can appear twice in the command picker. Pick one install method per machine.
</Note>

## Claude Code marketplace

Claude Code is the most common harness. The marketplace manifest lives at `.claude-plugin/marketplace.json` (plugin name `last30days`, source `./`). Plugin metadata is in `.claude-plugin/plugin.json`; Claude Code auto-discovers `skills/*/SKILL.md` without a `"skills": ["./"]` entry (that pattern was removed after path-escape errors on newer Claude Code builds).

<Steps>
<Step title="Add marketplace and install plugin">

```text
/plugin marketplace add mvanhorn/last30days-skill
/plugin install last30days@last30days-skill
```

</Step>
<Step title="Force an update when needed">

```text
claude plugin update last30days@last30days-skill
```

Or in-session: `/plugin update last30days` then `/reload-plugins`.

</Step>
<Step title="Verify cached version">

```bash
cat ~/.claude/plugins/cache/last30days-skill/last30days/*/.claude-plugin/plugin.json | grep version
```

Expect `"version": "3.3.1"` (or your target release).

</Step>
</Steps>

Optional Agent Skills path on Claude Code (coexists with marketplace — avoid unless you intend dual installs):

```bash
npx skills add mvanhorn/last30days-skill -g -a claude-code
```

Invocation: `/last30days <topic>` (user-invocable skill; `argument-hint` documents example topics).

## Agent Skills CLI (`npx skills`)

Canonical install for Codex, Cursor, GitHub Copilot, Gemini CLI, and [50+ hosts](https://agentskills.io) via [vercel-labs/skills](https://github.com/vercel-labs/skills).

<CodeGroup>

```bash title="Global (recommended)"
npx skills add mvanhorn/last30days-skill -g
```

```bash title="Project-local"
npx skills add mvanhorn/last30days-skill
```

```bash title="Target specific harnesses"
npx skills add mvanhorn/last30days-skill -g -a codex
npx skills add mvanhorn/last30days-skill -g -a cursor
npx skills add mvanhorn/last30days-skill -g -a gemini-cli
npx skills add mvanhorn/last30days-skill -g -a codex -a cursor
```

</CodeGroup>

| Flag | Effect |
| --- | --- |
| `-g` | User-global install (available across projects) |
| `-a <harness>` | Target one or more harness adapters (`codex`, `cursor`, `gemini-cli`, `claude-code`, `github-copilot`, `windsurf`, `cline`, `continue`, `roo`, `aider-desk`, `opencode`, `goose`, …) |
| (no `-a`) | Installs for whichever harness `npx skills` detects |

Maintenance:

```bash
npx skills update last30days -g    # this skill only
npx skills update -g               # all global skills
npx skills list -g
npx skills remove last30days -g
```

<Warning>
`npx skills add` copies into `~/.agents/skills/last30days/` and symlinks per-host dirs where supported. That copy is **frozen at install time** — repo edits do not propagate until you re-run `npx skills add mvanhorn/last30days-skill -g` (or use a dev symlink). Contributors syncing a working tree use `npx skills add . -g -y` from the repo root.
</Warning>

### Codex

The `.codex-plugin/` scaffold was removed (v3.3.0). Codex users install via `npx skills add` or manual copy to `~/.codex/skills/last30days/`. Agent metadata for Codex-style hosts is in `skills/last30days/agents/openai.yaml` (`display_name`, `default_prompt`, `allow_implicit_invocation`).

Historical note: early Codex docs referenced `$last30days`; current installs follow the host’s skill/slash-command naming.

### Cursor, Copilot, Windsurf, and other `-a` targets

Same `npx skills add` flow with `-a <harness>`. Invocation is the host’s skill mechanism (slash command or skill picker per client). Do not pass shell pipes or flags through the slash form — express intent in natural language and let the model map to engine flags, or call the engine directly for scripting.

### Gemini CLI

README no longer documents a separate native-extension install path; use `npx skills add … -a gemini-cli`. The repo still ships `gemini-extension.json` (version **3.3.1**) listing optional env vars (`SCRAPECREATORS_API_KEY`, `BRAVE_API_KEY`, `XAI_API_KEY`, cookie vars, etc.) for extension-style configuration if your Gemini setup consumes that manifest.

## claude.ai (web)

Web Claude does not use the marketplace cache. Package a uploadable bundle from source:

```bash
bash skills/last30days/scripts/build-skill.sh   # from repo root; requires clean git tree
```

Produces `dist/last30days.skill` (zip with top-level `last30days/`, ≤200 files). Upload via [claude.ai Settings → Capabilities → Skills](https://claude.ai/settings/capabilities). Enable **Code execution and file creation** or skills will not run.

## Hermes

Hermes uses its own installer (not `npx skills`):

```bash
hermes skills install mvanhorn/last30days-skill --force
```

Deploys to `~/.hermes/skills/research/last30days/`. Invoke as:

```text
last30days "your research topic"
last30days "best mechanical keyboards 2025" --search=reddit,youtube
```

Prerequisites: Python 3.12+, optional `yt-dlp`. Diagnose from the install dir:

```bash
cd ~/.hermes/skills/research/last30days
python3.12 scripts/last30days.py --diagnose
```

Developer live-edit: symlink `skills/last30days` into `~/.hermes/skills/research/last30days` (see `HERMES_SETUP.md`).

## OpenClaw

OpenClaw distributes via ClawHub, not the public GitHub marketplace:

```bash
clawhub install last30days-official
clawhub update last30days-official
```

`SKILL.md` frontmatter includes `metadata.openclaw` (required bins `node`, `python3`, optional env keys, `clawhub` tag). SKILL contract documents a failure mode where models improvised `for dir in …` path discovery and hit a stale `~/.openclaw/skills/last30days/` engine — use `SKILL_DIR` from the loaded `SKILL.md`, not ad hoc directory walks.

## SKILL_DIR binding (all harnesses)

Every engine invocation must use the install that loaded `SKILL.md`:

```bash
SKILL_DIR="<absolute path of the directory containing the SKILL.md you Read>"
"${LAST30DAYS_PYTHON}" "${SKILL_DIR}/scripts/last30days.py" "<topic>" --emit=compact ...
```

Valid examples from the skill contract:

- `~/.claude/skills/last30days`
- `~/.codex/skills/last30days`
- `~/.claude/plugins/cache/last30days-skill/last30days/3.3.1/skills/last30days`
- `~/.agents/skills/last30days` (npx global copy)
- Repo checkout `…/skills/last30days`

If `scripts/last30days.py` is missing under `SKILL_DIR`, the install is broken or the model substituted the wrong path.

Badge version resolution: per-harness installs may lack `.claude-plugin/plugin.json`; the engine falls back to `SKILL.md` frontmatter `version` (see `scripts/lib/render.py`).

## Stale-install avoidance

### Claude Code marketplace clone (STEP 0)

`~/.claude/plugins/marketplaces/last30days-skill/` is a git clone Claude Code can restore to `origin/main` on session start. It may lag the **versioned cache** under `~/.claude/plugins/cache/last30days-skill/last30days/{version}/`.

`SKILL.md` **STEP 0** instructs the model: if the loaded path contains `/.claude/plugins/marketplaces/` and a newer cache `SKILL.md` exists, stop and re-read from the cache before proceeding. Documented impact: stale `--help` missed flags such as `--competitors` and broke comparison workflows.

Cache layout may be nested (`{version}/skills/last30days/SKILL.md`) or flat (`{version}/SKILL.md`); STEP 0 resolves whichever exists.

<Check>
Other paths (`~/.codex/skills/`, `~/.agents/skills/`, npx install dirs, repo checkouts) are valid load points — STEP 0 does not force a cache hop on those paths.
</Check>

### Frozen `npx skills` copy

Re-run `npx skills add` or `npx skills update` after upgrading the GitHub release. For local development, symlink: `ln -sfn "$PWD/skills/last30days" ~/.agents/skills/last30days`.

### Dual installs

Marketplace + `npx skills` on the same Claude Code machine → duplicate `/last30days` entries (fixed in v3.3.1 by removing redundant `commands/last30days.md` wrapper; dual skill trees can still coexist).

### OpenClaw and path-discovery loops

Do not implement custom install discovery. Follow `SKILL_DIR` from the `Read` of `SKILL.md`. Update via `clawhub update last30days-official` when behavior drifts.

### Verification smoke test

After updating Claude Code cache:

```text
/last30days birthday gift for 40 year old
```

A healthy install asks a clarifying question before running the engine. If it runs immediately with thin or improvised output, repeat plugin update and cache verification.

## Manifest and repo anchors

| File | Purpose |
| --- | --- |
| `.claude-plugin/marketplace.json` | Claude Code marketplace listing (`plugins[0].version`) |
| `.claude-plugin/plugin.json` | Plugin identity for cache installs |
| `.agents/plugins/marketplace.json` | Agents-style marketplace metadata (local `source`) |
| `gemini-extension.json` | Optional Gemini extension env schema |
| `skills/last30days/agents/openai.yaml` | Codex/OpenAI-agent interface display metadata |
| `skills/last30days/SKILL.md` | Runtime authority (STEP 0, `SKILL_DIR`, invocation contract) |

`tests/test_plugin_contract.py` enforces version alignment across manifests and asserts `.codex-plugin/` stays removed.

## Beta and experimental installs

Experimental work ships from `mvanhorn/last30days-skill-private` as `/last30days-beta` (parallel slash command). Do not mix beta-only behavior into the public marketplace without a review PR to the public repo.

## Related pages

<CardGroup>
<Card title="Installation" href="/installation">
Marketplace add, `npx skills` global vs project scope, Hermes, symlinks, and version sync expectations.
</Card>
<Card title="Skill contract" href="/skill-contract">
STEP 0 stale-clone guard, pre-flight protocol, engine invocation rules, and `SKILL_DIR` substitution.
</Card>
<Card title="Skill, engine, and harness" href="/skill-engine-harness">
Boundaries between the Agent Skills package, Python engine, and multi-harness runtimes.
</Card>
<Card title="Troubleshooting" href="/troubleshooting">
`--diagnose`, missing sources, stale marketplace clones, and thin results.
</Card>
<Card title="Configuration reference" href="/configuration-reference">
Env vars, reasoning provider priority when running headless, and web-backend order.
</Card>
<Card title="Beta channel" href="/beta-channel">
Parallel `/last30days-beta` install and promotion workflow.
</Card>
</CardGroup>

---

## 10. Comparison mode

> --competitors discovery fan-out, --competitors-list and --competitors-plan JSON schema, per-entity Step 0.55 targeting, and comparison synthesis template.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/10-comparison-mode.md
- Generated: 2026-06-04T23:21:24.129Z

### Source Files

- `skills/last30days/SKILL.md`
- `skills/last30days/scripts/last30days.py`
- `skills/last30days/scripts/lib/competitors.py`
- `skills/last30days/scripts/lib/fanout.py`
- `tests/test_cli_competitors.py`
- `tests/test_competitor_fanout.py`

---
title: "Comparison mode"
description: "--competitors discovery fan-out, --competitors-list and --competitors-plan JSON schema, per-entity Step 0.55 targeting, and comparison synthesis template."
---

Comparison mode runs one full `pipeline.run()` per entity in parallel, merges evidence for synthesis, and emits a Head-to-Head scaffold. Entry paths are a `vs`/`versus` topic string (automatic vs-mode), `--competitors` with optional engine-side peer discovery, or `--competitors-list` with explicit peer names. Per-entity Step 0.55 targeting is supplied through `--competitors-plan` (preferred) or engine `auto_resolve` when a web backend is configured.

## When comparison mode activates

| Trigger | Behavior |
| --- | --- |
| Topic contains ` vs `, ` versus `, ` vs.`, or `/` between entities | `planner._comparison_entities()` splits the topic; first entity is main, rest become peers; stderr logs `[Competitors] vs-mode: routing to N-pass fanout` |
| `--competitors` or `--competitors-list` | Enables fanout even without a vs-string; peers from discovery or the list |
| Neither of the above | Single-entity pipeline only |

Vs-mode detection uses a richer splitter than render (supports `difference between X and Y`, `compared to`, slash-separated lists, trailing context stripping, dedup). Entity count from the topic is capped at four subqueries for comparison intent. Render’s `_parse_comparison_entities()` caps table columns at four for readability.

`--competitors-plan` does not enable fanout by itself. Pass it together with a vs-topic or `--competitors` / `--competitors-list` so `parse_competitors_plan()` can thread targeting into peer sub-runs.

## End-to-end flow

```mermaid
sequenceDiagram
    participant Harness as Hosting model / CLI
    participant CLI as last30days.py
    participant Disc as competitors.discover_competitors
    participant Fan as fanout.run_competitor_fanout
    participant Pipe as pipeline.run
    participant Rend as render.render_comparison_multi

    Harness->>CLI: topic + flags (--competitors-plan, outer --x-handle, ...)
    CLI->>CLI: resolve_competitors_args / vs_entities
    alt explicit --competitors-list
        CLI->>CLI: peers = list
    else --competitors N and web backend
        CLI->>Disc: 3 parallel SERP queries
        Disc-->>CLI: ranked peer names
    else no backend (hosted path)
        CLI-->>Harness: exit 2 + SKILL.md recovery text
    end
    CLI->>Fan: main_runner + competitor_runner × N
    par MAX_PARALLEL_SUBRUNS=6
        Fan->>Pipe: main topic
        Fan->>Pipe: each peer (internal_subrun=True)
    end
    Fan-->>CLI: ordered (entity, Report)[]
    CLI->>Rend: merged compact/md/json/context/html
    Rend-->>Harness: evidence envelope + Head-to-Head scaffold
```

## CLI flags

<ParamField body="--competitors" type="integer (optional)" default="2 when flag present without value">
Auto-discover N peer entities and fan out. Bare `--competitors` defaults to N=2 (main + two peers → three-way). Range 1–6; values below 1 exit 2; values above 6 clamp with stderr warning. Ignored when `--competitors-list` is set (list length wins).
</ParamField>

<ParamField body="--competitors-list" type="comma-separated string">
Explicit peer names; whitespace-trimmed. Enables comparison mode without `--competitors`. Empty list exits 2. More than six entries clamp to `COMPETITORS_MAX` (6).
</ParamField>

<ParamField body="--competitors-plan" type="inline JSON or file path">
Per-entity Step 0.55 targeting for peer sub-runs. Parsed by `parse_competitors_plan()`; keys normalized to lowercase entity names. Preferred over list-only mode when the hosting model has already resolved handles and subreddits.
</ParamField>

<ParamField body="--polymarket-keywords" type="comma-separated string">
Optional disambiguation for ambiguous single-token topics during comparison runs (documented alongside competitor mode in SKILL.md).
</ParamField>

### Resolution precedence (`resolve_competitors_args`)

1. No flags → comparison disabled.
2. `--competitors-list` present → enabled; count = list length; numeric `--competitors` only warns if mismatched.
3. `--competitors=N` only → enabled; discovery count = N (after clamp).

## Engine-side discovery (`--competitors` without a list)

`lib/competitors.discover_competitors()` runs when auto-discovery is needed and a web backend exists (`BRAVE_API_KEY`, `EXA_API_KEY`, `SERPER_API_KEY`, etc., via `resolve._has_backend`).

<Steps>
<Step title="Fan out three SERP queries in parallel">
`{topic} competitors`, `{topic} alternatives`, `{topic} vs` — each through `grounding.web_search()` with the run’s lookback window.
</Step>
<Step title="Extract brand-shaped phrases">
Regex mining on titles and snippets; stopword filtering; reject candidates overlapping topic tokens.
</Step>
<Step title="Rank and return">
Frequency across SERP items; deterministic tie-break; return up to `count` names.
</Step>
</Steps>

If no backend is configured and `--mock` is not set, the engine exits 2 and prints recovery steps: hosting models should WebSearch peers, run Step 0.55 per entity, then re-invoke with a vs-topic and `--competitors-plan`; headless runs should configure API keys or pass `--competitors-list`.

Empty discovery also exits 2 (`Pass --competitors-list to override`).

## Parallel fan-out (`lib/fanout.py`)

`run_competitor_fanout()` schedules the main topic plus each peer in a `ThreadPoolExecutor` with `max_workers = min(len(competitors) + 1, MAX_PARALLEL_SUBRUNS)` where `MAX_PARALLEL_SUBRUNS = 6`.

- Per-entity failures log to stderr and are dropped; order preserved from submission order (main first, then peers in list order).
- Surviving reports must be at least two total; otherwise exit 1 (`Fewer than 2 sub-runs survived`).
- Empty competitor list degenerates to a single main run.

Each peer sub-run sets `internal_subrun=True`, deep-copies config, and records `report.artifacts["resolved"]` for the Resolved Entities block.

## `--competitors-plan` JSON schema

Top-level object: entity name → targeting object. Entity keys are case-insensitive after parse (stored lowercased).

| Field | Type | Role |
| --- | --- | --- |
| `x_handle` | string | Primary X handle (`@` stripped) |
| `x_related` | string[] | Related handles (`@` stripped) |
| `subreddits` | string[] | Subreddit names (`r/` prefix stripped) |
| `github_user` | string | GitHub username (lowercased, `@` stripped) |
| `github_repos` | string[] | `owner/repo` pairs; entries without `/` filtered out |
| `context` | string | News/events context for planner subqueries |

Unknown fields log a stderr warning and are dropped. Malformed JSON or a non-object top level exits 2. Non-dict entries for an entity are skipped with a warning.

`subrun_kwargs_for()` merges plan entries over `auto_resolve` results. Plan values win when present.

### Engine `auto_resolve` skip

When a plan entry includes both `x_handle` and `subreddits`, the engine skips `resolve.auto_resolve()` for that peer (hosting-model-driven Step 0.55). Otherwise, if a web backend exists, `auto_resolve(entity)` runs before `pipeline.run()`.

### File path vs inline JSON

`parse_competitors_plan()` accepts a filesystem path (same pattern as `--plan`). SKILL.md recommends a quoted heredoc tmpfile to avoid shell apostrophe breaks in `context` strings.

<RequestExample>

```bash
COMPETITORS_PLAN_FILE=$(mktemp "${TMPDIR:-/tmp}/last30days-competitors.XXXXXX")
trap 'rm -f "$COMPETITORS_PLAN_FILE"' EXIT
cat > "$COMPETITORS_PLAN_FILE" <<'PLAN_EOF'
{
  "Anthropic": {"x_handle":"AnthropicAI","subreddits":["ClaudeAI","MachineLearning"],"github_user":"anthropics","context":"Claude 4 rollout"},
  "xAI": {"x_handle":"xai","subreddits":["singularity"],"github_user":"xai-org"}
}
PLAN_EOF

python3 skills/last30days/scripts/last30days.py "OpenAI vs Anthropic vs xAI" \
  --emit=compact \
  --x-handle=OpenAI \
  --subreddits=OpenAI,MachineLearning \
  --competitors-plan "$COMPETITORS_PLAN_FILE"
```

</RequestExample>

Main topic (first entity in the vs-string) uses outer CLI flags (`--x-handle`, `--subreddits`, `--github-user`, `--github-repo`, TikTok/IG flags). Peers use `--competitors-plan` entries keyed by entity name.

## Per-entity Step 0.55 (hosting model)

On platforms with WebSearch (Claude Code, Codex, Hermes, Gemini, etc.), SKILL.md requires Step 0.55 once per entity before engine invocation: X handles, subreddits (including category-peer expansion for product topics), GitHub user/repos, and news context.

The four-step competitor protocol when the user asks for comparison without naming peers:

1. WebSearch `"{topic} competitors"` / `"{topic} alternatives"` (default N=2 peers unless `--competitors=N`).
2. Step 0.55 for main topic and each peer.
3. Build vs-topic: `"{main} vs {peer1} vs {peer2}"`.
4. Invoke engine with vs-topic, outer flags for main, and `--competitors-plan` for peers.

<Warning>
List-only mode (`--competitors-list` without plan) leaves peers on planner defaults — visibly thinner evidence. Dashes in `## Resolved Entities` for a peer mean Step 0.55 was skipped for that entity.
</Warning>

Platforms without WebSearch should use `--auto-resolve` on the engine command so `resolve.auto_resolve()` backs each sub-run when keys exist.

## Engine output shape

`render_comparison_multi()` produces:

- Badge and comparison header (`# last30days v{VERSION}: {entity1} vs {entity2} ...`)
- Optional aggregated `## Warnings`
- `<!-- EVIDENCE FOR SYNTHESIS -->` with per-entity `## {label}` evidence subsections (lower `cluster_limit` than single-entity runs)
- `## Resolved Entities` — per-entity X / Subs / GitHub / Context summary from artifacts
- `## Head-to-Head` empty table scaffold (nine axes from the April 9 exemplar: What it is, GitHub stars, Philosophy, Skills, Memory, Models, Security, Best for, Install)
- Pass-through emoji footer from the main report

`--emit=json` returns `{"comparison": true, "entities": [...], "reports": [...]}`. With `--save-dir`, the merged comparison saves once; each peer also gets an individual `{slug}-raw.md` (historical N-pass behavior).

Passing `--competitors-plan` or `--plan` suppresses the runtime web-promo nudge in the UI layer.

## Hosting model synthesis template

Comparison queries use a dedicated template (not `What I learned:` / KEY PATTERNS). LAW 2 and LAW 4 comparison exceptions apply.

Required structure after the badge:

| Section | Purpose |
| --- | --- |
| `# {A} vs {B} [vs {C}]: What the Community Says (/Last30Days)` | Title line |
| `## Quick Verdict` | Thesis, scale stats, one quotable community framing |
| `## {Entity}` (one per entity) | Community Sentiment; Strengths; Weaknesses bullets with `per <source>` |
| `## Head-to-Head` | Fill engine scaffold cells (5–15 words; ` - ` not em-dashes) |
| `## The Bottom Line` | `**Choose {Entity} if**` per entity |
| `## The emerging stack` | Combination pattern or explicit “no pattern yet” |

Forbidden for comparisons: `What I learned:`, bold-lead-in general voice, `KEY PATTERNS` lists, extra `##` headers beyond the six above, fabricated `## Notable Stats` (footer is LAW 5 pass-through).

Supplement after engine run: WebSearch `{A} vs {B} comparison {YEAR}` and `{A} vs {B} which is better` for rivalry coverage.

## Hosting model vs headless CLI

| Concern | Hosting model (slash command) | Direct CLI / cron |
| --- | --- | --- |
| Peer discovery | Model WebSearch + vs-topic + `--competitors-plan` | `--competitors` + web API keys, or `--competitors-list` |
| Per-entity targeting | `--competitors-plan` from Step 0.55 | Plan file or `auto_resolve` per peer |
| Synthesis | Model follows SKILL.md comparison template | Engine emits evidence + scaffold; synthesis still model-driven in skill hosts |
| Thin peer data | Avoid `--competitors-list` alone | Same; check Resolved Entities block |

`--competitors` on the engine signals intent in skill hosts; discovery and Step 0.55 remain the hosting model’s job before the vs-topic invocation.

## Failure modes and exit codes

| Condition | Exit / signal |
| --- | --- |
| `--competitors` below 1 | Exit 2 |
| Empty `--competitors-list` | Exit 2 |
| Auto-discovery with no web backend (non-mock) | Exit 2 + recovery stderr |
| Zero peers discovered | Exit 2 |
| Fewer than two sub-runs survived | Exit 1 |
| Invalid `--competitors-plan` JSON | Exit 2 |

Partial peer failure is graceful: remaining entities still render if at least two reports survive.

## Related pages

<CardGroup>
<Card title="Query types" href="/query-types">
COMPARISON intent detection, vs-string parsing, and pre-flight refusal paths.
</Card>
<Card title="Comparison recipes" href="/comparison-recipes">
Copy-paste workflows for vs-topic phrasing, fan-out, and Head-to-Head expectations.
</Card>
<Card title="Skill contract" href="/skill-contract">
STEP 0 stale-clone guard, LAW exceptions for comparison voice, and engine invocation rules.
</Card>
<Card title="CLI reference" href="/cli-reference">
Full flag inventory including `--competitors*` and emit modes.
</Card>
<Card title="Output contract" href="/output-contract">
Badge line, evidence envelope boundaries, and footer pass-through (LAW 5).
</Card>
<Card title="Per-client setup" href="/per-client-setup">
Recurring `--competitors-plan` templates and project-scoped env isolation.
</Card>
</CardGroup>

---

## 11. HTML briefs

> Shareable offline HTML artifacts via --emit=html, --synthesis-file, save paths under LAST30DAYS_MEMORY_DIR, and SKILL.md-driven save flow.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/11-html-briefs.md
- Generated: 2026-06-04T23:21:50.794Z

### Source Files

- `skills/last30days/references/save-html-brief.md`
- `skills/last30days/scripts/lib/html_render.py`
- `skills/last30days/SKILL.md`
- `skills/last30days/scripts/last30days.py`
- `README.md`

---
title: "HTML briefs"
description: "Shareable offline HTML artifacts via --emit=html, --synthesis-file, save paths under LAST30DAYS_MEMORY_DIR, and SKILL.md-driven save flow."
---

Shareable HTML briefs are produced by the Python engine’s `--emit=html` mode (`html_render.py` wrapping `render.render_for_html`), with the harness agent writing model synthesis to a temp file via `--synthesis-file` and saving stdout under `LAST30DAYS_MEMORY_DIR`. Chat remains the primary surface; the `.html` file is an optional artifact for Slack, email, or Notion.

## When HTML briefs run

HTML save flow activates only when the user asks for it. `SKILL.md` gates the flow on either:

| Trigger | Examples |
|---|---|
| Explicit flag in `$ARGUMENTS` | `--emit=html`, `--emit:html`, `--html` |
| Natural language | “shareable HTML brief”, “for Slack”, “export as HTML”, “for Notion” |

If neither applies, skip HTML entirely and proceed to the normal wait-for-response step.

<Warning>
Slash commands in Agent Skills hosts do not pass shell pipes or redirects. `/last30days OpenClaw --emit=html | pbcopy` is invalid. The model must translate intent into engine flags and a real shell redirect (or use direct CLI invocation with full `python3 ...` syntax).
</Warning>

## Two save path conventions

The repo uses two related but distinct filenames under `LAST30DAYS_MEMORY_DIR` (default `~/Documents/Last30Days/`, overridable via env or `.claude/last30days.env`):

| Path pattern | Who writes it | Purpose |
|---|---|---|
| `{slug}-brief.html` | Harness agent (`references/save-html-brief.md`) | Canonical shareable brief from skill flow |
| `{slug}-raw-html[-suffix][-YYYY-MM-DD].html` | Engine `save_output()` when `--save-dir` is set | Engine-persisted HTML alongside other emit saves |

Slug rules: lowercase topic with non-alphanumeric runs collapsed to `-` (`slugify()` in `last30days.py`).

Collision handling in `save_output()`: if the target file already exists, append `-YYYY-MM-DD` before the extension. The skill reference documents the same behavior for agents saving briefs.

## End-to-end flow (skill / harness)

```mermaid
sequenceDiagram
  participant User
  participant Agent as Harness agent
  participant Engine as last30days.py
  participant Disk as LAST30DAYS_MEMORY_DIR

  User->>Agent: /last30days topic (HTML intent)
  Agent->>Engine: --emit=compact --save-dir ...
  Engine-->>Agent: compact stdout (evidence + footer)
  Agent-->>User: synthesis in chat (badge, What I learned, KEY PATTERNS, footer, invitation)
  Agent->>Agent: Write synthesis verbatim to temp .md
  Agent->>Engine: --emit=html --synthesis-file temp.md
  Engine-->>Agent: full HTML document on stdout
  Agent->>Disk: redirect stdout to {slug}-brief.html
  Agent-->>User: 📎 Shareable brief saved to path
```

<Steps>
<Step title="Run research as usual">
Invoke the engine with `--emit=compact` and `--save-dir="${LAST30DAYS_MEMORY_DIR}"` (plus plan/targeting flags). Emit the full chat synthesis before any HTML step.
</Step>
<Step title="Write synthesis to a temp file">
Persist only the narrative the user saw: `What I learned:` paragraphs, inline `[name](url)` citations, and `KEY PATTERNS from the research:` list. Exclude the badge, engine footer, invitation block, evidence scratchpad, and debug headers.
</Step>
<Step title="Render HTML">
Re-invoke the engine with the same topic and the same plan/targeting flags as the compact run, plus `--emit=html` and `--synthesis-file`. Redirect stdout to `${LAST30DAYS_MEMORY_DIR}/${SLUG}-brief.html`.
</Step>
<Step title="Confirm in chat">
Append one line after the invitation: `📎 Shareable brief saved to <absolute or ~ path>`.
</Step>
</Steps>

Canonical shell pattern (from `references/save-html-brief.md`):

```bash
SYNTHESIS_FILE="/tmp/last30days-synthesis-${CLAUDE_SESSION_ID}.md"
# cat synthesis verbatim into SYNTHESIS_FILE (quoted heredoc)

SLUG=$(echo "$TOPIC" | tr '[:upper:]' '[:lower:]' | tr -cs 'a-z0-9' '-' | sed 's/^-//;s/-$//')
HTML_PATH="${LAST30DAYS_MEMORY_DIR}/${SLUG}-brief.html"

"${LAST30DAYS_PYTHON}" "${SKILL_ROOT}/scripts/last30days.py" "${TOPIC}" \
  --emit=html \
  --synthesis-file "$SYNTHESIS_FILE" \
  > "$HTML_PATH"
```

<Note>
`SKILL.md` requires reading `references/save-html-brief.md` before the wait-for-response step whenever HTML is requested. Do not improvise paths or synthesis scope from memory.
</Note>

### Follow-up turn without re-research

If the user already received a compact synthesis and later asks to “save as HTML”, reuse the synthesis from conversation history. Write it to the temp file and call `--emit=html --synthesis-file` only—do not restart WebSearch or rewrite the narrative.

## Direct CLI usage

For scripting, cron, or engine testing without the skill loop:

<ParamField body="--emit" type="string" required>
Must be `html`. Other values ignore `--synthesis-file` with a stderr warning.
</ParamField>

<ParamField body="--synthesis-file" type="path">
UTF-8 markdown file embedded as the HTML body. Only read when `--emit=html`. Missing or unreadable files exit with code 2.
</ParamField>

<ParamField body="--save-dir" type="path">
Optional. When set, the engine writes `{slug}-raw-html[-suffix][-date].html` via `save_output()` and still prints HTML to stdout.
</ParamField>

<CodeGroup>
```bash title="Stdout only"
python3 skills/last30days/scripts/last30days.py "OpenClaw" \
  --emit=html \
  --synthesis-file /tmp/synthesis.md
```

```bash title="Engine save + stdout"
python3 skills/last30days/scripts/last30days.py "OpenClaw" \
  --emit=html \
  --synthesis-file /tmp/synthesis.md \
  --save-dir "$HOME/Documents/Last30Days" \
  --save-suffix=v3
```
</CodeGroup>

Without `--synthesis-file`, `render_for_html()` still emits badge, date/source metadata, and the pass-through footer, but the body stays sparse—unsuitable as a shareable brief.

## What the HTML document contains

Rendering pipeline: `render_for_html()` → markdown intermediate → `html_render.render_html()` → single self-contained file.

| Section | Source | Notes |
|---|---|---|
| Badge | Engine `_render_badge()` | `🌐 last30days vX.Y.Z · synced YYYY-MM-DD` as styled pill |
| Metadata line | `<!-- META: range · sources -->` | Promoted to `<div class="meta">` after conversion |
| Synthesis body | `--synthesis-file` content | Verbatim; `What I learned:` / `KEY PATTERNS from the research:` promoted to `<h2>` |
| Citations | Inline markdown links | Converted to `<a href="...">` |
| Engine footer | Pass-through footer markers | Wrapped in `<div class="engine-footer"><pre>…</pre></div>`; tree preserved |
| Colophon | `_build_colophon()` | Generation date, skill version, topic, `/last30days {topic}` rerun hint |

Stripped before share (not in the artifact):

- `# last30days vX.Y.Z: TOPIC` debug file header (`<h1>` suppressed)
- `> Safety note:` blockquote
- `<!-- EVIDENCE FOR SYNTHESIS -->` blocks
- Invitation block (`I'm now an expert… Just ask.`)
- `---` / `# END OF last30days CANONICAL OUTPUT` boundary
- Data quality warnings (degraded run, thin evidence, quota messages)—by design these stay in stderr for the generator, not recipients

Presentation details:

- Inline CSS in `<style>`; no JavaScript (`test_self_containedness`)
- Google Fonts link for Inter and JetBrains Mono (offline-friendly content except font fetch)
- `prefers-color-scheme: light` and `@media print` rules (A4, link URLs in print)

## Comparison mode

For `X vs Y` topics, the engine routes to `render_for_html_comparison()` / `html_render.render_html_comparison()`. The agent uses the same save flow; the synthesis temp file should match the comparison template from chat (`## Quick Verdict`, per-entity sections, `## Head-to-Head`, `## The Bottom Line`, etc.). Saved comparison HTML uses topic slug `entity-a-vs-entity-b` when `--save-dir` persists via `comparison_topic()`.

## Synthesis file contract

| Include | Exclude |
|---|---|
| `What I learned:` bold-lead-in paragraphs with `[name](url)` | Badge line |
| `KEY PATTERNS from the research:` numbered list | `✅ All agents reported back!` footer tree |
| Comparison `##` sections when in comparison mode | Invitation block |
| Unicode and emoji as in chat | Evidence clusters, stats, source coverage |
| | Data quality warning text |

Use a quoted heredoc (`<<'SYNTHESIS_EOF'`) so shell metacharacters in the topic or synthesis do not expand. Match chat text exactly—no paraphrase or reorder.

## Configuration

| Variable / knob | Role |
|---|---|
| `LAST30DAYS_MEMORY_DIR` | Root for `{slug}-brief.html` and `{slug}-raw*.md` / `raw-html` saves; defaults per platform in `CONFIGURATION.md` |
| `--save-dir` | Per-run override passed on engine invocations |
| `--save-suffix` | Inserts `-{suffix}` before extension on engine saves (e.g. per-client `v3`) |

Project-scoped `.claude/last30days.env` can set `LAST30DAYS_MEMORY_DIR` per client without wrapper scripts.

## Troubleshooting

| Symptom | Likely cause | Mitigation |
|---|---|---|
| Thin HTML with no narrative | HTML run without `--synthesis-file` or empty temp file | Ensure synthesis file matches chat output scope |
| `[last30days] Warning: --synthesis-file is only used with --emit=html` | `--synthesis-file` on non-html emit | Add `--emit=html` |
| `Cannot read --synthesis-file` (exit 2) | Bad path or permissions | Fix path; use absolute paths in scripts |
| Warnings missing from file but in stderr | Expected—warnings excluded from artifact | Read stderr when debugging run quality |
| Slash command with pipe fails | Harness does not pass shell mechanics | Use agent redirect or direct CLI |
| Duplicate briefs | Same slug, same day | Engine adds date suffix on `--save-dir` saves; agent should report actual path after redirect |

## Related pages

<CardGroup>
<Card title="Output contract" href="/output-contract">
Badge, LAWs voice, all `--emit` modes, evidence blocks, and footer pass-through rules that define what goes into synthesis vs HTML.
</Card>
<Card title="Skill contract" href="/skill-contract">
SKILL.md HTML trigger section, `save-html-brief.md` reference load, and wait-for-response ordering.
</Card>
<Card title="CLI reference" href="/cli-reference">
Full `last30days.py` flags including `--emit` choices and direct CLI vs slash-command constraints.
</Card>
<Card title="Configuration reference" href="/configuration-reference">
`LAST30DAYS_MEMORY_DIR`, `--save-dir`, `--save-suffix`, and per-client `.env` patterns.
</Card>
<Card title="Comparison mode" href="/comparison-mode">
Comparison synthesis shape and `render_for_html_comparison` behavior.
</Card>
</CardGroup>

---

## 12. Trend monitoring

> SQLite --store persistence, watchlist.py scheduling and delivery, briefing.py digests, and recommended baseline cadence.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/12-trend-monitoring.md
- Generated: 2026-06-04T23:21:36.095Z

### Source Files

- `CONFIGURATION.md`
- `skills/last30days/scripts/store.py`
- `skills/last30days/scripts/watchlist.py`
- `skills/last30days/scripts/briefing.py`
- `tests/test_watchlist_commands.py`
- `skills/last30days/scripts/last30days.py`

---
title: "Trend monitoring"
description: "SQLite --store persistence, watchlist.py scheduling and delivery, briefing.py digests, and recommended baseline cadence."
---

Trend monitoring turns one-off `/last30days` research into a local time series: ranked findings land in SQLite at `~/.local/share/last30days/research.db`, `watchlist.py` drives recurring engine runs and optional webhooks, and `briefing.py` emits structured digest JSON for agent-side synthesis. Markdown snapshots under `LAST30DAYS_MEMORY_DIR` still save on every engine run; the database is the cross-run substrate for deltas, budgets, and digests.

## Snapshot mode vs accumulation

| Mode | Trigger | Primary artifact | Cross-run memory |
|------|---------|------------------|------------------|
| Snapshot (default) | `/last30days "<topic>"` or engine without persistence | `<slug>-raw.md` (or suffix) under `LAST30DAYS_MEMORY_DIR` | None — same topic + suffix overwrites or date-stamps per save rules |
| Accumulation | `--store`, `LAST30DAYS_STORE=1`, or watchlist `run-one` / `run-all` | `research.db` + optional brief JSON | URL-deduped `findings`, per-run `finding_sightings`, run history |

<Note>
Watchlist runs do **not** pass `--store` to the engine. They spawn `last30days.py` with `--emit=json`, then persist up to 25 findings via `store.findings_from_report()` and `store.store_findings()` inside `watchlist.py`.
</Note>

## Architecture

```mermaid
flowchart TB
  subgraph user["Operator / scheduler"]
    Cron["cron / Task Scheduler / CI"]
    Slash["/last30days or direct CLI"]
  end

  subgraph engine["skills/last30days/scripts/last30days.py"]
    Pipeline["v3 pipeline"]
  end

  subgraph persist["~/.local/share/last30days/"]
    DB["research.db — store.py"]
    Briefs["briefs/*.json — briefing.py"]
  end

  subgraph ops["Operational scripts"]
    WL["watchlist.py"]
    BR["briefing.py"]
  end

  subgraph delivery["Optional notifications"]
    Slack["Slack incoming webhook"]
    Hook["Generic HTTPS webhook"]
  end

  Slash --> Pipeline
  Cron --> WL
  WL -->|"subprocess: --emit=json --quick --lookback-days 90"| Pipeline
  Pipeline -->|"--store or LAST30DAYS_STORE"| DB
  WL --> DB
  WL -->|"new findings only"| Slack
  WL --> Hook
  BR --> DB
  BR --> Briefs
  Pipeline --> Mem["LAST30DAYS_MEMORY_DIR markdown"]
```

## SQLite store (`store.py`)

Default path: `~/.local/share/last30days/research.db` (overridable in tests via `store._db_override`). Initialization uses WAL mode, `PRAGMA foreign_keys=ON`, and lightweight versioned migrations.

### Schema (v1 + migration v2)

```mermaid
erDiagram
    topics ||--o{ research_runs : "topic_id"
    topics ||--o{ findings : "topic_id"
    research_runs ||--o{ findings : "run_id"
    findings ||--o{ finding_sightings : "finding_id"
    research_runs ||--o{ finding_sightings : "run_id"

    topics {
        int id PK
        text name UK
        text search_queries
        text schedule
        int enabled
    }

    research_runs {
        int id PK
        int topic_id FK
        text run_date
        text status
        real token_cost
        int findings_new
        int findings_updated
    }

    findings {
        int id PK
        text source_url UK
        int sighting_count
        real engagement_score
    }

    finding_sightings {
        int id PK
        int run_id FK
        int finding_id FK
    }
```

| Table / object | Role |
|----------------|------|
| `topics` | Watchlist entries; `name` is unique; optional `search_queries` JSON |
| `research_runs` | Per-execution metadata, cost, `findings_new` / `findings_updated` |
| `findings` | One row per `source_url` (UNIQUE); re-sightings bump `sighting_count` and `last_seen` |
| `finding_sightings` | Per-run ledger for `compute_topic_delta()` (new / continued / dropped URLs) |
| `findings_fts` | FTS5 (porter + unicode61) over content, summary, title, author |
| `settings` | Global knobs: budget, delivery, briefing defaults |

Default `settings` rows on first init:

| Key | Default |
|-----|---------|
| `daily_budget` | `5.00` |
| `delivery_channel` | empty |
| `delivery_mode` | `announce` |
| `briefing_format` | `concise` |
| `default_schedule` | `0 8 * * *` |

### Dedup and persistence rules

- Only findings with a `source_url` (or `url` in the input dict) are inserted or updated; URL-less items are skipped silently.
- Re-sighting the same URL updates engagement to `max(new, existing)`, increments `sighting_count`, and refreshes `last_seen`.
- `findings_from_report()` prefers `ranked_candidates`; supplements Hacker News and Polymarket from `items_by_source` when ranked; if ranking is empty, supplements all sources from raw items.

### Store query CLI

Run from the scripts directory or with the full repo path:

```bash
python3 skills/last30days/scripts/store.py stats
python3 skills/last30days/scripts/store.py query "your topic" --since 7d
python3 skills/last30days/scripts/store.py search "keyword" --limit 20
python3 skills/last30days/scripts/store.py trending --days 7
```

## Engine persistence: `--store` and `LAST30DAYS_STORE`

<ParamField body="--store" type="flag">
When set (or when env/config enables storage), after a successful pipeline run the engine calls `persist_report()`: upsert topic, record run, convert report to findings, `store_findings()`, mark run completed. Stderr prints `[last30days] Stored N new, M updated findings`.
</ParamField>

<ParamField body="LAST30DAYS_STORE" type="env">
Read from `os.environ` and loaded config (`.env` is not always mirrored into `os.environ`). Truthy values: `1`, `true`, `yes` (case-insensitive). Additive with `--store` — either enables persistence.
</ParamField>

<CodeGroup>
```bash title="Per-run flag"
python3 skills/last30days/scripts/last30days.py "british airways middle east" \
  --emit=compact --days=30 --store
```

```bash title="Always-on (.env)"
LAST30DAYS_STORE=1
```
</CodeGroup>

Manual `--store` runs still write markdown to `LAST30DAYS_MEMORY_DIR` when the save path is active; SQLite is additional, not a replacement.

## Watchlist (`watchlist.py`)

Manages scheduled topics and headless research loops. The `schedule` field on each topic is **metadata only** — you must invoke `run-one` or `run-all` from an external scheduler (cron, launchd, GitHub Actions, etc.).

### Commands

| Command | Purpose |
|---------|---------|
| `add <topic>` | Register topic; optional `--schedule`, `--weekly`, `--queries "q1,q2"` |
| `remove <topic>` | Delete topic and its findings / runs |
| `list` | JSON list of topics plus `budget_used` / `budget_limit` |
| `delta <topic>` | Compare latest two **completed** runs (new / continued / dropped URLs) |
| `run-one <topic>` | Single headless engine invocation + DB persist |
| `run-all` | All `enabled` topics until daily budget exceeded |
| `config delivery <url>` | Set `delivery_channel` (Slack or HTTPS) |
| `config budget <amount>` | Set `daily_budget` |

Schedule defaults:

| Flag | Stored cron | Description |
|------|-------------|-------------|
| (default) | `0 8 * * *` | Daily 8:00 |
| `--weekly` | `0 8 * * 1` | Mondays 8:00 |
| `--schedule "..."` | Custom | Stored as-is on the topic row |

### What `run-one` / `run-all` execute

Each enabled topic triggers:

```text
python3 last30days.py <search_term> --emit=json --quick --lookback-days 90
```

- `search_term` = first entry in `search_queries` JSON, else topic `name`
- Subprocess timeout: **300 seconds**
- On success: parse JSON → `findings_from_report(..., limit=25)` → `store_findings()`
- Failed runs set `research_runs.status = failed` with truncated stderr

`run-all` skips remaining topics when `store.get_daily_cost()` ≥ `daily_budget` (default **$5.00** per calendar day, UTC date on `run_date`).

### Webhook delivery

Fires only when `delivery_channel` is set **and** `counts["new"] > 0` after a successful run. Delivery failures are logged to stderr and do not fail the research run.

| Channel pattern | Payload |
|-----------------|---------|
| URL contains `hooks.slack.com` | `{"text": "<message>"}` |
| `https://...` otherwise | JSON: `message`, `source: "last30days"`, `timestamp` |

`delivery_mode` (DB default `announce`) controls message shape: `announce` (emoji + counts), `silent` (no emoji), or other (generic “complete” line). The `watchlist config` subcommand sets **delivery URL** and **budget** only — change `delivery_mode` via direct `store.set_setting()` if needed.

<Steps>
<Step title="Register a topic">
```bash
python3 skills/last30days/scripts/watchlist.py add "british airways middle east" --weekly
```
</Step>
<Step title="Configure delivery and budget (optional)">
```bash
python3 skills/last30days/scripts/watchlist.py config delivery "https://hooks.slack.com/services/..."
python3 skills/last30days/scripts/watchlist.py config budget 5.00
```
</Step>
<Step title="Schedule recurring execution">
Point cron (or equivalent) at:
```bash
python3 skills/last30days/scripts/watchlist.py run-all
```
Or one topic: `run-one "british airways middle east"`.
</Step>
<Step title="Inspect run-to-run change">
```bash
python3 skills/last30days/scripts/watchlist.py delta "british airways middle east"
```
Requires at least two completed runs; otherwise status is `insufficient_history`.
</Step>
</Steps>

<Info>
Headless watchlist runs use the engine’s internal planner path (`--emit=json` without `--plan`). Host-model reasoning keys are not required for cron-style execution; configure source API keys and `.env` the same as interactive runs.
</Info>

## Briefings (`briefing.py`)

Collects structured JSON for synthesis — the script does not render final prose; the skill/agent turns the payload into a readable briefing.

| Command | Output |
|---------|--------|
| `generate` | Daily digest data; default window: findings with `first_seen` since yesterday |
| `generate --weekly` | Week-over-week counts and engagement % change per topic |
| `generate --since YYYY-MM-DD` | Daily window override |
| `show [--date DATE]` | Load saved JSON from archive |

Archive directory: `~/.local/share/last30days/briefs/` as `{YYYY-MM-DD}.json` or `{YYYY-MM-DD}-weekly.json`.

Daily payload highlights:

- Per-topic `findings`, `new_count`, `last_run`, `last_status`
- `stale: true` when last completed run is **> 36 hours** ago
- `top_finding` per topic and cross-topic `top_finding` for TL;DR
- `cost.daily` vs `cost.budget`
- `failed_topics` list

Weekly payload adds `this_week_count`, `last_week_count`, `engagement_change_pct`, and top five findings per topic.

```bash
python3 skills/last30days/scripts/briefing.py generate
python3 skills/last30days/scripts/briefing.py generate --weekly
python3 skills/last30days/scripts/briefing.py show --date 2026-06-01
```

## Recommended baseline cadence

| Step | Cadence | Action |
|------|---------|--------|
| Baseline corpus | One-time per topic | `/last30days "<topic>"` with `--days=30` and `--store` (or `LAST30DAYS_STORE=1`) |
| Watchlist enrollment | One-time per topic | `watchlist.py add "<topic>"` (`--weekly` if Monday cadence fits) |
| Recurring research | Daily or weekly (your scheduler) | `watchlist.py run-all` or per-topic `run-one` |
| Digest | Weekly (typical) | `briefing.py generate --weekly`; use daily `generate` for operational triage |

<Warning>
The `schedule` column does not trigger runs by itself. Missing cron means topics go stale (`briefing.py` flags > 36h).
</Warning>

## Operational checks

| Symptom | Check |
|---------|-------|
| Empty brief | No enabled topics — `watchlist.py list`; add with `add` |
| Delta unavailable | Fewer than two completed runs for topic |
| Webhook never fires | `delivery_channel` unset, or zero **new** findings (updates alone do not notify) |
| `run-all` skips topics | Daily budget exhausted — `list` shows `budget_used` vs `budget_limit` |
| Thin watchlist results | Engine uses `--quick` and 90-day lookback; not full default-depth profile |

## Related pages

<CardGroup>
<Card title="Configuration reference" href="/configuration-reference">
Environment variables, `.env` lookup, and trend-monitoring toggles including `LAST30DAYS_STORE`.
</Card>
<Card title="CLI reference" href="/cli-reference">
`last30days.py` flags (`--store`, `--days`, `--emit`) and direct CLI vs slash-command constraints.
</Card>
<Card title="Research pipeline" href="/research-pipeline">
What `--quick` changes inside the v3 pipeline that watchlist runs hardcode.
</Card>
<Card title="Output contract" href="/output-contract">
Markdown save paths under `LAST30DAYS_MEMORY_DIR` alongside SQLite accumulation.
</Card>
<Card title="Quickstart" href="/quickstart">
First successful `/last30days` run before enabling `--store` or watchlist.
</Card>
</CardGroup>

---

## 13. Per-client setup

> Project-scoped .claude/last30days.env, save-dir and --save-suffix isolation, category peer subreddits, and recurring --competitors-plan templates.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/13-per-client-setup.md
- Generated: 2026-06-04T23:22:17.441Z

### Source Files

- `CONFIGURATION.md`
- `skills/last30days/scripts/lib/env.py`
- `skills/last30days/scripts/lib/categories.py`
- `skills/last30days/SKILL.md`
- `skills/last30days/scripts/last30days.py`

---
title: "Per-client setup"
description: "Project-scoped .claude/last30days.env, save-dir and --save-suffix isolation, category peer subreddits, and recurring --competitors-plan templates."
---

Per-client isolation in last30days-skill is configuration-driven: a project-scoped `.claude/last30days.env` overrides global credentials, `LAST30DAYS_MEMORY_DIR` and `--save-suffix` partition saved research files, `scripts/lib/categories.py` supplies vertical-specific Reddit peers, and checked-in `--competitors-plan` JSON templates reuse Step 0.55 targeting for recurring comparison runs.

## Configuration layers

```text
  Process environment (highest)
           │
           ▼
  .claude/last30days.env   ← walk-up from cwd (stops at $HOME)
           │
           ▼
  ~/.config/last30days/.env   ← global default
           │
           ▼
  macOS Keychain (last30days-* prefix, Darwin only)
```

<ParamField body="LAST30DAYS_CONFIG_DIR" type="string">
Override the global config directory. Set to an empty string for no-config (clean) mode. When unset, global config lives at `~/.config/last30days/.env`.
</ParamField>

The engine loads API keys and source toggles from this stack via `get_config()` in `scripts/lib/env.py`. Project env discovery walks parent directories from `Path.cwd()` until it finds `.claude/last30days.env` or reaches `$HOME`.

<Warning>
POSIX secret files should be mode `600`. The engine warns on every run when group/other read bits are set; Claude Code’s `check-config.sh` hook applies the same check at session start.
</Warning>

## Pattern 1: Project-scoped `.claude/last30days.env`

Place one env file per client workspace:

```text
acme-corp/
  .claude/
    last30days.env
  ...
```

Typical contents isolate credentials, enabled sources, and the save root:

```bash
LAST30DAYS_MEMORY_DIR=/Users/you/Clients/acme/Research/Last30Days
SCRAPECREATORS_API_KEY=<client-or-shared-key>
INCLUDE_SOURCES=tiktok,instagram
BSKY_HANDLE=acme.bsky.social
BSKY_APP_PASSWORD=<app-password>
```

<Steps>
<Step title="Create the file">
Add `.claude/last30days.env` at the client repo root (or any ancestor of where you run research). Restrict permissions: `chmod 600 .claude/last30days.env`.
</Step>
<Step title="Work from the client directory">
`cd` into the client folder before `/last30days <topic>`. The engine resolves the nearest project env on each run; no wrapper script is required for the common case.
</Step>
<Step title="Verify keys loaded">
Run `python3 skills/last30days/scripts/last30days.py --diagnose` from the client directory. Confirm per-source availability matches the client’s `.env` contents.
</Step>
</Steps>

<Note>
Slash-command invocations pass `--save-dir="${LAST30DAYS_MEMORY_DIR}"` from the SKILL.md bash preamble. Export `LAST30DAYS_MEMORY_DIR` in the shell (or have the hosting model read it from `.claude/last30days.env` before the engine call) so saves land in the client folder. API keys in the project file are picked up by `get_config()` without extra export.
</Note>

## Pattern 2: Save-dir and suffix isolation

Saved artifacts use slug-based filenames under the active save directory.

| Mechanism | Scope | Effect |
|-----------|-------|--------|
| `LAST30DAYS_MEMORY_DIR` | Persistent per client | Root directory for all saves when passed as `--save-dir` |
| `--save-dir <path>` | Single run | One-off output location |
| `--save-suffix <name>` | Single run | Inserts `-<suffix>` before the extension |

**Filename rules** (`save_output()` in `scripts/last30days.py`):

- Default markdown: `<slug>-raw[-suffix].md`
- JSON: `<slug>-raw[-suffix].json`
- HTML: `<slug>-raw-html[-suffix].html`
- Same topic + suffix on the same calendar day overwrites; a collision on a later day appends `-YYYY-MM-DD` before the extension.

Default save root when unset: `~/Documents/Last30Days/` (Windows: `C:\Users\<you>\Documents\Last30Days\`).

### Wrapper for batch or multi-client hosts

When you do not `cd` into each client tree, set the memory dir and suffix in one shell function:

<CodeGroup>
```bash title="Bash"
l30d-client() {
  local client=$1; shift
  LAST30DAYS_MEMORY_DIR="$HOME/Clients/$client/Research/Last30Days" \
    python3 skills/last30days/scripts/last30days.py "$@" \
      --emit=compact \
      --save-dir="$LAST30DAYS_MEMORY_DIR" \
      --save-suffix="$client"
}
# l30d-client acme "british airways middle east"
```

```powershell title="PowerShell"
function Run-L30D-Client {
    param([string]$ClientSlug, [Parameter(ValueFromRemainingArguments=$true)]$Args)
    $env:LAST30DAYS_MEMORY_DIR = "C:\Users\$env:USERNAME\Clients\$ClientSlug\Research\Last30Days"
    python3 skills/last30days/scripts/last30days.py @Args `
      --emit=compact `
      --save-dir=$env:LAST30DAYS_MEMORY_DIR `
      --save-suffix=$ClientSlug
}
# Run-L30D-Client acme "british airways middle east"
```
</CodeGroup>

<Info>
Combine Pattern 1 and Pattern 2: project env sets `LAST30DAYS_MEMORY_DIR` for the client tree; add `--save-suffix=<client-slug>` when multiple clients share one save root and you need distinct filenames for the same topic slug.
</Info>

### Comparison mode saves

Vs-mode (`--competitors-plan` or `--competitors`) writes one file per entity plus merged stdout output. Peer entities save as `{peer-slug}-raw[-suffix].md`; the main topic uses the leading entity slug. All paths honor the same `--save-dir` and `--save-suffix`.

## Pattern 3: Category peer subreddits

Brand-specific subreddits from WebSearch miss cross-product technique threads (the documented `GPT Image 2` failure mode). Step 0.55 Section 2a requires appending category peers for product topics.

**Hosting-model path (Claude Code, Codex, Gemini, etc.):** SKILL.md carries the canonical category table and merging rule—start with WebSearch subs, append 2–3 peers in priority order, dedupe case-insensitively, cap at 10 (WebSearch subs win when trimming). Emit `(+ {category_id} peers)` on the Resolved Reddit line when peers were added.

**Engine `--auto-resolve` path:** `scripts/lib/resolve.py` calls `categories.detect_category()` and `categories.peer_subs_for()` with the same cap and preservation rules.

### Extending the map

`scripts/lib/categories.py` defines `CATEGORY_PEERS`: each entry has `patterns` (compound terms, first-match-wins in declaration order) and `peer_subs` (priority-ordered subreddit names without `r/` prefix).

```python
# Illustrative new row — add inside CATEGORY_PEERS in categories.py
"legal_tech": {
    "patterns": ["legal tech", "contract automation", "clm platform"],
    "peer_subs": ["legaltech", "LawFirm", "SaaS"],
},
```

<Warning>
There is no runtime override file for categories. New verticals require a code change (or a beta/private fork). Keep patterns specific—bare tokens like `ai` or `model` are intentionally avoided to limit false positives.
</Warning>

| Category id | Example triggers | Peer subs (first entries) |
|-------------|------------------|---------------------------|
| `ai_image_generation` | gpt image, midjourney, stable diffusion | StableDiffusion, midjourney, dalle2, aiArt |
| `ai_coding_agent` | claude code, cursor ide, openclaw | ChatGPTCoding, LocalLLaMA, singularity |
| `saas_screen_recording` | loom video, tella screen | SaaS, screenrecording, productivity |
| `prediction_markets` | polymarket, kalshi | Polymarket, Kalshi, predictionmarkets |

Person topics, news stories, and topics outside any category skip peer expansion.

## Pattern 4: Recurring `--competitors-plan` templates

Store per-client competitor targeting as JSON beside the client workspace. Reuse across scheduled or ad hoc comparison runs instead of re-running full Step 0.55 discovery every time.

### Schema

Top-level object: entity name → targeting dict. Keys are normalized to lowercase. Inline JSON or a file path is accepted (same behavior as `--plan`).

<ParamField body="x_handle" type="string">
Official X/Twitter handle for the entity (`@` optional).
</ParamField>

<ParamField body="x_related" type="string[]">
Related handles searched at lower weight.
</ParamField>

<ParamField body="subreddits" type="string[]">
Subreddit names without `r/` prefix.
</ParamField>

<ParamField body="github_user" type="string">
GitHub username for person-mode search.
</ParamField>

<ParamField body="github_repos" type="string[]">
`owner/repo` strings for project-mode search.
</ParamField>

<ParamField body="context" type="string">
Free-text news/founding context passed into sub-runs.
</ParamField>

Unknown fields log a stderr warning and are ignored. Malformed JSON or a non-object top level exits with code 2.

### Example template

```json
{
  "Anthropic": {
    "x_handle": "AnthropicAI",
    "subreddits": ["ClaudeAI", "LocalLLaMA", "artificial"],
    "github_user": "anthropics",
    "context": "Founded 2021, Claude family of models"
  },
  "xAI": {
    "x_handle": "xai",
    "subreddits": ["singularity", "LocalLLaMA"],
    "github_user": "xai-org",
    "context": "Grok models, X integration"
  }
}
```

### Invocation

Main topic (first in the `vs` string) uses outer flags (`--x-handle`, `--subreddits`, `--github-user`, …). Peers take targeting from the plan via `subrun_kwargs_for()`—plan values beat engine `auto_resolve`, and main-topic flags cannot leak into peer sub-runs.

<RequestExample>
```bash
# Write JSON to a tmpfile when context strings contain apostrophes (SKILL.md heredoc pattern)
COMPETITORS_PLAN_FILE=$(mktemp "${TMPDIR:-/tmp}/last30days-competitors.XXXXXX")
trap 'rm -f "$COMPETITORS_PLAN_FILE"' EXIT
cat > "$COMPETITORS_PLAN_FILE" <<'PLAN_EOF'
{"Anthropic":{"x_handle":"AnthropicAI","subreddits":["ClaudeAI","LocalLLaMA"]}}
PLAN_EOF

python3 skills/last30days/scripts/last30days.py "OpenAI vs Anthropic vs xAI" \
  --emit=compact \
  --save-dir="${LAST30DAYS_MEMORY_DIR:-$HOME/Documents/Last30Days}" \
  --save-suffix=acme \
  --x-handle=OpenAI \
  --subreddits=OpenAI,ChatGPT,singularity \
  --competitors-plan "$COMPETITORS_PLAN_FILE"
```
</RequestExample>

<Check>
Prefer `--competitors-plan` over `--competitors-list` when peers need handles and subreddits. List-only mode falls back to thin planner defaults; empty fields in the Resolved Entities block indicate skipped Step 0.55 for that entity.
</Check>

File-path form for checked-in templates:

```bash
--competitors-plan clients/acme/competitors-plan.json
```

Refresh templates when handles, primary repos, or category peers change—peer sub-runs do not inherit the main topic’s subreddit list.

## Composing patterns

```text
┌─────────────────────────────────────────────────────────────┐
│ Client workspace                                            │
│  .claude/last30days.env  → keys, INCLUDE_SOURCES, save root │
│  competitors-plan.json   → recurring vs-mode targeting      │
│  (optional) categories.py fork row → vertical peer subs     │
└───────────────────────────┬─────────────────────────────────┘
                            │
         cd + /last30days    │    wrapper: LAST30DAYS_MEMORY_DIR
                            │              + --save-suffix
                            ▼
              Engine: get_config() + --save-dir + plan JSON
                            │
                            ▼
              ~/Clients/<slug>/Research/Last30Days/
                <topic>-raw-<suffix>.md
                <peer>-raw-<suffix>.md   (comparison)
```

| Client need | Primary pattern | Secondary |
|-------------|-----------------|-----------|
| Separate API keys per account | `.claude/last30days.env` | — |
| Separate output folders | `LAST30DAYS_MEMORY_DIR` in project env | `--save-dir` override |
| Same folder, same topic, different clients | `--save-suffix` | wrapper export |
| Niche SaaS vertical | `categories.py` row | SKILL.md 2a for model path |
| Monthly competitor brief | `competitors-plan.json` | `--competitors` for discovery refresh |

## Beta and private forks

Client-specific category rows, internal subreddit lists, and industry plan templates that should not ship on the public marketplace belong on the beta/private companion install (`/last30days-beta` from `mvanhorn/last30days-skill-private`). Promote upstream only through review against the public repo.

<Tip>
After editing `categories.py` or plan templates in a dev checkout, re-run `npx skills add . -g -y` or symlink `~/.agents/skills/last30days` to the working tree so harnesses load the updated skill copy.
</Tip>

## Verification

| Check | Command / signal |
|-------|------------------|
| Project env found | `--diagnose` shows expected keys; `_CONFIG_SOURCE` in debug paths may read `project:.claude/last30days.env` |
| Saves isolated | Footer line `📎 Raw results saved to …/<slug>-raw-<suffix>.md` points under the client `LAST30DAYS_MEMORY_DIR` |
| Category peers applied | Resolved block includes `(+ ai_image_generation peers)` or engine log `[Resolve] Matched category=…` |
| Plan threaded | No dashes in per-entity Resolved block; peer raw files exist for each competitor |

## Related pages

<CardGroup>
<Card title="Configure sources" href="/configure-sources">
Credential layers, `INCLUDE_SOURCES`, setup wizard, and `--diagnose` availability checks.
</Card>
<Card title="Configuration reference" href="/configuration-reference">
Full environment variable and `.env` lookup order.
</Card>
<Card title="Comparison mode" href="/comparison-mode">
`--competitors` fan-out, plan schema details, and synthesis scaffold.
</Card>
<Card title="Comparison recipes" href="/comparison-recipes">
Copy-paste vs-topic workflows and Head-to-Head table expectations.
</Card>
<Card title="Model clients" href="/model-clients">
Per-harness install patterns and stale-install avoidance.
</Card>
<Card title="Beta channel" href="/beta-channel">
Private-repo customizations that stay off the public marketplace.
</Card>
</CardGroup>

---

## 14. Configuration reference

> Environment variables, .env lookup order, reasoning provider priority, web-backend priority, output paths, and trend-monitoring toggles.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/14-configuration-reference.md
- Generated: 2026-06-04T23:22:46.853Z

### Source Files

- `CONFIGURATION.md`
- `skills/last30days/scripts/lib/env.py`
- `skills/last30days/scripts/lib/providers.py`
- `skills/last30days/scripts/last30days.py`
- `skills/last30days/SKILL.md`

---
title: "Configuration reference"
description: "Environment variables, .env lookup order, reasoning provider priority, web-backend priority, output paths, and trend-monitoring toggles."
---

Runtime configuration for `/last30days` resolves through `skills/last30days/scripts/lib/env.py` (`get_config()`), then per-run CLI flags in `last30days.py` override or extend those defaults. The slash-command contract in `skills/last30days/SKILL.md` governs voice and orchestration; this page lists every knob the engine reads.

## Configuration layers

Three layers stack from most transient to most persistent:

| Layer | Mechanism | Typical use |
|---|---|---|
| Per-run flags | `python3 skills/last30days/scripts/last30days.py ...` | `--save-dir`, `--web-backend`, `--store`, `--quick` / `--deep` |
| Process environment | `export VAR=value` before invocation | CI secrets, one-off overrides |
| Files + Keychain | `.claude/last30days.env`, `~/.config/last30days/.env`, macOS Keychain | Long-lived API keys and defaults |

Per-key resolution inside `get_config()` (highest wins):

```mermaid
flowchart TB
  subgraph highest["Highest priority"]
    OS["os.environ"]
  end
  subgraph files["File sources"]
    PROJ[".claude/last30days.env\n(walk up from cwd)"]
    GLOB["~/.config/last30days/.env\nor LAST30DAYS_CONFIG_DIR/.env"]
  end
  subgraph lowest["Lowest priority"]
    KC["macOS Keychain\nlast30days-{KEY}"]
  end
  KC --> MERGE["merged_env"]
  GLOB --> MERGE
  PROJ --> MERGE
  MERGE --> RESOLVE["config[key] = os.environ[key] or merged_env[key]"]
  OS --> RESOLVE
```

<Note>
`LAST30DAYS_CONFIG_DIR=""` disables global file loading (clean/no-config mode). `LAST30DAYS_CONFIG_DIR=/path` relocates the global `.env` to `/path/.env`.
</Note>

<Warning>
POSIX hosts: the engine warns on stderr when a secrets file is group- or world-readable. Set mode `600` on `.env` files.
</Warning>

## `.env` lookup paths

| Path | Scope | Precedence |
|---|---|---|
| `.claude/last30days.env` | Project tree (walk parents from `cwd` until home or root) | Overrides global file |
| `~/.config/last30days/.env` | User default | Fallback when no project file |
| `LAST30DAYS_CONFIG_DIR/.env` | Custom global directory | Replaces default `~/.config/last30days` |

Project-scoped env is the preferred pattern for per-client folders: `cd` into the client directory and run `/last30days` without wrapper scripts.

### macOS Keychain (additive, lowest priority)

On Darwin, `security find-generic-password` loads items with service name `last30days-{KEY}` (for example `last30days-BRAVE_API_KEY`). Keys mirrored in `scripts/setup-keychain.sh` include all major API credentials in `env.KEYCHAIN_KEYS`. Keychain never overrides values already present in `.env` or `os.environ`.

## Output paths

### `LAST30DAYS_MEMORY_DIR`

| Platform | Default when unset |
|---|---|
| Linux / macOS | `~/Documents/Last30Days/` |
| Windows | `C:\Users\<you>\Documents\Last30Days\` |

The skill passes `--save-dir="${LAST30DAYS_MEMORY_DIR}"` on agent invocations. Override with the env var or per-run `--save-dir <path>`.

### Filename rules

Each saved run uses a slug derived from the topic:

| Pattern | When |
|---|---|
| `<slug>-raw[-suffix].md` | Default markdown artifact |
| `<slug>-raw-html[-suffix].html` | `--emit=html` |
| `<slug>-raw[-suffix].json` | `--emit=json` |

- **`--save-suffix <name>`** — isolates variants (for example per client: `--save-suffix=acme`).
- **Collision** — same slug + suffix on the same calendar day overwrites; on a different day the engine appends `-YYYY-MM-DD` before the extension.

The footer line `📎 Raw results saved to …/<slug>-raw.md` is the canonical pointer in synthesized output.

### Trend-monitoring storage (separate from markdown saves)

| Artifact | Default path |
|---|---|
| SQLite research store | `~/.local/share/last30days/research.db` |
| Briefing JSON snapshots | `~/.local/share/last30days/briefs/` |
| Last-run metadata | `{LAST30DAYS_CONFIG_DIR or ~/.config/last30days}/last-run.json` |

Markdown under `LAST30DAYS_MEMORY_DIR` and SQLite under `~/.local/share/last30days/` are independent; `--store` / `LAST30DAYS_STORE` only affects SQLite.

## Reasoning provider priority

Headless runs (cron, watchlist, direct CLI without a hosting model supplying `--plan`) need one provider for planner + rerank. Pin explicitly or rely on auto-detect.

<ParamField body="LAST30DAYS_REASONING_PROVIDER" type="string" default="auto">
`auto` | `gemini` | `openai` | `xai` | `openrouter`. Resolved in `lib/providers.py` → `resolve_runtime()`.
</ParamField>

**Auto-detect order** (first match wins):

| Priority | Provider | Credential signal |
|---|---|---|
| 1 | Gemini | `GOOGLE_API_KEY`, `GEMINI_API_KEY`, or `GOOGLE_GENAI_API_KEY` |
| 2 | OpenAI | `OPENAI_API_KEY` with `OPENAI_AUTH_STATUS=ok` |
| 3 | xAI | `XAI_API_KEY` |
| 4 | OpenRouter | `OPENROUTER_API_KEY` |
| 5 | Local | No keys — deterministic planner, `local-score` rerank |

<Info>
When `/last30days` runs inside Claude Code, Codex, Gemini CLI, or similar, the **host model** is the reasoning provider for planning and synthesis. API keys above matter for headless `last30days.py` runs and for engine-side rerank when the host does not pass `--plan`.
</Info>

### Model pins

| Variable | Role |
|---|---|
| `LAST30DAYS_PLANNER_MODEL` | Override planner model for the resolved provider |
| `LAST30DAYS_RERANK_MODEL` | Override rerank model; Gemini `deep` profile defaults rerank to `gemini-3.1-pro-preview` |
| `OPENAI_MODEL_PIN` | Legacy pin alias consumed alongside provider defaults |
| `XAI_MODEL_PIN` | Legacy pin alias for X search and xAI defaults |
| `LAST30DAYS_X_MODEL` | X search model when using xAI backend |

Provider defaults (when pins unset): Gemini `gemini-3.1-flash-lite`, OpenAI `gpt-5.4-nano`, xAI `grok-4-1-fast`, OpenRouter `google/gemini-3.1-flash-lite-preview`. Gemini planner/rerank models must use the `gemini-3.1-*` prefix or the engine raises at startup.

### X search backend pin

<ParamField body="LAST30DAYS_X_BACKEND" type="string">
Optional pin: `xai` or `bird`. When unset, resolution order is `XAI_API_KEY` → explicit `AUTH_TOKEN` + `CT0` (Bird) → `xurl` CLI (OAuth2).
</ParamField>

## Web search backend priority

Grounding supplements (`grounding` source) power `--auto-resolve` and Step 2 web supplements when the host has no native WebSearch.

<ParamField body="--web-backend" type="choice" default="auto">
CLI choices: `auto`, `brave`, `exa`, `serper`, `parallel`, `none`. Implemented in `lib/grounding.py` → `web_search()`.
</ParamField>

**Auto-detect order** (in `grounding.web_search` and `pipeline.diagnose`):

| Priority | Backend | Key |
|---|---|---|
| 1 | Brave | `BRAVE_API_KEY` |
| 2 | Exa | `EXA_API_KEY` |
| 3 | Serper | `SERPER_API_KEY` |
| 4 | Parallel | `PARALLEL_API_KEY` |
| 5 | None / host | Empty key set — rely on harness native WebSearch (Claude Code, Codex, Gemini) |

`--web-backend=none` drops `grounding` from the active source list. Pinning `brave` without `BRAVE_API_KEY` raises a runtime error.

<Warning>
Thin cross-host results often trace to missing web keys on headless runs. Run `--diagnose` and compare `native_web_backend` against a machine that has Brave (or another key) configured.
</Warning>

## Source credentials and toggles

### API keys by source

| Source | Key(s) | Notes |
|---|---|---|
| Reddit, HN, Polymarket | none | Always on |
| GitHub | `gh` CLI auth or `GITHUB_TOKEN` | CLI preferred |
| YouTube | `yt-dlp` on PATH; optional `LAST30DAYS_YOUTUBE_SSH_HOST` | SSH-routes yt-dlp for datacenter bot-wall |
| X / Twitter | `XAI_API_KEY` **or** `AUTH_TOKEN`+`CT0` **or** `FROM_BROWSER` **or** xurl OAuth | See X backend table above |
| TikTok / Instagram / Threads | `SCRAPECREATORS_API_KEY` | Comma-separated keys rotate per run; legacy alias `SCRAPE_CREATORS_API_KEY` accepted |
| Pinterest | `SCRAPECREATORS_API_KEY` + request `pinterest` via `--search` | Opt-in at query time |
| Bluesky | `BSKY_HANDLE`, `BSKY_APP_PASSWORD` | App password `xxxx-xxxx-xxxx-xxxx` |
| Bluesky host | `BSKY_SEARCH_HOST` | Default `api.bsky.app` |
| Truth Social | `TRUTHSOCIAL_TOKEN` | Bearer from browser session |
| Web grounding | `BRAVE_API_KEY` / `EXA_API_KEY` / `SERPER_API_KEY` / `PARALLEL_API_KEY` | One is enough for auto mode |
| Perplexity Sonar | `OPENROUTER_API_KEY` + `INCLUDE_SOURCES=perplexity` | Additive source |
| Deep Research | `OPENROUTER_API_KEY` + `--deep-research` | ~$0.90/query |
| Apify (legacy) | `APIFY_API_TOKEN` | Fallback when ScrapeCreators exhausted |
| Xiaohongshu | `XIAOHONGSHU_API_BASE` | Default `http://host.docker.internal:18060` |
| Xquik | `XQUIK_API_KEY` | Alternate X search |

### `INCLUDE_SOURCES` and `EXCLUDE_SOURCES`

Comma-separated, case-insensitive lists in `.env` or environment.

| Variable | Behavior |
|---|---|
| `INCLUDE_SOURCES` | **Required** for `perplexity`, `youtube_comments`, `tiktok_comments`. When non-empty, quality scoring treats sources not listed as intentionally disabled. |
| `EXCLUDE_SOURCES` | Removes matching sources from `pipeline.available_sources()` (for example `EXCLUDE_SOURCES=threads,pinterest`). |

ScrapeCreators-backed TikTok, Instagram, and Threads enter the available set when `SCRAPECREATORS_API_KEY` is set; use `EXCLUDE_SOURCES` or `--search` to narrow the fan-out.

### Browser cookie extraction

<ParamField body="FROM_BROWSER" type="string">
`off` — disable extraction. `firefox` / `chrome` / `safari` — single browser. `auto` — try Firefox, Safari, then Chrome (Chrome may trigger Keychain prompts on macOS). **Default (unset):** Firefox and Safari only (silent).
</ParamField>

Extracted cookies map to `AUTH_TOKEN`/`CT0` (X) and `TRUTHSOCIAL_TOKEN` without overwriting explicit env values.

### Other engine tunables

| Variable | Default | Purpose |
|---|---|---|
| `LAST30DAYS_TRANSCRIPT_TIMEOUT` | `30` (seconds) | Instagram/SC transcript fetch timeout |
| `LAST30DAYS_DEBUG` | off | HTTP debug logging to stderr (`1` / `true` / `yes`) |
| `LAST30DAYS_SKIP_PREFLIGHT` | off | Bypass Class-1 keyword-trap refuse gate (exit 2) |
| `SETUP_COMPLETE` | — | Set by setup wizard after first configuration |
| `CODEX_AUTH_FILE` | `~/.codex/auth.json` | Codex auth path (OpenAI via API key is preferred; Codex JWT path is not used for planner auth in current `get_openai_auth`) |

## Trend monitoring toggles

Snapshot mode (default) writes slug-named markdown only. Continuous monitoring adds SQLite persistence plus optional scheduling scripts.

### Persistence toggle

| Mechanism | Effect |
|---|---|
| `--store` | Persist ranked findings to SQLite after the run |
| `LAST30DAYS_STORE=1` (or `true` / `yes`) | Same as `--store` on every run; flag and env are additive |

Findings dedupe on `source_url` (UNIQUE). Tables: `topics`, `research_runs`, `findings`, `settings` (`scripts/store.py`).

### Watchlist and briefings

| Script | Role |
|---|---|
| `scripts/watchlist.py` | `add`, `remove`, `list`, `run-one`, `run-all`, `config` — schedule metadata + Slack/webhook delivery on new findings |
| `scripts/briefing.py` | `generate`, `generate --weekly`, `show [--date DATE]` — structured digest data under `briefs/` |

Watchlist spawns the engine with `--quick` and `--lookback-days 90`. The stored `schedule` field is metadata; **cron / Task Scheduler / GitHub Actions** must call `run-one` or `run-all`.

<Steps>
<Step title="Baseline snapshot">
Run once with persistence: `/last30days "<topic>" --days=30 --store` (or `LAST30DAYS_STORE=1` in `.env`).
</Step>
<Step title="Register topic">
`python3 skills/last30days/scripts/watchlist.py add "<topic>" --weekly`
</Step>
<Step title="Schedule runs">
External scheduler: `python3 skills/last30days/scripts/watchlist.py run-all`
</Step>
<Step title="Weekly digest">
`python3 skills/last30days/scripts/briefing.py generate --weekly`
</Step>
</Steps>

## Verification

```bash
python3 skills/last30days/scripts/last30days.py --diagnose
```

Emits JSON with `providers`, `reasoning_provider`, `x_backend`, `native_web_backend`, `has_scrapecreators`, `has_github`, and `available_sources` — no full search executed.

<Check>
After editing `.env`, re-run `--diagnose` before a production topic. Confirm `native_web_backend` and expected sources appear under `available_sources`.
</Check>

## Example `.env` skeleton

```bash
# Reasoning (one provider; auto picks first match)
GOOGLE_API_KEY=<gemini-key>

# Web grounding (one key is enough)
BRAVE_API_KEY=<brave-key>

# Optional ScrapeCreators family
SCRAPECREATORS_API_KEY=<sc-key>
INCLUDE_SOURCES=tiktok,instagram,threads

# Always-on persistence
LAST30DAYS_STORE=1

# Output location
LAST30DAYS_MEMORY_DIR=$HOME/Clients/acme/Research/Last30Days
```

```bash
chmod 600 ~/.config/last30days/.env
# or: chmod 600 .claude/last30days.env
```

## Related pages

<CardGroup>
<Card title="Configure sources" href="/configure-sources">
Credential layers, setup wizard, and per-source availability checks.
</Card>
<Card title="Per-client setup" href="/per-client-setup">
Project-scoped `.claude/last30days.env`, save-dir isolation, and client wrappers.
</Card>
<Card title="Trend monitoring" href="/trend-monitoring">
SQLite schema, watchlist delivery, briefing synthesis, and cadence patterns.
</Card>
<Card title="CLI reference" href="/cli-reference">
Full `last30days.py` flag surface including `--emit` and targeting flags.
</Card>
<Card title="Troubleshooting" href="/troubleshooting">
`--diagnose` recovery, thin results, and stale marketplace installs.
</Card>
</CardGroup>

---

## 15. CLI reference

> last30days.py positional topic, all argparse flags, --emit choices, targeting flags, and direct CLI vs slash-command constraints.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/15-cli-reference.md
- Generated: 2026-06-04T23:22:55.940Z

### Source Files

- `skills/last30days/scripts/last30days.py`
- `CONFIGURATION.md`
- `tests/test_regression.py`
- `skills/last30days/scripts/verify_v3.py`
- `AGENTS.md`
- `pyproject.toml`

---
title: "CLI reference"
description: "last30days.py positional topic, all argparse flags, --emit choices, targeting flags, and direct CLI vs slash-command constraints."
---

`skills/last30days/scripts/last30days.py` is the v3 research engine: a Python 3.12+ `argparse` program that accepts a multi-word positional topic, fans out retrieval through `lib.pipeline`, and prints a rendered report on stdout while progress, promos, saves, and store writes go to stderr. The Agent Skills slash command (`/last30days`) is the primary product surface; direct CLI invocation is the supported path for scripting, cron, watchlist, verification, and engine testing.

## Invocation surfaces

```text
  Harness (Claude Code, Codex, Cursor, Gemini, …)
       │
       │  User: /last30days <topic>   (no shell pipes; model maps intent → flags)
       ▼
  SKILL.md contract ──► Bash: ${LAST30DAYS_PYTHON} ${SKILL_DIR}/scripts/last30days.py …
       │
       │  Operator / cron / watchlist / pytest
       ▼
  python3 skills/last30days/scripts/last30days.py "<topic>" [flags…]
       │
       ▼
  stdout: rendered --emit payload
  stderr: banner, promos, saves, store counts, quality nudges
```

<Warning>
Slash commands do not pass shell mechanics through. `/last30days OpenClaw --emit=html | pbcopy` is invalid in any harness. Use natural language in the slash form and let the hosting model translate intent into engine flags, or run the full `python3 …` command in a real shell when you need pipes, redirection, or explicit flags.
</Warning>

| Surface | When to use | Typical command shape |
| --- | --- | --- |
| Slash `/last30days` | Interactive research in an agent harness | Topic only; model adds `--emit=compact`, `--plan`, targeting, `--save-dir` per SKILL.md |
| Direct CLI | Cron, CI, watchlist, local debugging | Full flag list; stdout captured or redirected |
| Topic `setup` | First-run credential wizard | `last30days.py setup` plus unknown args (see Setup) |

From a repo checkout (dev/fallback):

```bash
python3 skills/last30days/scripts/last30days.py "openclaw skills" --emit=compact
uv run pytest tests/test_cli_v3.py   # CLI contract tests
```

Installed skill layout (production):

```bash
"${LAST30DAYS_PYTHON}" "${SKILL_DIR}/scripts/last30days.py" "$TOPIC" --emit=compact --save-dir="${LAST30DAYS_MEMORY_DIR}" --save-suffix=v3
```

`SKILL_DIR` must be the directory containing the `SKILL.md` the harness loaded. `LAST30DAYS_PYTHON` must be Python 3.12+ (resolved once per SKILL.md session).

## Positional topic

<ParamField body="topic" type="string (0+ tokens)" required>
Multi-word research query. Tokens are joined with spaces: `biosecurity ai agents` → `"biosecurity ai agents"`.
</ParamField>

| Case | Behavior |
| --- | --- |
| Normal topic | Required unless `--diagnose` alone (see Diagnostics) |
| Empty topic | Usage printed to stderr; exit code `2` |
| `setup` (case-insensitive) | Routes to setup wizard instead of research |
| `… vs …` / `… versus …` | Auto-enters comparison fan-out when the planner splits ≥2 entities (see Comparison) |

## `--emit` modes

Default: `compact`. All modes write human-oriented output to **stdout**; operational messages use **stderr**.

| Value | Renderer | Primary use |
| --- | --- | --- |
| `compact` | `render.render_compact` | Default skill path; badge + synthesis-oriented envelope |
| `md` | Same as `compact` | Debug/inspection; SKILL.md disallows as primary user-facing emit in harness runs |
| `json` | `schema.to_dict(report)` | Automation, tests, watchlist, `verify_v3.py` |
| `context` | `render.render_context` | Trimmed agent context (cluster-limited) |
| `html` | `html_render.render_html` | Shareable offline brief; pairs with `--synthesis-file` |

Comparison runs (`--competitors`, explicit list, or vs-topic) use parallel emit helpers: JSON wraps `{comparison, entities, reports[]}`; compact/md/context/html use multi-entity renderers.

<Note>
`--synthesis-file` is only honored when `--emit=html`. Otherwise the engine logs a warning and ignores the file.
</Note>

## Depth and source selection

| Flag | Effect |
| --- | --- |
| `--quick` | `depth="quick"` — lower latency (`per_stream_limit` 6, `pool_limit` 15, `rerank_limit` 12) |
| *(none)* | `depth="default"` |
| `--deep` | `depth="deep"` — higher recall (limits 20 / 60 / 60) |

<ParamField body="--search" type="comma-separated sources">
Restricts retrieval to named sources. Aliases normalized via `SEARCH_ALIAS` (`hn`→`hackernews`, `web`→`grounding`, `bsky`→`bluesky`, `truth`→`truthsocial`, `xhs`→`xiaohongshu`). Unknown or empty lists exit with an error.
</ParamField>

Valid source keys (also used by `--mock`): `reddit`, `x`, `youtube`, `tiktok`, `instagram`, `hackernews`, `bluesky`, `truthsocial`, `polymarket`, `grounding`, `xiaohongshu`, `github`, `perplexity`, `threads`, `pinterest`, `xquik`, `digg`.

## Targeting flags

Per-run targeting merges into `pipeline.run()` and is recorded in `report.artifacts["resolved"]`. When any of these are set, the renderer can emit a pre-research status warning if Step 0.55 was skipped in harness runs.

| Flag | Purpose |
| --- | --- |
| `--x-handle` | Primary X handle (leading `@` stripped) |
| `--x-related` | Comma-separated related handles (lower weight) |
| `--subreddits` | Comma-separated subs (`r/` prefix stripped) |
| `--tiktok-hashtags` | Hashtags without `#` |
| `--tiktok-creators` | Creator handles |
| `--ig-creators` | Instagram handles |
| `--github-user` | GitHub username for person-mode |
| `--github-repo` | Comma-separated `owner/repo` (canonicalized unless from `--auto-resolve`) |
| `--polymarket-keywords` | Comma-separated title filters for Polymarket disambiguation |
| `--days` / `--lookback-days` | Lookback window (default `30`; watchlist uses `90`) |

### Plan and auto-resolve

<ParamField body="--plan" type="JSON string or file path">
External query plan; skips the internal LLM planner when valid JSON loads. Invalid JSON logs to stderr and leaves planning unset.
</ParamField>

<ParamField body="--auto-resolve" type="boolean">
Engine-side Step 0.55/0.75 for hosts without WebSearch: discovers subreddits, X handle, GitHub user/repos before planning. Skipped when `--plan` is already provided.
</ParamField>

Harness models with WebSearch are expected to resolve targeting themselves and pass `--plan` or per-entity `--competitors-plan` instead of relying on `--auto-resolve`.

## Web backends and deep research

<ParamField body="--web-backend" type="enum" default="auto">
Choices: `auto`, `brave`, `exa`, `serper`, `parallel`, `none`. `auto` tries Brave → Exa → Serper → Parallel per config.
</ParamField>

| Flag | Requirement |
| --- | --- |
| `--deep-research` | Sets `OPENROUTER_API_KEY`; auto-appends `perplexity` to `INCLUDE_SOURCES`; exits `1` if key missing |

## Comparison mode

Comparison activates when any of these is true:

1. `--competitors` or `--competitors-list`
2. Topic contains ` vs ` or ` versus ` and splits into ≥2 entities

<ParamField body="--competitors" type="optional int (1–6)" default="2 when bare">
`--competitors` alone → discover **2** peers (3-way: topic + 2). `--competitors 4` → four discovered peers. Counts &lt; 1 exit `2`; counts &gt; 6 clamp with stderr warning.
</ParamField>

<ParamField body="--competitors-list" type="comma-separated names">
Skips discovery; list length wins over numeric `--competitors`. Empty list exits `2`. More than six entries clamp.
</ParamField>

<ParamField body="--competitors-plan" type="JSON object or file path">
Per-entity Step 0.55 targeting when comparison mode is already active. Top-level map: `{entity_name: {x_handle?, x_related?, subreddits?, github_user?, github_repos?, context?}}`. Keys normalized to lowercase; unknown fields warned and dropped. Invalid JSON or non-dict top-level exits `2`.
</ParamField>

Discovery requires a web backend or `--mock`; otherwise stderr prints the recommended WebSearch + `--competitors-plan` path and exits `2`. Fewer than two surviving sub-runs exits `1`.

Vs-topic routing sets the first entity as the main topic and remaining segments as the explicit competitor list. With `--save-dir`, comparison HTML saves one combined file plus per-entity raw files for peers.

## Output persistence

| Flag | Behavior |
| --- | --- |
| `--save-dir` | Writes slug-named file under directory; collision same day → date-stamped name |
| `--save-suffix` | Inserts suffix into filename (`topic-raw-SUFFIX.md`) |
| `--store` | Persists findings to SQLite (`scripts/store.py`) |

`LAST30DAYS_STORE=1` in environment or `~/.config/last30days/.env` enables store writes without the flag (same hybrid pattern as `LAST30DAYS_DEBUG`).

Save extensions follow emit: `.json`, `.html` (`raw-html` infix), else `.md`. Footer display uses `~/…` posix paths under the user home.

## Diagnostics, mock, and debug

| Flag | Behavior |
| --- | --- |
| `--diagnose` | Prints JSON availability report (`pipeline.diagnose`); **no topic required**; exit `0` |
| `--mock` | Fixture retrieval; bypasses live APIs |
| `--debug` | Sets `LAST30DAYS_DEBUG=1` for HTTP logging |

Preflight refusals (class-1 traps) honor `LAST30DAYS_SKIP_PREFLIGHT` and return exit `2` with message on stderr.

## Setup subcommand

`parse_known_args` leaves trailing argv available for setup-only flags:

| Invocation | Extra argv | Result |
| --- | --- | --- |
| `last30days.py setup` | *(none)* | `run_auto_setup`; writes config; human-readable status on stderr |
| `last30days.py setup` | `--device-auth` | JSON device-auth results on stdout |
| `last30days.py setup` | `--github` | JSON GitHub auth results |
| `last30days.py setup` | `--openclaw` | JSON OpenClaw setup results |

## Exit codes

| Code | Typical cause |
| --- | --- |
| `0` | Success; diagnose-only; setup completed |
| `1` | Comparison fan-out &lt; 2 entities; `--deep-research` without `OPENROUTER_API_KEY` |
| `2` | Missing topic; preflight refusal; competitor parse/validation errors; unreadable `--synthesis-file` |

Unsupported `--emit` values raise `SystemExit` inside emit helpers (parser choices prevent this in normal use).

## Flag inventory

### Output and persistence

| Flag | Type | Default |
| --- | --- | --- |
| `--emit` | `compact\|json\|context\|md\|html` | `compact` |
| `--save-dir` | path | — |
| `--save-suffix` | string | — |
| `--synthesis-file` | path | — (html only) |
| `--store` | boolean | off (unless env) |

### Retrieval profile

| Flag | Type |
| --- | --- |
| `--quick` | flag |
| `--deep` | flag |
| `--search` | string |
| `--mock` | flag |
| `--lookback-days` / `--days` | int (30) |
| `--web-backend` | enum (`auto`) |
| `--deep-research` | flag |

### Targeting and planning

| Flag | Type |
| --- | --- |
| `--plan` | JSON or file |
| `--auto-resolve` | flag |
| `--x-handle`, `--x-related` | string |
| `--subreddits` | string |
| `--tiktok-hashtags`, `--tiktok-creators` | string |
| `--ig-creators` | string |
| `--github-user`, `--github-repo` | string |
| `--polymarket-keywords` | string |

### Comparison

| Flag | Type |
| --- | --- |
| `--competitors` | optional int (const 2) |
| `--competitors-list` | string |
| `--competitors-plan` | JSON or file |

### Operations

| Flag | Type |
| --- | --- |
| `--diagnose` | flag |
| `--debug` | flag |

## Examples

<CodeGroup>

```bash title="Smoke test (repo root)"
python3 skills/last30days/scripts/last30days.py "test topic" --mock --emit=json
```

```bash title="Availability probe"
python3 skills/last30days/scripts/last30days.py --diagnose
```

```bash title="Headless comparison with explicit peers"
python3 skills/last30days/scripts/last30days.py "OpenAI" \
  --competitors-list "Anthropic,xAI" \
  --emit=json --mock
```

```bash title="HTML brief with saved synthesis"
python3 skills/last30days/scripts/last30days.py "OpenClaw" \
  --emit=html \
  --synthesis-file /tmp/synthesis.md \
  --save-dir ~/Documents/Last30Days
```

```bash title="Cron-friendly quick pass"
python3 skills/last30days/scripts/last30days.py "british airways middle east" \
  --quick --search=grounding,hackernews --emit=json
```

</CodeGroup>

<RequestExample>

```bash
python3 skills/last30days/scripts/last30days.py "openclaw vs nanoclaw" --mock --emit=json
```

</RequestExample>

<ResponseExample>

```json
{
  "comparison": true,
  "entities": ["openclaw", "nanoclaw"],
  "reports": [
    { "entity": "openclaw", "report": { "topic": "openclaw", "query_plan": { } } },
    { "entity": "nanoclaw", "report": { "topic": "nanoclaw", "query_plan": { } } }
  ]
}
```

</ResponseExample>

## Harness vs CLI constraints

<Steps>

<Step title="Slash command (default)">
User supplies topic and intent in natural language. The model reads SKILL.md, resolves `SKILL_DIR` and `LAST30DAYS_PYTHON`, runs Step 0.55 / plan generation when required, and invokes the engine with `--emit=compact` (not `md` as the primary path), plus `--save-dir` / `--save-suffix=v3` and targeting flags. Output is passed through verbatim (badge, evidence envelope, footer).
</Step>

<Step title="Direct CLI (scripting)">
Operator supplies every flag explicitly. Shell pipes, env exports, and cron wrappers are valid. Use `--emit=json` or `--mock` for automation; use `--diagnose` before debugging missing sources.
</Step>

</Steps>

<Tip>
New engine flags are incomplete until SKILL.md documents them — harness models only learn flags present in the skill contract. For env-backed defaults and save paths, see the configuration reference rather than duplicating every variable here.
</Tip>

## Related pages

<CardGroup>

<Card title="Skill, engine, and harness" href="/skill-engine-harness">
Boundaries between the Agent Skills package, Python engine, and multi-harness runtimes.
</Card>

<Card title="Output contract" href="/output-contract">
Badge line, LAWs voice rules, evidence envelopes, and emit semantics the harness must pass through.
</Card>

<Card title="Configuration reference" href="/configuration-reference">
Environment variables, `.env` lookup order, reasoning and web-backend priority, and store toggles.
</Card>

<Card title="Comparison mode" href="/comparison-mode">
Discovery fan-out, `--competitors-plan` schema, and multi-entity synthesis expectations.
</Card>

<Card title="Develop and test" href="/develop-and-test">
`uv run pytest`, `verify_v3.py`, and regression fixtures for the CLI surface.
</Card>

</CardGroup>

---

## 16. Data sources

> Per-platform retrieval modules, keyless vs keyed sources, --search alias map, and engagement scoring inputs by source.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/16-data-sources.md
- Generated: 2026-06-04T23:23:29.213Z

### Source Files

- `skills/last30days/scripts/lib/pipeline.py`
- `CONFIGURATION.md`
- `skills/last30days/scripts/lib/reddit_public.py`
- `skills/last30days/scripts/lib/bird_x.py`
- `skills/last30days/scripts/lib/github.py`
- `skills/last30days/scripts/lib/polymarket.py`
- `skills/last30days/scripts/lib/youtube_yt.py`

---
title: "Data sources"
description: "Per-platform retrieval modules, keyless vs keyed sources, --search alias map, and engagement scoring inputs by source."
---

Each research run fans out through `skills/last30days/scripts/lib/pipeline.py`, which resolves which platforms are callable (`available_sources`), normalizes CLI `--search` tokens, dispatches per-source retrieval in `_retrieve_stream`, and passes normalized `SourceItem` records through `signals.annotate_stream` for engagement-aware local ranking before fusion and LLM rerank.

## Source inventory

Canonical source names match `pipeline.MOCK_AVAILABLE_SOURCES` and `last30days.parse_search_flag` validation. Retrieval modules live under `skills/last30days/scripts/lib/`; `_retrieve_stream` is the single dispatch table.

| Source | Module | Auth / prerequisites | Default availability |
|--------|--------|----------------------|----------------------|
| `reddit` | `reddit_public.py` → `reddit_keyless.py`; backup `reddit.py` (ScrapeCreators) | Keyless tiers first; `SCRAPECREATORS_API_KEY` for paid backup | Always |
| `hackernews` | `hackernews.py` | None (Algolia) | Always |
| `polymarket` | `polymarket.py` | None (Gamma API) | Always |
| `github` | `github.py` | `GITHUB_TOKEN` or `gh` CLI | When token/CLI present |
| `youtube` | `youtube_yt.py` | `yt-dlp` on PATH; optional `SCRAPECREATORS_API_KEY` fallback + comment enrichment | When `yt-dlp` or SC YouTube path available |
| `grounding` | `grounding.py` | One of `BRAVE_API_KEY`, `EXA_API_KEY`, `SERPER_API_KEY`, `PARALLEL_API_KEY` (or forced via `--web-backend`) | When a web-search key is set |
| `x` | `bird_x.py`, `xai_x.py`, or `xurl_x.py` | Bird: `AUTH_TOKEN`+`CT0`+Node; xAI: `XAI_API_KEY`; xurl: authenticated CLI | When `env.get_x_source()` resolves a backend |
| `tiktok` | `tiktok.py` | `SCRAPECREATORS_API_KEY` or `APIFY_API_TOKEN` | When SC/Apify key set (see opt-out below) |
| `instagram` | `instagram.py` | `SCRAPECREATORS_API_KEY` | When SC key set |
| `threads` | `threads.py` | `SCRAPECREATORS_API_KEY` | When SC key set |
| `bluesky` | `bluesky.py` | `BSKY_HANDLE` + `BSKY_APP_PASSWORD` | When credentials set |
| `truthsocial` | `truthsocial.py` | `TRUTHSOCIAL_TOKEN` | When token set |
| `digg` | `digg.py` | `digg-pp-cli` on PATH | When CLI installed |
| `perplexity` | `perplexity.py` | `OPENROUTER_API_KEY` + `INCLUDE_SOURCES` contains `perplexity` (or explicit `--search=perplexity`) | Opt-in |
| `pinterest` | `pinterest.py` | `SCRAPECREATORS_API_KEY` | Only when `--search` (or planner) explicitly requests `pinterest` |
| `xiaohongshu` | `xiaohongshu_api.py` | Reachable `XIAOHONGSHU_API_BASE` with logged-in session | Only when explicitly requested and health probe passes |
| `xquik` | `xquik.py` | `XQUIK_API_KEY` | When key set |

<Note>
`available_sources()` is the runtime gate. User-facing docs sometimes mention `INCLUDE_SOURCES=tiktok`; in code, TikTok and Instagram are added whenever `SCRAPECREATORS_API_KEY` is set. Use `EXCLUDE_SOURCES=tiktok,instagram` to deny them without removing the key.
</Note>

## Keyless vs keyed retrieval

**Keyless (no API key in config):** Reddit (public/keyless pipeline), Hacker News, Polymarket, and GitHub when only the unauthenticated search API is used (higher rate limits with `GITHUB_TOKEN` or `gh`).

**Keyed or tool-backed:** Everything else depends on env vars, CLI tools, or session cookies as in the table above.

### Reddit tiering

Reddit is the most layered source:

1. **Tier 0** — legacy `reddit.com/search.json` via `reddit_public` (often 403; cheap one-shot).
2. **Tier 1** — RSS discovery (`reddit_rss.py`) inside `reddit_keyless.py`.
3. **Tier 2** — Shreddit comment enrichment (`reddit_shreddit.py`).
4. **Backup** — `reddit.search_and_enrich` via ScrapeCreators when keyless tiers return empty and `SCRAPECREATORS_API_KEY` is set.

Pipeline passes `raw_topic` (not only the planner sub-query) into Reddit/YouTube/TikTok/Instagram so query expansion stays anchored to the user topic.

### X backend selection

`env.get_x_source()` picks the active X path: `LAST30DAYS_X_BACKEND` pin, else `xai` if `XAI_API_KEY`, else Bird when explicit `AUTH_TOKEN`/`CT0` and vendored `bird-search.mjs`+Node, else `xurl`. Browser cookie probing is disabled in normal runs (`BIRD_DISABLE_BROWSER_COOKIES=1`).

### YouTube dual path

`_retrieve_stream` tries `youtube_yt.search_and_transcribe` (yt-dlp) first, then `search_youtube_sc` when SC is configured. Optional `youtube_comments` in `INCLUDE_SOURCES` triggers `enrich_with_comments` on top-N videos by engagement.

### Web grounding (`grounding`)

Registered as source `grounding`. CLI alias `web` maps to it. Backends: Brave, Exa, Serper, Parallel — selected from keys or `--web-backend`. Planner may append `grounding` to every sub-query when available; `--web-backend none` removes it.

## `--search` alias map

The `--search` flag accepts a comma-separated list. Tokens are lowercased, aliased, deduped, and validated against `MOCK_AVAILABLE_SOURCES`.

| CLI token | Canonical source |
|-----------|------------------|
| `hn` | `hackernews` |
| `bsky` | `bluesky` |
| `truth` | `truthsocial` |
| `web` | `grounding` |
| `xhs` | `xiaohongshu` |
| `xquik` | `xquik` |
| *(any other valid name)* | unchanged (e.g. `reddit`, `youtube`) |

```python
# pipeline.SEARCH_ALIAS (representative)
SEARCH_ALIAS = {
    "hn": "hackernews",
    "bsky": "bluesky",
    "truth": "truthsocial",
    "web": "grounding",
    "xhs": "xiaohongshu",
    "xquik": "xquik",
}
```

After normalization, `run()` intersects the list with `available_sources()`. Unknown tokens exit with `Unknown search source`. An empty list exits with `--search requires at least one source.`

<Warning>
Slash-command invocations do not pass shell flags. Restricting sources belongs in engine flags the hosting model translates, or in direct CLI: `python3 skills/last30days/scripts/last30days.py "topic" --search=reddit,youtube`.
</Warning>

## INCLUDE_SOURCES, EXCLUDE_SOURCES, and per-run opt-in

| Mechanism | Effect |
|-----------|--------|
| `EXCLUDE_SOURCES` | Removed from `available_sources()` (comma-separated, case-insensitive). Primary way to suppress TikTok/Instagram while keeping `SCRAPECREATORS_API_KEY`. |
| `INCLUDE_SOURCES=perplexity` | Enables Perplexity Sonar when `OPENROUTER_API_KEY` is set (unless `--search=perplexity` forces it). |
| `INCLUDE_SOURCES=youtube_comments` | Top-video comment enrichment via ScrapeCreators (requires SC key). |
| `INCLUDE_SOURCES=tiktok_comments` | Top-post TikTok comment enrichment (requires SC key). |
| Explicit `--search=pinterest` / `xiaohongshu` | Required for those sources even when keys/health checks pass. |

`quality_nudge.py` treats a **non-empty** `INCLUDE_SOURCES` as an allowlist when judging silent Instagram failures (zero items may be expected if `instagram` was omitted intentionally).

## Retrieval dispatch flow

```mermaid
flowchart TB
  subgraph cli [CLI / harness]
    L30[last30days.py]
    SF["--search comma list"]
  end
  subgraph pipe [pipeline.py]
    NS[normalize_requested_sources]
    AV[available_sources]
    PL[planner.plan_query]
    RS["_retrieve_stream per subquery × source"]
    NORM["_normalize_score_dedupe"]
  end
  subgraph mods [lib/*.py source modules]
    R[reddit_public / reddit]
    X[bird_x / xai_x / xurl_x]
    YT[youtube_yt]
    PM[polymarket]
    GH[github]
    GR[grounding]
    OTH[tiktok instagram hn bluesky ...]
  end
  L30 --> SF --> NS
  NS --> AV --> PL --> RS
  RS --> R & X & YT & PM & GH & GR & OTH
  RS --> NORM
```

Parallel fan-out uses a `ThreadPoolExecutor` sized from sub-query × source count. `MAX_SOURCE_FETCHES` caps **X** at two fetches per run to limit duplicate API cost.

## Normalization contract

`normalize.normalize_source_items()` maps each source’s raw dicts into `schema.SourceItem` with a shared `engagement` dict. Per-source normalizers are registered by source name (`reddit`, `x`, `youtube`, `polymarket`, etc.). Comment threads from Reddit, YouTube, and TikTok are remapped to `{score, excerpt}` in metadata `top_comments` for downstream scoring and rendering.

## Engagement scoring

Engagement is computed in two stages: **raw fields** from retrieval modules, then **pipeline scores** in `signals.py`.

### Stage 1 — Raw `engagement` dict by source

| Source | Typical `engagement` keys (from retrieval / normalize) |
|--------|--------------------------------------------------------|
| `reddit` | `score`, `num_comments`, `upvote_ratio`; top comment via metadata |
| `x` / `xquik` | `likes`, `reposts`, `replies`, `quotes` |
| `youtube` | `views`, `likes`, `comments` |
| `tiktok` / `instagram` | `views`, `likes`, `comments` (short-form video normalizer) |
| `hackernews` | `points`, `comments` |
| `bluesky` / `truthsocial` / `threads` | `likes`, `reposts`, `replies`, `quotes` (microblog shape) |
| `polymarket` | `volume` (from `volume1mo` or `volume24hr`), `liquidity` |
| `digg` | `postCount`, `uniqueAuthors`, `rank_score` |
| `pinterest` | `saves`, `comments` |
| `github` | `reactions`, `comments` (issue/PR search); profile modes use `reactions` as star-like signals |
| `xiaohongshu` | `likes`, `comments`, `favorites` |
| `grounding` / `perplexity` | Usually empty — ranking leans on text relevance and freshness |

### Stage 2 — `engagement_raw` and `local_rank_score`

`signals.engagement_raw()` applies per-source formulas (all use `log1p` on numeric inputs):

| Source | Formula class |
|--------|----------------|
| `reddit` | Custom: 50% score + 35% comments + 5% upvote_ratio + 10% top comment |
| `youtube` | Custom: 45% views + 32% likes + 13% comments + 10% top comment |
| `tiktok` | Custom: 45% views + 27% likes + 18% comments + 10% top comment |
| `x` | Weighted: likes 0.55, reposts 0.25, replies 0.15, quotes 0.05 |
| `instagram` | Weighted: views 0.50, likes 0.30, comments 0.20 |
| `hackernews` | Weighted: points 0.55, comments 0.45 |
| `bluesky` | Weighted: likes 0.40, reposts 0.30, replies 0.20, quotes 0.10 |
| `truthsocial` | Weighted: likes 0.45, reposts 0.30, replies 0.25 |
| `polymarket` | Weighted: volume 0.60, liquidity 0.40 |
| `digg` | Weighted: postCount 0.40, uniqueAuthors 0.30, rank_score 0.30 |
| Others with sparse metrics | `_generic_engagement`: mean of log1p across all numeric engagement values |

Scores are **min–max normalized to 0–100** within each stream batch (`signals.normalize`), stored on `SourceItem.engagement_score`.

**Local rank** (pre-fusion sort within a stream):

```
local_rank_score = 0.65 × local_relevance
                 + 0.25 × (freshness / 100)
                 + 0.10 × (engagement_score / 100)
```

`local_relevance` uses token overlap (`relevance.token_overlap_relevance`) with source-specific floors (e.g. YouTube views > 100k → min 0.3; GitHub `project-mode` label → min 0.8).

**Source quality** (`SOURCE_QUALITY` in `signals.py`) is a separate editorial multiplier used later in fusion weighting — not the same as engagement fields.

### Pruning rules tied to engagement

- `prune_low_relevance`: social sources with zero `engagement_score` need ~1.5× the relevance floor to survive.
- TikTok/Instagram: items with **&lt; 1000 views** drop when other sources contributed items (unless that source is the only one in the batch).

## Planner capabilities (source roles)

`planner.SOURCE_CAPABILITIES` tags each source with roles (`discussion`, `social`, `video`, `web`, `market`, …). The planner (LLM or deterministic fallback) picks sub-queries and source sets from **available** sources and query intent. `INTENT_SOURCE_EXCLUSIONS` drops `polymarket` for `concept` and `how_to` intents.

## Verification

<Steps>
<Step title="List resolved sources">

```bash
python3 skills/last30days/scripts/last30days.py --diagnose
```

Confirms keys, X backend, native web backend, and `available_sources` for the current config.

</Step>
<Step title="Run a source-scoped smoke test">

```bash
python3 skills/last30days/scripts/last30days.py "test topic" --quick --search=grounding,hackernews --emit=compact
```

Uses aliases (`hn` → `hackernews`, `web` → `grounding`) the same way as full names.

</Step>
</Steps>

## Related pages

<CardGroup>
<Card title="Configure sources" href="/configure-sources">
Credential layers, INCLUDE_SOURCES, setup wizard, and --diagnose availability checks.
</Card>
<Card title="Research pipeline" href="/research-pipeline">
Planner sub-queries, parallel fan-out, fusion, dedupe, and depth profiles after retrieval.
</Card>
<Card title="CLI reference" href="/cli-reference">
--search, targeting flags, and direct CLI vs slash-command constraints.
</Card>
<Card title="Configuration reference" href="/configuration-reference">
Environment variables and web-backend priority that gate grounding and reasoning.
</Card>
</CardGroup>

---

## 17. Skill contract

> SKILL.md runtime authority: STEP 0 stale-clone guard, pre-flight protocol, engine invocation rules, and SKILL_DIR substitution.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/17-skill-contract.md
- Generated: 2026-06-04T23:23:05.702Z

### Source Files

- `skills/last30days/SKILL.md`
- `skills/last30days/scripts/lib/skill_meta.py`
- `AGENTS.md`
- `.claude-plugin/plugin.json`
- `skills/last30days/scripts/last30days.py`

---
title: "Skill contract"
description: "SKILL.md runtime authority: STEP 0 stale-clone guard, pre-flight protocol, engine invocation rules, and SKILL_DIR substitution."
---

`skills/last30days/SKILL.md` is the runtime authority for `/last30days`: the harness loads it when the slash command fires, the model must follow its step order before any engine Bash call, and `scripts/last30days.py` is invoked only through the `SKILL_DIR`-anchored path of the SKILL.md instance that was actually read. The contract is not optional metadata — it defines stale-install guards, pre-flight resolution, plan JSON handoff, and output-shape LAWs that the engine and `lib/render.py` enforce on stdout.

## Skill, engine, and harness boundaries

| Layer | Artifact | Role |
|-------|----------|------|
| **Skill** | `SKILL.md` + sibling `scripts/` | Agent-facing prose contract: steps, flags, synthesis LAWs, failure-mode guards |
| **Engine** | `scripts/last30days.py` + `scripts/lib/*` | Retrieval, planning intake, clustering, `--emit=compact` rendering |
| **Harness** | Claude Code, Codex, Cursor, `~/.agents/skills/`, etc. | Loads SKILL.md via Read; does not pass shell flags through slash invocations |

<Note>
Slash commands cannot carry pipes or flags (`/last30days topic --emit=html` is invalid in harnesses). The model translates user intent into engine flags per SKILL.md. Direct CLI (`python3 …/last30days.py`) is for scripting, cron, and engine testing only.
</Note>

```mermaid
flowchart TB
  subgraph harness["Harness"]
    U["User: /last30days topic"]
    R["Read SKILL.md"]
  end
  subgraph skill["Skill contract — SKILL.md"]
    S0["STEP 0 stale-clone guard"]
    PF["Pre-flight 0.45 → 0.5 → 0.55 → 0.75"]
    INV["SKILL_DIR engine invocation"]
    LAW["OUTPUT CONTRACT / LAWs 1–8"]
  end
  subgraph engine["Engine — last30days.py"]
    PRE["lib/preflight.py Class 1 gate"]
    PIPE["lib/pipeline.py"]
    REN["lib/render.py compact + warnings"]
  end
  U --> R --> S0 --> PF --> INV --> PRE --> PIPE --> REN
  REN --> LAW
```

Version strings stay aligned across three surfaces: SKILL.md frontmatter (`version: "3.3.1"`), `.claude-plugin/plugin.json`, and `lib/skill_meta.read_skill_version()` used by `lib/render.py` for badge emission.

## STEP 0: Stale-clone self-check

STEP 0 runs **before** the rest of SKILL.md. It targets one known bad load path: Claude Code’s **marketplaces** git clone at `~/.claude/plugins/marketplaces/last30days-skill/`, which auto-restores to `origin/main` and can lag the versioned plugin cache.

<Steps>
<Step title="Detect cache SKILL.md">
Run the bash probe in STEP 0 to resolve `CLAUDE_CACHE_SKILL_MD` from `~/.claude/plugins/cache/last30days-skill/last30days/{version}/` (nested `skills/last30days/SKILL.md` or flat `SKILL.md`).
</Step>
<Step title="Compare load path">
If the SKILL.md you Read contains `/.claude/plugins/marketplaces/` **and** the cache path is non-empty, **stop** and re-read `$CLAUDE_CACHE_SKILL_MD` before continuing.
</Step>
<Step title="Otherwise continue">
Codex (`~/.codex/skills/`), `~/.agents/skills/`, `npx skills add` installs, and repo checkouts are valid — do not hop installs on those paths.
</Step>
</Steps>

<Warning>
A stale marketplace clone caused real regressions: models ran `--help` from the stale tree, missed flags such as `--competitors`, and improvised comparison flows. STEP 0 exists only for that Claude Code layout mismatch.
</Warning>

## Skill contract preface (three structural anchors)

Immediately after STEP 0, the **SKILL CONTRACT** block states that `/last30days` is a fixed research tool, not a generic “last 30 days” prompt. Three anchors prevent the documented v3.0.6-style improvisations:

1. **Mandatory badge** — first line `🌐 last30days v{VERSION} · synced {YYYY-MM-DD}` (engine emits it in `--emit=compact`; model passes through or builds it from `SKILL_DIR` + `plugin.json` / SKILL frontmatter).
2. **SKILL_DIR substitution** — engine path is the directory of the SKILL.md just Read; no install-path resolver walk.
3. **Read top-to-bottom** — no WebSearch-only answers, no `for dir in …` path discovery, no bare `python3 scripts/last30days.py` without pre-flight flags on named entities.

The **OUTPUT CONTRACT** (LAWs 1–8) was moved high in the file so long SKILL.md reads still load synthesis rules before the model stops reading (~line 1000). LAWs cover: no trailing `Sources:` (LAW 1), `What I learned:` vs comparison title (LAW 2), hyphen not em-dash (LAW 3), no `##` in GENERAL bodies (LAW 4), emoji-tree footer pass-through (LAW 5), no raw evidence clusters in user output (LAW 6), `--plan` on named entities (LAW 7), inline `[name](url)` citations (LAW 8).

## Invocation order (after STEP 0)

| Step | Name | Requirement |
|------|------|-------------|
| Runtime preflight | `LAST30DAYS_PYTHON` | Python 3.12+ (`python3.14` → `python3.12` → `python3`) |
| First-run | Step 0 wizard | If `~/.config/last30days/.env` lacks `SETUP_COMPLETE=true`, run `nux-wizard.md` |
| WebSearch load | STEP 0 (invoke section) | `ToolSearch select:WebSearch` first on Claude Code (deferred tool) |
| Intent | CRITICAL: Parse User Intent | `TOPIC`, `QUERY_TYPE`, optional `TARGET_TOOL`; branded one-liner, then 0.45 |
| 0.45 | Query quality pre-flight | Keyword-trap classes 1–4; ask or reframe before engine |
| 0.5 | Handle / repo resolution | `--x-handle`, `--x-related`, `--github-user`, `--github-repo` per checklist |
| 0.55 | Pre-research intelligence | Subreddits, TikTok/IG targets; mandatory when WebSearch exists |
| 0.75 | Query plan | Model-authored JSON → tmpfile → `--plan` |
| 1 | Research execution | `SKILL_DIR/scripts/last30days.py` foreground, 300s timeout, `--emit=compact` |
| 2 | WebSearch supplements | 2–3 post-engine searches (separate budget from 0.55) |

<Info>
Without a user topic, ask once for a topic — do not run WebSearch or the engine. With a topic, the Python engine is mandatory; missing the `✅ All agents reported back!` footer means the skill was not run.
</Info>

## Pre-flight protocol

### Step 0.45 — Topic quality (model-side)

Four keyword-trap classes drive clarify-or-reframe behavior **before** Bash:

| Class | Pattern | Action |
|-------|---------|--------|
| 1 | Demographic shopping (`gift for 42 year old man`) | One clarifying question, or reframe + gift subreddits |
| 2 | Numeric collisions (`42`, `100`) | Strip number from engine query when not semantically load-bearing |
| 3 | Tutorial phrasing (`how to use Docker`) | Reframe to discussion vocabulary |
| 4 | Generic single noun (`sneakers`) | Ask for facet before running |

Engine mirror: `lib/preflight.check_class_1_trap()` refuses Class 1 on stderr (exit 2) unless qualifiers (budget, hobbies, `who loves …`) are present. Bypass: `LAST30DAYS_SKIP_PREFLIGHT=1`.

### Step 0.5 — Targeting checklist (model-side)

The checklist is the **full** contract — not only X handles:

| Flag | When |
|------|------|
| `--x-handle` | Person, brand, product, creator with X presence |
| `--x-related` | Founders, collaborators, commentators |
| `--github-user` | Person who ships code (peer to X resolution) |
| `--github-repo` | Product / OSS project |
| `--subreddits` | Nearly always |
| `--tiktok-hashtags` / `--tiktok-creators` / `--ig-creators` | Per topic class |
| `--auto-resolve` | Fallback when WebSearch ran but 0.55 incomplete |

<Warning>
Person topics require **minimum** `--x-handle` + `--github-user` + `--subreddits` (typically `--x-related` too). X-handle-only invocations are a documented regression shape.
</Warning>

### Step 0.55 and 0.75 — WebSearch platform gate

When WebSearch is available, both steps are mandatory before engine invocation. When unavailable (OpenClaw, headless CLI), skip 0.55/0.75 and pass `--auto-resolve` so the engine discovers subreddits/handles via configured web backends.

Step 0.75 produces `QUERY_PLAN_JSON` (intent, freshness_mode, cluster_mode, 1–4 subqueries with sources/weights). Primary subquery must include reddit, x, youtube, tiktok, instagram, hackernews, polymarket.

### Engine-visible warnings

If pre-flight flags or `--plan` are missing on eligible named-entity topics, `lib/render.py` injects:

- `## Pre-Research Status` — Step 0.55 skip (keyword-only retrieval)
- `## DEGRADED RUN WARNING` — deterministic planner + bare invocation (LAW 7 backstop on stdout, not stderr)

Pass these blocks through verbatim (LAW 4 exception for engine-emitted status blocks; LAW 5/7 for degraded banner).

## SKILL_DIR substitution and engine invocation

`SKILL_DIR` is the absolute directory containing the SKILL.md the harness Read. `scripts/last30days.py` is always `${SKILL_DIR}/scripts/last30days.py`. Substitute the real path from the Read result — do not walk `~/.openclaw/skills/`, marketplace trees, or multi-install precedence lists.

```bash
SKILL_DIR="<absolute path of the directory containing the SKILL.md you Read>"

if [ ! -f "$SKILL_DIR/scripts/last30days.py" ]; then
  echo "ERROR: scripts/last30days.py not found under SKILL_DIR=$SKILL_DIR" >&2
  exit 1
fi

"${LAST30DAYS_PYTHON}" "${SKILL_DIR}/scripts/last30days.py" "$TOPIC" \
  --emit=compact \
  --save-dir="${LAST30DAYS_MEMORY_DIR}" \
  --save-suffix=v3 \
  --plan "$QUERY_PLAN_FILE" \
  --x-handle=... \
  --subreddits=...
```

<ParamField body="QUERY_PLAN_FILE" type="path" required>
Tmpfile from quoted heredoc (`mktemp …/last30days-plan.XXXXXX` + `<<'PLAN_EOF'`). Engine `parse_plan` reads file paths. **Never** inline `--plan '$JSON'` — apostrophes in plan strings break shell parsing.
</ParamField>

<ParamField body="COMPETITORS_PLAN_FILE" type="path">
Same tmpfile pattern for comparison `--competitors-plan` JSON (entity → handles, subreddits, github_user, context).
</ParamField>

<ParamField body="LAST30DAYS_MEMORY_DIR" type="path">
Defaults to `~/Documents/Last30Days` for raw saves; overridable per run.
</ParamField>

### Research execution precondition gate

Before Bash, confirm:

1. Platform branch chosen (WebSearch vs `--auto-resolve`).
2. WebSearch path: 0.55 + 0.75 completed.
3. `--emit=compact` (not `md` as primary user flow).
4. Command includes `--plan` plus every resolved flag from the checklist.

Run in the **foreground** with **300000 ms** timeout. Read full stdout (evidence envelope + pass-through footer). Post-engine: 2–3 WebSearch supplements (Step 2), distinct from 0.55 budget.

### Comparison and agent modes

- **COMPARISON** (`vs` / `versus`): per-entity 0.55 resolution, single engine call with `--competitors-plan` file + main entity outer flags; synthesis uses required `##` headers per template.
- **`--agent`**: skip intro, AskUserQuestion, wait states, and invitation; emit structured research report; still runs engine + supplements.

## Version and metadata helpers

`lib/skill_meta.read_skill_version(path)` parses SKILL.md YAML `version:` (quoted or unquoted). Used by render badge logic and release consistency tests — one regex, shared with `tests/test_plugin_contract.py` and `tests/test_version_consistency.py`.

Plugin manifest `.claude-plugin/plugin.json` carries the same semantic version for marketplace installs; badge resolution tries `jq` on `"$SKILL_DIR/../../.claude-plugin/plugin.json"` then SKILL frontmatter.

## Contract vs direct CLI

| Surface | Who sets flags | SKILL.md authority |
|---------|----------------|-------------------|
| `/last30days topic` | Model after reading SKILL.md | Full step + LAW contract |
| `python3 …/last30days.py …` | Human/script | Same flags exist, but no model pre-flight unless scripted |

<Check>
A successful harness run: STEP 0 satisfied → WebSearch loaded (if Claude Code) → 0.45/0.5/0.55/0.75 as applicable → engine from `SKILL_DIR` with `--emit=compact` + plan + targeting → synthesis obeys LAWs → engine footer present.
</Check>

## Related pages

<CardGroup>
<Card title="Skill, engine, and harness" href="/skill-engine-harness">
Boundaries between the Agent Skills package, Python engine, and multi-harness runtimes.
</Card>
<Card title="Query types" href="/query-types">
QUERY_TYPE classification, intent parsing, and engine refusal paths.
</Card>
<Card title="Output contract" href="/output-contract">
Badge line, LAWs voice contract, `--emit` modes, and footer pass-through.
</Card>
<Card title="CLI reference" href="/cli-reference">
Direct `last30days.py` flags vs slash-command constraints.
</Card>
<Card title="Troubleshooting" href="/troubleshooting">
Stale marketplace clones, missing sources, and `--diagnose`.
</Card>
</CardGroup>

---

## 18. Comparison recipes

> Copy-paste comparison workflows: vs-topic phrasing, --competitors fan-out, and multi-entity Head-to-Head table expectations.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/18-comparison-recipes.md
- Generated: 2026-06-04T23:23:48.379Z

### Source Files

- `README.md`
- `skills/last30days/SKILL.md`
- `skills/last30days/scripts/lib/competitors.py`
- `tests/test_render_comparison_multi.py`
- `fixtures/eval_topics.json`

---
title: "Comparison recipes"
description: "Copy-paste comparison workflows: vs-topic phrasing, --competitors fan-out, and multi-entity Head-to-Head table expectations."
---

Comparison runs classify as `QUERY_TYPE=COMPARISON`, fan out one full `pipeline.run()` per entity in parallel (via `lib/fanout.py`), merge stdout through `render.render_comparison_multi()`, and require a hosting-model synthesis that fills the engine-emitted `## Head-to-Head` scaffold—not the general-query `What I learned:` shape.

## When to use which recipe

| User intent | Slash-command phrasing | Engine path |
| --- | --- | --- |
| Named entities already known | `/last30days OpenAI vs Anthropic` or `/last30days OpenClaw vs Hermes vs Paperclip` | vs-string auto-routes to N-pass fanout |
| Main topic + discover peers | `/last30days OpenAI --competitors` or `/last30days Cursor --competitors=3` | Model discovers peers → builds vs-string → `--competitors-plan` |
| Headless / cron with API keys | `python3 .../last30days.py "OpenAI" --competitors` | Engine `discover_competitors()` via Brave/Exa/Serper/Parallel |
| Peers known, targeting thin | `--competitors-list "Anthropic,xAI"` | Names only; peers use planner defaults |
| Full per-entity depth | `--competitors-plan` JSON or file path | Preferred; drives Step 0.55 targeting per peer |

<Note>
Slash commands do not pass shell flags. The hosting model translates user phrasing (`--competitors`, `vs` topics) into engine flags and a tmpfile-backed `--competitors-plan`. Direct CLI invocation is the fallback for scripting and verification.
</Note>

## Recipe 1: Explicit vs-topic (2-way or 3-way)

**Trigger phrases** (Step 0.5 intent parse): `X vs Y`, `X versus Y`, `compare X and Y`, `X or Y which is better`. Topics split on spaced ` vs ` or ` versus ` (also accepts `vs.`).

<Steps>
<Step title="Confirm comparison intent">
Display the branded one-liner (not a multi-line parsed-intent block):

```
/last30days - comparing {TOPIC_A} vs {TOPIC_B} across {ACTIVE_SOURCES_LIST}.
```

For three entities, extend the vs-list in the confirmation message.
</Step>

<Step title="Run Step 0.55 per entity">
For each entity, resolve X handle, subreddits (include category-peer subs for products), GitHub user/repos, and news context. Batch WebSearch across entities (e.g. three entities × four lookup types → 3–4 batched queries, not twelve serial searches).
</Step>

<Step title="Invoke the engine once">
Write per-peer targeting to a tmpfile and pass the path (avoids apostrophe breakage in inline JSON):

```bash
COMPETITORS_PLAN_FILE=$(mktemp "${TMPDIR:-/tmp}/last30days-competitors.XXXXXX")
trap 'rm -f "$COMPETITORS_PLAN_FILE"' EXIT
cat > "$COMPETITORS_PLAN_FILE" <<'PLAN_EOF'
{
  "Anthropic": {"x_handle":"AnthropicAI","subreddits":["ClaudeAI","MachineLearning"],"github_user":"anthropics","context":"Claude 4.x ship cycle"},
  "xAI": {"x_handle":"xai","subreddits":["LocalLLaMA"],"github_user":"","context":"Grok release chatter"}
}
PLAN_EOF

python3 skills/last30days/scripts/last30days.py "OpenAI vs Anthropic vs xAI" \
  --emit=compact \
  --save-dir="${LAST30DAYS_MEMORY_DIR}" \
  --x-handle=OpenAI \
  --subreddits=OpenAI,MachineLearning \
  --competitors-plan "$COMPETITORS_PLAN_FILE"
```

Topic A (first in the vs-string) uses outer `--x-handle`, `--subreddits`, `--github-user`, `--github-repo`, `--tiktok-*`, `--ig-creators`. Peers read overrides from `--competitors-plan` keys (case-insensitive).
</Step>

<Step title="WebSearch supplements (skill path)">
After the engine returns, supplement with rivalry queries the per-entity passes may miss:

- `{TOPIC_A} vs {TOPIC_B} comparison {YEAR}`
- `{TOPIC_A} vs {TOPIC_B} which is better`
</Step>

<Step title="Synthesize with the comparison template">
Skip general-query synthesis. Use the comparison sections below and pass through the engine footer (LAW 5).
</Step>
</Steps>

**Engine routing:** If the positional topic contains ` vs ` / ` versus ` and splits into ≥2 entities (`planner._comparison_entities`), `last30days.py` rewrites the main topic to the first entity, sets competitor list to the remainder, and runs the same fanout path as `--competitors` (stderr: `[Competitors] vs-mode: routing to N-pass fanout`).

**Eval fixture example:** `OpenClaw vs NanoClaw vs ZeroClaw` is tagged `comparison` in `fixtures/eval_topics.json` for multi-entity extraction tests.

## Recipe 2: `--competitors` (discover peers)

Use when the user names one product or company and wants peers auto-selected.

<Steps>
<Step title="Discover peers (hosting model)">
WebSearch:

- `{topic} competitors`
- `{topic} alternatives`

Default peer count: **2** (3-way total: main + 2 peers). User may pass `--competitors=N` where N is 1..6 (out-of-range values clamp with stderr warning).
</Step>

<Step title="Step 0.55 for main and every peer">
Same resolution stack as Recipe 1—for each discovered peer, not only the main topic.
</Step>

<Step title="Build vs-string and invoke">
Construct `"{main} vs {peer1} vs {peer2}"` and call the engine with `--competitors-plan` covering peers (and optionally the main entity if overriding outer flags).
</Step>
</Steps>

**Headless discovery:** With `BRAVE_API_KEY`, `EXA_API_KEY`, `SERPER_API_KEY`, or `PARALLEL_API_KEY`, `lib/competitors.discover_competitors()` runs three parallel SERP queries (`{topic} competitors`, `{topic} alternatives`, `{topic} vs`), scores capitalized brand phrases, filters topic overlap and listicle stopwords, and returns up to N names. Without a web backend and without `--competitors-list`, the engine exits with stderr instructions to use the hosting-model path.

<Warning>
`--competitors-list "A,B,C"` is the minimum escape hatch. Without `--competitors-plan`, peer sub-runs use thin deterministic planner queries. The `## Resolved Entities` block shows dashes for skipped Step 0.55—treat that as a failed recipe, not a finished comparison.
</Warning>

## Recipe 3: Direct CLI quick checks

For engine testing without the skill loop:

<CodeGroup>
```bash title="2-way vs-string"
python3 skills/last30days/scripts/last30days.py "OpenAI vs Anthropic" --emit=compact --mock
```

```bash title="Explicit peer list"
python3 skills/last30days/scripts/last30days.py "OpenAI" \
  --competitors-list "Anthropic,xAI" \
  --emit=compact --mock
```

```bash title="JSON plan file"
python3 skills/last30days/scripts/last30days.py "OpenAI vs Anthropic" \
  --competitors-plan /tmp/plan.json \
  --emit=compact --mock
```
</CodeGroup>

`--emit=compact` and `--emit=md` both route comparison output through `render_comparison_multi`. `--emit=json` nests `comparison: true`, `entities: [...]`, and per-entity `reports`.

## `--competitors-plan` schema

Top-level object: entity name → targeting dict. Parsed by `parse_competitors_plan()`; accepts inline JSON or a file path.

| Field | Type | Role |
| --- | --- | --- |
| `x_handle` | string | Primary X handle for the entity sub-run |
| `x_related` | string | Related handles (comma-separated) |
| `subreddits` | array of strings | Subreddit names without `r/` prefix |
| `github_user` | string | GitHub person-mode username |
| `github_repos` | array of strings | `owner/repo` project targets |
| `context` | string | News/context snippet for planner subqueries |

Unknown fields log a warning and are ignored. Missing entries fall back to engine `auto_resolve()` when a web backend exists.

<ParamField body="--competitors" type="integer (optional)" default="2 when flag present without value">
Signals competitor mode. Bare `--competitors` → discover 2 peers. `--competitors=3` → main + 3 peers (4-way). Range 1..6.
</ParamField>

<ParamField body="--competitors-list" type="string">
Comma-separated peer names; implies `--competitors`. Count overrides `--competitors=N` when both are set.
</ParamField>

<ParamField body="--competitors-plan" type="JSON or file path">
Per-entity Step 0.55 targeting; implies `--competitors`. Preferred over `--competitors-list`.
</ParamField>

## Engine stdout shape (before synthesis)

Merged compact output includes:

```text
🌐 last30days v{VERSION} · synced {YYYY-MM-DD}
# last30days v{VERSION}: {Entity1} vs {Entity2} vs ...
- Comparison mode: N entities (Entity1, Entity2, ...)
<!-- EVIDENCE FOR SYNTHESIS -->   ← read, do not emit verbatim
## Resolved Entities              ← when any sub-run has artifacts.resolved
- **Entity**: X @handle | Subs r/a, r/b | GitHub @user (repos) | Context: ...
## {Entity}                       ← per-entity evidence (clusters or placeholder)
<!-- END EVIDENCE FOR SYNTHESIS -->
## Head-to-Head                   ← empty table scaffold for the model to fill
| Dimension | Entity1 | Entity2 | ...
<!-- PASS-THROUGH FOOTER -->      ← emit verbatim (LAW 5)
```

**Thin peer signal:** An entity section containing `(no significant discussion this month)` means that sub-run returned no clusters—still synthesize, but do not invent volume.

**Per-entity artifacts:** `--save-dir` receives `{slug}-raw.md` per entity (main + each peer).

## Head-to-Head table expectations

The engine emits the scaffold; the hosting model **fills cells** in the final user-facing response (5–15 words per cell; use ` - ` not em-dashes; `N/A` when an axis does not apply).

**Default axes** (from `_render_comparison_scaffold` in `lib/render.py`):

| Dimension |
| --- |
| What it is |
| GitHub stars |
| Philosophy |
| Skills |
| Memory |
| Models |
| Security |
| Best for |
| Install |

Column headers match entity labels in order (`| Dimension | OpenAI | Anthropic | xAI |` for three-way runs). For non-tool topics (geopolitics, people, sports), substitute topic-appropriate row meaning or `N/A` rather than inventing GitHub-star cells.

<Check>
Verification checklist after a comparison run:
- Stderr shows `[Competitors] Comparing: …` or vs-mode fanout line
- Compact stdout includes `## Head-to-Head` with one column per entity
- `## Resolved Entities` has handles/subs for every peer (not all dashes)
- One `*-raw.md` per entity under `--save-dir`
- Final synthesis uses comparison headers only—not `What I learned:`
</Check>

## Required synthesis shape (hosting model)

Comparison output **replaces** the general template. Required `##` headers (LAW 4 exception):

1. `## Quick Verdict` — one paragraph; competitors vs layers; inline scale stats; one quotable community line
2. `## {Entity}` — per entity: **Community Sentiment**, **Strengths**, **Weaknesses** with `per <source>` attribution
3. `## Head-to-Head` — filled table (pass through scaffold structure; fill cells)
4. `## The Bottom Line` — `**Choose {Entity} if**` per entity
5. `## The emerging stack` — one paragraph on combination patterns, or explicit “No emerging stack pattern has crystallized…”

**Title line** (after badge): `# {TOPIC_A} vs {TOPIC_B} [vs {TOPIC_C}]: What the Community Says (/Last30Days)`

**Do not use:** `What I learned:`, bold-lead-in general paragraphs, `KEY PATTERNS from the research:`, fabricated `## Notable Stats`, or a trailing `Sources:` block.

**Invitation** (comparison-specific):

```
I've compared {TOPIC_A} vs {TOPIC_B} using the latest community data. Some things you could ask:
- Deep dive into {Entity} alone with /last30days {Entity}
- …
```

Reference exemplar path (skill contract): `$LAST30DAYS_MEMORY_DIR/openclaw-vs-hermes-vs-paperclip-LAUNCH-VIDEO-april9-exemplar.md`.

## Comparison vs RECOMMENDATIONS

| Signal | COMPARISON | RECOMMENDATIONS |
| --- | --- | --- |
| User phrasing | `X vs Y`, `compare`, `which is better` | `best X`, `top X`, `what X should I use` |
| Ranking logic | Side-by-side entities + Head-to-Head axes | Signal-weighted picks, not mention counts |
| Body structure | Per-entity sections + table | `🏆 Top recommendations` list |
| Engine fanout | N parallel pipelines | Single pipeline |

## Failure modes and fixes

| Symptom | Likely cause | Fix |
| --- | --- | --- |
| No `--competitors` in `--help` | Stale Claude marketplace clone (STEP 0) | Re-read cache SKILL.md per STEP 0 guard |
| `[Competitors] Cannot auto-discover peers` | No web API keys on headless CLI | Use slash-command WebSearch + `--competitors-plan`, or set Brave/Exa/Serper |
| Resolved Entities all `-` for a peer | Skipped Step 0.55 for that entity | Re-run with plan entry for that peer |
| Raw `### 1. (score N, M items…)` in chat | LAW 6 violation—emitted evidence verbatim | Regenerate synthesis from evidence block only |
| `What I learned:` on a vs query | Wrong template | Switch to comparison synthesis sections |
| Fewer than 2 sub-runs survived | Fanout dropped failed entities | Check stderr warnings; widen sources or fix targeting |

## Related pages

<CardGroup>
<Card title="Comparison mode" href="/comparison-mode">
Engine flags, discovery fan-out, `--competitors-plan` wiring, and comparison synthesis contract in SKILL.md.
</Card>
<Card title="Query types" href="/query-types">
`QUERY_TYPE=COMPARISON` classification, intent parsing, and pre-flight paths.
</Card>
<Card title="Output contract" href="/output-contract">
Badge line, LAWs voice rules, `--emit` modes, and footer pass-through.
</Card>
<Card title="CLI reference" href="/cli-reference">
Full `last30days.py` flags including `--competitors`, `--competitors-list`, and `--competitors-plan`.
</Card>
<Card title="Per-client setup" href="/per-client-setup">
Recurring `--competitors-plan` templates and project-scoped env isolation.
</Card>
</CardGroup>

---

## 19. Prompting recipes

> PROMPTING query workflows: community technique extraction, KEY PATTERNS output shape, and copy-paste prompt generation patterns.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/19-prompting-recipes.md
- Generated: 2026-06-04T23:24:25.637Z

### Source Files

- `README.md`
- `skills/last30days/SKILL.md`
- `skills/last30days/scripts/lib/planner.py`
- `tests/test_pipeline_v3.py`
- `skills/last30days/scripts/lib/render.py`

---
title: "Prompting recipes"
description: "PROMPTING query workflows: community technique extraction, KEY PATTERNS output shape, and copy-paste prompt generation patterns."
---

`/last30days` treats **PROMPTING** as a skill-level `QUERY_TYPE` in `SKILL.md`: the hosting model classifies the user topic, runs the v3 engine with resolved communities and a JSON `--plan`, synthesizes community techniques into the canonical `What I learned:` + `KEY PATTERNS from the research:` shape, then—only after the user describes what to create—writes one paste-ready prompt that matches the format the research surfaced. The Python engine does not expose a `PROMPTING` enum; it plans with intents such as `how_to`, `product`, and `concept` via `scripts/lib/planner.py`.

## PROMPTING vs engine intent

| Layer | Identifier | Where it lives | Role for prompting runs |
|-------|------------|----------------|-------------------------|
| Skill | `QUERY_TYPE = PROMPTING` | `skills/last30days/SKILL.md` | Voice contract, WebSearch supplements, invitation, follow-up prompt rules |
| Engine | `intent` on `QueryPlan` | `scripts/lib/planner.py`, `--plan` JSON | Source priority, `freshness_mode`, `cluster_mode` (often `how_to` → `workflow`, `evergreen_ok`) |
| Search modules | Local `_infer_query_intent` | e.g. `scripts/lib/reddit.py` | Reddit query expansion only (`how_to`, `product`, …)—not the skill `QUERY_TYPE` |

<Note>
Eval fixtures label topics like `nano banana pro prompting` as engine `product` or `how_to`, while the skill still classifies the same phrasing as **PROMPTING** for synthesis and the post-research invitation. Plan for both: skill workflow + engine `how_to`/`product` subqueries.
</Note>

## Classify a PROMPTING query

The model parses **TOPIC**, optional **TARGET_TOOL**, and **QUERY_TYPE** before any engine call.

| User phrasing | `QUERY_TYPE` | `TARGET_TOOL` |
|---------------|--------------|---------------|
| `X prompts`, `prompting for X`, `X best practices` | PROMPTING | From `for [tool]` if present; else `unknown` until after research |
| `UI design prompts for Midjourney` | PROMPTING | `Midjourney` (specified in query) |
| `iOS design mockups` (no tool) | PROMPTING or GENERAL | `unknown`—do **not** ask for tool before research |

Confirmation line (same shape as GENERAL/NEWS/RECOMMENDATIONS):

```
/last30days - searching {ACTIVE_SOURCES_LIST} for what people are saying about {TOPIC}.
```

Do not emit a multi-line `Parsed intent:` block with `TOPIC=` / `QUERY_TYPE=` variables.

## End-to-end workflow

```mermaid
sequenceDiagram
  participant User
  participant Model as Hosting model (SKILL.md)
  participant Engine as last30days.py
  participant Sources as Reddit X YouTube TikTok …

  User->>Model: /last30days {topic} [for {tool}]
  Model->>Model: QUERY_TYPE=PROMPTING, Step 0.45 pre-flight
  Model->>Model: Step 0.55 resolve + category peers
  Model->>Model: Step 0.75 JSON --plan
  Model->>Engine: --emit=compact --plan --x-handle --subreddits …
  Engine->>Sources: parallel fan-out
  Sources-->>Engine: ranked clusters + footer
  Engine-->>Model: evidence envelope + PASS-THROUGH FOOTER
  Model->>Model: WebSearch supplements (prompting queries)
  Model->>User: badge, What I learned, KEY PATTERNS, footer, invitation
  User->>Model: describe what to create
  Model->>User: one prompt in research-recommended format
```

<Steps>
<Step title="Pre-flight and reframe (Step 0.45)">
Tutorial-shaped topics (`how to use X`, `tutorial for Z`) are **Class 3 keyword traps**. Reframe to discussion vocabulary (`Docker tips tricks workflows`) before the engine runs. Wait for user confirmation if you asked a clarifying question.
</Step>
<Step title="Resolve communities (Step 0.55)">
Run 2–3 WebSearches for subreddits and news context. For product topics, **mandatory category-peer expansion**: brand subs alone miss cross-product technique threads (documented `GPT Image 2` failure). Merge WebSearch subs with peers from `scripts/lib/categories.py`; cap at 10; annotate `(+ ai_image_generation peers)` on the Resolved line when peers were added.
</Step>
<Step title="Plan and execute (Steps 0.75 + Research Execution)">
Generate 1–4 subqueries as JSON; primary subquery must include `reddit`, `x`, `youtube`, `tiktok`, `instagram`, `hackernews`, `polymarket`. Pass `--plan` via a tmpfile (never inline JSON with apostrophes). Use `--emit=compact`, resolved `--x-handle`, `--subreddits`, hashtags, creators. Named-entity topics require `--plan` (LAW 7).
</Step>
<Step title="Supplement and synthesize (Steps 2 + Judge)">
Run 2–3 WebSearches: `{TOPIC} prompts examples 2026`, `{TOPIC} techniques tips`. Append bullets to the saved raw file under `## WebSearch Supplemental Results`. Transform engine clusters into prose—do not dump `## Ranked Evidence Clusters` (LAW 6).
</Step>
<Step title="Invite, then write one prompt">
After synthesis, use the PROMPTING invitation. When the user describes something to **create** or asks for a **prompt**, deliver **one** paste-ready prompt in the format the research recommends (JSON, structured params, etc.).
</Step>
</Steps>

## Community technique extraction

### Query stripping (`scripts/lib/query.py`)

Platform search uses `extract_core_subject()` to remove prompting meta-words (`prompt`, `prompts`, `prompting`, `techniques`, `tips`, …) and prefixes like `how to use`, `tips for`, `best practices for`. Multi-word suffixes `prompting techniques` / `prompting tips` strip when `strip_suffixes=True` (used on some modules). Compound terms (`Claude Code`, `multi-agent`) stay quoted in expansion.

### Category-peer subreddits (`scripts/lib/categories.py`)

`detect_category(topic)` returns IDs such as `ai_image_generation`, `ai_coding_agent`, `ai_video_generation`. `peer_subs_for()` returns priority-ordered subs—for image gen: `StableDiffusion`, `midjourney`, `dalle2`, `aiArt`, `PromptEngineering`, `MediaSynthesis`. Tests in `tests/test_categories.py` lock the **Prompting GPT Image 2** regression: OpenAI-brand subs only → thin technique signal; peers required.

| Category ID | Example trigger terms | Peer subs (first entries) |
|-------------|----------------------|---------------------------|
| `ai_image_generation` | GPT Image, Nano Banana, Midjourney, Stable Diffusion | `StableDiffusion`, `midjourney`, `dalle2` |
| `ai_coding_agent` | Claude Code, Cursor, OpenClaw, Hermes Agent | `ChatGPTCoding`, `LocalLLaMA`, `singularity` |
| `ai_video_generation` | Sora, Veo, Runway Gen, Kling | `aivideo`, `StableDiffusion`, `runwayml` |

### Planner hints for technique-heavy topics

When building `--plan` JSON for prompting runs, align with Step 0.75 rules:

- **Product-style topics**: route subqueries to YouTube (reviews/demos), Reddit (discussions), TikTok (demos).
- **How-to-shaped retrieval**: `intent: "how_to"`, `cluster_mode: "workflow"`, `freshness_mode: "evergreen_ok"`.
- **Search queries**: keyword-heavy, match how posts are **titled**; no `last 30 days`, `recent`, or year literals in `search_query`.
- Strip intent modifiers (`tutorial`, `workflow`, `examples`) from `search_query`; keep meaning in `ranking_query`; prefer 4–5 paraphrased subqueries for broad retrieval.

### WebSearch supplements (Step 2)

| Query type | Supplement searches | Goal |
|------------|---------------------|------|
| PROMPTING | `{TOPIC} prompts examples 2026`, `{TOPIC} techniques tips` | Examples and structures for copy-paste prompts |
| All types | Exclude `reddit.com`, `x.com`, `twitter.com` (engine already covers) | Blogs, tutorials, docs, GitHub |

Use the user's **exact terminology** in supplements—do not substitute model-known aliases.

### Evidence → synthesis fields

From **Judge Agent** instructions (all query types; critical for PROMPTING):

- **PROMPT FORMAT** — JSON vs natural language vs tag lists, as stated in sources
- **Top 3–5 patterns** across platforms
- **Keywords/structures** and **pitfalls** cited by sources—not invented

## KEY PATTERNS output shape

PROMPTING uses the same voice contract as GENERAL, NEWS, and RECOMMENDATIONS (not the COMPARISON template).

### Required sequence

1. **Line 1**: Badge pass-through — `🌐 last30days v{VERSION} · synced {YYYY-MM-DD}` (engine emits on `--emit=compact`).
2. **Line 3**: Prose label `What I learned:` (no invented title, no `#` heading).
3. **Body**: 3+ paragraphs, each starting `**Headline phrase** - ` then 1–2 sentences with inline `[name](url)` citations (LAW 8).
4. **Prose label**: `KEY PATTERNS from the research:` then a **numbered** list (typically 3–5 items).
5. **Engine footer**: `<!-- PASS-THROUGH FOOTER -->` block verbatim (LAW 5).
6. **Invitation**: PROMPTING-specific block (below).
7. **End**: No trailing `Sources:` (LAW 1).

### KEY PATTERNS list rules

| Rule | Requirement |
|------|-------------|
| Structure | Numbered list only; no `## Key patterns` header (LAW 4) |
| Bold | Skill template uses `**bold**` for pattern lead-ins; do not strip for global "no bold" prefs |
| Citations | One source per pattern: `per [@handle](url)` or `per [r/sub](url)` |
| Separators | Use ` - ` (spaced hyphen), never em-dash or en-dash (LAW 3) |
| Density | Do not chain `per @a, @b, @c`; pick the strongest source |

<RequestExample>
```text
🌐 last30days v3.3.1 · synced 2026-06-04

What I learned:

**JSON blocks are replacing tag soup** - Creators on X describe nested JSON fields to stop concept bleed between subjects, per [@pictsbyai](https://x.com/pictsbyai).

**Edit-first beats full regeneration** - r/StableDiffusion threads treat img2img and mask edits as the default loop for iteration, per [r/StableDiffusion](https://reddit.com/r/StableDiffusion).

KEY PATTERNS from the research:
1. **Nested JSON with explicit device/scene keys** - per [@pictsbyai](https://x.com/pictsbyai)
2. **Separate negative prompt channel** - per [r/midjourney](https://reddit.com/r/midjourney)
3. **Reference image + strength slider for style lock** - per [r/dalle2](https://reddit.com/r/dalle2)

--- 
✅ All agents reported back!
…
---

---
I'm now an expert on Nano Banana Pro prompting for Nano Banana Pro. What do you want to make? For example:
- …

Just describe your vision and I'll write a prompt you can paste straight into Nano Banana Pro.
```
</RequestExample>

HTML briefs (`--emit=html`) map labels via `scripts/lib/html_render.py`: `KEY PATTERNS from the research:` → display title "Key patterns from the research". Synthesis temp files include `What I learned:` + KEY PATTERNS only—not the badge or footer (`references/save-html-brief.md`).

## Copy-paste prompt generation

Research completes in one turn; **prompt delivery is a second turn** after the invitation.

### PROMPTING invitation (after synthesis)

```
---
I'm now an expert on {TOPIC} for {TARGET_TOOL}. What do you want to make? For example:
- [specific idea from popular technique in research]
- [specific idea from trending style in research]
- [specific idea from what people are actually creating]

Just describe your vision and I'll write a prompt you can paste straight into {TARGET_TOOL}.
```

If `TARGET_TOOL` was `unknown`, infer or ask **after** showing results—not before the engine run.

### When the user responds

| User intent | Action |
|-------------|--------|
| Question about the topic | Answer from research only—no new WebSearch, no prompt |
| Go deeper on a subtopic | Elaborate from existing findings |
| Describe something to **create** or asks for a **prompt** | Write **one** tailored prompt |
| Different topic | New `/last30days` run |

### Prompt quality contract

<Warning>
**Anti-pattern:** Research recommends JSON or structured params but the model outputs plain prose. That voids the research pass.
</Warning>

<Check>
**Pre-delivery checklist**
- Format matches research (JSON / structured / natural language)
- Addresses the user's stated creation goal
- Uses patterns and keywords from KEY PATTERNS and narrative
- Paste-ready with clear `[PLACEHOLDERS]` only where needed
- Length and style fit `TARGET_TOOL`
</Check>

**Delivery shape:**

```
Here's your prompt for {TARGET_TOOL}:

---

[Prompt in the format the research recommends]

---

This uses [one-line research insight applied].
```

Provide 2–3 variations **only** if the user asks for more options. After each prompt, optional expert footer with source counts; then: `Want another prompt? Just tell me what you're creating next.`

**Context memory** for the session: retain `TOPIC`, `TARGET_TOOL`, KEY PATTERNS list, and findings; answer follow-ups without re-searching unless the topic changes.

## Recipe catalog

### Slash command (primary)

| Goal | Example invocation | Notes |
|------|-------------------|-------|
| Tool-specific prompting | `/last30days Nano Banana Pro prompting` | README exemplar: JSON prompts, edit-first workflow |
| Image model techniques | `/last30days Prompting GPT Image 2` | Requires `ai_image_generation` peer subs in Resolved block |
| Coding agent prompts | `/last30days Claude Code prompting` | Category `ai_coding_agent`; plan with `how_to`/`product` subqueries |
| Video model | `/last30days Seedance prompting` | `ai_video_generation` peers + prompting supplements |

### Direct CLI (dev/scripting only)

```bash
python3 skills/last30days/scripts/last30days.py "Nano Banana Pro prompting" \
  --emit=compact \
  --plan /tmp/query-plan.json \
  --subreddits=StableDiffusion,midjourney,dalle2,aiArt \
  --save-dir="${LAST30DAYS_MEMORY_DIR}"
```

The hosting model must still perform Step 0.55 resolution and Step 0.75 planning when using the skill; bare CLI omits synthesis, KEY PATTERNS, invitation, and prompt generation unless a separate agent applies `SKILL.md`.

### Depth and timing

| Flag | Effect on prompting runs |
|------|--------------------------|
| `--quick` | Fewer items per source (faster, thinner technique coverage) |
| (default) | Balanced counts |
| `--deep` | More Reddit/X items—use when peer subs are broad |
| `--days=N` | Lookback window (default 30) |

## Failure modes and fixes

| Symptom | Likely cause | Fix |
|---------|--------------|-----|
| Brand-only subreddits, weak patterns | Skipped category-peer expansion | Re-run Step 0.55; add `(+ {category_id} peers)` subs from `categories.py` |
| Literal `how to use X` retrieval noise | Class 3 keyword trap | Reframe to tips/workflows phrasing in Resolved block |
| Raw `## Ranked Evidence Clusters` in chat | LAW 6 violation | Regenerate synthesis from evidence envelope only |
| Plain `@handle` without links | LAW 8 violation | Regenerate with `[name](url)` from raw dump |
| Prose prompt after JSON-heavy research | Format anti-pattern | Re-read PROMPT FORMAT from Judge section; output structured prompt |
| `Sources:` after invitation | LAW 1 / WebSearch override | Delete trailing list; keep engine `🌐 Web:` footer only |

## Related pages

<CardGroup>
<Card title="Query types" href="/query-types">
GENERAL, NEWS, PROMPTING, RECOMMENDATIONS, COMPARISON classification and pre-flight refusal paths.
</Card>
<Card title="Output contract" href="/output-contract">
Badge, LAWs voice contract, `--emit` modes, evidence blocks, footer pass-through.
</Card>
<Card title="Research pipeline" href="/research-pipeline">
Planner sub-queries, fan-out, fusion, clustering, dedupe, rerank, depth profiles.
</Card>
<Card title="Skill contract" href="/skill-contract">
SKILL.md authority: Step 0, pre-flight, engine invocation, `SKILL_DIR` substitution.
</Card>
<Card title="Configure sources" href="/configure-sources">
Credentials and `--diagnose` for Reddit, X, YouTube, ScrapeCreators, and web backends.
</Card>
<Card title="Per-client setup" href="/per-client-setup">
Project-scoped `.claude/last30days.env` and save-dir isolation for recurring prompt research.
</Card>
</CardGroup>

---

## 20. Troubleshooting

> --diagnose probes, missing-source recovery, stale Claude marketplace clones, thin results, and documented failure modes.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/20-troubleshooting.md
- Generated: 2026-06-04T23:25:32.691Z

### Source Files

- `CONFIGURATION.md`
- `skills/last30days/scripts/last30days.py`
- `skills/last30days/SKILL.md`
- `skills/last30days/scripts/lib/setup_wizard.py`
- `docs/solutions/workflow-issues/release-consistency-test-cascade-2026-05-16.md`
- `docs/solutions/architecture/search-quality-eval-manual-by-default-2026-05-10.md`

---
title: "Troubleshooting"
description: "--diagnose probes, missing-source recovery, stale Claude marketplace clones, thin results, and documented failure modes."
---

The engine exposes a non-destructive availability probe via `--diagnose`: `last30days.py` loads config from `.claude/last30days.env` or `~/.config/last30days/.env`, calls `pipeline.diagnose()`, prints JSON, and exits before any retrieval. Full runs add post-research quality nudges, pipeline thin-source retries, and structured warnings in the report footer.

```mermaid
flowchart TD
  subgraph probe [Probe path]
    CLI["last30days.py --diagnose"]
    ENV["env.get_config()"]
    DIAG["pipeline.diagnose()"]
    JSON["stdout: JSON + exit 0"]
    CLI --> ENV --> DIAG --> JSON
  end
  subgraph run [Research path]
    RUN["last30days.py TOPIC"]
    PF["preflight.check_class_1_trap()"]
    PIPE["pipeline.run()"]
    RETRY["_retry_thin_sources()"]
    NUDGE["quality_nudge → stderr"]
    RUN --> PF
    PF -->|pass| PIPE
    PIPE --> RETRY
    PIPE --> NUDGE
  end
```

## Run `--diagnose` first

Use the direct CLI form from the skill directory (not a slash command with pipes):

<CodeGroup>
```bash title="From repo checkout"
cd skills/last30days
python3 scripts/last30days.py --diagnose
```

```bash title="Scoped sources"
python3 scripts/last30days.py --diagnose --search reddit,x,youtube
```
</CodeGroup>

<Check>
Exit code `0` and JSON on stdout means config loaded and probes completed. No topic argument is required.
</Check>

Hermes and other hosts that pin Python 3.12+ should use the same entrypoint documented in `HERMES_SETUP.md`: `python3.12 scripts/last30days.py --diagnose`.

Maintainers can fold diagnose into release verification via `scripts/verify_v3.py`, which parses `--diagnose` stdout as a smoke gate.

## Diagnose JSON fields

`pipeline.diagnose(config, requested_sources)` returns a single object. Use it to separate “key missing” from “source errored mid-run.”

| Field | Meaning |
| --- | --- |
| `providers` | `google`, `openai`, `xai`, `openrouter` booleans (OpenAI requires `OPENAI_AUTH_STATUS == ok`) |
| `local_mode` | `true` when no cloud reasoning keys are configured |
| `reasoning_provider` | `LAST30DAYS_REASONING_PROVIDER` or `"auto"` |
| `available_sources` | Sources `pipeline.run()` would fan out to (after `EXCLUDE_SOURCES`, `INCLUDE_SOURCES`, CLI `--search`) |
| `x_backend` | Resolved X path: Bird cookies, xAI, ScrapeCreators, etc. |
| `bird_installed` / `bird_authenticated` / `bird_username` | Bundled Bird CLI state |
| `native_web_backend` | First configured web key: `brave`, `exa`, `serper`, or `parallel` |
| `has_scrapecreators` | `SCRAPECREATORS_API_KEY` present |
| `has_github` | `GITHUB_TOKEN` or `gh` on `PATH` |

<Warning>
`available_sources` empty for a normal topic (no `--search` narrowing) leads to `RuntimeError: No sources are available for this run.` Fix credentials before researching.
</Warning>

## Missing-source recovery

Default availability is computed in `pipeline.available_sources()` from env keys and CLI presence. The table maps common gaps to fixes.

| Symptom in diagnose / footer | Typical cause | Recovery |
| --- | --- | --- |
| `x` not in `available_sources` | No X auth | Log into x.com in a supported browser and re-run, or set `AUTH_TOKEN`+`CT0`, `XAI_API_KEY`, `SCRAPECREATORS_API_KEY`, or `FROM_BROWSER=firefox` |
| `youtube` missing | `yt-dlp` not on `PATH` | `brew install yt-dlp` (setup wizard can auto-install via Homebrew) |
| `grounding` missing | No web API key | Add `BRAVE_API_KEY`, `EXA_API_KEY`, `SERPER_API_KEY`, or `PARALLEL_API_KEY`; on Claude Code/Codex/Gemini the host WebSearch may suffice without keys |
| `tiktok` / `instagram` missing | No SC key or not in `INCLUDE_SOURCES` | Set `SCRAPECREATORS_API_KEY` and `INCLUDE_SOURCES=tiktok,instagram` |
| `bluesky` missing | Credentials absent | `BSKY_HANDLE` + `BSKY_APP_PASSWORD` (19-char app password); default search host is `api.bsky.app` — override with `BSKY_SEARCH_HOST` if infrastructure moves |
| Reddit “public only” | No `SCRAPECREATORS_API_KEY` | Reddit still works keyless; SC unlocks full threads with comments |
| `.env` ignored | Wrong path | Project `.claude/last30days.env` wins over `~/.config/last30days/.env`; set `LAST30DAYS_CONFIG_DIR` to relocate |

<Steps>
<Step title="Run first-time setup">
```bash
python3 scripts/last30days.py setup
```
Extracts browser cookies for configured domains, checks or installs `yt-dlp`, writes `SETUP_COMPLETE=true` and `FROM_BROWSER` to the active config file.
</Step>
<Step title="Re-probe">
```bash
python3 scripts/last30days.py --diagnose
```
Confirm `available_sources` includes the platforms you expect.
</Step>
<Step title="Run a smoke topic">
```bash
python3 scripts/last30days.py "your topic" --emit=compact --quick
```
</Step>
</Steps>

Optional setup subcommands: `setup --openclaw`, `setup --github`, `setup --device-auth` (JSON on stdout).

<Tip>
On POSIX hosts, secrets files should be mode `600`. The engine warns every run if group/other can read `.env` — run `chmod 600` on the path shown in the warning.
</Tip>

After a full run, missing or degraded core sources also surface through `quality_nudge.compute_quality_score()` on stderr (advisory, not a hard failure).

## Stale Claude marketplace clone

Claude Code can load `SKILL.md` from `~/.claude/plugins/marketplaces/last30days-skill/`, a git clone that auto-restores to `origin/main` and may lag the versioned plugin cache by one or more releases. Documented impact: `--help` and engine flags from the stale tree (e.g. missing `--competitors`) while the cache already had them.

**Model-side guard (STEP 0 in SKILL.md):** If the loaded path contains `/.claude/plugins/marketplaces/` and a newer cache exists under `~/.claude/plugins/cache/last30days-skill/last30days/{version}/`, re-read `SKILL.md` from the cache before any tool calls.

**Operator-side fixes:**

| Action | Command / note |
| --- | --- |
| Force plugin refresh | `claude plugin update last30days@last30days-skill` |
| Avoid duplicate installs | Use marketplace **or** `npx skills add … -a claude-code`, not both — duplicate `/last30days` entries mean two skill copies |
| Prefer versioned cache | Let marketplace update populate `~/.claude/plugins/cache/…` before relying on marketplaces path |

<Info>
`~/.codex/skills/`, `~/.agents/skills/`, `npx skills` global installs, and repo checkouts are valid; STEP 0 does not redirect those paths. Engine invocation uses `SKILL_DIR` from the `SKILL.md` the harness actually loaded.
</Info>

## Stale Agent Skills / dev-tree install

For Codex, Cursor, Gemini, and other Agent Skills hosts, `npx skills add` copies into `~/.agents/skills/<name>/` **frozen at install time**. Working-tree edits do not propagate until you re-sync:

```bash
npx skills add mvanhorn/last30days-skill -g -y
# or from a local repo:
npx skills add . -g -y
```

Live development symlink (repo root):

```bash
ln -sfn "$PWD/skills/last30days" ~/.agents/skills/last30days
```

Update published installs: `npx skills update last30days -g`.

## Thin results and low evidence

Thin output can come from query traps, sparse platforms, missing web backend, or per-source retrieval limits — not only “broken install.”

### Engine behaviors

| Mechanism | When it runs | Effect |
| --- | --- | --- |
| `_retry_thin_sources()` | Default/deep depth only; skipped in `--quick` | Re-queries sources with fewer than 3 items using `query.extract_core_subject(topic, max_words=3)`; skips sources in `errors_by_source` and rate-limited sources |
| `_warnings()` | End of `pipeline.run()` | Adds `"Evidence is thin for this topic."` when fewer than 5 ranked candidates; lists `errors_by_source` |
| Cluster `uncertainty` | `cluster._cluster_uncertainty()` | `thin-evidence` when max cluster score &lt; 55; `single-source` when one platform dominates |
| `--quick` | User flag | Disables thin-source retry — faster but sparser |

### Operator levers

- Add or fix web search keys (`BRAVE_API_KEY`, etc.) — CONFIGURATION.md notes client setups often look thinner than a host with native WebSearch.
- Use `--deep` for higher-recall depth settings (more pool/rerank budget).
- Narrow with `--subreddits`, `--x-handle`, `--github-repo` when the topic is broad.
- Avoid demographic-shopping phrasing without hobbies/relationship/budget (see preflight below).

### Post-run quality nudge (stderr)

After retrieval, `quality_nudge` scores five core sources (`hn`, `polymarket`, `x`, `youtube`, `reddit`) and may emit actionable text:

| Signal | Detection | Typical fix |
| --- | --- | --- |
| Missing X | No creds or `x_error` | Browser login or `XAI_API_KEY` |
| Missing YouTube | No `yt-dlp` | `brew install yt-dlp` |
| Degraded YouTube | Videos found, transcript ratio below `DEGRADED_TRANSCRIPT_THRESHOLD` (default 0.5) | `brew upgrade yt-dlp` / `pip install -U yt-dlp` — stale binary is the canonical failure mode |
| Silent Instagram | SC configured, zero items, not excluded | Topic may lack reel coverage; SC v2 endpoint flakiness; skill retries hashtag form automatically |

Captions-disabled YouTube videos are excluded from the degraded ratio so one uploader-disabled track does not false-trigger a stale-`yt-dlp` nudge.

## Preflight refuse gates (exit code 2)

Before the pipeline starts, `preflight.check_class_1_trap(topic)` blocks Class 1 demographic-shopping patterns unless qualifiers (budget, hobbies, relationship, activity noun) are present. Stderr prints a structured `REFUSE:` message; the model should ask for context and re-run with an enriched query.

Bypass only when the user insists:

```bash
LAST30DAYS_SKIP_PREFLIGHT=1 python3 scripts/last30days.py "birthday gift for 40 year old man"
```

SKILL.md Step 0.45 documents additional trap classes (numeric keyword collision, tutorial phrasing, generic single nouns) handled at the harness layer before engine invocation.

## Documented failure modes

| Failure mode | Surface | Mitigation |
| --- | --- | --- |
| Stale marketplace SKILL.md | Missing flags, wrong Step 0 behavior | STEP 0 cache redirect; `claude plugin update` |
| v3.0.6 “generic research” regression | Invented titles, `Sources:` footer, skipped Step 0.55 | Mandatory badge, LAWs block at top of SKILL.md, `SKILL_DIR`-scoped engine |
| Class 1 keyword trap | Irrelevant Reddit/HN noise | Engine refuse + Step 0.45 reframe/`--subreddits` |
| Stale `yt-dlp` | Videos without transcripts | Quality nudge + upgrade binary |
| Instagram SC silent zero | No section, no error | Documented in `quality_nudge`; check `INCLUDE_SOURCES` / topic shape |
| No sources available | Immediate `RuntimeError` | `--diagnose`; add keys per table above |
| Open PR CI cascade on version pins | Unrelated PRs red after release | Resolved by removing `sync.sh` pin test (see `docs/solutions/workflow-issues/release-consistency-test-cascade-2026-05-16.md`) |
| Search-quality regressions | Not caught in default CI | `evaluate_search_quality.py` is manual / `workflow_dispatch` by design (`docs/solutions/architecture/search-quality-eval-manual-by-default-2026-05-10.md`) |

<Note>
Search-quality eval intentionally stays out of default PR CI: live APIs, cost, and LLM-judge non-determinism. Request manual eval on retrieval/ranking changes; do not expect pytest alone to catch relevance regressions.
</Note>

## Runtime UI and errors in output

During a run, `ProgressDisplay` shows per-source counts at completion. If Reddit and web grounding are both unavailable, `show_promo()` nudges configuration (suppressed when `--plan` or `--competitors-plan` indicates a hosting model with native WebSearch).

Rendered `--emit=md` / compact output includes **Source Coverage** and **Source Errors** sections when `report.errors_by_source` is non-empty. Warnings from `_warnings()` pass through in the report payload for synthesis.

For SSH/datacenter YouTube blocks, set `LAST30DAYS_YOUTUBE_SSH_HOST` in config so `yt-dlp` routes through `ssh` (see `youtube_yt.py`).

## Debug and verification flags

<ParamField body="LAST30DAYS_DEBUG" type="string">
Set `1` in environment or config (or pass `--debug`) to enable HTTP debug logging.
</ParamField>

<ParamField body="LAST30DAYS_SKIP_PREFLIGHT" type="string">
Bypass Class 1 refuse gate when set before invocation.
</ParamField>

<ParamField body="--mock" type="flag">
Use fixture retrieval; useful for tests without live API keys.
</ParamField>

## Related pages

<CardGroup>
<Card title="Configure sources" href="/configure-sources">
Credential layers, `INCLUDE_SOURCES`, and per-source API keys tied to diagnose output.
</Card>
<Card title="Quickstart" href="/quickstart">
First invocation, setup wizard, and `--diagnose` verification in the happy path.
</Card>
<Card title="CLI reference" href="/cli-reference">
Full flag list including `--diagnose`, `--search`, depth profiles, and direct CLI constraints.
</Card>
<Card title="Skill contract" href="/skill-contract">
STEP 0 stale-clone guard, pre-flight protocol, and `SKILL_DIR` engine substitution.
</Card>
<Card title="Model clients" href="/model-clients">
Per-harness install, marketplace vs `npx skills`, and stale-install avoidance.
</Card>
<Card title="Configuration reference" href="/configuration-reference">
Env var catalog, `.env` lookup order, and web-backend priority.
</Card>
</CardGroup>

---

## 21. Develop and test

> uv/pytest workflow, coverage omit rules, validate and security CI, optional search-quality eval, and contributor constraints from AGENTS.md.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/21-develop-and-test.md
- Generated: 2026-06-04T23:26:24.252Z

### Source Files

- `AGENTS.md`
- `pyproject.toml`
- `tests/test_pipeline_v3.py`
- `.github/workflows/validate.yml`
- `.github/workflows/security.yml`
- `docs/search-quality-eval.md`
- `skills/last30days/scripts/evaluate_search_quality.py`

---
title: "Develop and test"
description: "uv/pytest workflow, coverage omit rules, validate and security CI, optional search-quality eval, and contributor constraints from AGENTS.md."
---

Local development centers on **Python 3.12+**, the **uv** toolchain (`pyproject.toml`, `uv.lock`), and a **pytest** suite under `tests/` that imports engine modules from `skills/last30days/scripts` via `tests/conftest.py`. Default CI on pull requests and `main` runs `uv run pytest` only; advisory security scans and optional retrieval-quality evaluation sit outside that blocking path.

## Prerequisites

| Requirement | Source |
|-------------|--------|
| Python ≥ 3.12 | `pyproject.toml` `requires-python` |
| [uv](https://docs.astral.sh/uv/) | `AGENTS.md`, `.github/workflows/validate.yml` |
| Virtualenv at `.venv/` | Created by `uv` on first `uv run` |

Runtime dependencies for the skill engine are stdlib-first (`[project] dependencies = []`). Dev tools live in `[dependency-groups] dev`: `pytest` and `pytest-cov`.

<Steps>
<Step title="Install toolchain and sync dev deps">

```bash
uv python install 3.12
uv sync --group dev
```

</Step>
<Step title="Run the full test suite">

```bash
uv run pytest
```

Default pytest options (`pyproject.toml`): quiet mode (`-q`), short tracebacks (`--tb=short`), `testpaths = ["tests"]`.

</Step>
<Step title="Optional: coverage report">

```bash
uv run pytest --cov
```

Vendor code under `skills/last30days/scripts/lib/vendor/` is omitted from coverage (see below).

</Step>
</Steps>

## Test layout and imports

`tests/conftest.py` prepends `skills/last30days/scripts` to `sys.path`, so tests import engine modules as `from lib import pipeline` (same layout as the engine at runtime).

The suite spans **94** `test_*.py` modules and **1600+** collected cases (pipeline, sources, CLI, rendering, env, competitors, watchlist, evaluator, and workflow contract tests). Representative offline pipeline coverage lives in `tests/test_pipeline_v3.py`, which exercises `pipeline.run(..., mock=True)` without live credentials.

### Targeted runs

```bash
uv run pytest tests/test_pipeline_v3.py
uv run pytest tests/test_dedupe_v3.py -k test_some_case
```

### Direct engine smoke test (dev only)

`AGENTS.md` treats the slash command as the product path; direct CLI invocation is for scripting, cron, and engine debugging:

```bash
python3 skills/last30days/scripts/last30days.py "test query" --emit=compact
```

## Coverage configuration

Coverage is configured under `[tool.coverage.run]` and `[tool.coverage.report]` in `pyproject.toml`:

| Setting | Value |
|---------|--------|
| `branch` | `true` |
| `source` | `skills/last30days/scripts`, `tests` |
| `omit` | `skills/last30days/scripts/lib/vendor/*`, `dist/*` |
| `skip_empty` | `true` (report) |
| `show_missing` | `true` (report) |

Vendored X client code (`lib/vendor/bird-search/`) is excluded so coverage reflects project-owned logic, not third-party trees.

## CI workflows

```mermaid
flowchart LR
  subgraph pr_main["PR / push to main"]
    V[validate.yml\nuv run pytest]
    S[security.yml\nadvisory scans]
  end
  subgraph tag["Tag push v*"]
    R[release.yml\nbuild-skill.sh → .skill]
  end
  pr_main --> V
  pr_main --> S
  tag --> R
```

### Validate (blocking)

`.github/workflows/validate.yml` runs on every pull request and push to `main`:

1. Checkout
2. `astral-sh/setup-uv@v5`
3. `uv python install 3.12`
4. `uv run pytest`

No search-quality eval, lint gate, or release build in this workflow.

### Security (advisory)

`.github/workflows/security.yml` runs on PR, push to `main`, and `workflow_dispatch`. Both jobs use **`continue-on-error: true`** until maintainers confirm a clean baseline; comments in the workflow describe flipping to blocking enforcement.

| Job | Tool | Purpose |
|-----|------|---------|
| `dependency-audit` | `uv export --locked --all-groups` → `pip-audit` | CVE visibility on locked dev deps |
| `secret-scan` | TruffleHog OSS `--only-verified` | Verified secrets in PR/push ranges |

`tests/test_security_workflow.py` locks the advisory policy (pip-audit and TruffleHog present, `continue-on-error: true`, contributor guidance in workflow comments). `AGENTS.md` requires not weakening this workflow without an explained PR rationale.

### Release (tags only)

`.github/workflows/release.yml` triggers on tags `v*`, runs `skills/last30days/scripts/build-skill.sh`, uploads `dist/last30days.skill` via `softprops/action-gh-release@v2`.

## Contract and policy tests

Beyond behavioral unit tests, several modules enforce repository invariants:

| Module | Enforces |
|--------|----------|
| `tests/test_plugin_contract.py` | Version alignment across `pyproject.toml`, `SKILL.md`, `.claude-plugin/plugin.json`, `gemini-extension.json`, marketplace shape; no removed `.codex-plugin/`; workflows must not reference legacy root `scripts/` paths |
| `tests/test_version_consistency.py` | `SKILL.md` double-quoted version, header badge, memory-dir env conventions |
| `tests/test_security_workflow.py` | Security workflow structure and `AGENTS.md` secret-hygiene mentions |

<Warning>
Avoid cross-file version pin tests that compare a bumped `SKILL.md` to a stale hardcoded path in another artifact — that pattern caused cascade CI failures across open PRs after releases (`docs/solutions/workflow-issues/release-consistency-test-cascade-2026-05-16.md`). Prefer single-source version reads or generated pins.
</Warning>

## Optional search-quality evaluation

`skills/last30days/scripts/evaluate_search_quality.py` compares two git revisions on ranked-candidate output. It is **not** part of default CI (`docs/solutions/architecture/search-quality-eval-manual-by-default-2026-05-10.md`): live API cost, latency, and judge non-determinism make it a maintainer-triggered check, not a per-PR gate.

### What it measures

**Deterministic stability** (baseline vs candidate):

- Jaccard overlap on ranked item keys
- Retention vs baseline
- Per-source counts and overlap

**Optional LLM judge** (when a Google/Gemini API key is available):

- `Precision@5`, `nDCG@5`, source-coverage recall over a judged union pool
- Judgments cached under `--output-dir` (default `tmp/search-quality`)

### Default topics

`fixtures/eval_topics.json` supplies eight reviewer topics (comparison, how_to, breaking_news, product, opinion, prediction, concept, factual). Override with `--topics-file`.

### CLI flags (script)

| Flag | Default | Role |
|------|---------|------|
| `--baseline` | `HEAD~1` | Git ref or label; materialized via `git worktree` unless `WORKTREE` |
| `--candidate` | `WORKTREE` | Same; `WORKTREE` uses the current checkout |
| `--search` | `""` | Passed to engine `--search` when set |
| `--output-dir` | `tmp/search-quality` | Metrics and judgment cache |
| `--judge-model` | `gemini-3.1-flash-lite` | Gemini judge model (`lib.providers.GEMINI_FLASH_LITE`) |
| `--timeout` | `240` | Per-topic subprocess timeout (seconds) |
| `--limit` | `20` | Ranked items considered per report |
| `--mock` | off | Forwards `--mock` to engine |
| `--quick` | off | Forwards `--quick` to engine |
| `--topics-file` | — | JSON list of `{topic, query_type}` |

Example:

```bash
uv run python skills/last30days/scripts/evaluate_search_quality.py \
  --baseline origin/main \
  --candidate WORKTREE \
  --quick
```

`create_eval_env()` clears `LAST30DAYS_CONFIG_DIR` and passes selected API keys from env/config (`GOOGLE_API_KEY`, `OPENAI_API_KEY`, `XAI_API_KEY`, `SCRAPECREATORS_API_KEY`, Bluesky/Truth Social tokens, etc.) so eval runs use a controlled config surface. See `docs/search-quality-eval.md` for judge key names and operational notes.

<Info>
Jaccard and retention are regression guards, not ground-truth quality. Judge metrics help compare revisions but do not replace a larger labeled benchmark.
</Info>

Unit tests in `tests/test_evaluator_v3.py` cover ranked-item extraction, worktree resolution, env construction, and summary writers without requiring a full live eval pass.

## Contributor constraints (`AGENTS.md`)

| Area | Rule |
|------|------|
| Product boundary | Agent Skills package; new engine flags need `SKILL.md` integration so harness models see them |
| `lib/__init__.py` | Bare package marker only — no eager imports |
| Install sync | `npx skills add . -g -y` copies a frozen skill tree; re-run after edits, or symlink `skills/last30days` → `~/.agents/skills/last30days` for live dev |
| Slash vs shell | Harness slash commands cannot pipe flags; use direct `python3 .../last30days.py` for shell mechanics |
| Secrets | Never commit real keys, cookies, or `.env`; use `lib/env.py` patterns; dummy values in tests/fixtures |
| Security CI | Do not disable `.github/workflows/security.yml` without PR explanation |
| `CONFIGURATION.md` | Update when adding env vars, CLI config flags, install patterns, or config precedence |
| Beta | Experimental work on private repo → `/last30days-beta`; public promotion via review PR |

## Skill install during development

To test harness behavior after skill edits:

```bash
npx skills add . -g -y
```

For iterative skill work without reinstalling:

```bash
ln -sfn "$PWD/skills/last30days" ~/.agents/skills/last30days
```

(run from repo root)

## Related pages

<CardGroup>
<Card title="Skill, engine, and harness" href="/skill-engine-harness">
Boundaries between SKILL.md, `last30days.py`, and multi-harness runtimes — context for why tests import `lib` from `scripts/`.
</Card>
<Card title="Research pipeline" href="/research-pipeline">
v3 orchestration exercised heavily by `test_pipeline_v3.py` and mock runs.
</Card>
<Card title="CLI reference" href="/cli-reference">
Direct engine flags (`--emit`, `--quick`, `--mock`) used in dev smoke tests and eval subprocesses.
</Card>
<Card title="Beta channel" href="/beta-channel">
Parallel `/last30days-beta` install path for pre-public experiments.
</Card>
<Card title="Troubleshooting" href="/troubleshooting">
`--diagnose` and runtime failure modes outside pytest.
</Card>
</CardGroup>

---

## 22. Beta channel

> Parallel /last30days-beta install from the private repo, promotion workflow, and where client-specific customizations belong.

- Page Markdown: https://grok-wiki.com/public/docs/mvanhorn-last30days-skill-50b5421a8cca/pages/22-beta-channel.md
- Generated: 2026-06-04T23:24:57.467Z

### Source Files

- `CONCEPTS.md`
- `CONFIGURATION.md`
- `AGENTS.md`
- `CHANGELOG.md`
- `docs/releases/v3.0.9.md`

---
title: "Beta channel"
description: "Parallel /last30days-beta install from the private repo, promotion workflow, and where client-specific customizations belong."
---

The beta channel is a second Agent Skills install sourced from the private companion repository `mvanhorn/last30days-skill-private`, exposed to harnesses as `/last30days-beta` alongside the public `/last30days` skill from `mvanhorn/last30days-skill`. Experimental SKILL.md and engine changes land in the private repo first; promotion to the public marketplace requires an explicit review pull request against the public repository—beta-only changes never ship on the public channel without that step.

## Public vs beta

| Dimension | Public channel | Beta channel |
|---|---|---|
| Repository | `mvanhorn/last30days-skill` (this repo) | `mvanhorn/last30days-skill-private` |
| Slash command | `/last30days` | `/last30days-beta` |
| Distribution | Claude Code marketplace, `npx skills add`, per-harness skill dirs | Private-repo install (see `BETA.md` in the private repo) |
| Badge line (first output line) | `🌐 last30days v{VERSION} · synced {YYYY-MM-DD}` | `🧪 last30days-beta · branch <name> · synced {YYYY-MM-DD}` |
| Promotion path | Default release target | Review PR → merge into public `main` |

Both channels share the same architectural split documented in [Skill, engine, and harness](/skill-engine-harness): `SKILL.md` is the runtime contract; `scripts/last30days.py` is the engine. The beta install is a parallel copy of that package with its own harness registration, not a flag on the public skill.

```mermaid
flowchart LR
  subgraph public["mvanhorn/last30days-skill"]
    P_SKILL["skills/last30days/SKILL.md"]
    P_ENGINE["scripts/last30days.py"]
  end
  subgraph private["mvanhorn/last30days-skill-private"]
    B_SKILL["SKILL.md + scripts/"]
    B_BETA["BETA.md workflow"]
  end
  subgraph harness["Harness runtime"]
    CMD_P["/last30days"]
    CMD_B["/last30days-beta"]
  end
  P_SKILL --> CMD_P
  P_ENGINE --> CMD_P
  B_SKILL --> CMD_B
  B_BETA -.->|promotion via review PR| public
```

<Note>
Detailed install, sync, and day-to-day beta operations live in `BETA.md` inside the private repository. That file is not vendored into the public tree; treat it as the operator runbook for beta maintainers.
</Note>

## Why the beta channel exists

Real-user validation on experimental SKILL.md and engine changes must not block the stable public marketplace. The beta channel lets maintainers iterate on contract changes (LAWs, badge enforcement, pre-flight flags) and client-specific data (category peers, subreddit lists, competitors-plan templates) before those changes appear in `mvanhorn/last30days-skill`.

The public `SKILL.md` documents why that separation matters in practice: on 2026-04-18 the same model achieved **10/10** compliance on the beta channel and **0/8** on public v3.0.6 with similar SKILL.md content—the delta was structural anchors (mandatory badge, `SKILL_DIR` substitution, preface ordering), not model capability. Beta is the environment where those anchors are proven before promotion.

## What belongs in beta vs public

Use the beta channel for customizations you do **not** intend to upstream—client-vertical category rows in `scripts/lib/categories.py`, internal subreddit lists, recurring `--competitors-plan` JSON skeletons, and other fork-specific SKILL.md tweaks.

Use the public repo’s [per-client setup](/per-client-setup) patterns when the customization is portable or may eventually ship to all users:

| Pattern | Public repo support | Typical beta use |
|---|---|---|
| Project-scoped `.claude/last30days.env` | Yes — [Configure sources](/configure-sources) | Rarely; env files work on either install |
| `--save-suffix` / `LAST30DAYS_MEMORY_DIR` wrappers | Yes | Beta runs often use `-raw-beta` suffix (see A/B testing) |
| Custom `categories.py` rows | Yes (upstreamable) | Vertical-specific peers you will not PR publicly |
| Pre-built `--competitors-plan` JSON | Yes (upstreamable) | Client-industry templates kept private |

<Warning>
Never publish beta-only changes to the public Claude Code marketplace or `main` without a review PR against `mvanhorn/last30days-skill`. The public marketplace plugin (`last30days` in `.claude-plugin/marketplace.json`) tracks only the public repository.
</Warning>

## Promotion workflow

Promotion is a deliberate merge from private experimentation into the public release line—not a separate release channel inside one repo.

<Steps>
<Step title="Develop and validate on beta">
Install or sync from `mvanhorn/last30days-skill-private` so `/last30days-beta` loads the private `SKILL.md` and engine. Run representative topics; confirm the beta badge line appears and output passes the LAWs contract.
</Step>
<Step title="Open a review PR on the public repo">
Cherry-pick or re-implement the validated changes on a branch of `mvanhorn/last30days-skill`. Scope the PR to what should ship to all users; leave client-only forks in the private repo.
</Step>
<Step title="Merge and release publicly">
After review, merge to public `main`, bump plugin version metadata, and let users pick up the release via marketplace update or `npx skills add` re-sync. Beta-only artifacts stay in the private repo.
</Step>
</Steps>

```mermaid
stateDiagram-v2
  [*] --> BetaDev: change in last30days-skill-private
  BetaDev --> BetaTest: /last30days-beta validation
  BetaTest --> BetaDev: fix failures
  BetaTest --> ReviewPR: open PR on last30days-skill
  ReviewPR --> PublicMain: merge after review
  PublicMain --> [*]: /last30days marketplace release
  BetaDev --> BetaOnly: client fork stays private
  BetaOnly --> [*]
```

The setup plan referenced in contributor docs (`docs/plans/2026-04-17-005-feat-beta-skill-from-private-repo-plan.md`) established this two-repo layout; that plan file is not present in the public tree—use `BETA.md` in the private repo for operational detail.

## A/B testing public vs beta

`skills/last30days/scripts/compare.sh` is a maintainer-side runner that invokes both slash commands sequentially (30s gap for rate limits) and prints saved raw file paths for manual diff review.

<CodeGroup>
```bash title="compare.sh invocation"
bash skills/last30days/scripts/compare.sh "Kevin Rose"
```

```text title="Expected output files"
$LAST30DAYS_MEMORY_DIR/<slug>-raw.md        # public /last30days
$LAST30DAYS_MEMORY_DIR/<slug>-raw-beta.md   # private /last30days-beta
```
</CodeGroup>

The script drives Claude Code headlessly:

```bash
claude -p --dangerously-skip-permissions "/last30days $TOPIC"
claude -p --dangerously-skip-permissions "/last30days-beta $TOPIC"
```

Verification signals after a run:

| Check | Public | Beta |
|---|---|---|
| First-line badge | `🌐 last30days v…` | `🧪 last30days-beta · branch …` |
| Raw filename suffix | `-raw.md` | `-raw-beta.md` |
| Regression signal | Missing public badge → stale install | Missing beta badge → beta badge regression |

If the beta badge is absent, treat it as a contract regression in the private `SKILL.md` synthesis section (the public repo’s `compare.sh` footer points maintainers at the beta setup plan naming pattern).

## Install and sync boundaries

**Public install (end users):** Claude Code marketplace (`/plugin install last30days@last30days-skill`), `npx skills add mvanhorn/last30days-skill -g -y`, or harness-specific paths documented in [Installation](/installation) and [Model clients](/model-clients).

**Beta install (maintainers / private forks):** Follow `BETA.md` in `mvanhorn/last30days-skill-private`. The public repo does not ship beta marketplace metadata.

Historical note: an older maintainer `sync.sh` script once pinned Claude plugin-cache paths; [#402](https://github.com/mvanhorn/last30days-skill/pull/402) fixed it pointing at the public cache instead of the private repo, and [#405](https://github.com/mvanhorn/last30days-skill/pull/405) removed `sync.sh` entirely in favor of `npx skills add . -g -y` (live symlink) for public dev deploys. Beta deploy semantics remain private-repo concerns.

<Info>
Public `SKILL.md` STEP 0 guards against a different stale-install bug: Claude Code’s `~/.claude/plugins/marketplaces/last30days-skill/` git clone lagging the versioned cache. That check applies to `/last30days`, not the beta install path.
</Info>

## Contributor rules (public repo)

From `AGENTS.md` and `CONFIGURATION.md`:

- Mirror new **public** configuration knobs in `CONFIGURATION.md` when they land in `SKILL.md` or the engine.
- Keep beta-only experiments out of public `main` until the review PR lands.
- When a config concept is beta-only today but becomes universal, promote code **and** docs in the same public PR.

## Related pages

<CardGroup>
<Card title="Per-client setup" href="/per-client-setup">
Project-scoped `.claude/last30days.env`, save-dir isolation, and portable wrapper patterns on the public skill.
</Card>
<Card title="Installation" href="/installation">
Marketplace and `npx skills add` flows for the public `/last30days` channel.
</Card>
<Card title="Skill contract" href="/skill-contract">
SKILL.md runtime authority, STEP 0 stale-clone guard, and engine invocation rules shared by both channels.
</Card>
<Card title="Develop and test" href="/develop-and-test">
`uv run pytest`, contributor constraints, and validation before promotion.
</Card>
</CardGroup>

---