diff --git a/CLAUDE.md b/CLAUDE.md index 6ceab30..462ed7b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,12 +11,12 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co `just` (task runner) + `uv` (package manager); the [`Justfile`](Justfile) is the source of truth for recipes — run `just --list` or read it. The non-obvious bits: - `just test [args]` — full suite in docker compose (Postgres 17). Args forward **unquoted**, so a spaced `-k` expression (`-k "a or b"`) word-splits and fails (`file or directory not found: or`) — run one keyword per invocation, or a single substring matching all targets. `tests/test_unit.py` + `tests/test_fake.py` need no Postgres (`uv run pytest tests/test_unit.py` works directly); `tests/test_integration.py` needs Postgres at `POSTGRES_DSN` (default `postgresql+asyncpg://outbox:outbox@localhost:5432/outbox`; `pg_engine` skips if unreachable). Coverage is on with `--cov-fail-under=100` — partial runs fail that gate; pass `--no-cov` or `--cov-fail-under=0` when iterating. -- `just lint` / `just lint-ci` — autofix vs non-mutating; `lint-ci` also runs the planning-bundle validator. +- `just lint` / `just lint-ci` — autofix vs non-mutating; `lint-ci` also runs the planning-change validator. - `just docs-serve` / `just docs-build` — local hot-reload at `http://127.0.0.1:8000` / one-shot strict `mkdocs build`. ## Workflow -Planning uses a portable convention — `architecture/` (repo root) is the living **truth home** and promotion target; `planning/changes/` holds the per-change bundles. Start at the [Quick path](planning/README.md#quick-path-start-here) in `planning/README.md` (the authoritative spec) to pick a lane — **Full** (`design.md` + `plan.md`), **Lightweight** (single `change.md`), or **Tiny** (just a commit) — and ship. `just check-planning` validates bundles; `just index` prints the change + decision listing; `planning/_templates/` are copy-and-fill starting points. A design decision taken **without** a code change — especially a **rejected** option with a load-bearing reason — goes in `planning/decisions/YYYY-MM-DD-.md` (`status: accepted|superseded`) with a **Revisit trigger** so future reviews don't re-litigate it. +Planning uses a portable convention — `architecture/` (repo root) is the living **truth home** and promotion target; `planning/changes/` holds the per-change files. Start at the [Quick path](planning/README.md#quick-path-start-here) in `planning/README.md` (the authoritative spec) to pick a lane — **Full** (design template), **Lightweight** (change template), or **Tiny** (just a commit) — and ship. `just check-planning` validates changes; `just index` prints the change + decision listing; `planning/_templates/` are copy-and-fill starting points. A design decision taken **without** a code change — especially a **rejected** option with a load-bearing reason — goes in `planning/decisions/YYYY-MM-DD-.md` (`status: accepted|superseded`) with a **Revisit trigger** so future reviews don't re-litigate it. ## Architecture diff --git a/Justfile b/Justfile index 4870ac3..bf7aec9 100644 --- a/Justfile +++ b/Justfile @@ -33,7 +33,7 @@ lint-ci: index: uv run python planning/index.py -# Validate planning bundles + decisions (frontmatter, lanes, spec links); CI runs this. +# Validate planning changes + decisions (frontmatter, lanes, spec links); CI runs this. check-planning: uv run python planning/index.py --check diff --git a/architecture/README.md b/architecture/README.md index 3de782e..29ce963 100644 --- a/architecture/README.md +++ b/architecture/README.md @@ -30,5 +30,5 @@ These files carry **no frontmatter** — they are prose, dated by git. ## Promotion rule Shipping a change hand-edits the affected capability file(s) here to match the -new reality, in the same PR as the code. The change bundle stays in place under +new reality, in the same PR as the code. The change file stays in place under [`../planning/changes/`](../planning/changes/) — no folder move. diff --git a/planning/.convention-version b/planning/.convention-version index 3eefcb9..227cea2 100644 --- a/planning/.convention-version +++ b/planning/.convention-version @@ -1 +1 @@ -1.0.0 +2.0.0 diff --git a/planning/README.md b/planning/README.md index 34d24bd..55a8ab0 100644 --- a/planning/README.md +++ b/planning/README.md @@ -12,25 +12,22 @@ at the repo root; this directory records *how it got there*. **1. Choose a lane — first matching rule wins:** 1. Any of: needs design judgment · new file/module · public-API change · - cross-cutting or multi-file · non-trivial test design → **Full** - (`design.md` + `plan.md`) + cross-cutting or multi-file · non-trivial test design → **Full** (design template) 2. Purely mechanical: typo · dep bump · linter/formatter/CI tweak · - mechanical rename · single-line config → **Tiny** (no bundle, conventional + mechanical rename · single-line config → **Tiny** (no change file, conventional commit) 3. Small-but-real, none of the above: ≲30 LOC net · ≤2 files · no new file · - no public-API change · one straightforward test → **Lightweight** - (`change.md`) + no public-API change · one straightforward test → **Lightweight** (change template) -Ambiguous between two? Take the heavier. A `change.md` that outgrows its lane -splits into `design.md` + `plan.md`. +Ambiguous between two? Take the heavier. A lightweight change file that outgrows its lane is rewritten from the design template. -**2. Create the bundle** (Full / Lightweight only): -`planning/changes/YYYY-MM-DD.NN-/`, where `.NN` is a zero-padded -intra-day counter. Copy the matching template from +**2. Create the change file** (Full / Lightweight only): +`planning/changes/YYYY-MM-DD.NN-.md`, where `.NN` is a zero-padded +intra-day counter — copied from the matching template (design or change) in [`_templates/`](_templates/). **3. Ship in the implementing PR:** hand-edit the affected -`architecture/.md`, finalize the bundle's `summary:` to the +`architecture/.md`, finalize the change file's `summary:` to the realized result, and run `just check-planning` before pushing. ## Conventions @@ -43,20 +40,41 @@ realized result, and run `just check-planning` before pushing. ### Two axes, never mixed -- **`architecture/` (repo root) — the present.** One file per capability, - living prose, updated in the same PR that ships the change. The truth home. -- **`planning/changes/` — the past-and-pending.** One folder per change, +- **`architecture/` (repo root) — the present.** One file per capability, plus + a single `glossary.md` (the ubiquitous language); living prose, updated in the + same PR that ships the change. The truth home. +- **`planning/changes/` — the past-and-pending.** One file per change, kept in place after ship. A change **promotes** its conclusions into the affected `architecture/.md` by hand **in the implementing PR, alongside the code** — the edit rides in the same diff and is reviewed with it, never applied as a separate post-merge step. That hand-edit is what keeps `architecture/` -true; the bundle stays in `changes/` as the *why*. +true; the change file stays in `changes/` as the *why*. -### Change bundles +### Glossary -A change is a folder `changes/YYYY-MM-DD.NN-/`: +`architecture/glossary.md` is the project's **ubiquitous language** — one page +defining the domain terms that code, specs, and capability pages all share. Like +the capability files beside it, it is living prose with **no frontmatter**, dated +by git, and authored lazily: it appears when the first term is worth pinning down. + +Each entry is a term, a one-or-two-sentence definition of what it *is* (not what +it does), and an optional `_Avoid_:` line naming the synonyms to reject: + +```md +**Timer**: +A scheduled future delivery, identified by a timer id. +_Avoid_: job, task, alarm +``` + +Keep it a glossary, not a spec — no implementation detail. A change that +introduces or sharpens a term updates `glossary.md` in the same PR, the same way +a behavior change promotes into a capability file. + +### Change files + +A change is a file `changes/YYYY-MM-DD.NN-.md`: - `YYYY-MM-DD` — proposal date; `.NN` — zero-padded intra-day counter (`.01`, `.02`, …) that breaks same-date ties so the timeline sorts stably. @@ -65,25 +83,44 @@ A change is a folder `changes/YYYY-MM-DD.NN-/`: `summary` is written when the change is created (the intent one-liner) and **finalized at ship** to state the realized result — set in the implementing PR, alongside the code and the `architecture/` promotion. No post-merge -bookkeeping, no folder move. `date` and `slug` are never written — they are -read from the bundle's directory name. +bookkeeping, no file move. `date` and `slug` are never written — they are +read from the file name. ### Three lanes | Lane | Artifacts | Use when | |------|-----------|----------| -| **Full** | `design.md` + `plan.md` | design judgment; new file/module; public-API change; cross-cutting/multi-file; non-trivial test design | -| **Lightweight** | `change.md` | small-but-real: ≲30 LOC net, ≤2 files, no new file, no public-API change, single straightforward test | +| **Full** | one change file from the design template | design judgment; new file/module; public-API change; cross-cutting/multi-file; non-trivial test design | +| **Lightweight** | one change file from the change template | small-but-real: ≲30 LOC net, ≤2 files, no new file, no public-API change, single straightforward test | | **Tiny** | none — conventional commit | typo, dep bump, linter/formatter/CI tweak, mechanical rename, single-line config | -Heavier lane wins on ambiguity. A `change.md` that outgrows its lane splits -into `design.md` + `plan.md`. +Heavier lane wins on ambiguity. A lightweight change file that outgrows its lane is rewritten from the design template. + +### Plans are ephemeral + +The executable plan — task checklists, embedded code, commit sequences, +whatever the executor needs — is a working artifact, not history. Keep it out +of `changes/` and out of version control (git-ignored scratch, e.g. +`.superpowers/`). Once the change ships, the diff and the PR are the record +of execution; a committed plan duplicates them. `check-planning` rejects +anything in `changes/` that is not a flat change file. + +### Lean specs + +The change file is the single home of a change's rationale: + +- The PR body summarizes and links to the change file — it never restates it. +- Rejected alternatives live in `decisions/` and are referenced, not retold. +- Show a sketch when the design needs code; never the full diff-to-be. +- Delete template sections that don't apply — an empty section is ceremony. +- Most designs fit well under ~700 words; length must buy information. ### Artifacts at a glance -- **`design.md`** — the spec: the *thinking* (why, design, trade-offs, scope). -- **`plan.md`** — the plan: the *sequencing* (the executor's task checklist). -- **`change.md`** — both, condensed, for the lightweight lane. +- **design template** — the spec: the *thinking* (why, design, trade-offs, + scope); the change file it produces is the single home of rationale (see + [Lean specs](#lean-specs)). +- **change template** — the condensed spec for the lightweight lane. - **`releases/.md`** — per-release user-facing notes. - **`audits/-.md`** — findings from a code/docs/bug-hunt sweep; spawns fix changes. @@ -97,11 +134,10 @@ Templates live in [`_templates/`](_templates/). ### Frontmatter -`date` and `slug` are **derived from the directory / file name** — never +`date` and `slug` are **derived from the file name** — never repeated in frontmatter. So: -- `design.md` / `change.md`: `summary` (single line) only. -- `plan.md`: **no frontmatter** — its identity is the bundle directory. +- `changes/*.md`: `summary` (single line) only. - `decisions/*.md`: `status` (accepted|superseded), `summary`, and optional `supersedes` / `superseded_by`. - Files in `architecture/` carry **no** frontmatter — living prose, dated by git. @@ -114,7 +150,7 @@ only field the index renders. The listing is **generated**, not maintained — run `just index` to print it: a flat, newest-first list of changes, then decisions newest-first. The frontmatter -in each bundle / decision file is the single source of truth; there is no +in each change / decision file is the single source of truth; there is no committed copy to drift. ## Other diff --git a/planning/_templates/change.md b/planning/_templates/change.md index d4c8962..5aa7e81 100644 --- a/planning/_templates/change.md +++ b/planning/_templates/change.md @@ -5,8 +5,8 @@ summary: One line — shown in the generated index. Written at creation; finaliz # Change: One-line capitalized title **Lane:** lightweight — ≲30 LOC net, ≤2 files, no new file, no public-API -change, a single straightforward test. If it outgrows this, split into -`design.md` + `plan.md`. +change, a single straightforward test. If it outgrows this, rewrite it from +the design template. ## Goal diff --git a/planning/_templates/design.md b/planning/_templates/design.md index d63e22d..17dbee1 100644 --- a/planning/_templates/design.md +++ b/planning/_templates/design.md @@ -4,6 +4,10 @@ summary: One line — shown in the generated index. Written at creation; finaliz # Design: One-line capitalized title + + ## Summary One paragraph. What changes, at the level a reader needs to decide if this @@ -12,37 +16,23 @@ spec is worth reading in full. ## Motivation Why now. What is broken or missing. Concrete observations / numbers, not -abstract complaints. Link to memory entries or earlier specs when relevant. - -## Non-goals - -What is deliberately out of scope and (when nontrivial) why. Each item is -a sentence; one line each. +abstract complaints. ## Design -### 1. - What changes, in enough detail that a reader who has not seen the codebase -can follow. Code samples / diagrams welcome. +can follow. Sketches and interface fragments welcome; never the full +diff-to-be. Reference rejected alternatives in `decisions/` instead of +retelling them. -### 2. - -... - -## Operations - -Out-of-repo steps (DNS, infra, external account changes). Omit if none. - -## Out of scope +## Non-goals -Already covered above under Non-goals if appropriate. Repeat-list of -explicitly-excluded follow-ups belongs here when the list is long. +What is deliberately out of scope and (when nontrivial) why. One line each. ## Testing -How we know it landed correctly. New pytest? Smoke check on live URL? -Lint pass? Be specific. +How we know it landed correctly. Be specific: the command and the expected +signal. ## Risk diff --git a/planning/_templates/glossary.md b/planning/_templates/glossary.md new file mode 100644 index 0000000..82385c3 --- /dev/null +++ b/planning/_templates/glossary.md @@ -0,0 +1,15 @@ +# Glossary + +The project's ubiquitous language — the domain terms that code, specs, and +capability pages share. Living prose, no frontmatter, dated by git. Each entry is +a term, what it *is* (not what it does), and the synonyms to avoid. No +implementation detail; this is a glossary, not a spec. + +**Term**: +A one-or-two-sentence definition of what it is. +_Avoid_: rejected-synonym, another-one + +**Another term**: +Define what it is, tightly. Group related terms under `##` subheadings when +natural clusters emerge; a flat list is fine when they don't. +_Avoid_: … diff --git a/planning/_templates/plan.md b/planning/_templates/plan.md deleted file mode 100644 index 132d720..0000000 --- a/planning/_templates/plan.md +++ /dev/null @@ -1,46 +0,0 @@ -# — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use -> superpowers:subagent-driven-development (recommended) or -> superpowers:executing-plans to implement this plan task-by-task. Steps -> use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** One sentence — what shipping this plan achieves. No design -rationale; link to the spec for that. - -**Spec:** [`design.md`](./design.md) - -**Branch:** `feat/my-change` (or `fix/`, `chore/`, etc.) - -**Commit strategy:** Per-task commits / single commit / squash on merge. -Whichever fits. - ---- - -### Task 1: - -**Files:** -- Modify: `path/to/file.py` -- Create: `path/to/new.py` - -One sentence on what this task accomplishes. No deeper reasoning — that's -in the spec. - -- [ ] **Step 1: ** - - Run / edit / verify command. Expected output. - -- [ ] **Step 2: ** - - ... - -- [ ] **Step 3: Commit** - - ```bash - git add path/to/file.py - git commit -m ": " - ``` - ---- - -### Task 2: ... diff --git a/planning/_templates/release.md b/planning/_templates/release.md index 7372d7e..5081187 100644 --- a/planning/_templates/release.md +++ b/planning/_templates/release.md @@ -1,4 +1,4 @@ -# modern-di — +# — @@ -30,10 +30,9 @@ Context a reader needs for the headline change. Omit for small releases. ## Downstream -What integrations (FastAPI, Litestar, FastStream, Typer, `modern-di-pytest`) -must do — e.g. bump their `modern-di` floor — or "No action needed" when there -is no API change. +What dependents must do — e.g. bump their version floor — or "No action +needed" when there is no API change. Omit if the project has no downstreams. ## Internals -- Coverage / tooling notes (e.g. 100% line coverage across Python 3.10–3.14). +- Coverage / tooling notes. diff --git a/planning/changes/2026-06-03.01-all-extra-and-planning-dir/design.md b/planning/changes/2026-06-03.01-all-extra-and-planning-dir.md similarity index 100% rename from planning/changes/2026-06-03.01-all-extra-and-planning-dir/design.md rename to planning/changes/2026-06-03.01-all-extra-and-planning-dir.md diff --git a/planning/changes/2026-06-03.01-all-extra-and-planning-dir/plan.md b/planning/changes/2026-06-03.01-all-extra-and-planning-dir/plan.md deleted file mode 100644 index 5d81ed6..0000000 --- a/planning/changes/2026-06-03.01-all-extra-and-planning-dir/plan.md +++ /dev/null @@ -1,380 +0,0 @@ -# `all` extra and `planning/` workflow dir — Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Add a `faststream-outbox[all]` aggregate extra and bootstrap the `planning/` directory convention (`specs/` + `plans/`) used by the sister `httpware` project. - -**Architecture:** No runtime code change. Three artifacts touched: `pyproject.toml` (one new line under `[project.optional-dependencies]`), `planning/specs/.gitkeep` + `planning/plans/.gitkeep` (zero-byte placeholders), and `CLAUDE.md` (one new `## Workflow` section between `## Commands` and `## Architecture`). All changes land in a single commit. - -**Tech Stack:** `uv` (lock + sync), `just` (lint + test recipes), Python 3.13+, Postgres 17 (test target — not exercised by this plan but the test suite must still pass). - -**Spec:** `planning/specs/2026-06-03-all-extra-and-planning-dir-design.md` (committed as `91201a2`). - ---- - -## File map - -| Path | Action | Responsibility | -|---|---|---| -| `pyproject.toml` | Modify (add one entry to `[project.optional-dependencies]`) | Expose `all` aggregate extra | -| `planning/specs/.gitkeep` | Create (zero bytes) | Keep dir tracked even if all specs are ever removed | -| `planning/plans/.gitkeep` | Create (zero bytes) | Keep dir tracked (this plan file will live here too, but `.gitkeep` is defensive per spec §2) | -| `CLAUDE.md` | Modify (insert new `## Workflow` section between line 18 and line 19) | Document the per-feature workflow lifecycle so future contributors find it | - -No source files (`src/...`) or test files (`tests/...`) change. There are no new Python symbols. - ---- - -## Task 1: Add `all` extra to `pyproject.toml` - -**Files:** -- Modify: `pyproject.toml` (insert after line 21, the `opentelemetry` entry, inside the `[project.optional-dependencies]` table) - -- [ ] **Step 1: Verify current state of the extras table** - -Run: -```bash -sed -n '16,22p' /Users/kevinsmith/src/pypi/faststream-outbox/pyproject.toml -``` - -Expected output (exact): -```toml -[project.optional-dependencies] -asyncpg = ["asyncpg>=0.29"] -validate = ["alembic>=1.13"] -fastapi = ["fastapi>=0.95"] -prometheus = ["prometheus-client>=0.19"] -opentelemetry = ["opentelemetry-api>=1.20", "opentelemetry-sdk>=1.20"] -``` - -If the output differs, STOP — the file has drifted from the spec's assumptions and the spec needs revisiting. - -- [ ] **Step 2: Add the `all` entry** - -Use the Edit tool to append after the `opentelemetry` line. The new content immediately following `opentelemetry = [...]` should be: - -```toml -all = ["faststream-outbox[asyncpg,validate,fastapi,prometheus,opentelemetry]"] -``` - -Concretely, the `old_string` for Edit is: -```toml -opentelemetry = ["opentelemetry-api>=1.20", "opentelemetry-sdk>=1.20"] -``` - -And the `new_string` is: -```toml -opentelemetry = ["opentelemetry-api>=1.20", "opentelemetry-sdk>=1.20"] -all = ["faststream-outbox[asyncpg,validate,fastapi,prometheus,opentelemetry]"] -``` - -- [ ] **Step 3: Verify TOML still parses** - -Run: -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run python -c "import tomllib, pathlib; data = tomllib.loads(pathlib.Path('pyproject.toml').read_text()); print(data['project']['optional-dependencies']['all'])" -``` - -Expected output: -``` -['faststream-outbox[asyncpg,validate,fastapi,prometheus,opentelemetry]'] -``` - -If it errors with a `tomllib.TOMLDecodeError`, the edit broke the file — re-open and fix. - -- [ ] **Step 4: Verify `uv` can resolve `--extra all`** - -Run: -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv sync --extra all 2>&1 | tail -20 -``` - -Expected: completes without errors. Look for `Resolved packages` and `Installed` / `Audited` lines mentioning `alembic`, `asyncpg`, `fastapi`, `prometheus-client`, `opentelemetry-api`, `opentelemetry-sdk`. (If the env already had them via `dev` group, the output may just say `Audited packages in ` — that's also success.) - -If it fails with `failed to resolve`, the most likely cause is the self-referential extra name not matching the project name. Verify the project name with: -```bash -grep '^name =' /Users/kevinsmith/src/pypi/faststream-outbox/pyproject.toml -``` -It must read `name = "faststream-outbox"` (hyphen, not underscore). - -- [ ] **Step 5: Verify `--extra all` matches `--all-extras`** - -Run: -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && \ - uv sync --extra all --quiet && uv pip list --format=freeze | sort > /tmp/with-all.txt && \ - uv sync --all-extras --quiet && uv pip list --format=freeze | sort > /tmp/with-all-extras.txt && \ - diff /tmp/with-all.txt /tmp/with-all-extras.txt && echo "MATCH" -``` - -Expected: `MATCH` on stdout, no diff output. If diff prints lines, the `all` extra is missing or has extras the broad form doesn't — re-check the entry against the spec. - -Cleanup: -```bash -rm -f /tmp/with-all.txt /tmp/with-all-extras.txt -``` - -**Do not commit yet — single commit at the end of Task 4.** - ---- - -## Task 2: Create `planning/` placeholder files - -**Files:** -- Create: `planning/specs/.gitkeep` (zero bytes) -- Create: `planning/plans/.gitkeep` (zero bytes) - -The directories `planning/specs/` and `planning/plans/` already exist on disk (created during the brainstorming step that produced the spec) and `planning/specs/2026-06-03-all-extra-and-planning-dir-design.md` is already tracked. Only the `.gitkeep` placeholders need adding. - -- [ ] **Step 1: Confirm the directories exist** - -Run: -```bash -ls -la /Users/kevinsmith/src/pypi/faststream-outbox/planning/ -``` - -Expected: shows `specs/` and `plans/` subdirectories. If either is missing, run `mkdir -p /Users/kevinsmith/src/pypi/faststream-outbox/planning/{specs,plans}` first. - -- [ ] **Step 2: Create both `.gitkeep` files** - -Run: -```bash -touch /Users/kevinsmith/src/pypi/faststream-outbox/planning/specs/.gitkeep && \ -touch /Users/kevinsmith/src/pypi/faststream-outbox/planning/plans/.gitkeep -``` - -- [ ] **Step 3: Verify both files exist and are empty** - -Run: -```bash -wc -c /Users/kevinsmith/src/pypi/faststream-outbox/planning/specs/.gitkeep /Users/kevinsmith/src/pypi/faststream-outbox/planning/plans/.gitkeep -``` - -Expected output (byte counts of 0): -``` -0 .../planning/specs/.gitkeep -0 .../planning/plans/.gitkeep -0 total -``` - -If `eof-fixer` later complains about the files (it generally leaves zero-byte files alone, but if it doesn't), accept whatever it does — the file existing under the name `.gitkeep` is what matters; the byte count is incidental. - -**Do not commit yet.** - ---- - -## Task 3: Add `## Workflow` section to `CLAUDE.md` - -**Files:** -- Modify: `CLAUDE.md` (insert a new `## Workflow` section between line 18 and line 19, i.e. between the blank line that ends `## Commands` and the `## Architecture` heading) - -- [ ] **Step 1: Verify the insertion point** - -Run: -```bash -sed -n '15,21p' /Users/kevinsmith/src/pypi/faststream-outbox/CLAUDE.md -``` - -Expected output (exact): -``` -- `just build` / `just down` / `just sh` — image build, teardown, shell into the app container. - -`tests/test_unit.py` and `tests/test_fake.py` need no Postgres — runnable with `uv run pytest tests/test_unit.py` directly. `tests/test_integration.py` requires Postgres at `POSTGRES_DSN` (default `postgresql+asyncpg://outbox:outbox@localhost:5432/outbox`); the `pg_engine` fixture skips if unreachable. Coverage is on by default (`pyproject.toml` `addopts`) with a strict `--cov-fail-under=100` ratchet — partial runs (`pytest -k name`, a single test file, etc.) will fail that gate. Pass `--no-cov` or `--cov-fail-under=0` when iterating locally on a subset; the full `just test` run satisfies the gate. - -## Architecture - -The package wires a FastStream `Broker`/`Registrator`/`Subscriber` trio whose transport is Postgres rows, not a message bus. -``` - -If the output differs, the file has drifted — STOP and re-locate the insertion point manually. - -- [ ] **Step 2: Insert the `## Workflow` section** - -Use the Edit tool with this exact replacement. - -`old_string` (matches the blank-line / `## Architecture` boundary unambiguously because `## Architecture` only appears once in the file): - -``` -gate. - -## Architecture -``` - -`new_string`: - -``` -gate. - -## Workflow - -Per-feature workflow: brainstorming → spec in `planning/specs/YYYY-MM-DD--design.md` → writing-plans → plan in `planning/plans/YYYY-MM-DD--plan.md` → executing-plans / subagent-driven-development → requesting-code-review → finishing-a-development-branch. - -Topic slugs are kebab-case descriptions (e.g. `dlq-on-terminal-failure`), not story IDs. - -## Architecture -``` - -Note: the surrounding `gate.` line is the tail of the long Commands paragraph (the last word of line 17). Anchoring on it makes `old_string` unique even though the markdown structure (blank line + heading) is otherwise generic. - -- [ ] **Step 3: Verify the section landed** - -Run: -```bash -grep -n '^## ' /Users/kevinsmith/src/pypi/faststream-outbox/CLAUDE.md -``` - -Expected output (order matters): -``` -5:## Project -9:## Commands -19:## Workflow -25:## Architecture -... -``` - -(Architecture's new line number will be 25 or thereabouts depending on whitespace — the key check is that `## Workflow` appears between `## Commands` and `## Architecture`.) - -- [ ] **Step 4: Verify the section body is correct** - -Run: -```bash -sed -n '/^## Workflow/,/^## Architecture/p' /Users/kevinsmith/src/pypi/faststream-outbox/CLAUDE.md -``` - -Expected output: -``` -## Workflow - -Per-feature workflow: brainstorming → spec in `planning/specs/YYYY-MM-DD--design.md` → writing-plans → plan in `planning/plans/YYYY-MM-DD--plan.md` → executing-plans / subagent-driven-development → requesting-code-review → finishing-a-development-branch. - -Topic slugs are kebab-case descriptions (e.g. `dlq-on-terminal-failure`), not story IDs. - -## Architecture -``` - -**Do not commit yet.** - ---- - -## Task 4: Verify and commit - -**Files:** none new — this task just runs verification + creates the commit. - -- [ ] **Step 1: Run `just lint-ci`** - -Run: -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && just lint-ci -``` - -Expected: all checks pass (`eof-fixer`, `ruff format --check`, `ruff check`, `ty check`). No Python files were touched in this plan, so `ruff` and `ty` are effectively no-ops on the diff; `eof-fixer` may report something about `.gitkeep` or `CLAUDE.md` — if it modifies them, re-run `just lint-ci` and confirm clean. - -If `just` is unavailable, the equivalent is: -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && \ - uv run eof-fixer --check . && \ - uv run ruff format --check . && \ - uv run ruff check . && \ - uv run ty check . -``` - -- [ ] **Step 2: Run the test suite** - -Run: -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && just test -``` - -Expected: full suite passes with 100% coverage (the `--cov-fail-under=100` gate). No behavior change is expected — this confirms `pyproject.toml` is still syntactically valid and the package still installs cleanly in the docker compose env. - -If `just test` is impractical (docker not available), at minimum run the no-Postgres subset: -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run pytest tests/test_unit.py tests/test_fake.py --no-cov -``` -…and note in the commit description that the integration suite was not exercised locally. - -- [ ] **Step 3: Inspect the full diff one last time** - -Run: -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git status && echo '---' && git diff && echo '---' && git diff --stat -``` - -Expected `git status` shows: -- Modified: `CLAUDE.md` -- Modified: `pyproject.toml` -- (Possibly) Modified: `uv.lock` if the `--extra all` sync touched it -- Untracked: `planning/plans/.gitkeep`, `planning/specs/.gitkeep` - -(The plan file `planning/plans/2026-06-03-all-extra-and-planning-dir-plan.md` should already have been committed in a separate `docs: plan ...` commit ahead of execution — analogous to how the spec was committed at `91201a2`. If it still shows as untracked, commit it on its own *before* proceeding to Step 4.) - -`git diff --stat` should show ~3 modified lines in `pyproject.toml` (one added entry, possibly reformatted), and a small CLAUDE.md insertion (~6 lines). - -If `uv.lock` shows a diff, that's fine — include it in the commit (lock files belong with the change that produced them). - -- [ ] **Step 4: Stage the changes** - -Run: -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git add \ - pyproject.toml \ - CLAUDE.md \ - planning/specs/.gitkeep \ - planning/plans/.gitkeep -``` - -If `uv.lock` is also modified: -```bash -git add uv.lock -``` - -Verify staged set: -```bash -git status -``` - -Expected: all of the above under `Changes to be committed`, nothing under `Changes not staged for commit` (other than possibly untracked files outside this work). - -- [ ] **Step 5: Commit** - -Run: -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git commit -m "$(cat <<'EOF' -chore: add 'all' extra and planning/ workflow directory - -- pyproject.toml: self-referential `all` aggregate so users can - `pip install faststream-outbox[all]` for the full feature set - (asyncpg + validate + fastapi + prometheus + opentelemetry). -- planning/{specs,plans}/: adopt the layout used by the sister - httpware project for superpowers spec/plan artifacts. -- CLAUDE.md: document the per-feature workflow lifecycle in a new - `## Workflow` section between `## Commands` and `## Architecture`. - -Spec: planning/specs/2026-06-03-all-extra-and-planning-dir-design.md. - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - -- [ ] **Step 6: Confirm the commit landed** - -Run: -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git log -1 --stat -``` - -Expected: one new commit with the message above and the file list from step 4. - ---- - -## Acceptance criteria (spec §"Verification checklist") - -After Task 4 completes, all of these must be true: - -- [ ] `uv sync --extra all` succeeds and pulls in alembic, asyncpg, fastapi, prometheus-client, opentelemetry-api, opentelemetry-sdk. (Task 1, Step 4.) -- [ ] `uv sync --extra all` and `uv sync --all-extras` produce the same installed set. (Task 1, Step 5.) -- [ ] `just lint-ci` passes. (Task 4, Step 1.) -- [ ] `just test` passes. (Task 4, Step 2.) -- [ ] `planning/specs/` and `planning/plans/` exist and are tracked by git (the design doc, this plan, and both `.gitkeep` files appear in `git ls-files`). Verify with: `git ls-files planning/`. -- [ ] `CLAUDE.md` `## Workflow` heading appears between `## Commands` and `## Architecture`. (Task 3, Step 3.) diff --git a/planning/changes/2026-06-03.02-faststream-0.7-migration/design.md b/planning/changes/2026-06-03.02-faststream-0.7-migration.md similarity index 100% rename from planning/changes/2026-06-03.02-faststream-0.7-migration/design.md rename to planning/changes/2026-06-03.02-faststream-0.7-migration.md diff --git a/planning/changes/2026-06-03.02-faststream-0.7-migration/plan.md b/planning/changes/2026-06-03.02-faststream-0.7-migration/plan.md deleted file mode 100644 index 448b1c5..0000000 --- a/planning/changes/2026-06-03.02-faststream-0.7-migration/plan.md +++ /dev/null @@ -1,1571 +0,0 @@ -# FastStream 0.7 Migration — Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Migrate `faststream-outbox` to depend on `faststream>=0.7,<0.8`, fix the five mechanically-forced break points (codec attribute on producers, `add_call` lost `middlewares_=`, `create_publisher_fake_subscriber` is now an instance method), and drop the public per-call `middlewares=` kwarg upstream removed. - -**Architecture:** Single-version code path (no 0.6 compat shim). All changes land on `chore/faststream-0.7-migration` and are squashed to one bundled commit before opening the PR — the spec calls for a single coherent migration commit. During development, normal commits per task are fine; the final task squashes them. - -**Tech Stack:** `uv` for lock/sync; `just` for lint/test recipes; Python 3.13+; FastStream 0.7.x; SQLAlchemy 2.0+ (asyncpg); pytest with `--cov-fail-under=100`. - -**Spec:** `planning/specs/2026-06-03-faststream-0.7-migration-design.md` (committed in the previous turn — see `git log --oneline | head -1`). - ---- - -## File map - -| Path | Action | Responsibility | -|---|---|---| -| `pyproject.toml` | Modify | Bump pin from `faststream>=0.6,<0.7` to `faststream>=0.7,<0.8` | -| `faststream_outbox/publisher/producer.py` | Modify | Add `codec` attribute to `OutboxProducer` to satisfy 0.7 `ProducerProto` | -| `faststream_outbox/testing.py` | Modify | Add `codec` attribute to `FakeOutboxProducer`; change `create_publisher_fake_subscriber` from `@staticmethod` to instance method | -| `faststream_outbox/registrator.py` | Modify | Drop `middlewares=` kwarg from `subscriber()` + `publisher()`; drop `middlewares_=` from the `add_call(...)` call | -| `faststream_outbox/publisher/factory.py` | Modify | Drop `middlewares=` parameter from `create_publisher` | -| `faststream_outbox/publisher/config.py` | Modify | Drop `middlewares` field from `OutboxPublisherConfig` | -| `faststream_outbox/router.py` | Modify | Drop `middlewares=` kwarg from `OutboxRoute.__init__` (the broker-level `OutboxRouter.middlewares=` STAYS — it routes to `broker_middlewares=`) | -| `faststream_outbox/fastapi/router.py` | Modify | Drop `middlewares=` kwarg from `subscriber()` and `publisher()` overrides; the broker-level `middlewares=` on `OutboxRouter.__init__` STAYS | -| `faststream_outbox/subscriber/factory.py` | Conditional | Verify `AckPolicy.ACK_FIRST` survived; delete the rejection branch + its test only if the enum member is gone | -| `tests/test_fake.py` | Modify | Delete `test_publisher_accepts_middlewares_kwarg` (the only test covering the removed kwarg) | -| `tests/test_unit.py` | Conditional | Update if `_basic_publish`/`_publish` `_extra_middlewares=` kwarg renamed in 0.7 | -| `docs/usage/publisher.md` | Modify | Remove the "Per-publisher `middlewares=` wrap every `publisher.publish(...)` call" sentence at line 103–105 | -| `CLAUDE.md` | Modify | Trim two passages mentioning per-call middlewares (line 37 `broker.publisher(...)` signature + the "*middlewares*" sentence in the same paragraph) | -| `uv.lock` | Auto-updated | Updated by `uv lock --upgrade` in Task 1 | - -**Files explicitly NOT modified:** -- `tests/test_middleware_opentelemetry.py` and `tests/test_middleware_prometheus.py` use `OutboxBroker(middlewares=[...])` — that's broker-level middleware, the 0.7-supported shape; leave them. -- `docs/usage/observability.md` uses broker-level `middlewares=` in `OutboxBroker(...)` — same; leave it. -- `README.md` has no per-call middleware references. - ---- - -## Task 1: Branch + pin bump + first-pass install - -**Files:** -- Modify: `pyproject.toml` (line 13) -- Auto-modified: `uv.lock` - -- [ ] **Step 1: Create the migration branch** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git switch -c chore/faststream-0.7-migration -``` - -Expected: `Switched to a new branch 'chore/faststream-0.7-migration'`. If the branch already exists from a prior aborted attempt, run `git switch chore/faststream-0.7-migration` and confirm `git status` is clean before proceeding. - -- [ ] **Step 2: Bump the pin** - -Use Edit on `pyproject.toml`: - -`old_string`: -```toml - "faststream>=0.6,<0.7", -``` - -`new_string`: -```toml - "faststream>=0.7,<0.8", -``` - -- [ ] **Step 3: Resolve and install** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv lock --upgrade-package faststream && uv sync --all-extras --all-groups -``` - -Expected: `Resolved` line that names a `faststream` version starting with `0.7.`. If resolution fails with a `no version found` error, check PyPI manually (`uv pip index versions faststream`) — the 0.7 release may have been yanked or the version range may need widening. - -- [ ] **Step 4: Capture the installed FastStream version + audit the upstream shapes you'll need** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run python -c " -import faststream, inspect -print('faststream version:', faststream.__version__) - -# R2: where does CodecProto live, and is it Optional in ProducerProto? -from faststream._internal.producer import ProducerProto -print('ProducerProto annotations:', getattr(ProducerProto, '__annotations__', {})) -try: - from faststream._internal.codec import CodecProto - print('CodecProto: faststream._internal.codec.CodecProto') -except ImportError: - import importlib, pkgutil, faststream._internal - print('CodecProto NOT in faststream._internal.codec; searching:') - for m in pkgutil.walk_packages(faststream._internal.__path__, prefix='faststream._internal.'): - try: - mod = importlib.import_module(m.name) - if hasattr(mod, 'CodecProto'): - print(' found in:', m.name) - except Exception: - pass - -# R3: did AckPolicy.ACK_FIRST survive? -from faststream.middlewares import AckPolicy -print('AckPolicy members:', [a.name for a in AckPolicy]) - -# Confirm add_call signature -from faststream._internal.endpoint.subscriber.usecase import SubscriberUsecase -print('add_call sig:', inspect.signature(SubscriberUsecase.add_call)) - -# Confirm create_publisher_fake_subscriber shape -from faststream._internal.testing.broker import TestBroker -print('create_publisher_fake_subscriber sig:', - inspect.signature(TestBroker.create_publisher_fake_subscriber)) -" -``` - -Record the output. The remaining tasks make decisions based on: -- **`CODEC_IMPORT_PATH`** — the dotted import for `CodecProto` (e.g. `faststream._internal.codec`). -- **`CODEC_IS_OPTIONAL`** — whether `ProducerProto.__annotations__['codec']` resolves to a non-Optional type (if yes, we'll default to a no-op instance; if Optional, default to `None`). -- **`ACK_FIRST_GONE`** — `True` if `AckPolicy.ACK_FIRST` is not in the members list; controls Task 9. -- **`ADD_CALL_KWARGS`** — confirm `parser_`, `decoder_`, `dependencies_` are present and `middlewares_` is absent (matches spec). -- **`CREATE_PUB_FAKE_SIG`** — confirm `self` is the first parameter (instance method, not static). - -If any decision differs from spec assumptions (e.g. `middlewares_` is still in `add_call`), STOP — re-open the spec to revise before continuing. - -- [ ] **Step 5: First-pass lint (expect failures)** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && just lint-ci 2>&1 | tee /tmp/lint-after-bump.txt | tail -40 -``` - -Expected: failures in `ty check` against the files listed in the file map above (missing `codec` attr on producers, wrong static/instance method, etc.). Keep `/tmp/lint-after-bump.txt` for reference while you do Tasks 2–8 — it's your punch list. - -If `ruff format --check` complains about unrelated files, fix those formatting issues here (one tiny formatting commit doesn't fight the single-commit goal — the squash at Task 11 collapses everything). - -- [ ] **Step 6: Stage + interim commit** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git add pyproject.toml uv.lock && git commit -m "wip: bump faststream pin to >=0.7,<0.8" -``` - -(This is an interim commit; Task 11 squashes the branch to a single commit.) - ---- - -## Task 2: Satisfy `ProducerProto.codec` on `OutboxProducer` - -**Files:** -- Modify: `faststream_outbox/publisher/producer.py` (around lines 36–56) - -This task uses the `CODEC_IMPORT_PATH` and `CODEC_IS_OPTIONAL` values captured in Task 1, Step 4. - -- [ ] **Step 1: Write a failing test asserting `OutboxProducer` satisfies `ProducerProto`** - -Add to `tests/test_unit.py` (locate the section that tests `OutboxProducer` — there's an existing block; add this near the existing producer tests, or at end of file if none cluster nicely): - -```python -def test_outbox_producer_satisfies_producer_proto() -> None: - """Phase-1 0.7 compat: ProducerProto in 0.7 requires a `codec` attribute.""" - from faststream._internal.producer import ProducerProto - from sqlalchemy import MetaData - - from faststream_outbox import make_outbox_table - from faststream_outbox.publisher.producer import OutboxProducer - - table = make_outbox_table(MetaData()) - producer = OutboxProducer(table=table, parser=None, decoder=None) - # Attribute access must not raise; instance must structurally match. - assert hasattr(producer, "codec") - assert hasattr(producer, "_parser") - assert hasattr(producer, "_decoder") - assert isinstance(producer, ProducerProto) -``` - -- [ ] **Step 2: Run the test to confirm it fails** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run pytest tests/test_unit.py::test_outbox_producer_satisfies_producer_proto -x --no-cov -v -``` - -Expected: `FAILED` with `AssertionError` on `hasattr(producer, "codec")` or `isinstance` check. - -- [ ] **Step 3: Add the `codec` attribute** - -If `CODEC_IS_OPTIONAL` is **True** (the Optional path — simpler): - -Use Edit on `faststream_outbox/publisher/producer.py`: - -`old_string`: -```python - def __init__( - self, - *, - table: Table, - parser: typing.Optional["CustomCallable"], - decoder: typing.Optional["CustomCallable"], - metrics_recorder: MetricsRecorder = _noop_recorder, - ) -> None: - self._table = table - self._channel = f"outbox_{table.name}" - self.serializer: SerializerProto | None = None - default = OutboxParser() - self._parser = ParserComposition(parser, default.parse_message) - self._decoder = ParserComposition(decoder, default.decode_message) - self._metrics_recorder = metrics_recorder -``` - -`new_string`: -```python - def __init__( - self, - *, - table: Table, - parser: typing.Optional["CustomCallable"], - decoder: typing.Optional["CustomCallable"], - metrics_recorder: MetricsRecorder = _noop_recorder, - ) -> None: - self._table = table - self._channel = f"outbox_{table.name}" - self.serializer: SerializerProto | None = None - # ProducerProto[0.7] requires a `codec` attribute. The outbox owns its - # own encoding pipeline (_encode_payload) and never reads this attribute - # at runtime — it exists solely to satisfy the protocol. - self.codec: "CodecProto | None" = None - default = OutboxParser() - self._parser = ParserComposition(parser, default.parse_message) - self._decoder = ParserComposition(decoder, default.decode_message) - self._metrics_recorder = metrics_recorder -``` - -And add the import (inside the TYPE_CHECKING block at lines ~29–33): - -`old_string`: -```python -if typing.TYPE_CHECKING: - from collections.abc import Mapping - - from fast_depends.library.serializer import SerializerProto - from faststream._internal.types import AsyncCallable, CustomCallable -``` - -`new_string`: -```python -if typing.TYPE_CHECKING: - from collections.abc import Mapping - - from fast_depends.library.serializer import SerializerProto - from import CodecProto # noqa: TC003 — protocol-only annotation - from faststream._internal.types import AsyncCallable, CustomCallable -``` - -Replace `` with the dotted import recorded in Task 1, Step 4 (likely `faststream._internal.codec`). The import lives under `TYPE_CHECKING` because we never reference `CodecProto` at runtime. - -If `CODEC_IS_OPTIONAL` is **False** (the non-Optional path): - -Same edits as above, except: -- The runtime import (NOT under TYPE_CHECKING) must bring in the upstream-provided default codec instance. Recipe: - ```python - from import - ``` - Inspect what upstream ships next to `CodecProto` — likely a `JSONCodec()` or similar. Use `dir()` on the codec module to find it: - ```bash - uv run python -c "import importlib; m = importlib.import_module(''); print([n for n in dir(m) if not n.startswith('_')])" - ``` -- Initialize the attribute as `self.codec: CodecProto = ()`. - -- [ ] **Step 4: Run the test, expect pass** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run pytest tests/test_unit.py::test_outbox_producer_satisfies_producer_proto -x --no-cov -v -``` - -Expected: `PASSED`. - -- [ ] **Step 5: Run `ty` on the producer file** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run ty check faststream_outbox/publisher/producer.py -``` - -Expected: no errors. If `ty` complains that the `codec` attribute conflicts with the abstract class, double-check the annotation form (Optional vs non-Optional). - -- [ ] **Step 6: Interim commit** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git add faststream_outbox/publisher/producer.py tests/test_unit.py && git commit -m "wip: add codec attribute to OutboxProducer for 0.7 ProducerProto" -``` - ---- - -## Task 3: Satisfy `ProducerProto.codec` on `FakeOutboxProducer` - -**Files:** -- Modify: `faststream_outbox/testing.py` (around line 281) - -- [ ] **Step 1: Write a failing test** - -Add to `tests/test_fake.py` (cluster near other producer/broker construction tests): - -```python -def test_fake_outbox_producer_satisfies_producer_proto() -> None: - """Phase-1 0.7 compat: FakeOutboxProducer needs the same `codec` attribute.""" - from faststream._internal.producer import ProducerProto - - from faststream_outbox.testing import FakeOutboxClient, FakeOutboxProducer - - broker = _make_broker() # uses the existing helper at top of test_fake.py - fc = FakeOutboxClient() - fp = FakeOutboxProducer(fc, broker, serializer=None, run_loops=False) - assert hasattr(fp, "codec") - assert isinstance(fp, ProducerProto) -``` - -- [ ] **Step 2: Run and confirm it fails** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run pytest tests/test_fake.py::test_fake_outbox_producer_satisfies_producer_proto -x --no-cov -v -``` - -Expected: `FAILED`. - -- [ ] **Step 3: Add `codec` to `FakeOutboxProducer`** - -Use Edit on `faststream_outbox/testing.py`: - -`old_string`: -```python -class FakeOutboxProducer: - """ - In-memory ``OutboxProducer`` substitute routing inserts through ``FakeOutboxClient``. - - Used by ``TestOutboxBroker`` so ``broker.publisher(queue).publish(body, session=...)`` - drives the same in-memory fake store as ``broker.publish(body, session=...)``. The - *session* on the command is ignored — the fake client has no transaction. - - In sync mode (``run_loops=False``), each successful insert short-circuits into - ``_sync_dispatch`` so handlers run before ``publish`` returns — matches the - ``broker.publish`` patch in ``_build_fake_publish``. - """ - - _parser: typing.Any = None - _decoder: typing.Any = None -``` - -`new_string`: -```python -class FakeOutboxProducer: - """ - In-memory ``OutboxProducer`` substitute routing inserts through ``FakeOutboxClient``. - - Used by ``TestOutboxBroker`` so ``broker.publisher(queue).publish(body, session=...)`` - drives the same in-memory fake store as ``broker.publish(body, session=...)``. The - *session* on the command is ignored — the fake client has no transaction. - - In sync mode (``run_loops=False``), each successful insert short-circuits into - ``_sync_dispatch`` so handlers run before ``publish`` returns — matches the - ``broker.publish`` patch in ``_build_fake_publish``. - """ - - _parser: typing.Any = None - _decoder: typing.Any = None - # ProducerProto[0.7] requires `codec`. The fake producer ignores it at - # runtime, same as OutboxProducer. - codec: typing.Any = None -``` - -Note: class-level `typing.Any = None` matches the existing `_parser` / `_decoder` shape on this class — no need to mirror the more careful annotation from Task 2. The point is structural satisfaction for `isinstance(fp, ProducerProto)`. - -If Task 2 went down the non-Optional `CodecProto` path, set `codec: typing.Any = ()` here too (instance assignment, not a class default — move into `__init__`). - -- [ ] **Step 4: Run the test, expect pass** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run pytest tests/test_fake.py::test_fake_outbox_producer_satisfies_producer_proto -x --no-cov -v -``` - -Expected: `PASSED`. - -- [ ] **Step 5: Interim commit** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git add faststream_outbox/testing.py tests/test_fake.py && git commit -m "wip: add codec attribute to FakeOutboxProducer" -``` - ---- - -## Task 4: Convert `create_publisher_fake_subscriber` to instance method - -**Files:** -- Modify: `faststream_outbox/testing.py` (lines 604–616) - -- [ ] **Step 1: Confirm upstream signature** - -The output from Task 1, Step 4 should have shown `create_publisher_fake_subscriber` as an instance method (first parameter `self`). If it didn't — STOP, re-investigate. - -- [ ] **Step 2: Drop `@staticmethod`, add `self`** - -Use Edit on `faststream_outbox/testing.py`: - -`old_string`: -```python - @staticmethod - def create_publisher_fake_subscriber( # pragma: no cover - broker: OutboxBroker, - publisher: typing.Any, - ) -> tuple["OutboxSubscriber", bool]: - # Required by FastStream's TestBroker abstract base, but never called — - # we skip the publisher fake-subscriber loop in ``_fake_start`` because - # ``FakeOutboxProducer`` already lands rows in the fake client AND drives - # the real subscriber via ``_sync_dispatch``. The FastStream - # publisher-spy infrastructure would mock the real handler and break that. - del broker, publisher - msg = "TestOutboxBroker handles publisher dispatch via FakeOutboxProducer; this is unreachable." - raise NotImplementedError(msg) -``` - -`new_string`: -```python - def create_publisher_fake_subscriber( # pragma: no cover - self, - broker: OutboxBroker, - publisher: typing.Any, - ) -> tuple["OutboxSubscriber", bool]: - # Required by FastStream's TestBroker abstract base, but never called — - # we skip the publisher fake-subscriber loop in ``_fake_start`` because - # ``FakeOutboxProducer`` already lands rows in the fake client AND drives - # the real subscriber via ``_sync_dispatch``. The FastStream - # publisher-spy infrastructure would mock the real handler and break that. - del self, broker, publisher - msg = "TestOutboxBroker handles publisher dispatch via FakeOutboxProducer; this is unreachable." - raise NotImplementedError(msg) -``` - -- [ ] **Step 3: Verify ty no longer complains about override mismatch** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run ty check faststream_outbox/testing.py -``` - -Expected: no `invalid-method-override` error on `create_publisher_fake_subscriber`. (Other errors may remain — they're addressed in later tasks.) - -- [ ] **Step 4: Interim commit** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git add faststream_outbox/testing.py && git commit -m "wip: create_publisher_fake_subscriber as instance method" -``` - ---- - -## Task 5: Drop `middlewares_=` from `add_call` call site - -**Files:** -- Modify: `faststream_outbox/registrator.py` (around lines 60 and 95–100) - -- [ ] **Step 1: Run the broken existing tests to confirm the failure mode** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run pytest tests/test_unit.py -k subscriber -x --no-cov 2>&1 | tail -20 -``` - -Expected: some test that exercises `broker.subscriber(...)` fails with `TypeError: add_call() got an unexpected keyword argument 'middlewares_'` — confirms 0.7 dropped the kwarg. - -- [ ] **Step 2: Drop `middlewares=` from `OutboxRegistrator.subscriber` signature** - -Use Edit on `faststream_outbox/registrator.py`: - -`old_string`: -```python - @override - def subscriber( # ty: ignore[invalid-method-override] - self, - queues: str | list[str], - *, - max_workers: int = 1, - retry_strategy: "RetryStrategyProto | None" = None, - fetch_batch_size: int = 10, - min_fetch_interval: float = 1.0, - max_fetch_interval: float = 10.0, - lease_ttl_seconds: float = 60.0, - max_deliveries: int | None = None, - ack_policy: AckPolicy | None = None, - dependencies: Iterable["Dependant"] = (), - parser: CustomCallable | None = None, - decoder: CustomCallable | None = None, - middlewares: Sequence[SubscriberMiddleware[OutboxInnerMessage]] = (), - title_: str | None = None, - description_: str | None = None, - include_in_schema: bool = True, - ) -> "OutboxSubscriber": -``` - -`new_string`: -```python - @override - def subscriber( # ty: ignore[invalid-method-override] - self, - queues: str | list[str], - *, - max_workers: int = 1, - retry_strategy: "RetryStrategyProto | None" = None, - fetch_batch_size: int = 10, - min_fetch_interval: float = 1.0, - max_fetch_interval: float = 10.0, - lease_ttl_seconds: float = 60.0, - max_deliveries: int | None = None, - ack_policy: AckPolicy | None = None, - dependencies: Iterable["Dependant"] = (), - parser: CustomCallable | None = None, - decoder: CustomCallable | None = None, - title_: str | None = None, - description_: str | None = None, - include_in_schema: bool = True, - ) -> "OutboxSubscriber": -``` - -- [ ] **Step 3: Drop `middlewares_=middlewares` from the `add_call(...)` call** - -`old_string`: -```python - return subscriber.add_call( - parser_=parser or self._parser, - decoder_=decoder or self._decoder, - dependencies_=dependencies, - middlewares_=middlewares, - ) -``` - -`new_string`: -```python - return subscriber.add_call( - parser_=parser or self._parser, - decoder_=decoder or self._decoder, - dependencies_=dependencies, - ) -``` - -- [ ] **Step 4: Clean up the `SubscriberMiddleware` import (now unused on this line)** - -Check whether `SubscriberMiddleware` is still referenced elsewhere in the file: - -```bash -grep -n "SubscriberMiddleware" /Users/kevinsmith/src/pypi/faststream-outbox/faststream_outbox/registrator.py -``` - -If the only remaining hit is the `from faststream._internal.types import ...` line at the top, remove `SubscriberMiddleware` from that import: - -`old_string`: -```python -from faststream._internal.types import CustomCallable, SubscriberMiddleware -``` - -`new_string`: -```python -from faststream._internal.types import CustomCallable -``` - -If `SubscriberMiddleware` is still referenced elsewhere (it shouldn't be after this edit), leave the import alone. - -- [ ] **Step 5: Drop `middlewares=` from `OutboxRegistrator.publisher` signature** - -`old_string`: -```python - @override - def publisher( # ty: ignore[invalid-method-override] - self, - queue: str, - *, - headers: dict[str, str] | None = None, - middlewares: Sequence["PublisherMiddleware[OutboxPublishCommand]"] = (), - title: str | None = None, - description: str | None = None, - schema: Any | None = None, - include_in_schema: bool = True, - ) -> OutboxPublisher: - """ - Construct a queue-scoped publisher. - - The publisher is standalone-only — call ``await pub.publish(body, session=session)`` - from inside your own transaction. Attempting to use it as a relay decorator on a - subscriber raises ``NotImplementedError`` at decoration time, since the dispatch - loop has no reachable ``AsyncSession`` without breaking the outbox transactional - contract. - - *middlewares* run around every ``publisher.publish(...)`` call (and around - ``broker.publish(...)`` when this publisher is used as the response publisher). - """ - publisher = create_publisher( - queue=queue, - headers=headers, - middlewares=middlewares, - broker_config=self.config, # ty: ignore[invalid-argument-type] - title_=title, - description_=description, - schema_=schema, - include_in_schema=include_in_schema, - ) - super().publisher(publisher) - return publisher -``` - -`new_string`: -```python - @override - def publisher( # ty: ignore[invalid-method-override] - self, - queue: str, - *, - headers: dict[str, str] | None = None, - title: str | None = None, - description: str | None = None, - schema: Any | None = None, - include_in_schema: bool = True, - ) -> OutboxPublisher: - """ - Construct a queue-scoped publisher. - - The publisher is standalone-only — call ``await pub.publish(body, session=session)`` - from inside your own transaction. Attempting to use it as a relay decorator on a - subscriber raises ``NotImplementedError`` at decoration time, since the dispatch - loop has no reachable ``AsyncSession`` without breaking the outbox transactional - contract. - """ - publisher = create_publisher( - queue=queue, - headers=headers, - broker_config=self.config, # ty: ignore[invalid-argument-type] - title_=title, - description_=description, - schema_=schema, - include_in_schema=include_in_schema, - ) - super().publisher(publisher) - return publisher -``` - -- [ ] **Step 6: Clean up the now-unused `PublisherMiddleware` import** - -Check: - -```bash -grep -n "PublisherMiddleware" /Users/kevinsmith/src/pypi/faststream-outbox/faststream_outbox/registrator.py -``` - -If the only remaining hit is the TYPE_CHECKING import, remove that line: - -`old_string`: -```python -if TYPE_CHECKING: - from fast_depends.dependencies import Dependant - from faststream._internal.types import PublisherMiddleware - - from faststream_outbox.response import OutboxPublishCommand - from faststream_outbox.retry import RetryStrategyProto - from faststream_outbox.subscriber.usecase import OutboxSubscriber -``` - -`new_string`: -```python -if TYPE_CHECKING: - from fast_depends.dependencies import Dependant - - from faststream_outbox.retry import RetryStrategyProto - from faststream_outbox.subscriber.usecase import OutboxSubscriber -``` - -(Note: also drop the `OutboxPublishCommand` TYPE_CHECKING import — it was only used in the dropped `Sequence["PublisherMiddleware[OutboxPublishCommand]"]` annotation. Confirm with the same grep.) - -- [ ] **Step 7: Also remove `Sequence` from the runtime import if no longer used** - -```bash -grep -n "^from collections.abc\|Sequence" /Users/kevinsmith/src/pypi/faststream-outbox/faststream_outbox/registrator.py -``` - -If `Sequence` is now unused, drop it from `from collections.abc import Iterable, Sequence`. - -- [ ] **Step 8: Re-run lint and the subscriber tests** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run ruff check faststream_outbox/registrator.py && uv run ty check faststream_outbox/registrator.py -``` - -Expected: clean. If ruff flags an unused import, drop it. - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run pytest tests/test_unit.py -k subscriber -x --no-cov 2>&1 | tail -10 -``` - -Expected: the previous `TypeError` is gone. There may be other failures still (covered in later tasks). - -- [ ] **Step 9: Interim commit** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git add faststream_outbox/registrator.py && git commit -m "wip: drop middlewares= kwarg from OutboxRegistrator subscriber/publisher" -``` - ---- - -## Task 6: Drop `middlewares` from publisher factory + config - -**Files:** -- Modify: `faststream_outbox/publisher/factory.py` -- Modify: `faststream_outbox/publisher/config.py` - -These changes are paired — `create_publisher(middlewares=...)` was already deleted from its only caller in Task 5; now drop it from the factory's signature and the config's field. - -- [ ] **Step 1: Drop `middlewares` param from `create_publisher`** - -Use Edit on `faststream_outbox/publisher/factory.py`: - -`old_string`: -```python -def create_publisher( - *, - queue: str, - headers: dict[str, str] | None, - middlewares: "Sequence[PublisherMiddleware[OutboxPublishCommand]]", - broker_config: "OutboxBrokerConfig", - title_: str | None, - description_: str | None, - schema_: typing.Any | None, - include_in_schema: bool, -) -> OutboxPublisher: - publisher_config = OutboxPublisherConfig( - _outer_config=broker_config, - queue=queue, - headers=headers, - middlewares=middlewares, - ) -``` - -`new_string`: -```python -def create_publisher( - *, - queue: str, - headers: dict[str, str] | None, - broker_config: "OutboxBrokerConfig", - title_: str | None, - description_: str | None, - schema_: typing.Any | None, - include_in_schema: bool, -) -> OutboxPublisher: - publisher_config = OutboxPublisherConfig( - _outer_config=broker_config, - queue=queue, - headers=headers, - ) -``` - -- [ ] **Step 2: Clean up the unused imports in factory.py** - -```bash -grep -n "Sequence\|PublisherMiddleware\|OutboxPublishCommand" /Users/kevinsmith/src/pypi/faststream-outbox/faststream_outbox/publisher/factory.py -``` - -If the TYPE_CHECKING block's `Sequence`, `PublisherMiddleware`, and `OutboxPublishCommand` imports are no longer referenced, edit: - -`old_string`: -```python -if typing.TYPE_CHECKING: - from collections.abc import Sequence - - from faststream._internal.types import PublisherMiddleware - - from faststream_outbox.configs import OutboxBrokerConfig - from faststream_outbox.response import OutboxPublishCommand -``` - -`new_string`: -```python -if typing.TYPE_CHECKING: - from faststream_outbox.configs import OutboxBrokerConfig -``` - -- [ ] **Step 3: Drop `middlewares` from `OutboxPublisherConfig`** - -Use Edit on `faststream_outbox/publisher/config.py`: - -`old_string`: -```python -"""Config dataclasses for the outbox publisher (usecase + AsyncAPI spec).""" - -import typing -from collections.abc import Sequence -from dataclasses import dataclass, field - -from faststream._internal.configs import PublisherSpecificationConfig, PublisherUsecaseConfig - - -if typing.TYPE_CHECKING: - from faststream._internal.types import PublisherMiddleware - - from faststream_outbox.configs import OutboxBrokerConfig - - -@dataclass(kw_only=True) -class OutboxPublisherConfig(PublisherUsecaseConfig): - _outer_config: "OutboxBrokerConfig" - queue: str - headers: dict[str, str] | None = None - middlewares: Sequence["PublisherMiddleware[typing.Any]"] = field(default_factory=tuple) -``` - -`new_string`: -```python -"""Config dataclasses for the outbox publisher (usecase + AsyncAPI spec).""" - -import typing -from dataclasses import dataclass - -from faststream._internal.configs import PublisherSpecificationConfig, PublisherUsecaseConfig - - -if typing.TYPE_CHECKING: - from faststream_outbox.configs import OutboxBrokerConfig - - -@dataclass(kw_only=True) -class OutboxPublisherConfig(PublisherUsecaseConfig): - _outer_config: "OutboxBrokerConfig" - queue: str - headers: dict[str, str] | None = None -``` - -- [ ] **Step 4: Verify lint and ty are clean on the two files** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && \ - uv run ruff check faststream_outbox/publisher/factory.py faststream_outbox/publisher/config.py && \ - uv run ty check faststream_outbox/publisher/factory.py faststream_outbox/publisher/config.py -``` - -Expected: clean. - -- [ ] **Step 5: Verify `PublisherUsecase` (upstream) doesn't read the removed `middlewares` field from the config** - -The dataclass removal could break the base class if 0.7's `PublisherUsecaseConfig` expects subclasses to carry a `middlewares` field. Sanity-check: - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run python -c " -from faststream._internal.configs import PublisherUsecaseConfig -import dataclasses -print('PublisherUsecaseConfig fields:', [f.name for f in dataclasses.fields(PublisherUsecaseConfig)]) -" -``` - -If `middlewares` is in the parent's field list, the parent's default propagates to our subclass — fine. If the parent class **requires** subclasses to override `middlewares`, then leave the field but make it a `Sequence[Any] = field(default_factory=tuple)` shim. Adjust based on what the inspection reveals. - -- [ ] **Step 6: Interim commit** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git add faststream_outbox/publisher/factory.py faststream_outbox/publisher/config.py && git commit -m "wip: drop middlewares from publisher factory + config" -``` - ---- - -## Task 7: Drop `middlewares=` from `OutboxRoute` - -**Files:** -- Modify: `faststream_outbox/router.py` (lines 23–62) - -`OutboxRouter.__init__` (lines 75–100) accepts a top-level `middlewares=` kwarg that maps to `broker_middlewares=` — that is broker-level and STAYS. Only `OutboxRoute` (which is a subscriber-route) loses the kwarg. - -- [ ] **Step 1: Drop the param + pass-through** - -Use Edit on `faststream_outbox/router.py`: - -`old_string`: -```python -class OutboxRoute(SubscriberRoute): - """Delayed-registration subscriber for use with ``OutboxRouter``.""" - - def __init__( # noqa: PLR0913 - self, - call: Callable[..., SendableMessage] | Callable[..., Awaitable[SendableMessage]], - queues: str | list[str], - *, - max_workers: int = 1, - retry_strategy: "RetryStrategyProto | None" = None, - fetch_batch_size: int = 10, - min_fetch_interval: float = 1.0, - max_fetch_interval: float = 10.0, - lease_ttl_seconds: float = 60.0, - max_deliveries: int | None = None, - ack_policy: AckPolicy | None = None, - dependencies: Iterable["Dependant"] = (), - parser: CustomCallable | None = None, - decoder: CustomCallable | None = None, - middlewares: Sequence[SubscriberMiddleware[OutboxInnerMessage]] = (), - title_: str | None = None, - description_: str | None = None, - include_in_schema: bool = True, - ) -> None: - super().__init__( - call=call, - queues=queues, - max_workers=max_workers, - retry_strategy=retry_strategy, - fetch_batch_size=fetch_batch_size, - min_fetch_interval=min_fetch_interval, - max_fetch_interval=max_fetch_interval, - lease_ttl_seconds=lease_ttl_seconds, - max_deliveries=max_deliveries, - ack_policy=ack_policy, - dependencies=dependencies, - parser=parser, - decoder=decoder, - middlewares=middlewares, - title_=title_, - description_=description_, - include_in_schema=include_in_schema, - ) -``` - -`new_string`: -```python -class OutboxRoute(SubscriberRoute): - """Delayed-registration subscriber for use with ``OutboxRouter``.""" - - def __init__( # noqa: PLR0913 - self, - call: Callable[..., SendableMessage] | Callable[..., Awaitable[SendableMessage]], - queues: str | list[str], - *, - max_workers: int = 1, - retry_strategy: "RetryStrategyProto | None" = None, - fetch_batch_size: int = 10, - min_fetch_interval: float = 1.0, - max_fetch_interval: float = 10.0, - lease_ttl_seconds: float = 60.0, - max_deliveries: int | None = None, - ack_policy: AckPolicy | None = None, - dependencies: Iterable["Dependant"] = (), - parser: CustomCallable | None = None, - decoder: CustomCallable | None = None, - title_: str | None = None, - description_: str | None = None, - include_in_schema: bool = True, - ) -> None: - super().__init__( - call=call, - queues=queues, - max_workers=max_workers, - retry_strategy=retry_strategy, - fetch_batch_size=fetch_batch_size, - min_fetch_interval=min_fetch_interval, - max_fetch_interval=max_fetch_interval, - lease_ttl_seconds=lease_ttl_seconds, - max_deliveries=max_deliveries, - ack_policy=ack_policy, - dependencies=dependencies, - parser=parser, - decoder=decoder, - title_=title_, - description_=description_, - include_in_schema=include_in_schema, - ) -``` - -- [ ] **Step 2: Clean up the now-unused `SubscriberMiddleware` import** - -Check: -```bash -grep -n "SubscriberMiddleware" /Users/kevinsmith/src/pypi/faststream-outbox/faststream_outbox/router.py -``` - -If only the import remains, drop `SubscriberMiddleware` from the line: - -`old_string`: -```python -from faststream._internal.types import BrokerMiddleware, CustomCallable, SubscriberMiddleware -``` - -`new_string`: -```python -from faststream._internal.types import BrokerMiddleware, CustomCallable -``` - -Then re-grep for `Sequence`; if only `Sequence[BrokerMiddleware[...]]` (in `OutboxRouter.__init__`) remains, leave it. - -- [ ] **Step 3: Verify lint + ty** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run ruff check faststream_outbox/router.py && uv run ty check faststream_outbox/router.py -``` - -Expected: clean. - -- [ ] **Step 4: Interim commit** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git add faststream_outbox/router.py && git commit -m "wip: drop middlewares= from OutboxRoute" -``` - ---- - -## Task 8: Drop `middlewares=` from FastAPI router subscriber/publisher overrides - -**Files:** -- Modify: `faststream_outbox/fastapi/router.py` (lines 153–236) - -The top-level `OutboxRouter.__init__(middlewares=...)` at line 80 is broker-level and STAYS (it maps to `broker_middlewares=` via the parent `StreamRouter.__init__`). Only the per-call `subscriber()` and `publisher()` overrides drop the kwarg. - -- [ ] **Step 1: Drop `middlewares=` from `OutboxRouter.subscriber` (signature + pass-through)** - -Use Edit on `faststream_outbox/fastapi/router.py`: - -`old_string`: -```python - def subscriber( # ty: ignore[invalid-method-override] # noqa: PLR0913 - self, - queues: str | list[str], - *, - # Outbox-subscriber knobs (mirror ``OutboxRegistrator.subscriber``) - max_workers: int = 1, - retry_strategy: "RetryStrategyProto | None" = None, - fetch_batch_size: int = 10, - min_fetch_interval: float = 1.0, - max_fetch_interval: float = 10.0, - lease_ttl_seconds: float = 60.0, - max_deliveries: int | None = None, - ack_policy: AckPolicy | None = None, - # FastStream subscriber-level knobs - dependencies: Iterable["params.Depends"] = (), - parser: CustomCallable | None = None, - decoder: CustomCallable | None = None, - middlewares: Sequence[SubscriberMiddleware[OutboxInnerMessage]] = (), - title_: str | None = None, - description_: str | None = None, - include_in_schema: bool = True, - # FastAPI response-model knobs (defaults match ``StreamRouter`` expectations) - response_model: typing.Any = _DEFAULT_RESPONSE_MODEL, - response_model_include: typing.Optional["IncEx"] = None, - response_model_exclude: typing.Optional["IncEx"] = None, - response_model_by_alias: bool = True, - response_model_exclude_unset: bool = False, - response_model_exclude_defaults: bool = False, - response_model_exclude_none: bool = False, - ) -> "OutboxSubscriber": - # ``StreamRouter.subscriber`` uses ``*extra: NameRequired | str`` — our - # ``queues: str | list[str]`` is wider; the actual broker-side - # ``OutboxRegistrator.subscriber`` accepts both. - return typing.cast( - "OutboxSubscriber", - super().subscriber( - queues, # ty: ignore[invalid-argument-type] - max_workers=max_workers, - retry_strategy=retry_strategy, - fetch_batch_size=fetch_batch_size, - min_fetch_interval=min_fetch_interval, - max_fetch_interval=max_fetch_interval, - lease_ttl_seconds=lease_ttl_seconds, - max_deliveries=max_deliveries, - ack_policy=ack_policy, - dependencies=dependencies, - parser=parser, - decoder=decoder, - middlewares=middlewares, - title_=title_, - description_=description_, - include_in_schema=include_in_schema, - response_model=response_model, - response_model_include=response_model_include, - response_model_exclude=response_model_exclude, - response_model_by_alias=response_model_by_alias, - response_model_exclude_unset=response_model_exclude_unset, - response_model_exclude_defaults=response_model_exclude_defaults, - response_model_exclude_none=response_model_exclude_none, - ), - ) -``` - -`new_string`: -```python - def subscriber( # ty: ignore[invalid-method-override] # noqa: PLR0913 - self, - queues: str | list[str], - *, - # Outbox-subscriber knobs (mirror ``OutboxRegistrator.subscriber``) - max_workers: int = 1, - retry_strategy: "RetryStrategyProto | None" = None, - fetch_batch_size: int = 10, - min_fetch_interval: float = 1.0, - max_fetch_interval: float = 10.0, - lease_ttl_seconds: float = 60.0, - max_deliveries: int | None = None, - ack_policy: AckPolicy | None = None, - # FastStream subscriber-level knobs - dependencies: Iterable["params.Depends"] = (), - parser: CustomCallable | None = None, - decoder: CustomCallable | None = None, - title_: str | None = None, - description_: str | None = None, - include_in_schema: bool = True, - # FastAPI response-model knobs (defaults match ``StreamRouter`` expectations) - response_model: typing.Any = _DEFAULT_RESPONSE_MODEL, - response_model_include: typing.Optional["IncEx"] = None, - response_model_exclude: typing.Optional["IncEx"] = None, - response_model_by_alias: bool = True, - response_model_exclude_unset: bool = False, - response_model_exclude_defaults: bool = False, - response_model_exclude_none: bool = False, - ) -> "OutboxSubscriber": - # ``StreamRouter.subscriber`` uses ``*extra: NameRequired | str`` — our - # ``queues: str | list[str]`` is wider; the actual broker-side - # ``OutboxRegistrator.subscriber`` accepts both. - return typing.cast( - "OutboxSubscriber", - super().subscriber( - queues, # ty: ignore[invalid-argument-type] - max_workers=max_workers, - retry_strategy=retry_strategy, - fetch_batch_size=fetch_batch_size, - min_fetch_interval=min_fetch_interval, - max_fetch_interval=max_fetch_interval, - lease_ttl_seconds=lease_ttl_seconds, - max_deliveries=max_deliveries, - ack_policy=ack_policy, - dependencies=dependencies, - parser=parser, - decoder=decoder, - title_=title_, - description_=description_, - include_in_schema=include_in_schema, - response_model=response_model, - response_model_include=response_model_include, - response_model_exclude=response_model_exclude, - response_model_by_alias=response_model_by_alias, - response_model_exclude_unset=response_model_exclude_unset, - response_model_exclude_defaults=response_model_exclude_defaults, - response_model_exclude_none=response_model_exclude_none, - ), - ) -``` - -- [ ] **Step 2: Drop `middlewares=` from `OutboxRouter.publisher`** - -`old_string`: -```python - def publisher( # ty: ignore[invalid-method-override] - self, - queue: str, - *, - headers: dict[str, str] | None = None, - middlewares: Sequence["PublisherMiddleware[OutboxPublishCommand]"] = (), - title: str | None = None, - description: str | None = None, - schema: typing.Any | None = None, - include_in_schema: bool = True, - ) -> "OutboxPublisher": - # ``StreamRouter.publisher`` forwards directly to ``self.broker.publisher``; - # mirror its delegation so outbox users get the right return type. - return self.broker.publisher( - queue, - headers=headers, - middlewares=middlewares, - title=title, - description=description, - schema=schema, - include_in_schema=include_in_schema, - ) -``` - -`new_string`: -```python - def publisher( # ty: ignore[invalid-method-override] - self, - queue: str, - *, - headers: dict[str, str] | None = None, - title: str | None = None, - description: str | None = None, - schema: typing.Any | None = None, - include_in_schema: bool = True, - ) -> "OutboxPublisher": - # ``StreamRouter.publisher`` forwards directly to ``self.broker.publisher``; - # mirror its delegation so outbox users get the right return type. - return self.broker.publisher( - queue, - headers=headers, - title=title, - description=description, - schema=schema, - include_in_schema=include_in_schema, - ) -``` - -- [ ] **Step 3: Clean up now-unused imports** - -```bash -grep -n "SubscriberMiddleware\|PublisherMiddleware\|OutboxPublishCommand" /Users/kevinsmith/src/pypi/faststream-outbox/faststream_outbox/fastapi/router.py -``` - -Remove from the runtime import line: -- `SubscriberMiddleware` from `from faststream._internal.types import BrokerMiddleware, CustomCallable, SubscriberMiddleware` if no longer referenced. - -Remove from the TYPE_CHECKING block: -- `from faststream._internal.types import PublisherMiddleware` (if unreferenced). -- `from faststream_outbox.response import OutboxPublishCommand` (if unreferenced). - -The `BrokerMiddleware` import STAYS — used by `OutboxRouter.__init__(middlewares=...)` at line 80. - -- [ ] **Step 4: Re-check whether `# noqa: PLR0913` should remain** - -After the drop, count the args on `subscriber()`. If it's still > 15, the `noqa` stays. If ≤ 15, drop the `# noqa: PLR0913` from the method line. - -```bash -uv run python -c " -import inspect -from faststream_outbox.fastapi.router import OutboxRouter -print('subscriber args:', len(inspect.signature(OutboxRouter.subscriber).parameters)) -print('publisher args:', len(inspect.signature(OutboxRouter.publisher).parameters)) -" -``` - -Adjust the `noqa` accordingly. - -- [ ] **Step 5: Verify lint + ty** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run ruff check faststream_outbox/fastapi/router.py && uv run ty check faststream_outbox/fastapi/router.py -``` - -Expected: clean. - -- [ ] **Step 6: Interim commit** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git add faststream_outbox/fastapi/router.py && git commit -m "wip: drop middlewares= from fastapi router subscriber/publisher" -``` - ---- - -## Task 9: Conditionally handle `AckPolicy.ACK_FIRST` - -**Files:** -- Modify (conditional): `faststream_outbox/subscriber/factory.py` (lines 108–116) -- Modify (conditional): the matching test in `tests/test_unit.py` - -This task branches on the `ACK_FIRST_GONE` value from Task 1, Step 4. - -**Branch A — `ACK_FIRST_GONE == False` (the enum member survived):** No change needed. The footgun rejection still applies; the existing test still passes. Skip to Task 10. - -**Branch B — `ACK_FIRST_GONE == True` (the enum member is gone):** - -- [ ] **Step 1: Locate and delete the rejection branch** - -Use Edit on `faststream_outbox/subscriber/factory.py`: - -`old_string`: -```python - is_no_retry = isinstance(retry_strategy, NoRetry) - if ack_policy is AckPolicy.ACK_FIRST: - msg = ( - "ack_policy=AckPolicy.ACK_FIRST is not supported by the outbox broker: it " - "deletes the row before the handler runs, so a handler crash silently drops " - "the message — defeating the outbox reliability guarantee. Use NACK_ON_ERROR " - "(default, retries via retry_strategy), REJECT_ON_ERROR (delete on first " - "failure, no retry), or MANUAL (handler calls msg.ack()/nack()/reject() itself)." - ) - raise ValueError(msg) - if ack_policy is AckPolicy.REJECT_ON_ERROR and retry_strategy is not None and not is_no_retry: -``` - -`new_string`: -```python - is_no_retry = isinstance(retry_strategy, NoRetry) - if ack_policy is AckPolicy.REJECT_ON_ERROR and retry_strategy is not None and not is_no_retry: -``` - -- [ ] **Step 2: Locate and delete the matching test** - -```bash -grep -n "ACK_FIRST" /Users/kevinsmith/src/pypi/faststream-outbox/tests/*.py -``` - -Delete the test that asserts the ACK_FIRST rejection. (Exact name/file depends on the test layout — likely `tests/test_unit.py::test_subscriber_rejects_ack_first` or similar.) - -- [ ] **Step 3: Verify lint + ty** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run ruff check faststream_outbox/subscriber/factory.py && uv run ty check faststream_outbox/subscriber/factory.py -``` - -Expected: clean. - -- [ ] **Step 4: Update `CLAUDE.md`** — the architecture section documents this rejection. Remove the "AckPolicy.ACK_FIRST is rejected at registration with ValueError" passage: - -```bash -grep -n "ACK_FIRST" /Users/kevinsmith/src/pypi/faststream-outbox/CLAUDE.md -``` - -If hits exist, trim them (whole sentence; leave surrounding paragraph coherent). - -- [ ] **Step 5: Interim commit** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git add faststream_outbox/subscriber/factory.py tests/ CLAUDE.md && git commit -m "wip: drop ACK_FIRST rejection (enum member gone in 0.7)" -``` - ---- - -## Task 10: Test cleanup + docs + CLAUDE.md - -**Files:** -- Modify: `tests/test_fake.py` (delete `test_publisher_accepts_middlewares_kwarg` at lines 406–423) -- Modify (conditional): `tests/test_unit.py` (if `_basic_publish` / `_publish` `_extra_middlewares=` kwarg changed in 0.7) -- Modify: `docs/usage/publisher.md` (lines 102–105) -- Modify: `CLAUDE.md` (line 37) - -- [ ] **Step 1: Delete `test_publisher_accepts_middlewares_kwarg`** - -Use Edit on `tests/test_fake.py`: - -`old_string`: -```python -async def test_publisher_accepts_middlewares_kwarg() -> None: - """B3: ``broker.publisher(..., middlewares=...)`` threads the middleware through to publish.""" - broker = _make_broker() - seen: list[str] = [] - - async def record_middleware(call_next: typing.Callable, cmd: typing.Any) -> typing.Any: - seen.append(f"before:{cmd.destination}") - result = await call_next(cmd) - seen.append(f"after:{cmd.destination}") - return result - - publisher = broker.publisher("orders", middlewares=(record_middleware,)) - - test_broker = TestOutboxBroker(broker) - async with test_broker: - await publisher.publish({"x": 1}, session=_fake_session()) - - assert seen == ["before:orders", "after:orders"] - - -async def test_outbox_route_accepts_ack_policy() -> None: -``` - -`new_string`: -```python -async def test_outbox_route_accepts_ack_policy() -> None: -``` - -- [ ] **Step 2: Audit `_extra_middlewares=` usage** - -```bash -grep -n "_extra_middlewares" /Users/kevinsmith/src/pypi/faststream-outbox/faststream_outbox/ /Users/kevinsmith/src/pypi/faststream-outbox/tests/ -r -``` - -Expected hits: -- `faststream_outbox/publisher/usecase.py` line ~99 (`await self._basic_publish(cmd, producer=..., _extra_middlewares=())`) -- `faststream_outbox/publisher/usecase.py` line ~111 (the `_publish` override signature) -- `tests/test_unit.py` line ~925 (the test that calls `_publish(cmd, _extra_middlewares=())`) - -Verify that upstream's `_basic_publish` / `_publish` still accept `_extra_middlewares=`: - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run python -c " -import inspect -from faststream._internal.broker.pub_base import BrokerPublishingMixin -from faststream._internal.endpoint.publisher import PublisherUsecase -print('_basic_publish sig:', inspect.signature(BrokerPublishingMixin._basic_publish)) -print('_publish sig:', inspect.signature(PublisherUsecase._publish)) -" -``` - -- **If `_extra_middlewares` is still accepted:** no change needed. -- **If `_extra_middlewares` was removed in 0.7:** drop those three usages (signature, call site, test line). Replace `_extra_middlewares=()` with whatever 0.7 uses (likely just remove it entirely if middlewares are now exclusively broker-scope). -- **If `_extra_middlewares` was renamed:** rename at the three sites. - -Make whatever edits are needed; if no change is needed, log "no change" and move on. - -- [ ] **Step 3: Trim `docs/usage/publisher.md`** - -`old_string`: -```markdown -Per-call `headers` are merged with the decorator's static headers -(per-call wins). Per-publisher `middlewares=` wrap every -`publisher.publish(...)` call — useful for tracing spans, metrics -counters, or audit-log writes scoped to a single queue. -``` - -`new_string`: -```markdown -Per-call `headers` are merged with the decorator's static headers -(per-call wins). -``` - -- [ ] **Step 4: Trim `CLAUDE.md`** - -Find the paragraph at line ~37 that documents `broker.publisher(..., middlewares=...)`: - -`old_string`: -``` -`broker.publisher(queue, *, headers=None, middlewares=(), title=None, description=None, schema=None, include_in_schema=True)` returns an `OutboxPublisher` — a typed, queue-scoped wrapper around `broker.publish` with the same transactional contract: `await pub.publish(body, *, session, headers=None, correlation_id=None, activate_in=None, activate_at=None, timer_id=None)`. Static headers passed to the decorator are merged with per-call headers (per-call wins). `middlewares=` wrap every `publisher.publish(...)` call. The publisher exists for AsyncAPI spec coverage and per-queue config — **not** for decorator-relay chaining. `OutboxPublisher.__call__` raises `NotImplementedError` at decoration time so `@pub @broker.subscriber(...)` fails fast with a message pointing at the manual `broker.publish(...)` pattern. Rationale: the dispatch loop has no reachable `AsyncSession` without breaking the outbox transactional contract (row commits with caller's domain writes), so a relay decorator would either silently open its own session (defeating the point) or require contextvar plumbing (over-engineered for the use case). -``` - -`new_string`: -``` -`broker.publisher(queue, *, headers=None, title=None, description=None, schema=None, include_in_schema=True)` returns an `OutboxPublisher` — a typed, queue-scoped wrapper around `broker.publish` with the same transactional contract: `await pub.publish(body, *, session, headers=None, correlation_id=None, activate_in=None, activate_at=None, timer_id=None)`. Static headers passed to the decorator are merged with per-call headers (per-call wins). The publisher exists for AsyncAPI spec coverage and per-queue config — **not** for decorator-relay chaining. `OutboxPublisher.__call__` raises `NotImplementedError` at decoration time so `@pub @broker.subscriber(...)` fails fast with a message pointing at the manual `broker.publish(...)` pattern. Rationale: the dispatch loop has no reachable `AsyncSession` without breaking the outbox transactional contract (row commits with caller's domain writes), so a relay decorator would either silently open its own session (defeating the point) or require contextvar plumbing (over-engineered for the use case). -``` - -- [ ] **Step 5: Final sweep — confirm no per-call middlewares= references remain** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && \ - grep -rn "middlewares=" faststream_outbox/ tests/ docs/ README.md CLAUDE.md 2>/dev/null | \ - grep -v "broker_middlewares" | \ - grep -v "BrokerMiddleware" | \ - grep -v "# noqa\|# ty:" -``` - -Expected hits to remain (each is broker-scope and stays): -- `tests/test_middleware_opentelemetry.py:45` (`OutboxBroker(middlewares=[...])`) -- `tests/test_middleware_prometheus.py:37,206` (`OutboxBroker(middlewares=[...])`) -- `docs/usage/observability.md:266` (`OutboxBroker(middlewares=[...])`) -- `faststream_outbox/fastapi/router.py:127` (`super().__init__(...middlewares=middlewares...)`) -- `faststream_outbox/router.py:92` (`broker_middlewares=middlewares`) - -If anything else turns up, audit it — it's likely a missed cleanup. - -- [ ] **Step 6: Interim commit** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git add tests/test_fake.py docs/usage/publisher.md CLAUDE.md && git commit -m "wip: drop test + docs for removed per-call middlewares=" -``` - -If Step 2 made code edits, include those files in the `git add`. - ---- - -## Task 11: Full test pass + squash + push - -**Files:** none new — verification + final commit shape. - -- [ ] **Step 1: Full lint** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && just lint-ci -``` - -Expected: all checks pass. If anything fails, fix in-place and re-run before proceeding. - -- [ ] **Step 2: No-Postgres test tier** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && uv run pytest tests/test_unit.py tests/test_fake.py --no-cov -v 2>&1 | tail -40 -``` - -Expected: all tests pass. Failures here are easier to debug locally than the full docker run. - -- [ ] **Step 3: Full suite with coverage gate** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && just test -``` - -Expected: full suite green at `--cov-fail-under=100`. If coverage drops below 100, look for orphaned branches (most likely candidates: the removed `middlewares=` branches in registrator/factory/config). The fix is usually one of: - 1. Branch is fully dead → delete it. - 2. Branch survives but the only test exercising it was deleted → re-add a focused test or restructure. - -Iterate until green. - -- [ ] **Step 4: Final repo-wide sanity checks** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && \ - echo '--- pin ---' && \ - grep -n 'faststream' pyproject.toml | grep -v 'faststream-outbox' && \ - echo '--- stale pin refs ---' && \ - (grep -rn 'faststream<0.7\|faststream>=0.6' . --include='*.py' --include='*.toml' --include='*.md' || echo 'none') && \ - echo '--- import sanity ---' && \ - uv run python -c "from faststream_outbox import OutboxBroker; from faststream_outbox.fastapi import OutboxRouter; print('OK')" -``` - -Expected: -- pin shows `"faststream>=0.7,<0.8"` -- stale pin refs prints `none` -- import sanity prints `OK` - -- [ ] **Step 5: Inspect the WIP commit list** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git log --oneline main..HEAD -``` - -Expected: a handful of `wip: ...` commits from Tasks 1–10. - -- [ ] **Step 6: Squash to a single bundled commit** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git reset --soft $(git merge-base main HEAD) && git status -``` - -Expected: all the touched files staged, nothing committed yet on the branch. - -- [ ] **Step 7: Create the single bundled commit** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git commit -m "$(cat <<'EOF' -chore: migrate to faststream 0.7 - -Pure compat migration; no new 0.7 features adopted. - -Internal break points fixed: -- pyproject.toml: pin bumped to faststream>=0.7,<0.8. -- publisher/producer.py: OutboxProducer gained a `codec` attribute to - satisfy ProducerProto[0.7]. The outbox owns its own encoding pipeline - via _encode_payload and ignores the attribute at runtime. -- testing.py: FakeOutboxProducer gained the same `codec` attribute; - create_publisher_fake_subscriber converted from @staticmethod to - instance method to match upstream's abstract base. -- registrator.py: dropped `middlewares_=` from the SubscriberUsecase - .add_call call (kwarg removed upstream in 0.7). - -Public surface (BREAKING): -- Dropped the per-call `middlewares=` kwarg from - OutboxRegistrator.subscriber, OutboxRegistrator.publisher, OutboxRoute, - and the FastAPI router's subscriber/publisher overrides. Upstream - removed publisher- and subscriber-level middlewares as a public - surface in 0.7; install middleware at the broker scope instead via - `OutboxBroker(middlewares=[...])`. Broker-scope middlewares on - OutboxBroker, OutboxRouter, and the FastAPI router are unchanged. - -Spec: planning/specs/2026-06-03-faststream-0.7-migration-design.md. - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - -- [ ] **Step 8: Confirm the commit** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git log -1 --stat -``` - -Expected: one new commit on the branch, with all the modified files listed. - -- [ ] **Step 9: Final test rerun on the squashed branch** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && just lint-ci && just test -``` - -Expected: green on both. (Sanity check that the squash didn't drop anything — `git reset --soft` preserves the working tree, so this should be identical to Step 3, but worth re-running on a paranoid basis.) - -- [ ] **Step 10: Push the branch + open the PR** - -```bash -cd /Users/kevinsmith/src/pypi/faststream-outbox && git push -u origin chore/faststream-0.7-migration -``` - -Then open the PR using `gh pr create` — refer to the standard PR-creation flow in `CLAUDE.md` (the operator will run this; do not auto-open without explicit user instruction). - ---- - -## Acceptance criteria (spec §"Verification") - -After Task 11, all of these must be true: - -- [ ] `uv lock --upgrade` resolved `faststream` to a `0.7.x` release (Task 1, Step 3 output). -- [ ] `uv sync --all-extras --all-groups --frozen` succeeds (Task 11, equivalent via `just install` — run manually if needed). -- [ ] `just lint-ci` clean (Task 11, Step 1). -- [ ] `just test` green at `--cov-fail-under=100` (Task 11, Step 3). -- [ ] `uv run pytest tests/test_unit.py tests/test_fake.py` green standalone (Task 11, Step 2). -- [ ] `git grep -n "middlewares_=\|middlewares=" faststream_outbox/ tests/ docs/` returns only `broker_middlewares=` / `BrokerMiddleware` references (Task 10, Step 5). -- [ ] `git grep -n "faststream<0.7\|faststream>=0.6" .` returns nothing (Task 11, Step 4). -- [ ] Manual import sanity passes (Task 11, Step 4). -- [ ] The branch has exactly one commit beyond `main` (Task 11, Step 8) — `git log --oneline main..HEAD | wc -l` returns `1`. -- [ ] The commit message documents each break point + the dropped kwarg (Task 11, Step 7). diff --git a/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/design.md b/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing.md similarity index 100% rename from planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/design.md rename to planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing.md diff --git a/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/plan.md b/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/plan.md deleted file mode 100644 index a4622a0..0000000 --- a/planning/changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/plan.md +++ /dev/null @@ -1,256 +0,0 @@ -# FastStream 0.7.1 TestBroker typing alignment — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Adopt FastStream 0.7.1's `TestBroker[Broker, EnterType]` typing fix here by binding `EnterType = OutboxBroker` and removing the two `# ty: ignore` directives that worked around the upstream `Broker | list[Broker]` return-type bug. - -**Architecture:** Bump the FastStream pin to `>=0.7.1`, switch `TestOutboxBroker`'s generic to `TestBroker[OutboxBroker, OutboxBroker]`, update the ASGI registry hook return annotation to the new two-param shape, delete the two suppressions that the upstream fix obsoletes, and add a regression test in `tests/test_fake.py` that proves the entered context yields a single `OutboxBroker` (not a list/tuple). - -**Tech Stack:** Python 3.13, FastStream 0.7.1, `uv` for deps, `ruff` for lint, `ty` for type check, `pytest` (under docker compose for the Postgres-backed suite). - -**Spec:** [`planning/specs/2026-06-04-faststream-0.7.1-testbroker-typing-design.md`](../specs/2026-06-04-faststream-0.7.1-testbroker-typing-design.md) - -**Commit strategy:** Single bundled commit at the end of Task 5. Tasks 1–4 stage incrementally without committing, so intermediate `ty check` runs reflect work-in-progress and the final commit captures one logical change. - -**Branch:** `chore/faststream-0.7.1-testbroker-typing` (matches the sibling project's naming). - ---- - -### Task 1: Bump the FastStream pin to >=0.7.1 - -**Files:** -- Modify: `pyproject.toml:12` - -- [ ] **Step 1: Create the feature branch from main** - -Run: `git switch -c chore/faststream-0.7.1-testbroker-typing` -Expected: switched to a new branch off `main`. - -- [ ] **Step 2: Edit the dependency line** - -In `pyproject.toml`, change: - -```toml -"faststream>=0.7,<0.8", -``` - -to: - -```toml -"faststream>=0.7.1,<0.8", -``` - -- [ ] **Step 3: Refresh the lockfile and sync** - -Run: `just install` - -This runs `uv lock --upgrade` followed by `uv sync --all-extras --all-groups --frozen`. Expected: `uv` resolves `faststream==0.7.1` (the only version satisfying both `>=0.7.1` and `<0.8` at time of writing). The sync should report no errors. - -- [ ] **Step 4: Confirm the resolved version** - -Run: `uv pip show faststream | grep -i version` -Expected output: `Version: 0.7.1` - -- [ ] **Step 5: Sanity-run the no-Postgres tests against the upgraded library** - -Run: `uv run pytest tests/test_unit.py tests/test_fake.py -v --no-cov` -Expected: all tests pass. (The two `ty: ignore` directives are still in place and still satisfy the type checker under 0.7.1, because `EnterType` defaults to `Any`. If anything fails here, **stop** — the upgrade has surfaced an unrelated regression that needs investigation before refactoring.) `--no-cov` is required because partial runs would otherwise trip the `--cov-fail-under=100` ratchet from `pyproject.toml`'s `addopts`. - -- [ ] **Step 6: Stage the change (do not commit yet)** - -Run: `git add pyproject.toml uv.lock` - -(Verify `uv.lock` is tracked first with `git status`; if it isn't, drop it from the `git add`.) - ---- - -### Task 2: Add the regression test for `__aenter__` return shape - -**Files:** -- Modify: `tests/test_fake.py` (append new test at the end of the file) - -This test is added *before* the refactor so we can prove the contract (single `OutboxBroker` returned from the context, never a list or tuple) survives the suppression removal. Under the current code (where `TestBroker[OutboxBroker]`'s `__aenter__` returns `OutboxBroker | list[OutboxBroker]` and we suppress with `ty: ignore`) the test should pass on the first run because the runtime `__aenter__` already returns the single broker — the bug was purely in the type annotation. - -- [ ] **Step 1: Append the new test to `tests/test_fake.py`** - -Add this test as the last function in the file (after `test_fake_dlq_not_emitted_on_handler_success`): - -```python -async def test_test_broker_aenter_returns_single_outbox_broker() -> None: - """0.7.1's EnterType binding means TestOutboxBroker yields a single OutboxBroker, not a list/tuple. - - Guards the contract through the upstream typing refactor: even if the base - class signature changes again, our single-broker subclass must always hand - back a single broker instance. - """ - broker = _make_broker() - async with TestOutboxBroker(broker) as br: - assert isinstance(br, OutboxBroker) -``` - -No new imports needed — `TestOutboxBroker`, `OutboxBroker`, and the module-local `_make_broker()` helper are already in scope in `tests/test_fake.py`. - -- [ ] **Step 2: Run the new test to confirm it passes against the current code** - -Run: `uv run pytest tests/test_fake.py::test_test_broker_aenter_returns_single_outbox_broker -v --no-cov` -Expected: PASS. (The runtime `__aenter__` already returns the single `OutboxBroker`; we're locking that contract in.) - -- [ ] **Step 3: Stage the change** - -Run: `git add tests/test_fake.py` - ---- - -### Task 3: Refactor `TestOutboxBroker` — bind `EnterType`, drop the type-arguments suppression - -**Files:** -- Modify: `faststream_outbox/testing.py:521` - -- [ ] **Step 1: Replace the class declaration** - -In `faststream_outbox/testing.py`, locate line 521: - -```python -class TestOutboxBroker(TestBroker[OutboxBroker]): # ty: ignore[invalid-type-arguments] -``` - -Replace with: - -```python -class TestOutboxBroker(TestBroker[OutboxBroker, OutboxBroker]): -``` - -The base `__aenter__` now returns `EnterType`, which we bind to `OutboxBroker`. The `invalid-type-arguments` suppression that worked around the 0.7.0 single-param shape is no longer needed. - -**Do not touch** the `# ty: ignore[invalid-argument-type]` on `patch_broker_calls(broker)` later in the same file (around line 626) — that's unrelated (config-generic invariance on `BrokerUsecase`) and is documented in `CLAUDE.md`'s ignore table. - -- [ ] **Step 2: Confirm `ty` is satisfied with the new annotation** - -Run: `uv run ty check faststream_outbox/testing.py` -Expected: no errors. (If `ty` flags a *different* diagnostic on the class declaration, **stop** — the upstream 0.7.1 fix isn't behaving as documented in our environment; investigate before adding a replacement suppression.) - -- [ ] **Step 3: Re-run the regression test from Task 2** - -Run: `uv run pytest tests/test_fake.py::test_test_broker_aenter_returns_single_outbox_broker -v --no-cov` -Expected: PASS. (The result now comes from the base class via `EnterType = OutboxBroker` instead of falling through the union.) - -- [ ] **Step 4: Re-run the full `test_fake.py` suite to confirm no regression** - -Run: `uv run pytest tests/test_fake.py -v --no-cov` -Expected: every test passes. Every test in this file uses `async with TestOutboxBroker(broker)`; if `EnterType` were wired wrong, the subsequent `await broker.publish(...)` calls would still work at runtime (the override never affected runtime), but any `ty` mismatch would surface here when the next steps run `just lint`. - -- [ ] **Step 5: Stage the change** - -Run: `git add faststream_outbox/testing.py` - ---- - -### Task 4: Update the ASGI registry hook annotation — drop the return-type suppression - -**Files:** -- Modify: `faststream_outbox/__init__.py:45-47` - -- [ ] **Step 1: Update the return type annotation and drop the inline ignore** - -In `faststream_outbox/__init__.py`, locate lines 45–47: - -```python - @functools.lru_cache(maxsize=1) - def get_broker_registry() -> dict[type[BrokerUsecase[typing.Any, typing.Any]], type[TestBroker[typing.Any]]]: - return {**original_get_broker_registry(), OutboxBroker: TestOutboxBroker} # ty: ignore[invalid-return-type] -``` - -Replace with: - -```python - @functools.lru_cache(maxsize=1) - def get_broker_registry() -> dict[ - type[BrokerUsecase[typing.Any, typing.Any]], - type[TestBroker[typing.Any, typing.Any]], - ]: - return {**original_get_broker_registry(), OutboxBroker: TestOutboxBroker} -``` - -This matches the new shape of `faststream.asgi.factories.asyncapi.try_it_out._get_broker_registry`, which 0.7.1 typed as `dict[..., type[TestBroker[Any, Any]]]`. With `TestOutboxBroker` now declared `TestBroker[OutboxBroker, OutboxBroker]` (from Task 3), the dict value type is structurally assignable and the `invalid-return-type` suppression comes off cleanly. - -- [ ] **Step 2: Confirm `ty` is satisfied with the new annotation** - -Run: `uv run ty check faststream_outbox/__init__.py` -Expected: no errors. (Same stop-and-investigate rule as Task 3 Step 2 if `ty` flags a different diagnostic.) - -- [ ] **Step 3: Stage the change** - -Run: `git add faststream_outbox/__init__.py` - ---- - -### Task 5: Final validation and bundled commit - -**Files:** -- All four changes above are now staged together. - -- [ ] **Step 1: Lint the staged changes** - -Run: `just lint` -Expected: `eof-fixer`, `ruff format`, `ruff check --fix`, and `ty check` all pass. If `ruff format` or `ruff check --fix` modifies any staged file, re-stage with `git add ` and re-run `just lint` until clean. - -- [ ] **Step 2: Run the full test suite under docker compose** - -Run: `just test` -Expected: every test in `tests/test_unit.py`, `tests/test_fake.py`, and `tests/test_integration.py` passes, and the `--cov-fail-under=100` ratchet is satisfied. The new regression test from Task 2 should appear in the output and pass. - -- [ ] **Step 3: Review staged diff one more time** - -Run: `git diff --staged` -Expected: changes only in `pyproject.toml`, `uv.lock` (if tracked), `tests/test_fake.py`, `faststream_outbox/testing.py`, and `faststream_outbox/__init__.py`. No drive-by edits. - -- [ ] **Step 4: Commit** - -Run: - -```bash -git commit -m "$(cat <<'EOF' -chore: adopt faststream 0.7.1 TestBroker typing fix - -ag2ai/faststream#2903 makes TestBroker generic over a second EnterType -TypeVar (default Any) and threads it through __aenter__. Bind -EnterType = OutboxBroker in our TestOutboxBroker and drop the -# ty: ignore[invalid-type-arguments] on the class declaration plus the -# ty: ignore[invalid-return-type] on get_broker_registry's return that -worked around the old Broker | list[Broker] return type. Update the -ASGI registry annotation to the new two-param shape and bump the -faststream floor to >=0.7.1. - -Adds a regression test guarding the single-broker contract through -future upstream changes. - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" - -``` - -- [ ] **Step 5: Verify clean tree** - -Run: `git status` -Expected: working tree clean; one new commit on `chore/faststream-0.7.1-testbroker-typing`. - ---- - -## Validation summary (post-implementation) - -After Task 5, the following should hold: - -- `pyproject.toml` requires `faststream>=0.7.1,<0.8`. -- `TestOutboxBroker` declares `TestBroker[OutboxBroker, OutboxBroker]` with **no** `# ty: ignore` on the class declaration. -- `get_broker_registry` returns `dict[..., type[TestBroker[typing.Any, typing.Any]]]` with **no** `# ty: ignore` on the return statement. -- The `# ty: ignore[invalid-argument-type]` on `patch_broker_calls(broker)` in `_fake_start` is **untouched** (documented in `CLAUDE.md` as a separate concern). -- `tests/test_fake.py::test_test_broker_aenter_returns_single_outbox_broker` is present and passing. -- `just lint` and `just test` both succeed. - -## Risks & mitigations - -- **Upstream AST helper depth.** `TestOutboxBroker.__init__` adds an extra frame on top of `TestBroker.__init__`. The 0.7.1 PR adds a `while … name == "__init__"` walk in `_internal/testing/ast.py` exactly for this, so the `async with` AST analysis still finds the user frame. Validated implicitly by Task 3 Step 4 (the full `test_fake.py` suite, which depends on this mechanism). No code change required on our side. -- **`uv lock --upgrade` pulling in unrelated upgrades.** `just install` runs `uv lock --upgrade` which refreshes *all* dependencies. If this surfaces incidental breakage in Task 1 Step 3, narrow the upgrade to `uv lock --upgrade-package faststream` and re-run `uv sync --all-extras --all-groups --frozen` to avoid pulling in unrelated changes. -- **`just test` requires Docker.** If Docker isn't running locally, `just test` will fail at the compose step. Start Docker before Task 5 Step 2. The no-Docker subset (`tests/test_unit.py`, `tests/test_fake.py`) already ran in Task 1 Step 5 and Task 3 Step 4; Task 5 Step 2 is what actually exercises `tests/test_integration.py` against real Postgres and satisfies the 100% coverage ratchet. -- **`ty` still flagging diagnostics after the suppressions come off.** If Task 3 Step 2 or Task 4 Step 2 surfaces a *different* `ty` diagnostic, the playbook is to investigate (not re-add the original ignore). The original suppressions targeted bugs that 0.7.1 fixes; a new diagnostic would mean either the upstream fix isn't behaving as documented in our environment, or our annotation has a separate issue that deserves its own justification entry in `CLAUDE.md`'s ignore table. diff --git a/planning/changes/2026-06-04.02-foreign-broker-relay/design.md b/planning/changes/2026-06-04.02-foreign-broker-relay.md similarity index 100% rename from planning/changes/2026-06-04.02-foreign-broker-relay/design.md rename to planning/changes/2026-06-04.02-foreign-broker-relay.md diff --git a/planning/changes/2026-06-04.02-foreign-broker-relay/plan.md b/planning/changes/2026-06-04.02-foreign-broker-relay/plan.md deleted file mode 100644 index a50fcd1..0000000 --- a/planning/changes/2026-06-04.02-foreign-broker-relay/plan.md +++ /dev/null @@ -1,1502 +0,0 @@ -# Foreign-broker relay — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Officially support the FastStream-native decorator-relay pattern (`@kafka_pub @broker_outbox.subscriber(...)`) so an `OutboxSubscriber` can serve as the source of an outbox-to-foreign-broker relay, with three small guardrails (block `OutboxResponse` + foreign-publisher dual-fire, WARN on unstarted foreign brokers, opt-in inbound-header propagation) and a docs push that promotes the relay tutorial to the top of the navigation. - -**Architecture:** Zero changes to the dispatch path — the FastStream chain `chain(self.__get_response_publisher(message), h.handler._publishers)` already runs foreign publishers, `OutboxFakePublisher` already self-gates on `OutboxPublishCommand`, and `AcknowledgementMiddleware.__aexit__` already turns publisher-chain exceptions into outbox nacks. Guardrails go in: G1 (override `OutboxSubscriber.process_message` to refuse the `OutboxResponse` + foreign-publisher combo), G2 (walk subscribers in `OutboxBroker.start()` and warn for unstarted foreign brokers), G3 (a `propagate_inbound_headers: bool = False` kwarg plumbed through config → factory → registrator → router → fastapi router, applied in the same `process_message` override). Docs: a new `docs/usage/relay.md`, nav promotion in `mkdocs.yml`, intro callout, `CLAUDE.md` subsection, and a README front-page rewrite. - -**Tech Stack:** Python 3.13+, FastStream 0.7.1, SQLAlchemy 2.x (async), `uv` for deps, `ruff` for lint, `ty` for type check, `pytest` (under docker compose for the Postgres-backed suite). New dev dependency: `faststream[kafka]` for `TestKafkaBroker` in the relay tests. - -**Spec:** [`planning/specs/2026-06-04-foreign-broker-relay-design.md`](../specs/2026-06-04-foreign-broker-relay-design.md) - -**Commit strategy:** Per-task commits. Each task ends with `git add` of named files only (never `git add -A`) and a `git commit` invocation. Tasks 8–10 (docs) can be folded into one commit if Task 8 completes cleanly; otherwise commit per docs task. The final Task 11 produces no commit — it only runs full lint + test suite and reports. - -**Branch:** `feat/foreign-broker-relay`. - ---- - -### Task 1: Branch and add `faststream[kafka]` dev dependency - -**Files:** -- Modify: `pyproject.toml` (dev dependency group) - -The `TestKafkaBroker` driver used by Tasks 2–5 lives behind FastStream's `kafka` extra. Without it, the relay tests `ImportError` on first run. We add it to the `dev` dependency group (not `[project.optional-dependencies]`, which is for runtime extras) so that user installs of `faststream-outbox` do not pick up Kafka transport unless they explicitly depend on it. - -- [ ] **Step 1: Create the feature branch from `main`** - -Run: `git switch -c feat/foreign-broker-relay` -Expected: `Switched to a new branch 'feat/foreign-broker-relay'`. - -- [ ] **Step 2: Edit `pyproject.toml`** - -Find the `[dependency-groups]` block (around line 24). Change the `dev` list from: - -```toml -dev = [ - "pytest", - "pytest-asyncio", - "pytest-cov", - "asyncpg>=0.29", - "alembic>=1.13", - "fastapi>=0.95", - "httpx2>=2.2", - "prometheus-client>=0.19", - "opentelemetry-api>=1.20", - "opentelemetry-sdk>=1.20", -] -``` - -to: - -```toml -dev = [ - "pytest", - "pytest-asyncio", - "pytest-cov", - "asyncpg>=0.29", - "alembic>=1.13", - "fastapi>=0.95", - "faststream[kafka]>=0.7.1,<0.8", - "httpx2>=2.2", - "prometheus-client>=0.19", - "opentelemetry-api>=1.20", - "opentelemetry-sdk>=1.20", -] -``` - -The version pin matches the runtime `faststream` pin on line 12 so resolver does not pull a different minor. - -- [ ] **Step 3: Refresh the lockfile** - -Run: `just install` -Expected: `uv` resolves `aiokafka` and supporting Kafka deps. Sync reports no errors. - -- [ ] **Step 4: Confirm Kafka imports work** - -Run: `uv run python -c "from faststream.kafka import KafkaBroker, KafkaRouter, TestKafkaBroker; print('ok')"` -Expected output: `ok` - -- [ ] **Step 5: Commit** - -```bash -git add pyproject.toml uv.lock -git commit -m "chore: add faststream[kafka] to dev deps for relay tests - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -### Task 2: Baseline relay test — naked decorator chain, default behavior - -**Files:** -- Create: `tests/test_relay.py` - -This test pins the FastStream-native cross-broker chain against `OutboxSubscriber` without any guardrail code (G1/G2/G3 not yet implemented). If it fails on first run, the spec's "How the wiring works today" analysis is wrong and the work stops here for re-investigation. - -We use `TestOutboxBroker(..., run_loops=False)` so each `await outbox.publish(...)` synchronously drives the subscriber's `dispatch_one` (the documented test idiom; see `CLAUDE.md` "Test broker" section). The destination `TestKafkaBroker` records published commands on its in-memory `_producer.mock`, which we assert against. - -- [ ] **Step 1: Create the test file** - -Create `tests/test_relay.py` with the following exact contents: - -```python -from typing import Any - -import pytest -from faststream.kafka import KafkaBroker, TestKafkaBroker - -from faststream_outbox import OutboxBroker -from faststream_outbox.testing import TestOutboxBroker - - -pytestmark = pytest.mark.asyncio - - -async def test_naked_decorator_chain_relays_plain_return_to_kafka() -> None: - """A handler decorated `@kafka_pub @outbox.subscriber(...)` returning a plain - value publishes the value through the Kafka publisher chain.""" - broker_outbox = OutboxBroker() - broker_kafka = KafkaBroker("kafka://test:9092") - publisher_kafka = broker_kafka.publisher("relay_topic") - - @publisher_kafka - @broker_outbox.subscriber("relay_queue") - async def relay(body: dict[str, Any]) -> dict[str, Any]: - return body - - # TestKafkaBroker first, TestOutboxBroker second — see "Context-manager - # ordering note" at the end of this plan. The foreign broker's mock producer - # must be wired before our subscriber starts (Task 6 introduces a startup - # warning that probes foreign producers). - async with TestKafkaBroker(broker_kafka), TestOutboxBroker(broker_outbox, run_loops=False) as outbox: - await outbox.publish({"hello": "world"}, queue="relay_queue", session=None) - publisher_kafka.mock.assert_called_once_with({"hello": "world"}) -``` - -Note: `session=None` is fine in `TestOutboxBroker` — the fake client ignores the session argument (see CLAUDE.md "Test broker" section). - -- [ ] **Step 2: Run the test** - -Run: `uv run pytest tests/test_relay.py::test_naked_decorator_chain_relays_plain_return_to_kafka -v --no-cov` -Expected: PASS. `--no-cov` is required because a single-file run would trip `--cov-fail-under=100` from `pyproject.toml`'s addopts. - -If this fails: -- `AttributeError: 'NoneType' object has no attribute 'publish'`: the Kafka broker did not start under `TestKafkaBroker` — re-check the context-manager order (Kafka must be inside the `async with` chain). -- `AssertionError: Expected 'mock' to have been called once. Called 0 times.`: the publisher chain did not fire. Re-read the spec's "How the wiring works today" — the OutboxParser must be setting `reply_to=msg.queue` (it does, line 23 of `parser.py`), and `process_message`'s `chain(...)` loop must be iterating `handler._publishers`. **Stop here and re-investigate.** - -- [ ] **Step 3: Commit** - -```bash -git add tests/test_relay.py -git commit -m "test: baseline relay from OutboxSubscriber to TestKafkaBroker - -Pins the FastStream-native cross-broker chain against OutboxSubscriber -with no guardrail code in place: a plain handler return relays to Kafka -via the @publisher decorator stack. - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -### Task 3: Router-shape relay tests - -**Files:** -- Modify: `tests/test_relay.py` (append) - -Adds two tests covering the router-publisher shapes from spec section "Router publishers as relay decorators": (a) Kafka publisher from a `KafkaRouter` included into the broker; (b) outbox subscriber on an `OutboxRouter` included into `broker_outbox`. Both have to happen *before* the broker starts. We exercise the `include_router` ordering inside each test. - -- [ ] **Step 1: Append the two new tests to `tests/test_relay.py`** - -Add the following at the bottom of the existing file (after the function added in Task 2). Also add the new imports at the top: - -```python -from faststream.kafka import KafkaRouter # add to existing kafka import line -from faststream_outbox import OutboxRouter -from faststream_outbox.router import OutboxRoute -``` - -(If `OutboxRouter` is not in `faststream_outbox.__init__`, import from `faststream_outbox.router import OutboxRouter` instead. Use whichever path resolves — check `faststream_outbox/__init__.py` once.) - -Then add the two functions: - -```python -async def test_relay_via_kafka_router_publisher() -> None: - """A publisher obtained from a KafkaRouter (then include_router'd into the - broker) relays the same way a broker-direct publisher does.""" - broker_outbox = OutboxBroker() - broker_kafka = KafkaBroker("kafka://test:9092") - kafka_router = KafkaRouter() - publisher_kafka = kafka_router.publisher("relay_topic") - - @publisher_kafka - @broker_outbox.subscriber("relay_queue") - async def relay(body: dict[str, Any]) -> dict[str, Any]: - return body - - broker_kafka.include_router(kafka_router) - - async with TestKafkaBroker(broker_kafka), TestOutboxBroker(broker_outbox, run_loops=False) as outbox: - await outbox.publish({"router": True}, queue="relay_queue", session=None) - publisher_kafka.mock.assert_called_once_with({"router": True}) - - -async def test_relay_via_outbox_router_subscriber() -> None: - """A subscriber registered on an OutboxRouter (then include_router'd into - broker_outbox) accepts a foreign-publisher decorator the same way a - broker-direct subscriber does.""" - broker_outbox = OutboxBroker() - broker_kafka = KafkaBroker("kafka://test:9092") - publisher_kafka = broker_kafka.publisher("relay_topic") - outbox_router = OutboxRouter() - - @publisher_kafka - @outbox_router.subscriber("relay_queue") - async def relay(body: dict[str, Any]) -> dict[str, Any]: - return body - - broker_outbox.include_router(outbox_router) - - async with TestKafkaBroker(broker_kafka), TestOutboxBroker(broker_outbox, run_loops=False) as outbox: - await outbox.publish({"outbox_router": True}, queue="relay_queue", session=None) - publisher_kafka.mock.assert_called_once_with({"outbox_router": True}) -``` - -- [ ] **Step 2: Run the two new tests** - -Run: -```bash -uv run pytest tests/test_relay.py::test_relay_via_kafka_router_publisher tests/test_relay.py::test_relay_via_outbox_router_subscriber -v --no-cov -``` -Expected: both PASS. - -If `test_relay_via_outbox_router_subscriber` fails with an `OutboxRouter` import error, look at `faststream_outbox/__init__.py` for the public name — the import-line guidance in Step 1 covers both possibilities. Use the correct import path. - -- [ ] **Step 3: Commit** - -```bash -git add tests/test_relay.py -git commit -m "test: relay via KafkaRouter and OutboxRouter publishers/subscribers - -Confirms include_router-before-start() resolves the router-publisher's -ConfigComposition to the real producer (Kafka side) and that -broker_outbox.include_router wires foreign-decorated subscribers from -an OutboxRouter the same as broker-direct ones. - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -### Task 4: G3 — `propagate_inbound_headers` kwarg - -**Files:** -- Modify: `faststream_outbox/subscriber/config.py` -- Modify: `faststream_outbox/subscriber/factory.py` -- Modify: `faststream_outbox/registrator.py` -- Modify: `faststream_outbox/router.py` (the `OutboxRoute.__init__` signature) -- Modify: `faststream_outbox/fastapi/router.py` (the `subscriber()` override) -- Modify: `faststream_outbox/subscriber/usecase.py` (introduce a `process_message` override that applies the toggle — this override also hosts G1 in Task 5) -- Modify: `tests/test_relay.py` (extend Task 2's test, add a new one) - -We add the kwarg with default `False` (matches FastStream-wide convention and `faststream-sqlbroker`). The dispatch-time application lives inside an override of `SubscriberUsecase.process_message` — that override also hosts G1 in Task 5, so design the override to be reusable for both. - -- [ ] **Step 1: Write the failing test for the ON behavior** - -Append the following function to `tests/test_relay.py`: - -```python -async def test_propagate_inbound_headers_true_forwards_outbox_headers_to_kafka() -> None: - """With propagate_inbound_headers=True, the inbound outbox row's headers - are forwarded onto the Response before the foreign-publisher chain fires.""" - broker_outbox = OutboxBroker() - broker_kafka = KafkaBroker("kafka://test:9092") - publisher_kafka = broker_kafka.publisher("relay_topic") - - @publisher_kafka - @broker_outbox.subscriber("relay_queue", propagate_inbound_headers=True) - async def relay(body: dict[str, Any]) -> dict[str, Any]: - return body - - async with TestKafkaBroker(broker_kafka), TestOutboxBroker(broker_outbox, run_loops=False) as outbox: - await outbox.publish( - {"hi": 1}, - queue="relay_queue", - session=None, - headers={"x-trace-id": "abc123", "content-type": "application/json"}, - ) - publisher_kafka.mock.assert_called_once() - # The OutboxFakePublisher and KafkaPublisher both ran; we assert on the - # Kafka mock's call arg. The exact spy shape depends on FastStream's - # TestKafkaBroker; we read it off the mock's call_args. - call_args = publisher_kafka.mock.call_args - assert call_args is not None - # FastStream's TestKafkaBroker spy records (body,) positional; headers on the - # PublishCommand are attached separately. Inspect what TestKafkaBroker exposes - # at runtime: typically `publisher_kafka.mock.call_args.args[0]` is the body, - # and the Kafka publisher's recorded outbound headers are on the captured - # publish_command. The reference test below uses the body-only shape since - # native KafkaBroker tests use the same pattern. - assert call_args.args[0] == {"hi": 1} -``` - -Also update the existing Task-2 test to assert on the **default-off** header behavior explicitly. Replace the bottom of `test_naked_decorator_chain_relays_plain_return_to_kafka` (the lines after `publisher_kafka.mock.assert_called_once_with({"hello": "world"})`) with: - -```python - # Default behavior: propagate_inbound_headers is False, so inbound - # outbox row headers do NOT reach the Kafka publish. We assert this - # by emitting a row with headers and confirming Kafka's mock recorded - # the body only (headers default-empty on Response(value)). - publisher_kafka.mock.reset_mock() - await outbox.publish( - {"second": True}, - queue="relay_queue", - session=None, - headers={"x-trace-id": "ignored"}, - ) - publisher_kafka.mock.assert_called_once_with({"second": True}) -``` - -(Drop the `_ = kafka` line — no longer needed.) - -- [ ] **Step 2: Run the new test and confirm it fails** - -Run: `uv run pytest tests/test_relay.py::test_propagate_inbound_headers_true_forwards_outbox_headers_to_kafka -v --no-cov` -Expected: FAIL with `TypeError: subscriber() got an unexpected keyword argument 'propagate_inbound_headers'`. This is the signal the kwarg plumbing is not yet wired — Steps 3–8 add it. - -- [ ] **Step 3: Add the config field** - -Edit `faststream_outbox/subscriber/config.py`. Add the field after `max_deliveries`: - -```python -@dataclass(kw_only=True) -class OutboxSubscriberConfig(SubscriberUsecaseConfig): - _outer_config: "OutboxBrokerConfig" - queues: list[str] - max_workers: int - retry_strategy: "RetryStrategyProto | None" - fetch_batch_size: int - min_fetch_interval: float - max_fetch_interval: float - lease_ttl_seconds: float - max_deliveries: int | None - propagate_inbound_headers: bool -``` - -The new field has no default so callers must pass it explicitly — Step 4 updates the factory accordingly. - -- [ ] **Step 4: Plumb through the factory** - -Edit `faststream_outbox/subscriber/factory.py`. Add the parameter to `create_subscriber` (keyword-only, default `False`) and pass it into the `OutboxSubscriberConfig(...)` call: - -```python -def create_subscriber( - *, - queues: list[str], - max_workers: int, - retry_strategy: "RetryStrategyProto | None", - fetch_batch_size: int, - min_fetch_interval: float, - max_fetch_interval: float, - lease_ttl_seconds: float, - max_deliveries: int | None, - config: "OutboxBrokerConfig", - ack_policy: AckPolicy | None = None, - propagate_inbound_headers: bool = False, - title_: str | None = None, - description_: str | None = None, - include_in_schema: bool = True, -) -> OutboxSubscriber: -``` - -And in the `OutboxSubscriberConfig(...)` block: - -```python - usecase_config = OutboxSubscriberConfig( - _outer_config=config, - _ack_policy=ack_policy if ack_policy is not None else EMPTY, - queues=queues, - max_workers=max_workers, - retry_strategy=retry_strategy, - fetch_batch_size=fetch_batch_size, - min_fetch_interval=min_fetch_interval, - max_fetch_interval=max_fetch_interval, - lease_ttl_seconds=lease_ttl_seconds, - max_deliveries=max_deliveries, - propagate_inbound_headers=propagate_inbound_headers, - ) -``` - -- [ ] **Step 5: Plumb through the registrator** - -Edit `faststream_outbox/registrator.py`. In `OutboxRegistrator.subscriber(...)`, add the kwarg (default `False`) after `ack_policy` and pass it through to `create_subscriber`: - -```python - def subscriber( # ty: ignore[invalid-method-override] - self, - queues: str | list[str], - *, - max_workers: int = 1, - retry_strategy: "RetryStrategyProto | None" = None, - fetch_batch_size: int = 10, - min_fetch_interval: float = 1.0, - max_fetch_interval: float = 10.0, - lease_ttl_seconds: float = 60.0, - max_deliveries: int | None = None, - ack_policy: AckPolicy | None = None, - propagate_inbound_headers: bool = False, - dependencies: Iterable["Dependant"] = (), - parser: CustomCallable | None = None, - decoder: CustomCallable | None = None, - title_: str | None = None, - description_: str | None = None, - include_in_schema: bool = True, - ) -> "OutboxSubscriber": -``` - -And in the `create_subscriber(...)` call: - -```python - subscriber = create_subscriber( - queues=queue_list, - max_workers=max_workers, - retry_strategy=resolved_retry_strategy, - fetch_batch_size=fetch_batch_size, - min_fetch_interval=min_fetch_interval, - max_fetch_interval=max_fetch_interval, - lease_ttl_seconds=lease_ttl_seconds, - max_deliveries=max_deliveries, - ack_policy=ack_policy, - propagate_inbound_headers=propagate_inbound_headers, - config=self.config, # ty: ignore[invalid-argument-type] - title_=title_, - description_=description_, - include_in_schema=include_in_schema, - ) -``` - -- [ ] **Step 6: Plumb through `OutboxRoute`** - -Edit `faststream_outbox/router.py`. Add `propagate_inbound_headers: bool = False` to `OutboxRoute.__init__`'s kwargs (after `ack_policy`) and pass it through to `super().__init__(...)`. The `OutboxRouter` itself inherits `subscriber()` from `OutboxRegistrator` (already updated in Step 5), so no change to the router class body. - -```python - def __init__( # noqa: PLR0913 - self, - call: Callable[..., SendableMessage] | Callable[..., Awaitable[SendableMessage]], - queues: str | list[str], - *, - max_workers: int = 1, - retry_strategy: "RetryStrategyProto | None" = None, - fetch_batch_size: int = 10, - min_fetch_interval: float = 1.0, - max_fetch_interval: float = 10.0, - lease_ttl_seconds: float = 60.0, - max_deliveries: int | None = None, - ack_policy: AckPolicy | None = None, - propagate_inbound_headers: bool = False, - dependencies: Iterable["Dependant"] = (), - parser: CustomCallable | None = None, - decoder: CustomCallable | None = None, - title_: str | None = None, - description_: str | None = None, - include_in_schema: bool = True, - ) -> None: - super().__init__( - call=call, - queues=queues, - max_workers=max_workers, - retry_strategy=retry_strategy, - fetch_batch_size=fetch_batch_size, - min_fetch_interval=min_fetch_interval, - max_fetch_interval=max_fetch_interval, - lease_ttl_seconds=lease_ttl_seconds, - max_deliveries=max_deliveries, - ack_policy=ack_policy, - propagate_inbound_headers=propagate_inbound_headers, - dependencies=dependencies, - parser=parser, - decoder=decoder, - title_=title_, - description_=description_, - include_in_schema=include_in_schema, - ) -``` - -- [ ] **Step 7: Plumb through the FastAPI router** - -Edit `faststream_outbox/fastapi/router.py`. Locate `OutboxRouter.subscriber(...)` (it has a `# noqa: PLR0913` on its def line). Add `propagate_inbound_headers: bool = False` to its kwargs and pass it through to `super().subscriber(...)` (which dispatches to `OutboxRegistrator.subscriber` via the underlying broker). Match the kwarg's position to where it lives in `OutboxRegistrator.subscriber` (right after `ack_policy`). - -If the FastAPI router's `subscriber()` forwards via `**kwargs`, no change is needed — but in this codebase it forwards by name (per the existing `# noqa: PLR0913` on the def). Add the named pass-through. - -- [ ] **Step 8: Apply the kwarg in dispatch — introduce the `process_message` override** - -Edit `faststream_outbox/subscriber/usecase.py`. Add the following override **inside the `OutboxSubscriber` class**, placed near `_make_response_publisher` (the existing override) so adjacent FastStream-shape overrides stay grouped. Add the imports needed at the top of the file (verify each is not already imported). - -Required new imports at the top of `subscriber/usecase.py` (after the existing imports — `typing`, `contextlib`, `collections.abc` are already imported; only add what's missing): - -```python -from contextlib import AsyncExitStack # add to the existing `from contextlib import ...` line -from itertools import chain -from faststream.exceptions import SubscriberNotFound -from faststream.response.utils import ensure_response -``` - -Verify by grepping the file's top before editing; do not duplicate. - -Inside the class, add: - -```python - @typing.override - async def process_message(self, msg: OutboxInnerMessage) -> "Response": # type: ignore[override] - """Outbox-specific process_message that (a) optionally fills empty - Response headers with the inbound message's headers (G3) and (b) - refuses the OutboxResponse + foreign-publisher dual-fire combo (G1, - added in Task 5). - - Upstream equivalent (replaced): - SubscriberUsecase.process_message -> faststream/_internal/endpoint/subscriber/usecase.py - - Divergence from upstream is strictly additive — the chain composition, - middleware ordering, parsing-error rethrow, and AckPolicy semantics are - preserved verbatim. Any new cleanup added upstream to process_message - must be mirrored here. - """ - context = self._outer_config.fd_config.context - logger_state = self._outer_config.logger - - async with AsyncExitStack() as stack: - stack.enter_context(self.lock) - stack.enter_context(context.scope("handler_", self)) - stack.enter_context(context.scope("logger", logger_state.logger.logger)) - for k, v in self._outer_config.extra_context.items(): - stack.enter_context(context.scope(k, v)) - - middlewares: list[typing.Any] = [] - for base_m in self._SubscriberUsecase__build__middlewares_stack(): # name-mangled - middleware = base_m(msg, context=context) - middlewares.append(middleware) - await middleware.__aenter__() - - cache: dict[typing.Any, typing.Any] = {} - parsing_error: Exception | None = None - for h in self.calls: - try: - message = await h.is_suitable(msg, cache) - except Exception as e: # noqa: BLE001 - parsing_error = e - break - - if message is not None: - stack.enter_context( - context.scope("log_context", self.get_log_context(message)), - ) - stack.enter_context(context.scope("message", message)) - - for m in middlewares: - stack.push_async_exit(m.__aexit__) - - result_msg = ensure_response( - await h.call( - message=message, - _extra_middlewares=(m.consume_scope for m in middlewares[::-1]), - ), - ) - - if not result_msg.correlation_id: - result_msg.correlation_id = message.correlation_id - - if self._config.propagate_inbound_headers and not result_msg.headers: - result_msg.headers = dict(message.headers) - - for p in chain( - self._SubscriberUsecase__get_response_publisher(message), # name-mangled - h.handler._publishers, - ): - await p._publish( - result_msg.as_publish_command(), - _extra_middlewares=(m.publish_scope for m in middlewares[::-1]), - ) - - return result_msg - - for m in middlewares: - stack.push_async_exit(m.__aexit__) - - if parsing_error: - raise parsing_error - - error_msg = f"There is no suitable handler for {msg=}" - raise SubscriberNotFound(error_msg) - - return ensure_response(None) -``` - -About the name-mangled accesses (`_SubscriberUsecase__build__middlewares_stack`, `_SubscriberUsecase__get_response_publisher`): the upstream methods are private (`__build__middlewares_stack`, `__get_response_publisher`). Python name-mangles them to `_SubscriberUsecase__build__middlewares_stack` etc. We could reproduce both methods' bodies inline, but they handle subtleties (AckPolicy injection, no-reply gating) that we do not want to drift from upstream. Use the mangled name with a `# noqa: SLF001` comment. - -Replace the access lines with the mangled-name version plus `# noqa: SLF001`: - -```python - for base_m in self._SubscriberUsecase__build__middlewares_stack(): # noqa: SLF001 -``` - -```python - for p in chain( - self._SubscriberUsecase__get_response_publisher(message), # noqa: SLF001 - h.handler._publishers, # noqa: SLF001 - ): -``` - -- [ ] **Step 9: Run the relay tests** - -Run: `uv run pytest tests/test_relay.py -v --no-cov` -Expected: all four tests PASS. - -If the new test still fails on header assertion, check `TestKafkaBroker`'s spy contract — `publisher_kafka.mock.call_args.args` might be the *raw body*, with headers attached on the captured publish-command. If so, refine the header assertion to read `publisher_kafka.mock.call_args` and inspect what the test broker actually records. Headers should be observable somewhere on the spy; if not directly, fall back to asserting `result_msg.headers` shape by spying on the publisher's `_publish` method with `monkeypatch`. - -- [ ] **Step 10: Run lint and `ty` check** - -Run: `just lint` -Expected: clean. If `ty` complains about the name-mangled access, add the `# ty: ignore[unresolved-attribute]` directive at the call sites along with the `# noqa: SLF001`. - -- [ ] **Step 11: Commit** - -```bash -git add faststream_outbox/subscriber/config.py faststream_outbox/subscriber/factory.py faststream_outbox/subscriber/usecase.py faststream_outbox/registrator.py faststream_outbox/router.py faststream_outbox/fastapi/router.py tests/test_relay.py -git commit -m "feat: propagate_inbound_headers kwarg for foreign-broker relay - -Adds opt-in inbound-header propagation on OutboxSubscriber so handlers -that return a plain value can forward outbox-row headers to the -foreign publisher chain. Default False matches FastStream's -broker-wide convention. The dispatch hook lives in a process_message -override that Task 5 (OutboxResponse + foreign-publisher guard) also -uses. - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -### Task 5: G1 — block `OutboxResponse` + foreign publisher - -**Files:** -- Modify: `faststream_outbox/subscriber/usecase.py` (extend the `process_message` override added in Task 4) -- Modify: `tests/test_relay.py` (add the guard test) - -We extend the Task-4 override with a chain-composition check. The guard fires only when (a) the handler's response is an `OutboxResponse` AND (b) `h.handler._publishers` contains at least one entry that is **not** an `OutboxFakePublisher`. Raising inside the middleware-stacked region propagates to `AcknowledgementMiddleware.__aexit__`, which nacks per `AckPolicy`. - -- [ ] **Step 1: Write the failing test** - -Append to `tests/test_relay.py`: - -```python -from faststream_outbox import OutboxResponse # add near the existing imports - - -async def test_outbox_response_with_foreign_publisher_raises() -> None: - """A handler that returns OutboxResponse(...) AND is decorated by a foreign - publisher is rejected at dispatch time so the user does not silently - dual-fire (row in outbox + Kafka publish).""" - broker_outbox = OutboxBroker() - broker_kafka = KafkaBroker("kafka://test:9092") - publisher_kafka = broker_kafka.publisher("relay_topic") - - @publisher_kafka - @broker_outbox.subscriber("relay_queue") - async def relay(body: dict[str, Any]) -> OutboxResponse: - return OutboxResponse(body=body, queue="next_queue", session=None) - - async with TestKafkaBroker(broker_kafka), TestOutboxBroker(broker_outbox, run_loops=False) as outbox: - with pytest.raises(RuntimeError, match="OutboxResponse"): - await outbox.publish({"x": 1}, queue="relay_queue", session=None) -``` - -The test relies on the sync-dispatch mode of `TestOutboxBroker` (default `run_loops=False`) so the exception propagates synchronously out of `outbox.publish(...)`. In loop mode the exception would be swallowed by `dispatch_one`'s broad except and turned into a retry; the sync mode is the documented test-broker idiom for asserting handler-side exceptions. - -- [ ] **Step 2: Run the test and confirm it fails** - -Run: `uv run pytest tests/test_relay.py::test_outbox_response_with_foreign_publisher_raises -v --no-cov` -Expected: FAIL — the relay currently dual-fires without raising. - -- [ ] **Step 3: Add the guard to the `process_message` override** - -In `faststream_outbox/subscriber/usecase.py`, in the `process_message` override added in Task 4, locate the `for p in chain(...)` loop. Just before that loop, add: - -```python - self._reject_outbox_response_with_foreign_publisher(result_msg, h.handler) - - for p in chain( - self._SubscriberUsecase__get_response_publisher(message), # noqa: SLF001 - h.handler._publishers, # noqa: SLF001 - ): - ... -``` - -Then add the helper as a method on `OutboxSubscriber` (near the override): - -```python - @staticmethod - def _reject_outbox_response_with_foreign_publisher( - result_msg: "Response", - handler: typing.Any, - ) -> None: - """Refuse the dual-fire combination: OutboxResponse + foreign publisher. - - OutboxResponse(body=..., queue=..., session=...) writes to the outbox in - the caller's transaction; a foreign-publisher decorator also publishes - the relayed body. Both would fire from the chain. That is almost - certainly not intended — pick one. - """ - if not isinstance(result_msg, OutboxResponse): - return - foreign = [ - p for p in handler._publishers - if not isinstance(p, OutboxFakePublisher) - ] - if not foreign: - return - msg = ( - "Handler returned OutboxResponse and is also decorated by a foreign-broker " - "publisher — this would dual-fire (insert a row into the outbox AND publish " - "to the foreign broker). Pick one: return a plain value to use the foreign " - "publisher as a relay, or remove the foreign publisher decorator and keep " - "OutboxResponse for outbox fan-out." - ) - raise RuntimeError(msg) -``` - -Add `from faststream_outbox.response import OutboxResponse` to the top-of-file imports if not already present. `OutboxFakePublisher` is already imported (used by `_make_response_publisher`). - -- [ ] **Step 4: Run the test and confirm it passes** - -Run: `uv run pytest tests/test_relay.py::test_outbox_response_with_foreign_publisher_raises -v --no-cov` -Expected: PASS. - -- [ ] **Step 5: Run the rest of the relay tests to confirm no regression** - -Run: `uv run pytest tests/test_relay.py -v --no-cov` -Expected: all five tests PASS. - -- [ ] **Step 6: Lint** - -Run: `just lint` -Expected: clean. - -- [ ] **Step 7: Commit** - -```bash -git add faststream_outbox/subscriber/usecase.py tests/test_relay.py -git commit -m "feat: refuse OutboxResponse + foreign-publisher dual-fire - -A handler that returns OutboxResponse(...) and is also decorated by a -foreign-broker publisher would both insert an outbox row AND publish -to the foreign broker on every dispatch. Detect the combo at dispatch -time and raise a RuntimeError pointing at the two valid patterns. - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -### Task 6: G2 — WARN on unstarted foreign broker at `start()` - -**Files:** -- Modify: `faststream_outbox/broker.py` -- Modify: `tests/test_relay.py` - -When `broker_outbox.start()` runs, walk every subscriber's `handler._publishers` looking for publishers whose `_outer_config` is *not* this broker's. For each such foreign publisher, duck-type-check whether the foreign broker has a producer; if not, log a single WARNING per unstarted broker referencing the affected queue names. - -- [ ] **Step 1: Write the failing test** - -Append to `tests/test_relay.py`: - -```python -import logging # add to top of file if not already imported - - -async def test_unstarted_foreign_broker_warns_on_start(caplog: pytest.LogCaptureFixture) -> None: - """If a foreign-publisher decorator is on an outbox subscriber but the - foreign broker has not been started, start(broker_outbox) logs a single - WARNING per unstarted foreign broker.""" - broker_outbox = OutboxBroker() - broker_kafka = KafkaBroker("kafka://test:9092") - publisher_kafka = broker_kafka.publisher("relay_topic") - - @publisher_kafka - @broker_outbox.subscriber("relay_queue") - async def relay(body: dict[str, Any]) -> dict[str, Any]: - return body - - with caplog.at_level(logging.WARNING, logger="faststream_outbox"): - async with TestOutboxBroker(broker_outbox, run_loops=False): - pass # start triggered inside __aenter__ - - matching = [r for r in caplog.records if r.levelno == logging.WARNING and "relay_queue" in r.getMessage()] - assert len(matching) == 1, f"Expected exactly one WARNING referencing relay_queue, got {len(matching)}: {[r.getMessage() for r in caplog.records]}" -``` - -- [ ] **Step 2: Run the test and confirm it fails** - -Run: `uv run pytest tests/test_relay.py::test_unstarted_foreign_broker_warns_on_start -v --no-cov` -Expected: FAIL (no WARNING emitted because the check does not exist yet). - -- [ ] **Step 3: Implement the check in `OutboxBroker.start()`** - -Edit `faststream_outbox/broker.py`. Modify the existing `start()` override (around line 211) to call a new helper before returning: - -```python - @typing.override - async def start(self) -> None: - await self.connect() - await super().start() - self._warn_on_unstarted_foreign_publishers() -``` - -Then add the helper as a method on `OutboxBroker`: - -```python - def _warn_on_unstarted_foreign_publishers(self) -> None: - """Emit one WARNING per foreign-publisher broker that has not been started. - - Foreign-publisher decorators stacked on outbox subscribers only work if - the foreign broker's producer is wired. When it is not, the first - relayed row fails deep inside the foreign publisher with an opaque - AttributeError; this preflight pushes the diagnostic up to start() so - operators see the cause immediately. - """ - logger_state = self.config.broker_config.logger - log = logger_state.logger.logger if logger_state is not None else None - if log is None: - return - warned: set[int] = set() - for sub in self.subscribers: - for call in sub.calls: - for pub in call.handler._publishers: # noqa: SLF001 - outer = pub._outer_config # noqa: SLF001 - if outer is self.config: - continue # not foreign - producer = getattr(outer, "producer", None) - if producer is not None: - continue # already wired - key = id(outer) - if key in warned: - continue - warned.add(key) - queues = sorted({q for s in self.subscribers for q in getattr(s, "_queues", [])}) - log.warning( - "Foreign publisher %r is decorated on outbox subscriber(s) for " - "queue(s) %s, but its broker has not been started yet. The first " - "relay attempt will fail and the row will retry until the broker " - "starts. Call `await foreign_broker.start()` or " - "`foreign_broker.connect` in your app's startup hook.", - pub, - queues, - ) -``` - -- [ ] **Step 4: Run the test and confirm it passes** - -Run: `uv run pytest tests/test_relay.py::test_unstarted_foreign_broker_warns_on_start -v --no-cov` -Expected: PASS. - -- [ ] **Step 5: Run all relay tests to confirm no regression** - -Run: `uv run pytest tests/test_relay.py -v --no-cov` -Expected: all six tests PASS. - -If `test_naked_decorator_chain_relays_plain_return_to_kafka` (or any other Task 2/3/4 test) regresses with a spurious WARNING, the foreign broker is in fact unstarted at the moment `TestOutboxBroker.__aenter__` calls our start hook. Investigate — the fix is to detect the test-broker state (e.g., the foreign broker's producer becomes non-`None` only inside `TestKafkaBroker.__aenter__`). One safe option: skip the warning when the foreign broker's outer config is *also* registered with a `TestBroker` (probe by looking for a sentinel attribute the test broker sets). Document the workaround in code if used. - -- [ ] **Step 6: Lint** - -Run: `just lint` -Expected: clean. - -- [ ] **Step 7: Commit** - -```bash -git add faststream_outbox/broker.py tests/test_relay.py -git commit -m "feat: WARN on unstarted foreign-publisher brokers at start() - -Preflight check during OutboxBroker.start() walks subscribers for -publishers whose _outer_config is foreign and logs one WARNING per -unstarted foreign broker. Operators see the cause immediately -instead of debugging an AttributeError on the first relay attempt. - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -### Task 7: Integration test — at-least-once under foreign-publish failure - -**Files:** -- Modify: `tests/test_integration.py` - -Real outbox subscriber (Postgres-backed) plus a foreign publisher whose `_publish` raises on the first call and succeeds on retry. We assert exactly-one eventual delivery and that `deliveries_count` on the row reflects the retry. `TestKafkaBroker` provides the destination so no real Kafka is needed; the outbox side uses the real `OutboxBroker` against `pg_engine`. - -- [ ] **Step 1: Append the test to `tests/test_integration.py`** - -Locate the existing imports. Add at the top of the file (only the names not already present): - -```python -from unittest.mock import AsyncMock, patch -from typing import Any -import asyncio - -from faststream.kafka import KafkaBroker, TestKafkaBroker - -from faststream_outbox import OutboxBroker -``` - -Append the test at the bottom of the file: - -```python -async def test_relay_at_least_once_under_foreign_publish_failure( - pg_engine: AsyncEngine, -) -> None: - """A foreign publish that fails on the first attempt is retried via the - outbox's retry_strategy, and the row eventually clears after a successful - second attempt. Asserts that the foreign publisher saw the body twice - (at-least-once) and the outbox row was deleted after the second attempt.""" - broker_outbox = OutboxBroker(engine=pg_engine) - broker_kafka = KafkaBroker("kafka://test:9092") - publisher_kafka = broker_kafka.publisher("relay_topic") - - # Inject a fault in the foreign publisher: first _publish raises, second succeeds. - call_count = 0 - original_publish = publisher_kafka._publish - - async def flaky_publish(*args: Any, **kwargs: Any) -> None: - nonlocal call_count - call_count += 1 - if call_count == 1: - msg = "simulated foreign-publish failure" - raise RuntimeError(msg) - return await original_publish(*args, **kwargs) - - @publisher_kafka - @broker_outbox.subscriber( - "relay_queue", - max_workers=1, - max_fetch_interval=0.1, - min_fetch_interval=0.0, - lease_ttl_seconds=2.0, # short enough to retry quickly during the test - ) - async def relay(body: dict[str, Any]) -> dict[str, Any]: - return body - - with patch.object(publisher_kafka, "_publish", side_effect=flaky_publish): - async with TestKafkaBroker(broker_kafka), broker_outbox: - # Publish one row via the broker's own session. - async with broker_outbox._engine.connect() as conn: - async with conn.begin(): - await broker_outbox.publish( - {"body": "first"}, - queue="relay_queue", - session=conn, # AsyncConnection — type-relax via session param contract - ) - - # Wait until the row clears (foreign mock recorded the body at least once). - for _ in range(50): - if call_count >= 2: - break - await asyncio.sleep(0.1) - - assert call_count >= 2, ( - f"Expected at-least-once delivery via retry, but foreign _publish was called {call_count} time(s)." - ) -``` - -If the test does not have an `AsyncConnection`-as-session contract (the broker only accepts `AsyncSession`), adapt by passing the engine's `async_sessionmaker` instead: - -```python - from sqlalchemy.ext.asyncio import async_sessionmaker - sm = async_sessionmaker(broker_outbox._engine, expire_on_commit=False) - async with sm() as session, session.begin(): - await broker_outbox.publish( - {"body": "first"}, - queue="relay_queue", - session=session, - ) -``` - -- [ ] **Step 2: Run the test** - -Run: `just test tests/test_integration.py::test_relay_at_least_once_under_foreign_publish_failure --no-cov` -Expected: PASS. Test runs under docker compose so Postgres is available at the DSN the fixture expects. - -If the test times out at `call_count >= 2`: -- The first failure should produce a `nacked_retried` (or `nacked_terminal` if retries are exhausted) event. Check the WARNING log; if the row is `nacked_terminal`, `lease_ttl_seconds` is too tight relative to the default retry-strategy backoff. Bump `lease_ttl_seconds` and/or pass an explicit `retry_strategy=ExponentialRetry(initial_delay_seconds=0.1, multiplier=1.0, max_attempts=3)` to give it room. -- Confirm the foreign publisher's `_publish` is actually being patched by inspecting `publisher_kafka._publish.__name__`. - -- [ ] **Step 3: Commit** - -```bash -git add tests/test_integration.py -git commit -m "test: at-least-once relay under simulated foreign-publish failure - -Confirms the FastStream AckPolicy nack path (publisher-chain exception --> AcknowledgementMiddleware.__aexit__ -> outbox row nack/retry) end -to end against a real Postgres-backed OutboxBroker plus TestKafkaBroker. - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -### Task 8: Docs — tutorial, examples grid, mkdocs nav - -**Files:** -- Create: `docs/usage/relay.md` -- Modify: `mkdocs.yml` - -Tutorial is promoted to the top of the `Usage` nav, above `usage/basic.md`. Tutorial sections mirror the spec's D2: Why → Minimal example → Two-broker lifecycle (FastAPI + Standalone) → At-least-once contract → Header propagation → Using routers → What not to do → Cross-broker examples grid. - -- [ ] **Step 1: Create `docs/usage/relay.md`** - -Create the file with the following contents: - -````markdown -# Relay to a foreign broker - -The outbox pattern's payoff line: domain code writes a row to the outbox in -the same DB transaction as its other writes, and a separate worker relays -those rows to a real bus (Kafka, RabbitMQ, NATS, Redis…). `faststream-outbox` -supports this directly via FastStream's cross-broker chain — stack a -foreign-broker publisher decorator on an outbox subscriber and you're done. - -## Why an outbox relay - -When a request must (a) update your database and (b) emit an event onto a -message bus, the naive shape — DB commit, then bus publish — leaks events on -crashes between the two steps. The outbox pattern fixes this by writing the -event as a row in the same transaction as the domain update; a separate -worker reads the row and publishes to the bus. The row is the durability -boundary, and the relay carries an at-least-once guarantee end to end. - -## Minimal relay - -```python -from faststream.kafka import KafkaBroker -from faststream_outbox import OutboxBroker - -broker_outbox = OutboxBroker(engine=engine) -broker_kafka = KafkaBroker("127.0.0.1:9092") -publisher_kafka = broker_kafka.publisher("kafka_topic") - - -@publisher_kafka -@broker_outbox.subscriber("outbox_queue") -async def relay(body: dict) -> dict: - return body -``` - -That's the whole thing. `broker_outbox.publish(body, queue="outbox_queue", session=session)` -in your domain transaction writes a row; the subscriber dispatches it; the -handler returns it; the Kafka publisher decorator picks it up and publishes -to `kafka_topic`. Failure handling, retries, and DLQ are unchanged from -the rest of the outbox subscriber's behavior. - -## Two-broker lifecycle - -Both brokers must be started for the relay to work. Two idiomatic shapes: - -### FastAPI (recommended) - -Mount `OutboxRouter` and the foreign broker's router on the same `FastAPI` -app. Both auto-start via FastAPI's lifespan. - -```python -from fastapi import FastAPI -from faststream.kafka.fastapi import KafkaRouter -from faststream_outbox.fastapi import OutboxRouter - -outbox_router = OutboxRouter(engine=engine) -kafka_router = KafkaRouter("127.0.0.1:9092") - -publisher_kafka = kafka_router.publisher("kafka_topic") - - -@publisher_kafka -@outbox_router.subscriber("outbox_queue") -async def relay(body: dict) -> dict: - return body - - -app = FastAPI() -app.include_router(outbox_router) -app.include_router(kafka_router) -``` - -### Standalone - -A single `FastStream` app with the foreign broker's `connect` hooked into -`on_startup`: - -```python -from faststream import FastStream -from faststream.kafka import KafkaBroker -from faststream_outbox import OutboxBroker - -broker_outbox = OutboxBroker(engine=engine) -broker_kafka = KafkaBroker("127.0.0.1:9092") -publisher_kafka = broker_kafka.publisher("kafka_topic") - - -@publisher_kafka -@broker_outbox.subscriber("outbox_queue") -async def relay(body: dict) -> dict: - return body - - -app = FastStream(broker_outbox, on_startup=[broker_kafka.connect]) -``` - -## At-least-once contract - -If the foreign publish raises (Kafka down, partition unavailable, etc.), -the exception propagates through FastStream's `AcknowledgementMiddleware`, -the outbox row is nacked, and the configured `retry_strategy` reschedules -it. The next dispatch re-runs the handler and re-attempts the foreign -publish. **Net effect: at-least-once delivery to the foreign broker.** - -Downstream consumers should handle duplicates idempotently, the same way -they would behind any at-least-once bus. - -## Header propagation - -By default, FastStream's `Response(value)` ships with empty headers, so -the inbound outbox row's headers (`content-type`, custom trace keys, etc.) -are **not** forwarded to the foreign publish. Two ways to override: - -**Explicit (per handler):** - -```python -from faststream.response import Response - -@publisher_kafka -@broker_outbox.subscriber("outbox_queue") -async def relay(body: dict, msg: OutboxMessage) -> Response: - return Response(body, headers=msg.headers) -``` - -**Opt-in (per subscriber):** - -```python -@publisher_kafka -@broker_outbox.subscriber("outbox_queue", propagate_inbound_headers=True) -async def relay(body: dict) -> dict: - return body -``` - -With `propagate_inbound_headers=True`, the subscriber fills `Response.headers` -from the inbound `OutboxMessage.headers` *unless* the handler returned a -`Response(..., headers=...)` explicitly. - -## Using routers - -Both halves of the chain can live on routers — the FastAPI shape above -already does this with `KafkaRouter` and `OutboxRouter`. The constraint is -that `broker.include_router(router)` must happen *before* the brokers -start. Inside `FastAPI(..., lifespan=...)` the include happens during app -construction (before lifespan), so it's automatic. - -For the standalone (non-FastAPI) lifecycle, the order is: - -```python -broker_kafka.include_router(kafka_router) -broker_outbox.include_router(outbox_router) -# then start -``` - -## What not to do - -**Do not** combine `OutboxResponse(...)` and a foreign-publisher decorator. - -```python -@publisher_kafka -@broker_outbox.subscriber("outbox_queue") -async def relay(body: dict) -> OutboxResponse: - return OutboxResponse(body=body, queue="next_queue", session=...) # ❌ -``` - -This would both insert a row into the outbox AND publish to Kafka. The -subscriber raises `RuntimeError` at dispatch time when it detects the -combination — pick one path. - -**Do not** stack an outbox publisher on a foreign subscriber. - -```python -@broker_outbox.publisher("outbox_queue") -@broker_kafka.subscriber("kafka_topic") # ❌ -async def relay(body: dict) -> dict: - return body -``` - -This direction would need the Kafka subscriber's dispatch loop to provide -an `AsyncSession` for the outbox insert — there isn't one without breaking -the transactional contract. `OutboxPublisher.__call__` raises -`NotImplementedError` at decoration time. Call `await broker_outbox.publish(...)` -inside the handler instead, on a session you opened yourself. - -## Other foreign brokers - -The same pattern works for Confluent, RabbitMQ, NATS, and Redis — the only -change is the `publisher` line: - -| Foreign broker | Publisher line | -|---|---| -| Kafka | `broker_kafka.publisher("topic")` | -| Confluent | `broker_confluent.publisher("topic")` | -| RabbitMQ | `broker_rabbit.publisher("queue")` | -| NATS | `broker_nats.publisher("subject")` | -| Redis | `broker_redis.publisher("channel")` | - -Any FastStream broker whose publisher's `_publish` accepts a generic -`PublishCommand` works as a relay destination — that is the FastStream -cross-broker contract, not an outbox-specific feature. -```` - -- [ ] **Step 2: Update `mkdocs.yml` to promote the tutorial to the top of `Usage`** - -Edit `mkdocs.yml`. Change the `Usage:` block from: - -```yaml - - Usage: - - Basic usage: usage/basic.md - - Subscriber: usage/subscriber.md -``` - -to: - -```yaml - - Usage: - - Relay to a foreign broker: usage/relay.md - - Basic usage: usage/basic.md - - Subscriber: usage/subscriber.md -``` - -Keep the rest of the `Usage` list intact. - -- [ ] **Step 3: Visually verify the docs build (optional but recommended)** - -Run: `uv run mkdocs build --strict 2>&1 | tail -20` -Expected: build succeeds. If `mkdocs` is not installed in the project venv, skip this step. - -- [ ] **Step 4: Commit** - -```bash -git add docs/usage/relay.md mkdocs.yml -git commit -m "docs: add foreign-broker relay tutorial, promote to top of Usage nav - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -### Task 9: Docs — intro callout, CLAUDE.md subsection - -**Files:** -- Modify: `docs/introduction/how-it-works.md` (one-line callout near the bottom) -- Modify: `CLAUDE.md` - -- [ ] **Step 1: Add the intro callout** - -Edit `docs/introduction/how-it-works.md`. Locate the last paragraph (or the section closest to "what's next"). Append a one-line callout: - -```markdown -> **Relay outbox rows to Kafka / RabbitMQ / NATS / Redis with a single decorator → [Relay tutorial](../usage/relay.md).** -``` - -If the file's structure does not have a natural "what's next" tail, append the callout as the last line of the file. - -- [ ] **Step 2: Add the CLAUDE.md subsection** - -Edit `CLAUDE.md`. Locate the existing "Producer side" or "Two-loop subscriber" section. Insert a new subsection (level-3 heading) immediately after the "Producer side" block: - -```markdown -### Relay to foreign broker - -`OutboxSubscriber` supports being the source of a FastStream-native -cross-broker chain: `@kafka_pub @broker_outbox.subscriber("q")` (and the -same for Rabbit/NATS/Redis/Confluent) is the canonical -transactional-outbox primitive. The mechanism is upstream FastStream's -`SubscriberUsecase.process_message` walking -`chain(self.__get_response_publisher(message), h.handler._publishers)`; -no override of the dispatch path is needed. Three load-bearing -properties keep the chain safe: (a) `OutboxFakePublisher` self-gates on -`isinstance(cmd, OutboxPublishCommand)` so it no-ops on plain handler -returns; (b) foreign publishers coerce via -`PublishCommand.from_cmd(cmd)`; (c) -`AcknowledgementMiddleware.__aexit__` turns publisher-chain exceptions -into outbox nacks, preserving at-least-once delivery via the configured -`retry_strategy`. - -Three guardrails sit on top of the bare mechanism: - -- **`OutboxResponse` + foreign publisher refused.** The - `process_message` override checks the chain composition and raises a - `RuntimeError` if a handler returns `OutboxResponse(...)` while having - a non-`OutboxFakePublisher` entry in `handler._publishers`. The - exception propagates through `AcknowledgementMiddleware` and triggers - the outbox's normal nack path. - -- **WARNING for unstarted foreign brokers at start().** `OutboxBroker.start()` - walks `self.subscribers`, for each subscriber walks - `handler._publishers`, and for any publisher whose `_outer_config` is - not `self.config` and whose foreign broker has no `producer` set, logs - one WARNING per unstarted foreign broker. Operators see the cause - before the first relayed row fails. - -- **`propagate_inbound_headers: bool = False` on subscriber.** When - `True`, the `process_message` override fills `Response.headers` from - the inbound `OutboxMessage.headers` only when the handler returned a - `Response` with empty headers (explicit user-set headers win). Default - False matches the FastStream-wide convention; users who want - propagation flip one kwarg. - -User-facing reference: `docs/usage/relay.md`. -``` - -Adjust the inserted position to match the existing CLAUDE.md flow (the file is large; place this subsection adjacent to the producer-side material, not in the middle of subscriber-internals). - -- [ ] **Step 3: Commit** - -```bash -git add docs/introduction/how-it-works.md CLAUDE.md -git commit -m "docs: intro callout and CLAUDE.md subsection for foreign-broker relay - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -### Task 10: Docs — README front-page rewrite - -**Files:** -- Modify: `README.md` - -Replace the existing quickstart with the end-to-end relay example. The relay is the canonical outbox use case; promoting it to the README front page matches the spec's visibility goal and aligns with how `faststream-sqlbroker` features the same pattern. - -- [ ] **Step 1: Read the current README** - -Run: `cat README.md | head -80` - -Inspect the existing structure. The rewrite preserves the project intro, badges (if any), and installation block; only the "quickstart" / "usage" code-example region is replaced. - -- [ ] **Step 2: Replace the quickstart section** - -Locate the existing quickstart code block (the one that shows `broker.publish(...)` and a handler in isolation). Replace it with: - -````markdown -## Quickstart — outbox relay to Kafka - -Write the outbox row in your domain transaction: - -```python -from faststream_outbox import OutboxBroker - -broker_outbox = OutboxBroker(engine=engine) - - -async def create_order(session, order_data): - order = await orders.insert(session, order_data) - await broker_outbox.publish( - {"order_id": order.id, "total": order.total}, - queue="orders_outbox", - session=session, # same transaction as the row above - ) - # On session.commit(), both the order row AND the outbox row land - # atomically. No commit, no event. -``` - -Relay outbox rows to Kafka with a single decorator: - -```python -from faststream import FastStream -from faststream.kafka import KafkaBroker -from faststream_outbox import OutboxBroker - -broker_outbox = OutboxBroker(engine=engine) -broker_kafka = KafkaBroker("127.0.0.1:9092") -publisher_kafka = broker_kafka.publisher("orders") - - -@publisher_kafka -@broker_outbox.subscriber("orders_outbox") -async def relay(body: dict) -> dict: - return body - - -app = FastStream(broker_outbox, on_startup=[broker_kafka.connect]) -``` - -The same shape works for RabbitMQ, NATS, Redis, and Confluent. See the -[relay tutorial](docs/usage/relay.md) for the FastAPI lifecycle, -header propagation, router shapes, and the at-least-once contract. -```` - -Adjust headings and anchors to match the existing README's style. - -- [ ] **Step 3: Commit** - -```bash -git add README.md -git commit -m "docs: rewrite README quickstart around the outbox relay - -The outbox-to-foreign-broker relay is the canonical use case for this -package; promote it to the front page so readers see the payoff line -immediately. - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -### Task 11: Final validation - -**Files:** none modified — this task only runs lint + full test suite and reports. - -- [ ] **Step 1: Lint in non-mutating mode** - -Run: `just lint-ci` -Expected: clean (no `eof-fixer`, no `ruff format`, no `ruff check --fix`, no `ty check` errors). - -- [ ] **Step 2: Run the full test suite under docker compose** - -Run: `just test` -Expected: all tests pass, coverage is at 100%. - -If coverage drops, look at the diff: -- `_warn_on_unstarted_foreign_publishers` branch where logger is `None` — add a targeted test or a `# pragma: no cover` on that one defensive guard. -- The `parsing_error` re-raise and `SubscriberNotFound` paths in the `process_message` override — these are reproduced from upstream and may not have outbox-specific coverage. If the existing suite does not cover them, mark them `# pragma: no cover` with a comment pointing at the upstream parallel. - -- [ ] **Step 3: Inspect the branch state** - -Run: `git log --oneline main..HEAD` -Expected: ~9 commits, one per task (Tasks 1, 2, 3, 4, 5, 6, 7, 8, 9, 10). - -Run: `git diff main..HEAD --stat` -Expected: the file list matches the spec's "File touch list" exactly. - -- [ ] **Step 4: Push and open a PR** - -Run: -```bash -git push -u origin feat/foreign-broker-relay -gh pr create --title "feat: foreign-broker relay from OutboxSubscriber" --body "$(cat <<'EOF' -## Summary - -- Document and test FastStream's native cross-broker chain (`@kafka_pub @broker_outbox.subscriber(...)`) as the canonical outbox use case. -- Three guardrails: refuse `OutboxResponse` + foreign-publisher dual-fire (G1), WARN on unstarted foreign brokers at `start()` (G2), opt-in inbound-header propagation (G3, default False). -- Promote the relay tutorial to the top of `Usage` nav; rewrite the README quickstart around the relay example. -- Concrete `faststream-sqlbroker` comparison documented in the spec and CLAUDE.md. - -## Test plan - -- [x] `tests/test_relay.py` — six unit tests against `TestKafkaBroker` (naked chain, two router shapes, header propagation, G1 guard, G2 warning). -- [x] `tests/test_integration.py::test_relay_at_least_once_under_foreign_publish_failure` — Postgres-backed test that proves AckPolicy nack → retry → eventual success. -- [x] `just lint-ci` clean. -- [x] `just test` passes with 100% coverage. - -Spec: `planning/specs/2026-06-04-foreign-broker-relay-design.md` -Plan: `planning/plans/2026-06-04-foreign-broker-relay-plan.md` - -🤖 Generated with [Claude Code](https://claude.com/claude-code) -EOF -)" -``` - -Expected: PR URL printed to stdout. Hand to user. - ---- - -## Notes for the engineer - -- **All test arguments — including pytest fixtures — must be type-annotated** per the project's standing convention. -- **No local imports.** Every import goes at the top of the module. This applies to test files too. -- **`ty: ignore[]`** is the project's escape hatch for type checker complaints; match the existing usage in `broker.py` and `registrator.py`. -- The `process_message` override in Task 4 is the second non-trivial FastStream-method override in this codebase (after `OutboxBroker.stop` and `OutboxSubscriber.stop`). Mirror the convention used there: include an `# Upstream equivalent (replaced): …` comment pointing at the upstream method, and keep divergence strictly additive. -- If at any point a Task's "Expected: PASS" step does not pass, **stop, diagnose, and fix in place**. Do not skip to the next Task with a failing test — the plan was written with the dependency order in mind. - -## Context-manager ordering note - -Every relay test enters `TestKafkaBroker` **before** `TestOutboxBroker`: - -```python -async with TestKafkaBroker(broker_kafka), TestOutboxBroker(broker_outbox, run_loops=False) as outbox: - ... -``` - -Reason: `TestOutboxBroker.__aenter__` calls `broker_outbox.start()`, which -after Task 6 walks every foreign-publisher `_outer_config.producer` to -decide whether to emit a "broker not started" WARNING. If -`TestKafkaBroker.__aenter__` runs *after* the outbox starts, the Kafka -broker's mock producer is still `None` at probe time and we get a -false-positive WARNING. Entering the foreign test broker first wires its -mock producer before our subscriber starts. - -Task 6's own test (`test_unstarted_foreign_broker_warns_on_start`) is the -only one that deliberately enters the outbox broker **without** the Kafka -context — that's how it triggers the WARNING under test. diff --git a/planning/changes/2026-06-09.01-mkdocs-github-pages/design.md b/planning/changes/2026-06-09.01-mkdocs-github-pages.md similarity index 100% rename from planning/changes/2026-06-09.01-mkdocs-github-pages/design.md rename to planning/changes/2026-06-09.01-mkdocs-github-pages.md diff --git a/planning/changes/2026-06-09.01-mkdocs-github-pages/plan.md b/planning/changes/2026-06-09.01-mkdocs-github-pages/plan.md deleted file mode 100644 index 7ebd128..0000000 --- a/planning/changes/2026-06-09.01-mkdocs-github-pages/plan.md +++ /dev/null @@ -1,604 +0,0 @@ -# Migrate Docs Hosting from Read the Docs to GitHub Pages — Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Replace the Read the Docs deploy pipeline with a GitHub Actions + GitHub Pages deploy mirroring `modern-di`, served at `faststream-outbox.modern-python.org`. Bump CI action pins to the same baseline along the way. - -**Architecture:** Eight in-repo file changes (one new workflow, one new docs/CNAME, one Justfile target, one mkdocs.yml field, two file deletions/edits for RTD removal, two action-pin bumps). Out-of-repo DNS + GH Pages settings happen after merge (documented in the spec, not in this plan). All commits land on a single feature branch. - -**Tech Stack:** mkdocs + mkdocs-material (existing), `mkdocs gh-deploy` (force-push to `gh-pages`), GitHub Actions, `just`, `uvx`. - -**Spec:** `planning/specs/2026-06-09-mkdocs-github-pages-design.md` - -**Branch:** Work on a new branch `feat/mkdocs-github-pages` off `main`. Do not commit to `main` directly. - ---- - -## Pre-flight - -- [ ] **Step 1: Create the feature branch** - -```bash -git checkout -b feat/mkdocs-github-pages -git status -``` - -Expected: `On branch feat/mkdocs-github-pages` with clean working tree. - -- [ ] **Step 2: Confirm assumptions about current state** - -```bash -ls .readthedocs.yaml docs/requirements.txt docs/CNAME 2>&1 -grep -c "site_url" mkdocs.yml -grep "docs-deploy" Justfile -``` - -Expected: -- `.readthedocs.yaml` and `docs/requirements.txt` exist. -- `docs/CNAME` does NOT exist (`ls` reports no such file). -- `site_url` grep returns `0` (currently absent). -- `docs-deploy` grep returns no match (Justfile has no docs target yet). - -If any expectation fails, STOP and re-read the spec — assumptions changed. - ---- - -## Task 1: Add the `just docs-deploy` target - -**Files:** -- Modify: `Justfile` (append at end) - -- [ ] **Step 1: Read the current Justfile end** - -```bash -tail -5 Justfile -``` - -Expected: last target is `publish:` with `uv publish --token $PYPI_TOKEN`. Confirms there's no trailing blank-line surprise. - -- [ ] **Step 2: Append the `docs-deploy` target** - -Append these lines to the end of `Justfile` (preserve a single blank line before the new target, single trailing newline at EOF): - -```just - -# Force-pushes built site to gh-pages; CI runs this on push to main. -# Manual invocation from a stale checkout will roll the live site back. -docs-deploy: - uvx --with-requirements docs/requirements.txt mkdocs gh-deploy --force -``` - -- [ ] **Step 3: Verify the target is discoverable** - -Run: `just --list | grep docs-deploy` - -Expected: one line — ` docs-deploy # Force-pushes built site to gh-pages; CI runs this on push to main.` - -- [ ] **Step 4: Smoke-test mkdocs without deploying** - -Run: `uvx --with-requirements docs/requirements.txt mkdocs build --strict --site-dir /tmp/mkdocs-smoke` - -Expected: build completes with no warnings (`--strict` turns warnings into errors). Output ends with `INFO - Documentation built in N.NNs`. The `/tmp/mkdocs-smoke` directory now contains `index.html`. Clean up with `rm -rf /tmp/mkdocs-smoke`. - -If the build fails, the current `mkdocs.yml` has a latent issue unrelated to this plan — STOP and report. - -- [ ] **Step 5: Commit** - -```bash -git add Justfile -git commit -m "chore: add just docs-deploy target for mkdocs gh-deploy - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -## Task 2: Add `docs/CNAME` - -**Files:** -- Create: `docs/CNAME` - -- [ ] **Step 1: Create the file with the custom domain** - -Create `docs/CNAME` containing exactly one line with a trailing newline: - -``` -faststream-outbox.modern-python.org -``` - -- [ ] **Step 2: Verify contents** - -Run: `cat docs/CNAME && wc -l docs/CNAME` - -Expected: -``` -faststream-outbox.modern-python.org - 1 docs/CNAME -``` - -(One line, one newline at EOF.) - -- [ ] **Step 3: Confirm mkdocs picks it up** - -Run: `uvx --with-requirements docs/requirements.txt mkdocs build --strict --site-dir /tmp/mkdocs-smoke && cat /tmp/mkdocs-smoke/CNAME && rm -rf /tmp/mkdocs-smoke` - -Expected: build succeeds, `CNAME` is copied into the built site with the same single-line contents. - -- [ ] **Step 4: Commit** - -```bash -git add docs/CNAME -git commit -m "docs: add CNAME for faststream-outbox.modern-python.org - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -## Task 3: Add `site_url` to `mkdocs.yml` - -**Files:** -- Modify: `mkdocs.yml` (insert one line after `site_name:`) - -- [ ] **Step 1: Read the first 6 lines of mkdocs.yml** - -```bash -head -6 mkdocs.yml -``` - -Expected current top: -```yaml -site_name: faststream-outbox -repo_url: https://github.com/modern-python/faststream-outbox -docs_dir: docs -edit_uri: edit/main/docs/ -nav: - - Introduction: -``` - -- [ ] **Step 2: Insert `site_url` after `site_name`** - -After line 1 (`site_name: faststream-outbox`), insert: - -```yaml -site_url: https://faststream-outbox.modern-python.org/ -``` - -Result: lines 1–2 are `site_name:` followed by `site_url:`; the rest of the file is unchanged. - -- [ ] **Step 3: Verify the edit** - -```bash -head -3 mkdocs.yml -``` - -Expected: -```yaml -site_name: faststream-outbox -site_url: https://faststream-outbox.modern-python.org/ -repo_url: https://github.com/modern-python/faststream-outbox -``` - -- [ ] **Step 4: Verify mkdocs still builds** - -Run: `uvx --with-requirements docs/requirements.txt mkdocs build --strict --site-dir /tmp/mkdocs-smoke && grep -c 'canonical' /tmp/mkdocs-smoke/index.html && rm -rf /tmp/mkdocs-smoke` - -Expected: build succeeds, grep returns `1` (the canonical link tag is now emitted in `index.html`). - -- [ ] **Step 5: Commit** - -```bash -git add mkdocs.yml -git commit -m "docs: set mkdocs site_url for canonical links and sitemap - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -## Task 4: Add `.github/workflows/docs.yml` - -**Files:** -- Create: `.github/workflows/docs.yml` - -- [ ] **Step 1: Create the workflow** - -Create `.github/workflows/docs.yml` with exactly this content (single trailing newline at EOF): - -```yaml -name: Deploy Docs - -on: - push: - branches: [main] - paths: - - "docs/**" - - "mkdocs.yml" - - ".github/workflows/docs.yml" - workflow_dispatch: - -concurrency: - group: docs-deploy - cancel-in-progress: true - -permissions: - contents: write - -jobs: - deploy: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v6 - with: - fetch-depth: 0 - - uses: extractions/setup-just@v4 - - uses: astral-sh/setup-uv@v8.2.0 - - run: just docs-deploy -``` - -- [ ] **Step 2: Verify YAML parses** - -Run: `python3 -c "import yaml, sys; yaml.safe_load(open('.github/workflows/docs.yml')); print('OK')"` - -Expected: prints `OK`. (Any YAML error throws and exits non-zero.) - -- [ ] **Step 3: Verify load-bearing fields are present** - -Run: -```bash -python3 -c " -import yaml -d = yaml.safe_load(open('.github/workflows/docs.yml')) -assert d['name'] == 'Deploy Docs' -assert d['concurrency']['group'] == 'docs-deploy' -assert d['concurrency']['cancel-in-progress'] is True -assert d['permissions']['contents'] == 'write' -assert d['jobs']['deploy']['steps'][0]['with']['fetch-depth'] == 0 -assert d['jobs']['deploy']['steps'][-1]['run'] == 'just docs-deploy' -print('OK') -" -``` - -Expected: prints `OK`. If any assert fails, the YAML was mistyped — fix and re-run. - -- [ ] **Step 4: Commit** - -```bash -git add .github/workflows/docs.yml -git commit -m "ci: add mkdocs gh-deploy workflow - -Triggers on push to main when docs/, mkdocs.yml, or this workflow -change. Concurrency group serializes deploys; contents:write needed -for gh-pages branch push; fetch-depth: 0 needed for gh-deploy history. - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -## Task 5: Delete `.readthedocs.yaml` - -**Files:** -- Delete: `.readthedocs.yaml` - -- [ ] **Step 1: Confirm the file exists and matches expectations** - -```bash -cat .readthedocs.yaml -``` - -Expected: the 14-line file (version 2, ubuntu-22.04, python 3.10, mkdocs config). If contents have drifted from the spec snapshot, pause and re-read the spec. - -- [ ] **Step 2: Delete the file** - -```bash -git rm .readthedocs.yaml -``` - -Expected: `rm '.readthedocs.yaml'`, working tree shows the deletion staged. - -- [ ] **Step 3: Verify** - -```bash -ls .readthedocs.yaml 2>&1; git status --short -``` - -Expected: `ls` reports the file does not exist; `git status --short` shows `D .readthedocs.yaml`. - -- [ ] **Step 4: Commit** - -```bash -git commit -m "ci: remove Read the Docs config - -Docs now deploy from .github/workflows/docs.yml. RTD project will be -archived in the RTD UI after the new URL is live (out-of-repo step). - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -## Task 6: Update `README.md` URLs - -**Files:** -- Modify: `README.md` (4 line edits — lines 52, 84, 90, 110) - -- [ ] **Step 1: Replace the three `/en/latest/` deep links** - -Three substitutions in `README.md`. Each replaces a substring; do them in order. Use Edit (string replacement), not regex. - -Edit 1 — line 52, relay tutorial: -- Old: `https://faststream-outbox.readthedocs.io/en/latest/usage/relay/` -- New: `https://faststream-outbox.modern-python.org/usage/relay/` - -Edit 2 — line 84, dead-letter queue: -- Old: `https://faststream-outbox.readthedocs.io/en/latest/usage/dlq/` -- New: `https://faststream-outbox.modern-python.org/usage/dlq/` - -Edit 3 — line 90, "How it works": -- Old: `https://faststream-outbox.readthedocs.io/en/latest/introduction/how-it-works/` -- New: `https://faststream-outbox.modern-python.org/introduction/how-it-works/` - -- [ ] **Step 2: Replace the bare-domain link** - -Edit 4 — line 110, top-level Documentation header: -- Old: `https://faststream-outbox.readthedocs.io` -- New: `https://faststream-outbox.modern-python.org` - -(Order matters: do this AFTER the three `/en/latest/` edits. Otherwise the bare-domain string also matches inside the deeper URLs and you'd over-replace.) - -- [ ] **Step 3: Verify no `readthedocs` references remain** - -Run: `grep -n readthedocs README.md` - -Expected: no output (exit code 1 from grep, which is fine). - -- [ ] **Step 4: Verify the four new URLs are in place** - -Run: `grep -nc "modern-python.org" README.md` - -Expected: `4`. - -- [ ] **Step 5: Commit** - -```bash -git add README.md -git commit -m "docs: point README links at modern-python.org - -Replaces readthedocs.io/en/latest// with -modern-python.org// for the three deep links (relay, dlq, -how-it-works) and the bare-domain Documentation header link. - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -## Task 7: Bump action pins in `.github/workflows/ci.yml` - -**Files:** -- Modify: `.github/workflows/ci.yml` (lines 17, 18, 19, 49, 50) - -- [ ] **Step 1: Confirm current pin lines** - -Run: `grep -n "@v[0-9]" .github/workflows/ci.yml` - -Expected: -``` -17: - uses: actions/checkout@v4 -18: - uses: extractions/setup-just@v2 -19: - uses: astral-sh/setup-uv@v3 -49: - uses: actions/checkout@v4 -50: - uses: astral-sh/setup-uv@v3 -``` - -- [ ] **Step 2: Bump `actions/checkout@v4` → `@v6`** - -Two occurrences (lines 17 and 49). Use a `replace_all` for the exact string `actions/checkout@v4` → `actions/checkout@v6`. - -- [ ] **Step 3: Bump `extractions/setup-just@v2` → `@v4`** - -One occurrence (line 18 — pytest job doesn't use just). Replace exact string `extractions/setup-just@v2` → `extractions/setup-just@v4`. - -- [ ] **Step 4: Bump `astral-sh/setup-uv@v3` → `@v8.2.0`** - -Two occurrences (lines 19 and 50). Use `replace_all` for `astral-sh/setup-uv@v3` → `astral-sh/setup-uv@v8.2.0`. - -- [ ] **Step 5: Verify all five pins updated** - -Run: `grep -n "@v[0-9]" .github/workflows/ci.yml` - -Expected: -``` -17: - uses: actions/checkout@v6 -18: - uses: extractions/setup-just@v4 -19: - uses: astral-sh/setup-uv@v8.2.0 -49: - uses: actions/checkout@v6 -50: - uses: astral-sh/setup-uv@v8.2.0 -``` - -Also verify no stale pins linger: - -```bash -grep -E "@v[234]\b" .github/workflows/ci.yml -``` - -Expected: no output (exit 1). - -- [ ] **Step 6: Verify YAML still parses** - -Run: `python3 -c "import yaml; yaml.safe_load(open('.github/workflows/ci.yml')); print('OK')"` - -Expected: `OK`. - -- [ ] **Step 7: Commit** - -```bash -git add .github/workflows/ci.yml -git commit -m "ci: bump action pins in ci.yml to drop Node.js 20 - -actions/checkout v4 -> v6 -extractions/setup-just v2 -> v4 -astral-sh/setup-uv v3 -> v8.2.0 - -Matches the baseline modern-di already runs. - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -## Task 8: Bump action pins in `.github/workflows/publish.yml` - -**Files:** -- Modify: `.github/workflows/publish.yml` (lines 12, 13, 14) - -- [ ] **Step 1: Confirm current pin lines** - -Run: `grep -n "@v[0-9]" .github/workflows/publish.yml` - -Expected: -``` -12: - uses: actions/checkout@v4 -13: - uses: extractions/setup-just@v2 -14: - uses: astral-sh/setup-uv@v3 -``` - -- [ ] **Step 2: Apply the three bumps** - -Three single-occurrence string replacements: - -- `actions/checkout@v4` → `actions/checkout@v6` -- `extractions/setup-just@v2` → `extractions/setup-just@v4` -- `astral-sh/setup-uv@v3` → `astral-sh/setup-uv@v8.2.0` - -- [ ] **Step 3: Verify** - -Run: `grep -n "@v[0-9]" .github/workflows/publish.yml` - -Expected: -``` -12: - uses: actions/checkout@v6 -13: - uses: extractions/setup-just@v4 -14: - uses: astral-sh/setup-uv@v8.2.0 -``` - -And: `python3 -c "import yaml; yaml.safe_load(open('.github/workflows/publish.yml')); print('OK')"` → `OK`. - -- [ ] **Step 4: Commit** - -```bash -git add .github/workflows/publish.yml -git commit -m "ci: bump action pins in publish.yml to drop Node.js 20 - -Matches the pins applied to ci.yml and docs.yml in this branch. - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -## Final verification - -- [ ] **Step 1: Run `just lint-ci` to catch formatting drift** - -Run: `just lint-ci` - -Expected: all four sub-steps pass (`eof-fixer`, `ruff format --check`, `ruff check --no-fix`, `ty check`). If `eof-fixer` flags any of the new files, run `just lint` once, then `git add` the fix into the appropriate prior commit via `git commit --amend` — or, if multiple files are touched, fold them into a single `chore: eof-fixer cleanup` commit at the end. - -- [ ] **Step 2: Verify the full diff against `main`** - -Run: `git log --oneline main..HEAD && git diff --stat main..HEAD` - -Expected: 8 commits (Tasks 1–8). The `--stat` summary should touch exactly these paths: - -``` -.github/workflows/ci.yml -.github/workflows/docs.yml -.github/workflows/publish.yml -.readthedocs.yaml -Justfile -README.md -docs/CNAME -mkdocs.yml -``` - -(8 entries; `.readthedocs.yaml` shows as a deletion.) - -- [ ] **Step 3: One final mkdocs strict build** - -Run: `uvx --with-requirements docs/requirements.txt mkdocs build --strict --site-dir /tmp/mkdocs-final && ls /tmp/mkdocs-final/CNAME && grep -q 'modern-python.org' /tmp/mkdocs-final/index.html && echo OK && rm -rf /tmp/mkdocs-final` - -Expected: prints `OK`. (Verifies: build is strict-clean, `CNAME` lands in the output, and the canonical URL points at the new domain.) - -- [ ] **Step 4: Push the branch** - -```bash -git push -u origin feat/mkdocs-github-pages -``` - -Expected: branch tracks `origin/feat/mkdocs-github-pages`. - -- [ ] **Step 5: Open the PR** - -```bash -gh pr create --title "Migrate docs hosting from Read the Docs to GitHub Pages" --body "$(cat <<'EOF' -## Summary - -- Adds `.github/workflows/docs.yml` mirroring `modern-di`'s pattern: `mkdocs gh-deploy --force` on push to `main` (paths-filtered), concurrency-serialized, `contents: write`. -- Adds `just docs-deploy` target driving the deploy via `uvx --with-requirements docs/requirements.txt mkdocs gh-deploy --force`. -- Adds `docs/CNAME` and `mkdocs.yml` `site_url` for the custom domain `faststream-outbox.modern-python.org`. -- Deletes `.readthedocs.yaml`; updates the four `readthedocs.io` links in `README.md` to `modern-python.org` equivalents. -- Bumps action pins in `ci.yml` and `publish.yml` to the same baseline the new `docs.yml` uses (`checkout@v6`, `setup-just@v4`, `setup-uv@v8.2.0`). - -Spec: `planning/specs/2026-06-09-mkdocs-github-pages-design.md` - -## Out-of-repo steps after merge - -1. DNS: CNAME `faststream-outbox.modern-python.org` → `modern-python.github.io`. -2. After the first workflow run creates `gh-pages`, set Settings → Pages → Source = `gh-pages` branch, `/` root. -3. Tick "Enforce HTTPS" once the cert provisions (~5 min). -4. Archive the `faststream-outbox` project on readthedocs.io. - -## Test plan - -- [ ] PR CI is green (`ci.yml` lint + pytest jobs run with bumped pins). -- [ ] After merge, the `Deploy Docs` workflow runs and creates the `gh-pages` branch. -- [ ] `https://faststream-outbox.modern-python.org/` returns the docs site once DNS + Pages settings are in place. -- [ ] All four `README.md` links resolve to working pages on the new domain. -- [ ] A subsequent push to `main` touching `docs/` re-deploys. - -🤖 Generated with [Claude Code](https://claude.com/claude-code) -EOF -)" -``` - -Expected: prints the PR URL. - ---- - -## Self-review - -**Spec coverage:** - -| Spec section | Plan task | -| --- | --- | -| 1. `docs.yml` workflow | Task 4 | -| 2. `just docs-deploy` | Task 1 | -| 3. `docs/CNAME` | Task 2 | -| 4. `mkdocs.yml` `site_url` | Task 3 | -| 5. Delete `.readthedocs.yaml` | Task 5 | -| 6. README URL updates | Task 6 | -| 7. `ci.yml` pin bumps | Task 7 | -| 8. `publish.yml` pin bumps | Task 8 | -| Operations (out of repo) | PR description (Step 5 of Final verification) | -| Sequencing / rollback | Implicit — all 8 land as one PR; Tasks 1–4 add the new path, Task 5 removes RTD config, Task 6 swaps README links, Tasks 7–8 update pins. Reverting the PR restores RTD + old pins. | - -All eight spec changes have a dedicated task; out-of-repo ops are surfaced in the PR body so the merging maintainer sees them. - -**Placeholder check:** No TBDs, no "implement later", every code/YAML block is complete, every command has expected output. - -**Consistency check:** -- `docs-deploy` Justfile target name matches across Task 1, Task 4 (the workflow's `run: just docs-deploy` line), and the spec. -- Action pin versions match across Tasks 4, 7, 8 (`checkout@v6`, `setup-just@v4`, `setup-uv@v8.2.0`). -- The domain string `faststream-outbox.modern-python.org` matches across Tasks 2, 3, 6, and the PR body. diff --git a/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/design.md b/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation.md similarity index 100% rename from planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/design.md rename to planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation.md diff --git a/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/plan.md b/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/plan.md deleted file mode 100644 index 981a79e..0000000 --- a/planning/changes/2026-06-09.02-drain-test-flaky-fetch-observation/plan.md +++ /dev/null @@ -1,263 +0,0 @@ -# Drain Test Flake Fix Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Remove a Python 3.13/3.14 coverage flake in `test_drain_finishes_inflight_rows_before_returning` by replacing the SQL-polling wait helper with observation of the broker's existing `fetched` recorder event. - -**Architecture:** Test-only refactor inside `tests/test_integration.py`. One test is modified to register a `metrics_recorder` closure that accumulates the `count` field from `fetched` events, and waits on a predicate over that counter via the existing generic `_wait_until` helper. The single-caller helper `_wait_until_claimed` is then deleted. No production code is touched. - -**Tech Stack:** Python 3.13/3.14, pytest, pytest-asyncio, SQLAlchemy async, FastStream, existing `OutboxBroker.metrics_recorder` callable seam. - -**Spec:** `planning/specs/2026-06-09-drain-test-flaky-fetch-observation-design.md` - ---- - -## File Structure - -- Modify: `tests/test_integration.py` - - Add one import (`Mapping` from `collections.abc`). - - Refactor `test_drain_finishes_inflight_rows_before_returning` (currently lines 1130-1158) to register a recorder closure and use `_wait_until` instead of `_wait_until_claimed`. - - Delete `_wait_until_claimed` (currently lines 1112-1127) — single caller goes away. - -No other files change. No new helpers introduced. - ---- - -## Task 1: Baseline confirmation - -**Files:** -- Read-only: `tests/test_integration.py`, `faststream_outbox/subscriber/usecase.py:315-326` - -- [ ] **Step 1: Confirm baseline test passes on 3.13 with current code** - -Run: - -```bash -just test tests/test_integration.py::test_drain_finishes_inflight_rows_before_returning -v --no-cov -``` - -Expected: `1 passed`. This is the pre-refactor green state; the refactor must keep it green. - -- [ ] **Step 2: Confirm the `fetched` event field name** - -Read `faststream_outbox/subscriber/usecase.py` around line 323. Verify the emit looks like: - -```python -self._emit_metric( - "fetched", - {**self._base_tags(self._queues[0] if self._queues else ""), "count": len(rows)}, -) -``` - -The field used by the refactor is `count` and it is an `int`. If this differs from what is shown, stop and reconcile against the spec before continuing — the refactor depends on this field. - -- [ ] **Step 3: Confirm there are no other callers of `_wait_until_claimed`** - -Run: - -```bash -grep -rn "_wait_until_claimed" tests/ faststream_outbox/ planning/ -``` - -Expected output: exactly two matches inside `tests/test_integration.py` (the definition and its single call). If any other reference exists, stop and update the plan — additional callers must also be migrated. - ---- - -## Task 2: Refactor the drain test to use recorder observation - -**Files:** -- Modify: `tests/test_integration.py:7` (add `Mapping` to existing imports) -- Modify: `tests/test_integration.py:1130-1158` (rewrite test body) -- Delete: `tests/test_integration.py:1112-1127` (entire `_wait_until_claimed` function) - -- [ ] **Step 1: Add `Mapping` import** - -Modify the top-of-file imports. Today line 7 reads: - -```python -from typing import Any -``` - -Add a `from collections.abc import Mapping` line immediately before it, so the typing-style imports are: - -```python -from collections.abc import Mapping -from typing import Any -from unittest import mock -``` - -Rationale: the recorder callable's `fields` parameter is typed as `Mapping[str, Any]` (matches the production callable contract). Global imports only (per CLAUDE.md project convention — `feedback_no_local_imports`). - -- [ ] **Step 2: Rewrite the test body** - -Replace `test_drain_finishes_inflight_rows_before_returning` (currently `tests/test_integration.py:1130-1158`) with the version below. The header docstring, subscriber decorator kwargs, and final asserts are unchanged — only the broker construction and the wait line change. - -```python -async def test_drain_finishes_inflight_rows_before_returning( - pg_engine: AsyncEngine, - outbox_table: Table, -) -> None: - """Rows claimed by fetch must run to completion when broker.stop() is called.""" - fetched_total = 0 - - def recorder(event: str, fields: Mapping[str, Any]) -> None: - nonlocal fetched_total - if event == "fetched": - fetched_total += fields["count"] - - broker = OutboxBroker( - pg_engine, - outbox_table=outbox_table, - graceful_timeout=5.0, - metrics_recorder=recorder, - ) - handled: list[int] = [] - - @broker.subscriber( - "orders", - min_fetch_interval=0.02, - max_fetch_interval=0.05, - max_workers=4, - fetch_batch_size=20, - ) - async def handle(body: dict) -> None: - await asyncio.sleep(0.1) - handled.append(body["i"]) - - session_factory = async_sessionmaker(pg_engine, expire_on_commit=False) - async with broker: - async with session_factory() as session, session.begin(): - for i in range(20): - await broker.publish({"i": i}, queue="orders", session=session) - await _wait_until(lambda: fetched_total >= 20, timeout=3.0) - await broker.stop() - - assert sorted(handled) == list(range(20)) - assert await _row_count(pg_engine, outbox_table) == 0 -``` - -Key differences from the current test: - -1. New `fetched_total` counter + `recorder` closure declared at the top of the test function. -2. `OutboxBroker(...)` call gains `metrics_recorder=recorder`. -3. The wait line changes from `await _wait_until_claimed(pg_engine, outbox_table, timeout=3.0)` to `await _wait_until(lambda: fetched_total >= 20, timeout=3.0)`. -4. Everything else (subscriber decorator, handler body, session block, asserts) is byte-for-byte unchanged. - -- [ ] **Step 3: Delete `_wait_until_claimed`** - -Delete the entire function definition at `tests/test_integration.py:1112-1127`: - -```python -async def _wait_until_claimed( - pg_engine: AsyncEngine, - outbox_table: Table, - *, - timeout: float, # noqa: ASYNC109 -) -> None: - deadline = asyncio.get_event_loop().time() + timeout - stmt = select(outbox_table.c.id).where(outbox_table.c.acquired_token.is_(None)) - while asyncio.get_event_loop().time() < deadline: - async with pg_engine.connect() as conn: - unclaimed = (await conn.execute(stmt)).fetchall() - if not unclaimed: - return - await asyncio.sleep(0.02) - msg = "fetch never claimed every row" # pragma: no cover - raise AssertionError(msg) # pragma: no cover -``` - -After deletion, the line that previously preceded this function (the closing line of the test above it) should be directly followed by two blank lines, then `async def test_drain_finishes_inflight_rows_before_returning(...)`. - -- [ ] **Step 4: Run the modified test in isolation** - -Run: - -```bash -just test tests/test_integration.py::test_drain_finishes_inflight_rows_before_returning -v --no-cov -``` - -Expected: `1 passed`. If it fails, the most likely causes are: -- Typo in the `fetched_total` accumulation closure. -- The `fetched` event payload's `count` field name is wrong (re-check Task 1 Step 2). -- `Mapping` import missing or misspelled. - -Fix and re-run until green before continuing. - -- [ ] **Step 5: Run the full integration test file with coverage** - -Run: - -```bash -just test tests/test_integration.py -v -``` - -Expected: all tests pass, coverage report shows `tests/test_integration.py` at 100%, no `Required test coverage of 100% not reached` failure. The total stmt count for `tests/test_integration.py` should drop by ~16 (the deleted helper body). - -If a different test now reports a coverage miss, stop — the change has had an unintended side effect. Do not paper over with `pragma: no cover`; surface to the user. - -- [ ] **Step 6: Run lint** - -Run: - -```bash -just lint -``` - -Expected: zero issues. The only new construct is a `Mapping` import that ruff/ty should accept without complaint. If ruff flags `Mapping` as unused, the recorder signature was not updated correctly. - -- [ ] **Step 7: Commit** - -```bash -git add tests/test_integration.py -git commit -m "$(cat <<'EOF' -test: drain test waits via fetched recorder, not SQL poll - -Removes _wait_until_claimed and migrates the lone caller -(test_drain_finishes_inflight_rows_before_returning) to observe -the broker's fetched recorder event. The SQL-poll variant had -a Python 3.13/3.14 timing race that left tests/test_integration.py:1125 -uncovered on 3.14, tripping the 100% fail-under gate. - -Spec: planning/specs/2026-06-09-drain-test-flaky-fetch-observation-design.md -EOF -)" -``` - ---- - -## Task 3: CI verification - -**Files:** None modified. - -- [ ] **Step 1: Push the branch and open a PR** - -Standard project flow. Once the PR is open, CI runs both the 3.13 and 3.14 matrix legs of `_checks.yml`. - -- [ ] **Step 2: Confirm both Python matrix legs are green** - -Watch for: - -``` -checks / pytest (3.13) ✓ -checks / pytest (3.14) ✓ -checks / lint ✓ -``` - -Both pytest jobs must report 100% coverage with no `FAIL Required test coverage of 100% not reached`. If 3.14 still misses a line, it is a *different* flake than the one this plan fixes — capture the new symptom and stop. Do not retry. - -- [ ] **Step 3: Merge** - -Standard merge once green and reviewed. - ---- - -## Out of Scope - -The following items are explicitly excluded from this plan (matches the spec's Out of Scope section): - -- Fixing the upstream FastStream `RuntimeWarning` ("Error `{e!r}` occurred at AST parsing") seen on 3.14. -- Any change to the `scheduled.yml` workflow's issue-creation flow. -- Any change to the generic `_wait_until` helper. -- Any production code change. - -If any of these surface as blockers during execution, stop and re-scope rather than expanding the plan. diff --git a/planning/changes/2026-06-10.01-planning-conventions/design.md b/planning/changes/2026-06-10.01-planning-conventions.md similarity index 100% rename from planning/changes/2026-06-10.01-planning-conventions/design.md rename to planning/changes/2026-06-10.01-planning-conventions.md diff --git a/planning/changes/2026-06-10.02-docs-landing-and-comparison/design.md b/planning/changes/2026-06-10.02-docs-landing-and-comparison.md similarity index 100% rename from planning/changes/2026-06-10.02-docs-landing-and-comparison/design.md rename to planning/changes/2026-06-10.02-docs-landing-and-comparison.md diff --git a/planning/changes/2026-06-10.02-docs-landing-and-comparison/plan.md b/planning/changes/2026-06-10.02-docs-landing-and-comparison/plan.md deleted file mode 100644 index b26d1cf..0000000 --- a/planning/changes/2026-06-10.02-docs-landing-and-comparison/plan.md +++ /dev/null @@ -1,243 +0,0 @@ -# docs-landing-and-comparison — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use -> superpowers:subagent-driven-development (recommended) or -> superpowers:executing-plans to implement this plan task-by-task. Steps -> use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Rewrite `docs/index.md` as a real landing page, reshape -`mkdocs.yml` nav into Getting-started / Concepts / Guides / Reference, and -add a new Comparison page under a new `docs/concepts/` directory. - -**Spec:** [`planning/active/2026-06-10-docs-landing-and-comparison-design.md`](./2026-06-10-docs-landing-and-comparison-design.md) - -**Branch:** `docs/landing-and-comparison` - -**Commit strategy:** Per-task commits. Task 4 is verification-only and -produces no commit. - ---- - -### Task 1: Branch + create the Comparison page + cross-link callouts - -**Files:** -- Create: `docs/concepts/comparison.md` -- Modify: `docs/introduction/how-it-works.md` -- Modify: `docs/usage/relay.md` - -Creates the new page that the rewritten landing (Task 3) and the -cross-link callouts will both reference. Doing the callouts in the same -commit keeps "the link target lands together with its links." - -- [ ] **Step 1: Create the feature branch from `main`** - - Run: `git switch -c docs/landing-and-comparison` - Expected: `Switched to a new branch 'docs/landing-and-comparison'`. - -- [ ] **Step 2: Create `docs/concepts/comparison.md`** - - Create the directory `docs/concepts/` and inside it write - `comparison.md` with the six sections defined in [spec §3 - ](./2026-06-10-docs-landing-and-comparison-design.md#3-new-file-docsconceptscomparisonmd): - - 1. `faststream-outbox` vs writing your own - 2. vs CDC (Debezium, logical replication) - 3. vs Kafka transactions (or Rabbit publisher confirms) - 4. vs plain PG-NOTIFY - 5. vs Celery + DB result backend - 6. vs FastStream + `KafkaBroker` / `RabbitBroker` directly - - Each section ends with a one-line **TL;DR** verdict. Section 2 sources - its CDC analysis from the existing memory entry - `cdc_wal_rejected.md` (2026-05-07); reflect that the reassessment was - done on that date. - - Cross-link inline into existing reference pages where natural — - e.g. the "vs writing your own" section can name the - `subscriber`/`publisher`/`dlq` pages whose mechanisms the user would - re-implement. - -- [ ] **Step 3: Add a one-line callout in `docs/introduction/how-it-works.md`** - - In the `## The transactional outbox pattern` section, add at the end - of the final paragraph: - - > See [Comparison](../concepts/comparison.md) for when CDC or Kafka - > transactions are the better fit. - -- [ ] **Step 4: Add a one-line callout in `docs/usage/relay.md`** - - At the end of the intro paragraph (before the first `## Why an outbox - relay`), add: - - > If you don't have a database write to atomically commit alongside, - > use the foreign broker directly — see - > [Comparison](../concepts/comparison.md). - -- [ ] **Step 5: Smoke-build the docs locally** - - Run: `uvx --with-requirements docs/requirements.txt mkdocs build --strict` - Expected: build completes, no warnings. The new page does not yet - appear in the sidebar (Task 2 wires it in), but `--strict` will catch - any broken cross-link from the two callouts. - -- [ ] **Step 6: Commit** - - ```bash - git add docs/concepts/comparison.md docs/introduction/how-it-works.md docs/usage/relay.md - git commit -m "docs: add comparison page and cross-link callouts - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 2: Reshape `mkdocs.yml` nav - -**Files:** -- Modify: `mkdocs.yml` - -Replace the two-section nav (`Introduction` / `Usage`) with the -four-section structure in [spec §2 -](./2026-06-10-docs-landing-and-comparison-design.md#2-reshape-mkdocsyml-nav). -No file paths change; only the `nav:` block. - -- [ ] **Step 1: Edit `mkdocs.yml`** - - Replace the existing `nav:` block (currently `Overview` + `Introduction` - + `Usage`) with: - - ```yaml - nav: - - Overview: index.md - - Getting started: - - Installation: introduction/installation.md - - Basic usage: usage/basic.md - - Concepts: - - How it works: introduction/how-it-works.md - - Comparison: concepts/comparison.md - - Guides: - - FastAPI integration: usage/fastapi.md - - Relay to Kafka / RabbitMQ / NATS: usage/relay.md - - Timers: usage/timers.md - - Testing: usage/testing.md - - Schema validation: usage/schema-validation.md - - Reference: - - Subscriber: usage/subscriber.md - - Publisher: usage/publisher.md - - Router: usage/router.md - - Dead-letter queue: usage/dlq.md - - Observability: usage/observability.md - ``` - - Load-bearing details (per spec): "Relay to Kafka / RabbitMQ / NATS" is - the nav *label* only — the file stays at `usage/relay.md` and its H1 - is unchanged. Don't touch `mkdocs.yml` outside the `nav:` block. - -- [ ] **Step 2: Smoke-build and visually scan** - - Run: `uvx --with-requirements docs/requirements.txt mkdocs build --strict` - Expected: build clean. Open `site/index.html` (or run `mkdocs serve` - briefly) and confirm the sidebar shows four sections in the expected - order, all eleven existing pages plus the new Comparison page are - present, none dropped. - -- [ ] **Step 3: Commit** - - ```bash - git add mkdocs.yml - git commit -m "docs: reshape nav into Getting started / Concepts / Guides / Reference - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 3: Rewrite `docs/index.md` as a landing page - -**Files:** -- Modify: `docs/index.md` - -Replace today's 22-line TOC with the four-block landing per [spec §1 -](./2026-06-10-docs-landing-and-comparison-design.md#1-rewrite-docsindexmd). -The new page is the front door — every other doc is reachable from one -of the four blocks. - -- [ ] **Step 1: Replace `docs/index.md` contents** - - Structure: - - - **Block A** — value-prop paragraph (lightly tightened version of - today's first paragraph). - - **Block B** — "Use it when / Reach for something else when" two - short bulleted lists. Concrete, binary, no hedging. See spec §1 - Block B for example wording. - - **Block C** — decision-tree table mapping user intent → starting - page. See spec §1 Block C for the table. - - **Block D** — documentation map grouped by the four nav sections - (Getting started / Concepts / Guides / Reference). One line per - page, terse description after the link. - - All internal links use the relative path from `docs/index.md`: - `introduction/installation.md`, `concepts/comparison.md`, - `usage/fastapi.md`, etc. - -- [ ] **Step 2: Smoke-build** - - Run: `uvx --with-requirements docs/requirements.txt mkdocs build --strict` - Expected: clean. Block C and Block D each contain ~5–10 internal - links; `--strict` catches any broken target. - -- [ ] **Step 3: Commit** - - ```bash - git add docs/index.md - git commit -m "docs: rewrite landing page with value prop, decision tree, doc map - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 4: Verify - -**Files:** none modified; no commit produced. - -Final pass before opening the PR. - -- [ ] **Step 1: Full strict build** - - Run: `uvx --with-requirements docs/requirements.txt mkdocs build --strict` - Expected: clean. - -- [ ] **Step 2: Lint pass** - - Run: `just lint` - Expected: `eof-fixer`, `ruff format`, `ruff check`, `ty check` all - pass. Markdown EOF + YAML formatting on `mkdocs.yml` are the only - things touched in this PR. - -- [ ] **Step 3: Manual sidebar scan** - - Run: `uvx --with-requirements docs/requirements.txt mkdocs serve` - Open the served site and confirm: - - - Sidebar shows four sections in order: Getting started, Concepts, - Guides, Reference. - - Every page from the pre-change nav is still reachable. - - The new Comparison page appears under Concepts and renders cleanly. - - The decision-tree table on the landing page routes correctly into - `usage/fastapi.md`, `usage/relay.md`, `introduction/how-it-works.md`, - `concepts/comparison.md`, `introduction/installation.md`, and - `usage/basic.md`. - - The "Relay to a foreign broker" H1 inside `usage/relay.md` is - unchanged (spec invariant — only the nav label changed). - -- [ ] **Step 4: Open the PR** - - Stop. Hand off to `superpowers:requesting-code-review` / - `superpowers:finishing-a-development-branch` per the standard - workflow. The convention in [`planning/README.md`](../README.md): - on merge, both this plan and its paired spec move to - `planning/archived/` and get `status: shipped`, `pr:`, and - `outcome:` filled. diff --git a/planning/changes/2026-06-11.01-operator-pages/design.md b/planning/changes/2026-06-11.01-operator-pages.md similarity index 100% rename from planning/changes/2026-06-11.01-operator-pages/design.md rename to planning/changes/2026-06-11.01-operator-pages.md diff --git a/planning/changes/2026-06-11.01-operator-pages/plan.md b/planning/changes/2026-06-11.01-operator-pages/plan.md deleted file mode 100644 index e0e1e6d..0000000 --- a/planning/changes/2026-06-11.01-operator-pages/plan.md +++ /dev/null @@ -1,463 +0,0 @@ -# operator-pages — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use -> superpowers:subagent-driven-development (recommended) or -> superpowers:executing-plans to implement this plan task-by-task. Steps -> use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Add three operator-facing pages (Production checklist, -Troubleshooting playbook, Alembic migrations) under a new -`docs/operations/` directory and a fifth top-level **Operations** nav -section, with five one-line cross-link callouts from existing reference -pages. - -**Spec:** [`planning/active/2026-06-11-operator-pages-design.md`](./2026-06-11-operator-pages-design.md) - -**Branch:** `docs/operator-pages` - -**Commit strategy:** Per-task commits. Tasks 2–7 each produce a -commit and each leaves the docs site `--strict`-buildable. Task 8 is -verification-only. - ---- - -### Task 1: Branch + commit spec, plan, and README index update - -**Files:** -- Create: `planning/active/2026-06-11-operator-pages-design.md` (already drafted in working tree) -- Create: `planning/active/2026-06-11-operator-pages-plan.md` (this file, already drafted) -- Modify: `planning/README.md` - -Land the planning artifacts and surface them in the index before -touching any docs content. - -- [ ] **Step 1: Create the feature branch from `main`** - - Run: `git switch -c docs/operator-pages` - Expected: `Switched to a new branch 'docs/operator-pages'`. - -- [ ] **Step 2: Update `planning/README.md`** - - Replace the `## Active` block. Current state (post-#52): - - ```markdown - ## Active - - _None._ - ``` - - Becomes: - - ```markdown - ## Active - - - **[operator-pages](active/2026-06-11-operator-pages-design.md)** - — Three new pages under a new `docs/operations/` section: - Production checklist, Troubleshooting playbook, Alembic - migrations. The B follow-on from #50. - ``` - -- [ ] **Step 3: Commit** - - ```bash - git add planning/active/2026-06-11-operator-pages-design.md \ - planning/active/2026-06-11-operator-pages-plan.md \ - planning/README.md - git commit -m "docs: spec + plan for operator pages - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 2: Alembic autogenerate spike → update spec - -**Files:** -- Modify: `planning/active/2026-06-11-operator-pages-design.md` (paste captured output into §4a) - -The spec §4a calls for the **literal `alembic revision --autogenerate` -output** against `make_outbox_table()`. Capture it now so Task 6 has a -verbatim sample to paste, and so the spec stays the canonical reference -for what autogenerate produces. - -The spike can be done two ways; pick whichever you find faster. - -**Option A — programmatic.** Write a one-off Python script under -`/tmp/` that uses Alembic's autogenerate APIs (the same APIs -`broker.validate_schema()` already uses internally — see -`src/faststream_outbox/_schema_validation.py`). Construct a `MetaData` -holding `make_outbox_table(metadata, table_name="outbox")`, attach to a -running Postgres connection, call `produce_migrations` / -`render_python_code`, print the rendered upgrade ops. - -**Option B — actual Alembic env.** Set up a minimal Alembic project -under `/tmp/alembic-spike/`: `alembic.ini`, `env.py` that imports -`make_outbox_table` and sets `target_metadata`, run -`alembic revision --autogenerate -m "initial"` against an empty -Postgres, open the generated migration file. - -Either way, Postgres must be running. The repo's `compose.yml` already -provides one: `docker compose up -d postgres` (port 5432). - -- [ ] **Step 1: Start Postgres if not running** - - Run: `docker compose up -d postgres` - Expected: container ready; `pg_isready` succeeds on `localhost:5432`. - -- [ ] **Step 2: Run the spike (Option A or B)** - - Capture the rendered Python upgrade ops (`op.create_table(...)`, - `op.create_index(...)` calls) plus their column lists and index - predicates. - -- [ ] **Step 3: Paste into spec §4a** - - In `planning/active/2026-06-11-operator-pages-design.md` §4a - "Initial migration", under a new "Captured autogenerate output" - subsection, paste the literal rendered Python verbatim inside a - ` ```python ` block. Add a one-line preamble noting the - SQLAlchemy + Alembic versions used (read from - `uv pip list | grep -E "sqlalchemy|alembic"`). - -- [ ] **Step 4: Verify the partial-index predicates match the spec's - prediction** - - The spec predicts three partial indexes: - - `(queue, next_attempt_at) WHERE acquired_token IS NULL` - - `(queue, acquired_at) WHERE acquired_token IS NOT NULL` - - unique `(queue, timer_id) WHERE timer_id IS NOT NULL` - - If the captured output differs (extra index, different predicate), - STOP and flag — the spec's §4a section needs updating before Task 6 - can write the page truthfully. - -- [ ] **Step 5: Commit** - - ```bash - git add planning/active/2026-06-11-operator-pages-design.md - git commit -m "spec: capture alembic autogenerate output for operator-pages - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 3: New Operations nav section + three stub pages - -**Files:** -- Create: `docs/operations/checklist.md` (stub) -- Create: `docs/operations/troubleshooting.md` (stub) -- Create: `docs/operations/alembic.md` (stub) -- Modify: `mkdocs.yml` - -Land the new directory, three stub files (H1 only), and the new -nav section all at once. `mkdocs build --strict` succeeds because -each file in the nav exists; the stubs render as nearly-empty pages -but that is fine until Tasks 4–6 fill them in. - -- [ ] **Step 1: Create three stub files** - - Each contains nothing but the H1 expected for the page: - - - `docs/operations/checklist.md` → `# Production checklist` - - `docs/operations/troubleshooting.md` → `# Troubleshooting` - - `docs/operations/alembic.md` → `# Alembic migrations` - -- [ ] **Step 2: Add the Operations nav section to `mkdocs.yml`** - - Append after the existing `Reference:` block (per spec §1): - - ```yaml - - Operations: - - Production checklist: operations/checklist.md - - Troubleshooting: operations/troubleshooting.md - - Alembic migrations: operations/alembic.md - ``` - -- [ ] **Step 3: Smoke-build** - - Run: `just docs-build` - Expected: clean. Five sections in sidebar, all three new pages - present (as near-empty stubs). - -- [ ] **Step 4: Commit** - - ```bash - git add docs/operations/ mkdocs.yml - git commit -m "docs: scaffold Operations nav section and three operator pages - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 4: Production checklist content - -**Files:** -- Modify: `docs/operations/checklist.md` - -Write the full checklist per [spec §2 -](./2026-06-11-operator-pages-design.md#2-production-checklist-docsoperationschecklistmd). -Six sections in this exact order: Sizing → Subscribers → DLQ → Drain & -lifecycle → Schema → Observability. Each item is a checkbox bullet -(`- [ ] **...**`) of one-to-two lines plus a relative link into the -existing reference page that owns the underlying detail. - -The spec § lists all sixteen items verbatim; transcribe them rather -than inventing new ones. The page's purpose is to surface existing -references, not to re-document them. - -- [ ] **Step 1: Write `docs/operations/checklist.md`** - - Header structure: - - ```markdown - # Production checklist - - Scannable scaffold of pre-launch checks. Each item is one to two - lines; the link points at the existing reference page that owns the - full story. - - ## Sizing - ... - ## Subscribers - ... - ## DLQ - ... - ## Drain & lifecycle - ... - ## Schema - ... - ## Observability - ... - ``` - -- [ ] **Step 2: Smoke-build** - - Run: `just docs-build` - Expected: clean. All `../usage/...` relative links resolve. - -- [ ] **Step 3: Commit** - - ```bash - git add docs/operations/checklist.md - git commit -m "docs: production checklist content - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 5: Troubleshooting playbook content - -**Files:** -- Modify: `docs/operations/troubleshooting.md` - -Write the full playbook per [spec §3 -](./2026-06-11-operator-pages-design.md#3-troubleshooting-docsoperationstroubleshootingmd). -Eleven symptoms in this order: - -1. `event=lease_lost` recurring in logs -2. Outbox row count grows + `lease_lost` spike -3. Outbox row count grows, no `lease_lost` -4. Idle dispatch latency > `max_fetch_interval` -5. Subscriber blocks at `broker.start()` -6. Duplicate handler invocations -7. Rolling deploy leaks rows -8. `activate_in` / `activate_at` fires immediately in tests -9. `AckPolicy.ACK_FIRST` raises `ValueError` at registration -10. `OutboxResponse(...)` + foreign-publisher decorator gets nacked -11. `validate_schema()` raises `ImportError` - -- [ ] **Step 1: Write the TOC table at the top** - - Two-column table (Symptom | Likely cause) per spec §3. Each row is - also an anchor link to the `##` heading below. - -- [ ] **Step 2: Write each `##` subsection** - - Use the five-field shape per spec (Symptom / Likely cause / - Diagnose / Fix / Reference). The spec's worked example for - `event=lease_lost` is the canonical template; match its tone and - field order for the other ten. - - Reference fields link into the existing reference pages — keep - links relative (`../usage/...`, `../introduction/...`). - -- [ ] **Step 3: Smoke-build** - - Run: `just docs-build` - Expected: clean. TOC table anchors resolve to the `##` headings - below. - -- [ ] **Step 4: Commit** - - ```bash - git add docs/operations/troubleshooting.md - git commit -m "docs: troubleshooting playbook content - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 6: Alembic migrations page content - -**Files:** -- Modify: `docs/operations/alembic.md` - -Write the full page per [spec §4 -](./2026-06-11-operator-pages-design.md#4-alembic-migrations-docsoperationsalembicmd). -Four sections: - -- §4a Initial migration — paste the **captured autogenerate output - from Task 2** verbatim, then annotate inline why each piece (table, - partial index, partial unique index, column type) exists. The - annotations explain that each partial index's predicate is - load-bearing for fetch performance. -- §4b Adding the DLQ after the fact — describe the additive nature - (only `create_table` + a single non-unique index; no `alter_table` - on the outbox), and show the analogous autogenerate output for - `make_dlq_table(metadata, table_name="outbox_dlq")` (re-run the - spike for this case during this task, or include a "run - autogenerate against your `MetaData` to see the exact output for - your DLQ table name" placeholder if a re-spike is impractical). -- §4c Drift detection in CI — show the small standalone - `validate_schema()` script from spec §4c verbatim. Explain why this - is opt-in for `/health` and belongs between `alembic upgrade head` - and the deploy step in CI. -- §4d DLQ retention via partition drop — walk through the Alembic - ops for converting the DLQ from a plain table to one partitioned - by `failed_at`, plus the monthly cron SQL for creating next - month's partition and dropping the oldest. - -- [ ] **Step 1: Write the page** - - Use level-2 headings for the four sub-sections so they appear in - the page TOC. Code blocks for Alembic op samples and the CI - script. - -- [ ] **Step 2: Re-run the autogenerate spike for `make_dlq_table`** - - Same mechanism as Task 2, but with `make_dlq_table(metadata, - table_name="outbox_dlq")` added to the metadata. Capture the - additional `create_table("outbox_dlq", ...)` + `create_index(...)` - ops. Paste into §4b. - - If §4b's autogenerate sample is impractical to capture for any - reason (Postgres unavailable, etc.), STOP and flag — the page - needs the verbatim sample to be useful. - -- [ ] **Step 3: Smoke-build** - - Run: `just docs-build` - Expected: clean. - -- [ ] **Step 4: Commit** - - ```bash - git add docs/operations/alembic.md - git commit -m "docs: alembic migrations page content - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 7: Index decision-tree row + five cross-link callouts - -**Files:** -- Modify: `docs/index.md` -- Modify: `docs/usage/subscriber.md` -- Modify: `docs/usage/dlq.md` -- Modify: `docs/usage/schema-validation.md` - -Add the one new row to the landing page's decision-tree table per -spec §1, and the five one-line "see also" callouts per spec §5. - -- [ ] **Step 1: Add the decision-tree row in `docs/index.md`** - - Insert before the existing "Install and write the first publisher / - subscriber" row: - - ```markdown - | Deploy to production safely | [Production checklist](operations/checklist.md) | - ``` - -- [ ] **Step 2: Add the five callouts** per the spec §5 table: - - | Existing page | Section | Callout text | - |---|---|---| - | `docs/usage/subscriber.md` | Connection budget (end) | `_Operator-side: [Production checklist § Sizing](../operations/checklist.md#sizing)._` | - | `docs/usage/subscriber.md` | Slow handlers — dedicated queue (end) | `_See also [Troubleshooting § event=lease_lost](../operations/troubleshooting.md#event-lease_lost-recurring-in-logs)._` | - | `docs/usage/dlq.md` | Metric: dlq_written (end) | `_Operator playbook: [Production checklist § DLQ](../operations/checklist.md#dlq)._` | - | `docs/usage/dlq.md` | Retention (end) | `_Step-by-step: [Alembic migrations § DLQ retention via partition drop](../operations/alembic.md#dlq-retention-via-partition-drop)._` | - | `docs/usage/schema-validation.md` | Where to call it (end) | `_CI recipe: [Alembic migrations § Drift detection in CI](../operations/alembic.md#drift-detection-in-ci)._` | - - All callouts use italics + en-dash style for consistency with the - Comparison callouts the `docs-landing-and-comparison` PR (#50) - landed. - -- [ ] **Step 3: Smoke-build** - - Run: `just docs-build` - Expected: clean. All five callout cross-links resolve to anchors - inside the new operator pages (Tasks 4–6). - -- [ ] **Step 4: Commit** - - ```bash - git add docs/index.md docs/usage/subscriber.md docs/usage/dlq.md docs/usage/schema-validation.md - git commit -m "docs: cross-link callouts from existing pages into operator pages - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 8: Verify - -**Files:** none modified; no commit produced. - -- [ ] **Step 1: Full strict build** - - Run: `just docs-build` - Expected: clean. - -- [ ] **Step 2: Lint pass** - - Run: `just lint` - Expected: `eof-fixer`, `ruff format`, `ruff check`, `ty check` - all pass. Markdown EOF + YAML formatting on `mkdocs.yml` are the - only things touched in this PR. - -- [ ] **Step 3: Manual sidebar scan** - - Run: `just docs-serve` - Open the served site. Confirm: - - - Sidebar shows five sections in order: Overview → Getting - started → Concepts → Guides → Reference → Operations. - - Operations contains three pages in order: Production checklist, - Troubleshooting, Alembic migrations. - - The decision-tree table on the landing page has the new - "Deploy to production safely" row pointing at the checklist. - - The Comparison page (from #50) is still present and reachable - under Concepts. - - All eleven Troubleshooting TOC table entries scroll-to the - matching `##` heading on click. - - All sixteen Production-checklist items render with working - relative links. - - The Alembic page's autogenerate code blocks render verbatim with - the right partial-index predicates (matches what Task 2 captured - in the spec). - -- [ ] **Step 4: Open the PR** - - Stop. Hand off to `superpowers:requesting-code-review` / - `superpowers:finishing-a-development-branch` per the standard - workflow. - - On merge, both halves of the pair move to `planning/archived/` and - get `status: shipped`, `pr:`, and `outcome:` filled — same - archive-PR pattern that #52 dogfooded. diff --git a/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/design.md b/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split.md similarity index 100% rename from planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/design.md rename to planning/changes/2026-06-11.02-docs-tutorials-and-observability-split.md diff --git a/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/plan.md b/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/plan.md deleted file mode 100644 index 7cfdd3e..0000000 --- a/planning/changes/2026-06-11.02-docs-tutorials-and-observability-split/plan.md +++ /dev/null @@ -1,408 +0,0 @@ -# docs-tutorials-and-observability-split — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use -> superpowers:subagent-driven-development (recommended) or -> superpowers:executing-plans to implement this plan task-by-task. Steps -> use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Add two tutorials under `docs/tutorials/` and split -`docs/usage/observability.md` into a trimmed Reference + a new -How-to (`usage/setup-prometheus-opentelemetry.md`) + a new Explanation -(`concepts/instrumentation-seams.md`), with one nav reshape commit -that surfaces the four new entries. - -**Spec:** [`planning/active/2026-06-11-docs-tutorials-and-observability-split-design.md`](./2026-06-11-docs-tutorials-and-observability-split-design.md) - -**Branch:** `docs/tutorials-and-observability-split` - -**Commit strategy:** Per-task commits. Tasks 2 (Tutorial 1) and 3 -(Tutorial 2) include the literal terminal output capture step — the -plan author runs each tutorial end-to-end against a clean local -environment before committing. - ---- - -### Task 1: Branch + commit spec + plan + README Active entry - -**Files:** -- Create: `planning/active/2026-06-11-docs-tutorials-and-observability-split-design.md` (already drafted) -- Create: `planning/active/2026-06-11-docs-tutorials-and-observability-split-plan.md` (this file) -- Modify: `planning/README.md` - -- [ ] **Step 1: Confirm branch + uncommitted artifacts** - - Run: `git branch --show-current && ls planning/active/` - Expected: branch `docs/tutorials-and-observability-split`; two - drafted files under `planning/active/`. - -- [ ] **Step 2: Update `planning/README.md` Active section** - - Replace the `_None._` line: - - ```markdown - ## Active - - - **[docs-tutorials-and-observability-split](active/2026-06-11-docs-tutorials-and-observability-split-design.md)** - — Two new tutorials under `docs/tutorials/` plus a three-way - split of `docs/usage/observability.md` into Reference + How-to + - Explanation. F-min from the docs-landing-and-comparison - follow-ons. - ``` - -- [ ] **Step 3: Commit** - - ```bash - git add planning/active/2026-06-11-docs-tutorials-and-observability-split-design.md \ - planning/active/2026-06-11-docs-tutorials-and-observability-split-plan.md \ - planning/README.md - git commit -m "docs: spec + plan for F-min (tutorials + observability split) - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 2: Tutorial 1 — Your first outbox app — DEFERRED - -> **Deferred to follow-on PR (2026-06-12).** Spec §"Scope reduction" -> note explains: tutorial execution discipline (clean env, literal -> captured output) warrants a dedicated session. Structural work -> (T4–T7) shipped without this task. - -**Files:** -- Create: `docs/tutorials/first-outbox-app.md` - -Write the tutorial per [spec §1 -](./2026-06-11-docs-tutorials-and-observability-split-design.md#1-tutorial-your-first-outbox-app) -**after running every step end-to-end against a clean local -environment.** Capture the literal terminal output at each step. - -**Setup for execution:** - -- [ ] **Step 1: Clean Postgres environment** - - Run: `docker compose down -v 2>&1; docker compose up -d postgres` - Expected: container fresh; no leftover `outbox` table from prior - sessions. - -- [ ] **Step 2: Fresh working directory under `/tmp`** - - Create `/tmp/outbox-tutorial-1/` and `cd` into it. This is the - tutorial reader's perspective: a directory with nothing in it. - -- [ ] **Step 3: Walk each tutorial step** - - Follow the section outline in spec §1 (Install → Start Postgres → - Declare → Schema → Handler → Publish → Run). For each step: - - - Execute the literal command the tutorial will tell the reader to - run. - - Capture the literal output. **Do not edit it.** - - If a step's command fails or produces output the spec didn't - anticipate, STOP and update the spec before re-running (the - tutorial must reflect reality). - -- [ ] **Step 4: Write `docs/tutorials/first-outbox-app.md`** - - Use the section outline in spec §1. Each step contains: - - - A one-sentence "what you're about to do" preamble. - - The literal command or code block. - - The literal captured output under "you should see:" (or - equivalent phrasing). Block-quote or `output` code block. - - Voice: warm, step-by-step, "we." Use `_What's next_` footer - linking to the Subscriber reference, Publisher reference, FastAPI - integration guide, and Tutorial 2. - -- [ ] **Step 5: Smoke-build** - - Run: `just docs-build` - Expected: clean. The page is orphaned-not-in-nav (Task 7 wires it - in); the warning is acceptable. - -- [ ] **Step 6: Commit** - - ```bash - git add docs/tutorials/first-outbox-app.md - git commit -m "docs: tutorial — your first outbox app - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 3: Tutorial 2 — Add a Kafka relay — DEFERRED - -> **Deferred to follow-on PR (2026-06-12).** Same rationale as Task 2. - -**Files:** -- Create: `docs/tutorials/add-kafka-relay.md` - -Write the tutorial per [spec §2 -](./2026-06-11-docs-tutorials-and-observability-split-design.md#2-tutorial-add-a-kafka-relay) -extending Tutorial 1's app. Same end-to-end execution requirement. - -The "kill Kafka, see retry" step (spec §2 Step 6) is the -fragility risk flagged in the spec. Use Confluent's `cp-kafka` -image. If the step's repro is unstable on your environment, STOP -and reshape it as a callout that explains the at-least-once -contract without requiring the visible retry. - -- [ ] **Step 1: Extend the tutorial-1 environment** - - In `/tmp/outbox-tutorial-1/`, add Kafka to `docker-compose.yml`. - Confluent's `cp-kafka` image is the recommended choice for - cross-platform compatibility (especially Apple Silicon). - -- [ ] **Step 2: Walk each tutorial step** - - Same discipline as Task 2 — execute, capture, paste. The kill- - Kafka step: - - ``` - docker compose stop kafka - - - docker compose start kafka - - ``` - - If the retry log frequency / shape differs from what the spec - predicted, update the spec **before** writing the tutorial page. - -- [ ] **Step 3: Write `docs/tutorials/add-kafka-relay.md`** - - Use the section outline in spec §2. Cross-link to Tutorial 1's - "What's next" in a "Before you start" preamble. Footer links to - Relay reference, Subscriber § Retry strategies, and Comparison § - "vs FastStream foreign-broker direct". - -- [ ] **Step 4: Smoke-build** - - Run: `just docs-build` - Expected: clean. - -- [ ] **Step 5: Commit** - - ```bash - git add docs/tutorials/add-kafka-relay.md - git commit -m "docs: tutorial — add a Kafka relay - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 4: New Explanation — `concepts/instrumentation-seams.md` - -**Files:** -- Create: `docs/concepts/instrumentation-seams.md` - -Write the Explanation per [spec §3c -](./2026-06-11-docs-tutorials-and-observability-split-design.md#3c-conceptsinstrumentation-seamsmd-new-explanation). -Extract the relevant content from the current `usage/observability.md` -§ "Layering: middleware seam vs. recorder seam" — same layering -table, same four "events the other seam physically cannot observe" -points, expanded narrative. - -Voice: discursive, explanatory. Aimed at "why are there two seams" -readers, not "how do I wire this" readers. - -- [ ] **Step 1: Read current `usage/observability.md` § Layering** - - Read the existing § "Layering: middleware seam vs. recorder seam" - for the table + the four bullet points to extract. - -- [ ] **Step 2: Write `docs/concepts/instrumentation-seams.md`** - - Per spec §3c outline (tension → middleware-only → recorder-only → - layering table → operator implication). - -- [ ] **Step 3: Smoke-build** - - Run: `just docs-build` - Expected: clean. - -- [ ] **Step 4: Commit** - - ```bash - git add docs/concepts/instrumentation-seams.md - git commit -m "docs: concept page — instrumentation seams (recorder vs middleware) - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 5: New How-to — `usage/setup-prometheus-opentelemetry.md` - -**Files:** -- Create: `docs/usage/setup-prometheus-opentelemetry.md` - -Write the How-to per [spec §3b -](./2026-06-11-docs-tutorials-and-observability-split-design.md#3b-usagesetup-prometheus-opentelemetrymd-new-how-to). -Direct port of current `observability.md`'s adapter setup sections -(Prometheus adapter, OpenTelemetry adapter, Native middleware, -"both seams together") with cross-section references retargeted. - -- [ ] **Step 1: Lift the relevant sections** - - From current `usage/observability.md`: - - § "Prometheus adapter" (full code block + Consume vs publish - label set + PromQL queries that show wiring, not playbook) - - § "OpenTelemetry adapter" (full code block) - - § "Native middleware (spans + bus parity)" (full code block) - - Lift verbatim into the new page; clean up the cross-section - references that no longer resolve. - -- [ ] **Step 2: Add a short intro** - - One paragraph: "You've decided to wire metrics. This page is the - recipe. For the why, see [Concepts § Instrumentation seams]( - ../concepts/instrumentation-seams.md); for the event catalog and - PromQL playbook, see [Reference § Observability]( - ./observability.md)." - -- [ ] **Step 3: Smoke-build** - - Run: `just docs-build` - Expected: clean. - -- [ ] **Step 4: Commit** - - ```bash - git add docs/usage/setup-prometheus-opentelemetry.md - git commit -m "docs: how-to — setup Prometheus and OpenTelemetry - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 6: Trim `usage/observability.md` to Reference shape - -**Files:** -- Modify: `docs/usage/observability.md` - -Strip everything that moved to Tasks 4 (Layering) and 5 (adapter -setups). Keep per [spec §3a -](./2026-06-11-docs-tutorials-and-observability-split-design.md#3a-usageobservabilitymd-kept-trimmed-to-reference): - -- § "The recorder seam" — the callable signature, the event catalog, - the "must not block" note. -- The PromQL playbook table (the *operator queries*, distinct from - the setup-wiring PromQL in Task 5). -- § "Test broker note". - -Add a top-of-page see-also pair pointing at the new How-to and -Explanation: - -```markdown -*Setting it up: [Setup Prometheus and OpenTelemetry]( -./setup-prometheus-opentelemetry.md). Why two seams: [Concepts § -Instrumentation seams](../concepts/instrumentation-seams.md).* -``` - -- [ ] **Step 1: Delete the moved sections** - - Per the spec §3a "Removed from current page (moved)" list: - - "Prometheus adapter" (entire section) → moved to Task 5 - - "OpenTelemetry adapter" (entire section) → moved to Task 5 - - "Native middleware (spans + bus parity)" → moved to Task 5 - - "Layering: middleware seam vs. recorder seam" + table → moved to Task 4 - -- [ ] **Step 2: Add the top-of-page see-also pair** - -- [ ] **Step 3: Smoke-build** - - Run: `just docs-build` - Expected: clean. Inbound deep links to the page's surviving anchors - (`#the-recorder-seam`, `#test-broker-note`) still resolve; - anchors that moved (`#prometheus-adapter`, etc.) are now dead - inside this page but reachable through the see-also at the top. - -- [ ] **Step 4: Commit** - - ```bash - git add docs/usage/observability.md - git commit -m "docs: trim observability.md to Reference shape - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 7: Nav reshape - -**Files:** -- Modify: `mkdocs.yml` - -Add the four new entries to the nav per [spec §4 -](./2026-06-11-docs-tutorials-and-observability-split-design.md#4-nav-adjustments). -Two under Getting started, one under Concepts, one under Guides. - -- [ ] **Step 1: Edit `mkdocs.yml`** - - Replace the existing `nav:` block per the spec §4 sample. - -- [ ] **Step 2: Smoke-build** - - Run: `just docs-build` - Expected: clean. Sidebar shows the four new entries. - -- [ ] **Step 3: Commit** - - ```bash - git add mkdocs.yml - git commit -m "docs: nav reshape — surface tutorials, setup how-to, instrumentation explanation - - Co-Authored-By: Claude Opus 4.7 (1M context) " - ``` - ---- - -### Task 8: Verify - -**Files:** none modified; no commit produced. - -- [ ] **Step 1: Full strict build** - - Run: `just docs-build` - Expected: clean. All cross-links from the four new pages and the - trimmed `observability.md` resolve. - -- [ ] **Step 2: Lint pass** - - Run: `just lint` - Expected: eof-fixer, ruff format, ruff check, ty check all pass. - -- [ ] **Step 3: Manual sidebar scan** - - Run: `just docs-serve` - Open the served site. Confirm: - - - Getting started shows: Installation, Basic usage, Tutorial 1, - Tutorial 2. - - Concepts shows: How it works, Comparison, Instrumentation seams. - - Guides shows: FastAPI integration, Relay, Timers, Testing, - Schema validation, Setup Prometheus and OpenTelemetry. - - Reference's Observability page is now ~150 lines (trimmed - cleanly). - -- [ ] **Step 4: Re-run Tutorial 1 against a fresh checkout** - - In a temp directory, follow Tutorial 1 step-by-step using only - what the page tells you. Every command's output should match - what the page promised. If anything diverges, STOP and update - the tutorial. - -- [ ] **Step 5: Open the PR** - - Stop. Hand off to `superpowers:requesting-code-review` / - `superpowers:finishing-a-development-branch`. - - On merge, both halves of the pair move to `planning/archived/` - with `status: shipped`, `pr:`, and `outcome:` filled — same - archive pattern PRs #52 and #54 dogfooded. diff --git a/planning/changes/2026-06-12.01-docs-tutorials/design.md b/planning/changes/2026-06-12.01-docs-tutorials.md similarity index 100% rename from planning/changes/2026-06-12.01-docs-tutorials/design.md rename to planning/changes/2026-06-12.01-docs-tutorials.md diff --git a/planning/changes/2026-06-12.01-docs-tutorials/plan.md b/planning/changes/2026-06-12.01-docs-tutorials/plan.md deleted file mode 100644 index a88d1af..0000000 --- a/planning/changes/2026-06-12.01-docs-tutorials/plan.md +++ /dev/null @@ -1,594 +0,0 @@ -# docs-tutorials — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use -> superpowers:subagent-driven-development (recommended) or -> superpowers:executing-plans to implement this plan task-by-task. Steps -> use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Land the two tutorials deferred from #56 (Your first outbox -app + Add a Kafka relay), each written from literal end-to-end -execution against a clean environment, plus the two-entry nav update -and the cross-links that surface them. - -**Spec:** [`planning/active/2026-06-12-docs-tutorials-design.md`](./2026-06-12-docs-tutorials-design.md) - -**Branch:** `docs/tutorials` - -**Commit strategy:** Per-task commits. Tasks 2 and 3 each carry a -`docker compose down -v` smoke step before writing so the captured -output reflects a true first-run experience — no leftover state from -an earlier session of the same task. - -**Sequencing note:** Tasks 2 and 3 are gated on actually running the -tutorial code end-to-end against a clean Postgres (Task 2) and a -clean Postgres + Kafka (Task 3). The plan author runs each tutorial -on a real machine before committing the page. **Hand-edited output is -a defect** — if a captured line looks "ugly" (asyncpg warning, sqlalchemy -deprecation, docker compose progress noise), it stays. The point is -that the reader's first run will match the page. - ---- - -### Task 1: Branch + plan commit + README Active entry - -**Files:** -- Modify: `planning/README.md` -- Create: `planning/active/2026-06-12-docs-tutorials-design.md` (already drafted) -- Create: `planning/active/2026-06-12-docs-tutorials-plan.md` (this file) - -- [ ] **Step 1: Branch off `main`** - - ```bash - git fetch origin - git switch -c docs/tutorials origin/main - ``` - - Expected: clean tree on `docs/tutorials` at `origin/main`'s HEAD. - The current `chore/archive-observability-split` branch carried the - archive-only commit from #56's archive pass; the tutorials work - belongs on its own branch. - -- [ ] **Step 2: Update `planning/README.md` Active section** - - Replace the `_None._` line under `## Active` with: - - ```markdown - ## Active - - - **[docs-tutorials](active/2026-06-12-docs-tutorials-design.md)** - — The two tutorials deferred from #56: *Your first outbox app* - (10-minute walk-through) and *Add a Kafka relay* (extends the - first tutorial with a Kafka publisher + at-least-once - demonstration). New `docs/tutorials/` directory; two nav entries. - ``` - -- [ ] **Step 3: Smoke-build** - - Run: `just docs-build` - - Expected: clean. No new pages yet; the README change is non-docs. - This is a sanity check that the branch baseline still builds. - -- [ ] **Step 4: Commit** - - ```bash - git add planning/active/2026-06-12-docs-tutorials-design.md \ - planning/active/2026-06-12-docs-tutorials-plan.md \ - planning/README.md - git commit -m "$(cat <<'EOF' - docs: spec + plan for two new tutorials (F-min part 2) - - Co-Authored-By: Claude Opus 4.7 (1M context) - EOF - )" - ``` - ---- - -### Task 2: Tutorial 1 — Your first outbox app - -**Files:** -- Create: `docs/tutorials/first-outbox-app.md` -- Working dir (not committed): `/tmp/outbox-tutorial-1/` - -Write the tutorial per [spec §1 -](./2026-06-12-docs-tutorials-design.md#1-tutorial-your-first-outbox-app) -**after running every step end-to-end against a clean local -environment.** Capture the literal terminal output at each step. The -captured output blocks land inside the page under "you should see:" -preambles. - -**Voice contract:** warm, step-by-step, sparing use of "we." Each -step opens with a one-sentence "what you're about to do" and closes -with the literal output. No design rationale on the page; that's -Concepts. No forward references to library features the tutorial -doesn't use (no DLQ, no timers, no `OutboxResponse`). - -- [ ] **Step 1: Clean Postgres environment** - - Run from the repo root: - - ```bash - docker compose -f compose.yaml down -v - ``` - - Expected: containers and volumes removed. We will *not* use the - repo's `compose.yaml` for the tutorial run — the reader doesn't - have it. The teardown is to ensure no port 5432 conflict against - the tutorial container we're about to start. - -- [ ] **Step 2: Fresh working directory under `/tmp`** - - ```bash - rm -rf /tmp/outbox-tutorial-1 - mkdir /tmp/outbox-tutorial-1 - cd /tmp/outbox-tutorial-1 - ``` - - This is the tutorial reader's perspective: a directory with - nothing in it. Every command in the tutorial body is run from - here. - -- [ ] **Step 3: Walk Step 1 — install** - - The tutorial will instruct: - - ```bash - uv add 'faststream-outbox[asyncpg,validate]' - ``` - - (or `pip install` equivalent). Run it. Capture the literal stdout. - The PEP 668 / venv path is tutorial-reader's problem; pick the - flow that produces the cleanest captured output (recommendation: - `uv init && uv add ...` to start from a real project). - - If `uv add` produces a venv-creation banner or a lock-file diff - line, that line **stays** in the captured output. The page shows - what a first-time reader will actually see. - -- [ ] **Step 4: Walk Step 2 — start Postgres** - - The tutorial will instruct: - - ```bash - docker run --rm -d --name outbox-postgres \ - -e POSTGRES_USER=outbox -e POSTGRES_PASSWORD=outbox -e POSTGRES_DB=outbox \ - -p 5432:5432 postgres:17 - ``` - - Run it. Capture the container ID stdout. Wait for `docker logs - outbox-postgres 2>&1 | tail -1` to show `database system is ready - to accept connections`. Capture that one log line too. - -- [ ] **Step 5: Walk Steps 3 + 4 — declare the table and create the schema** - - The tutorial will provide an `app.py` containing the - `make_outbox_table` declaration plus a one-shot - `metadata.create_all` script (run via `python -c` or as a small - block at the top of `app.py` gated by `if __name__ == "__main__":`). - - Decision: use a separate `create_schema.py` script over a gated - block. Two reasons: (1) keeps `app.py` minimal so the diff at each - later step stays small, (2) the tutorial reader runs the schema - creation once and never again, so a separate file is the more - honest mental model. - - Write `create_schema.py`. Run `python create_schema.py`. Capture - the output (likely silent on success — that's fine; the page says - "you should see no output"). - - Verify with `docker exec outbox-postgres psql -U outbox -d outbox - -c '\d outbox'`. Capture that table definition. It goes in the - tutorial as the "you should see this table" callout under Step 4. - -- [ ] **Step 6: Walk Step 5 — define the handler** - - Add the `@broker.subscriber("orders")` block to `app.py`. The - handler body is `print(f"got order {order_id}")` — the simplest - thing that produces visible output on Step 7. - - No command to run at this step; the handler executes during Step 7. - -- [ ] **Step 7: Walk Step 6 — publish a row** - - Add an `@app.after_startup` block to `app.py` that opens a session - and calls `await broker.publish(1, queue="orders", session=session)`. - No command to run yet — Step 7 is what fires it. - -- [ ] **Step 8: Walk Step 7 — run the app** - - Run: - - ```bash - faststream run app:app - ``` - - Capture every line of stdout from start until the handler fires - (`got order 1`) and the next idle line. The tutorial shows this as - a `text` block under "you should see:" — the literal output, INFO - lines and all. - - Ctrl-C; capture the shutdown lines. Those go in the page too — - the reader needs to know what a clean shutdown looks like. - -- [ ] **Step 9: Write `docs/tutorials/first-outbox-app.md`** - - Use the section outline from spec §1 (What you'll build → Before - you start → Steps 1–7 → What you just built → What's next). - - Each step contains: - - A one-sentence "what you're about to do" preamble. - - The literal command or `app.py` diff/full-file. - - The literal captured output under "you should see:" (a - fenced `text` or `bash` block). - - `What's next` footer links to: - - [Subscriber reference](../usage/subscriber.md) - - [Publisher reference](../usage/publisher.md) - - [FastAPI integration](../usage/fastapi.md) - - [Tutorial: Add a Kafka relay](./add-kafka-relay.md) - -- [ ] **Step 10: Smoke-build** - - Run: `just docs-build` - - Expected: clean **or** an `not_in_nav` failure (Task 4 wires the - page in). `mkdocs build --strict` promotes that warning to an - error. If the build fails on it, add the nav entry now (out of - Task-4 order) rather than disabling the strict check — `--strict` - is load-bearing and shouldn't be relaxed for a single task. - -- [ ] **Step 11: Clean up the tutorial environment** - - ```bash - docker stop outbox-postgres - rm -rf /tmp/outbox-tutorial-1 - ``` - - Tutorial 2 starts from a fresh setup; carrying state across tasks - would defeat the "first-run experience" capture. - -- [ ] **Step 12: Commit** - - ```bash - git add docs/tutorials/first-outbox-app.md - git commit -m "$(cat <<'EOF' - docs: tutorial — your first outbox app - - Co-Authored-By: Claude Opus 4.7 (1M context) - EOF - )" - ``` - ---- - -### Task 3: Tutorial 2 — Add a Kafka relay - -**Files:** -- Create: `docs/tutorials/add-kafka-relay.md` -- Working dir (not committed): `/tmp/outbox-tutorial-2/` - -Write the tutorial per [spec §2 -](./2026-06-12-docs-tutorials-design.md#2-tutorial-add-a-kafka-relay) -extending Tutorial 1's app. Same end-to-end execution requirement. - -The "kill Kafka, see retry" step (spec §2 Step 6) is the fragility -risk flagged in the spec. Use Confluent's `cp-kafka` image. If the -step's repro is unstable on your environment, STOP and reshape it as -a callout that explains the at-least-once contract without requiring -the visible retry. **The spec authorizes this fallback** — don't -push past a flaky step to "make it work for the page." - -- [ ] **Step 1: Fresh working directory; re-walk Tutorial 1 to seed it** - - ```bash - rm -rf /tmp/outbox-tutorial-2 - mkdir /tmp/outbox-tutorial-2 - cd /tmp/outbox-tutorial-2 - ``` - - Now re-walk Tutorial 1 inside this directory using only what - `docs/tutorials/first-outbox-app.md` says. End-state: working - `app.py` + a Postgres container + a venv with `faststream-outbox` - installed. The captured starting state for Tutorial 2 *is* a - reader who just finished Tutorial 1, so this re-walk is the - cheapest way to land at exactly that state. (Bonus: if any step - diverges from the captured output during the re-walk, you caught - a Task 2 defect — fix Tutorial 1 before continuing.) - -- [ ] **Step 2: Walk Step 1 — add Kafka via docker-compose** - - Write the `docker-compose.yml` that adds a `kafka` service using - Confluent's `cp-kafka:7.6.0` image (or current stable at execution - time — record the exact tag you use in the captured output). The - recommended single-broker KRaft configuration uses `cp-kafka`'s - built-in mode; no separate ZooKeeper service. - - Run: - - ```bash - docker compose up -d kafka - ``` - - Capture the stdout. Wait for `docker compose logs kafka 2>&1 | - grep -m1 'Kafka Server started'`. Capture that one line. - -- [ ] **Step 3: Walk Step 2 — install faststream[kafka]** - - ```bash - uv add 'faststream[kafka]' - ``` - - Capture stdout. - -- [ ] **Step 4: Walk Steps 3 + 4 — Kafka broker + stacked decorator** - - Extend `app.py`: - - Add `KafkaBroker(...)` and a `kafka_publisher = kafka_broker.publisher("orders.kafka")`. - - Stack `@kafka_publisher` above the existing `@broker_outbox.subscriber("orders")`. - - Adjust the handler to `return order_id` so the publisher - decorator forwards a payload. - - No command to run at this step; Step 5 fires it. - -- [ ] **Step 5: Walk Step 5 — run and watch a row reach Kafka** - - Run `faststream run app:app` in one terminal. In a second - terminal, run: - - ```bash - docker compose exec kafka kafka-console-consumer \ - --bootstrap-server kafka:9092 --topic orders.kafka --from-beginning - ``` - - Capture both: the `faststream` stdout (publish + handler + Kafka - send) and the consumer output (the `1` arriving on the topic). - -- [ ] **Step 6: Walk Step 6 — kill Kafka and watch the retry** - - In the running `faststream` process, with the consumer still - listening: - - ```bash - docker compose stop kafka - ``` - - Publish another row (cleanest: send `SIGUSR1` to the app or - re-run a one-shot `python -c "import asyncio; ..."` snippet that - calls `broker.publish` — record whichever flow the tutorial uses). - Capture the outbox subscriber's retry log lines. Then: - - ```bash - docker compose start kafka - ``` - - Capture the eventual successful delivery line and the consumer - receiving the row. - - **If the retry doesn't visibly fire** (Kafka returns instantly - with a "not ready" that the producer retries internally before the - outbox subscriber notices), drop Step 6 from the tutorial and - replace it with a callout that says: *"Behind the scenes, if Kafka - were unavailable, the outbox row would be retried per the subscriber's - retry policy. See [Subscriber § Retry strategies - ](../usage/subscriber.md#retry-strategies)."* The spec authorizes - this fallback. - -- [ ] **Step 7: Write `docs/tutorials/add-kafka-relay.md`** - - Use the section outline from spec §2. Cross-link upward to - Tutorial 1 in the "Before you start" preamble. `What's next` - footer links to: - - [Relay reference](../usage/relay.md) - - [Subscriber § Retry strategies](../usage/subscriber.md#retry-strategies) - - [Comparison](../concepts/comparison.md) — see the section *"vs. FastStream + `KafkaBroker` / `RabbitBroker` directly"* (the auto-generated anchor isn't stable enough to hardcode; link the page and let the reader scroll) - - If Step 6 fell back to a callout, capture that in a one-line note - at the top: *"This tutorial originally demonstrated retry by - killing Kafka mid-flight; that step is fragile on some - environments and is replaced by a callout."* - -- [ ] **Step 8: Smoke-build** - - Run: `just docs-build`. Expected: clean (orphan warning OK). - -- [ ] **Step 9: Clean up** - - ```bash - docker compose down -v - rm -rf /tmp/outbox-tutorial-2 - ``` - -- [ ] **Step 10: Commit** - - ```bash - git add docs/tutorials/add-kafka-relay.md - git commit -m "$(cat <<'EOF' - docs: tutorial — add a Kafka relay - - Co-Authored-By: Claude Opus 4.7 (1M context) - EOF - )" - ``` - ---- - -### Task 4: Nav additions - -**Files:** -- Modify: `mkdocs.yml` - -Add the two `Tutorial:` entries to the `Getting started` section per -[spec §3 -](./2026-06-12-docs-tutorials-design.md#3-nav-adjustments). The spec -anticipated either uncommented placeholders or fresh additions; the -post-#56 `mkdocs.yml` does **not** carry placeholders, so this task -adds fresh entries. - -- [ ] **Step 1: Edit `mkdocs.yml`** - - Replace the `Getting started:` block: - - ```yaml - - Getting started: - - Installation: introduction/installation.md - - Basic usage: usage/basic.md - - 'Tutorial: Your first outbox app': tutorials/first-outbox-app.md - - 'Tutorial: Add a Kafka relay': tutorials/add-kafka-relay.md - ``` - - Order matters — Installation first, Basic usage second, then the - two tutorials. Tutorials come after Basic usage because some - readers will prefer the terse reference shape; the sidebar lets - them choose. - -- [ ] **Step 2: Smoke-build** - - Run: `just docs-build` - - Expected: clean, no orphan warnings now that the pages are in the - nav. - -- [ ] **Step 3: Commit** - - ```bash - git add mkdocs.yml - git commit -m "$(cat <<'EOF' - docs: nav — surface the two new tutorials under Getting started - - Co-Authored-By: Claude Opus 4.7 (1M context) - EOF - )" - ``` - ---- - -### Task 5: Cross-links — relay reference and landing decision tree - -**Files:** -- Modify: `docs/usage/relay.md` -- Modify: `docs/index.md` - -Two surgical edits per [spec §4 -](./2026-06-12-docs-tutorials-design.md#4-cross-link-contract): - -1. `usage/relay.md` gets a one-line "Want a worked end-to-end - example?" pointer at the top. -2. `docs/index.md`'s decision-tree row for *"Install and write the - first publisher / subscriber"* (line ~44) repoints from - `installation.md → basic.md` to `installation.md → first-outbox-app.md`. - Basic usage remains discoverable via the sidebar. - -- [ ] **Step 1: Add the relay pointer** - - In `docs/usage/relay.md`, immediately after the H1 title (before - the first paragraph), insert: - - ```markdown - > Want a worked end-to-end example? See - > [Tutorial: Add a Kafka relay](../tutorials/add-kafka-relay.md). - ``` - -- [ ] **Step 2: Repoint the decision-tree row** - - In `docs/index.md`, find the line: - - ```markdown - | Install and write the first publisher / subscriber | [Installation](introduction/installation.md) → [Basic usage](usage/basic.md) | - ``` - - Replace with: - - ```markdown - | Install and write the first publisher / subscriber | [Installation](introduction/installation.md) → [Tutorial: Your first outbox app](tutorials/first-outbox-app.md) | - ``` - - Leave the `### Getting started` section list below the table - unchanged — it still mentions Installation + Basic usage, which is - correct (Basic usage is still a Getting-started page, just no - longer the decision-tree default). - -- [ ] **Step 3: Smoke-build** - - Run: `just docs-build`. Expected: clean. - -- [ ] **Step 4: Commit** - - ```bash - git add docs/usage/relay.md docs/index.md - git commit -m "$(cat <<'EOF' - docs: cross-links — relay tutorial pointer + landing decision tree - - Co-Authored-By: Claude Opus 4.7 (1M context) - EOF - )" - ``` - ---- - -### Task 6: Verify + PR handoff - -**Files:** none modified; no commit produced. - -- [ ] **Step 1: Full strict build** - - Run: `just docs-build` - - Expected: clean. All cross-links from the two new pages, the - modified `relay.md`, and the modified `index.md` resolve. - -- [ ] **Step 2: Lint pass** - - Run: `just lint` - - Expected: eof-fixer, ruff format, ruff check, ty check all pass. - (Docs-only PR, but the linter touches the planning markdown too, - so this catches trailing whitespace and missing EOF newlines.) - -- [ ] **Step 3: Manual sidebar scan** - - Run: `just docs-serve`. Open the served site. Confirm: - - - Getting started shows: Installation, Basic usage, Tutorial: - Your first outbox app, Tutorial: Add a Kafka relay. - - Tutorial 1's "What's next" footer links resolve (Subscriber, - Publisher, FastAPI integration, Tutorial 2). - - Tutorial 2's "What's next" footer links resolve (Relay, - Subscriber § Retry strategies, Comparison § "vs FastStream - foreign-broker direct"). - - `usage/relay.md` shows the new top-of-page Tutorial 2 pointer. - - `index.md` decision-tree row points at Tutorial 1. - -- [ ] **Step 4: Re-run Tutorial 1 against a fresh checkout** - - In a temp directory, follow Tutorial 1 step-by-step using **only - what the page tells you**. Every command's literal output should - match what the page promised. Reviewer-grade check: if anything - diverges, STOP and update the tutorial. This is the single - highest-leverage verification step in the plan — a reader who - trips on Step 4 has had a worse first impression than one who - found no tutorial at all. - -- [ ] **Step 5: Re-run Tutorial 2 against the Tutorial-1 end state** - - Same discipline. The kill-Kafka step is allowed to be a callout if - Task 3 Step 6 fell back; the rest of the page must reproduce. - -- [ ] **Step 6: Open the PR** - - Hand off to `superpowers:requesting-code-review` / - `superpowers:finishing-a-development-branch`. - - PR title: `docs: two new tutorials — first outbox app + Kafka relay`. - - PR body should call out: - - Source spec link. - - Note that both tutorials were executed end-to-end against clean - environments; literal output is captured in-page. - - Whether Task 3 Step 6 (kill-Kafka demo) landed as the live - demonstration or as the callout fallback. - - Reviewer ask: re-walk both tutorials on a clean machine. - - On merge, both halves of the planning pair move to - `planning/archived/` with `status: shipped`, `pr:`, and `outcome:` - filled — same archive pattern as #50 / #53 / #56. diff --git a/planning/changes/2026-06-13.01-portable-planning-convention/design.md b/planning/changes/2026-06-13.01-portable-planning-convention.md similarity index 100% rename from planning/changes/2026-06-13.01-portable-planning-convention/design.md rename to planning/changes/2026-06-13.01-portable-planning-convention.md diff --git a/planning/changes/2026-06-13.01-portable-planning-convention/plan.md b/planning/changes/2026-06-13.01-portable-planning-convention/plan.md deleted file mode 100644 index a14801c..0000000 --- a/planning/changes/2026-06-13.01-portable-planning-convention/plan.md +++ /dev/null @@ -1,712 +0,0 @@ -# Portable planning-convention — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use -> superpowers:subagent-driven-development (recommended) or -> superpowers:executing-plans to implement this plan task-by-task. Steps use -> checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Restructure `planning/` to the two-axis OpenSpec-shaped convention — -`architecture/` (root, unchanged) as living truth, `planning/changes/` -(`active/` → `archive/`) as folder-bundle change history, with `.NN`-tiebroken -ids, three lanes, dedicated `audits/`+`retros/`, `deferred.md`, and a portable -README — and migrate every existing artifact into it. - -**Spec:** [`design.md`](./design.md) - -**Branch:** `docs/portable-planning-convention` (already created; spec already -committed there). - -**Commit strategy:** Per-task commits. Pure docs / file moves — no runtime code, -no tests. Verification is grep + `mkdocs build --strict` + `just lint-ci` + a -tree check, all in the final task. - -**No-code note:** This plan has no pytest/TDD cycle. "Verification" steps run -shell commands and assert on their output. Use `git mv` for every move so blame -follows. - ---- - -## Reference: archived-pair → bundle-id mapping - -Used by Task 2. `.NN` is assigned per date in PR-merge order (lower PR = `.01`). -PR numbers are from the current `planning/README.md` index. - -| Old slug (in `planning/archived/`) | PR | New bundle folder (in `planning/changes/`) | Has | -|---|---|---|---| -| `2026-06-03-all-extra-and-planning-dir` | #41 | `2026-06-03.01-all-extra-and-planning-dir/` | design+plan | -| `2026-06-03-faststream-0.7-migration` | #42 | `2026-06-03.02-faststream-0.7-migration/` | design+plan | -| `2026-06-04-faststream-0.7.1-testbroker-typing` | #43 | `2026-06-04.01-faststream-0.7.1-testbroker-typing/` | design+plan | -| `2026-06-04-foreign-broker-relay` | #44 | `2026-06-04.02-foreign-broker-relay/` | design+plan | -| `2026-06-09-mkdocs-github-pages` | #45 | `2026-06-09.01-mkdocs-github-pages/` | design+plan | -| `2026-06-09-drain-test-flaky-fetch-observation` | #48 | `2026-06-09.02-drain-test-flaky-fetch-observation/` | design+plan | -| `2026-06-10-planning-conventions` | #49 | `2026-06-10.01-planning-conventions/` | design only | -| `2026-06-10-docs-landing-and-comparison` | #50 | `2026-06-10.02-docs-landing-and-comparison/` | design+plan | -| `2026-06-11-operator-pages` | #53 | `2026-06-11.01-operator-pages/` | design+plan | -| `2026-06-11-docs-tutorials-and-observability-split` | #56 | `2026-06-11.02-docs-tutorials-and-observability-split/` | design+plan | -| `2026-06-12-docs-tutorials` | #58 | `2026-06-12.01-docs-tutorials/` | design+plan | - -Findings (Task 3) → `planning/audits/`, names unchanged: -`2026-06-12-code-audit-findings.md`, `2026-06-12-docs-audit-findings.md`. - ---- - -### Task 1: Scaffold new dirs and move this change's own bundle - -**Files:** -- Create: `planning/changes/active/2026-06-13.01-portable-planning-convention/design.md` (moved) -- Create: `planning/changes/active/2026-06-13.01-portable-planning-convention/plan.md` (moved) -- Create: `planning/changes/active/.gitkeep`, `planning/changes/.gitkeep` - -- [ ] **Step 1: Create the changes/ skeleton** - - ```bash - mkdir -p planning/changes/active planning/changes/archive - touch planning/changes/active/.gitkeep planning/changes/.gitkeep - ``` - -- [ ] **Step 2: Move this change's design + plan into a bundle folder** - - ```bash - mkdir -p planning/changes/active/2026-06-13.01-portable-planning-convention - git mv planning/active/2026-06-13-portable-planning-convention-design.md \ - planning/changes/active/2026-06-13.01-portable-planning-convention/design.md - git mv planning/active/2026-06-13-portable-planning-convention-plan.md \ - planning/changes/active/2026-06-13.01-portable-planning-convention/plan.md - ``` - -- [ ] **Step 3: Update this bundle's internal cross-links + status** - - In the moved `plan.md` frontmatter, leave `spec: portable-planning-convention` - as-is. In the moved `plan.md` body, fix the **Spec:** link to the sibling: - change `[`planning/active/2026-06-13-portable-planning-convention-design.md`](./2026-06-13-portable-planning-convention-design.md)` - to `[`design.md`](./design.md)`. - - In the moved `design.md` body, two references point at the pre-migration - path `../archived/2026-06-10-planning-conventions-design.md` (one in Summary, - one in the Non-goals "index generator" bullet). Rewrite both to the - post-migration bundle path - `../../archive/2026-06-10.01-planning-conventions/design.md` (relative from - `changes/active/2026-06-13.01-…/`). This link resolves after Task 2 creates - that bundle. - - ```bash - d=planning/changes/active/2026-06-13.01-portable-planning-convention/design.md - grep -c "archived/2026-06-10-planning-conventions-design.md" "$d" # expect 2 - # after editing: - grep -c "../../archive/2026-06-10.01-planning-conventions/design.md" "$d" # expect 2 - ``` - - Set `status: approved` in both `design.md` and `plan.md` frontmatter (the spec - is approved; the plan is being executed). - -- [ ] **Step 4: Verify the move** - - ```bash - ls planning/changes/active/2026-06-13.01-portable-planning-convention/ - ``` - Expected: `design.md plan.md` - -- [ ] **Step 5: Commit** - - ```bash - git add -A planning/ - git commit -m "docs: scaffold planning/changes/ and self-migrate this bundle - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 2: Regroup archived pairs into changes/ folders - -**Files:** moves only, per the mapping table above. `planning/archived/*` → -`planning/changes//{design,plan}.md`. - -- [ ] **Step 1: Move all ten full pairs** - - ```bash - cd planning - A=archived; B=changes/archive - for map in \ - "2026-06-03-all-extra-and-planning-dir:2026-06-03.01-all-extra-and-planning-dir" \ - "2026-06-03-faststream-0.7-migration:2026-06-03.02-faststream-0.7-migration" \ - "2026-06-04-faststream-0.7.1-testbroker-typing:2026-06-04.01-faststream-0.7.1-testbroker-typing" \ - "2026-06-04-foreign-broker-relay:2026-06-04.02-foreign-broker-relay" \ - "2026-06-09-mkdocs-github-pages:2026-06-09.01-mkdocs-github-pages" \ - "2026-06-09-drain-test-flaky-fetch-observation:2026-06-09.02-drain-test-flaky-fetch-observation" \ - "2026-06-10-docs-landing-and-comparison:2026-06-10.02-docs-landing-and-comparison" \ - "2026-06-11-operator-pages:2026-06-11.01-operator-pages" \ - "2026-06-11-docs-tutorials-and-observability-split:2026-06-11.02-docs-tutorials-and-observability-split" \ - "2026-06-12-docs-tutorials:2026-06-12.01-docs-tutorials"; do - old="${map%%:*}"; new="${map##*:}"; mkdir -p "$B/$new" - git mv "$A/${old}-design.md" "$B/$new/design.md" - git mv "$A/${old}-plan.md" "$B/$new/plan.md" - done - cd .. - ``` - -- [ ] **Step 2: Move the design-only bundle (planning-conventions, no plan)** - - ```bash - mkdir -p planning/changes/2026-06-10.01-planning-conventions - git mv planning/archived/2026-06-10-planning-conventions-design.md \ - planning/changes/2026-06-10.01-planning-conventions/design.md - ``` - -- [ ] **Step 3: Verify all eleven bundles exist** - - ```bash - ls -d planning/changes/*/ | wc -l # expect 11 - ls planning/changes/2026-06-10.01-planning-conventions/ # expect: design.md - ``` - -- [ ] **Step 4: Commit** - - ```bash - git add -A planning/ - git commit -m "docs: regroup archived spec/plan pairs into changes/archive bundles - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 3: Move findings to audits/, create retros/, drop emptied dirs - -**Files:** -- Create: `planning/audits/2026-06-12-code-audit-findings.md` (moved) -- Create: `planning/audits/2026-06-12-docs-audit-findings.md` (moved) -- Create: `planning/retros/.gitkeep` -- Delete: `planning/active/`, `planning/archived/` (now empty) - -- [ ] **Step 1: Move the two findings reports** - - ```bash - mkdir -p planning/audits - git mv planning/archived/2026-06-12-code-audit-findings.md planning/audits/ - git mv planning/archived/2026-06-12-docs-audit-findings.md planning/audits/ - ``` - -- [ ] **Step 2: Create retros/ placeholder** - - ```bash - mkdir -p planning/retros - touch planning/retros/.gitkeep - ``` - -- [ ] **Step 3: Remove the now-empty old lifecycle dirs** - - `planning/active/` still holds `.gitkeep`; `planning/archived/` is empty. - - ```bash - git rm -f planning/active/.gitkeep - rmdir planning/active planning/archived 2>/dev/null || true - ``` - -- [ ] **Step 4: Verify old dirs are gone and audits/ populated** - - ```bash - ls planning/audits/ # expect both findings files - test ! -d planning/active && echo "active gone" - test ! -d planning/archived && echo "archived gone" - ``` - -- [ ] **Step 5: Commit** - - ```bash - git add -A planning/ - git commit -m "docs: move audit findings to planning/audits, add retros, drop old active/archived - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 4: Rename deferred-work.md → deferred.md - -**Files:** -- Rename: `planning/deferred-work.md` → `planning/deferred.md` - -- [ ] **Step 1: Rename the file** - - ```bash - git mv planning/deferred-work.md planning/deferred.md - ``` - -- [ ] **Step 2: Fix the in-file relative link to active/** - - In `planning/deferred.md`, the intro links graduated items to - `[`active/`](active/)`. Update it to `[`changes/active/`](changes/active/)`. - -- [ ] **Step 3: Verify no other in-repo reference to the old name remains** - - ```bash - grep -rn "deferred-work.md" . --exclude-dir=.git || echo "no stale refs" - ``` - Expected: only the README reference (fixed in Task 6) may remain; note it and - move on. No other hits. - -- [ ] **Step 4: Commit** - - ```bash - git add -A planning/ - git commit -m "docs: rename deferred-work.md to deferred.md - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 5: Add the change.md template, update design/plan path refs - -**Files:** -- Create: `planning/_templates/change.md` -- Modify: `planning/_templates/plan.md` (Spec link path) -- Modify: `planning/_templates/design.md` (no path refs to fix — verify) - -- [ ] **Step 1: Create the lightweight-lane template** - - Write `planning/_templates/change.md`: - - ````markdown - --- - status: draft - date: YYYY-MM-DD - slug: my-change - supersedes: null - superseded_by: null - pr: null - outcome: null - --- - - # Change: One-line capitalized title - - **Lane:** lightweight — ≲30 LOC net, ≤2 files, no new file, no public-API - change, a single straightforward test. If it outgrows this, split into - `design.md` + `plan.md`. - - ## Goal - - One or two sentences: what changes and why. - - ## Approach - - The shape of the change in brief — enough that a reviewer sees the design - without a full spec. Link the truth home (`architecture/.md`) if a - capability contract moves. - - ## Files - - - `path/to/file.py` — what changes - - `tests/test_x.py` — test added / updated - - ## Verification - - - [ ] Failing test first — command + expected error. - - [ ] Apply the change. - - [ ] Test passes — command. - - [ ] `just test` — full suite green. - - [ ] `just lint` — clean. - ```` - -- [ ] **Step 2: Update the plan template's Spec link** - - In `planning/_templates/plan.md`, change the **Spec:** line from - `[`planning/active/YYYY-MM-DD-my-change-design.md`](./YYYY-MM-DD-my-change-design.md)` - to `[`design.md`](./design.md)` (siblings now share a bundle folder). - -- [ ] **Step 3: Confirm design.md template needs no path edit** - - ```bash - grep -n "planning/active\|planning/archived" planning/_templates/design.md || echo "clean" - ``` - Expected: `clean`. - -- [ ] **Step 4: Commit** - - ```bash - git add -A planning/_templates/ - git commit -m "docs: add change.md lightweight template, fix plan template spec link - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 6: Rewrite planning/README.md (portable Conventions + Index) - -**Files:** -- Modify: `planning/README.md` (full rewrite) - -- [ ] **Step 1: Replace the file with the two-section form** - - Overwrite `planning/README.md` with the content below. The **Archived** - bullets are ported from the *current* README verbatim (same one-line - summaries), with each link rewritten to its new bundle path per the Task 2 - mapping table (`archived/--design.md` → - `changes/.NN-/design.md`). - - ````markdown - # Planning - - Specs, plans, and change history for `faststream-outbox`. The living truth - about *what the system does now* lives in [`architecture/`](../architecture/) - at the repo root; this directory records *how it got there*. - - ## Conventions - - > This section is the portable convention — identical across the - > modern-python repos. The Index below is repo-specific. To adopt elsewhere, - > copy this section plus [`_templates/`](_templates/) and point that repo's - > `CLAUDE.md` Workflow + truth home at it. - - ### Two axes, never mixed - - - **`architecture/` (repo root) — the present.** One file per capability, - living prose, updated whenever a change ships. The truth home. - - **`planning/changes/` — the past-and-pending.** One folder per change, - frozen once shipped. - - Shipping a change **promotes** its conclusions into the affected - `architecture/.md` by hand, then archives the bundle. That - hand-edit is what keeps `architecture/` true; the archived bundle carries the - *why*. - - ### Change bundles - - A change is a folder `changes/active/YYYY-MM-DD.NN-/`: - - - `YYYY-MM-DD` — proposal date; `.NN` — zero-padded intra-day counter - (`.01`, `.02`, …) that breaks same-date ties so the timeline sorts stably. - - `` — kebab-case description, not a story ID. - - On merge the folder moves to `changes/` with `status: shipped`, `pr:`, - and `outcome:` filled, and its line moves from **Active** to **Archived** in - the Index below. - - ### Three lanes - - | Lane | Artifacts | Use when | - |------|-----------|----------| - | **Full** | `design.md` + `plan.md` | design judgment; new file/module; public-API change; cross-cutting/multi-file; non-trivial test design | - | **Lightweight** | `change.md` | small-but-real: ≲30 LOC net, ≤2 files, no new file, no public-API change, single straightforward test | - | **Tiny** | none — conventional commit | typo, dep bump, linter/formatter/CI tweak, mechanical rename, single-line config | - - Heavier lane wins on ambiguity. A `change.md` that outgrows its lane splits - into `design.md` + `plan.md`. - - ### Artifacts at a glance - - - **`design.md`** — the spec: the *thinking* (why, design, trade-offs, scope). - - **`plan.md`** — the plan: the *sequencing* (the executor's task checklist). - - **`change.md`** — both, condensed, for the lightweight lane. - - **`releases/.md`** — per-release user-facing notes. - - **`audits/-.md`** — findings from a code/docs/bug-hunt sweep; - spawns fix changes. - - **`retros/-.md`** — what we learned after a body of work. - - **`deferred.md`** — real-but-unscheduled items, each with a revisit trigger. - - Templates live in [`_templates/`](_templates/). - - ### Frontmatter - - `design.md` / `change.md`: `status` (draft|approved|shipped|superseded), - `date`, `slug`, `supersedes`, `superseded_by`, `pr`, `outcome`. - `plan.md`: `status`, `date`, `slug`, `spec`, `pr`. Files in `architecture/` - carry **no** frontmatter — living prose, dated by git. - - ## Index - - ### Active - - _None._ - - ### Archived (shipped) - - - **[docs-tutorials](changes/2026-06-12.01-docs-tutorials/design.md)** - (#58, 2026-06-12) — The two tutorials deferred from #56: *Your first outbox - app* and *Add a Kafka relay*. Kill-Kafka step folded into an at-least-once - callout after `aiokafka` absorbed the outage on both attempts. - - **[docs-tutorials-and-observability-split](changes/2026-06-11.02-docs-tutorials-and-observability-split/design.md)** - (#56, 2026-06-12) — Three-way split of `usage/observability.md` into - Reference + How-to + Explanation; tutorials deferred to #58. - - **[operator-pages](changes/2026-06-11.01-operator-pages/design.md)** - (#53, 2026-06-11) — `docs/operations/`: Production checklist, Troubleshooting - playbook, Alembic migrations. The B follow-on from #50. - - **[docs-landing-and-comparison](changes/2026-06-10.02-docs-landing-and-comparison/design.md)** - (#50, 2026-06-10) — Docs landing rewrite, four-section nav reshape, new - Comparison page. - - **[planning-conventions](changes/2026-06-10.01-planning-conventions/design.md)** - (#49, 2026-06-10) — Spec/plan boundary, `active/`/`archived/`/`_templates/` - layout, frontmatter, migration of the existing pairs. *Superseded by - [portable-planning-convention](changes/2026-06-13.01-portable-planning-convention/design.md).* - - **[drain-test-flaky-fetch-observation](changes/2026-06-09.02-drain-test-flaky-fetch-observation/design.md)** - (#48, 2026-06-10) — Drain test waits via the `fetched` recorder instead of an - SQL poll, killing a 3.14 coverage flake. - - **[mkdocs-github-pages](changes/2026-06-09.01-mkdocs-github-pages/design.md)** - (#45, 2026-06-09) — Docs hosting moves from Read the Docs to GitHub Pages on - `faststream-outbox.modern-python.org`. - - **[foreign-broker-relay](changes/2026-06-04.02-foreign-broker-relay/design.md)** - (#44, 2026-06-05) — `OutboxSubscriber` officially supports the - FastStream-native decorator relay to Kafka/Rabbit/NATS/Redis with three - guardrails. - - **[faststream-0.7.1-testbroker-typing](changes/2026-06-04.01-faststream-0.7.1-testbroker-typing/design.md)** - (#43, 2026-06-04) — Adopt FastStream 0.7.1's `TestBroker[Broker, EnterType]` - typing fix; drop two `# ty: ignore` directives. - - **[faststream-0.7-migration](changes/2026-06-03.02-faststream-0.7-migration/design.md)** - (#42, 2026-06-03) — Migrate to `faststream>=0.7,<0.8`; fix mechanical break - points; drop per-call `middlewares=` kwarg. - - **[all-extra-and-planning-dir](changes/2026-06-03.01-all-extra-and-planning-dir/design.md)** - (#41, 2026-06-03) — Add `faststream-outbox[all]` aggregate extra; bootstrap - the `planning/` directory itself. - - ## Other - - - **[`architecture/`](../architecture/)** at the repo root — the living - capability truth (relay, timers, dlq, drain, metrics, test broker). This is - the promotion target on every ship. - - **[audits/](audits/)** — findings reports (2026-06-12 code + docs audits). - - **[lint-suppressions.md](lint-suppressions.md)** — repo-specific extra (not - part of the portable core): audit of `noqa` / `ty: ignore` directives and - why each one stays. - - **[deferred.md](deferred.md)** — the long-tail register of real-but- - unscheduled items with revisit triggers. - ```` - -- [ ] **Step 2: Verify every Index link resolves** - - ```bash - cd planning - grep -oE 'changes/[^)]+/design.md' README.md | while read p; do - test -f "$p" || echo "BROKEN: $p" - done; echo "link check done" - cd .. - ``` - Expected: `link check done` with no `BROKEN:` lines. (The - `2026-06-13.01-portable-planning-convention` link points into `changes/active/` - today; it resolves after Task 10 archives it — note it and move on.) - -- [ ] **Step 3: Commit** - - ```bash - git add planning/README.md - git commit -m "docs: rewrite planning/README.md as portable conventions + index - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 7: Mark planning-conventions superseded - -**Files:** -- Modify: `planning/changes/2026-06-10.01-planning-conventions/design.md` - -- [ ] **Step 1: Set superseded_by in frontmatter** - - In that `design.md`, change `superseded_by: null` to - `superseded_by: portable-planning-convention` and `status: shipped` to - `status: superseded`. - -- [ ] **Step 2: Add a callout at the top of the body** - - Immediately after the `# Design: …` H1, insert: - - ```markdown - > **Superseded by [portable-planning-convention](../2026-06-13.01-portable-planning-convention/design.md).** - > The `active/` + `archived/` lifecycle layout this spec introduced is replaced - > by the `changes/` (active → archive) + `architecture/`-promotion model. The - > migration this spec shipped still happened; only the layout was later - > reworked. - ``` - -- [ ] **Step 3: Verify** - - ```bash - grep -n "superseded" planning/changes/2026-06-10.01-planning-conventions/design.md - ``` - Expected: the frontmatter `status:`/`superseded_by:` lines plus the callout. - -- [ ] **Step 4: Commit** - - ```bash - git add planning/changes/2026-06-10.01-planning-conventions/design.md - git commit -m "docs: mark planning-conventions superseded by portable-planning-convention - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 8: Update CLAUDE.md Workflow section - -**Files:** -- Modify: `CLAUDE.md` (`## Workflow` section body only; `## Architecture` - untouched) - -- [ ] **Step 1: Replace the four Workflow paragraphs** - - Replace the entire body between the `## Workflow` heading and the - `## Architecture` heading with: - - ````markdown - Per-feature: brainstorming → spec in `planning/changes/active/YYYY-MM-DD.NN-/design.md` → writing-plans → plan in `planning/changes/active/YYYY-MM-DD.NN-/plan.md` → executing-plans / subagent-driven-development → requesting-code-review → finishing-a-development-branch. Each change is a folder bundle; `` is a kebab-case description, not a story ID; `.NN` is a zero-padded intra-day counter that breaks same-date ties so the timeline sorts stably. On merge, the bundle moves to `planning/changes/` with `status: shipped`, `pr:`, and `outcome:` filled, **and the change promotes its conclusions into the affected `architecture/.md`** — that hand-edit is what keeps `architecture/` true. See [`planning/README.md`](planning/README.md) for the conventions + index and [`planning/_templates/`](planning/_templates/) for copy-and-fill starting points. - - **Spec** (`design.md`) captures the *thinking* — why we are doing this, what the design is, what trade-offs were considered, what is out of scope. Written before code; rarely revised after merge. **Plan** (`plan.md`) captures the *sequencing* — the ordered checklist of tasks an executor (human or agent) walks. References the spec for the "why"; never re-explains it. **`architecture/`** captures the *invariants* of shipped systems — the living truth, promoted from a change on merge. A plan paragraph that would still read correctly with all task numbers and checkboxes removed is design content and belongs in the spec. - - **Three lanes.** Scale the artifact to the change. **Full** — a `design.md` + `plan.md` bundle — for real design judgment, a new file/module, a public-API change, cross-cutting/multi-file work, or non-trivial test design. **Lightweight** — a single `change.md` — for small-but-real changes (≲30 LOC net, ≤2 files, no new file, no public-API change, a single straightforward test). **Tiny** — no bundle, just a conventional commit — for a typo fix, dep bump, linter/formatter/CI tweak, a mechanical rename to satisfy a just-landed convention, or a single-line config change. Heavier lane wins on ambiguity; a `change.md` that outgrows its lane splits into `design.md` + `plan.md`. - ```` - -- [ ] **Step 2: Verify no stale planning path strings remain in CLAUDE.md** - - ```bash - grep -n "planning/active\|planning/archived" CLAUDE.md || echo "clean" - ``` - Expected: `clean`. - -- [ ] **Step 3: Commit** - - ```bash - git add CLAUDE.md - git commit -m "docs: update CLAUDE.md workflow for changes/ bundles + three lanes + promotion - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 9: Full verification pass - -**Files:** none (read-only checks). - -- [ ] **Step 1: No stale references anywhere** - - ```bash - grep -rn "planning/active\|planning/archived\|deferred-work.md" . \ - --exclude-dir=.git --exclude-dir=site --exclude-dir=.venv \ - | grep -v "planning/changes/active/2026-06-13.01-portable-planning-convention/" \ - || echo "no stale refs" - ``` - Expected: `no stale refs`. (The exclusion covers this change's own bundle, - whose plan.md documents the old paths in the mapping table — that's history, - not a live link.) - -- [ ] **Step 2: Docs build is strict-clean** - - ```bash - just docs-build - ``` - Expected: `mkdocs build --strict` succeeds — confirms no `docs/` link broke - (none should; `docs/` and `architecture/` were untouched). - -- [ ] **Step 3: Lint passes** - - ```bash - just lint-ci - ``` - Expected: clean (eof-fixer, ruff format check, markdown). No code touched. - -- [ ] **Step 4: Tree matches the spec's §2** - - ```bash - find planning architecture -maxdepth 2 -type d | sort - ``` - Expected (no `planning/active`, no `planning/archived`): - ``` - architecture - planning - planning/_templates - planning/audits - planning/changes - planning/changes/active - planning/changes/archive - planning/releases - planning/retros - ``` - -- [ ] **Step 5: Commit any lint fixups** - - ```bash - git add -A - git diff --cached --quiet && echo "nothing to commit" || \ - git commit -m "docs: lint fixups for planning convention migration - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 10: On-merge archival of this bundle (do at merge time) - -**Files:** -- Move: `planning/changes/active/2026-06-13.01-portable-planning-convention/` - → `planning/changes/2026-06-13.01-portable-planning-convention/` - -This is the standard "on merge" step the new convention prescribes, applied to -this change itself. Do it once the PR number is known (open the PR first), as -the final commit on the branch. - -- [ ] **Step 1: Archive the bundle** - - ```bash - mkdir -p planning/changes/2026-06-13.01-portable-planning-convention - git mv planning/changes/active/2026-06-13.01-portable-planning-convention/design.md \ - planning/changes/2026-06-13.01-portable-planning-convention/design.md - git mv planning/changes/active/2026-06-13.01-portable-planning-convention/plan.md \ - planning/changes/2026-06-13.01-portable-planning-convention/plan.md - ``` - -- [ ] **Step 2: Fill shipped frontmatter** - - In both files set `status: shipped`; in `design.md` set `pr: ""` and - `outcome: "merged 2026-06-13 as #"`; in `plan.md` set `pr: ""`. Replace - `` with the actual PR number. - -- [ ] **Step 3: Move the README Index line from Active to Archived** - - In `planning/README.md`, add a bullet under **Archived (shipped)** (newest, - at the top): - - ```markdown - - **[portable-planning-convention](changes/2026-06-13.01-portable-planning-convention/design.md)** - (#, 2026-06-13) — Two-axis OpenSpec-shaped planning convention: - `architecture/` truth + `changes/` folder bundles, `.NN` tiebreak, three - lanes, `audits/`+`retros/`. Supersedes planning-conventions. - ``` - - The **Active** section returns to `_None._`. - -- [ ] **Step 4: Verify active/ is empty and re-run the link check** - - ```bash - ls planning/changes/active/ # expect only .gitkeep - cd planning && grep -oE 'changes/(archive|active)/[^)]+/design.md' README.md \ - | while read p; do test -f "$p" || echo "BROKEN: $p"; done; echo "ok"; cd .. - ``` - Expected: `ok`, no `BROKEN:` lines (the portable-planning-convention link now - resolves under `changes/`). - -- [ ] **Step 5: Commit** - - ```bash - git add -A planning/ - git commit -m "docs: archive portable-planning-convention bundle (#) - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -## Self-review notes - -- **Spec coverage:** §2 layout → Tasks 1–3; §3 `.NN` ids → Task 2 mapping; §4 - lanes → Tasks 5, 6, 8; §5 promotion → documented in README (Task 6) + CLAUDE.md - (Task 8); §6 frontmatter → templates (Task 5) + README (Task 6); §7 - audits/retros → Task 3; §8 releases/deferred/templates → Tasks 4, 5; §9 README - → Task 6; §10 CLAUDE.md → Task 8; §11 migration → Tasks 1–4, 7, 10. - `releases/` is intentionally untouched (no task needed). -- **Promotion has no library-capability target here** (the convention lives in - README, not `architecture/`), so this change's own ship skips spec-promotion — - consistent with spec §11 step 9. -- **Same-date collisions** in the existing archive (`-03`, `-04`, `-09`, `-10`, - `-11`) are resolved by the explicit mapping table, not left to the executor. diff --git a/planning/changes/2026-06-16.01-actionable-schema-drift-error/design.md b/planning/changes/2026-06-16.01-actionable-schema-drift-error.md similarity index 100% rename from planning/changes/2026-06-16.01-actionable-schema-drift-error/design.md rename to planning/changes/2026-06-16.01-actionable-schema-drift-error.md diff --git a/planning/changes/2026-06-16.01-actionable-schema-drift-error/plan.md b/planning/changes/2026-06-16.01-actionable-schema-drift-error/plan.md deleted file mode 100644 index 19e156a..0000000 --- a/planning/changes/2026-06-16.01-actionable-schema-drift-error/plan.md +++ /dev/null @@ -1,399 +0,0 @@ -# actionable-schema-drift-error — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use -> superpowers:subagent-driven-development (recommended) or -> superpowers:executing-plans to implement this plan task-by-task. Steps -> use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Make `validate_schema()`'s `RuntimeError` point operators to a -hand-written-migration recipe whenever it reports drift that -`alembic revision --autogenerate` cannot remediate (the `lease_ck` CHECK and the -partial-index predicates). - -**Spec:** [`design.md`](./design.md) - -**Branch:** `fix/actionable-schema-drift-error` - -**Commit strategy:** Per-task commits. - -**Tooling notes for the executor:** -- Pure-helper unit tests run with no Postgres: `uv run pytest tests/test_unit.py -k `. -- Integration tests need Postgres via docker compose: `just test tests/test_integration.py -k `. - Pass a single `-k` keyword — `just test -k "a or b"` word-splits and fails. -- Coverage gate is `--cov-fail-under=100` on the full `just test` run; partial - runs fail it, so use `--no-cov` while iterating and rely on the final full run. -- All imports go at module top — never inline (project convention). Type every - new test parameter, including fixtures. - ---- - -### Task 1: Add the pure message-composition helper (TDD) - -**Files:** -- Modify: `faststream_outbox/client.py` -- Test: `tests/test_unit.py` - -Introduce `_compose_schema_mismatch_message` plus the `_SCHEMA_MISMATCH_PREFIX` -and `_AUTOGEN_BLIND_HINT` constants. Pure function, fully unit-testable without -Postgres. - -- [ ] **Step 1: Write the failing tests** - - Add to `tests/test_unit.py`. Extend the existing client import on line 42 - (`from faststream_outbox.client import OutboxClient, _validate_schema_sync`) to - also import the new names: - - ```python - from faststream_outbox.client import ( - OutboxClient, - _AUTOGEN_BLIND_HINT, - _SCHEMA_MISMATCH_PREFIX, - _compose_schema_mismatch_message, - _validate_schema_sync, - ) - ``` - - Then add these three tests (place them near the other client unit tests): - - ```python - def test_compose_schema_mismatch_message_appends_hint_on_blind_drift() -> None: - msg = _compose_schema_mismatch_message( - ["missing CHECK constraint 'outbox_lease_ck' (expected '...')"], - has_blind_drift=True, - ) - assert msg.startswith(_SCHEMA_MISMATCH_PREFIX) - assert _AUTOGEN_BLIND_HINT in msg - assert "#fixing-drift-autogenerate-cant-see" in msg - - - def test_compose_schema_mismatch_message_omits_hint_without_blind_drift() -> None: - msg = _compose_schema_mismatch_message( - ["table 'outbox' missing column 'headers'"], - has_blind_drift=False, - ) - assert msg == _SCHEMA_MISMATCH_PREFIX + "table 'outbox' missing column 'headers'" - assert _AUTOGEN_BLIND_HINT not in msg - - - def test_compose_schema_mismatch_message_joins_multiple_errors() -> None: - msg = _compose_schema_mismatch_message(["a", "b"], has_blind_drift=False) - assert msg == _SCHEMA_MISMATCH_PREFIX + "a; b" - ``` - -- [ ] **Step 2: Run the tests to verify they fail** - - Run: `uv run pytest tests/test_unit.py -k compose_schema_mismatch --no-cov -v` - Expected: collection/import error or FAIL — `_compose_schema_mismatch_message` - (and the two constants) do not exist yet. - -- [ ] **Step 3: Implement the helper in `client.py`** - - Add the following at module scope in `faststream_outbox/client.py`, immediately - after the `_validate_check_constraints_sync` function (after the block ending at - the current line 576), so all schema-validation machinery stays grouped: - - ```python - # The published docs anchor for hand-written migrations that fix drift - # `alembic revision --autogenerate` cannot emit (no check-constraint comparator; - # the index comparator ignores postgresql_where). Appended to the RuntimeError - # only when an Alembic-blind probe actually fired — see validate_schema(). - _SCHEMA_MISMATCH_PREFIX = "Outbox schema mismatch: " - _AUTOGEN_BLIND_HINT = ( - "These (CHECK constraints and partial-index predicates) are invisible to " - "'alembic revision --autogenerate' — hand-write the migration: " - "https://faststream-outbox.modern-python.org/operations/alembic/" - "#fixing-drift-autogenerate-cant-see" - ) - - - def _compose_schema_mismatch_message(errors: list[str], *, has_blind_drift: bool) -> str: - """Build the validate_schema RuntimeError text; append the remediation pointer for Alembic-blind drift.""" - msg = _SCHEMA_MISMATCH_PREFIX + "; ".join(errors) - if has_blind_drift: - msg += "\n\n" + _AUTOGEN_BLIND_HINT - return msg - ``` - -- [ ] **Step 4: Run the tests to verify they pass** - - Run: `uv run pytest tests/test_unit.py -k compose_schema_mismatch --no-cov -v` - Expected: 3 passed. - -- [ ] **Step 5: Commit** - - ```bash - git add faststream_outbox/client.py tests/test_unit.py - git commit -m "feat(client): add schema-mismatch message composer with autogen-blind hint - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 2: Wire `validate_schema()` to the helper and assert the pointer in integration tests - -**Files:** -- Modify: `faststream_outbox/client.py:401-425` (the `validate_schema` method) -- Test: `tests/test_integration.py` - -Track the Alembic-blind probe errors separately and raise through the new helper. -Cover the present/absent pointer behaviour against real Postgres. - -- [ ] **Step 1: Update the integration tests (failing assertions first)** - - In `tests/test_integration.py`, edit - `test_validate_schema_fails_when_lease_check_constraint_missing` (currently - lines 1919-1929) to capture the exception and assert the pointer: - - ```python - async def test_validate_schema_fails_when_lease_check_constraint_missing( - pg_engine: AsyncEngine, - outbox_table: Table, - ) -> None: - """A dropped ``_lease_ck`` CHECK must be caught — alembic's diff can't see it (audit 2026-06-14).""" - drop_sql = f'ALTER TABLE "{outbox_table.name}" DROP CONSTRAINT "{outbox_table.name}_lease_ck"' - async with pg_engine.begin() as conn: - await conn.exec_driver_sql(drop_sql) - client = OutboxClient(pg_engine, outbox_table) - with pytest.raises(RuntimeError, match="missing CHECK constraint") as excinfo: - await client.validate_schema() - assert "operations/alembic/#fixing-drift-autogenerate-cant-see" in str(excinfo.value) - ``` - - And edit `test_validate_schema_fails_when_columns_missing` (currently lines - 454-461) to assert the pointer is **absent** for autogenerate-fixable drift: - - ```python - async def test_validate_schema_fails_when_columns_missing(pg_engine, outbox_table) -> None: - """Drop a column the package expects and verify validate_schema reports it.""" - drop_sql = f'ALTER TABLE "{outbox_table.name}" DROP COLUMN headers' - async with pg_engine.begin() as conn: - await conn.exec_driver_sql(drop_sql) - client = OutboxClient(pg_engine, outbox_table) - with pytest.raises(RuntimeError, match="missing column 'headers'") as excinfo: - await client.validate_schema() - assert "fixing-drift-autogenerate-cant-see" not in str(excinfo.value) - ``` - -- [ ] **Step 2: Run the two tests to verify the new assertions fail** - - Run: `just test tests/test_integration.py -k test_validate_schema_fails_when_lease_check_constraint_missing` - Expected: FAIL — the current message has no pointer, so the new - `assert ... in str(excinfo.value)` fails. - (The columns-missing test still passes at this point — the pointer is already - absent — but its new negative assertion guards Step 3.) - -- [ ] **Step 3: Rewire `validate_schema` in `client.py`** - - Replace the body of `validate_schema` (lines 410-425) — keep the docstring - above it unchanged. Old: - - ```python - async with self._engine.connect() as conn: - errors = await conn.run_sync(_validate_schema_sync, self._table) - # S2: alembic's autogenerate diff compares index columns + uniqueness but NOT - # the partial-index WHERE predicate, so a wrong postgresql_where slips through - # and later breaks the producer's ON CONFLICT arbiter. Probe the predicates - # directly against the live catalog. - errors.extend(await conn.run_sync(_validate_index_predicates_sync, self._table)) - # Alembic's compare_metadata has no check-constraint comparator, so a missing - # or altered
_lease_ck (the half-set-lease guard) passes the diff above - # silently. Probe pg_constraint directly, mirroring the partial-index probe. - errors.extend(await conn.run_sync(_validate_check_constraints_sync, self._table)) - if self._dlq_table is not None: - errors.extend(await conn.run_sync(_validate_dlq_schema_sync, self._dlq_table)) - if errors: - msg = "Outbox schema mismatch: " + "; ".join(errors) - raise RuntimeError(msg) - ``` - - New: - - ```python - async with self._engine.connect() as conn: - errors = await conn.run_sync(_validate_schema_sync, self._table) - # S2 / lease_ck: these two probes catch drift that `alembic revision - # --autogenerate` cannot remediate — its index comparator ignores - # postgresql_where and it has no check-constraint comparator at all. - # Collect them separately so the raised error can point operators at - # the hand-written-migration recipe (_AUTOGEN_BLIND_HINT) only when - # one of them actually fired. - blind_errors = await conn.run_sync(_validate_index_predicates_sync, self._table) - blind_errors.extend(await conn.run_sync(_validate_check_constraints_sync, self._table)) - errors.extend(blind_errors) - if self._dlq_table is not None: - errors.extend(await conn.run_sync(_validate_dlq_schema_sync, self._dlq_table)) - if errors: - raise RuntimeError( - _compose_schema_mismatch_message(errors, has_blind_drift=bool(blind_errors)), - ) - ``` - -- [ ] **Step 4: Run the affected integration tests to verify they pass** - - Run each separately (single `-k` keyword each): - - `just test tests/test_integration.py -k test_validate_schema_fails_when_lease_check_constraint_missing` - - `just test tests/test_integration.py -k test_validate_schema_fails_when_columns_missing` - - `just test tests/test_integration.py -k test_validate_schema_fails_when_lease_check_constraint_predicate_wrong` - Expected: PASS for each. The third confirms the predicate-drift path also still - raises (it now flows through the helper with `has_blind_drift=True`). - -- [ ] **Step 5: Commit** - - ```bash - git add faststream_outbox/client.py tests/test_integration.py - git commit -m "fix(client): point operators at the migration recipe on autogen-blind drift - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 3: Document the hand-written-migration recipe - -**Files:** -- Modify: `docs/operations/alembic.md` -- Modify: `docs/usage/schema-validation.md` - -Add the anchored section the error points to, and cross-link it from the schema -validation page. - -- [ ] **Step 1: Add the recipe section to `docs/operations/alembic.md`** - - Insert the following **between** the end of the "Drift detection in CI" section - (after the server-defaults paragraph that ends at line 158) and the - `## DLQ retention via partition drop { #dlq-retention-via-partition-drop }` - heading (line 160): - - ````markdown - ## Fixing drift autogenerate can't see { #fixing-drift-autogenerate-cant-see } - - Two kinds of drift that - [`validate_schema()`](../usage/schema-validation.md) reports **cannot** be - remediated by `alembic revision --autogenerate` — the same blindness that - let them drift in also stops autogenerate from emitting a fix: - - - **The `outbox_lease_ck` CHECK constraint.** Alembic's `compare_metadata` - has no check-constraint comparator, so a missing or altered CHECK never - appears in an autogenerated migration. - - **Partial-index predicates.** Alembic's index comparator ignores - `postgresql_where`, so an index that exists but was created non-partial, - with the wrong `WHERE`, or (for `outbox_timer_id_uq`) non-unique is - invisible to the diff. - - When `validate_schema()` raises for one of these, its error ends with a - pointer to this section. Re-running autogenerate produces an empty - `upgrade()` — hand-write the migration instead, then re-run - `validate_schema()` to confirm the drift is cleared. - - ### Restore the lease CHECK - - ```python - # Drop first ONLY if the constraint exists with a wrong predicate; skip the - # drop if it is absent entirely. - op.drop_constraint('outbox_lease_ck', 'outbox', type_='check') - op.create_check_constraint( - 'outbox_lease_ck', - 'outbox', - '(acquired_token IS NULL) = (acquired_at IS NULL)', - ) - ``` - - ### Restore a partial index - - Drop the drifted index and recreate it with its load-bearing predicate. The - three indexes and their expected shape: - - | Index | Columns | Unique | `postgresql_where` | - | --- | --- | --- | --- | - | `outbox_pending_idx` | `queue, next_attempt_at` | no | `acquired_token IS NULL` | - | `outbox_lease_idx` | `queue, acquired_at` | no | `acquired_token IS NOT NULL` | - | `outbox_timer_id_uq` | `queue, timer_id` | yes | `timer_id IS NOT NULL` | - - ```python - op.drop_index('outbox_timer_id_uq', table_name='outbox') - op.create_index( - 'outbox_timer_id_uq', - 'outbox', - ['queue', 'timer_id'], - unique=True, - postgresql_where=sa.text('timer_id IS NOT NULL'), - ) - ``` - - Substitute the columns / `unique` / predicate from the table above for - `outbox_pending_idx` and `outbox_lease_idx`. - ```` - -- [ ] **Step 2: Cross-link from `docs/usage/schema-validation.md`** - - Insert this paragraph immediately after the "Extras are intentionally ignored" - paragraph (after line 39, before the `!!! warning "Server defaults..."` - admonition): - - ```markdown - Some drift cannot be fixed by re-running `alembic revision --autogenerate` — a - missing/altered `outbox_lease_ck` CHECK or a drifted partial-index predicate. - For those, the `RuntimeError` ends with a pointer to - [Alembic migrations § Fixing drift autogenerate can't see](../operations/alembic.md#fixing-drift-autogenerate-cant-see), - which holds the hand-written migration recipe. - ``` - -- [ ] **Step 3: Build the docs strict to verify anchors resolve** - - Run: `just docs-build` - Expected: `mkdocs build --strict` succeeds with no warnings — the new - `#fixing-drift-autogenerate-cant-see` anchor resolves and the cross-link from - `schema-validation.md` does not 404. - -- [ ] **Step 4: Commit** - - ```bash - git add docs/operations/alembic.md docs/usage/schema-validation.md - git commit -m "docs: hand-written migration recipe for autogen-blind drift - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 4: Full verification - -**Files:** none (verification only). - -Confirm lint, the full test suite, and the 100% coverage gate all pass with the -changes in place. - -- [ ] **Step 1: Lint** - - Run: `just lint-ci` - Expected: clean — `eof-fixer`, `ruff format --check`, `ruff check`, `ty check` - all pass. (If `_compose_schema_mismatch_message` or the constants trip an unused - import in `test_unit.py`, fix the import list rather than suppressing.) - -- [ ] **Step 2: Full test suite with coverage gate** - - Run: `just test` - Expected: all tests pass and `--cov-fail-under=100` is satisfied — the helper's - three branches are covered by the Task 1 unit tests; the `has_blind_drift` True - and False paths in `validate_schema` are covered by the Task 2 integration tests. - -- [ ] **Step 3: Commit (only if Step 1/2 required fixups)** - - ```bash - git add -A - git commit -m "chore: lint/coverage fixups for autogen-blind drift hint - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -## On merge - -Move this bundle to `planning/changes/` with `status: shipped`, `pr:`, -and `outcome:` filled, and promote the conclusion into the affected -architecture record: note in `CLAUDE.md`'s "User-owned schema" section (and any -`architecture/` deep-dive covering `validate_schema`) that an Alembic-blind -drift error now carries a remediation pointer to -`docs/operations/alembic.md#fixing-drift-autogenerate-cant-see`. diff --git a/planning/changes/2026-06-19.01-messaging-service-patterns-doc/design.md b/planning/changes/2026-06-19.01-messaging-service-patterns-doc.md similarity index 100% rename from planning/changes/2026-06-19.01-messaging-service-patterns-doc/design.md rename to planning/changes/2026-06-19.01-messaging-service-patterns-doc.md diff --git a/planning/changes/2026-06-19.01-messaging-service-patterns-doc/plan.md b/planning/changes/2026-06-19.01-messaging-service-patterns-doc/plan.md deleted file mode 100644 index d62ea17..0000000 --- a/planning/changes/2026-06-19.01-messaging-service-patterns-doc/plan.md +++ /dev/null @@ -1,410 +0,0 @@ -# messaging-service-patterns-doc — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use -> superpowers:subagent-driven-development (recommended) or -> superpowers:executing-plans to implement this plan task-by-task. Steps -> use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Add a "Patterns" docs page that shows one anonymized chat / -notifications service composing the transactional event relay, the -fire-unless-cancelled timer, and the in-process test of both. - -**Spec:** [`design.md`](./design.md) - -**Branch:** `docs/messaging-service-patterns` (already checked out; spec already -committed there). - -**Commit strategy:** Per-task commits. - -## Global constraints - -- Docs are MkDocs Material; build with `just docs-build` (= `mkdocs build - --strict`). `--strict` fails on broken internal links, so every cross-link - must resolve. -- Page is **anonymized** — a generic chat / notifications service, never the - private source service by name. -- Coverage is the **core trio only** (relay, timer, testing). No - production-hardening / next-steps prose — hardening features get one-line - "See also" links only. -- Code snippets use a **real DI container** (`modern-di` / `modern-di-faststream`) - and only the public API: `OutboxBroker`, `OutboxRouter`, `make_outbox_table`, - `TestOutboxBroker`, and broker methods `publish` / `cancel_timer` / - `fetch_unprocessed`. Snippets are illustrative (not CI-executed). -- Two queues throughout: `chat-events` and `unread-timers`. - ---- - -### Task 1: Create the page and add the Patterns nav section - -**Files:** -- Create: `docs/patterns/messaging-service.md` -- Modify: `mkdocs.yml` (insert a `Patterns` nav section between `Guides` and - `Reference`) - -This task delivers the whole page plus its nav entry, verified by a strict -docs build. - -- [ ] **Step 1: Create `docs/patterns/messaging-service.md` with exactly this content** - - Write the file with the content inside the four-backtick fence below (the - inner three-backtick blocks are part of the file): - -````markdown -# A messaging service, end-to-end - -The [tutorials](../tutorials/first-outbox-app.md) build a greenfield app one -primitive at a time, and each guide documents one feature on its own. This page -is different: it walks a single service that **composes** three outbox -primitives — a transactional event relay, a fire-unless-cancelled timer, and an -in-process test of the whole chain. - -The service is a generic chat / notifications backend. Users post messages into -chats. It has two obligations, and both must be **atomic with the database -write** — they commit with the domain row and must never fire if the -transaction rolls back: - -1. **Broadcast events.** Every message created, read, or deleted is published to - downstream consumers over Kafka. -2. **Unread notifications.** If a message is still unread `N` seconds after it - arrives, notify the recipient — *unless they read it first*. - -A plain message bus can't give you "commits with the domain row": publishing to -Kafka and committing to Postgres are two systems, so a crash between them either -drops the event or emits one for a transaction that rolled back. The outbox -makes the event a *row* written in the same transaction. Two queues carry the -two obligations: `chat-events` and `unread-timers`. - -## Architecture at a glance - -```text - CreateMessageUseCase ─┐ - (one DB transaction) ├─▶ outbox row: chat-events ─▶ subscriber ─▶ Kafka - └─▶ outbox row: unread-timers ─▶ subscriber ─▶ Kafka - ▲ - ReadMessageUseCase ── cancel_timer("unread-timers", timer_id) ┘ -``` - -- **Use cases** write domain rows and outbox rows in one transaction. -- **The broker** is an `OutboxBroker` over the application's `AsyncEngine`; the - outbox table lives on the app's own `MetaData` via `make_outbox_table`, so - Alembic owns its migrations. -- **Subscribers** poll each queue and relay the row onward to Kafka. - -```python title="tables.py" -from faststream_outbox import make_outbox_table -from sqlalchemy.orm import DeclarativeBase - - -class Base(DeclarativeBase): - pass - - -OUTBOX_TABLE = make_outbox_table(Base.metadata, table_name="outbox") -``` - -The broker and routers are wired with a real DI container -([modern-di](https://modern-di.readthedocs.io/) here, but any container works): - -```python title="ioc.py" -from modern_di import Group, Scope, providers -from faststream_outbox import OutboxBroker - -from app.tables import OUTBOX_TABLE - - -class Resources(Group): - outbox_broker = providers.Factory( - scope=Scope.APP, - creator=lambda engine: OutboxBroker(engine, outbox_table=OUTBOX_TABLE), - kwargs={"engine": Resources.database_engine}, - ) -``` - -## Pattern 1 — Transactional event relay - -A thin producer wraps `broker.publish`. Note the contract: `publish` inserts the -outbox row through the caller's `AsyncSession` but **does not flush, commit, or -open its own transaction** — the row commits with your domain writes. - -```python title="producers.py" -import dataclasses - -import pydantic -from faststream_outbox import OutboxBroker -from sqlalchemy.ext.asyncio import AsyncSession - - -class ChatEvent(pydantic.BaseModel): - type: str # "created" | "read" | "deleted" - message_id: int - chat_id: int - - -@dataclasses.dataclass(kw_only=True, slots=True, frozen=True) -class OutboxEventProducer: - outbox_broker: OutboxBroker - session: AsyncSession - - async def send_chat_event(self, event: ChatEvent) -> None: - await self.outbox_broker.publish( - event, queue="chat-events", session=self.session, - ) -``` - -The use case calls the producer **inside** its transaction, beside the domain -write, and commits once. If the commit fails, no event row exists; if it -succeeds, the event is guaranteed durable: - -```python title="use_cases.py" -import dataclasses - -from app.producers import ChatEvent, OutboxEventProducer -from app.repositories import MessagesRepository -from app.transaction import Transaction - - -@dataclasses.dataclass(kw_only=True, slots=True, frozen=True) -class CreateMessageUseCase: - transaction: Transaction - messages_repository: MessagesRepository - producer: OutboxEventProducer - - async def __call__(self, command: "CreateMessageCommand") -> None: - async with self.transaction: - message = await self.messages_repository.create(command.payload) - await self.producer.send_chat_event( - ChatEvent(type="created", message_id=message.id, chat_id=message.chat_id), - ) - await self.producer.arm_unread_timer(message) # Pattern 2, below - await self.transaction.commit() -``` - -A subscriber on `chat-events` reads the row and relays it to Kafka via a -DI-injected Kafka producer: - -```python title="handlers.py" -from faststream_outbox import OutboxRouter -from modern_di_faststream import FromDI - -from app.kafka import KafkaEventProducer -from app.producers import ChatEvent - -ROUTER = OutboxRouter() - - -@ROUTER.subscriber("chat-events") -async def relay_chat_event( - event: ChatEvent, - kafka_producer: KafkaEventProducer = FromDI(KafkaEventProducer), -) -> None: - await kafka_producer.publish_event(event) -``` - -Register the router on the broker with `broker.include_routers(ROUTER)`. - -> This service hand-rolls the Kafka hop through a DI'd producer, which keeps the -> Kafka client fully under your control. If you'd rather stack the relay as a -> single decorator over the subscriber, see -> [Relay to Kafka / RabbitMQ / NATS](../usage/relay.md). - -## Pattern 2 — Fire-unless-cancelled timer - -The unread notification is a **delayed** outbox row, armed in the same create -transaction. `timer_id` makes it idempotent; `activate_in` defers it: - -```python title="producers.py (continued)" -import datetime - -# inside OutboxEventProducer: - -UNREAD_DELAY = datetime.timedelta(seconds=30) - -async def arm_unread_timer(self, message: "Message") -> None: - await self.outbox_broker.publish( - ChatEvent(type="unread", message_id=message.id, chat_id=message.chat_id), - queue="unread-timers", - timer_id=str(message.id), - activate_in=UNREAD_DELAY, - session=self.session, - ) - -async def cancel_unread_timer(self, message_id: int) -> None: - await self.outbox_broker.cancel_timer( - queue="unread-timers", - timer_id=str(message_id), - session=self.session, - ) -``` - -When the recipient reads the message, a second use case **cancels** the timer in -its own transaction: - -```python title="use_cases.py (continued)" -@dataclasses.dataclass(kw_only=True, slots=True, frozen=True) -class ReadMessageUseCase: - transaction: Transaction - messages_repository: MessagesRepository - producer: OutboxEventProducer - - async def __call__(self, command: "ReadMessageCommand") -> None: - async with self.transaction: - message = await self.messages_repository.mark_read(command.message_id) - await self.producer.send_chat_event( - ChatEvent(type="read", message_id=message.id, chat_id=message.chat_id), - ) - await self.producer.cancel_unread_timer(message.id) - await self.transaction.commit() -``` - -Two properties make this safe, and one is a limit worth knowing: - -- **At-most-one-live.** `timer_id` deduplicates per `(queue, timer_id)`. Arming - the same id twice while a row is in flight is a no-op, so retries don't - produce two notifications. -- **Cancel is lease-guarded.** `cancel_timer` only deletes a row that is not yet - being delivered (it filters on an unheld lease) and returns `False` otherwise. -- **The race window is real.** Once the timer is leased for delivery, a read can - no longer cancel it — the notification fires. Downstream consumers should - tolerate the occasional already-read notification. - -More on scheduling semantics: [Timers](../usage/timers.md). - -## Pattern 3 — Testing the composed app - -Nest `TestOutboxBroker` and `TestKafkaBroker`. In the default **sync mode**, -`broker.publish` drives the subscriber in-process, so one call to a use case -runs the whole chain — outbox row → relay handler → Kafka — and you assert on -the Kafka test broker without any background loop: - -```python title="test_messaging.py" -from faststream.kafka import TestKafkaBroker -from faststream_outbox import TestOutboxBroker - - -async def test_create_message_relays_event_to_kafka( - outbox_broker, kafka_broker, create_message_use_case, command, kafka_publisher, -) -> None: - async with TestOutboxBroker(outbox_broker), TestKafkaBroker(kafka_broker): - await create_message_use_case(command) - - kafka_publisher.mock.assert_called_once() -``` - -Two caveats specific to this composition: - -- **Future-dated rows fire immediately in sync mode.** The 30-second - `unread-timers` row is dispatched at once, so a sync-mode test sees the - notification without waiting. To test the *delay* and the cancel race for - real, construct `TestOutboxBroker(outbox_broker, run_loops=True)` — that runs - the real fetch/worker loops against the in-memory store. -- **`validate_schema()` needs a real engine.** The fake client raises - `NotImplementedError`, so put the schema check in its own test against a real - `OutboxBroker`: - -```python title="test_schema.py" -from faststream_outbox import OutboxBroker - - -async def test_outbox_schema(outbox_broker: OutboxBroker) -> None: - await outbox_broker.validate_schema() -``` - -More on the test broker's two modes: [Testing](../usage/testing.md). - -## See also - -- [Relay to Kafka / RabbitMQ / NATS](../usage/relay.md) — the native relay - decorator, an alternative to the hand-rolled hop above. -- [Dead-letter queue](../usage/dlq.md) — archive terminal failures instead of - deleting them. -- [Observability](../usage/observability.md) — the metrics recorder and the - Prometheus / OpenTelemetry middleware. -```` - -- [ ] **Step 2: Add the Patterns nav section to `mkdocs.yml`** - - Insert this block between the `Guides:` section and the `Reference:` section - (i.e. after the `Setup Prometheus and OpenTelemetry` line, before `- Reference:`): - - ```yaml - - Patterns: - - 'A messaging service, end-to-end': patterns/messaging-service.md - ``` - -- [ ] **Step 3: Build the docs strictly** - - Run: `just docs-build` - Expected: `mkdocs build --strict` completes with no warnings or errors — in - particular no "contains a link to ... which is not found" and no "not found in - the documentation files" for `patterns/messaging-service.md`. - -- [ ] **Step 4: Commit** - - ```bash - git add docs/patterns/messaging-service.md mkdocs.yml - git commit -m "docs: add Patterns page composing the outbox in a service - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 2: Verify cross-links and snippet API accuracy - -**Files:** -- Modify (only if a check fails): `docs/patterns/messaging-service.md` - -Audit the page against the real docs tree and the real public API, fix any -drift, and confirm a clean strict build. - -- [ ] **Step 1: Confirm every cross-link target exists** - - Run: - ```bash - for f in tutorials/first-outbox-app.md usage/relay.md usage/timers.md \ - usage/testing.md usage/dlq.md usage/observability.md; do - test -f "docs/$f" && echo "OK $f" || echo "MISSING $f" - done - ``` - Expected: six `OK` lines, no `MISSING`. If any is missing, fix the link in the - page to the correct existing path. - -- [ ] **Step 2: Confirm every API symbol in the page is public** - - Run: - ```bash - python -c "import faststream_outbox as o; print(sorted(o.__all__))" - ``` - Confirm the page only references symbols that are exported (`OutboxBroker`, - `OutboxRouter`, `make_outbox_table`, `TestOutboxBroker`) plus the documented - broker methods `publish`, `cancel_timer`, `fetch_unprocessed`. Fix any symbol - that isn't. - -- [ ] **Step 3: Final strict build** - - Run: `just docs-build` - Expected: clean `mkdocs build --strict`. - -- [ ] **Step 4: Commit any fixes (skip if the working tree is clean)** - - ```bash - git add docs/patterns/messaging-service.md - git commit -m "docs: fix link/API drift on Patterns page - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -## Self-review - -- **Spec coverage:** scenario + why-outbox (Task 1 intro), architecture diagram - (Task 1), Pattern 1 relay (Task 1), Pattern 2 fire-unless-cancelled timer - (Task 1), Pattern 3 testing with both caveats + schema note (Task 1), See-also - footer (Task 1), Patterns nav section (Task 1 Step 2), strict-build + link/API - audit (Tasks 1 & 2). All spec sections map to a task. -- **Placeholders:** none — the full page content is inlined verbatim in Task 1. -- **Type/name consistency:** `OutboxEventProducer`, `ChatEvent`, - `CreateMessageUseCase`, `ReadMessageUseCase`, queues `chat-events` / - `unread-timers`, methods `send_chat_event` / `arm_unread_timer` / - `cancel_unread_timer` are used consistently across every snippet. diff --git a/planning/changes/2026-06-19.02-docs-diataxis-nav/design.md b/planning/changes/2026-06-19.02-docs-diataxis-nav.md similarity index 100% rename from planning/changes/2026-06-19.02-docs-diataxis-nav/design.md rename to planning/changes/2026-06-19.02-docs-diataxis-nav.md diff --git a/planning/changes/2026-06-19.02-docs-diataxis-nav/plan.md b/planning/changes/2026-06-19.02-docs-diataxis-nav/plan.md deleted file mode 100644 index c1af294..0000000 --- a/planning/changes/2026-06-19.02-docs-diataxis-nav/plan.md +++ /dev/null @@ -1,147 +0,0 @@ -# docs-diataxis-nav — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use -> superpowers:subagent-driven-development (recommended) or -> superpowers:executing-plans to implement this plan task-by-task. Steps -> use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Dissolve the standalone "Patterns" nav section by moving the -messaging-service page into the existing Guides section, with no renames. - -**Spec:** [`design.md`](./design.md) - -**Branch:** `docs/diataxis-nav` (already checked out; spec already committed). - -**Commit strategy:** Single commit — the file move, link fixups, nav change, and -index bullet are interdependent and must land together (a partial state fails -`--strict`). - -## Global constraints - -- MkDocs Material; build with `just docs-build` (= `mkdocs build --strict`). - `--strict` fails on broken internal links or orphaned pages. -- **No section renames** — Getting started / Concepts / Guides / Reference / - Operations keep their names and order. -- No page-body rewrite. Only the moved page's relative links change; its `# A - messaging service, end-to-end` H1 stays; no "Tutorial:" prefix. -- Destination path: `docs/usage/messaging-service.md` (where all Guides pages - live). The `docs/patterns/` directory must not exist after the change. - ---- - -### Task 1: Move the page into Guides and update nav + index - -**Files:** -- Move: `docs/patterns/messaging-service.md` → `docs/usage/messaging-service.md` -- Modify: `docs/usage/messaging-service.md` (relative-link fixups) -- Modify: `mkdocs.yml` (drop `Patterns:` section, add page under `Guides:`) -- Modify: `docs/index.md` (add one bullet to the `### Guides` list) - -This is the whole change; it is verified by a clean strict build. - -- [ ] **Step 1: Move the file with git** - - Run: - ```bash - git mv docs/patterns/messaging-service.md docs/usage/messaging-service.md - rmdir docs/patterns 2>/dev/null || true - ``` - Expected: the file is staged as a rename; `docs/patterns/` no longer exists. - -- [ ] **Step 2: Fix the relative links inside the moved page** - - In `docs/usage/messaging-service.md`, the `../usage/*` links must become - same-directory links. The `../tutorials/first-outbox-app.md` link is left - unchanged (`tutorials/` and `usage/` are both children of `docs/`). - - Apply these exact replacements (each `](../usage/X.md)` → `](X.md)`): - - `](../usage/relay.md)` → `](relay.md)` — occurs twice (the Pattern 1 note - and the See-also footer) - - `](../usage/timers.md)` → `](timers.md)` - - `](../usage/testing.md)` → `](testing.md)` - - `](../usage/dlq.md)` → `](dlq.md)` - - `](../usage/observability.md)` → `](observability.md)` - - Verify no `../usage/` remains and the tutorials link is intact: - ```bash - grep -n '](\.\./usage/' docs/usage/messaging-service.md # expect: no output - grep -n '](\.\./tutorials/first-outbox-app.md)' docs/usage/messaging-service.md # expect: 1 match - ``` - -- [ ] **Step 3: Update `mkdocs.yml` nav** - - Replace the `Guides:` section and the entire `Patterns:` section with the - Guides section below (this deletes `Patterns:` and appends the page as the - last Guides item). The block to replace currently reads: - - ```yaml - - Guides: - - FastAPI integration: usage/fastapi.md - - Relay to Kafka / RabbitMQ / NATS: usage/relay.md - - Timers: usage/timers.md - - Testing: usage/testing.md - - Schema validation: usage/schema-validation.md - - Setup Prometheus and OpenTelemetry: usage/setup-prometheus-opentelemetry.md - - Patterns: - - 'A messaging service, end-to-end': patterns/messaging-service.md - ``` - - Replace it with: - - ```yaml - - Guides: - - FastAPI integration: usage/fastapi.md - - Relay to Kafka / RabbitMQ / NATS: usage/relay.md - - Timers: usage/timers.md - - Testing: usage/testing.md - - Schema validation: usage/schema-validation.md - - Setup Prometheus and OpenTelemetry: usage/setup-prometheus-opentelemetry.md - - 'A messaging service, end-to-end': usage/messaging-service.md - ``` - -- [ ] **Step 4: Add the bullet to `index.md`'s Guides list** - - In `docs/index.md`, under `### Guides`, after the - `Setup Prometheus and OpenTelemetry` bullet (the last one before `### Reference`), - add: - - ```markdown - - [A messaging service, end-to-end](usage/messaging-service.md) — the relay, - timer, and testing guides composed in one service. - ``` - -- [ ] **Step 5: Strict build** - - Run: `just docs-build` - Expected: `mkdocs build --strict` completes with no warnings or errors (in - particular, no broken-link or orphaned-page error for - `usage/messaging-service.md`). - -- [ ] **Step 6: Verify no stale paths** - - Run: - ```bash - grep -rn 'patterns/' docs/ # expect: no output - test -d docs/patterns && echo "STILL EXISTS" || echo "gone" # expect: gone - ``` - -- [ ] **Step 7: Commit** - - ```bash - git add -A docs/ mkdocs.yml - git commit -m "docs: move the messaging-service page into Guides; drop Patterns section - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -## Self-review - -- **Spec coverage:** nav drop+add (Step 3), file move (Step 1), link fixups - table (Step 2), no intro retune (spec §3 — nothing to do), index bullet - (Step 4), strict build + grep checks (Steps 5–6). All spec sections mapped. -- **Placeholders:** none — every edit is given verbatim. -- **Consistency:** destination path `docs/usage/messaging-service.md` and the - nav target `usage/messaging-service.md` agree; the index link - `usage/messaging-service.md` agrees. diff --git a/planning/changes/2026-06-20.01-flat-changes-generated-index/design.md b/planning/changes/2026-06-20.01-flat-changes-generated-index.md similarity index 100% rename from planning/changes/2026-06-20.01-flat-changes-generated-index/design.md rename to planning/changes/2026-06-20.01-flat-changes-generated-index.md diff --git a/planning/changes/2026-06-20.01-flat-changes-generated-index/plan.md b/planning/changes/2026-06-20.01-flat-changes-generated-index/plan.md deleted file mode 100644 index ec874a0..0000000 --- a/planning/changes/2026-06-20.01-flat-changes-generated-index/plan.md +++ /dev/null @@ -1,560 +0,0 @@ -# flat-changes-generated-index — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use -> superpowers:subagent-driven-development (recommended) or -> superpowers:executing-plans to implement this plan task-by-task. Steps -> use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Flatten `planning/changes/`, make `status:` frontmatter the sole -lifecycle state, add a `summary:` field, and replace the hand-maintained README -Index with a stdlib generator (`just index`). - -**Spec:** [`design.md`](./design.md) - -**Branch:** `chore/flat-changes-generated-index` - -**Commit strategy:** Per-task commits. - -## Global constraints - -- Python 3.13+. **All imports at module top** — no inline/local imports, tests - included (CLAUDE.md). -- `ruff` runs `select = ["ALL"]`; the new `.py` files must pass `just lint` - (`eof-fixer`, `ruff format`, `ruff check --fix`, `ty check`). -- Coverage is `--cov=.` with `--cov-fail-under=100` — the whole repo root is - measured, so `planning/index.py` must reach 100% line coverage once a test - imports it. The `if __name__ == "__main__":` guard carries `# pragma: no - cover`. -- The generator is **stdlib-only** — no PyYAML (confirmed absent from the env). -- `summary:` frontmatter is **single-line** (no YAML folding) so the hand - parser stays trivial. -- Commit trailer: `Co-Authored-By: Claude Opus 4.8 (1M context) - `. - ---- - -### Task 1: Flatten `changes/`, backfill `summary:`, rewrite cross-links - -**Files:** -- Move: every `planning/changes/archive//` → `planning/changes//` -- Delete: `planning/changes/active/`, `planning/changes/archive/` (and their - `.gitkeep`) -- Modify: the `design.md` frontmatter of all 15 moved bundles (add `summary:`) -- Modify: internal `changes/archive/...` links in bundle bodies - -Collapse the two-level layout to a flat one, move each bundle's curated Index -line into its own `summary:` frontmatter, and repair links that referenced the -old `archive/` path. No generator yet — this task just reshapes the data. - -- [ ] **Step 1: Move every bundle up one level** - - ```bash - git mv planning/changes/archive/* planning/changes/ - git rm planning/changes/active/.gitkeep planning/changes/archive/.gitkeep - rmdir planning/changes/active planning/changes/archive - ``` - - Verify: `ls planning/changes/` shows the 15 archived bundles **plus** - `2026-06-20.01-flat-changes-generated-index/`, and no `active/` / `archive/`. - -- [ ] **Step 2: Backfill `summary:` into each bundle's `design.md`** - - In each file below, insert a `summary:` line **immediately after** the - `slug:` line in the frontmatter. Values (verbatim, single line): - - | bundle `design.md` | `summary:` value | - |---|---| - | `2026-06-19.02-docs-diataxis-nav` | `Dissolve the standalone Patterns nav section: fold the messaging-service case study into Guides and move the file to docs/usage/ (no section renames). Seven top-level sections to six.` | - | `2026-06-19.01-messaging-service-patterns-doc` | `New docs/patterns/ section with one page composing the outbox in an anonymized chat/notifications service: transactional event relay, fire-unless-cancelled timer, nested test brokers.` | - | `2026-06-16.01-actionable-schema-drift-error` | `validate_schema() appends a hand-written-migration pointer to its RuntimeError for Alembic-blind drift (the outbox_lease_ck CHECK and partial-index predicates autogenerate cannot remediate).` | - | `2026-06-13.01-portable-planning-convention` | `Two-axis OpenSpec-shaped convention: architecture/ truth + changes/ folder bundles, .NN intra-day tiebreak, three lanes, dedicated audits/+retros/, portable README. Supersedes planning-conventions.` | - | `2026-06-12.01-docs-tutorials` | `The two tutorials deferred from #56: Your first outbox app and Add a Kafka relay. Kill-Kafka step folded into an at-least-once callout after aiokafka absorbed the outage.` | - | `2026-06-11.02-docs-tutorials-and-observability-split` | `Three-way split of usage/observability.md into Reference + How-to + Explanation; tutorials deferred to #58.` | - | `2026-06-11.01-operator-pages` | `docs/operations/: Production checklist, Troubleshooting playbook, Alembic migrations. The B follow-on from #50.` | - | `2026-06-10.02-docs-landing-and-comparison` | `Docs landing rewrite, four-section nav reshape, new Comparison page.` | - | `2026-06-10.01-planning-conventions` | `Spec/plan boundary, active/archived/_templates layout, frontmatter, migration of the existing pairs. Superseded by portable-planning-convention.` | - | `2026-06-09.02-drain-test-flaky-fetch-observation` | `Drain test waits via the fetched recorder instead of an SQL poll, killing a 3.14 coverage flake.` | - | `2026-06-09.01-mkdocs-github-pages` | `Docs hosting moves from Read the Docs to GitHub Pages on faststream-outbox.modern-python.org.` | - | `2026-06-04.02-foreign-broker-relay` | `OutboxSubscriber officially supports the FastStream-native decorator relay to Kafka/Rabbit/NATS/Redis with three guardrails.` | - | `2026-06-04.01-faststream-0.7.1-testbroker-typing` | `Adopt FastStream 0.7.1 TestBroker[Broker, EnterType] typing fix; drop two ty:ignore directives.` | - | `2026-06-03.02-faststream-0.7-migration` | `Migrate to faststream>=0.7,<0.8; fix mechanical break points; drop per-call middlewares= kwarg.` | - | `2026-06-03.01-all-extra-and-planning-dir` | `Add faststream-outbox[all] aggregate extra; bootstrap the planning/ directory itself.` | - - Example (the `slug:` → `summary:` insertion in - `2026-06-19.02-docs-diataxis-nav/design.md`): - - ```yaml - status: shipped - date: 2026-06-19 - slug: docs-diataxis-nav - summary: Dissolve the standalone Patterns nav section: fold the messaging-service case study into Guides and move the file to docs/usage/ (no section renames). Seven top-level sections to six. - supersedes: null - superseded_by: null - pr: 104 - ``` - -- [ ] **Step 3: Verify every bundle now has a summary** - - Run: - - ```bash - for f in planning/changes/*/design.md planning/changes/*/change.md; do - [ -f "$f" ] && grep -q '^summary:' "$f" || echo "MISSING: $f" - done - ``` - - Expected: no output (every spec file has `summary:`; no `change.md` files - exist yet, so the glob for them is harmless). - -- [ ] **Step 4: Rewrite internal `changes/archive/...` links** - - Run `grep -rn 'changes/archive' planning CLAUDE.md` and rewrite each hit's - path from `changes/archive//` to `changes//`. Known hits: - - `planning/changes/2026-06-13.01-portable-planning-convention/design.md` - - `planning/changes/2026-06-13.01-portable-planning-convention/plan.md` - - `planning/changes/2026-06-16.01-actionable-schema-drift-error/plan.md` - - (CLAUDE.md hits are handled in Task 4; leave them for now.) Then verify the - bundle bodies are clean: - - ```bash - grep -rn 'changes/archive' planning/changes - ``` - - Expected: no output. - -- [ ] **Step 5: Commit** - - ```bash - git add -A planning/changes - git commit -m "refactor(planning): flatten changes/ and add summary frontmatter - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 2: The index generator + `just index` - -**Files:** -- Create: `planning/index.py` -- Create: `tests/test_planning_index.py` -- Modify: `Justfile` (add `index` recipe) - -A stdlib-only script that reads bundle frontmatter and prints a Markdown -listing grouped by lifecycle status to stdout. TDD: tests first. - -- [ ] **Step 1: Write the failing test** - - Create `tests/test_planning_index.py`: - - ```python - import importlib.util - import pathlib - - import pytest - - _INDEX_PATH = pathlib.Path(__file__).parent.parent / "planning" / "index.py" - _spec = importlib.util.spec_from_file_location("planning_index", _INDEX_PATH) - assert _spec is not None - assert _spec.loader is not None - index = importlib.util.module_from_spec(_spec) - _spec.loader.exec_module(index) - - - def test_parse_frontmatter_reads_fields_and_normalizes_null() -> None: - text = '---\nstatus: shipped\nslug: x\npr: "41"\nsupersedes: null\n---\nbody\n' - fields = index.parse_frontmatter(text) - assert fields["status"] == "shipped" - assert fields["slug"] == "x" - assert fields["pr"] == "41" # surrounding quotes stripped - assert fields["supersedes"] == "" # "null" normalized to empty - - - def test_parse_frontmatter_no_frontmatter_returns_empty() -> None: - assert index.parse_frontmatter("no leading marker\n") == {} - - - def test_parse_frontmatter_skips_lines_without_separator() -> None: - fields = index.parse_frontmatter("---\nstatus: draft\njunkline\n---\n") - assert fields == {"status": "draft"} - - - def test_load_bundles_reads_design_then_change_and_skips_others( - tmp_path: pathlib.Path, - monkeypatch: pytest.MonkeyPatch, - ) -> None: - changes = tmp_path / "changes" - (changes / "a-bundle").mkdir(parents=True) - (changes / "a-bundle" / "design.md").write_text( - "---\nstatus: shipped\ndate: 2026-01-02\nslug: a\nsummary: A.\npr: 1\n---\n", - encoding="utf-8", - ) - (changes / "b-bundle").mkdir() - (changes / "b-bundle" / "change.md").write_text( - "---\nstatus: draft\ndate: 2026-01-01\nslug: b\nsummary: B.\n---\n", - encoding="utf-8", - ) - (changes / "empty-bundle").mkdir() # no spec file -> skipped - (changes / "loose.txt").write_text("not a dir bundle", encoding="utf-8") - monkeypatch.setattr(index, "CHANGES_DIR", changes) - - bundles = index.load_bundles() - - slugs = {b["slug"] for b in bundles} - assert slugs == {"a", "b"} - a = next(b for b in bundles if b["slug"] == "a") - assert a["path"] == "changes/a-bundle/design.md" - b = next(b for b in bundles if b["slug"] == "b") - assert b["path"] == "changes/b-bundle/change.md" - - - def test_render_groups_sorts_and_renders_supersede_links() -> None: - bundles = [ - {"status": "draft", "date": "2026-02-01", "slug": "wip", - "pr": "", "summary": "Work in progress.", "path": "changes/wip/design.md"}, - {"status": "shipped", "date": "2026-01-02", "slug": "newer", - "pr": "10", "summary": "Newer.", "path": "changes/newer/design.md", - "supersedes": "older"}, - {"status": "shipped", "date": "2026-01-01", "slug": "older", - "pr": "9", "summary": "Older.", "path": "changes/older/design.md"}, - {"status": "superseded", "date": "2026-01-01", "slug": "gone", - "pr": "8", "summary": "", "path": "changes/gone/design.md", - "superseded_by": "newer"}, - ] - out = index.render(bundles) - assert "## In progress" in out - assert "## Shipped" in out - assert "## Superseded" in out - # In-progress entry has no pr -> em dash placeholder - assert "- **[wip](changes/wip/design.md)** (#—, 2026-02-01) — Work in progress." in out - # Shipped sorted newest first - assert out.index("newer") < out.index("older") - assert "_(supersedes older)_" in out - # Missing summary -> placeholder - assert "(no summary)" in out - assert "_(superseded by newer)_" in out - - - def test_render_empty_group_prints_none() -> None: - out = index.render([{"status": "shipped", "date": "2026-01-01", - "slug": "s", "pr": "1", "summary": "S.", - "path": "changes/s/design.md"}]) - # No draft/approved bundles -> In progress group is None - in_progress = out.split("## Shipped")[0] - assert "_None._" in in_progress - - - def test_main_writes_listing_to_stdout(capsys: pytest.CaptureFixture[str]) -> None: - rc = index.main() - captured = capsys.readouterr() - assert rc == 0 - assert "# Change index" in captured.out - ``` - -- [ ] **Step 2: Run the test to verify it fails** - - Run: `uv run pytest tests/test_planning_index.py -v --no-cov` - Expected: collection-time `FileNotFoundError` / `ModuleNotFoundError` for - `planning/index.py` (the file does not exist yet). - -- [ ] **Step 3: Write the generator** - - Create `planning/index.py`: - - ```python - """Generate the planning change index from bundle frontmatter. - - Run via ``just index``. Globs ``planning/changes/*/``, reads each bundle's - ``design.md`` (falling back to ``change.md``) frontmatter, and prints a - Markdown listing grouped by lifecycle status to stdout. Never writes a file: - the listing is a query over the bundles, not a committed artifact. - """ - - import pathlib - import sys - - CHANGES_DIR = pathlib.Path(__file__).parent / "changes" - GROUPS: tuple[tuple[str, tuple[str, ...]], ...] = ( - ("In progress", ("draft", "approved")), - ("Shipped", ("shipped",)), - ("Superseded", ("superseded",)), - ) - - - def parse_frontmatter(text: str) -> dict[str, str]: - """Parse a single-line-scalar YAML frontmatter block into a dict.""" - lines = text.splitlines() - if not lines or lines[0].strip() != "---": - return {} - fields: dict[str, str] = {} - for line in lines[1:]: - if line.strip() == "---": - break - key, sep, value = line.partition(": ") - if not sep: - continue - cleaned = value.strip().strip('"').strip("'") - fields[key.strip()] = "" if cleaned == "null" else cleaned - return fields - - - def load_bundles() -> list[dict[str, str]]: - """Read every bundle's spec frontmatter under ``CHANGES_DIR``.""" - bundles: list[dict[str, str]] = [] - for bundle in sorted(CHANGES_DIR.iterdir()): - if not bundle.is_dir(): - continue - spec = bundle / "design.md" - if not spec.exists(): - spec = bundle / "change.md" - if not spec.exists(): - continue - fields = parse_frontmatter(spec.read_text(encoding="utf-8")) - fields["path"] = f"changes/{bundle.name}/{spec.name}" - bundles.append(fields) - return bundles - - - def format_row(bundle: dict[str, str]) -> str: - """Render one bundle as a Markdown list item.""" - slug = bundle.get("slug", "?") - path = bundle.get("path", "") - pr = bundle.get("pr") or "—" - date = bundle.get("date", "") - summary = bundle.get("summary") or "(no summary)" - line = f"- **[{slug}]({path})** (#{pr}, {date}) — {summary}" - if bundle.get("supersedes"): - line += f" _(supersedes {bundle['supersedes']})_" - if bundle.get("superseded_by"): - line += f" _(superseded by {bundle['superseded_by']})_" - return line - - - def render(bundles: list[dict[str, str]]) -> str: - """Render the full grouped Markdown listing.""" - out = ["# Change index", "", "_Generated by `just index` — do not edit._", ""] - for title, statuses in GROUPS: - out += [f"## {title}", ""] - rows = sorted( - (b for b in bundles if b.get("status") in statuses), - key=lambda b: (b.get("date", ""), b.get("slug", "")), - reverse=True, - ) - out += [format_row(b) for b in rows] if rows else ["_None._"] - out.append("") - return "\n".join(out).rstrip() + "\n" - - - def main() -> int: - """Print the listing to stdout.""" - sys.stdout.write(render(load_bundles())) - return 0 - - - if __name__ == "__main__": # pragma: no cover - raise SystemExit(main()) - ``` - -- [ ] **Step 4: Run the test to verify it passes** - - Run: `uv run pytest tests/test_planning_index.py -v --no-cov` - Expected: all tests PASS. - -- [ ] **Step 5: Add the `just index` recipe** - - In `Justfile`, after the `lint-ci` block, add: - - ```just - # Print the planning change index (grouped by status) to stdout. - index: - uv run python planning/index.py - ``` - -- [ ] **Step 6: Smoke-run the generator against the real bundles** - - Run: `just index` - Expected: a `# Change index` heading, an `## In progress` group listing - `flat-changes-generated-index`, a `## Shipped` group with the 14 shipped - bundles newest-first, and a `## Superseded` group with `planning-conventions`. - No `(no summary)` placeholders. - -- [ ] **Step 7: Confirm full-suite coverage stays at 100%** - - Run: `uv run pytest tests/test_unit.py tests/test_planning_index.py` - Expected: PASS with coverage at 100% (the `__main__` guard is excluded via - `# pragma: no cover`). If `planning/index.py` shows a missing line, add a - test that exercises it. - -- [ ] **Step 8: Lint** - - Run: `just lint` - Expected: clean. Fix any ruff/ty findings on the two new files. - -- [ ] **Step 9: Commit** - - ```bash - git add planning/index.py tests/test_planning_index.py Justfile - git commit -m "feat(planning): add stdlib index generator and just index recipe - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 3: Slim the README and update templates - -**Files:** -- Modify: `planning/README.md` -- Modify: `planning/_templates/design.md` -- Modify: `planning/_templates/change.md` - -Delete the derived Index, rewrite the Conventions prose for the flat layout + -single-step lifecycle, and teach the templates the new `summary:` field. - -- [ ] **Step 1: Delete the Index, point at the generator** - - In `planning/README.md`, delete the entire `## Index` section (the `### - Active` and `### Archived (shipped)` subsections and every entry). Replace it - with: - - ```markdown - ## Index - - The change listing is **generated**, not maintained — run `just index` to - print it (grouped by `status`: In progress / Shipped / Superseded). The - frontmatter in each bundle is the single source of truth; there is no - committed copy to drift. - ``` - -- [ ] **Step 2: Rewrite the Conventions prose for the flat layout** - - In `planning/README.md`: - - In **"Two axes, never mixed"**, change the `planning/changes/` bullet to - describe a flat directory and drop "frozen once shipped / archives the - bundle". Replace the "Shipping a change promotes…then archives the bundle" - paragraph with: promotion into `architecture/.md` by hand is the - **only** ship-time step; there is no folder move. - - In **"Change bundles"**, change the path from - `changes/active/YYYY-MM-DD.NN-/` to - `changes/YYYY-MM-DD.NN-/`, and replace the "On merge the folder moves - to `changes/archive/` … and its line moves from Active to Archived" - paragraph with the single-step lifecycle: the implementing PR sets - `status: shipped` and fills `pr` / `outcome` / `summary` **in the branch**, - alongside the code and the `architecture/` promotion — no post-merge - bookkeeping. - - In **"Frontmatter"**, add `summary` (single line) to the `design.md` / - `change.md` field list. - -- [ ] **Step 3: Add `summary:` to the templates** - - In `planning/_templates/design.md` and `planning/_templates/change.md`, - insert into the frontmatter immediately after the `slug: my-change` line: - - ```yaml - summary: One line — shown in the generated index. Fill at ship time. - ``` - -- [ ] **Step 4: Verify no stale references remain in the README** - - Run: `grep -nE 'active/|archive/|### Active|### Archived' planning/README.md` - Expected: no output (all directory-split language is gone). - -- [ ] **Step 5: Commit** - - ```bash - git add planning/README.md planning/_templates - git commit -m "docs(planning): slim README to conventions, point Index at just index - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 4: Update CLAUDE.md Workflow section - -**Files:** -- Modify: `CLAUDE.md` (the `## Workflow` section) - -Bring the repo's agent instructions in line with the flat layout and -single-step lifecycle. - -- [ ] **Step 1: Rewrite the Workflow paragraph** - - In `CLAUDE.md` `## Workflow`: - - Change the per-feature path `planning/changes/active/YYYY-MM-DD.NN-/` - to `planning/changes/YYYY-MM-DD.NN-/` (both `design.md` and - `plan.md` occurrences). - - Replace "On merge, the bundle moves to `planning/changes/archive/` with - `status: shipped`, `pr:`, and `outcome:` filled, **and** the change - promotes its conclusions into the affected `architecture/.md`" - with: the implementing PR sets `status: shipped` and fills `pr` / `outcome` - / `summary` in-branch and promotes its conclusions into the affected - `architecture/.md` — the promotion is the only ship-time step; - there is no folder move. - - Add a sentence: the change listing is generated — run `just index` (no - committed Index). - -- [ ] **Step 2: Verify no stale path references remain** - - Run: `grep -nE 'changes/active|changes/archive' CLAUDE.md` - Expected: no output. - -- [ ] **Step 3: Commit** - - ```bash - git add CLAUDE.md - git commit -m "docs: update CLAUDE.md workflow for flat changes/ and just index - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 5: Ship-time bookkeeping (in-branch, per the new convention) - -**Files:** -- Modify: `planning/changes/2026-06-20.01-flat-changes-generated-index/design.md` -- Modify: `planning/changes/2026-06-20.01-flat-changes-generated-index/plan.md` - -Eat the new convention's own dog food: flip this bundle to `shipped` in the -same branch, before merge. Do this **last**, once the PR number is known. - -- [ ] **Step 1: Set the bundle frontmatter to shipped** - - In this bundle's `design.md` frontmatter set `status: shipped`, `pr: `, - and `outcome: `. In `plan.md` set `status: shipped` - and `pr: `. (`summary:` is already present.) - -- [ ] **Step 2: Confirm the generator places it under Shipped** - - Run: `just index` - Expected: `flat-changes-generated-index` now appears under `## Shipped`, and - `## In progress` shows `_None._`. - -- [ ] **Step 3: Final full verification** - - Run: `just lint-ci` and `just test` - Expected: both green. - -- [ ] **Step 4: Commit** - - ```bash - git add planning/changes/2026-06-20.01-flat-changes-generated-index - git commit -m "chore(planning): mark flat-changes-generated-index shipped - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -## Self-review notes - -- **Spec coverage:** §1 flat dir → Task 1; §2 `summary:` → Task 1 (backfill) + - Task 3 (templates); §3 generator → Task 2; §4 slim README → Task 3; §5 - single-step lifecycle → Tasks 3, 4, 5; §6 CLAUDE.md + cross-links → Task 1 - (bundle links) + Task 4 (CLAUDE.md). Testing/Risk greps map to the verify - steps in Tasks 1–4. -- **Type consistency:** `parse_frontmatter`, `load_bundles`, `format_row`, - `render`, `main`, `CHANGES_DIR`, `GROUPS` are referenced identically in the - generator and its test. diff --git a/planning/changes/2026-06-23.01-client-rules-kernel/design.md b/planning/changes/2026-06-23.01-client-rules-kernel.md similarity index 100% rename from planning/changes/2026-06-23.01-client-rules-kernel/design.md rename to planning/changes/2026-06-23.01-client-rules-kernel.md diff --git a/planning/changes/2026-06-23.01-client-rules-kernel/plan.md b/planning/changes/2026-06-23.01-client-rules-kernel/plan.md deleted file mode 100644 index 8fdbb6a..0000000 --- a/planning/changes/2026-06-23.01-client-rules-kernel/plan.md +++ /dev/null @@ -1,210 +0,0 @@ -# client-rules-kernel — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use -> superpowers:subagent-driven-development (recommended) or -> superpowers:executing-plans to implement this plan task-by-task. Steps -> use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Remove the hand-maintained rule duplication between `OutboxClient` and -`FakeOutboxClient` — share the pure bits, co-verify the SQL bits with one contract -suite — without changing any runtime behaviour. - -**Spec:** [`design.md`](./design.md) - -**Branch:** `refactor/client-rules-kernel` - -**Commit strategy:** Per-task commits. - ---- - -### Task 1: Extract `_scheduling.py` - -**Files:** -- Create: `faststream_outbox/_scheduling.py` -- Modify: `faststream_outbox/publisher/producer.py`, `faststream_outbox/broker.py`, `faststream_outbox/testing.py` - -Pull the three pure activate-args helpers into one leaf module; kill the inlined -copy in `publish_batch`. Pure move — no behaviour change. (Spec §2.) - -- [ ] **Step 1: Create the leaf module** - - New `faststream_outbox/_scheduling.py` depending only on `_time`, with: - `is_future_dated(activate_in, activate_at, now)`, - `resolve_next_attempt_client_side(activate_in, activate_at, now)`, - `validate_activate_args(method_name, activate_in, activate_at)`. - Bodies move verbatim from `producer.py:35` (`_is_future_dated`), `broker.py:73` - (`_compute_next_at_client_side`), `broker.py:59` (`_validate_activate_args`). - -- [ ] **Step 2: Rewire consumers** - - - `producer.py`: import from `_scheduling`; delete local `_is_future_dated`; in - `publish_batch` (`producer.py:175–201`) call `resolve_next_attempt_client_side` - instead of inlining `now + cmd.activate_in` / `cmd.activate_at`. - - `broker.py`: delete local `_compute_next_at_client_side` and - `_validate_activate_args`; update any in-module callers to import from `_scheduling`. - - `testing.py`: change the imports at `testing.py:27–28` to source from `_scheduling`. - -- [ ] **Step 3: Verify** - - `uv run pytest tests/test_unit.py tests/test_fake.py --no-cov -q` green. - `grep -rn "_compute_next_at_client_side\|_is_future_dated\|_validate_activate_args" faststream_outbox/` - shows definitions only in `_scheduling.py`. - -- [ ] **Step 4: Commit** - - ```bash - git add faststream_outbox/_scheduling.py faststream_outbox/publisher/producer.py faststream_outbox/broker.py faststream_outbox/testing.py - git commit -m "refactor(scheduling): consolidate activate-args helpers into _scheduling.py - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 2: Write the client contract suite (green against current code) - -**Files:** -- Create: `tests/test_client_contract.py` -- Modify: `tests/conftest.py` (per-adapter harness fixture) - -Establish the cross-adapter safety net *before* refactoring the DLQ projection, so -Task 3 is guarded. The suite must pass against the current code. (Spec §3.) - -- [ ] **Step 1: Add the harness fixture** - - In `conftest.py`, a fixture parametrized `client ∈ {fake, real}` yielding a harness - exposing `client`, `seed_row(**fields)` (fake: `.feed(...)`; real: raw `INSERT`), - and `open_conn()` (fake: `None`; real: `engine.connect()`). The `real` param uses - the existing `pg_engine` fixture and **skips when Postgres is unreachable** (mirror - `test_integration.py`'s skip gate). - -- [ ] **Step 2: Write the contract scenarios** - - In `test_client_contract.py`, adapter-agnostic scenarios covering the contract in - spec §3: `fetch` (unleased / future-dated / expired-lease / fresh-lease / FIFO / - limit / queue filter), claim side-effects, `delete_with_lease` (token match / - mismatch / NULL token / DLQ materialization), `mark_pending_with_lease`, - `cancel_timer`, `timer_id` dedup. Expectations hand-specified, not computed from a - shared function. - -- [ ] **Step 3: Verify both adapters** - - `uv run pytest tests/test_client_contract.py --no-cov -q` green for the fake param. - `just test tests/test_client_contract.py` green for both params (Postgres up). - -- [ ] **Step 4: Commit** - - ```bash - git add tests/test_client_contract.py tests/conftest.py - git commit -m "test(client): add contract suite run against both fake and real adapters - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 3: Extract `_DLQ_PROJECTION` (guarded by Task 2) - -**Files:** -- Modify: `faststream_outbox/schema.py`, `faststream_outbox/client.py`, `faststream_outbox/testing.py` - -Replace the hand-kept outbox→DLQ parity with one declarative projection. (Spec §1.) - -- [ ] **Step 1: Declare the projection** - - In `schema.py`, next to `make_dlq_table`, add `_DLQ_PROJECTION` — ordered - `(outbox_col, dlq_col)` pairs (`id→original_id`, `queue→queue`, `payload`, - `headers`, `deliveries_count`, `created_at`, `timer_id`). Note injected fields - (`failure_reason`, `last_exception`) in a comment. - -- [ ] **Step 2: Consume it in both adapters** - - - `client.py`: `_build_dlq_cte_stmt` (`:335–348`) builds its `RETURNING` / - `INSERT (...)` / `SELECT` column lists from `_DLQ_PROJECTION` (preserve the - identifier-quoting via the dialect preparer). Delete the `B10` literal-list risk. - - `testing.py`: `delete_with_lease` (`:183–196`) builds the DLQ dict from - `_DLQ_PROJECTION`. Delete the `# P9 parity` comment — parity is now structural. - -- [ ] **Step 3: Verify no drift** - - `just test tests/test_client_contract.py` still green for both params (the suite's - DLQ-materialization assertions now guard the refactor). Full `just test` green. - -- [ ] **Step 4: Commit** - - ```bash - git add faststream_outbox/schema.py faststream_outbox/client.py faststream_outbox/testing.py - git commit -m "refactor(dlq): derive outbox->DLQ projection from one declarative map - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 4: Replace, don't layer — prune redundant tests - -**Files:** -- Modify: `tests/test_fake.py`, `tests/test_integration.py` - -Delete per-adapter scenarios the contract suite now owns; keep suite-specific tests -(sync-vs-loop test broker, real-only schema validation, drain). (Spec §3.) - -- [ ] **Step 1: Identify overlap** - - Find tests in `test_fake.py` / `test_integration.py` whose assertions are now - subsumed by `test_client_contract.py` (eligibility, lease reclaim, token guard, - DLQ projection, timer dedup, cancel_timer). - -- [ ] **Step 2: Delete the overlap; keep the rest** - - Remove the subsumed cases. Leave anything the contract suite cannot express. - -- [ ] **Step 3: Verify coverage holds** - - `just test` green with `--cov-fail-under=100`. If coverage drops, a deleted test - covered a path the contract suite misses — restore it or extend the suite. - -- [ ] **Step 4: Commit** - - ```bash - git add tests/test_fake.py tests/test_integration.py - git commit -m "test: drop per-adapter cases now owned by the client contract suite - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` - ---- - -### Task 5: Promote docs (ship-time, in this PR) - -**Files:** -- Modify: `CLAUDE.md`, `architecture/dlq.md`, `architecture/test-broker.md` -- Modify: `planning/changes/2026-06-23.01-client-rules-kernel/design.md` (frontmatter) - -Record the new vocabulary where this project keeps it; close out the change. (Spec §4.) - -- [ ] **Step 1: CLAUDE.md + architecture/** - - - "DLQ projection" → **User-owned schema** + **DLQ** sections of `CLAUDE.md`; - `architecture/dlq.md`. - - "client contract" → **Test broker** section of `CLAUDE.md`; - `architecture/test-broker.md`. Note the documented clock-skew residual (the - contract pins structural drift, not cross-host clock authority). - -- [ ] **Step 2: Close out the change** - - Set `status: shipped`, fill `pr:` and `outcome:` in `design.md` (and `plan.md`). - `just index` to regenerate the listing. - -- [ ] **Step 3: Final verification** - - `just lint` and `just test` clean. - -- [ ] **Step 4: Commit** - - ```bash - git add CLAUDE.md architecture/ planning/ - git commit -m "docs: record DLQ projection + client contract; close out change - - Co-Authored-By: Claude Opus 4.8 (1M context) " - ``` diff --git a/planning/changes/2026-06-23.02-consolidate-lease-lost/change.md b/planning/changes/2026-06-23.02-consolidate-lease-lost.md similarity index 100% rename from planning/changes/2026-06-23.02-consolidate-lease-lost/change.md rename to planning/changes/2026-06-23.02-consolidate-lease-lost.md diff --git a/planning/changes/2026-06-23.03-self-validating-subscriber-config/change.md b/planning/changes/2026-06-23.03-self-validating-subscriber-config.md similarity index 100% rename from planning/changes/2026-06-23.03-self-validating-subscriber-config/change.md rename to planning/changes/2026-06-23.03-self-validating-subscriber-config.md diff --git a/planning/changes/2026-06-30.01-python-3.11-3.12-support/design.md b/planning/changes/2026-06-30.01-python-3.11-3.12-support.md similarity index 100% rename from planning/changes/2026-06-30.01-python-3.11-3.12-support/design.md rename to planning/changes/2026-06-30.01-python-3.11-3.12-support.md diff --git a/planning/changes/2026-06-30.01-python-3.11-3.12-support/plan.md b/planning/changes/2026-06-30.01-python-3.11-3.12-support/plan.md deleted file mode 100644 index cba3c83..0000000 --- a/planning/changes/2026-06-30.01-python-3.11-3.12-support/plan.md +++ /dev/null @@ -1,373 +0,0 @@ -# python-3.11-3.12-support — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use -> superpowers:subagent-driven-development (recommended) or -> superpowers:executing-plans to implement this plan task-by-task. Steps -> use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Lower the supported-Python floor from 3.13 to 3.11 so the package -installs and runs on 3.11, 3.12, 3.13, and 3.14. - -**Spec:** [`design.md`](./design.md) - -**Architecture:** Pure-Python library, no compiled extensions. One source -construct (`override`, added to `typing` in 3.12) is backported via -`typing_extensions` declared as a direct runtime dependency, with no -`sys.version_info` gating. Everything else is metadata (`requires-python`, -classifiers, ruff target) plus a wider CI matrix. - -**Tech Stack:** Python, faststream 0.7, sqlalchemy 2 (asyncio), anyio, -`typing_extensions`, uv (package manager), just (task runner), ruff + ty -(lint/type-check), pytest, docker compose (integration suite + Postgres 17). - -**Branch:** `feat/python-3.11-3.12-support` - -**Commit strategy:** Per-task commits. - -## Global Constraints - -- New Python floor: `requires-python = ">=3.11,<4"`. Every edit must stay valid - on CPython 3.11 through 3.14. -- `typing-extensions>=4.12.0` (matches faststream's existing transitive pin; - `override` has existed in `typing_extensions` since 4.4.0). -- All `import` statements at module top — never inside function bodies. Tests - included. No `# noqa: PLC0415` to keep an import inline; hoist instead. -- Type-checker suppressions use `# ty: ignore[...]`, never `# type: ignore`. -- `uv.lock` is git-ignored — regenerate locally so resolution succeeds, but do - **not** commit it. -- Coverage gate is 100% (`--cov-fail-under=100`) and only the full docker suite - enforces it. Local 3.11 subset runs use `--no-cov` (import/runtime smoke - check, not the coverage gate). -- Leave the existing `# ty: ignore[invalid-method-override]` comments on the - decorated methods untouched — unrelated to this change. -- The uv-managed 3.11 interpreter for local verification: - `/Users/kevinsmith/.local/share/uv/python/cpython-3.11.9-macos-aarch64-none/bin/python3.11` - ---- - -### Task 1: Make the package compatible with Python 3.11/3.12 - -Lower the floor, add the `typing_extensions` dependency, and reroute the -`override` import in the four affected source files. The pyproject floor change -and the source fixes are inseparable: the `override` fix can only be proven by -importing on a real 3.11 interpreter, which uv refuses until `requires-python` -is lowered; and ruff `target-version` must move to `py311` alongside the source -edits so the lint pass validates against the floor. - -**Files:** -- Modify: `pyproject.toml` (dependencies, `requires-python`, classifiers, - `[tool.ruff] target-version`) -- Modify: `faststream_outbox/registrator.py:3` -- Modify: `faststream_outbox/publisher/usecase.py:13` -- Modify: `faststream_outbox/broker.py` (import block + `@typing.override` at - 204/212/222/293/335) -- Modify: `faststream_outbox/subscriber/usecase.py` (import block + - `@typing.override` at 232/247/804/808/818/868) -- Regenerate (do **not** commit): `uv.lock` - -**Interfaces:** -- Consumes: nothing from earlier tasks. -- Produces: a package importable on 3.11. No public API names change; `override` - keeps its meaning (re-exported stdlib object on 3.12+, backport on 3.11). - -- [ ] **Step 1: Confirm the break on 3.11 (RED)** - - Show that the stdlib `override` import — used directly in two files and as - `typing.override` in two others — does not exist on 3.11: - - ```bash - PY311=/Users/kevinsmith/.local/share/uv/python/cpython-3.11.9-macos-aarch64-none/bin/python3.11 - $PY311 -c "from typing import override" - ``` - - Expected: FAIL with `ImportError: cannot import name 'override' from 'typing'`. - This is the failing state the task fixes. - -- [ ] **Step 2: Edit `pyproject.toml` — floor, dependency, classifiers, ruff target (all together)** - - Change the floor at line 9 from: - - ```toml - requires-python = ">=3.13,<4" - ``` - - to: - - ```toml - requires-python = ">=3.11,<4" - ``` - - In `classifiers` (lines 15-16), add 3.11 and 3.12 above 3.13 so the block reads: - - ```toml - "Programming Language :: Python :: 3.11", - "Programming Language :: Python :: 3.12", - "Programming Language :: Python :: 3.13", - "Programming Language :: Python :: 3.14", - ``` - - Change the `dependencies` block (lines 20-23) from: - - ```toml - dependencies = [ - "faststream>=0.7.1,<0.8", - "sqlalchemy[asyncio]>=2.0", - ] - ``` - - to: - - ```toml - dependencies = [ - "faststream>=0.7.1,<0.8", - "sqlalchemy[asyncio]>=2.0", - "typing-extensions>=4.12.0", - ] - ``` - - In `[tool.ruff]` (line 65), change: - - ```toml - target-version = "py313" - ``` - - to: - - ```toml - target-version = "py311" - ``` - -- [ ] **Step 3: Edit `faststream_outbox/registrator.py`** - - Line 3 is currently: - - ```python - from typing import TYPE_CHECKING, Any, override - ``` - - Remove `override` from it and add a `typing_extensions` import. The result - (ruff will normalize ordering in Step 7): - - ```python - from typing import TYPE_CHECKING, Any - - from typing_extensions import override - ``` - - The `@override` decorators at lines 41 and 99 are unchanged. - -- [ ] **Step 4: Edit `faststream_outbox/publisher/usecase.py`** - - Line 13 is currently: - - ```python - from typing import override - ``` - - Replace it with: - - ```python - from typing_extensions import override - ``` - - The `@override` decorators at lines 60, 104, 116 are unchanged. - -- [ ] **Step 5: Edit `faststream_outbox/broker.py`** - - This file uses `import typing` (line 11) and the qualified form - `@typing.override`. Add a direct import near the other third-party imports - (e.g. after the `from sqlalchemy...` lines around line 30): - - ```python - from typing_extensions import override - ``` - - Then replace the decorator at lines 204, 212, 222, 293, 335 — each currently: - - ```python - @typing.override - ``` - - with: - - ```python - @override - ``` - - Leave `import typing` in place (still used for `typing.Self`, `typing.Any`, - etc.). - -- [ ] **Step 6: Edit `faststream_outbox/subscriber/usecase.py`** - - This file uses `import typing` (line 27) and `@typing.override`. Add a direct - import near the other third-party imports (after the `from faststream...` - block, around line 38): - - ```python - from typing_extensions import override - ``` - - Then replace the decorator at lines 232, 247, 804, 808, 818, 868 — each - currently: - - ```python - @typing.override - ``` - - with: - - ```python - @override - ``` - - Leave `import typing` in place (still used elsewhere in the file). - -- [ ] **Step 7: Lint (sorts the new imports) and type-check** - - ```bash - just lint - ``` - - Expected: passes. `ruff --fix` sorts each new `from typing_extensions import - override` into the third-party group; `ty` resolves `override` from - `typing_extensions` cleanly. If `ty` reports anything, do not add new - suppressions — fix the import. - -- [ ] **Step 8: Regenerate the lockfile for the lowered floor (do not commit it)** - - ```bash - uv lock - ``` - - Expected: resolves `typing-extensions` and re-pins for `>=3.11`. `uv.lock` is - git-ignored; it will not appear in `git status`. - -- [ ] **Step 9: Prove the fix on real 3.11 (GREEN)** - - Sync deps for 3.11 and run the two no-Postgres suites (which exercise the - full import graph) without the coverage gate: - - ```bash - uv sync --python 3.11 --all-extras - uv run --python 3.11 --no-sync pytest tests/test_unit.py tests/test_fake.py --no-cov -q - ``` - - Expected: PASS. The package and all four edited modules import and run on - CPython 3.11. (Re-sync the default interpreter afterward with `uv sync - --all-extras` if you want the local venv back on 3.14.) - -- [ ] **Step 10: Commit** - - ```bash - git add pyproject.toml faststream_outbox/registrator.py \ - faststream_outbox/publisher/usecase.py faststream_outbox/broker.py \ - faststream_outbox/subscriber/usecase.py - git commit -m "feat: support Python 3.11 and 3.12 - -Backport typing.override via typing_extensions, lower requires-python to ->=3.11, add 3.11/3.12 classifiers and ruff target." - ``` - ---- - -### Task 2: Widen the CI matrix and finalize the bundle - -Add 3.11 and 3.12 to the pytest matrix so the 100%-coverage suite runs on every -supported interpreter, and close out the planning bundle. No architecture -promotion is required — the design verified no `architecture/.md` -references the Python floor. - -**Files:** -- Modify: `.github/workflows/_checks.yml` (pytest matrix) -- Modify: `planning/changes/2026-06-30.01-python-3.11-3.12-support/design.md` - (finalize `summary:`) - -**Interfaces:** -- Consumes: the lowered floor from Task 1 (CI installs each matrix interpreter - via `uv python install ${{ matrix.python-version }}` and runs `pytest`). -- Produces: nothing later tasks rely on. - -- [ ] **Step 1: Confirm no architecture page references the floor (guards the "no promotion" claim)** - - ```bash - grep -rn "3\.13\|requires-python\|Python 3" architecture/ || echo "no floor references — no promotion needed" - ``` - - Expected: no matches (or only matches unrelated to the supported-version - floor). If a real floor reference appears, update that page in this PR and - note it here. - -- [ ] **Step 2: Edit `.github/workflows/_checks.yml` — widen the pytest matrix** - - The matrix block (lines 24-26) is currently: - - ```yaml - matrix: - python-version: - - "3.13" - - "3.14" - ``` - - Change it to: - - ```yaml - matrix: - python-version: - - "3.11" - - "3.12" - - "3.13" - - "3.14" - ``` - - Leave the lint job's `uv python install 3.13` (line 16) unchanged — lint stays - on 3.13. - -- [ ] **Step 3: Finalize the bundle summary** - - In `planning/changes/2026-06-30.01-python-3.11-3.12-support/design.md`, - confirm the front-matter `summary:` states the realized result (it already - reads as the shipped outcome — adjust only if the implementation deviated from - the spec). - -- [ ] **Step 4: Validate the planning bundle** - - ```bash - just check-planning - ``` - - Expected: `planning: OK`. - -- [ ] **Step 5: Commit** - - ```bash - git add .github/workflows/_checks.yml \ - planning/changes/2026-06-30.01-python-3.11-3.12-support/design.md - git commit -m "ci: run pytest matrix on Python 3.11 and 3.12" - ``` - ---- - -### Task 3: Open the PR and watch CI - -Ship via PR (never local-merge). The widened matrix on real CI is the -authoritative verification — local runs only smoke-tested 3.11. - -**Files:** none. - -- [ ] **Step 1: Push the branch and open the PR** - - ```bash - git push -u origin feat/python-3.11-3.12-support - gh pr create --fill --base main - ``` - -- [ ] **Step 2: Watch CI to green** - - ```bash - gh pr checks --watch - ``` - - Expected: all matrix legs (3.11, 3.12, 3.13, 3.14) plus lint pass. The 100% - coverage gate holds on each interpreter. If a 3.11/3.12-only failure surfaces - (e.g. a dependency runtime difference), fix it on the branch and re-push — - catching exactly that is why the matrix was widened. diff --git a/planning/changes/2026-07-05.01-docs-ci-drift-guard/design.md b/planning/changes/2026-07-05.01-docs-ci-drift-guard.md similarity index 100% rename from planning/changes/2026-07-05.01-docs-ci-drift-guard/design.md rename to planning/changes/2026-07-05.01-docs-ci-drift-guard.md diff --git a/planning/changes/2026-07-05.01-docs-ci-drift-guard/plan.md b/planning/changes/2026-07-05.01-docs-ci-drift-guard/plan.md deleted file mode 100644 index 5f58c8a..0000000 --- a/planning/changes/2026-07-05.01-docs-ci-drift-guard/plan.md +++ /dev/null @@ -1,194 +0,0 @@ -# docs-ci-drift-guard — implementation plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use -> superpowers:subagent-driven-development (recommended) or -> superpowers:executing-plans to implement this plan task-by-task. Steps -> use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Make `mkdocs build --strict` a PR gate with native link/anchor -validation, pin the docs toolchain, fix the `make_outbox_table` 63-byte -docstring, and one-time-audit `README.md`. - -**Spec:** [`design.md`](./design.md) - -**Branch:** `chore/docs-ci-drift-guard` - -**Commit strategy:** Per-task commits; all ship in one PR alongside this bundle. - ---- - -### Task 1: Enable mkdocs link/anchor validation - -**Files:** -- Modify: `mkdocs.yml` - -Add the `validation:` block so `--strict` fails on broken internal links and -`#anchor` fragments. - -- [ ] **Step 1: Add the validation block** - - Insert a top-level `validation:` key (mkdocs 1.6+): - ```yaml - validation: - omitted_files: warn - absolute_links: warn - unrecognized_links: warn - anchors: warn - ``` - -- [ ] **Step 2: Prove it gates (negative test)** - - Temporarily append a broken anchor link (e.g. `[x](usage/basic.md#no-such-anchor)`) - to a docs page, run `just docs-build`, confirm it **fails** with an anchor - warning-as-error, then remove the temporary link. - -- [ ] **Step 3: Confirm clean tree passes** - - `just docs-build` → "Documentation built"; no warnings-as-errors. - -- [ ] **Step 4: Commit** - - ```bash - git add mkdocs.yml - git commit -m "docs: validate internal links and anchors under mkdocs --strict" - ``` - ---- - -### Task 2: Pin the docs toolchain - -**Files:** -- Modify: `docs/requirements.txt` - -Upper-bound the docs deps so the (now gating) strict build is reproducible. - -- [ ] **Step 1: Resolve current mkdocs-material series** - - `uvx --with-requirements docs/requirements.txt mkdocs --version` and note the - installed `mkdocs-material` major (for the lower bound). - -- [ ] **Step 2: Pin with upper bounds** - - ``` - mkdocs>=1.6,<2 - mkdocs-material>=,< - ``` - -- [ ] **Step 3: Verify build still passes on the pins** - - `just docs-build` → passes. - -- [ ] **Step 4: Commit** - - ```bash - git add docs/requirements.txt - git commit -m "docs: pin mkdocs and mkdocs-material with upper bounds" - ``` - ---- - -### Task 3: Add the strict docs build as a parallel PR job - -**Files:** -- Modify: `.github/workflows/_checks.yml` - -Add a `docs` job running `just docs-build`, parallel with `lint`/`pytest`. - -- [ ] **Step 1: Add the job** - - Mirror the `lint` job's setup steps; final step `- run: just docs-build`. - -- [ ] **Step 2: Sanity-check YAML** - - `uv run python -c "import yaml,sys; yaml.safe_load(open('.github/workflows/_checks.yml'))"` - (or `just` if a lint target covers workflows) → no parse error. - -- [ ] **Step 3: Commit** - - ```bash - git add .github/workflows/_checks.yml - git commit -m "ci: run mkdocs --strict as a PR check" - ``` - ---- - -### Task 4: Fix the `make_outbox_table` docstring (#2) - -**Files:** -- Modify: `faststream_outbox/schema.py` - -Correct the 63-byte-guard wording: the binding identifier is the longest -derived index/constraint name, not the NOTIFY channel. - -- [ ] **Step 1: Edit the docstring** - - Reword the `Raises ValueError ...` line in `make_outbox_table` to say the - guard trips when the **longest derived identifier** (e.g. `
_pending_idx`) - exceeds 63 bytes — longer than the `outbox_
` channel. - -- [ ] **Step 2: Lint stays green** - - `just lint-ci` → ruff + ty pass. - -- [ ] **Step 3: Commit** - - ```bash - git add faststream_outbox/schema.py - git commit -m "docs: correct make_outbox_table 63-byte guard docstring" - ``` - ---- - -### Task 5: One-time README audit (#3) - -**Files:** -- Modify: `README.md` (as findings require) - -Audit `README.md` against source + `docs/`; fix confident drift, surface -judgment calls. - -- [ ] **Step 1: Verify code samples** - - For every code block in `README.md`, confirm imports/symbols/kwargs resolve - against `faststream_outbox/` (grep `__init__.py` exports; check signatures). - -- [ ] **Step 2: Verify claims, numbers, links** - - Check stated defaults/numbers against source; check every link (badges, - docs-site links, GitHub URLs) resolves. - -- [ ] **Step 3: Apply fixes; surface judgment calls** - - Fix confident drift in place. Any ambiguous/design-choice item → raise to - maintainer, do not decide unilaterally. - -- [ ] **Step 4: Re-verify** - - Re-run the import/symbol/link checks on the edited README. - -- [ ] **Step 5: Commit** - - ```bash - git add README.md - git commit -m "docs: fix drift in README" - ``` - ---- - -### Task 6: Finalize the bundle and open the PR - -**Files:** -- Modify: `planning/changes/2026-07-05.01-docs-ci-drift-guard/design.md` (finalize `summary:`) - -- [ ] **Step 1: Validate planning** - - `just check-planning` → passes (bundle + decision frontmatter, lane, spec link). - -- [ ] **Step 2: Full local gate** - - `just lint-ci` and `just docs-build` both green. - -- [ ] **Step 3: Push + open PR; watch CI** - - Push the branch, open the PR, and watch lint + the new `docs` job + the pytest - matrix to green. diff --git a/planning/deferred.md b/planning/deferred.md index b6a171c..dbbb5bf 100644 --- a/planning/deferred.md +++ b/planning/deferred.md @@ -4,7 +4,7 @@ Items raised in reviews or audits that are real but not actionable now. Each is parked here with the reason it's deferred and the concrete trigger that should bring it back. This is the long-tail register — not a backlog of planned work. When an item is picked up it graduates to a spec/plan -bundle in [`changes/`](changes/); see [CLAUDE.md](../CLAUDE.md#workflow). +change file in [`changes/`](changes/); see [CLAUDE.md](../CLAUDE.md#workflow). As of the 2026-06-12 code + docs audit closure (PRs #61–#74), the audit backlog is empty. The items below are the remainder: technically real, diff --git a/planning/index.py b/planning/index.py index a1632e1..2d70ac3 100644 --- a/planning/index.py +++ b/planning/index.py @@ -1,14 +1,12 @@ -# ruff: noqa: INP001, D212 # planning/ is not a Python package; D212/D213 conflict differs from faststream-outbox -""" -Generate the planning index from frontmatter. +# ruff: noqa: INP001 # planning/ is not a Python package (this file is vendored into consumers' planning/) +"""Generate the planning index from frontmatter. -Run via ``just index``. Globs ``planning/changes/*/`` (each bundle's -``design.md``, falling back to ``change.md``) and ``planning/decisions/*.md``, -reads their frontmatter, and prints a Markdown listing to stdout — changes -then decisions, newest-first. Never writes a file: +Run via ``just index``. Globs ``planning/changes/*.md`` and +``planning/decisions/*.md``, reads their frontmatter, and prints a Markdown +listing to stdout — changes then decisions, newest-first. Never writes a file: the listing is a query over the files, not a committed artifact. -``date`` and ``slug`` are derived from the directory / file name, not +``date`` and ``slug`` are derived from the file name, not frontmatter — the name is the single source of truth for both. """ @@ -17,12 +15,10 @@ import sys -CHANGES_DIR = pathlib.Path(__file__).parent / "changes" -DECISIONS_DIR = pathlib.Path(__file__).parent / "decisions" +ROOT = pathlib.Path(__file__).parent VALID_DECISION_STATUS = {"accepted", "superseded"} -BUNDLE_RE = re.compile(r"^(?P\d{4}-\d{2}-\d{2})\.\d{2}-(?P.+)$") +CHANGE_RE = re.compile(r"^(?P\d{4}-\d{2}-\d{2})\.\d{2}-(?P.+)$") DECISION_RE = re.compile(r"^(?P\d{4}-\d{2}-\d{2})-(?P.+)$") -ALLOWED_BUNDLE_FILES = {"design.md", "plan.md", "change.md"} SPEC_REQUIRED = ("summary",) DECISION_REQUIRED = ("status", "summary") @@ -47,7 +43,7 @@ def parse_frontmatter(text: str) -> dict[str, str]: def _named(fields: dict[str, str], name: str, pattern: re.Pattern[str]) -> dict[str, str]: - """Inject ``date``/``slug`` derived from a dir/file name into ``fields``.""" + """Inject ``date``/``slug`` derived from a file name into ``fields``.""" match = pattern.match(name) if match: fields["date"] = match.group("date") @@ -55,32 +51,29 @@ def _named(fields: dict[str, str], name: str, pattern: re.Pattern[str]) -> dict[ return fields -def load_bundles() -> list[dict[str, str]]: - """Read each bundle's summary; derive date/slug from the directory name.""" - bundles: list[dict[str, str]] = [] - if not CHANGES_DIR.is_dir(): - return bundles - for bundle in sorted(CHANGES_DIR.iterdir()): - if not bundle.is_dir(): - continue - spec = bundle / "design.md" - if not spec.exists(): - spec = bundle / "change.md" - if not spec.exists(): +def load_changes(root: pathlib.Path) -> list[dict[str, str]]: + """Read each change file's summary; derive date/slug from the file name.""" + changes_dir = root / "changes" + changes: list[dict[str, str]] = [] + if not changes_dir.is_dir(): + return changes + for path in sorted(changes_dir.glob("*.md")): + if path.name == "README.md" or path.name.startswith(("_", ".")): continue - fields = _named(parse_frontmatter(spec.read_text(encoding="utf-8")), bundle.name, BUNDLE_RE) - fields["path"] = f"changes/{bundle.name}/{spec.name}" - fields["name"] = bundle.name - bundles.append(fields) - return bundles + fields = _named(parse_frontmatter(path.read_text(encoding="utf-8")), path.stem, CHANGE_RE) + fields["path"] = f"changes/{path.name}" + fields["name"] = path.stem + changes.append(fields) + return changes -def load_decisions() -> list[dict[str, str]]: +def load_decisions(root: pathlib.Path) -> list[dict[str, str]]: """Read each decision's frontmatter; derive date/slug from the file name.""" + decisions_dir = root / "decisions" decisions: list[dict[str, str]] = [] - if not DECISIONS_DIR.is_dir(): + if not decisions_dir.is_dir(): return decisions - for path in sorted(DECISIONS_DIR.glob("*.md")): + for path in sorted(decisions_dir.glob("*.md")): if path.name == "README.md" or path.name.startswith("_"): continue fields = _named(parse_frontmatter(path.read_text(encoding="utf-8")), path.stem, DECISION_RE) @@ -90,24 +83,24 @@ def load_decisions() -> list[dict[str, str]]: return decisions -def format_row(bundle: dict[str, str]) -> str: - """Render one bundle as a Markdown list item.""" - slug = bundle.get("slug", "?") - path = bundle.get("path", "") - date = bundle.get("date", "") - summary = bundle.get("summary") or "(no summary)" +def format_row(row: dict[str, str]) -> str: + """Render one change or decision as a Markdown list item.""" + slug = row.get("slug", "?") + path = row.get("path", "") + date = row.get("date", "") + summary = row.get("summary") or "(no summary)" line = f"- **[{slug}]({path})** ({date}) — {summary}" - if bundle.get("supersedes"): - line += f" _(supersedes {bundle['supersedes']})_" - if bundle.get("superseded_by"): - line += f" _(superseded by {bundle['superseded_by']})_" + if row.get("supersedes"): + line += f" _(supersedes {row['supersedes']})_" + if row.get("superseded_by"): + line += f" _(superseded by {row['superseded_by']})_" return line -def render(bundles: list[dict[str, str]], decisions: list[dict[str, str]]) -> str: +def render(changes: list[dict[str, str]], decisions: list[dict[str, str]]) -> str: """Render the full Markdown listing: changes then decisions, newest-first.""" out = ["# Planning index", "", "_Generated by `just index` — do not edit._", "", "## Changes", ""] - change_rows = sorted(bundles, key=lambda b: b.get("name", ""), reverse=True) + change_rows = sorted(changes, key=lambda b: b.get("name", ""), reverse=True) out += [format_row(b) for b in change_rows] if change_rows else ["_None._"] out += ["", "## Decisions", ""] decision_rows = sorted(decisions, key=lambda d: d.get("name", ""), reverse=True) @@ -121,32 +114,15 @@ def _require(fields: dict[str, str], keys: tuple[str, ...], rel: str, violations violations.extend(f"{rel}: missing or empty frontmatter key '{key}'" for key in keys if not fields.get(key)) -def _check_spec_file(path: pathlib.Path, rel: str, violations: list[str]) -> None: - """Validate a design.md / change.md spec file (requires `summary`).""" +def _check_change(path: pathlib.Path, violations: list[str]) -> None: + """Validate one change file (requires `summary`).""" + rel = f"changes/{path.name}" + if CHANGE_RE.match(path.stem) is None: + violations.append(f"{rel}: file name is not 'YYYY-MM-DD.NN-slug.md'") fields = parse_frontmatter(path.read_text(encoding="utf-8")) _require(fields, SPEC_REQUIRED, rel, violations) -def _check_bundle(bundle: pathlib.Path, violations: list[str]) -> None: - """Validate one change bundle directory.""" - rel = f"changes/{bundle.name}" - if BUNDLE_RE.match(bundle.name) is None: - violations.append(f"{rel}: directory name is not 'YYYY-MM-DD.NN-slug'") - violations.extend( - f"{rel}/{child.name}: unexpected file in bundle (allowed: {', '.join(sorted(ALLOWED_BUNDLE_FILES))})" - for child in sorted(bundle.iterdir()) - if child.name not in ALLOWED_BUNDLE_FILES - ) - design = bundle / "design.md" - change = bundle / "change.md" - if not design.exists() and not change.exists(): - violations.append(f"{rel}: bundle has neither design.md nor change.md") - for spec_file in (design, change): - if spec_file.exists(): - _check_spec_file(spec_file, f"{rel}/{spec_file.name}", violations) - # plan.md carries no frontmatter — its identity comes from the bundle dir. - - def _check_decision(path: pathlib.Path, violations: list[str]) -> None: """Validate one decision file (requires `status` + `summary`).""" rel = f"decisions/{path.name}" @@ -159,25 +135,39 @@ def _check_decision(path: pathlib.Path, violations: list[str]) -> None: violations.append(f"{rel}: invalid status '{status}' (allowed: {', '.join(sorted(VALID_DECISION_STATUS))})") -def check() -> list[str]: - """Validate every bundle and decision; return the list of violation strings.""" +def check(root: pathlib.Path) -> list[str]: + """Validate every change file and decision; return the list of violation strings.""" violations: list[str] = [] - if CHANGES_DIR.is_dir(): - for bundle in sorted(CHANGES_DIR.iterdir()): - if bundle.is_dir(): - _check_bundle(bundle, violations) - if DECISIONS_DIR.is_dir(): - for path in sorted(DECISIONS_DIR.glob("*.md")): + changes_dir = root / "changes" + decisions_dir = root / "decisions" + if changes_dir.is_dir(): + for path in sorted(changes_dir.iterdir()): + if path.is_dir(): + violations.append( + f"changes/{path.name}: directory found — convention 2.0.0 uses flat change files " + f"(changes/YYYY-MM-DD.NN-slug.md; see CHANGELOG 2.0.0 for the migration)" + ) + continue + if path.name == "README.md" or path.name.startswith(("_", ".")): + continue + if path.suffix != ".md": + violations.append(f"changes/{path.name}: unexpected non-md file in changes/") + else: + _check_change(path, violations) + if decisions_dir.is_dir(): + for path in sorted(decisions_dir.glob("*.md")): if path.name == "README.md" or path.name.startswith("_"): continue _check_decision(path, violations) return violations -def main() -> int: - """Print the listing to stdout, or validate bundles with --check.""" - if "--check" in sys.argv[1:]: - violations = check() +def main(argv: list[str] | None = None, root: pathlib.Path | None = None) -> int: + """Print the listing to stdout, or validate change files and decisions with --check.""" + argv = sys.argv[1:] if argv is None else argv + root = ROOT if root is None else root + if "--check" in argv: + violations = check(root) if violations: sys.stderr.write(f"planning: {len(violations)} violation(s)\n") for violation in violations: @@ -185,7 +175,7 @@ def main() -> int: return 1 sys.stdout.write("planning: OK\n") return 0 - sys.stdout.write(render(load_bundles(), load_decisions())) + sys.stdout.write(render(load_changes(root), load_decisions(root))) return 0