#153 chore: catch up co-core 0.37.0 -> 0.39.1

.socraticodecontextartifacts.json

@@ -153,7 +153,7 @@
     {
       "name": "dependency-contract",
       "path": "./pyproject.toml",
-      "description": "Project and tooling contract. The co-core / co-core-aio / co-core-sync pins (>=0.37.0,<0.38 since #136, for FetchPolicyState.generation and fetch_policy_supersedes; 0.36 was #140's no-feature catch-up; each raise's wheel diff covers the whole import closure; 0.19.6 brought the content.persist contract, cannobserv#493, and 0.19.3 the shared content-addressed BlobStore, cannobserv#475) with no extra on co-core (its [extract] parsers were unused, dropped by #142) and [bus,gcs] / [gcs] on the other two, the floor history that explains each raise, the [tool.uv] find-links wheelhouse resolution, the ruff rule set and its skills-vendor exclusion, the ty exclusion, the replicator-worker console script, pytest configuration (asyncio auto mode, the integration and gcs markers both deselected by default, timeout) and the 80% coverage gate."
+      "description": "Project and tooling contract. The co-core / co-core-aio / co-core-sync pins (>=0.39.0,<0.40 since #153, a no-feature catch-up ahead of #138's capture consumer; 0.37 was #136's, for FetchPolicyState.generation and fetch_policy_supersedes; 0.36 was #140's catch-up; each raise's wheel diff covers the whole import closure; 0.19.6 brought the content.persist contract, cannobserv#493, and 0.19.3 the shared content-addressed BlobStore, cannobserv#475) with no extra on co-core (its [extract] parsers were unused, dropped by #142) and [bus,gcs] / [gcs] on the other two, the floor history that explains each raise, the [tool.uv] find-links wheelhouse resolution, the ruff rule set and its skills-vendor exclusion, the ty exclusion, the replicator-worker console script, pytest configuration (asyncio auto mode, the integration and gcs markers both deselected by default, timeout) and the 80% coverage gate."
     },
     {
       "name": "main-checkout-guard",

AGENTS.md

@@ -34,7 +34,7 @@ Python ≥3.12, uv, pytest, ruff. `ty` (`uv run ty check`) is a **non-gating** t
 uv run --no-project --with 'google-cloud-storage>=2,<4' python scripts/sync_wheelhouse.py
 ```
 
-Auth is ADC. Pin the current minor — `>=0.37.0,<0.38` — and raise the **patch** floor with every co-core feature the code starts depending on; the ways a skew has already failed are in [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md).
+Auth is ADC. Pin the current minor — `>=0.39.0,<0.40` — and raise the **patch** floor with every co-core feature the code starts depending on; the ways a skew has already failed are in [docs/DEPLOYMENT.md](docs/DEPLOYMENT.md).
 
 <!-- BEGIN socraticode-policy -->
 ## Code Exploration Policy

docs/DEPLOYMENT.md

@@ -72,11 +72,11 @@ declined: [MEMORY-PROTECTION.md](MEMORY-PROTECTION.md).
 mirrored into `./.wheelhouse` by `scripts/sync_wheelhouse.py` and resolved
 through `[tool.uv] find-links` — never from PyPI.
 
-Auth is ADC, with one substitution: on the VM the sync uses the read-only key named by `REPLICATOR_WHEELHOUSE_CREDENTIALS` (`/etc/replicator/co-pypi-reader.json`) rather than the worker's own `GOOGLE_APPLICATION_CREDENTIALS`, which is the writer (`/etc/replicator/co-gcs-replicator-writer.json`, #29, #114); in CI it is a keyless WIF token. Pin the current minor — `>=0.37.0,<0.38`, on all three. The **patch** floor is load-bearing, not tidiness: the change-bus payloads are `extra="ignore"`, so on an older wheel a model constructed with fields it does not have yet succeeds and silently discards them. Raise the floor with every co-core feature the code starts depending on, or a version skew publishes facts that look right and carry nothing (#10). Both floors since fail *loudly* instead — a ValidationError at construction (0.8.0 requires `info_source_id` on all three fetch payloads, #19/#28) or an ImportError at load, never reaching a running worker (0.9.4 cuts the replicate contracts, #29; 0.19.3 ships the shared `BlobStore`, #114; 0.19.6 cuts the persist contract, #114 step 6).
+Auth is ADC, with one substitution: on the VM the sync uses the read-only key named by `REPLICATOR_WHEELHOUSE_CREDENTIALS` (`/etc/replicator/co-pypi-reader.json`) rather than the worker's own `GOOGLE_APPLICATION_CREDENTIALS`, which is the writer (`/etc/replicator/co-gcs-replicator-writer.json`, #29, #114); in CI it is a keyless WIF token. Pin the current minor — `>=0.39.0,<0.40`, on all three. The **patch** floor is load-bearing, not tidiness: the change-bus payloads are `extra="ignore"`, so on an older wheel a model constructed with fields it does not have yet succeeds and silently discards them. Raise the floor with every co-core feature the code starts depending on, or a version skew publishes facts that look right and carry nothing (#10). Both floors since fail *loudly* instead — a ValidationError at construction (0.8.0 requires `info_source_id` on all three fetch payloads, #19/#28) or an ImportError at load, never reaching a running worker (0.9.4 cuts the replicate contracts, #29; 0.19.3 ships the shared `BlobStore`, #114; 0.19.6 cuts the persist contract, #114 step 6).
 
 **The 0.10 → 0.13 jump (#77) was the first departure from "pin the current minor", and what justified it is evidence rather than judgement.** The hazard that rule guards is payload fields moving under `extra="ignore"`; every module this service imports proved byte-identical between the two wheels — `pure/models/changes.py`, `pure/adapters/bus/{envelope,exceptions}.py`, `effects/{bus,fetch,gcs}.py`, `pure/util/{hashing,gcs}.py`, and all of `co_core_aio` bar its `__version__`. Only `streams.py` gained anything (`group_name`, `stream_kind` — cannobserv#384), and Replicator imports no `co_v1` adapter, which is where 0.13.2's other changes landed. That `diff -rq` of both wheels from `.wheelhouse/` took a minute and turned a deferred change into a safe one.
 
-**Diff the import closure before every pin move**, a single minor included: #136 needed it for one. The version distance is not the risk, the moved surface is, and the import lines are not the surface. At 0.19 → 0.36 (#140), `changes.py`'s import of `extract.canonical` brought in the `pure/extract` package, which had changed (cannobserv#523), and no import line in Replicator names it. `scripts/co_core_closure_diff.py OLD NEW` (#145) walks the closure from `src/` and `scripts/` through all three wheels at each version, after the wheel sync. It prints the counts, a verdict per file that isn't identical, the Requires-Dist delta and the closure's third-party imports, and `--diff` adds each changed file's code diff. Paste the report into the pin comment and write the reasons around it by hand. `version-only` and `docstrings-only` need no reading. `changed` does, a model under `extra="ignore"` above all. `added` and `removed` are modules joining or leaving the closure. An **Unresolved imports** line names a module that is imported but isn't in the wheel, which fails at import. A **Not parsed** line names a file whose imports the walk couldn't follow. Each prints only when there's something to list. The walk follows `TYPE_CHECKING` imports and lists dynamic ones without following them. At 0.37.0 that's co-core-sync's two lazy `__getattr__` sites in `drivers/` and `drivers/blobstore/`. Both are expected, because each target is named again under `TYPE_CHECKING`. A new site whose targets aren't named there is a gap to read by hand. Flags: `--root`, `--package`, `--wheelhouse`; the defaults are this repo's.
+**Diff the import closure before every pin move**, a single minor included: #136 needed it for one. The version distance is not the risk, the moved surface is, and the import lines are not the surface. At 0.19 → 0.36 (#140), `changes.py`'s import of `extract.canonical` brought in the `pure/extract` package, which had changed (cannobserv#523), and no import line in Replicator names it. `scripts/co_core_closure_diff.py OLD NEW` (#145) walks the closure from `src/` and `scripts/` through all three wheels at each version, after the wheel sync. It prints the counts, a verdict per file that isn't identical, the Requires-Dist delta and the closure's third-party imports, and `--diff` adds each changed file's code diff. Paste the report into the pin comment and write the reasons around it by hand. `version-only` and `docstrings-only` need no reading. `changed` does, a model under `extra="ignore"` above all. `added` and `removed` are modules joining or leaving the closure. An **Unresolved imports** line names a module that is imported but isn't in the wheel, which fails at import. A **Not parsed** line names a file whose imports the walk couldn't follow. Each prints only when there's something to list. The walk follows `TYPE_CHECKING` imports and lists dynamic ones without following them. At 0.39.1 that's co-core-sync's two lazy `__getattr__` sites in `drivers/` and `drivers/blobstore/`. Both are expected, because each target is named again under `TYPE_CHECKING`. A new site whose targets aren't named there is a gap to read by hand. Flags: `--root`, `--package`, `--wheelhouse`; the defaults are this repo's.
 
 **Sync the wheelhouse before concluding a version is unavailable.** The #77 review reported 0.13.1 "not mirrored" from a local `ls` while the private index already carried it; the mirror had simply not been synced, and what silently fixed it was `replicator.service`'s own non-fatal `ExecStartPre` running `sync_wheelhouse.py` on the next restart. A local listing is a cache, not the index.
 

pyproject.toml

@@ -7,7 +7,82 @@ license = "MIT"
 license-files = ["LICENSE"]
 requires-python = ">=3.12"
 dependencies = [
-    # Floor is 0.37.0, for `FetchPolicyState.generation` and
+    # Floor is 0.39.0, and it carries no new feature: **two minors (0.37 -> 0.39)**
+    # taken as a catch-up (#153), ahead of the capture consumer (#138), so that
+    # this diff does not land with a behaviour change. #138's types
+    # (`ContentCaptureCommand`, `CaptureStoredEmit` / `CaptureFailedEmit`, the
+    # `CONTENT_CAPTURE` constant) arrived in 0.37.1 (cannobserv#571), so every
+    # 0.39 release has them. The floor is 0.39.0, not 0.39.1: 0.39.1's diff over
+    # 0.39.0 is `__version__` in the three `__init__`s and nothing else, and the
+    # lock resolves 0.39.1. Both `co-core-aio` and `co-core-sync` 0.39.0 and
+    # 0.39.1 require `co-core>=0.39,<0.40`, so the three move together.
+    #
+    # **`scripts/co_core_closure_diff.py 0.37.0 0.39.1`** (#145), verbatim:
+    #
+    # ## Import closure, 0.37.0 → 0.39.1
+    #
+    # Roots `src`, `scripts`; packages `co_core`, `co_core_aio`, `co_core_sync`.
+    #
+    # 57 files at 0.37.0, 58 at 0.39.1. 14 not identical:
+    #
+    # - `version-only` co_core/__init__.py
+    # - `changed` co_core/pure/adapters/bus/envelope.py
+    # - `changed` co_core/pure/adapters/bus/streams.py
+    # - `changed` co_core/pure/extract/__init__.py
+    # - `added` co_core/pure/extract/_canonical_json.py
+    # - `docstrings-only` co_core/pure/extract/base.py
+    # - `changed` co_core/pure/extract/config.py
+    # - `changed` co_core/pure/extract/errors.py
+    # - `changed` co_core/pure/extract/extraction_defaults.py
+    # - `changed` co_core/pure/extract/media_type.py
+    # - `changed` co_core/pure/extract/spec_fingerprint.py
+    # - `changed` co_core/pure/models/changes.py
+    # - `version-only` co_core_aio/__init__.py
+    # - `version-only` co_core_sync/__init__.py
+    #
+    # **Requires-Dist:**
+    #
+    # - co_core: added `python-jsonpath<3,>=2.2; extra == 'extract'`, `soupsieve<3,>=2.0; extra == 'extract'`
+    # - co_core_aio: removed `co-core<0.38,>=0.37`; added `co-core<0.40,>=0.39`
+    # - co_core_sync: removed `co-core<0.38,>=0.37`; added `co-core<0.40,>=0.39`
+    #
+    # **Third-party imports** (at 0.39.1): `google`, `google_auth_oauthlib`, `googleapiclient`, `httpx`, `pydantic`, `redis`, `requests`, `urllib3`.
+    #
+    # **Dynamic imports, not followed:**
+    #
+    # - co_core_sync/drivers/__init__.py:89 `import_module(f"{__name__}.{module}")`
+    # - co_core_sync/drivers/blobstore/__init__.py:32 `import_module(f"{__name__}.{module}")`
+    #
+    # Read by hand, the `changed` files:
+    #
+    # - `pure/models/changes.py`: the capture models and their Emit twins, added.
+    #   Every class that exists at 0.37.0 is AST-identical at 0.39.1 bar
+    #   docstrings, so no field, default or config moved under `extra="ignore"`;
+    # - `pure/adapters/bus/envelope.py`: `from_wire`'s global table gains
+    #   `content_capture`, `capture_stored` and `capture_failed`. On 0.37.0 one of
+    #   those on a stream read here raised `BusMessageUnknownEventTypeError`; now
+    #   it decodes and falls to the `isinstance` guards. A command stream still
+    #   dead-letters it, under "payload is not a ... command" instead of a poison
+    #   frame; `content.fetch-policy` skips it with a wrong-type warning instead
+    #   of a poison skip.
+    #   `idempotency_key` adds `occurred_at` to `InfoItemPrimaryChangedEvent`'s key,
+    #   a type Replicator never publishes;
+    # - `pure/adapters/bus/streams.py`: `CONTENT_CAPTURE`, a command stream;
+    # - `pure/extract/*`: 0.38.0's JSON extractor and `ignore_selectors`, and
+    #   `spec_fingerprint` routed through the added `_canonical_json.py`. They load
+    #   only because `changes.py` imports `extract.canonical`; nothing here imports
+    #   the package or calls `spec_fingerprint`, `extraction_config_from_spec` or
+    #   `refuse_unhonoured`.
+    #
+    # The new Requires-Dist is gated on `extra == 'extract'`, which isn't
+    # installed. 0.39.0's co/v1 lookup refusals and 0.39.1's CRM-model
+    # datetimes fall outside the closure. All of `co_core_aio`, `bus` and `fetch`
+    # with it, is identical bar `__version__`. The two dynamic imports are the
+    # same two as at 0.37.0, each target named again under `TYPE_CHECKING`.
+    #
+    # Previously:
+    #
+    # Floor was 0.37.0, for `FetchPolicyState.generation` and
     # `fetch_policy_supersedes` (cannobserv#566, #568; #136). `FetchPolicyMap`
     # applies every policy frame through the helper, so on 0.36.0 the worker fails
     # at import. The field alone would fail silently instead: under
@@ -186,9 +261,9 @@ dependencies = [
     # at module load, so a skew cannot reach a running worker at all. Additive over
     # 0.8.0 for everything the fetch path uses — no field on ContentFetchCommand,
     # BlobAvailableEvent or FetchFailedEvent moved between the two.
-    "co-core>=0.37.0,<0.38",
-    "co-core-aio[bus,gcs]>=0.37.0,<0.38",
-    "co-core-sync[gcs]>=0.37.0,<0.38",
+    "co-core>=0.39.0,<0.40",
+    "co-core-aio[bus,gcs]>=0.39.0,<0.40",
+    "co-core-sync[gcs]>=0.39.0,<0.40",
     "fastapi>=0.126.0,<1",
     "pydantic>=2.9,<3",
     "pydantic-settings>=2.5.0,<3",

tests/test_co_core_pin.py

@@ -0,0 +1,66 @@
+"""The co-core pin is one specifier, written the same way everywhere it is named (#153).
+
+The three packages move together: `co-core-aio` and `co-core-sync` each require
+`co-core` at their own minor, so a skewed specifier fails at resolve time at
+best. The prose sites are worse. AGENTS.md, docs/DEPLOYMENT.md and the
+SocratiCode manifest each name the current pin, nothing reads them, and a
+bump that forgets one stays green while an agent is told the wrong minor. The
+manifest is the costliest to leave stale: its description is what semantic
+search ranks first (#19).
+"""
+
+import json
+import re
+import tomllib
+from pathlib import Path
+
+REPO_ROOT = Path(__file__).resolve().parent.parent
+PACKAGES = ("co-core", "co-core-aio", "co-core-sync")
+_REQ = re.compile(r"^(?P<name>[A-Za-z0-9_.-]+)(?:\[[^\]]*\])?(?P<spec>.*)$")
+
+
+def co_core_specifiers() -> dict[str, str]:
+    """Map each co-core package to its specifier in pyproject.toml."""
+    deps = tomllib.loads((REPO_ROOT / "pyproject.toml").read_text())["project"]["dependencies"]
+    found = {}
+    for dep in deps:
+        m = _REQ.match(dep.replace(" ", ""))
+        if m and m["name"] in PACKAGES:
+            found[m["name"]] = m["spec"]
+    return found
+
+
+def the_pin() -> str:
+    specs = co_core_specifiers()
+    assert set(specs) == set(PACKAGES), f"pyproject.toml pins {sorted(specs)}"
+    return specs["co-core"]
+
+
+def test_the_three_packages_share_one_specifier():
+    specs = co_core_specifiers()
+    assert set(specs) == set(PACKAGES)
+    assert len(set(specs.values())) == 1, specs
+
+
+def test_the_pin_is_one_minor():
+    m = re.fullmatch(r">=0\.(\d+)\.\d+,<0\.(\d+)", the_pin())
+    assert m, the_pin()
+    assert int(m[2]) == int(m[1]) + 1, the_pin()
+
+
+def test_agents_md_names_the_pin():
+    assert f"Pin the current minor — `{the_pin()}`" in (REPO_ROOT / "AGENTS.md").read_text()
+
+
+def test_deployment_md_names_the_pin():
+    text = (REPO_ROOT / "docs" / "DEPLOYMENT.md").read_text()
+    assert f"Pin the current minor — `{the_pin()}`, on all three" in text
+
+
+def test_the_manifest_names_the_pin():
+    manifest = json.loads((REPO_ROOT / ".socraticodecontextartifacts.json").read_text())
+    descriptions = [
+        a["description"] for a in manifest["artifacts"] if "co-core-sync pins" in a["description"]
+    ]
+    assert len(descriptions) == 1, "one manifest artifact describes the co-core pins"
+    assert f"pins ({the_pin()} since" in descriptions[0]