Catch up co-core 0.19.6 → 0.36.0 ahead of the generation release #140

Activity

1. gregoryfoster commented on Oct 17, 2026

   

   gregoryfoster

   on Oct 17, 2026

   ContributorAuthor

   More actions

   Brief (coordinator, epic #135, 2026-10-17)

   First child of #135. Pre-flight run against origin/main at f638266.

   Authorities. This issue's body. The precedent for a multi-minor jump is docs/DEPLOYMENT.md, "The co-core pin, and why the patch floor is load-bearing", together with the 0.13 → 0.19 comment in pyproject.toml. If they disagree with this brief, stop and ask.

   Target. The newest co-core at ship time; 0.36.0 (2026-10-16) when this was written [measured, cannobserv CHANGELOG]. Unreleased work in that CHANGELOG is the CannObserv/cannobserv#568 release that #136 waits for. Don't wait for it.

   Blockers: none.

   Hazards, to weigh

   1. The wheelhouse is stale. .wheelhouse stops at 0.34.1 [measured]. Run scripts/sync_wheelhouse.py (the command is in AGENTS.md) before you conclude a version is missing. docs/DEPLOYMENT.md records the time a local ls was misread this way (Bus consumer name is hostname-derived, and the VM migration will orphan it (archiver#156 precedent) #77).

   2. Don't use the shared venv. The main checkout's .venv is the one replicator.service imports from. In the worktree, rm .venv (it's a symlink), then uv lock --offline && uv sync --frozen --offline (the Converge storage on the cluster content-addressed model #114 rule) [stated, host memory and AGENTS.md]. Run uv sync --frozen in main only as the deploy step, after the merge.

   3. The risk is the surface that moved, not the version distance [stated, DEPLOYMENT.md]. Unzip the 0.19.6 and target wheels and run diff -rq on every module Replicator imports [measured, by grep]:

      • pure/models/changes.py
      • pure/adapters/bus/{envelope,exceptions,dead_letter,streams}
      • effects/{bus,fetch,gcs}
      • pure/util/{blobstore,hashing,gcs}
      • co_core_aio.{bus,fetch,gcs}
      • co_core_sync.drivers.blobstore

      Watch hardest for a payload field that moved under extra="ignore": the failure is silent. Record what you find in the pin comment.

   4. Breaking entries since 0.19.6 [measured, CHANGELOG]. About a dozen, nearly all WordPress or co/v1, which Replicator doesn't import. The exception is CannObserv/cannobserv#549, which makes every datetime aware-only. src/ has no utcnow(), no bare datetime.now() and no naive literals [measured, git grep]. Confirm that in the tests, and confirm none of the fakes build naive datetimes.

   5. FetchPolicyState and the stream constants. src/worker/policy.py depends on revoked, min_interval_seconds, schema_version and occurred_at. CannObserv/cannobserv#563–#565 are unreleased, so 0.36.0 shouldn't move them. Check anyway. Apply FetchPolicyState.generation only if newer #136 builds on this file.

   6. Floors. Raise all three packages together. They share one version.

   7. AGENTS.md is at 5955 of 6000 tokens [measured, this week's curation]. The pin line already exists; change only its version numbers. replicator-boundaries.md: Archiver produces content.fetch-policy; link the cohort charter #137 adds the charter link after you, and that's the overlap that set the order.

   Expected: CI's gcs (T4 against the test bucket) job runs against the new co_core_sync GCS driver. Treat that job as evidence. Its failing isn't noise.

   Post-merge host steps (write them on the issue; the operator runs or approves them): git pull --ff-only && uv sync --frozen && sudo systemctl restart replicator in the main checkout. Then read the journal for a clean boot, the policy replay summary (tracked_hosts), and the consume loop picking up work.

   Out of scope:

   • generation and any change to the apply rule (Apply FetchPolicyState.generation only if newer #136).
   • The contract doc (replicator-boundaries.md: Archiver produces content.fetch-policy; link the cohort charter #137).
   • Any behaviour change. If the diff forces one, stop and report.

   Template: #77, the 0.10 → 0.13 jump, for the wheel diff and the pin comment.
2. gregoryfoster commented on Oct 17, 2026

   

   gregoryfoster

   on Oct 17, 2026

   ContributorAuthor

   More actions

   Brief review against f638266 — corrections before build

   Wheelhouse synced (24 new files, newest 0.36.0 on all three). Target stays 0.36.0, so the pin becomes >=0.36.0,<0.37. co_core_aio and co_core_sync 0.36.0 each require co-core>=0.36,<0.37, so the three floors can't drift apart anyway.

   1. The module list in hazard 3 misses the transitive closure. I walked the imports from every module Replicator imports, through to the end. That's 56 files at 0.19.6 and 57 at 0.36.0, and I diffed all of them, using an AST comparison with docstrings removed. The list in the brief leaves out one path that matters: changes.py imports co_core.pure.extract.canonical, and that runs the co_core.pure.extract package __init__. So extract/{config,extraction_defaults,media_type} and the new extract/errors.py load in the worker. All of it is #523 (XML extraction, UnsupportedExtractionError), and all of it is additive at import. Replicator calls none of it.

   Result, file by file:

   • Identical apart from docstrings (AST-equal): pure/models/changes.py, pure/adapters/bus/streams.py, pure/util/blobstore.py, pure/util/gcs.py. FetchPolicyState has the same fields, defaults and extra="ignore": schema_version, event_type, occurred_at, host, min_interval_seconds, revoked. No payload field moved.
   • Byte-identical: the rest of pure/adapters/bus/*, effects/{bus,fetch,gcs}, pure/util/hashing, and all of co_core_aio apart from __version__.
   • Code changed, additive only: co_core_sync/drivers/blobstore/gcs.py. It adds PointerEntry, list_pointer_entries and delete_pointer (#496), and POINTER_LISTING_FIELDS now asks for generation. Replicator calls none of the pointer verbs. create_if_absent, open, exists and delete are unchanged.
   • Requires-Dist / extras: unchanged on all three, apart from the self-pins.

   2. #549 doesn't reach Replicator's surface. The brief says it makes every datetime aware-only. In fact it covers the co/v1 domain models (EventModel, TimelineModel, MilestoneModel, SourceURLModel, ObservationModel, AddressModel) and the wp/v2 formatters. None of those is in the closure, and changes.py is AST-identical. I checked the tests anyway. src/ and tests/ have no naive datetime(20xx, …) literals, no utcnow, no bare .now(), no strptime and no tzinfo=None. Of the other Breaking entries, #523 (extraction_config_from_spec) is the only one that touches a loaded module, and Replicator doesn't call it.

   3. One more place carries the pin text. The pyproject.toml entry in .socraticodecontextartifacts.json spells out >=0.19.6,<0.20. Per AGENTS.md, a description edit ships with a codebase_update. That's in scope here, alongside AGENTS.md and docs/DEPLOYMENT.md. The other 0.19.6 mentions stay as they are: the persist contract's P5, and the test comments on the store's digest check. They date a feature, not the pin.

   Nothing in the diff forces a behaviour change. Building.
3. gregoryfoster commented on Oct 17, 2026

   

   gregoryfoster

   on Oct 17, 2026

   Last edited by gregoryfoster

   ContributorAuthor

   More actions

   Post-merge host steps (operator runs or approves)

   Run these on co-replicator, in the main checkout /home/exedev/replicator, after the branch lands on origin/main.

   The wheelhouse already holds 0.36.0. I synced it into the main checkout's .wheelhouse on 2026-10-17, and the unit's ExecStartPre re-syncs it anyway.

   1. Deploy

   cd /home/exedev/replicator
   git status -sb                      # must read "## main...origin/main" with nothing ahead
   git pull --ff-only                  # if the merge happened here, `git push` instead; the guard refuses unpushed commits
   uv sync --frozen                    # installs co-core / -aio / -sync 0.36.0 into the shared .venv
   sudo systemctl restart replicator

   2. Verify the install and the build

   .venv/bin/python -c 'import co_core, co_core_aio, co_core_sync as s; print(co_core.__version__, co_core_aio.__version__, s.__version__)'   # 0.36.0 x3
   cat /run/replicator/build-id        # = short SHA of the merged main
   systemctl is-active replicator      # active

   3. Read the journal for this boot

   journalctl -u replicator --since "-5min" -o cat --no-pager | grep -vE '^\s*$' | tail -40

   Expect these lines, in this order:

   • "replaying the fetch policy stream", then "fetch policy replay complete" with tracked_hosts (4 at the last boot, 2026-10-13), a messages count and hosts_stricter_than_default.
   • "worker ready", with build = the merged SHA, persist: "enabled", and the three groups replicator.fetch, replicator.replicate and replicator.persist.
   • No ImportError / AttributeError / ValidationError. No "poll cycle failed" beyond an isolated broker read timeout. No journalctl -t replicator-failure entry since the restart.

   The consume loop picks up work when the first fetch, replicate or persist outcome after worker ready lands on content.blobs or content.artifacts. If nothing is issued for a while, a quiet loop with no poll cycle failed is the clean state. Command and outcome volume are the broker operator's to read, because our ACL denies XINFO.

   4. Agent step, after the deploy: run codebase_update against /home/exedev/replicator. The PR changes three entries in .socraticodecontextartifacts.json: the pyproject.toml and docs/DEPLOYMENT.md descriptions, and a new docs/MEMORY-PROTECTION.md entry. SocratiCode indexes the main checkout's absolute path, so a worktree can't do this step.

   Rollback, if step 2 or 3 fails:

   git revert --no-edit <docs-sha> <chore-sha> && git push
   uv sync --frozen
   sudo systemctl restart replicator

   This lands back on 0.19.6, which is still in the wheelhouse.
4. gregoryfoster

   added 3 commits that reference this issue on Oct 17, 2026

   #140 docs: CR 1 — trim the closure-diff lesson to the rule and its case

   ...

   0a34455

   #140 docs: curate DEPLOYMENT.md under budget — demote Memory protection

   ...

   2b1fd5b

   #140 chore: record curation telemetry (AGENTS.md 5955 -> 5991, DEPLOY…

   ...

   3abaa81
5. gregoryfoster commented on Oct 17, 2026

   

   gregoryfoster

   on Oct 17, 2026

   ContributorAuthor

   More actions

   Build report: PR #141, ready to ship (CI green on 3abaa81)

   The pin change, CR, and a curating-context pass over AGENTS.md and DEPLOYMENT.md are all done. The PR body has every disposition.

   For #136:

   • FetchPolicyState is unchanged through 0.36.0, so generation will be its first field change since 0.19.
   • co-core-aio and co-core-sync 0.36.0 pin co-core>=0.36,<0.37 themselves, so all three must move together.
   • If cannobserv#568 ships as 0.37.0 rather than 0.36.x, the ceiling moves to <0.38 and the change is a minor bump, not a patch.
   • Move Catch up co-core 0.19.6 → 0.36.0 ahead of the generation release #140's top pin-comment block under "Previously:" and name generation as the floor's reason.
   • Run the wheel sync first.

   For #137: AGENTS.md is now 5991 of 6000 (exact), down to 9 tokens of headroom from 45. Indexing the new docs/MEMORY-PROTECTION.md cost 36 tokens. The curation found nothing else in AGENTS.md to cut honestly:

   • every section was classified a week ago (99d8a22);
   • the only candidate shaped like a duplicate, the Overview's ARCHITECTURE.md pointer, was a CR decision (a19d77e).

   A charter link probably won't fit without a trim. One option is to drop the MEMORY-PROTECTION.md index line and treat the doc as an annex that DEPLOYMENT.md and COMMANDS.md already reach. FAILURE-NOTIFICATION.md is indexed, though, so that would break precedent.

   For #142: the CI step co-core import smoke (extras wired) may assert [extract]. Check it when the extra goes.

   Skipped verification:

   • The deploy hasn't run. It's the operator's step (see the host-steps comment).
   • codebase_update hasn't run. It comes after the merge, from the main checkout.
   • The closure walk is static: dynamic imports in co-core aren't followed.
   • In the CHANGELOG I audited only the Breaking entries. The closure diff is the authority for everything else.
   • test_deploy.py:829 skips outside an exe.dev session, the same as the baseline.

   Accepted limits:

   • There's no failing-test-first commit (CR finding 2, stet): there's no behaviour to make fail.
   • The 0.36.0 floor carries no feature, so a skew within the minor wouldn't fail loudly. That doesn't matter while nothing here depends on a 0.36 feature.
   • DEPLOYMENT.md still carries 17 rot-prone counts. They predate this change and moved unchanged.
6. gregoryfoster commented on Oct 17, 2026

   

   gregoryfoster

   on Oct 17, 2026

   ContributorAuthor

   More actions

   Shipped to main: f638266..3abaa81 (PR #141). Not deployed yet.

   CI on main @ 3abaa81 is green: lint, test, and gcs (T4 against the test bucket).

   What changed:

   • 03ce721: co-core / -aio / -sync pinned at >=0.36.0,<0.37. The lock moves only those three. The pin comment records the diff of the whole import closure.
   • e121177, 0a34455: AGENTS.md, DEPLOYMENT.md and the manifest name the 0.36 pin. DEPLOYMENT.md records the import-closure lesson; CR 1 trimmed it.
   • 2b1fd5b, 3abaa81: the curating-context pass. DEPLOYMENT.md drops from 10889 to 7570 tokens, with Memory protection moved to docs/MEMORY-PROTECTION.md. AGENTS.md goes from 5955 to 5991. The manifest's false StartLimitBurst=6 is fixed. Telemetry is recorded.
   • CR 3 is filed as Drop the unused co-core [extract] extra #142, dropping the unused [extract] extra.

   Done-when:

   • Pins at the newest minor; uv.lock resolves from the wheelhouse.
   • Every imported co-core module diffed, the whole closure included; the pin comment records it.
   • Suite at baseline (1408 passed, 1 skipped), plus -m integration 55 passed and -m gcs 16 passed.
   • Pin text in AGENTS.md and docs/DEPLOYMENT.md.
   • Deployed. Waiting on the operator. Steps 1–3 of the host-steps comment above, starting from uv sync --frozen: the pull isn't needed, because main was merged in this checkout and pushed from it.

   Still open:

   • Step 4, codebase_update. It returned "Another process is already indexing this project, skipping": another session holds the index lock. docs/MEMORY-PROTECTION.md reads "not yet indexed" until a retry succeeds.

   Leaving this issue open until the deploy is verified in the journal.
7. gregoryfoster commented on Oct 17, 2026

   

   gregoryfoster

   on Oct 17, 2026

   ContributorAuthor

   More actions

   Verified and closed (coordinator, epic #135, check-in 1, 2026-10-17).

   • Range and CI: f638266..3abaa81 (PR #140 chore: catch up co-core 0.19.6 -> 0.36.0 #141). CI is green by job (lint, test, gcs T4) on 3abaa81, which is main's tip.
   • Deployed: replicator.service ActiveEnterTimestamp is 12:50:09Z, after the 12:38Z merge. /run/replicator/build-id is 3abaa81. The venv has co-core, -aio and -sync at 0.36.0. The main checkout is on main, clean, at origin/main.
   • Journal: the fetch policy replay completed (tracked_hosts 4, 506 messages), then worker ready with build 3abaa81 and persist enabled. There are no Import, Attribute or Validation errors, no poll cycle failed, and no replicator-failure entries.
   • Step 4: the manifest's context artifacts were re-indexed on read, and memory-protection-reference answers search.
   • Verification debt: no command has been processed since the restart. The first fetch, replicate or persist outcome on 0.36.0 is tracked on the epic.
   • Moved on: the "For Apply FetchPolicyState.generation only if newer #136" notes went to Apply FetchPolicyState.generation only if newer #136, and Drop the unused co-core [extract] extra #142 is bundled into Apply FetchPolicyState.generation only if newer #136 (operator decision).
8. gregoryfoster

   closed this as completedon Oct 17, 2026
9. gregoryfoster

   added a commit that references this issue on Oct 20, 2026

   #145 feat: keep the co-core import-closure diff as a script

   ...

   607be67