Transport finite self-referencing family batches with typed refs

agent-notes/self-reference-family-transport-303.md

@@ -0,0 +1,26 @@
+# Self-reference family batch transport (#303 consumer)
+
+## Goal
+Extend existing occurrence transport with an explicit finite batch operation so self-reference targets' family-owned identities are known before rewriting references. No consumer semantics or identity invented in Quire.
+
+## State
+- REVIEW correction: reproduced real Propstore merge `1409906437aed51275a43505fc3fc8a9d29d27df` collapsing valid equal-support legacy A/B mutual attacks into self-attacks; preferred clusters changed 1 -> 0. Source/parents valid and old rows round-trip. Dung p.327 Example 9 vs p.328 Lemma 15 establishes different semantics. This is a consumer invariant, so Quire retains non-injective maps. Added optional typed `validate_transport(originals, images)` callback to family declaration/decorator/contract and finite-batch exits, before consumer coalescence. Coreference owns its injectivity admission; no policy/domain string/hash changes in Quire. Focused new/edited owner file: **20 passed in 0.77s**, `family-transport-admission-20261019-233812.log`, including explicit acceptance of a non-injective reciprocal-link map without family policy and delivery of complete before/after batches to a rejecting family callback. Consumer edited file: **12 passed in 13.94s**, `legacy-cycle-collapse-rejection-20261019-233809.log`. Original reproduction retained; corrected draft publication/CI next.
+- Isolated branch `feat/self-reference-family-transport-303`, base/default `03f7173cd2cc0eb72cd595498ac762f92708bc8c`. Main checkout dirty files preserved; read global and project AGENTS/CLAUDE, existing transport note, family_transport.py, reference metadata/rewriter, charter callback and tests.
+- Consumer source proof: Propstore coreference IDs hash supports only; attacks are a separate relation updated without changing IDs. Missing self-FK metadata and sequential row transport prevent complete attack transport. Dung 1995 Definition 2 p.326 models attacks as a binary relation over abstract arguments; Schultz/Wisnesky 2015 p.11 requires the image map commute with reference operations. Primary page images read, no extracted text.
+- Initial implementation added 56 generic lines plus owner tests. Focused new-file run PASSED: **17 passed in 1.06s**, `logs/test-runs/self-reference-family-transport-20261019-230808.log`. Covers all six orderings with one/two moved external targets, legacy referrer closure, self loop, unchanged objects/IDs, dangling refs, missing callback, and explicit rejection of identity depending incompatibly on resolved self refs.
+- Consumer first focused run FAILED (12 failed, 23 passed): fixtures omit the canonical tree on their fixed proposal branch; an existing merge preparation edge passes string identities to a typed reference placement (`AttributeError: 'str' object has no attribute 'argument_id'`). Stopped all implementation/publication under global AGENTS.md. No draft, pin update or full CI yet. This generic operation is not claimed integrated until those consumer edges are corrected and verified with parent direction.
+- Parent explicitly authorized diagnosed corrections and continued ordinary test diagnosis within scope. Added `FamilyDefinition.ref_from_document`: the declared flat placement's existing factory constructs the actual typed ref from the declared document identity; no application convention or fallback. Propstore consumes that owner operation. Corrected fixtures seed canonical proposal snapshots and use existing author/transaction owners for legacy records.
+- Corrected owner tests PASSED: **18 passed in 0.72s**, `logs/test-runs/self-reference-typed-ref-20261019-231748.log`, including real transport -> typed ref -> store preparation. Corrected consumer tests PASSED: **35 passed in 32.32s**, consumer `coreference-owner-corrections-20261019-231745.log`; includes actual app authoring, cyclic/self attacks, Dung sidecar query, evidence conflicts, legacy reads and retained graph/path-collision behavior. No checks weakened. Original failed logs remain intact.
+- FINAL draft https://github.com/propstore/quire/pull/20 at exact source `55f1ad123ba37bb3cbd64a8ede4d96c8f3da0e97`, OPEN/DRAFT. PR CI https://github.com/propstore/quire/actions/runs/39374405169 and push CI https://github.com/propstore/quire/actions/runs/39374322926 both completed SUCCESS at this exact head. All four Ubuntu/Windows Python 3.11/3.13 jobs per run passed lint/typecheck/tests/build, each **605 passed, 12 benchmarks deselected**. All eight checks green.
+- Consumer draft https://github.com/propstore/propstore/pull/324 at `f5f5713fc49e020afa76050e726311f49e20c614` pins this exact source in manifest and both lock references; full CI https://github.com/propstore/propstore/actions/runs/39374494927 succeeded: **1981 passed, 4 skipped, 4 warnings in 77.25s**. Both drafts remain unmerged; #303 remains open and untouched. No blocker. Final note-only checkpoint stays local; untracked logs preserve original test evidence.
+
+## Decisions and why
+- Smallest operation (~60 lines): caller supplies Sequence of typed documents; use charter self-FKs to build reverse dependencies; seed affected rows from currently mapped FKs, plan each affected row once with its existing callback and propagate actual identity changes to referrers; perform existing one-row transport after all images are assigned. This is finite reachability, not a guessed hash fixed point. Final single-image enforcement rejects callback identities inconsistent with resolved self references.
+- Unchanged objects remain unchanged. No-self-FK families use existing transport. Source refs validate with existing owner. No new FK parser, callback metadata, source scan, compatibility alias, storage format, or commit/ref concurrency mechanism.
+- Active cleanup-refactor control: generic transport is already owned here (keep/extend); consumer replaces its row-at-a-time caller with this owner batch. No wrong file/surface requires deletion. Search: no Propstore imports/domain strings, no recursion/hash algorithm. Runtime: edited owner tests plus consumer production-path regressions, full PR CI.
+
+## Surprises
+- None yet.
+
+## Next step
+Publish this bounded family-admission extension to existing draft #20, update consumer's exact pin, publish its coreference-owned correction to draft #324, and follow full fresh CI. Parent re-review remains required; no merge/ready transition or recursive identity/concurrency redesign.

quire/charter_class.py

@@ -18,7 +18,7 @@
 
 from __future__ import annotations
 
-from collections.abc import Callable, Mapping
+from collections.abc import Callable, Mapping, Sequence
 from dataclasses import dataclass, field as _dc_field, replace
 from enum import Enum
 import sys
@@ -639,6 +639,7 @@ def charter(
     accessor: str | None = None,
     family_foreign_keys: tuple[ForeignKeySpec, ...] = (),
     transport_identity: Callable[[Any], Any] | None = None,
+    validate_transport: Callable[[Sequence[Any], Sequence[Any]], None] | None = None,
     reference_keys: tuple[Any, ...] = (),
     family_metadata: Mapping[str, object] | None = None,
     semantic_metadata: Mapping[str, object] | None = None,
@@ -752,6 +753,7 @@ def decorate(cls: _T) -> _T:
             accessor=accessor,
             foreign_keys=family_foreign_keys,
             transport_identity=transport_identity,
+            validate_transport=validate_transport,
             identity_field=identity_field,
             reference_keys=cast("tuple[Any, ...]", reference_keys),
             metadata=family_metadata,

quire/families.py

@@ -10,6 +10,7 @@
     ArtifactFamily,
     ArtifactHandle,
     ArtifactPlacementPolicy,
+    FlatYamlPlacement,
     PreparedArtifact,
     RefBlobLocator,
 )
@@ -87,6 +88,7 @@ class FamilyDeclaration(Generic[TOwner, TKey, TRef, TDoc]):
     identity_field: str | None = None
     reference_keys: tuple[ReferenceKey, ...] = ()
     metadata: Mapping[str, object] | None = None
+    validate_transport: Callable[[Sequence[TDoc], Sequence[TDoc]], None] | None = None
 
     def to_artifact_family(self) -> ArtifactFamily[TOwner, TRef, TDoc]:
         return ArtifactFamily(
@@ -114,6 +116,7 @@ def to_definition(self) -> FamilyDefinition[TOwner, TKey, TRef, TDoc]:
             foreign_keys=self.foreign_keys,
             identity_policy=self.identity_policy,
             transport_identity=self.transport_identity,
+            validate_transport=self.validate_transport,
             identity_field=self.identity_field,
             reference_keys=self.reference_keys,
             metadata=self.metadata,
@@ -133,6 +136,7 @@ class FamilyDefinition(Generic[TOwner, TKey, TRef, TDoc]):
     identity_field: str | None = None
     reference_keys: tuple[ReferenceKey, ...] = ()
     metadata: Mapping[str, object] | None = None
+    validate_transport: Callable[[Sequence[TDoc], Sequence[TDoc]], None] | None = None
 
     def __post_init__(self) -> None:
         if not self.name:
@@ -178,6 +182,18 @@ def reference_index_from_records(
             keys=self.reference_keys,
         )
 
+    def ref_from_document(self, document: TDoc) -> TRef:
+        """Construct a flat document's typed ref with its declared factory."""
+        placement = self.artifact_family.placement
+        if not isinstance(placement, FlatYamlPlacement):
+            raise TypeError(f"family {self.name!r} does not have a flat identity placement")
+        if self.identity_field is None:
+            raise ValueError(f"family {self.name!r} has no identity field")
+        identities = ReferenceKey.field(self.identity_field)(document)
+        if len(identities) != 1:
+            raise ValueError("a document ref requires a single identity")
+        return placement.ref_factory(identities[0])
+
     def contract_body(self) -> dict[str, object]:
         body: dict[str, object] = {
             "accessor": self.accessor_name,
@@ -195,6 +211,10 @@ def contract_body(self) -> dict[str, object]:
             body["transport_identity"] = (
                 f"{self.transport_identity.__module__}.{self.transport_identity.__qualname__}"
             )
+        if self.validate_transport is not None:
+            body["validate_transport"] = (
+                f"{self.validate_transport.__module__}.{self.validate_transport.__qualname__}"
+            )
         if self.identity_field is not None:
             body["identity_field"] = self.identity_field
         if self.reference_keys:

quire/family_transport.py

@@ -8,7 +8,7 @@
 
 from __future__ import annotations
 
-from collections.abc import Iterator, Mapping
+from collections.abc import Iterator, Mapping, Sequence
 from dataclasses import dataclass, field
 from types import MappingProxyType
 from typing import Any, TypeVar
@@ -97,6 +97,66 @@ def transport(
         self.assign(family.name, old_ids[0], new_ids[0])
         return result
 
+    def transport_family(
+        self,
+        family: FamilyDefinition[Any, Any, Any, TDoc],
+        documents: Sequence[TDoc],
+    ) -> tuple[TDoc, ...]:
+        """Transport a finite batch after planning its affected self-reference IDs.
+
+        The family owns identity. Planning uses its callback with external FKs
+        mapped and self FKs unchanged, then follows incoming references when an
+        ID moves. Final transport must produce the same planned IDs; callbacks
+        that require a different recursive identity construction fail the
+        existing single-image check. No recursive hashes are guessed here.
+        """
+        self_specs = tuple(s for s in family.foreign_keys if s.target_family == family.name)
+        if not self_specs:
+            results = tuple(self.transport(family, document) for document in documents)
+            if family.validate_transport is not None:
+                family.validate_transport(documents, results)
+            return results
+        if family.identity_field is None:
+            raise ValueError(f"family {family.name!r} has no identity field")
+        identity = ReferenceKey.field(family.identity_field)
+        external_specs = tuple(s for s in family.foreign_keys if s.target_family != family.name)
+        dependents: dict[str, list[int]] = {}
+        pending: list[int] = []
+        for index, document in enumerate(documents):
+            for spec in family.foreign_keys:
+                validate_foreign_key(spec, document, self.source_indexes[spec.target_family])
+            if map_foreign_keys(document, family.foreign_keys, self._reference_image) is not document:
+                pending.append(index)
+            for spec in self_specs:
+                for reference in ReferenceKey.field(spec.source_field)(document):
+                    source_id = self.source_indexes[family.name].require_id(reference)
+                    dependents.setdefault(source_id, []).append(index)
+        planned: set[int] = set()
+        while pending:
+            index = pending.pop()
+            if index in planned:
+                continue
+            planned.add(index)
+            if family.transport_identity is None:
+                raise UnsupportedFamilyTransportError(
+                    f"family {family.name!r} has no identity operation for reference transport"
+                )
+            document = documents[index]
+            external = map_foreign_keys(document, external_specs, self._reference_image)
+            old_ids = identity(document)
+            new_ids = identity(family.transport_identity(external))
+            if len(old_ids) != 1 or len(new_ids) != 1:
+                raise ValueError("family transport requires a single source and result identity")
+            self.assign(family.name, old_ids[0], new_ids[0])
+            if old_ids[0] != new_ids[0]:
+                pending.extend(dependents.get(old_ids[0], ()))
+        results = tuple(self.transport(family, document) for document in documents)
+        for index in planned:
+            self.assign(family.name, identity(documents[index])[0], identity(results[index])[0])
+        if family.validate_transport is not None:
+            family.validate_transport(documents, results)
+        return results
+
     def iter_images(self) -> Iterator[FamilyReferenceImage]:
         """Yield assigned occurrence images, including unchanged explicit images."""
         for family, images in sorted(self._images.items()):