Skip to content

feat: add portable archive export and restore - #1561

Open
larry-zy wants to merge 2 commits into
oceanbase:masterfrom
larry-zy:feat/1421-portable-export-backup-restore
Open

larry-zy wants to merge 2 commits into
oceanbase:masterfrom
larry-zy:feat/1421-portable-export-backup-restore

Conversation

@larry-zy

@larry-zy larry-zy commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Closes #1421

Rationale for this change

PowerContext needs a backend-neutral logical archive for disaster recovery, offline transfer, customer-controlled backup, and SQLite/OceanBase migration without losing immutable history, lineage, review decisions, or Handoff evidence. Database-native backup remains a separate operational recovery mechanism.

What changes are included in this PR?

  • Adds a versioned .pcb contract with deterministic manifests, producer/schema versions, per-record SHA-256 digests, an aggregate checksum, deterministic ZIP output, and optional compression.
  • Streams scope-selective export, inspect, target-aware dry-run validation, and transactional/idempotent restore with content-free progress.
  • Preserves Scope relationships, Sources and journal positions, immutable Artifact Revisions, lineage and cross-Scope publication provenance, Memory versions/heads, Candidate decisions, Work Sources, Handoffs, and receipts.
  • Requires export authorization before database enumeration, rejects credentials/provider secrets/absolute host paths, and excludes access bindings, audit/usage/evaluation data, host-local state, cursors, and projections.
  • Adds immutable conflict detection, rollback-safe restore, durable readiness receipts, and Memory/Experience/Skill projection rebuilds.
  • Supports configured SQLite, SeekDB, and OceanBase deployments through the local CLI; includes an opt-in live SQLite-to-OceanBase acceptance test.
  • Documents format compatibility, recovery workflow, exclusions, and distinct SQLite/OceanBase native backup guidance.

Are there any user-facing changes?

Adds powercontext archive export, powercontext archive inspect, and powercontext archive restore. Restore writes require --yes; --dry-run performs target compatibility and conflict checks without domain writes. This introduces portable bundle format version 1 and the pc_portable_restore_receipts table. No existing API or bundle format is changed.

How was this change tested?

  • prek run -a
  • ty check
  • Focused archive/schema/CLI/E2E suite: 22 passed
  • make docs-test: 792 public pages and internal links verified
  • Full macOS suite: 2471 passed and 101 skipped; the remaining three tests are Linux systemd-only platform guards

AI usage statement

Codex (GPT-5) was used to assist implementation, tests, documentation, and review; all changes were reviewed and validated locally.

@Teingi

Teingi commented Sep 10, 2026

Copy link
Copy Markdown
Member

resolve conflicts

@larry-zy
larry-zy force-pushed the feat/1421-portable-export-backup-restore branch from ea1688e to fc1521a Compare September 11, 2026 13:26
@larry-zy
larry-zy force-pushed the feat/1421-portable-export-backup-restore branch from fc1521a to 3c0e6fb Compare September 11, 2026 19:27
@larry-zy
larry-zy marked this pull request as ready for review September 11, 2026 19:28
@larry-zy larry-zy changed the title feat: add local portable archive CLI feat: add portable archive export and restore Sep 11, 2026

@Teingi Teingi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed bffe534. The focused archive suite passed (16 tests; the large-bundle memory test was excluded). I reproduced the six issues below in isolated SQLite databases; vector coverage used a deterministic embedding model. I did not run live OceanBase validation.

scope_field: str = "scope_id"


_SPECS: dict[RecordType, _RecordSpec] = {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve managed Skill package contents

_SPECS omits pc_skill_packages. An approved two-file Skill passes archive validation, but restore commits the Artifact rows and then raises RepositoryNotFoundError while rebuilding projections. Reopening the target Runtime fails with the same error. Please include referenced package bytes and validate those dependencies before committing the restored rows.

async def _establish_export_snapshot(connection: AsyncConnection, /) -> None:
"""Pin all export reads to one database snapshot before enumeration."""

if connection.dialect.name == "mysql":

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Establish an explicit SQLite read snapshot

This only pins a snapshot for MySQL. With the configured SQLite driver, a separate writer can commit between the journal-head and Source reads. I reproduced an archive with journal head 1 and a Source at position 2; validation and restore succeeded, but the next capture failed the journal-position unique constraint. Please explicitly begin a SQLite read transaction and validate journal consistency.

database,
projection_rebuilder=self.rebuild_portable_projections,
supported_source_types=tuple(definition.name for definition in self.source_registry.definitions),
supported_artifact_families=(Handoff.family, Memory.family, Experience.family, Skill.family),

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Include the registered Artifact families in restore support

The repository below supports Profile, Prompt, and Topic Memory, but this list rejects all three. Creating a Prompt through the public Runtime produces an archive that the same version cannot restore. Please align export and restore support, including the CLI validation list and each family's durable dependencies.

for scope_id in scope_ids:
services = self._services_for(scope_id)
_, catalog = services.sources()
await services.memory(catalog).rebuild_projections()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Rebuild vectors before reporting readiness

rebuild_projections() defaults its embedding model to None; it does not use the model configured on the Memory service. Restoring into a vector-enabled target therefore reports ready, while explicit vector search raises CapabilityNotSupportedError and auto falls back to FTS. Please pass the target embedding model and verify vector completeness before marking the restore ready.


# Parent rows always precede their dependent rows. This matters for SQLite as
# well as for MySQL/OceanBase foreign-key enforcement.
_EXPORT_ORDER: tuple[RecordType, ...] = tuple(_SPECS)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Preserve tag assignments in the archive

The export list excludes pc_artifact_tags. A Memory entry tagged backup-critical restores with an empty tag set even though the receipt reports ready, silently losing metadata and changing tag-filtered retrieval. Please include tag assignments and validate their target references.


def _emit_validation_result(source: Path, /, *, env_file: Path | None) -> None:
try:
_emit(asyncio.run(_validate_only(source, env_file=env_file)))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Return a failure exit code for an incompatible dry-run

_validate_only() can return compatible=False without raising, so this path prints the report and exits successfully. I reproduced compatible: false with exit code 0 for a Prompt bundle. Scripts checking the exit status will treat that validation as successful. Please inspect the result and return a nonzero exit code for incompatibility or conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add portable export, backup, and restore

2 participants