Skip to content

Document that restore_backup checks a backup before replacing the database - #721

Open
cb1kenobi wants to merge 11 commits into
mainfrom
docs/restore-verify-before-replace
Open

cb1kenobi wants to merge 11 commits into
mainfrom
docs/restore-verify-before-replace

Conversation

@cb1kenobi

@cb1kenobi cb1kenobi commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

⊙ Problem

HarperFast/harper#3121: Verify a backup can be read before restore_backup replaces the database changes restore_backup in 5.4. The backup reference still describes the old close-purge-restore order, and this PR's earlier text described a staging design the core PR has since dropped.

💡 Solution

Document the shipped behavior as of v5.4.0: a restore checks the backup in place before it deletes the database, needs no extra disk, and refuses a backup this version can't read with the database left as it was. An online restore's check runs inside its downtime, and a crash during the check leaves the database loadable.

🔧 Changes

  • reference/backups/operations.md, restore_backup.
    • A v5.4.0 "changed" badge and the new order: close, check, restore, reload.
    • What the check covers: checksums of every database file (not blob files), the transaction logs, each table file's format and compression, and a declared blob snapshot's presence. Also what a refusal says, the exception for a rerun over an incomplete restore, and that verify_backup does not open tables.
    • What it costs: one extra read of the backup, inside the downtime for an online restore, with no extra disk. Writes during that time fail rather than being lost. Blob contents are not checked.
    • Interruptions: only an interruption after the restore has started replacing files leaves the database blocked. One during the check leaves it loadable.
  • reference/backups/overview.md.
  • release-notes/v5-lincoln/5.4.md. A "Restore Checks a Backup Before Deleting the Database" entry under a new Backups section. The 5.3 entry this PR used to add is removed.

✅ Verification

  • npm run build passes and npm run format:check is clean.
  • Every behavior claim was checked against the core branch at its current head: verification order and coverage in dataLayer/rocksdbBackup.ts, marker timing in dataLayer/restoreMarker.ts, and the design note in dataLayer/DESIGN.md. The core branch's unit tests cover each claim (refusals online and offline, check-after-close, a loadable database after a claim that never reached the purge).

🤖 Generated by Anthropic Claude (Opus); posted via @cb1kenobi.

Related PRs: none found
Complexity: easy

Review-Coverage: authored=claude; ran=gemini,cursor-composer,codex,cursor-muse; adjudicated=domain; declined=cursor-grok,cursor-kimi; rounds=5; full=3 @ 7fb257e

Review-Attention: skim ~2m (decisions: summary-brevity) @ 7fb257e

cb1kenobi and others added 5 commits October 8, 2026 13:59
…base

Documents HarperFast/harper#3121: restore stages and opens the backup first, refuses one it
cannot read with the database untouched, needs free space for a second copy, and cannot restore
a database whose directory is a symlink.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…unt points and 5.3.2

Docs review round 1: blobs are neither staged nor checked, so the guarantee is stated for the
database files only; the refusal message is quoted by its phrase rather than as a suffix; staging
space is the backup's restored size and a rerun after an interrupted swap can need one more copy;
mount-point database directories are refused like symlinks (core change in the same PR); and the
5.3 release notes record the behavior change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fit, and when the aside copy goes

Docs review round 2: an online restore stages while the database keeps serving, and writes made
then are replaced; a staging copy without headroom is refused with 507 (core change in the same
PR); an uninterrupted restore removes the set-aside copy itself; a crash during staging leaves the
files untouched but blocked; the refusal comes from "the restore", which offline has no job; and
the release note counts transaction logs in the space needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hind

Docs review round 3: a refused rerun over an unfinished restore leaves the database blocked, not
untouched; a rerun needs no more room than the first attempt (core drops the stale candidate
first); repointing a symlinked path after an interrupted restore needs an offline rerun before
the next start; the overview's worked example describes staging; and a restore refused before the
database closes leaves its subscriptions running (verified against the branch).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gemini-code-assist[bot]

This comment was marked as resolved.

@github-actions
github-actions Bot temporarily deployed to pr-721 October 8, 2026 20:36 Inactive
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-721

This preview will update automatically when you push new commits.

Gemini review on #721: per the docs guidelines, a patch-level behavior change to an existing
operation carries a `changed` VersionBadge, under the heading and inline in the Limitations item.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions
github-actions Bot temporarily deployed to pr-721 October 8, 2026 20:42 Inactive
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-721

This preview will update automatically when you push new commits.

cb1kenobi and others added 2 commits October 13, 2026 16:24
The restore no longer stages a copy. It verifies the backup in place
(checksums, transaction logs, every table file's format and compression,
and the presence of a declared blob snapshot) before it deletes the
database, needs no extra disk space, and checks after an online restore
has closed the database. Moves the release note from 5.3 to 5.4 to match
the harper PR's milestone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cb1kenobi cb1kenobi changed the title Document that restore_backup checks a backup before replacing the database Document that restore_backup checks a backup before replacing the database Oct 13, 2026
…un exception

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions
github-actions Bot temporarily deployed to pr-721 October 14, 2026 02:55 Inactive
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-721

This preview will update automatically when you push new commits.

@cb1kenobi
cb1kenobi marked this pull request as ready for review October 14, 2026 03:05
@cb1kenobi
cb1kenobi requested a review from a team as a code owner October 14, 2026 03:05

@kriszyp kriszyp left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Reviewed with Codex

Comment thread reference/backups/operations.md Outdated
@github-actions
github-actions Bot temporarily deployed to pr-721 October 14, 2026 20:35 Inactive
@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-721

This preview will update automatically when you push new commits.

Comment thread reference/backups/operations.md Outdated

As of v5.3.1, subscriptions opened before a restore do not carry over to the restored data. When Harper reloads the database, each one ends, and its last message is a `DatabaseGenerationChangedError` (status code 409, code `DATABASE_GENERATION_CHANGED`). Resubscribe to resynchronize against the restored state. A restore that fails before it changes anything reloads the database as it was; those subscriptions end instead with a retryable `DatabaseClosingError` (status code 503, code `DATABASE_CLOSING`); resubscribe to continue.

As of v5.4.0, a restore checks the backup before it deletes anything. It verifies the checksum of every database file in the backup (not blob files), checks the backup's transaction logs, and opens each of its table files to confirm this version of Harper can read their format and compression. For a backup that captured blobs, it also checks that the blob snapshot is present. A backup that fails any of these is refused: the restore fails with an error saying it did not modify the database, followed by the cause, and the database keeps its data and its blobs. Previously the restore deleted the database first, so a backup this version could not read, such as one taken by a newer version of Harper, destroyed the database it was meant to restore. A rerun over an earlier restore that did not finish is the exception: that database is already incomplete, so it stays blocked until a restore succeeds, and the error says so. [`verify_backup`](#verify_backup) does not open the table files, so a backup it reports as valid can still be refused here for a format this version cannot read.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any way we could tuck "Previously the restore..." a bit more? It'll quickly become dated information, and unnecessary knowledge. Sticking it somehow under a collapsed < 5.4 section or something would let us identify it as old knowledge, and avoid communicating it to new users, you know?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 09082ba675bb.

@github-actions

Copy link
Copy Markdown

🚀 Preview Deployment

Your preview deployment is ready!

🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-721

This preview will update automatically when you push new commits.

This branch was successfully deployed

1 active deployment
pr-721 — 09082ba6 Deployed Oct 19, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants