fix(backup): resolve the effective interval identically everywhere (audit C1, H3) - #425
Merged
Merged
Conversation
Two halves of the same disagreement about what a backup schedule means. The staleness/reconcile scan set (`scan_rows`) inner-joined `server_group_backup_schedule`, so a `(group, type)` with no override row — the out-of-the-box case, since `tamanu-postgres` ships a canopy-wide 6h default and rows are only written by explicit `set_schedule` — was never scanned. The schedulers back those pairs up happily; nothing then checked that the backups kept arriving, so a device that stopped backing up raised no `backup-staleness`/`backup-never` issue, and the reconcile checks (report-gap, size-mismatch, reconcile-missing) never ran for it either. Conversely the resolvers collapsed "no override row" (inherit the default) into "override row with a NULL interval", which the model and the `set_schedule` contract both document as manual-only. Setting a group to manual-only silently reverted it to the type default: still commanded to back up, and shown in `group_schedules` with an interval and a next run. Both now go through `database::backups::effective_interval`: an override row answers on its own, NULL included; only its absence inherits the type default. The scan left-joins both sources and resolves the same way, so the set that is commanded to back up and the set that is monitored are one set. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SGfH1cdFKPnKpM7ytRThft
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes C1 (critical) and H3 (high) from the audit in #370. They're two halves of the same disagreement about what a backup schedule means, so they're one change.
C1 — the staleness scan skipped every group on the inherited default
backup::staleness::scan_rowsinner-joinedserver_group_backup_scheduleand filteredexpected_interval IS NOT NULL, with no fallback to the type default. A(group, type)with no override row — the out-of-the-box case, sincetamanu-postgresships a canopy-wide 6h default and rows are only written by an explicitset_schedule— was never in the scan set.The schedulers back those pairs up happily via
backups_due_now_for_server, but nothing then checked that the backups kept arriving. A device that stopped backing up raised nobackup-staleness/backup-neverissue, and becausereconcile::sweepconsumes the same rows, report-gap, size-mismatch and reconcile-missing never ran for it either.H3 — a manual-only schedule silently reverted to the default cadence
The three resolvers (
effective_interval_for_type,effective_interval_secs,group_schedules) used.and_then(|s| s.expected_interval).or_else(default), collapsing "no override row" (inherit) into "override row whose interval is NULL". The model and theset_schedulecontract both document the latter as manual-only, deliberately unlike the same row'sretention: Nonewhich does mean inherit.So an operator setting
expected_interval: nullgot a group that was still commanded to back up on the 6h default, still showed an interval and anext_run_atingroup_schedules— and, because the scan honoured NULL as manual-only, could never raise a staleness alert either.The fix
One resolver,
database::backups::effective_interval: an override row answers on its own, NULL included; only its absence inherits the type'sdefault_interval. All three call sites now go through it.scan_rowsleft-joins bothserver_group_backup_scheduleandbackup_type_defaultsand resolves the same way in Rust — a SQLCOALESCEcan't express it, because it would flatten manual-only back into the default. The set that is commanded to back up and the set that is monitored are now one set.Tests
database:effective_interval_precedencecovers all four states (no row, row with interval, row with NULL, row deleted).database:scan_includes_pair_inheriting_the_type_default_interval— the C1 regression, asserting the inherited 6h shows up on the scan row.database:scan_excludes_pair_whose_override_makes_it_manual_only— the default must not resurrect a manual-only pair.private-server:group_schedules_reports_manual_only_override_as_no_schedule— the API surface of H3.Ran green against a local Postgres 16 alongside the existing backup suites.
Generated by Claude Code