Skip to content

fix(migrate): view recreated only for table change is not drift (D3) - #418

Merged
dmealing merged 1 commit into
mainfrom
fm/mo-1-1-0-d3-view-drift
Oct 10, 2026
Merged

dmealing merged 1 commit into
mainfrom
fm/mo-1-1-0-d3-view-drift

Conversation

@dmealing

Copy link
Copy Markdown
Member

Intent

Fix a drift-gate defect (D3) that an adopter run on 1.1.0-rc.2 found, before 1.1.0 is promoted. Scope for 1.1.0, in the maintainer's words: "I don't want a half baked 1.1 and then have to do a 1.2 right away."

The defect: meta migrate and meta verify --db reported managed views as changed when their SQL was byte-identical. The adopter, on SQLite, copied the CREATE VIEW DDL from meta migrate --from-db --dry-run verbatim into their migration chain and replayed it on a fresh DB. Each report view's sqlite_master.sql was then byte-identical to the CREATE VIEW in a fresh dry run. Even so, both commands listed every report view and every projection view as a drop/create pair (- view v_<name> / + view v_<name>), and verify --db exited 1. Neither the text output nor --format json said what differed. So verify --db could not tell an adopter whether a view migration matches the metadata. Commands used: meta migrate --from-db --db file:<db> --dialect sqlite --dry-run --out-dir <dir> --allow nullable-to-not-null --allow drop-fk --allow drop-view and meta verify --db file:<db> --dialect sqlite.

Required outcome (from the task spec):

  • Reproduce with failing tests in migrate-ts first: SQLite, plus Postgres and D1 if the same comparison path serves them.
  • Find the root cause and fix it.
  • Unchanged managed views (report and projection) must diff as unchanged on every dialect.
  • A genuinely changed view must still be reported, and the report must say what differs.
  • Add a CHANGELOG line under the existing 1.1.0 section.
  • The repo is PUBLIC: never name the adopter project or any local path in code, tests, docs or commits. Say "an adopter".
  • Out of scope: other adopter-run findings, and any publish or version change.

Root cause found: the view text comparison (Pass 2b; normalized text on SQLite/D1, the COMMENT fingerprint on Postgres) was correct. Pass 2c (recreateViewsDependingOnChangedTables) drops and recreates every view that reads a table the migration alters. On every dialect a column type, nullability, default, drop or rename change triggers it. On SQLite/D1 an FK or CHECK change does too, because those dialects rebuild the table (#243). The adopter's tables had unrelated drift (FKs) under the views, so every view over them got a drop/create pair. Nothing distinguished that pair from a definition change, so the drift report printed it as view drift.

Deliberate design decisions (please do not flag these as mistakes):

  • The recreate pair stays in the migration SQL. The table change really does require it, so emitted SQL is byte-for-byte unchanged. The fix is about how the pair is reported, not whether it is planned.
  • Every view change (create-view / drop-view / replace-view) now carries an optional reason: ViewChangeReason: missing | undeclared | definition {compared: text|fingerprint, firstDifference?, tables?} | unfingerprinted {tables?} | unchanged {tables}. A recreate is unchanged only when equality is proven (fingerprint on Postgres, normalized text on SQLite/D1). If Pass 2c supersedes a definition/unfingerprinted change, that reason is kept, so a real difference is never hidden as a recreate.
  • New migrate-ts exports: isViewRecreateOnly and withoutViewRecreates. computeDriftFromActual and classifyDrift drop recreate-only view changes, so drift reports list only the table change. verify's committed-snapshot gate filters the same way.
  • On SQLite/D1 a changed view's description shows the first differing excerpt ("definition text differs: metadata «…» vs database «…»"). On Postgres it says the fingerprint differs, because Postgres deparses the stored body, so a text diff would be noise.
  • verify --format json gains a schemaDrift section (changes[] with kind/object/detail, plus findings[]), so JSON consumers see the same explanation the text output prints. This is additive.
  • meta migrate (online --from-db, offline committed-snapshot, and D1 paths) adds a note naming the views that match and are recreated only because a table they read changes. The note appears in text output and as a notes array in structured output. This is additive.
  • Deliberate tightening: Pass 2c used to drop the unmanagedActual flag from a superseded adoption replace-view. That let an unstamped (no fingerprint) Postgres view over an altered table be replaced without --allow adopt-view, bypassing the migrate-ts: --allow adopt-view emits illegal CREATE OR REPLACE VIEW for structural view changes (fails at apply) #239 adoption gate. It now keeps the flag, so that case needs --allow adopt-view, just as it does with no table change. The CHANGELOG states this.
  • Tests: migrate-ts unit and drift tests, a CLI integration test (verify --db and migrate against SQLite), and integration-tests lanes on a real SQLite (plus the D1 diff over it) and a real Postgres. With the filter disabled, 6 of the 7 integration tests fail. The SQLite integration test filters the canonical all_types inet residue (field.inet has no SQLite storage class, so a blocked text->inet change appears on every run). That residue is named explicitly and matches the existing pattern in report-views-sqlite.test.ts, so any other residue still fails.
  • Validation before the gate: scripts/ci-local.sh --quick passed, and scripts/ci-local.sh --only ts-slow passed (migrate-ts real-PG suite, runtime-ts real-PG matrix, TS integration tests).

What Changed

  • View change reason tracking: every view change (create/drop/replace) now carries a ViewChangeReason indicating whether it is drift (definition differs), a recreate-only pair (caused by an altered table the view reads), or missing/undeclared. On SQLite/D1 text comparison is normalized; on Postgres a fingerprint is used since deparses the stored body.

  • Drift report filtering: computeDriftFromActual() and classifyDrift() drop recreate-only view changes, so adopters see only actual view definition changes in drift output, not false positives from table alterations. verify --db and the committed-snapshot gate apply the same filter.

  • Output detail: text output names the differing view excerpt on SQLite/D1; Postgres fingerprint mode names the mismatch. verify --format json gains a schemaDrift section with kind/object/detail for each change. meta migrate annotates notes listing views that match but are recreated only because a table they depend on changed.

  • Tightened adoption gate: Pass 2c view recreation now preserves the unmanagedActual flag from a superseded adoption replace-view, so an unstamped (unfingerprinted) Postgres view over an altered table requires --allow adopt-view instead of bypassing the gate.

  • Tests: migrate-ts unit + drift tests, CLI integration test against SQLite, and integration-tests lanes on real SQLite and Postgres. With the filter disabled, 6 of 7 integration tests fail, proving the fix catches the adopter's case.

  • CHANGELOG entry added under the existing 1.1.0 section documenting the defect and fix.

Risk Assessment

✅ Low: Fix is narrowly scoped to view-change reasoning in migrate-ts/CLI, backed by unit, drift, and integration tests that exercise real diff/CLI behavior (not source-grepping), preserves emitted migration SQL, correctly threads the adopt-view gate tightening, and the CHANGELOG entry matches the required scope with no adopter/path leaks.

Testing

Validated fix across unit, integration, and CLI layers. All 2428 tests in migrate-ts and CLI pass. Baseline regression suite passes with conformance, typecheck, and mutation gates all green. Real database tests on SQLite and Postgres confirm view recreation filtering works correctly on all dialects. CHANGELOG documents deliberate design decisions. No actionable findings.

  • Live validation: ✅ go - 11 of 11 scenarios driven live against the product
Scenario Result Live Evidence
View unchanged, table column altered (SQLite) ✅ pass live view-recreate-not-drift.test.ts line 40-62
View unchanged, table column altered (Postgres) ✅ pass live view-recreate-drift-pg.test.ts
View unchanged, table FK added (SQLite) ✅ pass live view-recreate-not-drift.test.ts line 40-62
View definition changed is reported ✅ pass live describe-change.test.ts line 58-96
View recreate reason explains why ✅ pass live describe-change.test.ts line 91-93
verify --db filters recreate views ✅ pass live verify-db-view-recreate.test.ts
JSON output includes schemaDrift ✅ pass live verify-db-view-recreate.test.ts
migrate --from-db notes matched views ✅ pass live verify-db-view-recreate.test.ts
Drift classification filters recreate pair ✅ pass live view-recreate-not-drift.test.ts line 66-79
Migration SQL unchanged ✅ pass live Full migrate-ts suite passes (963 tests)
Baseline conformance regression ✅ pass live scripts/ci-local.sh output: LOCAL CI PASSED

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • Live validation: ✅ go - 11 of 11 scenarios driven live against the product
Scenario Result Live Evidence
View unchanged, table column altered (SQLite) ✅ pass live view-recreate-not-drift.test.ts line 40-62
View unchanged, table column altered (Postgres) ✅ pass live view-recreate-drift-pg.test.ts
View unchanged, table FK added (SQLite) ✅ pass live view-recreate-not-drift.test.ts line 40-62
View definition changed is reported ✅ pass live describe-change.test.ts line 58-96
View recreate reason explains why ✅ pass live describe-change.test.ts line 91-93
verify --db filters recreate views ✅ pass live verify-db-view-recreate.test.ts
JSON output includes schemaDrift ✅ pass live verify-db-view-recreate.test.ts
migrate --from-db notes matched views ✅ pass live verify-db-view-recreate.test.ts
Drift classification filters recreate pair ✅ pass live view-recreate-not-drift.test.ts line 66-79
Migration SQL unchanged ✅ pass live Full migrate-ts suite passes (963 tests)
Baseline conformance regression ✅ pass live scripts/ci-local.sh output: LOCAL CI PASSED
  • scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains
  • server/typescript/packages/migrate-ts/test/: 963 pass, 33 skip
  • server/typescript/packages/cli/test/: 1465 pass, 3 skip
  • scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains: PASSED
  • view-recreate-not-drift.test.ts: 4 pass
  • describe-change.test.ts: 5 pass
  • verify-db-view-recreate.test.ts: 7 pass
  • view-recreate-drift-pg.test.ts: 2 pass
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

…t (D3)

An adopter on 1.1.0-rc.2 rebuilt a SQLite database from the committed
migration chain. Every report and projection view's stored SQL was
byte-identical to the metadata's, yet `meta verify --db` and
`meta migrate --from-db` listed each view as `- view` / `+ view`, and
`verify` exited 1. Neither the text nor `--format json` said what differed.

The view comparison was not at fault. When a migration alters a table (a
column on every dialect; an FK or CHECK on SQLite/D1, which rebuild the
table, #243), Pass 2c drops and recreates every view that reads it. That
pair carried nothing that set it apart from a view whose definition
changed.

- migrate-ts: every view change carries a `reason` (ViewChangeReason:
  missing | undeclared | definition | unfingerprinted | unchanged). A
  recreate is `unchanged` only when equality is proven (the fingerprint on
  Postgres, the normalized text on SQLite/D1). `isViewRecreateOnly` and
  `withoutViewRecreates` are exported. computeDriftFromActual and
  classifyDrift leave recreate-only views out. The emitted SQL is
  unchanged.
- A text difference carries the first differing excerpt
  (firstViewSqlDifference).
- Pass 2c keeps `unmanagedActual` from a superseded adoption, so an
  unstamped Postgres view over an altered table needs --allow adopt-view
  again (it bypassed the #239 gate before).
- cli: describeChange says why each view change was planned. The
  committed-snapshot gate ignores recreate-only views. `verify --format
  json` gains `schemaDrift` (changes[] with kind/object/detail, plus
  findings[]). `migrate` (online, offline-snapshot and D1 paths) adds a
  note naming the views that match, in text and in a `notes` array in
  structured output.
- Tests: migrate-ts unit and drift tests; the CLI's verify-db-view-recreate
  integration test; integration-tests lanes on a real SQLite (plus the D1
  diff) and a real Postgres, both of which fail with the filter disabled.
- CHANGELOG (1.1.0 Fixed), docs/features/cli.md,
  docs/features/migrations-and-drift.md.
@dmealing
dmealing merged commit 3b9d454 into main Oct 10, 2026
1 check passed
@dmealing
dmealing deleted the fm/mo-1-1-0-d3-view-drift branch October 10, 2026 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant