diff --git a/SAFETY.md b/SAFETY.md index 3ebeb82..467202d 100644 --- a/SAFETY.md +++ b/SAFETY.md @@ -30,6 +30,7 @@ The invariant registry (invariant IDs referenced below) lives in | `pkg/statement`, `pkg/planner`, `pkg/schemadiff`, `pkg/router`, `pkg/plan`, `pkg/lint`, `pkg/suggest` — classify/diff/route/report | ❌ periphery¹ | `pkg/statement` (parse boundary), `pkg/schemadiff` (introspect/diff via scratch execute-and-introspect), `pkg/planner` (classifier), `pkg/router` (backend assignment + availability policy), `pkg/plan` (versioned dry-run plan report), `pkg/lint` (offline typed findings), and `pkg/suggest` (advisory rewrites with typed caveats) exist (Phases 2.1–2.5) | (CO-7 holds at the parse boundary) | | `pkg/verdict` — structured outcome contract, rendering, exit codes | ❌ periphery | exists (Phase 1) | — | | `pkg/diffplan` — desired schema → routed convergence plan, the declarative front door as a library (the CLI `diff` and embedding orchestrators share it) | ❌ periphery | exists | — | +| `pkg/migrate` — one gated statement → resolve, classify, route, execute → one verdict; the imperative front door as a library (the CLI `migrate` and embedding orchestrators share it) | ❌ periphery² | exists | — | | `internal/cli` — CLI, flags, help, prompts | ❌ periphery | `migrate`, `status`, `diff`, `fmt`, `lint`, and `suggest` exist | — | | `pkg/progress` — strategy-wide progress snapshots; the executors' observation seam (core imports it, so its locking discipline is core-critical); copy counters reserved for later | ✅ core | native progress exists | — | | orchestrator adapter | ❌ periphery | planned (Phase 11) | OC-* hold *at* the boundary | @@ -41,6 +42,12 @@ Today a "copy" route reports unavailable; once copy-and-swap exists, a wrong "co produce a wasteful but *correct* schema change because the checksum will still gate it. The core executors re-verify their own preconditions and never trust that the planner checked. +² **The imperative front door is periphery for the same reason.** `pkg/migrate` sequences the +pipeline — gate, resolve, preflight, execute — but every dangerous step it requests is enforced +by the core packages it calls: the executors re-verify admission and run under their own bounded +budgets, and preflight's proof types gate what may execute. A wrong sequencing decision in +`pkg/migrate` yields a refusal or a bounded failed attempt, never an unbounded lock. + ## Rules inside the core The short version — the full rules live in [docs/tcb-model.md](docs/tcb-model.md): diff --git a/docs/architecture.md b/docs/architecture.md index c26744c..111c292 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -190,6 +190,7 @@ different levels of commitment: | `pkg/suggest` | Offline advisory surface: maps risky DDL to the safer native form the engine would run, with typed caveats and manual-path guidance; emits the versioned suggest report ([suggest-report.md](suggest-report.md)) | exists | | `pkg/plan` | Versioned machine-readable dry-run plan report — the one JSON contract both front doors emit and an orchestrator consumes | exists (Phase 2.5) | | `pkg/diffplan` | The declarative front door as a library: desired schema in, routed `plan.Report` out — the CLI `diff` and embedding orchestrators share this one pipeline | exists | +| `pkg/migrate` | The imperative front door as a library: one parsed statement in — gate, resolve, classify, route, execute — one `verdict.Verdict` out; the CLI `migrate` and embedding orchestrators share this one pipeline | exists | | `pkg/router` | Route classified statements to native / copy-and-swap / refuse dispositions; copy-and-swap reports unavailable until that backend lands | exists (Phase 2.4) | | `pkg/executor` | Bounded optimistic native attempt, the concurrent index build, and the autocommit safer-sequence runner, with stable outcome codes; the full `Executor` contract (`Plan`/`Execute`/`Status`/`Abort`) arrives with the copy-and-swap backend | native execution exists | | `pkg/progress` | Strategy-wide, pollable progress snapshots: native phase/elapsed time, sequence position, retry attempt, and server-reported concurrent-index work; optional copy counters are reserved for copy-and-swap | native progress exists | diff --git a/docs/schemabot-integration.md b/docs/schemabot-integration.md index 996479d..9144ee1 100644 --- a/docs/schemabot-integration.md +++ b/docs/schemabot-integration.md @@ -77,7 +77,7 @@ the integration phase starts; they drift.) | --- | --- | | `Name()` | a stable identifier, e.g. `"pg-sprite"` | | `Plan` | run parse → declarative diff when applicable → classify → route — exported as `diffplan.Plan` in [`pkg/diffplan`](../pkg/diffplan/diffplan.go) (parse via `statement.ParseDesired`, connect via `dbconn.NewPool`, inputs named by `diffplan.Request{Schema, Desired}`); return a `PlanResult` whose `SchemaChange.TableChanges` are `engine.TableChange{Table, Operation (statement.StatementType), DDL, IsUnsafe, UnsafeReason, ExecutionMode, ModeReason}`; map a **not native-safe** refusal to `engine.ExecutionModeBlocked` with the refusal reason as `ModeReason` (see [execution-mode verdicts](#execution-mode-verdicts-and-direct-execution)) | -| `Apply` | start the native executor asynchronously and return immediately; re-resolve the routing decision at execution time rather than trusting the stored plan-time verdict (see [execution-mode verdicts](#execution-mode-verdicts-and-direct-execution)) | +| `Apply` | start the native executor asynchronously and return immediately — the synchronous core is exported as `migrate.Run` in [`pkg/migrate`](../pkg/migrate/migrate.go) (parse via `statement.ParseOne`, gate via `migrate.Gate` before dialing, connect via `dbconn.NewPool`, policy via `migrate.DefaultOptions` tuned per table): one statement in, one `verdict.Verdict` out, with a three-shape contract — refusal (verdict, nil error), execution failure (failed verdict carrying the stable code and the committed prefix, plus the operational error), or an error with a zero verdict (stopped before executing). `Run` re-resolves the routing decision at execution time, so the adapter never trusts the stored plan-time verdict (see [execution-mode verdicts](#execution-mode-verdicts-and-direct-execution)) | | `Progress` | per-table rows-copied / total / percent / ETA / checksum state | | `Stop` / `Start` | checkpoint and resume (slot + copy + applier watermark) | | `Cutover` | the deferred, operator-gated atomic swap | diff --git a/internal/cli/dryrun.go b/internal/cli/dryrun.go index 60f23f0..465979e 100644 --- a/internal/cli/dryrun.go +++ b/internal/cli/dryrun.go @@ -2,18 +2,15 @@ package cli import ( "context" - "errors" "fmt" "io" - "github.com/jackc/pgx/v5/pgxpool" - "github.com/block/pg-sprite/pkg/dbconn" + "github.com/block/pg-sprite/pkg/migrate" "github.com/block/pg-sprite/pkg/plan" "github.com/block/pg-sprite/pkg/planner" "github.com/block/pg-sprite/pkg/preflight" "github.com/block/pg-sprite/pkg/router" - "github.com/block/pg-sprite/pkg/schemadiff" "github.com/block/pg-sprite/pkg/statement" "github.com/block/pg-sprite/pkg/verdict" ) @@ -33,7 +30,7 @@ func (c *MigrateCmd) runDryRun(ctx context.Context, out io.Writer) error { return err } logger.Debug("statement parsed", "kind", st.Kind(), "schema", st.Schema(), "table", st.Table()) - if v, refused := gateVerdict(st); refused { + if v, refused := migrate.Gate(st); refused { return c.emit(out, v) } @@ -43,7 +40,7 @@ func (c *MigrateCmd) runDryRun(ctx context.Context, out io.Writer) error { } defer pool.Close() - facts, targetFacts, tableExists, err := dryRunFacts(ctx, pool, st) + facts, err := migrate.LiveFacts(ctx, pool, st) if err != nil { return err } @@ -54,7 +51,7 @@ func (c *MigrateCmd) runDryRun(ctx context.Context, out io.Writer) error { if err != nil { return err } - classified, err := planner.Classify(canonical, facts) + classified, err := planner.Classify(canonical, facts.Classifier) if err != nil { return err } @@ -63,9 +60,9 @@ func (c *MigrateCmd) runDryRun(ctx context.Context, out io.Writer) error { "route", string(classified.Route), "disposition", string(routed.Disposition)) report := plan.NewReport(plan.SourceAlter) - report.Schema = resolvedSchema(st) + report.Schema = migrate.ResolvedSchema(st) report.Table = st.Table() - report.TableExists = tableExists + report.TableExists = facts.TableExists if report.ServerVersion, err = dbconn.ServerVersion(ctx, pool); err != nil { return err } @@ -77,11 +74,11 @@ func (c *MigrateCmd) runDryRun(ctx context.Context, out io.Writer) error { } report.Statements = append(report.Statements, ps) } - if targetFacts.Partitioned() { + if facts.Target.Partitioned() { refused := make([]bool, len(report.Statements)) for i := range report.Statements { var cause preflight.PartitionRefusalCause - cause, err = preflight.RefusesPartitionedParent(targetFacts.ServerMajor(), report.Statements[i].ExecSQL) + cause, err = preflight.RefusesPartitionedParent(facts.Target.ServerMajor(), report.Statements[i].ExecSQL) if err != nil { return err } @@ -108,41 +105,3 @@ func (c *MigrateCmd) runDryRun(ctx context.Context, out io.Writer) error { } return nil } - -// resolvedSchema is the schema the engine plans against: the statement's -// qualification, or public — the default the engine introspects — when a -// table-targeted statement leaves it unqualified. The report carries the -// resolved name, never the submitted one: a stored plan must not depend on -// the reader's search_path to say which table it describes. -func resolvedSchema(st statement.Statement) string { - if st.Schema() == "" && st.Table() != "" { - return "public" - } - return st.Schema() -} - -// dryRunFacts introspects the statement's target table for classifier -// facts. Statements without a single table target (index drops, REINDEX) -// and missing tables classify with zero facts. The returned tableExists -// mirrors the introspection outcome for the report: true when the table -// was found, false when it was looked up and missing, nil when the -// statement has no single table target to introspect. -func dryRunFacts(ctx context.Context, pool *pgxpool.Pool, st statement.Statement) (planner.Facts, preflight.TargetFacts, *bool, error) { - if st.Table() == "" { - return planner.Facts{}, preflight.TargetFacts{}, nil, nil - } - live, err := schemadiff.Introspect(ctx, pool, resolvedSchema(st), st.Table()) - switch { - case errors.Is(err, schemadiff.ErrTableNotFound): - exists := false - return planner.Facts{}, preflight.TargetFacts{}, &exists, nil - case err != nil: - return planner.Facts{}, preflight.TargetFacts{}, nil, err - } - targetFacts, err := preflight.LookupTargetFacts(ctx, pool, resolvedSchema(st), st.Table()) - if err != nil { - return planner.Facts{}, preflight.TargetFacts{}, nil, err - } - exists := true - return planner.FactsFrom(live), targetFacts, &exists, nil -} diff --git a/internal/cli/migrate.go b/internal/cli/migrate.go index f247c26..5db5b89 100644 --- a/internal/cli/migrate.go +++ b/internal/cli/migrate.go @@ -2,32 +2,25 @@ package cli import ( "context" - "errors" "fmt" "io" "log/slog" - "time" - - "github.com/jackc/pgx/v5" - "github.com/jackc/pgx/v5/pgxpool" "github.com/block/pg-sprite/pkg/dbconn" "github.com/block/pg-sprite/pkg/executor" - "github.com/block/pg-sprite/pkg/planner" - "github.com/block/pg-sprite/pkg/preflight" - "github.com/block/pg-sprite/pkg/router" + "github.com/block/pg-sprite/pkg/migrate" "github.com/block/pg-sprite/pkg/statement" "github.com/block/pg-sprite/pkg/verdict" ) -// run is the migrate flow: gate the statement type, classify and route it -// exactly as dry-run would, execute the routed SQL — the planner's safer -// native sequence by default when the submitted form blocks — and end in -// exactly one verdict. Refusal verdicts are printed to out and returned as -// verdict.ErrRefused so the entry point maps them to the refusal exit code; -// an execution failure prints a failed verdict — the stable executor code -// plus the committed prefix — and still returns the operational error. -// --dry-run diverts to the classify-and-route plan instead. +// run is the migrate flow: parse the statement, gate its type before +// dialing, and hand it to the engine's imperative pipeline (pkg/migrate), +// which classifies, routes, executes, and ends in exactly one verdict. +// Refusal verdicts are printed to out and returned as verdict.ErrRefused so +// the entry point maps them to the refusal exit code; an execution failure +// prints a failed verdict — the stable executor code plus the committed +// prefix — and still returns the operational error. --dry-run diverts to +// the classify-and-route plan instead. func (c *MigrateCmd) run(ctx context.Context, out io.Writer) error { if c.DryRun { return c.runDryRun(ctx, out) @@ -38,8 +31,11 @@ func (c *MigrateCmd) run(ctx context.Context, out io.Writer) error { return err } logger.Debug("statement parsed", "kind", st.Kind(), "schema", st.Schema(), "table", st.Table()) - if v, refused := gateVerdict(st); refused { - return c.emit(out, v) + // Gate before dialing so an unsupported statement kind refuses without + // a database connection. Run re-checks the gate — this early check is + // an ordering choice, not the safety boundary. + if v, refused := migrate.Gate(st); refused { + return c.emit(out, cliSaferIdiom(st, v)) } pool, err := dbconn.NewPool(ctx, c.Config()) @@ -48,351 +44,54 @@ func (c *MigrateCmd) run(ctx context.Context, out io.Writer) error { } defer pool.Close() - st, err = resolveTarget(ctx, pool, st, logger) - if err != nil { - return err - } - if c.Force != "" { - if err := c.checkForceAck(st); err != nil { - return err - } - } - - facts, _, _, err := dryRunFacts(ctx, pool, st) - if err != nil { - return err - } - canonical, err := statement.Canonical(st.SQL()) - if err != nil { - return err - } - classified, err := planner.Classify(canonical, facts) - if err != nil { - return err - } - routed := router.Route([]planner.Plan{classified}) - rs := routed.Statements[0] - logger.Debug("statement routed", - "route", string(classified.Route), "disposition", string(rs.Disposition)) - - switch rs.Disposition { - case router.DispositionExecute: - execSQL := rs.ExecSQL - substituted := len(execSQL) != 1 || execSQL[0] != rs.Statement - forced := substituted && c.Force != "" - if forced { - // The acknowledged override: run the submitted form as a - // blind bounded attempt instead of the safer sequence. - execSQL, substituted = []string{canonical}, false - c.auditForce(st, rs) - } - return c.execute(ctx, out, pool, st, execSQL, rs.Plan, substituted, forced, logger) - case router.DispositionRewriteRequired: - if c.Force == "" { - return c.emit(out, rewriteRequiredVerdict(st)) - } - c.auditForce(st, rs) - return c.execute(ctx, out, pool, st, []string{canonical}, rs.Plan, false, true, logger) - case router.DispositionUnavailable: - if c.Force == "" { - return c.emit(out, backendUnavailableVerdict(st, rs)) - } - c.auditForce(st, rs) - return c.execute(ctx, out, pool, st, []string{canonical}, rs.Plan, false, true, logger) - case router.DispositionRefuse: - // A planner refusal means no known safe path — there is nothing - // bounded to acknowledge, so --force does not apply. - return c.emit(out, routeRefusalVerdict(st, rs)) - default: - // A disposition this build does not know is a router/CLI version - // skew; refuse to act rather than guess. - return fmt.Errorf("unknown disposition %q", rs.Disposition) - } -} - -// resolveTarget qualifies an unqualified statement against the session's -// search_path exactly once and re-emits it in schema-qualified form, so -// every later stage — facts, classification, the planner's safer sequences, -// preflight, and the executor — names the same relation regardless of any -// session's search_path. The executor's own unqualified-table refusals stay: -// this is the CLI resolving its user's intent, not the executor trusting a -// name. Statements already qualified (or without a table target) pass -// through unchanged. -func resolveTarget(ctx context.Context, pool *pgxpool.Pool, st statement.Statement, - logger *slog.Logger) (statement.Statement, error) { - if st.Schema() != "" || st.Table() == "" { - return st, nil - } - // Re-emitting goes through the deparser, which drops comments; refuse - // commented input instead of silently discarding content. - if err := statement.CheckNoComments(st.SQL()); err != nil { - return statement.Statement{}, err - } - const q = ` - SELECT n.nspname - FROM pg_class c - JOIN pg_namespace n ON n.oid = c.relnamespace - WHERE c.oid = to_regclass(quote_ident($1))` - var schema string - err := pool.QueryRow(ctx, q, st.Table()).Scan(&schema) - if errors.Is(err, pgx.ErrNoRows) { - return statement.Statement{}, fmt.Errorf("%w: %s is not visible on the session search_path", - preflight.ErrTableNotFound, st.Table()) - } - if err != nil { - return statement.Statement{}, fmt.Errorf("resolve %s against search_path: %w", st.Table(), err) - } - sql, err := statement.Qualify(st.SQL(), schema) - if err != nil { - return statement.Statement{}, fmt.Errorf("qualify %s as %s.%s: %w", st.Table(), schema, st.Table(), err) - } - logger.Debug("unqualified table resolved", "table", st.Table(), "schema", schema) - return statement.ParseOne(sql) -} - -// checkForceAck validates the --force acknowledgement: it must name the -// resolved schema-qualified target table exactly, proving the operator -// names the relation whose lock they are accepting. A mismatch is a usage -// error — nothing has executed. -func (c *MigrateCmd) checkForceAck(st statement.Statement) error { - if c.Force == qualified(st) { - return nil - } - return fmt.Errorf("--force must acknowledge the resolved target table %q, got %q; nothing was executed", - qualified(st), c.Force) -} - -// auditForce records the override decision before anything executes: the -// operator chose the submitted form over the engine's routing. The record -// is warn-level and unconditional — an audit trail must not depend on -// --debug — and the verdict's Forced field is its machine-readable twin. -func (c *MigrateCmd) auditForce(st statement.Statement, rs router.Statement) { - c.audit().Warn("forced execution of submitted form", - "table", qualified(st), - "kind", st.Kind().String(), - "disposition", string(rs.Disposition)) -} - -// execute runs execSQL through the sequence executor: the planner's safer -// sequence when one was substituted, otherwise the submitted form. A forced -// run bypasses the sequence executor's shape admission — the acknowledged -// override runs the submitted form as one blind bounded attempt, whatever -// its kind — so it goes through the optimistic executor directly, under the -// same brief budgets. Blind attempts of the submitted form — including -// forced ones — are size-guarded; substituted sequences and planner-proven -// online idioms are not — long work on large tables is their purpose, and -// every brief step is still budget-bounded. Before anything runs, the -// connected role is checked at the tier the routed steps actually need -// (engine-role contract), so a role that would die mid-change is refused -// with the exact provisioning statement instead. -func (c *MigrateCmd) execute(ctx context.Context, out io.Writer, pool *pgxpool.Pool, - st statement.Statement, execSQL []string, plan planner.Plan, - substituted, forced bool, logger *slog.Logger) error { - limit := int64(c.MaxTableSize) - if !sizeGuardApplies(plan, substituted) { - limit = preflight.NoSizeLimit - } - pt, err := preflight.CheckTable(ctx, pool, st.Schema(), st.Table(), limit) - var sizeErr *preflight.SizeError - if errors.As(err, &sizeErr) { - return c.emit(out, sizeGuardVerdict(st, sizeErr, forced)) - } - if err != nil { - return err - } - serverMajor, err := dbconn.ServerMajor(ctx, pool) - if err != nil { - return err - } - if err := preflight.CheckPartitionSupport(pt, serverMajor, execSQL); err != nil { - var partitionErr *preflight.UnsupportedPartitionedParentError - if errors.As(err, &partitionErr) { - return c.emit(out, partitionedParentVerdict(st, partitionErr, forced)) - } - return err - } - tier, err := preflight.RequiredTier(execSQL) - if err != nil { - return err - } - priv, err := preflight.CheckPrivileges(ctx, pool, st.Schema(), st.Table(), - preflight.Requirement{Tier: tier}) - var privErr *preflight.PrivilegeError - if errors.As(err, &privErr) { - return c.emit(out, privilegeVerdict(st, privErr, forced)) - } - if err != nil { - return err - } - logger.Debug("privilege preflight passed", - "role", priv.Role(), "owner", priv.Owner(), "tier", tier.String()) - logger.Debug("preflight passed", - "table", qualified(st), "total_bytes", pt.TotalBytes(), "limit_bytes", limit) - if substituted { - logger.Debug("substituting safer native sequence", - "table", qualified(st), "steps", len(execSQL)) - } - - budget := executor.SequenceBudget{ - Brief: executor.Budget{LockTimeout: c.LockTimeout, StatementTimeout: c.StatementTimeout}, - Concurrent: executor.ConcurrentBudget{Overall: c.IndexBuildTimeout}, - Validate: executor.ValidateBudget{LockTimeout: c.LockTimeout, Overall: c.ValidateTimeout}, - } - retry := c.retryPolicy() - start := time.Now() - var rep executor.SequenceReport - if forced { - err = executor.ExecuteNative(ctx, pool, pt, st, budget.Brief, retry) - } else { - rep, err = executor.RunSequence(ctx, pool, pt, execSQL, budget, retry) - } - elapsed := time.Since(start) - if v, refused := execRefusal(st, err, substituted, forced, onlineIdiomPlan(plan)); refused { - logger.Debug("execution refused", - "reason", string(v.Reason), "cause", string(v.Cause), "attempts", v.Attempts, "elapsed", elapsed) - return c.emit(out, v) - } - if err != nil { - // Everything else is an operational failure, not a refusal: the - // typed *SequenceStepError names the failed step and the committed - // prefix that remains, and an *InvalidIndexError carries the - // operator recovery guidance. The failed verdict is the error's - // machine-readable twin on stdout — the stable executor code, the - // failed step, and the committed prefix — while the error itself - // still returns, so the process exits 1, not the refusal code. - v := failureVerdict(st, err, rep, forced) - logger.Debug("execution failed", - "code", v.Code, "failed_step", v.FailedStep, "committed_steps", len(v.ExecutedSQL), "elapsed", elapsed) - if emitErr := c.emit(out, v); emitErr != nil { - return emitErr + v, runErr := migrate.Run(ctx, pool, st, c.options(logger)) + if runErr != nil { + // The failed verdict is the error's machine-readable twin on + // stdout, while the error itself still returns so the process + // exits 1, not the refusal code. An error without a verdict means + // the pipeline stopped before reaching one. + if v.Outcome == verdict.OutcomeFailed { + if emitErr := c.emit(out, v); emitErr != nil { + return emitErr + } } - return fmt.Errorf("run schema change on %s: %w", qualified(st), err) - } - logger.Debug("schema change committed", - "table", qualified(st), "steps", len(execSQL), "elapsed", elapsed) - - v := verdict.Verdict{ - Outcome: verdict.OutcomeExecuted, - Statement: st.SQL(), - Table: qualified(st), - Forced: forced, - Detail: fmt.Sprintf("committed within budgets (lock %s, statement %s): the change was effectively instant", - c.LockTimeout, c.StatementTimeout), - } - if substituted { - v.ExecutedSQL = execSQL - v.Detail = fmt.Sprintf("the submitted form blocks; pg-sprite ran the safer native sequence instead — all %d steps committed", - len(execSQL)) - } - if forced { - v.Detail = fmt.Sprintf("forced: the submitted form ran as-is under budgets (lock %s, statement %s), overriding the engine's routing", - c.LockTimeout, c.StatementTimeout) + return runErr } return c.emit(out, v) } -// failureVerdict maps an operational execution failure to its failed -// verdict: the executor's stable outcome code, and for a mid-sequence -// failure the failed step and the committed prefix whose state remains. -// It is the machine-readable twin of the returned error — automation -// branches on Code and ExecutedSQL instead of parsing stderr prose. A -// single bounded attempt (the submitted form, forced or not) rolls back on -// failure, so it carries no step and an empty committed prefix. -func failureVerdict(st statement.Statement, err error, - rep executor.SequenceReport, forced bool) verdict.Verdict { - v := verdict.Verdict{ - Outcome: verdict.OutcomeFailed, - Code: string(executor.OutcomeCode(err)), - Statement: st.SQL(), - Table: qualified(st), - Forced: forced, - Detail: "execution failed; nothing committed — a started bounded attempt rolls back", - } - var stepErr *executor.SequenceStepError - if !errors.As(err, &stepErr) { - return v - } - v.FailedStep = stepErr.Step - v.FailedStepSQL = stepErr.SQL - for _, s := range rep.Steps { - v.ExecutedSQL = append(v.ExecutedSQL, s.SQL) - } - if len(v.ExecutedSQL) > 0 { - v.Detail = fmt.Sprintf("sequence step %d of %d failed; the %d committed steps' state remains — the planner sequence's partial-failure contract says how a retry resumes", - stepErr.Step, stepErr.Total, len(v.ExecutedSQL)) - } else { - v.Detail = fmt.Sprintf("sequence step %d of %d failed; no earlier steps had committed — Code names the outcome and any state the failed step itself left", - stepErr.Step, stepErr.Total) +// options is the engine policy this command wires from its flags. With no +// flags set it must equal [migrate.DefaultOptions] field for field — the +// defaults test pins the two together so the CLI's flag defaults and the +// library's sanctioned starting point cannot drift. The audit logger is +// wired here, always on: the library discards a nil Audit, and a +// deliberate safety override on this front door must be visible even +// without --debug. +func (c *MigrateCmd) options(logger *slog.Logger) migrate.Options { + return migrate.Options{ + Force: c.Force, + MaxTableSizeBytes: int64(c.MaxTableSize), + Budget: executor.SequenceBudget{ + Brief: executor.Budget{LockTimeout: c.LockTimeout, StatementTimeout: c.StatementTimeout}, + Concurrent: executor.ConcurrentBudget{Overall: c.IndexBuildTimeout}, + Validate: executor.ValidateBudget{LockTimeout: c.LockTimeout, Overall: c.ValidateTimeout}, + }, + Retry: c.retryPolicy(), + Logger: logger, + Audit: c.audit(), + } +} + +// cliSaferIdiom attaches this front door's actionable spelling to a gate +// refusal whose safer path is the declarative front door: the library +// names the concept, each front door owns its own syntax. +func cliSaferIdiom(st statement.Statement, v verdict.Verdict) verdict.Verdict { + if st.Kind() == statement.KindCreateTable { + v.SaferIdiom = "pg-sprite diff --desired schema.sql" } return v } -// execRefusal maps an execution failure to its refusal verdict, when the -// failure belongs to the refusal contract rather than the operational-error -// exit: a static admission refusal (decided before anything executed), or a -// budget cancellation of a non-substituted attempt. An *InvalidIndexError -// is never a refusal, even when a budget cancellation is buried inside it — -// invalid-index debris is the one outcome that needs an operator, and its -// typed error carries the recovery guidance a budget verdict would conceal. -func execRefusal(st statement.Statement, err error, - substituted, forced, online bool) (verdict.Verdict, bool) { - if err == nil { - return verdict.Verdict{}, false - } - if isAdmissionRefusal(err) { - return admissionRefusalVerdict(st, err, forced), true - } - var invalidErr *executor.InvalidIndexError - if errors.As(err, &invalidErr) { - return verdict.Verdict{}, false - } - var budgetErr *executor.BudgetError - if !substituted && errors.As(err, &budgetErr) { - // The blind attempt of the submitted form exceeded a budget and was - // cancelled without committing — the Phase 1 refusal contract. A - // forced attempt is bounded by the same budgets: --force overrides - // routing, never the executor's protections. - return budgetVerdict(st, budgetErr, forced, online), true - } - return verdict.Verdict{}, false -} - -// isAdmissionRefusal reports whether err is one of the executor's static -// admission refusals: decided from the statement's shape before anything -// executes, so it maps to a refusal verdict, not an operational error. A -// *SequenceStepError wrapper means execution started, which is never an -// admission refusal. -func isAdmissionRefusal(err error) bool { - var stepErr *executor.SequenceStepError - if errors.As(err, &stepErr) { - return false - } - return errors.Is(err, executor.ErrUnsupportedSequenceStep) || - errors.Is(err, executor.ErrUnnamedIndex) || - errors.Is(err, executor.ErrIfNotExistsUnsupported) -} - -// onlineIdiomPlan reports whether the plan proved every operation an online -// idiom (CONCURRENTLY, NOT VALID, VALIDATE): the submitted form already is -// the safe pattern, and running long on a large table is its purpose. -func onlineIdiomPlan(p planner.Plan) bool { - for _, d := range p.Decisions { - if d.Reason != planner.ReasonOnlineIdiom { - return false - } - } - return true -} - -// sizeGuardApplies reports whether the size guard protects this run. It -// guards exactly the blind attempt of the submitted form: when the engine -// substituted the planner's safer sequence, or when the plan proved every -// operation an online idiom, long work on a large table is the pattern's -// purpose and the guard would refuse the very tables the pattern serves. -func sizeGuardApplies(p planner.Plan, substituted bool) bool { - return !substituted && !onlineIdiomPlan(p) -} - func (c *MigrateCmd) retryPolicy() executor.RetryPolicy { // Programmatic callers do not pass through Kong's default population. // Preserve the safe defaults for a zero-valued command while rejecting @@ -423,216 +122,3 @@ func (c *MigrateCmd) emit(out io.Writer, v verdict.Verdict) error { } return nil } - -// gateVerdict is the statement-type gate: ALTER TABLE and CREATE INDEX -// proceed to classification (a blocking CREATE INDEX is substituted with -// its concurrent build, a submitted concurrent build is driven directly); -// the index-maintenance forms the executor cannot drive yet are pointed at -// their concurrent idiom, everything else is unsupported. Refused -// statements are never executed. -func gateVerdict(st statement.Statement) (verdict.Verdict, bool) { - v := verdict.Verdict{Outcome: verdict.OutcomeRefused, Statement: st.SQL()} - switch st.Kind() { - case statement.KindAlterTable, statement.KindCreateIndex: - return verdict.Verdict{}, false - case statement.KindDropIndex, statement.KindReindex: - v.Reason = verdict.ReasonIndexStatement - v.Detail, v.SaferIdiom = indexAdvice(st) - case statement.KindCreateTable: - v.Reason = verdict.ReasonUnsupportedStatement - v.Detail = "migrate changes an existing table; to converge a table onto a desired-state CREATE TABLE, use the declarative front-end" - v.SaferIdiom = "pg-sprite diff --desired schema.sql" - case statement.KindOther: - v.Reason = verdict.ReasonUnsupportedStatement - v.Detail = "only ALTER TABLE and CREATE INDEX statements are supported by the imperative front door" - } - return v, true -} - -// indexAdvice explains an index-statement refusal for the maintenance forms -// the executor does not drive (DROP INDEX, REINDEX). The already-concurrent -// forms carry no safer idiom: suggesting the statement the user submitted -// would confuse a human once and send a resubmitting automation into a loop. -func indexAdvice(st statement.Statement) (detail, saferIdiom string) { - if st.Concurrent() { - return "this is already the safe concurrent idiom; pg-sprite does not drive this maintenance form yet — run it directly against the database", "" - } - switch st.Kind() { - case statement.KindDropIndex: - return "a plain DROP INDEX takes ACCESS EXCLUSIVE on the table; the concurrent drop does not", "DROP INDEX CONCURRENTLY" - case statement.KindReindex: - return "a plain REINDEX blocks writes; the concurrent rebuild does not", "REINDEX ... CONCURRENTLY" - default: - return "", "" - } -} - -// privilegeVerdict is the refusal for a connected role that lacks the access -// the routed change needs. The error already names the failed catalog check -// and the exact provisioning statement, so it is the detail verbatim. A -// refused forced attempt still records the override: the operator asked for -// the submitted form and the role could not run it. -func privilegeVerdict(st statement.Statement, privErr *preflight.PrivilegeError, forced bool) verdict.Verdict { - return verdict.Verdict{ - Outcome: verdict.OutcomeRefused, - Reason: verdict.ReasonInsufficientPrivileges, - Statement: st.SQL(), - Table: qualified(st), - Forced: forced, - Detail: privErr.Error(), - } -} - -// partitionedParentVerdict refuses unsupported execution steps on a -// partitioned parent before the sequence executor runs anything. -func partitionedParentVerdict(st statement.Statement, partitionErr *preflight.UnsupportedPartitionedParentError, - forced bool) verdict.Verdict { - return verdict.Verdict{ - Outcome: verdict.OutcomeRefused, - Reason: verdict.ReasonUnsupportedPartitionedParent, - Statement: st.SQL(), - Table: qualified(st), - Forced: forced, - Detail: partitionErr.Error(), - } -} - -// rewriteRequiredVerdict is the refusal for a statement whose submitted -// form blocks but for which the planner could not construct the safer -// native sequence — a multi-operation statement, or a pattern it cannot -// build. Running the submitted form would falsify the plan's own reason. -func rewriteRequiredVerdict(st statement.Statement) verdict.Verdict { - return verdict.Verdict{ - Outcome: verdict.OutcomeRefused, - Reason: verdict.ReasonRewriteRequired, - Statement: st.SQL(), - Table: qualified(st), - Detail: "the submitted form blocks and must run as a safer native sequence, but pg-sprite could not " + - "construct one for this statement; submit each operation as its own single-operation statement " + - "so the engine can build its safer form (run with --dry-run to see each operation's classification)", - } -} - -// backendUnavailableVerdict is the refusal for a change that routes to an -// execution strategy this build does not implement. -func backendUnavailableVerdict(st statement.Statement, rs router.Statement) verdict.Verdict { - return verdict.Verdict{ - Outcome: verdict.OutcomeRefused, - Reason: verdict.ReasonBackendUnavailable, - Statement: st.SQL(), - Table: qualified(st), - Detail: fmt.Sprintf("the change requires the %s strategy, which this build does not implement yet: "+ - "PostgreSQL would rewrite the table under ACCESS EXCLUSIVE for the whole operation", rs.Backend), - } -} - -// routeRefusalVerdict is the refusal for a statement the planner refused: -// it carries the refused operations by name so the operator knows which -// part of the statement the engine does not know a safe path for. -func routeRefusalVerdict(st statement.Statement, rs router.Statement) verdict.Verdict { - v := verdict.Verdict{ - Outcome: verdict.OutcomeRefused, - Reason: verdict.ReasonUnsupportedStatement, - Statement: st.SQL(), - Table: qualified(st), - Detail: "the planner knows no safe path for this statement", - } - for _, d := range rs.Decisions { - if d.Route == planner.RouteRefuse { - v.Detail = fmt.Sprintf("the planner knows no safe path for %s", d.Operation) - break - } - } - return v -} - -// sizeGuardVerdict is the refusal for tables above the size threshold, where -// even a budget-bounded attempt would visibly stall the table. A refused -// forced attempt still records the override: the operator asked for the -// submitted form and the guard said no. -func sizeGuardVerdict(st statement.Statement, sizeErr *preflight.SizeError, forced bool) verdict.Verdict { - return verdict.Verdict{ - Outcome: verdict.OutcomeRefused, - Reason: verdict.ReasonTableTooLarge, - Statement: st.SQL(), - Table: qualified(st), - Forced: forced, - Detail: fmt.Sprintf("table is %d bytes on disk (heap, indexes, and TOAST), above the %d-byte "+ - "--max-table-size threshold. pg-sprite cannot yet prove this change is instant on a table this "+ - "size; if it requires a rewrite, a cancelled attempt is not a free probe — it would hold "+ - "ACCESS EXCLUSIVE doing rewrite work for the whole budget", - sizeErr.TotalBytes, sizeErr.LimitBytes), - } -} - -// admissionRefusalVerdict is the refusal for a statement the gate admits but -// the executor's static admission refuses before anything executes: an -// unnamed index build, IF NOT EXISTS on a concurrent build, or a substituted -// step shape the sequence executor does not drive yet (DETACH PARTITION -// CONCURRENTLY). The typed error carries the explanation. -func admissionRefusalVerdict(st statement.Statement, err error, forced bool) verdict.Verdict { - return verdict.Verdict{ - Outcome: verdict.OutcomeRefused, - Reason: verdict.ReasonUnsupportedStatement, - Statement: st.SQL(), - Table: qualified(st), - Forced: forced, - Detail: fmt.Sprintf("the engine cannot run this statement safely: %v; nothing was executed", err), - } -} - -// budgetVerdict is the refusal for an attempt that exceeded its lock or -// statement budget and was cancelled without executing. online tailors the -// statement-budget advice: a submitted form the plan proved an online idiom -// (a concurrent build, a lone VALIDATE) needs a larger budget, not a -// different strategy, while a blind attempt that ran past its budget is -// doing rewrite work. A refused forced attempt still records the override. -func budgetVerdict(st statement.Statement, budgetErr *executor.BudgetError, forced, online bool) verdict.Verdict { - v := verdict.Verdict{ - Outcome: verdict.OutcomeRefused, - Reason: verdict.ReasonBudgetExceeded, - Statement: st.SQL(), - Table: qualified(st), - Forced: forced, - } - switch budgetErr.Cause { - case executor.CauseLock: - v.Cause = verdict.CauseLockBudget - v.Attempts = budgetErr.Attempts - if budgetErr.Attempts > 1 { - v.Detail = fmt.Sprintf("the lock was not granted within the %s lock budget on any of %d bounded "+ - "attempts: the table is too contended for a blind attempt; nothing was executed", - budgetErr.Budget, budgetErr.Attempts) - break - } - v.Detail = fmt.Sprintf("the lock was not granted within the %s lock budget: the table is too "+ - "contended right now; nothing was executed", budgetErr.Budget) - case executor.CauseStatement: - v.Cause = verdict.CauseStatementBudget - if online { - v.Detail = fmt.Sprintf("cancelled after the %s budget: the statement already is the safe online "+ - "idiom — the work needs more time, not a different strategy; retry with a larger budget for "+ - "this step class", budgetErr.Budget) - } else { - v.Detail = fmt.Sprintf("cancelled after the %s statement budget: the change does real rewrite work, "+ - "not an in-place catalog change, and needs a copy-and-swap rewrite that pg-sprite does not perform yet. "+ - "If it adds a constraint, ADD CONSTRAINT ... NOT VALID followed by VALIDATE CONSTRAINT avoids the long lock", - budgetErr.Budget) - } - default: - v.Detail = budgetErr.Error() - } - return v -} - -// qualified renders the statement's target table for the verdict, empty when -// the statement has none. -func qualified(st statement.Statement) string { - if st.Table() == "" { - return "" - } - if st.Schema() == "" { - return st.Table() - } - return st.Schema() + "." + st.Table() -} diff --git a/internal/cli/migrate_test.go b/internal/cli/migrate_test.go index 7d8f738..3f626be 100644 --- a/internal/cli/migrate_test.go +++ b/internal/cli/migrate_test.go @@ -1,7 +1,6 @@ package cli import ( - "fmt" "testing" "time" @@ -10,8 +9,7 @@ import ( "github.com/stretchr/testify/require" "github.com/block/pg-sprite/pkg/executor" - "github.com/block/pg-sprite/pkg/statement" - "github.com/block/pg-sprite/pkg/verdict" + "github.com/block/pg-sprite/pkg/migrate" ) // parseMigrate runs args through the real command grammar so these tests @@ -63,6 +61,20 @@ func TestRetryFlagsWireIntoRetryPolicy(t *testing.T) { }, c.retryPolicy()) } +// The library's sanctioned starting point and this front door's flag +// defaults are one policy by contract: an embedder starting from +// migrate.DefaultOptions and an operator running the CLI with no flags get +// identical budgets, size guard, and retry. Full-struct equality on each +// policy field, so a drift on either side fails here. +func TestMigrateDefaultsMatchLibraryDefaults(t *testing.T) { + c := parseMigrate(t) + got := c.options(nil) + want := migrate.DefaultOptions() + assert.Equal(t, want.MaxTableSizeBytes, got.MaxTableSizeBytes) + assert.Equal(t, want.Budget, got.Budget) + assert.Equal(t, want.Retry, got.Retry) +} + func TestRetryPolicyDefaults(t *testing.T) { t.Run("kong defaults match the executor defaults", func(t *testing.T) { c := parseMigrate(t) @@ -74,161 +86,3 @@ func TestRetryPolicyDefaults(t *testing.T) { assert.Equal(t, executor.DefaultRetryPolicy(), c.retryPolicy()) }) } - -func TestBudgetVerdict(t *testing.T) { - st, err := statement.ParseOne("ALTER TABLE billing.invoices ALTER COLUMN id TYPE bigint") - require.NoError(t, err) - - t.Run("lock budget", func(t *testing.T) { - v := budgetVerdict(st, &executor.BudgetError{Cause: executor.CauseLock, Budget: 3 * time.Second, Attempts: 3}, false, false) - assert.Equal(t, verdict.OutcomeRefused, v.Outcome) - assert.Equal(t, verdict.ReasonBudgetExceeded, v.Reason) - assert.Equal(t, verdict.CauseLockBudget, v.Cause) - assert.Equal(t, 3, v.Attempts, "the exhausted attempt count must reach the verdict") - assert.Equal(t, "billing.invoices", v.Table) - assert.False(t, v.Forced) - assert.NotEmpty(t, v.Detail) - }) - - t.Run("statement budget", func(t *testing.T) { - v := budgetVerdict(st, &executor.BudgetError{Cause: executor.CauseStatement, Budget: 30 * time.Second}, false, false) - assert.Equal(t, verdict.ReasonBudgetExceeded, v.Reason) - assert.Equal(t, verdict.CauseStatementBudget, v.Cause) - assert.NotEmpty(t, v.Detail) - }) - - t.Run("statement budget on the online idiom advises a larger budget", func(t *testing.T) { - blind := budgetVerdict(st, &executor.BudgetError{Cause: executor.CauseStatement, Budget: time.Minute}, false, false) - online := budgetVerdict(st, &executor.BudgetError{Cause: executor.CauseStatement, Budget: time.Minute}, false, true) - assert.Equal(t, verdict.CauseStatementBudget, online.Cause) - assert.NotEqual(t, blind.Detail, online.Detail, - "a cancelled online idiom needs a larger budget, not the copy-and-swap advice") - }) - - t.Run("a forced refusal records the override", func(t *testing.T) { - v := budgetVerdict(st, &executor.BudgetError{Cause: executor.CauseStatement, Budget: time.Minute}, true, false) - assert.True(t, v.Forced, "the machine-readable audit record must survive a refusal") - }) - - t.Run("unknown cause falls back to the error text", func(t *testing.T) { - budgetErr := &executor.BudgetError{Budget: time.Second} - v := budgetVerdict(st, budgetErr, false, false) - assert.Equal(t, verdict.ReasonBudgetExceeded, v.Reason) - assert.Equal(t, verdict.CauseNone, v.Cause) - assert.Equal(t, budgetErr.Error(), v.Detail) - }) -} - -func TestFailureVerdict(t *testing.T) { - st, err := statement.ParseOne("ALTER TABLE billing.invoices ALTER COLUMN status SET NOT NULL") - require.NoError(t, err) - - t.Run("a mid-sequence failure discloses the step and the committed prefix", func(t *testing.T) { - stepErr := &executor.SequenceStepError{ - Step: 2, - Total: 4, - Kind: executor.StepValidateConstraint, - SQL: "ALTER TABLE billing.invoices VALIDATE CONSTRAINT c", - Err: fmt.Errorf("server error"), - } - rep := executor.SequenceReport{Steps: []executor.StepReport{ - {SQL: "ALTER TABLE billing.invoices ADD CONSTRAINT c CHECK (status IS NOT NULL) NOT VALID", Kind: executor.StepBrief}, - }} - v := failureVerdict(st, fmt.Errorf("wrapped: %w", stepErr), rep, false) - assert.Equal(t, verdict.OutcomeFailed, v.Outcome) - assert.Equal(t, string(executor.CodeExecutionFailed), v.Code) - assert.Equal(t, 2, v.FailedStep) - assert.Equal(t, stepErr.SQL, v.FailedStepSQL) - assert.Equal(t, []string{rep.Steps[0].SQL}, v.ExecutedSQL, - "the committed prefix is what distinguishes partial state from nothing happened") - assert.Equal(t, "billing.invoices", v.Table) - assert.False(t, v.Forced) - }) - - t.Run("the failed step's typed cause maps to its own stable code", func(t *testing.T) { - stepErr := &executor.SequenceStepError{ - Step: 2, Total: 4, Kind: executor.StepValidateConstraint, - SQL: "ALTER TABLE billing.invoices VALIDATE CONSTRAINT c", - Err: &executor.BudgetError{Cause: executor.CauseStatement, Budget: time.Minute}, - } - v := failureVerdict(st, stepErr, executor.SequenceReport{}, false) - assert.Equal(t, string(executor.CodeBudgetStatementExceeded), v.Code) - assert.Empty(t, v.ExecutedSQL, "an empty committed prefix means nothing committed") - }) - - t.Run("a non-sequence failure carries no step and an empty prefix", func(t *testing.T) { - v := failureVerdict(st, fmt.Errorf("server error"), executor.SequenceReport{}, true) - assert.Equal(t, verdict.OutcomeFailed, v.Outcome) - assert.Equal(t, string(executor.CodeExecutionFailed), v.Code) - assert.Zero(t, v.FailedStep) - assert.Empty(t, v.FailedStepSQL) - assert.Empty(t, v.ExecutedSQL) - assert.True(t, v.Forced, "the machine-readable audit record must survive a failure") - }) -} - -func TestExecRefusal(t *testing.T) { - st, err := statement.ParseOne("CREATE INDEX CONCURRENTLY i ON billing.invoices (customer_id)") - require.NoError(t, err) - - t.Run("nil error is not a refusal", func(t *testing.T) { - _, refused := execRefusal(st, nil, false, false, false) - assert.False(t, refused) - }) - - t.Run("a budget-cancelled attempt is a refusal", func(t *testing.T) { - budgetErr := &executor.BudgetError{Cause: executor.CauseStatement, Budget: time.Minute} - v, refused := execRefusal(st, fmt.Errorf("wrapped: %w", budgetErr), false, true, true) - require.True(t, refused) - assert.Equal(t, verdict.ReasonBudgetExceeded, v.Reason) - assert.Equal(t, verdict.CauseStatementBudget, v.Cause) - assert.True(t, v.Forced) - }) - - t.Run("a substituted sequence's budget failure is operational", func(t *testing.T) { - budgetErr := &executor.BudgetError{Cause: executor.CauseStatement, Budget: time.Minute} - _, refused := execRefusal(st, fmt.Errorf("wrapped: %w", budgetErr), true, false, false) - assert.False(t, refused, "a failed substituted step is an operational failure with a committed prefix") - }) - - t.Run("invalid-index debris is never a budget refusal", func(t *testing.T) { - // The exact chain a budget-cancelled concurrent build that left an - // invalid index produces: the buried *BudgetError must not map to a - // budget verdict that conceals the operator-recovery outcome. - invalid := &executor.InvalidIndexError{ - Schema: "billing", - Index: "i", - Build: &executor.BudgetError{Cause: executor.CauseStatement, Budget: time.Minute}, - Cleanup: executor.ErrBuildLeftInvalidIndex, - } - stepErr := &executor.SequenceStepError{Step: 1, Total: 1, Err: invalid} - _, refused := execRefusal(st, stepErr, false, false, true) - assert.False(t, refused, "invalid-index debris needs an operator, not a budget verdict") - }) - - t.Run("static admission refusals map to a typed refusal verdict", func(t *testing.T) { - for name, admissionErr := range map[string]error{ - "unsupported step": fmt.Errorf("sequence step 1 of 1: blocking CREATE INDEX: %w", executor.ErrUnsupportedSequenceStep), - "unnamed index": fmt.Errorf("sequence step 1 of 1: %w", executor.ErrUnnamedIndex), - "if not exists": fmt.Errorf("sequence step 1 of 1: %w", executor.ErrIfNotExistsUnsupported), - } { - t.Run(name, func(t *testing.T) { - v, refused := execRefusal(st, admissionErr, true, false, false) - require.True(t, refused, "an admission refusal decided before execution is a refusal verdict") - assert.Equal(t, verdict.ReasonUnsupportedStatement, v.Reason) - assert.NotEmpty(t, v.Detail) - }) - } - }) - - t.Run("an admission sentinel inside a step failure stays operational", func(t *testing.T) { - stepErr := &executor.SequenceStepError{Step: 2, Total: 3, Err: executor.ErrUnnamedIndex} - _, refused := execRefusal(st, stepErr, true, false, false) - assert.False(t, refused, "a step failure means execution started; the committed prefix must surface") - }) - - t.Run("an operational server error is not a refusal", func(t *testing.T) { - _, refused := execRefusal(st, fmt.Errorf("connection reset"), false, false, false) - assert.False(t, refused) - }) -} diff --git a/internal/cli/verdict_text.go b/internal/cli/verdict_text.go index fa61423..0cc2339 100644 --- a/internal/cli/verdict_text.go +++ b/internal/cli/verdict_text.go @@ -31,7 +31,7 @@ func writeVerdictText(out io.Writer, pal palette, v verdict.Verdict) error { fmt.Fprintf(&b, "\n %s %s", pal.bold("safer:"), v.SaferIdiom) } if v.Forced { - fmt.Fprintf(&b, "\n %s the submitted form ran as-is (--force)", pal.bold("forced:")) + fmt.Fprintf(&b, "\n %s the submitted form ran as-is (force acknowledged)", pal.bold("forced:")) } if v.FailedStep > 0 { fmt.Fprintf(&b, "\n %s step %d: %s", pal.bold("failed at:"), v.FailedStep, v.FailedStepSQL) diff --git a/pkg/migrate/example_test.go b/pkg/migrate/example_test.go new file mode 100644 index 0000000..6a2c67b --- /dev/null +++ b/pkg/migrate/example_test.go @@ -0,0 +1,68 @@ +package migrate_test + +import ( + "context" + "fmt" + "log" + + "github.com/block/pg-sprite/pkg/dbconn" + "github.com/block/pg-sprite/pkg/migrate" + "github.com/block/pg-sprite/pkg/statement" + "github.com/block/pg-sprite/pkg/verdict" +) + +// Example_run is the full library flow an orchestrator embeds: parse the +// statement, gate it before dialing, connect, and drive the imperative +// pipeline to exactly one verdict. It is compile-checked but not executed — +// Run needs a live PostgreSQL database. +func Example_run() { + ctx := context.Background() + + // Parse failures surface here, at the boundary where the embedder can + // render them. + st, err := statement.ParseOne("ALTER TABLE users ALTER COLUMN email SET NOT NULL") + if err != nil { + log.Print(err) + return + } + + // Gate needs no database: an unsupported statement kind refuses before + // dialing. Run re-checks it, so skipping this early gate is safe — + // only slower. + if v, refused := migrate.Gate(st); refused { + fmt.Println(v.Reason, v.Detail) + return + } + + pool, err := dbconn.NewPool(ctx, dbconn.Config{URL: "postgres://engine@localhost:5432/app"}) + if err != nil { + log.Print(err) + return + } + defer pool.Close() + + // The zero Options is not a runnable policy — Run rejects it. + // DefaultOptions is the sanctioned starting point (the CLI's flag + // defaults); tune it per table: a large table needs more generous + // concurrent-build and validate bounds, a hot table a tighter lock + // budget. + opts := migrate.DefaultOptions() + + // The verdict-and-error contract has three shapes: a refusal returns + // the verdict with a nil error; an execution failure returns the + // failed verdict (the stable code and the committed prefix) together + // with the operational error; an error with a zero verdict means the + // pipeline stopped before executing anything. + v, err := migrate.Run(ctx, pool, st, opts) + if err != nil { + log.Print(err) + } + switch v.Outcome { + case verdict.OutcomeExecuted: + fmt.Println(v.ExecutedSQL) + case verdict.OutcomeRefused: + fmt.Println(v.Reason, v.Detail) + case verdict.OutcomeFailed: + fmt.Println(v.Code, v.ExecutedSQL) + } +} diff --git a/pkg/migrate/facts.go b/pkg/migrate/facts.go new file mode 100644 index 0000000..23b1a11 --- /dev/null +++ b/pkg/migrate/facts.go @@ -0,0 +1,71 @@ +package migrate + +import ( + "context" + "errors" + + "github.com/jackc/pgx/v5/pgxpool" + + "github.com/block/pg-sprite/pkg/planner" + "github.com/block/pg-sprite/pkg/preflight" + "github.com/block/pg-sprite/pkg/schemadiff" + "github.com/block/pg-sprite/pkg/statement" +) + +// Facts is one introspection pass over the statement's target table. Both +// [Run] and a dry-run plan classify from one Facts value, so execution and +// its plan describe the same live state. +type Facts struct { + // Classifier feeds [planner.Classify]. Statements without a single + // table target (index drops, REINDEX) and missing tables classify + // with zero facts — a strictly more conservative plan. + Classifier planner.Facts + + // Target carries the preflight facts (partitioning, server major) + // for plan-time partition checks; zero whenever Classifier is. + Target preflight.TargetFacts + + // TableExists mirrors the introspection outcome for a plan report: + // true when the table was found, false when it was looked up and + // missing, nil when the statement has no single table target to + // introspect. + TableExists *bool +} + +// LiveFacts introspects the statement's target table for classifier facts. +func LiveFacts(ctx context.Context, pool *pgxpool.Pool, + st statement.Statement) (Facts, error) { + if st.Table() == "" { + return Facts{}, nil + } + live, err := schemadiff.Introspect(ctx, pool, ResolvedSchema(st), st.Table()) + switch { + case errors.Is(err, schemadiff.ErrTableNotFound): + exists := false + return Facts{TableExists: &exists}, nil + case err != nil: + return Facts{}, err + } + targetFacts, err := preflight.LookupTargetFacts(ctx, pool, ResolvedSchema(st), st.Table()) + if err != nil { + return Facts{}, err + } + exists := true + return Facts{ + Classifier: planner.FactsFrom(live), + Target: targetFacts, + TableExists: &exists, + }, nil +} + +// ResolvedSchema is the schema the engine plans against: the statement's +// qualification, or public — the default the engine introspects — when a +// table-targeted statement leaves it unqualified. A report carries the +// resolved name, never the submitted one: a stored plan must not depend on +// the reader's search_path to say which table it describes. +func ResolvedSchema(st statement.Statement) string { + if st.Schema() == "" && st.Table() != "" { + return "public" + } + return st.Schema() +} diff --git a/pkg/migrate/migrate.go b/pkg/migrate/migrate.go new file mode 100644 index 0000000..fcaa4dc --- /dev/null +++ b/pkg/migrate/migrate.go @@ -0,0 +1,420 @@ +// Package migrate is the imperative front door as a library: one parsed +// statement in — gate, resolve, introspect, classify, route, execute — and +// exactly one verdict out. The CLI's migrate command and orchestrators +// embedding pg-sprite share this one pipeline, so a verdict means the same +// thing no matter which caller produced it. +// +// Callers own the boundary concerns: parse the statement through +// [statement.ParseOne] (a parse failure surfaces at the caller) and build +// the connection through [dbconn.NewPool]. [Gate] is exported so a caller +// can refuse an unsupported statement kind before dialing; [Run] re-checks +// it, so a caller that skips the early gate still cannot execute a gated +// kind. +// +// [Run] takes the concrete [pgxpool.Pool] that [dbconn.NewPool] returns — +// a deliberate concrete dependency, not an oversight: the execution paths +// need the full pool surface (dedicated sessions for concurrent builds, +// per-step transactions), a narrower interface would admit handles those +// paths cannot use, and it is the same handle the declarative front door +// (diffplan.Plan) takes, so the two front doors embed identically. +// +// Before a v1 module tag the Go API carries no compatibility promise: the +// JSON [verdict.Verdict] is the stability boundary, the Go API follows at +// v1 (see docs/architecture.md). +package migrate + +import ( + "context" + "errors" + "fmt" + "log/slog" + "time" + + "github.com/jackc/pgx/v5" + "github.com/jackc/pgx/v5/pgxpool" + + "github.com/block/pg-sprite/pkg/dbconn" + "github.com/block/pg-sprite/pkg/executor" + "github.com/block/pg-sprite/pkg/planner" + "github.com/block/pg-sprite/pkg/preflight" + "github.com/block/pg-sprite/pkg/router" + "github.com/block/pg-sprite/pkg/statement" + "github.com/block/pg-sprite/pkg/verdict" +) + +// Options carries the execution policy for one [Run]: the safety budgets, +// the retry policy, the size guard, and the operator's force +// acknowledgement. The zero value is not a runnable policy: callers set the +// budgets and the size guard deliberately — [DefaultOptions] is the +// sanctioned starting point (the same policy the CLI's flag defaults +// wire), an embedding orchestrator tunes from there. +type Options struct { + // Force is the typed acknowledgement to run the submitted form as-is, + // overriding a safer-sequence substitution or a rewrite-required / + // backend-unavailable refusal. It must name the resolved + // schema-qualified target table exactly; empty means the engine's + // routing decides. Planner refusals (no known safe path) and gated + // statement kinds cannot be forced. + Force string + + // MaxTableSizeBytes is the threshold above which a blind bounded + // attempt of the submitted form is refused, measured as the table's + // full on-disk footprint: heap, indexes, and TOAST, all partitions. + // Substituted safer sequences and planner-proven online idioms are not + // size-guarded — long work on large tables is their purpose. + MaxTableSizeBytes int64 + + // Budget bounds every executed step: brief steps under lock and + // statement timeouts, concurrent index builds and constraint + // validation under their overall bounds. + Budget executor.SequenceBudget + + // Retry is the bounded retry policy when native DDL exceeds its lock + // budget. The zero value uses [executor.DefaultRetryPolicy]; a + // partially configured policy is rejected by the executor. + Retry executor.RetryPolicy + + // Logger receives decision diagnostics (routing, preflight, execution + // transitions); nil discards them. + Logger *slog.Logger + + // Audit receives the force-override audit record; nil discards it. + // The verdict's Forced field is the machine-readable record and is + // always set, so the run's outcome never depends on this logger; the + // CLI wires an always-on stderr handler here so an operator's + // deliberate safety override is visible even without diagnostics. + Audit *slog.Logger +} + +// DefaultOptions is the sanctioned starting point for an embedding caller: +// the same budgets, size guard, and retry policy the CLI's flag defaults +// wire. These are defaults to tune, not a recommendation — a large table +// needs a more generous concurrent-build and validate bound, a hot table a +// tighter lock budget. Force, Logger, and Audit stay zero: overriding +// safety and receiving diagnostics are always deliberate choices. +func DefaultOptions() Options { + return Options{ + MaxTableSizeBytes: 1 << 30, + Budget: executor.SequenceBudget{ + Brief: executor.Budget{LockTimeout: 3 * time.Second, StatementTimeout: 30 * time.Second}, + Concurrent: executor.ConcurrentBudget{Overall: 30 * time.Minute}, + Validate: executor.ValidateBudget{LockTimeout: 3 * time.Second, Overall: 30 * time.Minute}, + }, + Retry: executor.DefaultRetryPolicy(), + } +} + +// logger returns the diagnostics logger, discarding when the caller wired +// none. +func (o Options) logger() *slog.Logger { + if o.Logger == nil { + return slog.New(slog.DiscardHandler) + } + return o.Logger +} + +// audit returns the audit logger, discarding when the caller wired none — +// the same split Logger has, so the library never writes to the host +// process's stderr behind an embedder's logging stack. +func (o Options) audit() *slog.Logger { + if o.Audit == nil { + return slog.New(slog.DiscardHandler) + } + return o.Audit +} + +// validate rejects an unrunnable policy at the front door, before any +// database work, so the documented "the zero value is not a runnable +// policy" contract holds on every path — not only the paths that happen to +// reach the size guard. Budgets and the retry policy carry their own +// validation in the executor. +func (o Options) validate() error { + if o.MaxTableSizeBytes <= 0 { + return fmt.Errorf("migrate: Options.MaxTableSizeBytes must be positive, got %d; DefaultOptions is the sanctioned starting point", o.MaxTableSizeBytes) + } + return nil +} + +// retry returns the retry policy, preserving the safe defaults for a +// zero-valued policy while leaving partially configured or invalid +// policies for the executor to reject. +func (o Options) retry() executor.RetryPolicy { + if o.Retry == (executor.RetryPolicy{}) { + return executor.DefaultRetryPolicy() + } + return o.Retry +} + +// Run drives one schema change end to end: gate the statement type, +// resolve the target, classify and route the statement exactly as a dry +// run would, execute the routed SQL — the planner's safer native sequence +// by default when the submitted form blocks — and end in exactly one +// verdict. +// +// The verdict-and-error contract has three shapes. A refusal returns the +// refusal verdict and a nil error; the caller maps it to its refusal exit +// path. An execution failure returns the failed verdict — the stable +// executor code plus the committed prefix — together with the operational +// error; the verdict is the error's machine-readable twin. An error with a +// zero verdict means the pipeline stopped before reaching a verdict (a +// resolution, introspection, or acknowledgement error) and nothing was +// executed. +// +// Run does not close the pool; one pool serves any number of calls. +func Run(ctx context.Context, pool *pgxpool.Pool, st statement.Statement, opts Options) (verdict.Verdict, error) { + if err := opts.validate(); err != nil { + return verdict.Verdict{}, err + } + logger := opts.logger() + if v, refused := Gate(st); refused { + return v, nil + } + st, err := resolveTarget(ctx, pool, st, logger) + if err != nil { + return verdict.Verdict{}, err + } + if opts.Force != "" { + if err := checkForceAck(st, opts.Force); err != nil { + return verdict.Verdict{}, err + } + } + + facts, err := LiveFacts(ctx, pool, st) + if err != nil { + return verdict.Verdict{}, err + } + canonical, err := statement.Canonical(st.SQL()) + if err != nil { + return verdict.Verdict{}, err + } + classified, err := planner.Classify(canonical, facts.Classifier) + if err != nil { + return verdict.Verdict{}, err + } + routed := router.Route([]planner.Plan{classified}) + rs := routed.Statements[0] + logger.Debug("statement routed", + "route", string(classified.Route), "disposition", string(rs.Disposition)) + + switch rs.Disposition { + case router.DispositionExecute: + execSQL := rs.ExecSQL + substituted := len(execSQL) != 1 || execSQL[0] != rs.Statement + forced := substituted && opts.Force != "" + if forced { + // The acknowledged override: run the submitted form as a + // blind bounded attempt instead of the safer sequence. + execSQL, substituted = []string{canonical}, false + auditForce(opts.audit(), st, rs) + } + return execute(ctx, pool, st, execSQL, rs.Plan, substituted, forced, opts, logger) + case router.DispositionRewriteRequired: + if opts.Force == "" { + return rewriteRequiredVerdict(st), nil + } + auditForce(opts.audit(), st, rs) + return execute(ctx, pool, st, []string{canonical}, rs.Plan, false, true, opts, logger) + case router.DispositionUnavailable: + if opts.Force == "" { + return backendUnavailableVerdict(st, rs), nil + } + auditForce(opts.audit(), st, rs) + return execute(ctx, pool, st, []string{canonical}, rs.Plan, false, true, opts, logger) + case router.DispositionRefuse: + // A planner refusal means no known safe path — there is nothing + // bounded to acknowledge, so the force acknowledgement does not + // apply. + return routeRefusalVerdict(st, rs), nil + default: + // A disposition this build does not know is a router version + // skew; refuse to act rather than guess. + return verdict.Verdict{}, fmt.Errorf("unknown disposition %q", rs.Disposition) + } +} + +// resolveTarget qualifies an unqualified statement against the session's +// search_path exactly once and re-emits it in schema-qualified form, so +// every later stage — facts, classification, the planner's safer sequences, +// preflight, and the executor — names the same relation regardless of any +// session's search_path. The executor's own unqualified-table refusals +// stay: this is the front door resolving its caller's intent, not the +// executor trusting a name. Statements already qualified (or without a +// table target) pass through unchanged. +func resolveTarget(ctx context.Context, pool *pgxpool.Pool, st statement.Statement, + logger *slog.Logger) (statement.Statement, error) { + if st.Schema() != "" || st.Table() == "" { + return st, nil + } + // Re-emitting goes through the deparser, which drops comments; refuse + // commented input instead of silently discarding content. + if err := statement.CheckNoComments(st.SQL()); err != nil { + return statement.Statement{}, err + } + const q = ` + SELECT n.nspname + FROM pg_class c + JOIN pg_namespace n ON n.oid = c.relnamespace + WHERE c.oid = to_regclass(quote_ident($1))` + var schema string + err := pool.QueryRow(ctx, q, st.Table()).Scan(&schema) + if errors.Is(err, pgx.ErrNoRows) { + return statement.Statement{}, fmt.Errorf("%w: %s is not visible on the session search_path", + preflight.ErrTableNotFound, st.Table()) + } + if err != nil { + return statement.Statement{}, fmt.Errorf("resolve %s against search_path: %w", st.Table(), err) + } + sql, err := statement.Qualify(st.SQL(), schema) + if err != nil { + return statement.Statement{}, fmt.Errorf("qualify %s as %s.%s: %w", st.Table(), schema, st.Table(), err) + } + logger.Debug("unqualified table resolved", "table", st.Table(), "schema", schema) + return statement.ParseOne(sql) +} + +// checkForceAck validates the force acknowledgement: it must name the +// resolved schema-qualified target table exactly, proving the operator +// names the relation whose lock they are accepting. A mismatch is a usage +// error — nothing has executed. +func checkForceAck(st statement.Statement, ack string) error { + if ack == qualified(st) { + return nil + } + return fmt.Errorf("the force acknowledgement must name the resolved target table %q, got %q; nothing was executed", + qualified(st), ack) +} + +// auditForce records the override decision before anything executes: the +// operator chose the submitted form over the engine's routing. The record +// is warn-level and unconditional — an audit trail must not depend on +// diagnostics being enabled — and the verdict's Forced field is its +// machine-readable twin. +func auditForce(audit *slog.Logger, st statement.Statement, rs router.Statement) { + audit.Warn("forced execution of submitted form", + "table", qualified(st), + "kind", st.Kind().String(), + "disposition", string(rs.Disposition)) +} + +// execute runs execSQL through the sequence executor: the planner's safer +// sequence when one was substituted, otherwise the submitted form. A forced +// run bypasses the sequence executor's shape admission — the acknowledged +// override runs the submitted form as one blind bounded attempt, whatever +// its kind — so it goes through the optimistic executor directly, under the +// same brief budgets. Blind attempts of the submitted form — including +// forced ones — are size-guarded; substituted sequences and planner-proven +// online idioms are not — long work on large tables is their purpose, and +// every brief step is still budget-bounded. Before anything runs, the +// connected role is checked at the tier the routed steps actually need +// (engine-role contract), so a role that would die mid-change is refused +// with the exact provisioning statement instead. +func execute(ctx context.Context, pool *pgxpool.Pool, st statement.Statement, + execSQL []string, plan planner.Plan, substituted, forced bool, + opts Options, logger *slog.Logger) (verdict.Verdict, error) { + limit := opts.MaxTableSizeBytes + if !sizeGuardApplies(plan, substituted) { + limit = preflight.NoSizeLimit + } + pt, err := preflight.CheckTable(ctx, pool, st.Schema(), st.Table(), limit) + var sizeErr *preflight.SizeError + if errors.As(err, &sizeErr) { + return sizeGuardVerdict(st, sizeErr, forced), nil + } + if err != nil { + return verdict.Verdict{}, err + } + serverMajor, err := dbconn.ServerMajor(ctx, pool) + if err != nil { + return verdict.Verdict{}, err + } + if err := preflight.CheckPartitionSupport(pt, serverMajor, execSQL); err != nil { + var partitionErr *preflight.UnsupportedPartitionedParentError + if errors.As(err, &partitionErr) { + return partitionedParentVerdict(st, partitionErr, forced), nil + } + return verdict.Verdict{}, err + } + tier, err := preflight.RequiredTier(execSQL) + if err != nil { + return verdict.Verdict{}, err + } + priv, err := preflight.CheckPrivileges(ctx, pool, st.Schema(), st.Table(), + preflight.Requirement{Tier: tier}) + var privErr *preflight.PrivilegeError + if errors.As(err, &privErr) { + return privilegeVerdict(st, privErr, forced), nil + } + if err != nil { + return verdict.Verdict{}, err + } + logger.Debug("privilege preflight passed", + "role", priv.Role(), "owner", priv.Owner(), "tier", tier.String()) + logger.Debug("preflight passed", + "table", qualified(st), "total_bytes", pt.TotalBytes(), "limit_bytes", limit) + if substituted { + logger.Debug("substituting safer native sequence", + "table", qualified(st), "steps", len(execSQL)) + } + + retry := opts.retry() + start := time.Now() + var rep executor.SequenceReport + if forced { + err = executor.ExecuteNative(ctx, pool, pt, st, opts.Budget.Brief, retry) + } else { + rep, err = executor.RunSequence(ctx, pool, pt, execSQL, opts.Budget, retry) + } + elapsed := time.Since(start) + if v, refused := execRefusal(st, err, substituted, forced, onlineIdiomPlan(plan)); refused { + logger.Debug("execution refused", + "reason", string(v.Reason), "cause", string(v.Cause), "attempts", v.Attempts, "elapsed", elapsed) + return v, nil + } + if err != nil { + // Everything else is an operational failure, not a refusal: the + // typed *SequenceStepError names the failed step and the committed + // prefix that remains, and an *InvalidIndexError carries the + // operator recovery guidance. The failed verdict is the error's + // machine-readable twin — the stable executor code, the failed + // step, and the committed prefix — while the error itself still + // returns, so the caller's operational-error exit applies, not its + // refusal exit. + v := failureVerdict(st, err, rep, forced) + logger.Debug("execution failed", + "code", v.Code, "failed_step", v.FailedStep, "committed_steps", len(v.ExecutedSQL), "elapsed", elapsed) + return v, fmt.Errorf("run schema change on %s: %w", qualified(st), err) + } + logger.Debug("schema change committed", + "table", qualified(st), "steps", len(execSQL), "elapsed", elapsed) + + v := verdict.Verdict{ + Outcome: verdict.OutcomeExecuted, + Statement: st.SQL(), + Table: qualified(st), + Forced: forced, + Detail: fmt.Sprintf("committed within budgets (lock %s, statement %s): the change was effectively instant", + opts.Budget.Brief.LockTimeout, opts.Budget.Brief.StatementTimeout), + } + if substituted { + v.ExecutedSQL = execSQL + v.Detail = fmt.Sprintf("the submitted form blocks; pg-sprite ran the safer native sequence instead — all %d steps committed", + len(execSQL)) + } + if forced { + v.Detail = fmt.Sprintf("forced: the submitted form ran as-is under budgets (lock %s, statement %s), overriding the engine's routing", + opts.Budget.Brief.LockTimeout, opts.Budget.Brief.StatementTimeout) + } + return v, nil +} + +// qualified renders the statement's target table for the verdict, empty when +// the statement has none. +func qualified(st statement.Statement) string { + if st.Table() == "" { + return "" + } + if st.Schema() == "" { + return st.Table() + } + return st.Schema() + "." + st.Table() +} diff --git a/pkg/migrate/migrate_test.go b/pkg/migrate/migrate_test.go new file mode 100644 index 0000000..7db4a5e --- /dev/null +++ b/pkg/migrate/migrate_test.go @@ -0,0 +1,260 @@ +package migrate + +import ( + "fmt" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/block/pg-sprite/pkg/executor" + "github.com/block/pg-sprite/pkg/statement" + "github.com/block/pg-sprite/pkg/verdict" +) + +func TestGate(t *testing.T) { + parse := func(t *testing.T, sql string) statement.Statement { + t.Helper() + st, err := statement.ParseOne(sql) + require.NoError(t, err) + return st + } + + t.Run("supported kinds pass", func(t *testing.T) { + for name, sql := range map[string]string{ + "alter table": "ALTER TABLE billing.invoices ADD COLUMN age int", + "create index": "CREATE INDEX i ON billing.invoices (age)", + } { + t.Run(name, func(t *testing.T) { + _, refused := Gate(parse(t, sql)) + assert.False(t, refused) + }) + } + }) + + t.Run("blocking index maintenance points at the concurrent idiom", func(t *testing.T) { + v, refused := Gate(parse(t, "DROP INDEX billing.i")) + require.True(t, refused) + assert.Equal(t, verdict.OutcomeRefused, v.Outcome) + assert.Equal(t, verdict.ReasonIndexStatement, v.Reason) + assert.Equal(t, "DROP INDEX CONCURRENTLY", v.SaferIdiom) + }) + + t.Run("already-concurrent maintenance carries no safer idiom", func(t *testing.T) { + v, refused := Gate(parse(t, "DROP INDEX CONCURRENTLY billing.i")) + require.True(t, refused) + assert.Equal(t, verdict.ReasonIndexStatement, v.Reason) + assert.Empty(t, v.SaferIdiom, + "suggesting the submitted statement back would loop a resubmitting automation") + }) + + t.Run("create table points at the declarative front door", func(t *testing.T) { + v, refused := Gate(parse(t, "CREATE TABLE t (id int)")) + require.True(t, refused) + assert.Equal(t, verdict.ReasonUnsupportedStatement, v.Reason) + assert.Empty(t, v.SaferIdiom, + "the library names the concept; each front door attaches its own actionable spelling") + }) + + t.Run("other kinds are unsupported", func(t *testing.T) { + v, refused := Gate(parse(t, "DROP TABLE t")) + require.True(t, refused) + assert.Equal(t, verdict.ReasonUnsupportedStatement, v.Reason) + assert.Empty(t, v.SaferIdiom) + }) +} + +func TestRunRejectsUnrunnableOptions(t *testing.T) { + st, err := statement.ParseOne("ALTER TABLE billing.invoices ADD COLUMN age int") + require.NoError(t, err) + + // Options validation happens before any database work, so no pool is + // needed: the zero value must be rejected by field name on every path, + // not only the paths that reach the size guard. + v, err := Run(t.Context(), nil, st, Options{}) + require.Error(t, err) + assert.Contains(t, err.Error(), "MaxTableSizeBytes", + "the rejection must name the Options field, not an internal preflight concept") + assert.Equal(t, verdict.Verdict{}, v, "an error before a verdict carries a zero verdict") +} + +func TestDefaultOptionsIsRunnable(t *testing.T) { + opts := DefaultOptions() + assert.NoError(t, opts.validate()) + assert.Equal(t, executor.DefaultRetryPolicy(), opts.Retry) +} + +func TestOptionsRetryDefaults(t *testing.T) { + t.Run("zero value falls back to the executor defaults", func(t *testing.T) { + var o Options + assert.Equal(t, executor.DefaultRetryPolicy(), o.retry()) + }) + + t.Run("a configured policy passes through untouched", func(t *testing.T) { + policy := executor.RetryPolicy{ + MaxAttempts: 5, + InitialBackoff: 250 * time.Millisecond, + MaxBackoff: 2 * time.Second, + } + assert.Equal(t, policy, Options{Retry: policy}.retry()) + }) +} + +func TestBudgetVerdict(t *testing.T) { + st, err := statement.ParseOne("ALTER TABLE billing.invoices ALTER COLUMN id TYPE bigint") + require.NoError(t, err) + + t.Run("lock budget", func(t *testing.T) { + v := budgetVerdict(st, &executor.BudgetError{Cause: executor.CauseLock, Budget: 3 * time.Second, Attempts: 3}, false, false) + assert.Equal(t, verdict.OutcomeRefused, v.Outcome) + assert.Equal(t, verdict.ReasonBudgetExceeded, v.Reason) + assert.Equal(t, verdict.CauseLockBudget, v.Cause) + assert.Equal(t, 3, v.Attempts, "the exhausted attempt count must reach the verdict") + assert.Equal(t, "billing.invoices", v.Table) + assert.False(t, v.Forced) + assert.NotEmpty(t, v.Detail) + }) + + t.Run("statement budget", func(t *testing.T) { + v := budgetVerdict(st, &executor.BudgetError{Cause: executor.CauseStatement, Budget: 30 * time.Second}, false, false) + assert.Equal(t, verdict.ReasonBudgetExceeded, v.Reason) + assert.Equal(t, verdict.CauseStatementBudget, v.Cause) + assert.NotEmpty(t, v.Detail) + }) + + t.Run("statement budget on the online idiom advises a larger budget", func(t *testing.T) { + blind := budgetVerdict(st, &executor.BudgetError{Cause: executor.CauseStatement, Budget: time.Minute}, false, false) + online := budgetVerdict(st, &executor.BudgetError{Cause: executor.CauseStatement, Budget: time.Minute}, false, true) + assert.Equal(t, verdict.CauseStatementBudget, online.Cause) + assert.NotEqual(t, blind.Detail, online.Detail, + "a cancelled online idiom needs a larger budget, not the copy-and-swap advice") + }) + + t.Run("a forced refusal records the override", func(t *testing.T) { + v := budgetVerdict(st, &executor.BudgetError{Cause: executor.CauseStatement, Budget: time.Minute}, true, false) + assert.True(t, v.Forced, "the machine-readable audit record must survive a refusal") + }) + + t.Run("unknown cause falls back to the error text", func(t *testing.T) { + budgetErr := &executor.BudgetError{Budget: time.Second} + v := budgetVerdict(st, budgetErr, false, false) + assert.Equal(t, verdict.ReasonBudgetExceeded, v.Reason) + assert.Equal(t, verdict.CauseNone, v.Cause) + assert.Equal(t, budgetErr.Error(), v.Detail) + }) +} + +func TestFailureVerdict(t *testing.T) { + st, err := statement.ParseOne("ALTER TABLE billing.invoices ALTER COLUMN status SET NOT NULL") + require.NoError(t, err) + + t.Run("a mid-sequence failure discloses the step and the committed prefix", func(t *testing.T) { + stepErr := &executor.SequenceStepError{ + Step: 2, + Total: 4, + Kind: executor.StepValidateConstraint, + SQL: "ALTER TABLE billing.invoices VALIDATE CONSTRAINT c", + Err: fmt.Errorf("server error"), + } + rep := executor.SequenceReport{Steps: []executor.StepReport{ + {SQL: "ALTER TABLE billing.invoices ADD CONSTRAINT c CHECK (status IS NOT NULL) NOT VALID", Kind: executor.StepBrief}, + }} + v := failureVerdict(st, fmt.Errorf("wrapped: %w", stepErr), rep, false) + assert.Equal(t, verdict.OutcomeFailed, v.Outcome) + assert.Equal(t, string(executor.CodeExecutionFailed), v.Code) + assert.Equal(t, 2, v.FailedStep) + assert.Equal(t, stepErr.SQL, v.FailedStepSQL) + assert.Equal(t, []string{rep.Steps[0].SQL}, v.ExecutedSQL, + "the committed prefix is what distinguishes partial state from nothing happened") + assert.Equal(t, "billing.invoices", v.Table) + assert.False(t, v.Forced) + }) + + t.Run("the failed step's typed cause maps to its own stable code", func(t *testing.T) { + stepErr := &executor.SequenceStepError{ + Step: 2, Total: 4, Kind: executor.StepValidateConstraint, + SQL: "ALTER TABLE billing.invoices VALIDATE CONSTRAINT c", + Err: &executor.BudgetError{Cause: executor.CauseStatement, Budget: time.Minute}, + } + v := failureVerdict(st, stepErr, executor.SequenceReport{}, false) + assert.Equal(t, string(executor.CodeBudgetStatementExceeded), v.Code) + assert.Empty(t, v.ExecutedSQL, "an empty committed prefix means nothing committed") + }) + + t.Run("a non-sequence failure carries no step and an empty prefix", func(t *testing.T) { + v := failureVerdict(st, fmt.Errorf("server error"), executor.SequenceReport{}, true) + assert.Equal(t, verdict.OutcomeFailed, v.Outcome) + assert.Equal(t, string(executor.CodeExecutionFailed), v.Code) + assert.Zero(t, v.FailedStep) + assert.Empty(t, v.FailedStepSQL) + assert.Empty(t, v.ExecutedSQL) + assert.True(t, v.Forced, "the machine-readable audit record must survive a failure") + }) +} + +func TestExecRefusal(t *testing.T) { + st, err := statement.ParseOne("CREATE INDEX CONCURRENTLY i ON billing.invoices (customer_id)") + require.NoError(t, err) + + t.Run("nil error is not a refusal", func(t *testing.T) { + _, refused := execRefusal(st, nil, false, false, false) + assert.False(t, refused) + }) + + t.Run("a budget-cancelled attempt is a refusal", func(t *testing.T) { + budgetErr := &executor.BudgetError{Cause: executor.CauseStatement, Budget: time.Minute} + v, refused := execRefusal(st, fmt.Errorf("wrapped: %w", budgetErr), false, true, true) + require.True(t, refused) + assert.Equal(t, verdict.ReasonBudgetExceeded, v.Reason) + assert.Equal(t, verdict.CauseStatementBudget, v.Cause) + assert.True(t, v.Forced) + }) + + t.Run("a substituted sequence's budget failure is operational", func(t *testing.T) { + budgetErr := &executor.BudgetError{Cause: executor.CauseStatement, Budget: time.Minute} + _, refused := execRefusal(st, fmt.Errorf("wrapped: %w", budgetErr), true, false, false) + assert.False(t, refused, "a failed substituted step is an operational failure with a committed prefix") + }) + + t.Run("invalid-index debris is never a budget refusal", func(t *testing.T) { + // The exact chain a budget-cancelled concurrent build that left an + // invalid index produces: the buried *BudgetError must not map to a + // budget verdict that conceals the operator-recovery outcome. + invalid := &executor.InvalidIndexError{ + Schema: "billing", + Index: "i", + Build: &executor.BudgetError{Cause: executor.CauseStatement, Budget: time.Minute}, + Cleanup: executor.ErrBuildLeftInvalidIndex, + } + stepErr := &executor.SequenceStepError{Step: 1, Total: 1, Err: invalid} + _, refused := execRefusal(st, stepErr, false, false, true) + assert.False(t, refused, "invalid-index debris needs an operator, not a budget verdict") + }) + + t.Run("static admission refusals map to a typed refusal verdict", func(t *testing.T) { + for name, admissionErr := range map[string]error{ + "unsupported step": fmt.Errorf("sequence step 1 of 1: blocking CREATE INDEX: %w", executor.ErrUnsupportedSequenceStep), + "unnamed index": fmt.Errorf("sequence step 1 of 1: %w", executor.ErrUnnamedIndex), + "if not exists": fmt.Errorf("sequence step 1 of 1: %w", executor.ErrIfNotExistsUnsupported), + } { + t.Run(name, func(t *testing.T) { + v, refused := execRefusal(st, admissionErr, true, false, false) + require.True(t, refused, "an admission refusal decided before execution is a refusal verdict") + assert.Equal(t, verdict.ReasonUnsupportedStatement, v.Reason) + assert.NotEmpty(t, v.Detail) + }) + } + }) + + t.Run("an admission sentinel inside a step failure stays operational", func(t *testing.T) { + stepErr := &executor.SequenceStepError{Step: 2, Total: 3, Err: executor.ErrUnnamedIndex} + _, refused := execRefusal(st, stepErr, true, false, false) + assert.False(t, refused, "a step failure means execution started; the committed prefix must surface") + }) + + t.Run("an operational server error is not a refusal", func(t *testing.T) { + _, refused := execRefusal(st, fmt.Errorf("connection reset"), false, false, false) + assert.False(t, refused) + }) +} diff --git a/pkg/migrate/run_integration_test.go b/pkg/migrate/run_integration_test.go new file mode 100644 index 0000000..f26abfe --- /dev/null +++ b/pkg/migrate/run_integration_test.go @@ -0,0 +1,210 @@ +package migrate_test + +import ( + "fmt" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/block/pg-sprite/internal/testutil" + "github.com/block/pg-sprite/pkg/dbconn" + "github.com/block/pg-sprite/pkg/executor" + "github.com/block/pg-sprite/pkg/migrate" + "github.com/block/pg-sprite/pkg/statement" + "github.com/block/pg-sprite/pkg/verdict" +) + +// runOptions is a runnable policy for direct [migrate.Run] calls: the +// sanctioned embedding starting point, with the long-phase bounds tightened +// the way a caller tunes them — here so a hung test fails in a minute, not +// thirty. +func runOptions() migrate.Options { + opts := migrate.DefaultOptions() + opts.Budget.Concurrent.Overall = time.Minute + opts.Budget.Validate.Overall = time.Minute + return opts +} + +func parseOne(t *testing.T, sql string) statement.Statement { + t.Helper() + st, err := statement.ParseOne(sql) + require.NoError(t, err) + return st +} + +// Run is the library front door: these tests drive its top-level dispatch — +// gate, force acknowledgement, and each router disposition — directly, +// without the CLI adapter, so a non-CLI embedding caller has the same +// coverage the CLI's integration tests give the flag surface. +func TestRunDispatch(t *testing.T) { + url := testutil.StartPostgres(t) + pool, err := dbconn.NewPool(t.Context(), dbconn.Config{URL: url}) + require.NoError(t, err) + defer pool.Close() + + t.Run("executes an instant change to one verdict", func(t *testing.T) { + schema := testutil.NewSchema(t, pool) + _, err := pool.Exec(t.Context(), fmt.Sprintf("CREATE TABLE %s.t (id int PRIMARY KEY)", schema)) + require.NoError(t, err) + + v, err := migrate.Run(t.Context(), pool, + parseOne(t, fmt.Sprintf("ALTER TABLE %s.t ADD COLUMN age int", schema)), runOptions()) + require.NoError(t, err) + assert.Equal(t, verdict.OutcomeExecuted, v.Outcome) + assert.Equal(t, schema+".t", v.Table) + + var typ string + require.NoError(t, pool.QueryRow(t.Context(), + `SELECT data_type FROM information_schema.columns + WHERE table_schema = $1 AND table_name = 't' AND column_name = 'age'`, schema).Scan(&typ)) + assert.Equal(t, "integer", typ) + }) + + t.Run("substitutes the safer native sequence", func(t *testing.T) { + schema := testutil.NewSchema(t, pool) + _, err := pool.Exec(t.Context(), fmt.Sprintf("CREATE TABLE %s.t (id int PRIMARY KEY, v text)", schema)) + require.NoError(t, err) + _, err = pool.Exec(t.Context(), fmt.Sprintf( + "INSERT INTO %s.t SELECT g, 'x' FROM generate_series(1, 100) g", schema)) + require.NoError(t, err) + + v, err := migrate.Run(t.Context(), pool, + parseOne(t, fmt.Sprintf("ALTER TABLE %s.t ALTER COLUMN v SET NOT NULL", schema)), runOptions()) + require.NoError(t, err) + assert.Equal(t, verdict.OutcomeExecuted, v.Outcome) + assert.Len(t, v.ExecutedSQL, 4, "the four-step SET NOT NULL sequence must be reported") + + var notNull bool + require.NoError(t, pool.QueryRow(t.Context(), + `SELECT attnotnull FROM pg_attribute + WHERE attrelid = ($1 || '.t')::regclass AND attname = 'v'`, schema).Scan(¬Null)) + assert.True(t, notNull) + }) + + t.Run("re-checks the gate for callers that skipped it", func(t *testing.T) { + schema := testutil.NewSchema(t, pool) + _, err := pool.Exec(t.Context(), fmt.Sprintf("CREATE TABLE %s.t (id int PRIMARY KEY)", schema)) + require.NoError(t, err) + + v, err := migrate.Run(t.Context(), pool, + parseOne(t, fmt.Sprintf("DROP TABLE %s.t", schema)), runOptions()) + require.NoError(t, err) + assert.Equal(t, verdict.OutcomeRefused, v.Outcome) + assert.Equal(t, verdict.ReasonUnsupportedStatement, v.Reason) + + var exists bool + require.NoError(t, pool.QueryRow(t.Context(), + "SELECT to_regclass($1 || '.t') IS NOT NULL", schema).Scan(&exists)) + assert.True(t, exists, "a gated statement must never execute") + }) + + t.Run("refuses a copy-and-swap route as backend-unavailable", func(t *testing.T) { + schema := testutil.NewSchema(t, pool) + _, err := pool.Exec(t.Context(), fmt.Sprintf("CREATE TABLE %s.t (id int PRIMARY KEY, v text)", schema)) + require.NoError(t, err) + _, err = pool.Exec(t.Context(), fmt.Sprintf( + "INSERT INTO %s.t SELECT g, 'x' FROM generate_series(1, 100) g", schema)) + require.NoError(t, err) + + v, err := migrate.Run(t.Context(), pool, + parseOne(t, fmt.Sprintf("ALTER TABLE %s.t ALTER COLUMN id TYPE bigint", schema)), runOptions()) + require.NoError(t, err) + assert.Equal(t, verdict.OutcomeRefused, v.Outcome) + assert.Equal(t, verdict.ReasonBackendUnavailable, v.Reason) + + var typ string + require.NoError(t, pool.QueryRow(t.Context(), + `SELECT data_type FROM information_schema.columns + WHERE table_schema = $1 AND table_name = 't' AND column_name = 'id'`, schema).Scan(&typ)) + assert.Equal(t, "integer", typ, "the refused change must not touch the schema") + }) + + t.Run("refuses an unconstructible rewrite as rewrite-required", func(t *testing.T) { + schema := testutil.NewSchema(t, pool) + _, err := pool.Exec(t.Context(), fmt.Sprintf("CREATE TABLE %s.t (id int PRIMARY KEY)", schema)) + require.NoError(t, err) + + v, err := migrate.Run(t.Context(), pool, + parseOne(t, fmt.Sprintf("ALTER TABLE %s.t ADD COLUMN nickname text UNIQUE", schema)), runOptions()) + require.NoError(t, err) + assert.Equal(t, verdict.OutcomeRefused, v.Outcome) + assert.Equal(t, verdict.ReasonRewriteRequired, v.Reason) + + var exists bool + require.NoError(t, pool.QueryRow(t.Context(), + `SELECT EXISTS (SELECT 1 FROM information_schema.columns + WHERE table_schema = $1 AND table_name = 't' AND column_name = 'nickname')`, schema).Scan(&exists)) + assert.False(t, exists, "the refused change must not touch the schema") + }) + + t.Run("runs the acknowledged submitted form as-is", func(t *testing.T) { + schema := testutil.NewSchema(t, pool) + _, err := pool.Exec(t.Context(), fmt.Sprintf("CREATE TABLE %s.t (id int PRIMARY KEY, v text)", schema)) + require.NoError(t, err) + _, err = pool.Exec(t.Context(), fmt.Sprintf( + "INSERT INTO %s.t SELECT g, 'x' FROM generate_series(1, 100) g", schema)) + require.NoError(t, err) + + opts := runOptions() + opts.Force = schema + ".t" + v, err := migrate.Run(t.Context(), pool, + parseOne(t, fmt.Sprintf("ALTER TABLE %s.t ALTER COLUMN v SET NOT NULL", schema)), opts) + require.NoError(t, err) + assert.Equal(t, verdict.OutcomeExecuted, v.Outcome) + assert.True(t, v.Forced, "the verdict must record the override") + assert.Empty(t, v.ExecutedSQL, "the submitted form ran as-is; no substitution to report") + + var notNull bool + require.NoError(t, pool.QueryRow(t.Context(), + `SELECT attnotnull FROM pg_attribute + WHERE attrelid = ($1 || '.t')::regclass AND attname = 'v'`, schema).Scan(¬Null)) + assert.True(t, notNull) + }) + + t.Run("returns the failed verdict together with the operational error", func(t *testing.T) { + schema := testutil.NewSchema(t, pool) + _, err := pool.Exec(t.Context(), fmt.Sprintf("CREATE TABLE %s.t (id int PRIMARY KEY, v text)", schema)) + require.NoError(t, err) + // The NULL row makes the substituted sequence's VALIDATE CONSTRAINT + // step fail after the scaffold CHECK ... NOT VALID has committed. + _, err = pool.Exec(t.Context(), fmt.Sprintf("INSERT INTO %s.t VALUES (1, NULL)", schema)) + require.NoError(t, err) + + v, err := migrate.Run(t.Context(), pool, + parseOne(t, fmt.Sprintf("ALTER TABLE %s.t ALTER COLUMN v SET NOT NULL", schema)), runOptions()) + require.Error(t, err, "an execution failure is an operational error") + assert.Equal(t, verdict.OutcomeFailed, v.Outcome, + "the failed verdict is the error's machine-readable twin — both return together") + assert.Equal(t, string(executor.CodeExecutionFailed), v.Code) + assert.Len(t, v.ExecutedSQL, 1, "exactly the scaffold step committed before the failure") + + var scaffolds int + require.NoError(t, pool.QueryRow(t.Context(), + `SELECT count(*) FROM pg_constraint con + JOIN pg_class c ON c.oid = con.conrelid + JOIN pg_namespace n ON n.oid = c.relnamespace + WHERE n.nspname = $1 AND c.relname = 't' AND con.contype = 'c'`, schema).Scan(&scaffolds)) + assert.Equal(t, 1, scaffolds, "the committed prefix must describe real surviving state") + }) + + t.Run("rejects a mismatched force acknowledgement before a verdict", func(t *testing.T) { + schema := testutil.NewSchema(t, pool) + _, err := pool.Exec(t.Context(), fmt.Sprintf("CREATE TABLE %s.t (id int PRIMARY KEY)", schema)) + require.NoError(t, err) + + opts := runOptions() + opts.Force = schema + ".wrong" + v, err := migrate.Run(t.Context(), pool, + parseOne(t, fmt.Sprintf("ALTER TABLE %s.t ADD COLUMN age int", schema)), opts) + require.Error(t, err) + assert.Equal(t, verdict.Verdict{}, v, "an error before a verdict carries a zero verdict") + + var exists bool + require.NoError(t, pool.QueryRow(t.Context(), + `SELECT EXISTS (SELECT 1 FROM information_schema.columns + WHERE table_schema = $1 AND table_name = 't' AND column_name = 'age')`, schema).Scan(&exists)) + assert.False(t, exists, "nothing may execute on an acknowledgement mismatch") + }) +} diff --git a/pkg/migrate/verdicts.go b/pkg/migrate/verdicts.go new file mode 100644 index 0000000..d8a4aba --- /dev/null +++ b/pkg/migrate/verdicts.go @@ -0,0 +1,319 @@ +package migrate + +import ( + "errors" + "fmt" + + "github.com/block/pg-sprite/pkg/executor" + "github.com/block/pg-sprite/pkg/planner" + "github.com/block/pg-sprite/pkg/preflight" + "github.com/block/pg-sprite/pkg/router" + "github.com/block/pg-sprite/pkg/statement" + "github.com/block/pg-sprite/pkg/verdict" +) + +// Gate is the statement-type gate: ALTER TABLE and CREATE INDEX proceed to +// classification (a blocking CREATE INDEX is substituted with its +// concurrent build, a submitted concurrent build is driven directly); the +// index-maintenance forms the executor cannot drive yet are pointed at +// their concurrent idiom, everything else is unsupported. Refused +// statements are never executed. Gate needs no database, so a caller can +// refuse before dialing; [Run] re-checks it regardless. +func Gate(st statement.Statement) (verdict.Verdict, bool) { + v := verdict.Verdict{Outcome: verdict.OutcomeRefused, Statement: st.SQL()} + switch st.Kind() { + case statement.KindAlterTable, statement.KindCreateIndex: + return verdict.Verdict{}, false + case statement.KindDropIndex, statement.KindReindex: + v.Reason = verdict.ReasonIndexStatement + v.Detail, v.SaferIdiom = indexAdvice(st) + case statement.KindCreateTable: + v.Reason = verdict.ReasonUnsupportedStatement + // The library names the concept; each front door attaches its own + // actionable spelling (the CLI points at its diff command). + v.Detail = "migrate changes an existing table; to converge a table onto a desired-state CREATE TABLE, " + + "use the declarative front door — diff the desired schema against the live database" + case statement.KindOther: + v.Reason = verdict.ReasonUnsupportedStatement + v.Detail = "only ALTER TABLE and CREATE INDEX statements are supported by the imperative front door" + } + return v, true +} + +// indexAdvice explains an index-statement refusal for the maintenance forms +// the executor does not drive (DROP INDEX, REINDEX). The already-concurrent +// forms carry no safer idiom: suggesting the statement the user submitted +// would confuse a human once and send a resubmitting automation into a loop. +func indexAdvice(st statement.Statement) (detail, saferIdiom string) { + if st.Concurrent() { + return "this is already the safe concurrent idiom; pg-sprite does not drive this maintenance form yet — run it directly against the database", "" + } + switch st.Kind() { + case statement.KindDropIndex: + return "a plain DROP INDEX takes ACCESS EXCLUSIVE on the table; the concurrent drop does not", "DROP INDEX CONCURRENTLY" + case statement.KindReindex: + return "a plain REINDEX blocks writes; the concurrent rebuild does not", "REINDEX ... CONCURRENTLY" + default: + return "", "" + } +} + +// privilegeVerdict is the refusal for a connected role that lacks the access +// the routed change needs. The error already names the failed catalog check +// and the exact provisioning statement, so it is the detail verbatim. A +// refused forced attempt still records the override: the operator asked for +// the submitted form and the role could not run it. +func privilegeVerdict(st statement.Statement, privErr *preflight.PrivilegeError, forced bool) verdict.Verdict { + return verdict.Verdict{ + Outcome: verdict.OutcomeRefused, + Reason: verdict.ReasonInsufficientPrivileges, + Statement: st.SQL(), + Table: qualified(st), + Forced: forced, + Detail: privErr.Error(), + } +} + +// partitionedParentVerdict refuses unsupported execution steps on a +// partitioned parent before the sequence executor runs anything. +func partitionedParentVerdict(st statement.Statement, partitionErr *preflight.UnsupportedPartitionedParentError, + forced bool) verdict.Verdict { + return verdict.Verdict{ + Outcome: verdict.OutcomeRefused, + Reason: verdict.ReasonUnsupportedPartitionedParent, + Statement: st.SQL(), + Table: qualified(st), + Forced: forced, + Detail: partitionErr.Error(), + } +} + +// rewriteRequiredVerdict is the refusal for a statement whose submitted +// form blocks but for which the planner could not construct the safer +// native sequence — a multi-operation statement, or a pattern it cannot +// build. Running the submitted form would falsify the plan's own reason. +func rewriteRequiredVerdict(st statement.Statement) verdict.Verdict { + return verdict.Verdict{ + Outcome: verdict.OutcomeRefused, + Reason: verdict.ReasonRewriteRequired, + Statement: st.SQL(), + Table: qualified(st), + Detail: "the submitted form blocks and must run as a safer native sequence, but pg-sprite could not " + + "construct one for this statement; submit each operation as its own single-operation statement " + + "so the engine can build its safer form (a dry-run classification shows each operation's route)", + } +} + +// backendUnavailableVerdict is the refusal for a change that routes to an +// execution strategy this build does not implement. +func backendUnavailableVerdict(st statement.Statement, rs router.Statement) verdict.Verdict { + return verdict.Verdict{ + Outcome: verdict.OutcomeRefused, + Reason: verdict.ReasonBackendUnavailable, + Statement: st.SQL(), + Table: qualified(st), + Detail: fmt.Sprintf("the change requires the %s strategy, which this build does not implement yet: "+ + "PostgreSQL would rewrite the table under ACCESS EXCLUSIVE for the whole operation", rs.Backend), + } +} + +// routeRefusalVerdict is the refusal for a statement the planner refused: +// it carries the refused operations by name so the operator knows which +// part of the statement the engine does not know a safe path for. +func routeRefusalVerdict(st statement.Statement, rs router.Statement) verdict.Verdict { + v := verdict.Verdict{ + Outcome: verdict.OutcomeRefused, + Reason: verdict.ReasonUnsupportedStatement, + Statement: st.SQL(), + Table: qualified(st), + Detail: "the planner knows no safe path for this statement", + } + for _, d := range rs.Decisions { + if d.Route == planner.RouteRefuse { + v.Detail = fmt.Sprintf("the planner knows no safe path for %s", d.Operation) + break + } + } + return v +} + +// sizeGuardVerdict is the refusal for tables above the size threshold, where +// even a budget-bounded attempt would visibly stall the table. A refused +// forced attempt still records the override: the operator asked for the +// submitted form and the guard said no. +func sizeGuardVerdict(st statement.Statement, sizeErr *preflight.SizeError, forced bool) verdict.Verdict { + return verdict.Verdict{ + Outcome: verdict.OutcomeRefused, + Reason: verdict.ReasonTableTooLarge, + Statement: st.SQL(), + Table: qualified(st), + Forced: forced, + Detail: fmt.Sprintf("table is %d bytes on disk (heap, indexes, and TOAST), above the configured "+ + "%d-byte size threshold. pg-sprite cannot yet prove this change is instant on a table this "+ + "size; if it requires a rewrite, a cancelled attempt is not a free probe — it would hold "+ + "ACCESS EXCLUSIVE doing rewrite work for the whole budget", + sizeErr.TotalBytes, sizeErr.LimitBytes), + } +} + +// admissionRefusalVerdict is the refusal for a statement the gate admits but +// the executor's static admission refuses before anything executes: an +// unnamed index build, IF NOT EXISTS on a concurrent build, or a substituted +// step shape the sequence executor does not drive yet (DETACH PARTITION +// CONCURRENTLY). The typed error carries the explanation. +func admissionRefusalVerdict(st statement.Statement, err error, forced bool) verdict.Verdict { + return verdict.Verdict{ + Outcome: verdict.OutcomeRefused, + Reason: verdict.ReasonUnsupportedStatement, + Statement: st.SQL(), + Table: qualified(st), + Forced: forced, + Detail: fmt.Sprintf("the engine cannot run this statement safely: %v; nothing was executed", err), + } +} + +// budgetVerdict is the refusal for an attempt that exceeded its lock or +// statement budget and was cancelled without executing. online tailors the +// statement-budget advice: a submitted form the plan proved an online idiom +// (a concurrent build, a lone VALIDATE) needs a larger budget, not a +// different strategy, while a blind attempt that ran past its budget is +// doing rewrite work. A refused forced attempt still records the override. +func budgetVerdict(st statement.Statement, budgetErr *executor.BudgetError, forced, online bool) verdict.Verdict { + v := verdict.Verdict{ + Outcome: verdict.OutcomeRefused, + Reason: verdict.ReasonBudgetExceeded, + Statement: st.SQL(), + Table: qualified(st), + Forced: forced, + } + switch budgetErr.Cause { + case executor.CauseLock: + v.Cause = verdict.CauseLockBudget + v.Attempts = budgetErr.Attempts + if budgetErr.Attempts > 1 { + v.Detail = fmt.Sprintf("the lock was not granted within the %s lock budget on any of %d bounded "+ + "attempts: the table is too contended for a blind attempt; nothing was executed", + budgetErr.Budget, budgetErr.Attempts) + break + } + v.Detail = fmt.Sprintf("the lock was not granted within the %s lock budget: the table is too "+ + "contended right now; nothing was executed", budgetErr.Budget) + case executor.CauseStatement: + v.Cause = verdict.CauseStatementBudget + if online { + v.Detail = fmt.Sprintf("cancelled after the %s budget: the statement already is the safe online "+ + "idiom — the work needs more time, not a different strategy; retry with a larger budget for "+ + "this step class", budgetErr.Budget) + } else { + v.Detail = fmt.Sprintf("cancelled after the %s statement budget: the change does real rewrite work, "+ + "not an in-place catalog change, and needs a copy-and-swap rewrite that pg-sprite does not perform yet. "+ + "If it adds a constraint, ADD CONSTRAINT ... NOT VALID followed by VALIDATE CONSTRAINT avoids the long lock", + budgetErr.Budget) + } + default: + v.Detail = budgetErr.Error() + } + return v +} + +// failureVerdict maps an operational execution failure to its failed +// verdict: the executor's stable outcome code, and for a mid-sequence +// failure the failed step and the committed prefix whose state remains. +// It is the machine-readable twin of the returned error — automation +// branches on Code and ExecutedSQL instead of parsing stderr prose. A +// single bounded attempt (the submitted form, forced or not) rolls back on +// failure, so it carries no step and an empty committed prefix. +func failureVerdict(st statement.Statement, err error, + rep executor.SequenceReport, forced bool) verdict.Verdict { + v := verdict.Verdict{ + Outcome: verdict.OutcomeFailed, + Code: string(executor.OutcomeCode(err)), + Statement: st.SQL(), + Table: qualified(st), + Forced: forced, + Detail: "execution failed; nothing committed — a started bounded attempt rolls back", + } + var stepErr *executor.SequenceStepError + if !errors.As(err, &stepErr) { + return v + } + v.FailedStep = stepErr.Step + v.FailedStepSQL = stepErr.SQL + for _, s := range rep.Steps { + v.ExecutedSQL = append(v.ExecutedSQL, s.SQL) + } + if len(v.ExecutedSQL) > 0 { + v.Detail = fmt.Sprintf("sequence step %d of %d failed; the %d committed steps' state remains — the planner sequence's partial-failure contract says how a retry resumes", + stepErr.Step, stepErr.Total, len(v.ExecutedSQL)) + } else { + v.Detail = fmt.Sprintf("sequence step %d of %d failed; no earlier steps had committed — Code names the outcome and any state the failed step itself left", + stepErr.Step, stepErr.Total) + } + return v +} + +// execRefusal maps an execution failure to its refusal verdict, when the +// failure belongs to the refusal contract rather than the operational-error +// exit: a static admission refusal (decided before anything executed), or a +// budget cancellation of a non-substituted attempt. An *InvalidIndexError +// is never a refusal, even when a budget cancellation is buried inside it — +// invalid-index debris is the one outcome that needs an operator, and its +// typed error carries the recovery guidance a budget verdict would conceal. +func execRefusal(st statement.Statement, err error, + substituted, forced, online bool) (verdict.Verdict, bool) { + if err == nil { + return verdict.Verdict{}, false + } + if isAdmissionRefusal(err) { + return admissionRefusalVerdict(st, err, forced), true + } + var invalidErr *executor.InvalidIndexError + if errors.As(err, &invalidErr) { + return verdict.Verdict{}, false + } + var budgetErr *executor.BudgetError + if !substituted && errors.As(err, &budgetErr) { + // The blind attempt of the submitted form exceeded a budget and was + // cancelled without committing — the Phase 1 refusal contract. A + // forced attempt is bounded by the same budgets: force overrides + // routing, never the executor's protections. + return budgetVerdict(st, budgetErr, forced, online), true + } + return verdict.Verdict{}, false +} + +// isAdmissionRefusal reports whether err is one of the executor's static +// admission refusals: decided from the statement's shape before anything +// executes, so it maps to a refusal verdict, not an operational error. A +// *SequenceStepError wrapper means execution started, which is never an +// admission refusal. +func isAdmissionRefusal(err error) bool { + var stepErr *executor.SequenceStepError + if errors.As(err, &stepErr) { + return false + } + return errors.Is(err, executor.ErrUnsupportedSequenceStep) || + errors.Is(err, executor.ErrUnnamedIndex) || + errors.Is(err, executor.ErrIfNotExistsUnsupported) +} + +// onlineIdiomPlan reports whether the plan proved every operation an online +// idiom (CONCURRENTLY, NOT VALID, VALIDATE): the submitted form already is +// the safe pattern, and running long on a large table is its purpose. +func onlineIdiomPlan(p planner.Plan) bool { + for _, d := range p.Decisions { + if d.Reason != planner.ReasonOnlineIdiom { + return false + } + } + return true +} + +// sizeGuardApplies reports whether the size guard protects this run. It +// guards exactly the blind attempt of the submitted form: when the engine +// substituted the planner's safer sequence, or when the plan proved every +// operation an online idiom, long work on a large table is the pattern's +// purpose and the guard would refuse the very tables the pattern serves. +func sizeGuardApplies(p planner.Plan, substituted bool) bool { + return !substituted && !onlineIdiomPlan(p) +} diff --git a/pkg/verdict/verdict.go b/pkg/verdict/verdict.go index f20a3ba..3d49761 100644 --- a/pkg/verdict/verdict.go +++ b/pkg/verdict/verdict.go @@ -181,7 +181,7 @@ func (v Verdict) String() string { fmt.Fprintf(&b, "\n safer: %s", v.SaferIdiom) } if v.Forced { - b.WriteString("\n forced: the submitted form ran as-is (--force)") + b.WriteString("\n forced: the submitted form ran as-is (force acknowledged)") } if v.FailedStep > 0 { fmt.Fprintf(&b, "\n failed at: step %d: %s", v.FailedStep, v.FailedStepSQL)