[stacurry]

Why

On 2026-10-09 disable_retired_mirroring on oauth_sessions in production deadlocked against live traffic for 53 seconds (AUTH-7221). DROP TRIGGER on a partitioned parent removes the cloned child triggers first, taking an ACCESS EXCLUSIVE lock on each partition before the parent, while every application statement locks the parent first and then the partitions it touches. Two walkers over the same relations in opposite orders is a deadlock, and Postgres killed an application statement roughly once a second until the drop itself lost.

• disable_mirroring and disable_retired_mirroring now lock the source table explicitly before dropping the trigger. Once the parent is held, no statement can be holding a partition the drop will wait on, so the walk over the partitions cannot form a cycle.
• enable_mirroring and enable_retired_mirroring take the same lock first. They drop any existing trigger before recreating it, so a re-run on a mirrored partitioned table had the same hazard.
• enable_mirroring, enable_retired_mirroring and both disable commands take --lock-timeout (default 5s, same as swap), so a contended run fails fast and is retried instead of queueing live traffic behind it.

Proof

New partitioned-table tests in src/mirroring.test.ts, run against the local Docker Postgres (28 files, 400 tests passing):

• The issued SQL order is LOCK TABLE ... IN ACCESS EXCLUSIVE MODE, then DROP TRIGGER, then DROP FUNCTION, and pg_locks shows the ACCESS EXCLUSIVE lock granted on the parent to this backend at that point.
• On enable, the LOCK TABLE precedes the DROP TRIGGER IF EXISTS, and re-running enable leaves exactly one trigger on the parent and each partition.
• The trigger is gone from the parent and every partition afterwards, and the function is dropped.
• lock_timeout is set to the requested value inside the transaction.

Not proven

• I did not reproduce the deadlock itself locally. Doing so needs a second session holding a row lock on a partition through the parent while the drop runs; the lock-order test pins the mechanism, not the race. If a reviewer wants the red case: on the previous fork commit, open one transaction that does SELECT ... FROM posts WHERE id = 1 FOR UPDATE on a partitioned posts, run disable_retired_mirroring in another, then issue a second statement through the parent from the first session. Postgres reports deadlock detected. With this branch the drop waits on the first transaction and completes after it commits.
• The real-traffic behavior on core-db-production is only observable when the runner runs this build against oauth_sessions.

Relates to AUTH-7221

🤖 Generated with Claude Code

[linear-code]

AUTH-7221

[greptile-apps]

Confidence Score: 5/5

[Critical impact] The PR appears safe to merge; no actionable issues remain from this review.

Summary

The PR locks the source table before enabling or disabling mirroring and adds --lock-timeout, defaulting to 5s.

• Mirroring changes lock the source table before changing triggers.
• Mirroring commands stop waiting after a configurable lock timeout.

Diagram

sequenceDiagram
  participant CLI as Mirroring command
  participant DB as PostgreSQL
  CLI->>DB: Begin transaction
  CLI->>DB: Set local lock_timeout
  CLI->>DB: Lock source table, parent first
  alt Lock acquired
    CLI->>DB: Drop existing trigger
    CLI->>DB: Create trigger or drop function
    CLI->>DB: Commit
  else Lock times out
    DB-->>CLI: Error
    CLI->>DB: Roll back
  end

Loading

_{Reviews (6) · Last reviewed commit: "make comments more accurate" · Reviewed by Greptile}

[stacurry]

Reworked in bc20425 after discussing the design: the disable-without-drop mode, the trigger enabled state in status, and the --drop flag are gone. The disable commands now lock the table and drop the trigger, with the lock wait bounded by --lock-timeout. The two earlier commits are left in place rather than force-pushed so the alternative is easy to revisit.

@greptileai review

[stacurry]

Good catch on the enable path: enable() runs DROP TRIGGER IF EXISTS before CREATE TRIGGER, so a re-run of enable_retired_mirroring on the live partitioned table would take the same unlocked walk. Fixed in e8c1956: both enable and disable now lock the source first through a shared step, and the partitioned-table tests assert the lock precedes the drop on enable and that a re-run leaves exactly one trigger on the parent and each partition.

@greptileai review

[stefanmb]

LGTM

[stefanmb]

Nit: The new option breaks deadlocks, but whether it's "safe" depends on how much contention is acceptable. Since we're realistically the only users no need to change the verbiage.

[stacurry]

Good call! Updated and I think the comments are more accurate now.