Skip to content

smp-server: rework message expiration - #1873

Open
shumvgolove wants to merge 1 commit into
masterfrom
sh/messages-improve
Open

shumvgolove wants to merge 1 commit into
masterfrom
sh/messages-improve

Conversation

@shumvgolove

@shumvgolove shumvgolove commented Sep 19, 2026

Copy link
Copy Markdown
Collaborator

Problem: the expiration sweep iterated every non-empty queue, doing a per-queue probe and COMMIT to find that most had nothing to expire. Cost is proportional to the number of non-empty queues times the per-queue commit, while only a small fraction of queues hold expired messages.

Change:

  • Messages-driven expiration: one indexed scan finds the queues that hold an expired message, then the unchanged delete_expired_msgs runs only on those, committing in batches.
  • Adds idx_messages_expire (msg_ts, recipient_id) WHERE NOT msg_quota.
  • Drops msg_queue_expire (column and idx_msg_queues_expire): no query reads it for a decision after this change, and removing it also removes the flag update that defeated HOT updates.
  • Drops idx_messages_recipient_id_msg_ts (unused; covered by recipient_id_message_id). Keeps recipient_id_msg_quota (in use).

Verification:

  • Functional: differential test (old schema vs migrated schema) over writes, quota markers, peek-ack, and expiration; msg_queues counters and messages are byte-identical.
  • Work: on a 20,000-queue / 1,000-expiring dataset, delete_expired_msgs calls drop from 20,000 to 1,000 for the identical result, with discovery via Index Only Scan.
  • Schema: server_schema.sql matches pg_dump 15 output; the down migration restores the prior schema except msg_queue_expire column order (added to the schema-test down-skip list, as with 20250320_short_links). Library and test suite compile.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant