Context
The inbound delivery loop (#9, PR #16) is delete-on-ack, at-least-once: only successfully delivered messages are acked, so a transient failure is retried on the next pull. A per-message failure — no analysisId, unknown sender, decrypt/parse error, or webhook delivery error — is isolated (logged, skipped, left unacked) in InboundDeliveryProcessor.processBatch / deliverMessage (apps/node-message-broker/src/core/inbound/processor.ts).
Problem
A permanently undecryptable or otherwise unprocessable ("poison") message is never acked, so the Hub redelivers it on every pull cycle indefinitely. It does not block newer messages (each is processed independently), but it:
- persists in the node's mailbox forever,
- wastes a decrypt/resolve attempt every cycle,
- produces a repeating warning log line.
Permanent failure modes include: a frame sealed under the wrong key, a corrupted ciphertext, a sender that is no longer a participant, or a payload that is not valid JSON after decryption.
Suggested approach
Add bounded retry / dead-lettering, e.g.:
- Track per-message attempt counts (keyed by message
id, the existing dedup key) and, after a configurable max, ack to drop (at-most-once for poison) while logging at error with the message id + reason.
- Optionally distinguish permanent failures (parse error, unknown sender) from transient ones (webhook 5xx, resolver/core outage): drop permanent failures immediately, only retry transient ones.
- Consider a small dead-letter sink (counter/metric and/or a local record) so dropped messages are observable rather than silently discarded.
Acceptance
- A permanently poison message is dropped after N attempts (or immediately for clearly-permanent failures) and stops being redelivered.
- Transient failures still retry.
- Dropped messages are observable (logged at error and/or counted), not silently lost.
Follow-up from #9 / PR #16.
Context
The inbound delivery loop (#9, PR #16) is delete-on-ack, at-least-once: only successfully delivered messages are acked, so a transient failure is retried on the next pull. A per-message failure — no
analysisId, unknown sender, decrypt/parse error, or webhook delivery error — is isolated (logged, skipped, left unacked) inInboundDeliveryProcessor.processBatch/deliverMessage(apps/node-message-broker/src/core/inbound/processor.ts).Problem
A permanently undecryptable or otherwise unprocessable ("poison") message is never acked, so the Hub redelivers it on every pull cycle indefinitely. It does not block newer messages (each is processed independently), but it:
Permanent failure modes include: a frame sealed under the wrong key, a corrupted ciphertext, a sender that is no longer a participant, or a payload that is not valid JSON after decryption.
Suggested approach
Add bounded retry / dead-lettering, e.g.:
id, the existing dedup key) and, after a configurable max, ack to drop (at-most-once for poison) while logging aterrorwith the message id + reason.Acceptance
Follow-up from #9 / PR #16.