Skip to content

[fix][ml] Fix batch ACK index loss when recovering cursor from MetadataStore - #26474

Open
void-ptr974 wants to merge 1 commit into
apache:masterfrom
void-ptr974:fix/metadata-store-batch-ack-recovery
Open

[fix][ml] Fix batch ACK index loss when recovering cursor from MetadataStore#26474
void-ptr974 wants to merge 1 commit into
apache:masterfrom
void-ptr974:fix/metadata-store-batch-ack-recovery

Conversation

@void-ptr974

Copy link
Copy Markdown
Contributor

Motivation

When a cursor is cleanly closed using the MetadataStore persistence path, partial batch acknowledgment indexes are written to ManagedCursorInfo. The cursorsLedgerId == -1 recovery path restored the mark-delete position, individual deleted-message ranges, and properties, but skipped batchedEntryDeletionIndexInfos. After topic unload/reload or another cursor reconstruction, already acknowledged messages within a batch could therefore be redelivered.

Modifications

  • Restore persisted batch deletion indexes from ManagedCursorInfo in the MetadataStore recovery path.
  • Keep recovery gated by batch-index acknowledgment being enabled and by metadata containing batch records, preserving compatibility with old metadata and disabled configurations.
  • Add ManagedCursor recovery coverage for MetadataStore and BookKeeper paths, persistence limits, cross-word bitmaps, missing metadata, disabled batch-index ACK, and combined cursor state.
  • Add a Shared consumer end-to-end test using real producer batching, topic unload, a new client, and an explicitly rebuilt cursor.

Verifying this change

  • ./gradlew :managed-ledger:test --tests '*ManagedCursorBatchAckRecoveryTest' -PtestRetryCount=0 -PtestFailFast=false
  • ./gradlew :pulsar-broker:test --tests '*ManagedCursorBatchAckMetadataStoreRecoveryTest' -PtestRetryCount=0 -PtestFailFast=false
  • ./gradlew quickCheck

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant