Skip to content

[ISSUE #S4] Keep persisted consumer filter data that has no bloom data - #11102

Open
zjncs wants to merge 1 commit into
apache:developfrom
zjncs:fix/consumer-filter-manager-null-bloom-decode
Open

[ISSUE #S4] Keep persisted consumer filter data that has no bloom data#11102
zjncs wants to merge 1 commit into
apache:developfrom
zjncs:fix/consumer-filter-manager-null-bloom-decode

Conversation

@zjncs

@zjncs zjncs commented Sep 9, 2026

Copy link
Copy Markdown

Motivation

ConsumerFilterManager.register stores ConsumerFilterData without bloom data whenever the filter bit map is disabled:

BloomFilterData bloomFilterData = null;
if (this.brokerController == null
        || this.brokerController.getBrokerConfig().isEnableCalcFilterBitMap()) {  // default: false
    bloomFilterData = bloomFilter.generate(consumerGroup + "#" + topic);
}

On restart, decode validates every persisted entry with bloomFilter.isValid(filterData.getBloomFilterData()), and isValid(null) returns false — so a single null-bloom entry flips bloomChanged and the entire persisted filter table is discarded ("Bloom filter is changed!So ignore all filter data persisted"). With the default configuration every broker restart therefore loses all registered SQL92 consumer filters; consumers that re-subscribe rebuild them, but any consumer that does not re-send its subscription silently stops filtering server-side (ExpressionMessageFilter matches everything when consumerFilterData == null).

Modifications

  • Only run the bloom-changed check for entries that actually carry bloom data: if (filterData.getBloomFilterData() != null && !this.bloomFilter.isValid(...)). Entries registered with the bit map disabled now survive restart; a genuinely changed bloom filter still discards the table as before.

Verification

Fail-before (new test on unpatched code — registers a SQL92 filter with enableCalcFilterBitMap=false, persists via encode(), reloads via decode()):

ConsumerFilterManagerTest.testDecodeKeepsFilterDataRegisteredWithoutBloomData:70
Expecting actual not to be null

Pass-after — full ConsumerFilterManagerTest (10 existing + 1 new):

mvn -pl broker test -Dtest='ConsumerFilterManagerTest'
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0

When the filter bit map is disabled (enableCalcFilterBitMap defaults to
false) register() persists ConsumerFilterData without bloom data. On
restart, decode() feeds that null into bloomFilter.isValid(), which
returns false for null, so one such entry flags the bloom filter as
changed and the whole persisted filter table is dropped. Every SQL92
subscription then re-registers from scratch (or, if the consumer does
not re-subscribe, the server-side filter silently matches everything).
Only treat entries that actually carry bloom data as evidence of a
changed bloom filter.

Signed-off-by: zjncs <18910855655@163.com>
Copilot AI lite review requested due to automatic review settings September 9, 2026 10:42

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@RockteMQ-AI RockteMQ-AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Important bug fix — with enableCalcFilterBitMap=false (the default), register stores filter entries with null bloom data, but decode unconditionally validates bloom via isValid(null) which returns false, causing the entire persisted filter table to be discarded on every broker restart. The null-guard in the bloom-changed check is the correct minimal fix: entries without bloom data now survive restart, while genuinely changed bloom filters still trigger the discard as before. Test covers the encode/decode round-trip with the default config.


Automated review by github-manager

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants