[ISSUE #11083] Release owned gRPC event loops and stabilize POP priority tests - #11084
Conversation
RockteMQ-AI
left a comment
There was a problem hiding this comment.
Summary
This PR correctly fixes a resource leak where GrpcServerBuilder-created boss/worker EventLoopGroups were never released on shutdown. The approach of tracking owned vs. borrowed groups via constructor parameters is clean, and the lifecycle handling covers startup failure, shutdown timeout, and interruption paths.
The test coverage is thorough — verifying owned group termination, borrowed group survival, bind-failure cleanup, timeout, and interrupt scenarios.
LGTM.
Automated review by github-manager-bot
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #11084 +/- ##
=============================================
+ Coverage 48.93% 48.97% +0.04%
- Complexity 13846 13860 +14
=============================================
Files 1382 1382
Lines 101602 101640 +38
Branches 13213 13224 +11
=============================================
+ Hits 49717 49782 +65
+ Misses 45858 45807 -51
- Partials 6027 6051 +24 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
ec1fe92 to
1ea4346
Compare
RockteMQ-AI
left a comment
There was a problem hiding this comment.
Summary
The new commits significantly improve test stability by replacing timing-dependent waits with proper polling helpers. The changes make the POP priority integration tests deterministic and resilient to CI timing issues.
Changes reviewed:
- Commit 1ea4346: Released owned gRPC event loop groups on shutdown (resource leak fix)
- Commit bd065d7: Stabilized POP priority tests with awaitStoredMessages/awaitRetryMessages helpers, deterministic queue distribution, and Bazel test result upload on failure
Excellent test stability improvements. The move from Random().nextInt() to deterministic i % writeQueueNum is particularly good for reproducibility.
LGTM
Automated review by RockteMQ-AI
Which Issue(s) This PR Fixes
Fixes #11083
Brief Description
GrpcServerBuildersupplies explicitly created boss and worker event loop groups to gRPC, butGrpcServer.shutdown()never releases them. gRPC treats these groups as caller-owned, so their threads remain alive after the server shuts down.Track internally created groups and shut them down when the server stops, including startup failure, shutdown timeout, and interruption. Create owned groups during
build()and release them if building fails. Preserve existing constructor defaults and add overloads for explicit thread counts or caller-owned groups; supplied groups remain the caller's responsibility and can outlive an individual server. Declare the gRPC test's directgrpc-protobufdependency for Bazel.Stabilize
PopPriorityITby waiting for the expected normal and retry messages to reach their consume queues before asserting priority. Waiting a fixedinvisibleTime + 3seconds can start consumption before a retry becomes available. Replace fixed queue-lock sleeps with bounded readiness checks so sequential requests do not skip a priority queue whose previous completion callback has not released its lock. Use deterministic input priorities and retain the existing ordering, count, and distribution assertions; parameter names and histogram failures now identify the tested configuration.Upload Bazel JUnit XML reports when a job fails. The failing CI run for this PR reported
PopPriorityITbut omitted its 25 MB output, including the original assertion, because it exceeded Bazel's console limit.How Did You Test This Change?
On JDK 11, the build, Checkstyle, and SpotBugs passed:
PopPriorityIT: all 32 enabled tests passed across legacy/KV POP and ascending/descending priority. The existing four ignored queue-expansion cases remain ignored.GrpcServerTest: six tests passed; the Epoll case was skipped on macOS. Coverage includes actual gRPC requests, owned-group termination, bind failures, shared caller-owned groups, shutdown timeout, and interruption.test_priority_consume_retry_as_highestfails withexpected:<1> but was:<0>; the updated test passes under the same delay. This independently reproduces the readiness race; the original CI assertion was unavailable in its truncated log.