[BP-2.2][FLINK-40070][tests] Fix flaky DynamicParameterITCase reading rolled JobManager logs - #29214
Open
MartijnVisser wants to merge 1 commit into
Open
MartijnVisser wants to merge 1 commit into
MartijnVisser wants to merge 1 commit into
Conversation
…JobManager logs
The distribution log4j configuration rolls the log file on startup, so the JobManager startup banner frequently lands in a rolled .log.N file that FlinkDistribution.searchAllLogs skips; the test then either spins unboundedly waiting for the banner (multi-hour e2e_4 hang) or parses a half-written arguments block ("Missing required option: c"). Search rolled logs for the startup banner and bound the wait so a missing banner fails fast.
Generated-by: Claude Opus 4.8 (1M context)
(cherry picked from commit f9e0756)
Collaborator
github-actions
Bot
added
target:release-2.2
community-reviewed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
Backport of #28636 (
f9e0756fbfd) to release-2.2.The distribution rolls its log file during JobManager startup, so the startup banner with the
program arguments frequently ends up in a rolled
.log.Nfile thatFlinkDistribution.searchAllLogsskips.DynamicParameterITCasethen waits for that banner inan unbounded loop, and the class has no
@Timeout, so the whole Java e2e leg runs until the CIwatchdog kills it at its time budget. On release-2.3 that took out E2E group 2 on the 2026-09-04,
09-11, 09-15 and 09-16 nightlies. The readiness loop,
FlinkDistributionand the distributionlog4j configuration are identical on this branch.
Brief change log
FlinkDistribution.searchAllLogsgains anincludeRolledLogsoverload, so the banner isfound when it has been rolled. The existing two-argument method delegates with
false, sothe other callers are unchanged.
DynamicParameterITCasewaits withCommonTestUtils.waitUtil(..., 1 minute, 500 ms, msg)instead of
while (!allProgramArgumentsLogged(dist)) Thread.sleep(500), so a banner thatnever arrives fails in a minute with a message rather than consuming the leg's budget.
Verifying this change
Clean cherry-pick: both changed files are byte-identical to the merged master commit, and this
commit is byte-identical to the release-2.3 backport, both verified by blob hash. The release-2.3
results carry over, so there is no separate run here. spotless green.
Does this pull request potentially affect one of the following parts:
@Public(Evolving): noDocumentation
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Opus 5 (1M context)