[BP-1.20][FLINK-40070][tests] Fix flaky DynamicParameterITCase reading rolled JobManager logs - #29215
Merged
MartijnVisser merged 1 commit intoSep 17, 2026
Conversation
…JobManager logs
The distribution log4j configuration rolls the log file on startup, so the JobManager startup banner frequently lands in a rolled .log.N file that FlinkDistribution.searchAllLogs skips; the test then either spins unboundedly waiting for the banner (multi-hour e2e_4 hang) or parses a half-written arguments block ("Missing required option: c"). Search rolled logs for the startup banner and bound the wait so a missing banner fails fast.
Generated-by: Claude Opus 4.8 (1M context)
(cherry picked from commit f9e0756)
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
Backport of #28636 (
f9e0756fbfd) to release-1.20.The distribution rolls its log file during JobManager startup, so the startup banner with the
program arguments frequently ends up in a rolled
.log.Nfile thatFlinkDistribution.searchAllLogsskips.DynamicParameterITCasethen waits for that banner inan unbounded loop, and the class has no
@Timeout, so the whole Java e2e leg runs until the CIwatchdog kills it at its time budget. On release-2.3 that took out E2E group 2 on the 2026-09-04,
09-11, 09-15 and 09-16 nightlies. The readiness loop,
FlinkDistributionand the distributionlog4j configuration are identical on this branch.
Brief change log
FlinkDistribution.searchAllLogsgains anincludeRolledLogsoverload, so the banner isfound when it has been rolled. The existing two-argument method delegates with
false, sothe other callers are unchanged.
DynamicParameterITCasewaits withCommonTestUtils.waitUtil(..., 1 minute, 500 ms, msg)instead of
while (!allProgramArgumentsLogged(dist)) Thread.sleep(500), so a banner thatnever arrives fails in a minute with a message rather than consuming the leg's budget.
Verifying this change
FlinkDistribution.javais byte-identical to the master commit.DynamicParameterITCase.javawasresolved by hand: release-1.20 passes the host and port positionally and asserts on
EntrypointClusterConfiguration#getHostnameand#getRestPort, so the argument building and theassertions are left alone. Master's
loadConfigurationhelper and itsConfigurationbasedassertions come from a later change that is not on this branch, and are not introduced here. The
three helpers the fix adds or rewrites are byte-identical to master's.
DynamicParameterITCasegreen twice against a built 1.20 distribution, 6 tests in 29.5 s and29.8 s. spotless and checkstyle green under JDK 11, since this branch pins google-java-format 1.7
rather than 1.24.0.
Does this pull request potentially affect one of the following parts:
@Public(Evolving): noDocumentation
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Opus 5 (1M context)