Skip to content

[flink] Fix flaky AppendTableITCase streaming compaction tests - #10184

Open
LuciferYang wants to merge 1 commit into
apache:masterfrom
LuciferYang:fix/append-streaming-compaction-flaky
Open

LuciferYang wants to merge 1 commit into
apache:masterfrom
LuciferYang:fix/append-streaming-compaction-flaky

Conversation

@LuciferYang

Copy link
Copy Markdown
Contributor

Purpose

AppendTableITCase.testCompactionInStreamingMode, and its sibling testCompactionInStreamingModeWithMaxWatermark, are flaky. They poll for up to 60 s for a COMPACT snapshot from a streaming append pipeline, and under CI load the first COMPACT snapshot occasionally arrives after the 60 s cut-off, so the test fails with Time up for streaming execute, don't get expected result.

On a healthy run the snapshot lands in under 20 s: the streaming job starts, four small files accumulate at about one per second, the coordinator picks them up on its next discovery cycle, compaction runs, and the compacted snapshot commits. So 60 s is normally comfortable. The failure is a benign wall-clock race in job deployment and checkpoint/commit progress under CI contention, not the file-count gate. #1634 floated lowering compaction.min.file-num, but that only trims the roughly 4 s accumulation phase and does not touch the part that actually stalls. Raising the wait to 5 minutes, the value the rest of paimon-flink already uses for generous waits, gives the rare tail room to finish while still failing on a genuine hang.

This closes #10183, the same flake reported earlier in #1634.

Tests

No new tests. This only raises the assertStreamingHasCompact timeout from 60 s to 5 minutes in the three AppendTableITCase copies (paimon-flink-1.18, paimon-flink-1.19, paimon-flink-common); the behavior under test is unchanged.

API and Format

No.

Documentation

No.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] AppendTableITCase.testCompactionInStreamingMode is flaky and can time out under CI load

1 participant