Skip to content

fix(tests): fix flaky test_capture_pane_flags[join_wrapped_numbers] - #655

Merged
tony merged 2 commits into
masterfrom
fix/flaky-join-wrapped-numbers-test
Nov 8, 2026
Merged

tony merged 2 commits into
masterfrom
fix/flaky-join-wrapped-numbers-test

Conversation

@tony

@tony tony commented Apr 1, 2026

Copy link
Copy Markdown
Member

Summary

  • Fix race condition in test_capture_pane_flags[join_wrapped_numbers] where the completion marker was detected in the shell command echo before the command actually executed

Test plan

  • uv run py.test tests/test_pane_capture_pane.py -v — all 19 tests pass
  • CI passes

Fixes #654

@tony
tony marked this pull request as ready for review April 1, 2026 23:58
@codecov

codecov Bot commented Apr 1, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 52.45%. Comparing base (15f1e6e) to head (2a4819d).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #655   +/-   ##
=======================================
  Coverage   52.45%   52.45%           
=======================================
  Files          26       26           
  Lines        3729     3729           
  Branches      747      747           
=======================================
  Hits         1956     1956           
  Misses       1469     1469           
  Partials      304      304           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tony
tony force-pushed the fix/flaky-join-wrapped-numbers-test branch from 0cecae7 to 7bb49f0 Compare April 26, 2026 10:22
@tony
tony force-pushed the fix/flaky-join-wrapped-numbers-test branch from 7bb49f0 to 33e94ab Compare May 10, 2026 16:02
@tony
tony force-pushed the fix/flaky-join-wrapped-numbers-test branch from 33e94ab to cb5ebd4 Compare May 30, 2026 11:54
@tony
tony force-pushed the fix/flaky-join-wrapped-numbers-test branch from cb5ebd4 to 6c451e1 Compare July 4, 2026 13:40
@tony
tony force-pushed the fix/flaky-join-wrapped-numbers-test branch from 6c451e1 to a468e8e Compare July 18, 2026 11:02
tony added a commit that referenced this pull request Oct 2, 2026
why: Four tests read a pane before the state they assert had arrived,
or waited on text the shell echoes back as it is typed. Under 40 busy
loops: `test_capture_pane`, `_start` and `_end` failed 10 of 192 runs
(each asserts the pane's exact text straight after `new_window` or
`send_keys`), and `test_capture_pane_flags[join_wrapped_numbers]` failed
3 of 96, because the echoed command line already contains the "done"
marker. #655 skips echo lines that start with "$", which holds only
while the echoed command fits on one line: with a 40-column pane the
wrapped continuation line, which has no "$", still matches. The
`automation_patterns` doctest has the same flaw, plus a single-shot read
after the marker: it failed on every run on a machine with a slow login
shell.

what:
- Poll each exact-text assertion in the three `capture_pane` tests until
  the pane holds the expected text, then assert it
- Type the `capture_pane_flags` marker as `"__DONE_""id__"` so only the
  command's output contains it
- Rewrite the "Capturing output between markers" example to wait for an
  end marker printed last, with split markers, and say why

Verified under the same load: 0 of 192 for each test file, and 0 of 24
for the doctest (24 of 24 before).
@tony
tony force-pushed the fix/flaky-join-wrapped-numbers-test branch from a468e8e to 3dd988f Compare November 7, 2026 02:18
@tony tony changed the title fix(tests): fix flaky test_capture_pane_flags[join_wrapped_numbers] fix(tests): fix flaky test_capture_pane_flags[join_wrapped_numbers] Nov 8, 2026
tony added 2 commits November 7, 2026 18:21
why: The command_complete() check matched the marker string inside the shell
command echo line ($ printf ... echo "__DONE_...") before the command actually
executed. On slower systems or CI, this race condition caused the test to
capture the pane before seq produced output.
what:
- Change marker detection to skip lines starting with $ (shell prompt echo)
- Marker now only matches when it appears as actual command output

Fixes #654
@tony
tony force-pushed the fix/flaky-join-wrapped-numbers-test branch from 3dd988f to 2a4819d Compare November 8, 2026 00:21
@tony
tony merged commit bfad66b into master Nov 8, 2026
15 checks passed
@tony
tony deleted the fix/flaky-join-wrapped-numbers-test branch November 8, 2026 00:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Flaky test: test_capture_pane_flags[join_wrapped_numbers] matches marker in command echo

1 participant