Skip to content

Fix GUI freeze on stop and prevent corrupted XDF footers - #146

Open
sappelhoff wants to merge 2 commits into
labstreaminglayer:masterfrom
sappelhoff:fix/instant-shutdown-and-cv
Open

sappelhoff wants to merge 2 commits into
labstreaminglayer:masterfrom
sappelhoff:fix/instant-shutdown-and-cv

Conversation

@sappelhoff

@sappelhoff sappelhoff commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Problem

When stopping a recording, recording::~recording() executes synchronously on the Qt UI thread. Because record_offsets() used an uninterruptible 5-second sleep (std::this_thread::sleep_for(offset_interval)) and boundary_thread joined with a 15-second timeout, the Qt event loop blocked, causing Windows to mark the window as "(Not Responding)" for 5–15+ seconds. Forcefully terminating the application when frozen aborted before stream footers could be written, resulting in corrupted XDF footers and truncated files.

Solution

  1. Interruptible Thread Teardown: Added a std::condition_variable shutdown_cv_ in recording and replaced std::this_thread::sleep_for / sleep_until across record_offsets(), record_boundaries(), and typed_transfer_loop() with condition variable waits. When shutdown is triggered, all worker threads wake up in < 1 ms.
  2. Active Socket Teardown: Track active lsl::stream_inlet instances and invoke in->close_stream() on shutdown to abort any blocking TCP socket calls or unreachable remote network routes immediately.
  3. Low-Latency First-Sample Check: Lowered the initial in->pull_sample() timeout from 4.0s to 0.1s so silent or late-starting streams check shutdown_ rapidly.
  4. Automated Integration Test: Added scripts/test_recording_teardown.py to verify that LabRecorderCLI stops in < 0.5s and produces fully valid, uncorrupted XDF files with intact footers.

Verification

  • Tested on Windows and Ubuntu via GitHub Actions CI and local automated integration tests.
  • Confirmed teardown completes in ~0.5s and resulting .xdf parses cleanly with pyxdf.

@sappelhoff
sappelhoff force-pushed the fix/instant-shutdown-and-cv branch from 01f99fe to c5d3038 Compare August 27, 2026 08:24
@cboulay

cboulay commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Thanks for tackling the stop latency. I recommend addressing these before merging:

  1. Missed shutdown notifications (high priority). shutdown_ and offset_shutdown are changed without holding shutdown_mut_, while the condition-variable predicates use that mutex. A notifier can set the flag and call notify_all() after the waiter reads false but before it enters the wait. The offset thread can then sleep for the full five-second interval. I reproduced this scheduling pattern in a standalone C++ harness. Update the predicate state under the same mutex, then notify, including both the normal and exception paths.

  2. The unreachable-stream stop hang is not fully resolved. close_stream() in liblsl v1.18.0.b3 stops the data receiver; it does not cancel the separate metadata receiver. record_from_streaminfo() still calls in->info() with its default infinite timeout, including after an interrupted open_stream(). With an unreachable metadata endpoint, Stop can therefore still block indefinitely. Also, the existing try_join_once() calls blocking std::thread::join(), so the reduced max_join_wait does not bound this wait. Please use bounded metadata waits that observe shutdown and explicitly handle stopping during subscription/header retrieval.

  3. Validation does not cover these guarantees yet. The new integration script is not called by the checked-in CI workflow. It verifies only the EEG footer, not the silent marker stream's footer, and permits 1.5 seconds despite documenting a 500 ms limit. Please run it in CI, check both footers and process exit status, and add stop-during-connect/metadata and repeated shutdown cases.

Review scope: source inspection against the PR head and liblsl v1.18.0.b3, plus the isolated condition-variable reproduction; I did not run the full integration script.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants