Fix connection data race probably causing flaky tests

[KonradBreitsprecherBkd]

Fix data race in ConnectKnownParticipants::UpdateStage

Problem

ITest_Internals_ParticipantModes.test_SyncAutonomousReq was intermittently failing with a std::future_error (promise_already_satisfied) thrown from VAsioConnection::OnConnectKnownParticipantsWaitingForAllReplies at _startWaitingForParticipantHandshakes.set_value().

Root cause

ConnectKnownParticipants::UpdateStage() performs a non-atomic check-then-set on _connectStage. Although _connectStage is std::atomic, the existing _mutex only guards the _peers iteration inside the lambdas — it never covers the stage transition itself.

UpdateStage() can run concurrently on two threads:

• the caller thread, via JoinSimulation → ConnectToKnownParticipants → StartConnecting (which calls UpdateStage() after issuing the async connects); and
• the IO worker thread, via a connect callback (OnConnectPeerSuccess → UpdateStage()).

Because the IO worker is already running the shared _asioIoContext when StartConnecting issues the connects, a fast (local domain-socket) connect can complete and drive the IO-thread UpdateStage() while the caller-thread UpdateStage() is still mid-transition. Both threads read _connectStage == CONNECTING, both observe allWaitingForReplies == true, and both fire OnConnectKnownParticipantsWaitingForAllReplies() → set_value() twice. The second call throws.

The try/catch(...) around set_value() swallows the exception, so it does not crash the process, but the underlying data race on _connectStage is undefined behavior and the real source of the flakiness. The WAITING_FOR_ALL_REPLIES → ALL_REPLIES_RECEIVED transition has the same exposure.

Fix

Serialize the entire UpdateStage() transition body with a dedicated _stageMutex, so the check-then-set runs atomically. Once one thread completes a transition, the next re-reads the already-advanced _connectStage and skips the block, so each stage callback fires exactly once.

Lock ordering is consistent (_stageMutex → _mutex); no listener callback invoked within UpdateStage() re-enters ConnectKnownParticipants, so there is no deadlock or recursion risk.

Changes

• ConnectKnownParticipants.hpp: add std::mutex _stageMutex.
• ConnectKnownParticipants.cpp: acquire _stageMutex at the top of UpdateStage().

Testing

• Rebuild and run ITest_Internals_ParticipantModes.test_SyncAutonomousReq in a loop to confirm the flake no longer reproduces.

[KonradBreitsprecherBkd]

Spun again on Debug and Release for 10k Times each with stress, could no longer reproduce the initial hang.

[KonradBreitsprecherBkd]

Took the wrong route here, this was NOT the cause of the hanging test. Although promise_already_satisfied is thrown in rare cases, this is catched and does not cause a deadlock or any hanging test.