test(integration): stop one failed SQL Server connect from failing every test

[samtrion]

Summary

A single failed connect to the SQL Server test container failed 96 net10.0 integration tests at once in run 36300167080 (PR #799, an unrelated change). The cause is SqlClient's connection pool blocking period, which turned one failure into 96. This PR stops that amplification, retries failed connects, and adds the container output to the failure message so the next failure shows its cause.

Root cause (with evidence)

From the log of job 108566282533 (attempt 1):

• failed: 96, all on net10.0 and all InvalidOperationException: Failed to create SQL Server test database .... net8.0 and net9.0 passed. The log shows 192 matching lines because each failure prints its message twice.
• Exactly one of the 96 is a real connect attempt:

  SqlException: A network-related or instance-specific error ... (provider: TCP Provider, error: 35 - An internal exception was caught)
     at Microsoft.Data.SqlClient.TdsParser.Connect(...)
     ...
     at Microsoft.Data.SqlClient.ConnectionPool.WaitHandleDbConnectionPool.CreateObject(...)
   ---> System.Net.Sockets.SocketException: Success
     at Microsoft.Data.SqlClient.ManagedSni.SniTcpHandle.Connect(...)
  

• The other 95 never touched the network. The pool served each of them the cached exception:

     at Microsoft.Data.SqlClient.ConnectionPool.BlockingPeriodErrorState.ThrowIfActive()
     at Microsoft.Data.SqlClient.ConnectionPool.WaitHandleDbConnectionPool.TryGetConnection(...)
  

  All 96 failures were reported within about 2 s, about 3 minutes into the net10.0 run.
• Every test database is created through the container's single master connection string, so they all share one pool. According to SqlConnectionStringBuilder.PoolBlockingPeriod, after a failed open "subsequent connection attempts will fail for the next five seconds ... up to a maximum of one minute". The default Auto disables this only for Azure SQL endpoints, so it applies to 127.0.0.1.
• SocketException: Success comes from managed SNI (SniTcpHandle.Connect). Socket.Select returned, but the socket was not connected and SO_ERROR was 0. SqlClient does not treat this error 35 as transient (dotnet/SqlClient#4665), so nothing retried it.
• No [SqlServerContainerFixture] ... crashed during startup line appears, so the container started normally and the test(integration): survive intermittent SQL Server 2025 startup crashes #796 startup retry did not apply. Passing tests are not printed, so the log cannot show whether the container was still alive after the failed connect (a crash during the run, as in [2025] Intermittent sqlpal crash on startup , ubuntu-24.04 only (~1.7% of starts, ~8.3% under concurrency) microsoft/mssql-docker#974). That state was not captured. This PR captures it from now on.

Local reproduction with Microsoft.Data.SqlClient 7.1.0 (3 pooled opens against a closed port, Connect Timeout=3):

--- default
open 1: 3135 ms, served from blocking period: False
open 2: 0 ms, served from blocking period: True
open 3: 0 ms, served from blocking period: True
--- Pool Blocking Period=NeverBlock
open 1: 3001 ms, served from blocking period: False
open 2: 3004 ms, served from blocking period: False
open 3: 3012 ms, served from blocking period: False

Changes

• SqlServerContainerFixture.ConnectionString appends Pool Blocking Period=NeverBlock. The per-test database connection strings are derived from it, so they get it as well.
• SqlServerDatabaseServiceFixture retries a failed OpenAsync up to 5 times with 1 s to 4 s backoff. The existing error 1807 retry stays the same, and both share the same attempt budget.
• If connecting still fails after all attempts, later fixtures fail immediately with that exception. Creation is serialized, so without this every remaining fixture would wait through the same connect timeouts against a dead server, which could hit the job timeout.
• The failure message now includes the last 40 lines of the container output (GetLogsAsync). That shows whether SQL Server crashed or the problem was on the network side.
• The test(integration): survive intermittent SQL Server 2025 startup crashes #796 startup crash retry is unchanged.

Test evidence

• dotnet build Pulse.slnx -c Release: 0 warnings, 0 errors. CSharpier formatted.
• CI Run Tests on 03ad4cf, 3 runs: 9285/9285 passed each time (jobs 108570481884, 108573560556, 108577188666). None logged a startup crash or a database creation failure.
• The same PR fix(postgresql): use TimeProvider instead of NOW() for outbox UpdatedAt #799 run that failed passed on its rerun (attempt 2), which fits a single transient connect failure that was amplified.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.82%. Comparing base (52d7df1) to head (b2ecf63).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #805      +/-   ##
==========================================
+ Coverage   95.75%   95.82%   +0.07%     
==========================================
  Files         270      270              
  Lines       10475    10475              
  Branches      953      953              
==========================================
+ Hits        10030    10038       +8     
+ Misses        241      232       -9     
- Partials      204      205       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.