Test: remove flaky markers from stabilised tests - #4483
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 64a6d241-8263-4451-b207-65459e29a980
There was a problem hiding this comment.
Pull request overview
Removes the xUnit flaky quarantine at the DateTimeVariantTests class level so stabilized DateTime/sql_variant parameter tests are eligible to run in the standard CI test set, while keeping the one known environment-sensitive test quarantined.
Changes:
- Removed
[Trait("Category", "flaky")]fromDateTimeVariantTestsso other tests in the class are no longer globally quarantined. - Added
[Trait("Category", "flaky")]specifically toTestSqlDataReaderParameterToTVP_Variantto keep only that test excluded from the default CI filter.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 64a6d241-8263-4451-b207-65459e29a980
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: e3e5c96e-86fb-4c23-a009-ebfcc9898f04
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (9)
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/DateTimeVariantTests.cs:166
- This test method is still marked
[Trait("Category", "flaky")], which conflicts with the PR description stating the stabilized DateTimeVariant tests should run in regular CI (onlyTestSqlDataReaderParameterToTVP_Variantshould remain flaky). Remove the flaky trait here so it is no longer quarantined.
[Trait("Category", "flaky")]
[ConditionalTheory(typeof(DataTestUtility), nameof(DataTestUtility.AreConnStringsSetup), nameof(DataTestUtility.IsNotAzureSynapse))]
[MemberData(nameof(GetParameterCombinations), DisableDiscoveryEnumeration = true)]
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/DateTimeVariantTests.cs:203
- This test method is still marked
[Trait("Category", "flaky")], which conflicts with the PR description stating the stabilized DateTimeVariant tests should run in regular CI (onlyTestSqlDataReaderParameterToTVP_Variantshould remain flaky). Remove the flaky trait here so it is no longer quarantined.
[Trait("Category", "flaky")]
[ConditionalTheory(typeof(DataTestUtility), nameof(DataTestUtility.AreConnStringsSetup), nameof(DataTestUtility.IsNotAzureSynapse))]
[MemberData(nameof(GetParameterCombinations), DisableDiscoveryEnumeration = true)]
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/DateTimeVariantTests.cs:247
- This test method is still marked
[Trait("Category", "flaky")], which conflicts with the PR description stating the stabilized DateTimeVariant tests should run in regular CI (onlyTestSqlDataReaderParameterToTVP_Variantshould remain flaky). Remove the flaky trait here so it is no longer quarantined.
[Trait("Category", "flaky")]
[ConditionalTheory(typeof(DataTestUtility), nameof(DataTestUtility.AreConnStringsSetup), nameof(DataTestUtility.IsNotAzureSynapse))]
[MemberData(nameof(GetParameterCombinations), DisableDiscoveryEnumeration = true)]
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/DateTimeVariantTests.cs:291
- This test method is still marked
[Trait("Category", "flaky")], which conflicts with the PR description stating the stabilized DateTimeVariant tests should run in regular CI (onlyTestSqlDataReaderParameterToTVP_Variantshould remain flaky). Remove the flaky trait here so it is no longer quarantined.
[Trait("Category", "flaky")]
[ConditionalTheory(typeof(DataTestUtility), nameof(DataTestUtility.AreConnStringsSetup), nameof(DataTestUtility.IsNotAzureSynapse))]
[MemberData(nameof(GetParameterCombinations), DisableDiscoveryEnumeration = true)]
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/DateTimeVariantTests.cs:387
- This test method is still marked
[Trait("Category", "flaky")], which conflicts with the PR description stating the stabilized DateTimeVariant tests should run in regular CI (onlyTestSqlDataReaderParameterToTVP_Variantshould remain flaky). Remove the flaky trait here so it is no longer quarantined.
[Trait("Category", "flaky")]
[ConditionalTheory(typeof(DataTestUtility), nameof(DataTestUtility.AreConnStringsSetup), nameof(DataTestUtility.IsNotAzureSynapse))]
[MemberData(nameof(GetParameterCombinations), DisableDiscoveryEnumeration = true)]
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/DateTimeVariantTests.cs:463
- This test method is still marked
[Trait("Category", "flaky")], which conflicts with the PR description stating the stabilized DateTimeVariant tests should run in regular CI (onlyTestSqlDataReaderParameterToTVP_Variantshould remain flaky). Remove the flaky trait here so it is no longer quarantined.
[Trait("Category", "flaky")]
[ConditionalTheory(typeof(DataTestUtility), nameof(DataTestUtility.AreConnStringsSetup), nameof(DataTestUtility.IsNotAzureSynapse))]
[MemberData(nameof(GetParameterCombinations), DisableDiscoveryEnumeration = true)]
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/DateTimeVariantTests.cs:539
- This test method is still marked
[Trait("Category", "flaky")], which conflicts with the PR description stating the stabilized DateTimeVariant tests should run in regular CI (onlyTestSqlDataReaderParameterToTVP_Variantshould remain flaky). Remove the flaky trait here so it is no longer quarantined.
[Trait("Category", "flaky")]
[ConditionalTheory(typeof(DataTestUtility), nameof(DataTestUtility.AreConnStringsSetup), nameof(DataTestUtility.IsNotAzureSynapse))]
[MemberData(nameof(GetParameterCombinations), DisableDiscoveryEnumeration = true)]
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/DateTimeVariantTests.cs:597
- This test method is still marked
[Trait("Category", "flaky")], which conflicts with the PR description stating the stabilized DateTimeVariant tests should run in regular CI (onlyTestSqlDataReaderParameterToTVP_Variantshould remain flaky). Remove the flaky trait here so it is no longer quarantined.
[Trait("Category", "flaky")]
[ConditionalTheory(typeof(DataTestUtility), nameof(DataTestUtility.AreConnStringsSetup), nameof(DataTestUtility.IsNotAzureSynapse))]
[MemberData(nameof(GetParameterCombinations), DisableDiscoveryEnumeration = true)]
src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/ParameterTest/DateTimeVariantTests.cs:129
- This test method is still marked
[Trait("Category", "flaky")], which conflicts with the PR description stating the stabilized DateTimeVariant tests should run in regular CI (onlyTestSqlDataReaderParameterToTVP_Variantshould remain flaky). Remove the flaky trait here so it is no longer quarantined.
This issue also appears in the following locations of the same file:
- line 164
- line 201
- line 245
- line 289
- line 385
- ...and 3 more
[Trait("Category", "flaky")]
[ConditionalTheory(typeof(DataTestUtility), nameof(DataTestUtility.AreConnStringsSetup), nameof(DataTestUtility.IsNotAzureSynapse))]
[MemberData(nameof(GetParameterCombinations), DisableDiscoveryEnumeration = true)]
CI Failure:
|
priyankatiwari08
left a comment
There was a problem hiding this comment.
CI is failing on macOS Set 3 (net8.0 + net9.0) due to SqlBulkCopyDataRow_Type — please address the comment above before merging.
Summary
DateTimeVariantTestsso the nine stabilised test methods run in regular CI.TestSqlDataReaderParameterToTVP_Variant, which was not in the stabilisation set and remains environment-sensitive.xsqlbecause it is a private helper and has no flaky marker of its own.Validation
dotnet build src/Microsoft.Data.SqlClient/tests/ManualTests/Microsoft.Data.SqlClient.ManualTests.csproj --no-restore --configuration Release --framework net8.0 --verbosity minimal --maxcpucount:1Checklist