Add sqlclient-ci-managed-instance CI pipeline

[paulmedynski]

Description

Adds an internal, package-triggered CI pipeline that runs the SqlClient Unit, Functional, and Manual test suites against an Azure SQL Managed Instance in Package reference mode, based on the exported Classic Test-SqlClient-Managed-Instance pipeline.

Managed Instance pipeline

• Adds the pipeline, stage, and job templates under eng/pipelines/ci/managed-instance/.
• Triggers after successful sqlclient-ci-package runs; it has no PR, commit, or schedule trigger.
• Is registered only in the internal ADO.Net project because it depends on the Managed-Instance-pool, MI connection strings from the ADO Test Configuration Properties variable group, and the sqlclient-test-key.snk secure file.
• Runs 11 parallel OS/SNI/TFM jobs:
  • Windows native SNI: net462, net8.0, net9.0, and net10.0.
  • Windows managed SNI: net462, net8.0, net9.0, and net10.0.
  • Linux managed SNI: net8.0, net9.0, and net10.0.
• Runs the normal Unit, Functional, and Manual suites in each job, including the regular Always Encrypted coverage. Quarantined flaky suites are omitted from this pipeline.
• Builds tests with ReferenceType=Package against package versions resolved from the triggering artifact. Microsoft.SqlServer.Server is temporarily pinned to stable 1.0.0 to avoid the prerelease-versus-stable NU1605 downgrade.
• Downloads the dedicated test signing key so test assemblies can access internals in the signed package build.

Source and package alignment

• Clones the ADO.Net dotnet-sqlclient repo.
  • Note that the Classic pipeline cloned the GitHub dotnet/SqlClient repo.
• Adds a shared source-alignment step used by the Managed Instance and stress pipelines. It checks out the upstream package run’s source commit so tests compile against the matching API surface.
• Restores eng/pipelines from the queued pipeline commit after alignment so file-based tasks and compiled YAML remain in lockstep.
• Adds a shared package-download step and documented sibling PowerShell script to stage .nupkg/.snupkg files and publish the resolved package versions as pipeline variables.
• Refactors sqlclient-ci-stress to reuse the shared source-alignment and package-staging steps.

Shared test configuration

• Adds runFlakyTests to run-all-tests-step.yml, defaulting to the existing behavior; the Managed Instance job sets it to false.
• Adds IsManagedInstance to update-config-file-step.yml so tests can opt into Managed-Instance-specific behavior.

No product code or public API changes are included.

Note: This pipeline targets main. Ports to release/7.0 and release/6.1 will be handled separately because those branches predate the sqlclient-ci-package single-artifact model and require scheduled, build-from-source variants.

Testing

• sqlclient-ci-managed-instance run 21102 completed successfully across all 11 OS/SNI/TFM jobs. Each job completed package setup and the Unit, Functional, and Manual suites; no quarantined flaky tasks were generated.

[Copilot]

Pull request overview

Adds a new Azure DevOps CI pipeline to run SqlClient Unit/Functional/Manual tests against an Azure SQL Managed Instance in Package reference mode, triggered by sqlclient-ci-package, and refactors package-download/version-resolution logic into a shared template to reduce duplication.

Changes:

• Introduces a new managed-instance pipeline trio under eng/pipelines/ci/managed-instance/ (pipeline + stage + job templates).
• Adds a shared download-driver-packages-step.yml template and updates the existing stress CI job to reuse it.
• Extends update-config-file-step.yml with an IsManagedInstance switch to enable MI-specific test behavior.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
eng/pipelines/common/templates/steps/update-config-file-step.yml	Adds IsManagedInstance parameter and writes it into generated test config.
eng/pipelines/common/steps/download-driver-packages-step.yml	New reusable step to download driver package artifacts and resolve package versions into pipeline variables.
eng/pipelines/ci/stress/sqlclient-ci-stress-job.yml	Refactors to use the new shared download/resolve template and consumes the new version variable name.
eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-stage.yml	New stage that fans out MI test jobs across Windows (native/managed SNI) and Linux.
eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-pipeline.yml	New pipeline definition triggered by sqlclient-ci-package runs.
eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-job.yml	New job template that sets up MI test config and runs the test suites across the requested TFMs.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

eng/pipelines/common/steps/download-driver-packages-step.yml:109

• After updating Resolve-PackageVersion to operate on the artifact directory, this call site should pass $artifactDir (the downloaded artifact contents) instead of $feed to avoid being affected by any pre-existing packages in the local feed.

        foreach ($p in $packages) {
            $version = Resolve-PackageVersion $feed $p.Pattern $p.Name
            Write-Host "Resolved $($p.Name) version: $version"
            Write-Host "##vso[task.setvariable variable=$($p.Variable)]$version"

eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-pipeline.yml:45

• Minor grammar issue in the parameter comment ("building the;") makes the pipeline description harder to read in the UI.

  # The build configuration to use when building the; defaults to Release.

eng/pipelines/common/steps/download-driver-packages-step.yml:74

• Resolve-PackageVersion searches in the destination feed directory, which may already contain older .nupkg files on self-hosted agents. That can cause non-deterministic version selection via Select-Object -First 1 and pass the wrong versions to downstream steps. Resolve versions from the just-downloaded artifact directory and fail if there isn’t exactly one match.

This issue also appears on line 106 of the same file.

        # Resolve the version of a single package from its .nupkg filename in the feed.
        function Resolve-PackageVersion($feed, $pattern, $name) {
            $pkg = Get-ChildItem "$feed/*.nupkg" | Where-Object { $_.Name -match $pattern } | Select-Object -First 1
            if (-not $pkg) { throw "$name package not found in $feed" }
            return [regex]::Match($pkg.Name, $pattern).Groups[1].Value
        }

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.77%. Comparing base (25dba6b) to head (3941f5b).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4482      +/-   ##
==========================================
- Coverage   64.61%   62.77%   -1.84%     
==========================================
  Files         288      283       -5     
  Lines       44046    66979   +22933     
==========================================
+ Hits        28459    42048   +13589     
- Misses      15587    24931    +9344     

Flag	Coverage Δ
CI-SqlClient	?
PR-SqlClient-Project	62.77% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (4)

eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-pipeline.yml:45

• Minor grammar issue in this comment: "when building the;" is missing the noun after "the".

  # The build configuration to use when building the; defaults to Release.

eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-stage.yml:38

• YAML indentation under parameters.dotnetVerbosity.values is invalid: the list items are aligned with values: instead of being indented beneath it. This will break template parsing.

  - name: dotnetVerbosity
    type: string
    values:
    - quiet
    - minimal

eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-job.yml:46

• YAML indentation under parameters.dotnetVerbosity.values is invalid: the list items are aligned with values: instead of being indented beneath it. This will break template parsing.

  - name: dotnetVerbosity
    type: string
    values:
    - quiet
    - minimal

eng/pipelines/common/steps/download-driver-packages-step.yml:74

• Package version resolution scans the feed directory and takes the first match. On self-hosted agents (like Managed-Instance-pool) the feed can contain leftover .nupkg files from previous runs, making version resolution nondeterministic and potentially selecting the wrong package version. Resolve versions from the downloaded artifact directory and enforce a single match.

        # Resolve the version of a single package from its .nupkg filename in the feed.
        function Resolve-PackageVersion($feed, $pattern, $name) {
            $pkg = Get-ChildItem "$feed/*.nupkg" | Where-Object { $_.Name -match $pattern } | Select-Object -First 1
            if (-not $pkg) { throw "$name package not found in $feed" }
            return [regex]::Match($pkg.Name, $pattern).Groups[1].Value
        }

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-pipeline.yml:45

• Minor doc/comment typo: the sentence is missing the object (“building the …”).

  # The build configuration to use when building the; defaults to Release.

eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-job.yml:129

• git checkout --force $sha will fail the job if the resource commit SHA isn't present in the fetched refs (e.g., manual runs, cross-branch triggers, or refspec limitations). It would be more robust to verify the commit exists locally and log a warning (then continue) when it doesn't.

          $sha = "$(resources.pipeline.sqlclient-ci-package.sourceCommit)"
          if ([string]::IsNullOrWhiteSpace($sha)) {
              Write-Host "No sqlclient-ci-package sourceCommit available; using checked-out source as-is."
          } else {
              Write-Host "Aligning source to sqlclient-ci-package commit $sha"

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-pipeline.yml:45

• This comment has a grammatical omission (“building the;”). It’s unclear what is being built (tests/projects/packages).

  # The build configuration to use when building the; defaults to Release.

eng/pipelines/common/steps/align-source-with-upstream-step.yml:49

• The script assumes the upstream commit SHA is always present locally. If the resource provides a SHA that isn’t in the checkout, git checkout --force will fail even though the comment says the step should fall back to the current source. Consider verifying the commit exists (or catching the failure) and falling back to the checked-out source with a clear message.

      $sha = "$(resources.pipeline.${{ parameters.upstreamPipeline }}.sourceCommit)"
      if ([string]::IsNullOrWhiteSpace($sha)) {
          Write-Host "No ${{ parameters.upstreamPipeline }} sourceCommit available; using checked-out source as-is."
      } else {
          Write-Host "Aligning source to ${{ parameters.upstreamPipeline }} commit $sha"

eng/pipelines/common/steps/download-driver-packages-step.yml:74

• Resolve-PackageVersion selects the first matching .nupkg in the local feed, which is nondeterministic if multiple matching versions exist (e.g., from previous steps/cached artifacts). It’s safer to enforce a single match so the pipeline doesn’t silently pick an arbitrary version.

        function Resolve-PackageVersion($feed, $pattern, $name) {
            $pkg = Get-ChildItem "$feed/*.nupkg" | Where-Object { $_.Name -match $pattern } | Select-Object -First 1
            if (-not $pkg) { throw "$name package not found in $feed" }
            return [regex]::Match($pkg.Name, $pattern).Groups[1].Value
        }

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (5)

eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-pipeline.yml:45

• Comment has a grammatical omission: "building the;" should specify what is being built (e.g., "building the tests").

  # The build configuration to use when building the; defaults to Release.

eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-stage.yml:38

• The dotnetVerbosity parameter values list is mis-indented (values: is followed by list items at the same indentation level), which makes the YAML invalid and will prevent the pipeline from compiling.

    values:
    - quiet
    - minimal
    - normal
    - detailed

eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-job.yml:46

• The dotnetVerbosity parameter values list is mis-indented (values: is followed by list items at the same indentation level), which makes the YAML invalid and will prevent template expansion.

    values:
    - quiet
    - minimal
    - normal
    - detailed

eng/pipelines/common/steps/align-source-with-upstream-step.yml:49

• git checkout --force $sha can fail when the upstream sourceCommit isn't in the refs fetched by checkout: self (e.g., if the triggering pipeline ran on a different branch). The template currently treats only an empty SHA as 'unavailable', so a missing commit will fail the job instead of falling back to the checked-out source.

      $sha = "$(resources.pipeline.${{ parameters.upstreamPipeline }}.sourceCommit)"
      if ([string]::IsNullOrWhiteSpace($sha)) {
          Write-Host "No ${{ parameters.upstreamPipeline }} sourceCommit available; using checked-out source as-is."
      } else {
          Write-Host "Aligning source to ${{ parameters.upstreamPipeline }} commit $sha"
          git checkout --force $sha
      }

eng/pipelines/common/steps/download-driver-packages-step.yml:84

• Resolve-PackageVersion scans the local feed directory and picks the first match. Since feedPath may already contain other .nupkg files (from prior steps or cached artifacts), this can resolve the wrong version. Resolve versions from the downloaded artifact directory and fail if multiple matches are found.

        function Resolve-PackageVersion($feed, $pattern, $name) {
            $pkg = Get-ChildItem "$feed/*.nupkg" | Where-Object { $_.Name -match $pattern } | Select-Object -First 1
            if (-not $pkg) { throw "$name package not found in $feed" }
            return [regex]::Match($pkg.Name, $pattern).Groups[1].Value
        }

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

eng/pipelines/common/steps/download-driver-packages-step.yml:122

• After changing Resolve-PackageVersion to search the downloaded artifact directory, update the call site to pass $artifactDir (not $feed). Otherwise the function will still enumerate the feed and the determinism issue remains.

            } else {
                $version = Resolve-PackageVersion $feed $p.Pattern $p.Name
                Write-Host "Resolved $($p.Name) version: $version"

eng/pipelines/common/steps/download-driver-packages-step.yml:84

• Resolve-PackageVersion searches the feed directory for matching .nupkg files. On self-hosted agents (or when the repo already has packages in $(Build.SourcesDirectory)/packages), the feed can contain multiple versions, and Select-Object -First 1 can resolve the wrong version (leading to restore/build against a different package than the one just downloaded). Resolve versions from the downloaded artifact directory (or otherwise isolate/clean the feed) so the resolved version is deterministic and matches the upstream artifact.

This issue also appears on line 120 of the same file.

        # Resolve the version of a single package from its .nupkg filename in the feed.
        function Resolve-PackageVersion($feed, $pattern, $name) {
            $pkg = Get-ChildItem "$feed/*.nupkg" | Where-Object { $_.Name -match $pattern } | Select-Object -First 1
            if (-not $pkg) { throw "$name package not found in $feed" }
            return [regex]::Match($pkg.Name, $pattern).Groups[1].Value

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

eng/pipelines/common/steps/align-source-with-upstream-step.yml:46

• git checkout --force $sha can fail when the upstream run’s sourceCommit isn’t reachable from the downstream pipeline’s initially checked-out branch (Azure Pipelines typically fetches only the current ref, even with fetchDepth: 0). Consider fetching the upstream run’s sourceBranch before checking out the SHA, and enable persistCredentials so the fetch can authenticate. Also consider cleaning the working tree after checkout so untracked files from the initial checkout can’t affect builds.

  # Check out the repo with full history so the working tree can be reset to an arbitrary commit.
  - checkout: self
    fetchDepth: 0

  # Align the working-tree source with the commit that built the upstream artifacts, so the projects

eng/pipelines/common/steps/download-driver-packages.ps1:97

• Resolve-PackageVersion selects the first matching *.nupkg it finds. If the artifact directory ever contains multiple versions of the same package ID (or an extra copy), the resolved version becomes nondeterministic and could silently point the build at the wrong package. Consider enforcing exactly one match (or throwing with a clear error listing the matches).

    $package = Get-ChildItem "$Path/*.nupkg" |
        Where-Object { $_.Name -match $Pattern } |
        Select-Object -First 1

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

eng/pipelines/common/steps/align-source-with-upstream-step.yml:48

• The git checkout calls here don’t fail the step when they return a non-zero exit code (PowerShell doesn’t throw on native command failures by default). That means the pipeline can silently proceed without actually aligning to the upstream commit, defeating the purpose of this template. It also assumes the upstream SHA is already present locally; depending on what refs checkout: self fetched, that SHA may not exist and git checkout will fail.

Consider explicitly fetching the SHA and checking $LASTEXITCODE, and only treating checkout failure as non-fatal when you truly want to fall back to the queued commit.

  - pwsh: |
      $sha = "$(resources.pipeline.${{ parameters.upstreamPipeline }}.sourceCommit)"
      $pipelineSourceSha = git rev-parse HEAD
      Write-Host "Aligning source to ${{ parameters.upstreamPipeline }} commit $sha"
      git checkout --force $sha

eng/pipelines/common/steps/download-driver-packages.ps1:75

• Copy-Item ... -ErrorAction SilentlyContinue will also suppress non-terminating copy errors (not just the “no matches” case), which can hide real filesystem problems when symbol packages are present. Since .snupkg files are optional, it’s safer to only suppress the “not found” case while still failing on actual copy errors.

Copy-Item "$ArtifactDirectory/*.nupkg" $FeedPath -Force
Copy-Item "$ArtifactDirectory/*.snupkg" $FeedPath -Force -ErrorAction SilentlyContinue

[paulmedynski]

These 2 new shared steps will be used by all of the new CI pipelines that are downstream from sqlclient-ci-package. I updated sqlclient-ci-stress in this PR to prove it all out. Kerberos will be updated separately.

[paulmedynski]

This is necessary until #4336 completes.

[paulmedynski]

This is a legacy CI template, but it works fine for now. We plan to common-ify the modern mechanism that sqlclient-pr uses, but that is future work.

[paulmedynski]

This is critical - the pipeline was compiled from HEAD, so we need to restore that part of the tree to ensure helper scripts match HEAD.

[cheenamalhotra]

Nit: You could remove the "Run Managed Instance Tests" from here, since that's also in the stage name. Adding here makes the job name too long.

[cheenamalhotra]

Sql Server version should be fixed forever IMO, should not play with the version so much.