Add azure.ai.training extension

[saanikaguptamicrosoft]

Notes

• Relevant PRs (refer PR overview for details)
  • Add artifact resolution for code and inputs in job create #7153
  • Implement cancel job command for custom training #7272
  • Implement delete job command for custom training #7273
  • Add validate command for custom training #7407 , Share offline validation between job validate and job submit commands #8068
  • Implement stream command for training job #7939
  • Implement connect-ssh to job node #7964
  • Implement download command for training job #8029
  • Add support for gpuCount for partial SKU scenario #8067
  • Bugfix: Honor --subscription and --project-endpoint flags over stored env values #8093
  • Implement show services command #8121
  • Add validations for UAMI requirement #8122
  • Add template flag in init command #8123

Testing

• Help command

• Init command

• Job submit

• Job show

• Jobs list

Other operations like validate, download, stream, delete, cancel, show-services, connect-ssh are also supported.

[jongio]

Addresses my prior feedback - TenantId fix, go version bump, SDK v2, and semver pin all look correct. One remaining item: the PR body is still empty. For a 14K-line new extension, even a brief summary (what training jobs are, how this relates to the finetune extension, key design decisions) would help reviewers and show up in the git log.

[wbreza]

Test comment

[wbreza]

Code review — supplementing prior reviews

Thanks for the substantial work here, @saanikaguptamicrosoft. Below are findings that aren't already covered by Copilot's or @jongio's earlier comments. Verified that the UserTenantId fix, go 1.26.1 bump, semver pin, and CODEOWNERS scope are all in place. ✅

🟧 High

1. --debug flag prints HTTP request URLs & bodies to stdout — risk of leaking SAS URIs / sensitive payloads
pkg/client/client.go:91–103. SetDebugBody() is called from production commands (job_submit.go:93, job_stream.go, job_download.go, job_connect_ssh.go) when --debug is passed. Logs include URLs (which for dataset blob/SAS operations may include SAS tokens) and request bodies that flow through GetDatasetCredentials / GetModelCredentials. Repo guidance forbids logging credentials/tokens.
Fix: Redact sasUri, Authorization, sig, signature, and known credential fields before printing; or move this behind a separate developer-only env var and document that it may expose tokens.

🟨 Medium

2. SSH proxy stdin goroutine intentionally untracked
internal/cmd/job_ssh_proxy.go:167–182. The upstream stdin → websocket goroutine is not added to the sync.WaitGroup and blocks on os.Stdin.Read(). The comment acknowledges the Windows limitation and relies on process exit. Works for the short-lived ProxyCommand model, but wg.Wait() at line 211 cannot guarantee clean shutdown on graceful cancellation.
Fix (optional): Close os.Stdin from the cancellation watcher to force Read() to return, or document the lifecycle expectation more visibly.

3. azcopy runner silently drops malformed JSON and parse errors
internal/azcopy/runner.go:167 (JSON unmarshal) and :175–177 (ParseInt). On azcopy output format drift, progress reporting silently degrades to "0 bytes" instead of surfacing the problem.
Fix: Log a one-time warning on first unparseable line; preserve raw text for diagnostics.

4. Nil-guard / pattern duplication on job.Properties.Services access
internal/service/stream_service.go:100, internal/cmd/job_download.go:350 (extractTrackingEndpoint), internal/cmd/job_show_services.go:45. Three inline reimplementations of "extract service endpoint" without consistent nil guards. Sparse API responses can omit Properties or Services.
Fix: Centralize as internal/utils/ExtractServiceEndpoint(job, name) with proper nil checks; replace the three call sites.

5. Critical untested code paths
Production files with zero test coverage: internal/service/upload_service.go (dedup + zombie recovery + sentinel tagging), internal/service/hash.go (deterministic directory hashing — drives dedup correctness), pkg/client/download.go (artifact/output downloads). The team's existing test files (resolver_test.go, job_validator_test.go, job_show_services_test.go) demonstrate solid patterns with fakes — this is a coverage gap on the most failure-prone code.
Fix (recommended before merge): Unit tests for hash.go (determinism + symlink + cross-platform path normalization) and upload_service.go (hash collision fallback, missing SAS URI, azcopy failure).

6. PR description is still empty
Reiterating @jongio's request — for a 14K-line new first-party extension, the empty description hurts git log archaeology, future reviewers, and release notes. A brief summary of what training jobs are, how this relates to azure.ai.finetune, key design decisions, and a pointer to the included Design/ docs would help significantly.

7. PR description references job get, but the actual command is job show
internal/cmd/job.go:32–42 registers newJobShowCommand() (defined in job_get.go). There is no newJobGetCommand. Commit history shows a rename from job get → job show. The PR description and screenshots still reference job get.
Fix: Update PR description and any docs (including content in Design/command-job-guide.md) to match the shipped command names.

🟦 Low

8. extension.yaml is missing an example for the metadata capability (other first-party extensions list examples for every declared capability).
9. proxyEndpointPattern (job_ssh_proxy.go) allows %; if any downstream consumer URL-decodes the endpoint, this could enable host/scheme bypass. Quick audit of buildTunnelWSURL recommended.
10. internal/azcopy/runner.go sourceArg lacks defense-in-depth shell-metachar validation (#nosec G204 only covers azcopyPath).
11. AdditionallyAllowedTenants: []string{"*"} in init.go:86 and job_ssh_proxy.go:265 is intentional for multi-tenant/guest support but should carry an inline security comment explaining the wide scope.
12. Missing godoc on exported symbols in pkg/client/ and pkg/models/ per repo standards.
13. ~140 KB of Design/ markdown shipped in the extension source tree (design.md 75 KB, execution-plan.md 40 KB, command-job-guide.md 25 KB). Will drift from code without a sync mechanism. Consider moving to repo-level docs/ or wiki, or trimming to a README link.
14. Client init boilerplate duplicated across ~15 command files. Extract newDefaultJobClient(ctx) helper to centralize credential/endpoint/client construction.

✅ Verified clean / resolved

• Subscription.UserTenantId correctly used at init.go:105.
• CODEOWNERS scope creep: only the training-extension line added — no co-owner additions to unrelated paths.
• go.mod is go 1.26.1 with semver-pinned azd dependency v1.25.1, aligned with extension.yaml's requiredAzdVersion.
• safeJoin in download.go correctly prevents zip-slip.
• resolver_test.go, job_validator_test.go, job_show_services_test.go, init_template_test.go are solid examples.
• All 9 required CI checks passing.

Posted by Copilot CLI on behalf of the reviewer.

[saanikaguptamicrosoft]

@wbreza

1. Traced every c.debugBody print site — only control-plane URLs (*.api.azureml.ms, auth via Authorization header) and request bodies (job definitions) are logged; SAS URIs come back in response bodies which are never logged, and GetBlobContent's SAS-bearing request URL has no debug print at all, so no token is exposed today.
2. az AML CLI takes the identical approach: stdin reader runs untracked, blocked in a blocking read, reaped by process exit when the short-lived ProxyCommand subprocess terminates. There's no portable way to interrupt os.Stdin.Read() on Windows, and az ml deliberately doesn't try. The existing comment block at lines 127–133 documents this; keeping as-is for consistency with the canonical AML SSH proxy behavior.
3. Addressed
4. Addressed
5. Added unit tests for hash.go (8 cases: determinism, order/content/path sensitivity, symlink skip, missing dir) and upload_service.go (12 cases including dedup hit, zombie recovery, hash collision, missing SAS URI, and azcopy failure propagation) — extracted small unexported interfaces on UploadService so the flow is testable without an HTTP server. Deferred pkg/client/download.go (HTTP plumbing — best covered via integration) to a follow-up.
6. PR description is added
7. Good catch — renamed job_get.go → job_show.go. PR already mentions job show.
8. Fixed 8 and 13. Other low priority skipped.