feat(appkit): migrate analytics to the modular @databricks/sdk-*

[MarioCadenas]

What

Migrates the analytics stack (SQLWarehouseConnector + type-generator) off the legacy monolithic @databricks/sdk-experimental onto the new modular per-service @databricks/sdk-* SDK (v0.46.0, ESM-only), behind the existing packages/shared/src/workspace-client/ facade seam.

Analytics doesn't touch the SDK directly — it delegates to SQLWarehouseConnector, which uses two services: warehouses and statementExecution. Both move together in this increment; every other service (files/genie/jobs/serving/currentUser/apiClient) still routes through the legacy client behind the same facade (the seam's designed incremental state).

How

• New boundary module packages/shared/src/workspace-client/modular.ts — the sole importer of @databricks/sdk-* (oxlint no-restricted-imports boundary), mirroring legacy.ts. Builds per-service WarehousesClient / StatementExecutionClient; maps wrapper options → ClientOptions (host scheme-normalization, credential resolution, and a httpClient transport that stamps the User-Agent).
• Connector + type-generator rewritten to the modular API: method renames (getStatement→getStatementResult, getStatementResultChunkN→getResultData), camelCase response model, CallOptions { signal } for cancellation.
• pnpm patch (patches/@databricks__sdk-statementexecution@0.46.0.patch) restores the undocumented Reyden attachment response field that the SDK's generated unmarshal transform would otherwise strip. Version-pinned — re-apply on any SDK bump; an upstream fix is preferred long-term.

Authentication (the foundation for migrating the remaining services)

Auth resolution is centralized in one service-agnostic function, mapToClientOptions(opts) in modular.ts, which produces the ClientOptions { host, credentials, httpClient } that every modular client constructor accepts. Warehouses and statementExecution build through it today; each remaining service will build through the same function, so the rest of the migration needs no per-service auth work (the credentials are generic bearer TokenCredentials, not service-scoped).

Resolution order — restores the legacy SDK's behavior for the modes AppKit uses:

• explicit token (asUser OBO) → PAT, guarded by token !== undefined so an empty/invalid OBO token fails loudly instead of silently falling through to the service principal (privilege-escalation guard);
• profile → profile file;
• otherwise the service principal from the environment — DATABRICKS_CLIENT_ID + DATABRICKS_CLIENT_SECRET → OAuth M2M (what the Databricks Apps runtime injects), else DATABRICKS_TOKEN → PAT, else the SDK's profile-based default chain.

Why resolve the SP explicitly rather than lean on the SDK's default chain: the default chain does read the DATABRICKS_* env vars, but its M2M strategy performs OAuth token-endpoint discovery against the raw DATABRICKS_HOST, which the Apps runtime injects scheme-less (e.g. e2-dogfood.staging.cloud.databricks.com). Discovery then throws Invalid URL and every deployed request fails as "Warehouse readiness check failed". Resolving the SP here with the scheme-normalized host avoids that (it's the credential-side facet of the host-normalization fix below). Verified end-to-end on a deployed app.

Not a full 1-to-1 with legacy's auth surface: the modular SDK 0.46.0 supports only PAT, OAuth M2M, and OAuth U2M — there are no factories for Azure (MSI/CLI/SP), GCP, or the instance-metadata service that legacy unified-auth covered. This is behaviorally equivalent for AppKit's real environments (Databricks Apps = SP via env; local dev = profile / PAT / env) but a reduced surface for those exotic modes; noted for release notes.

Per-service OBO scopes: the auth code is uniform, but each OBO-consuming service needs its matching user_api_scopes declared in the app's databricks.yml — sql (warehouse/statement), files.files, dashboards.genie, serving.serving-endpoints. That's app configuration, not auth code.

Runtime bugs found + fixed (caught against live warehouses / a deployed app, not unit tests)

1. Host scheme (requests) — the modular SDK passes host straight to fetch; a bare DATABRICKS_HOST → Invalid URL. Fixed via normalizeHost on the request host.
2. User-Agent — setProduct("@databricks/appkit") throws (the @// aren't token chars) and client-info is process-global. AppKit dashboards filter traffic on the literal @databricks/appkit UA, so we set it on a per-client httpClient transport wrapper instead — preserving the exact string. (A pnpm patch of the validator wouldn't reach deployed apps, which is where the dashboard traffic originates.)
3. ApiError.code — the modular ApiError carries the Databricks error code on .code, not .errorCode; the connector now reads either, so the arrow disposition/format capability-rejection fallback (INLINE↔EXTERNAL_LINKS) fires again.
4. BigInt serialization — the modular SDK types row/byte counts as bigint; Reyden's cached INLINE+ARROW_STREAM result then broke JSON.stringify. Counts are coerced back to number at the connector boundary.
5. Deployed service-principal auth — the SDK's M2M strategy builds credentials from the raw scheme-less DATABRICKS_HOST the Apps runtime injects, so OAuth token discovery failed (Invalid URL) and every deployed query died as "Warehouse readiness check failed". Fixed by resolving the SP with a scheme-normalized host (see Authentication).

Testing

• Full appkit + shared suites green, including new regression tests for the .code read, bigint normalization, the env-based credential resolution (M2M-from-env, PAT-from-env, OBO-token-wins, M2M-needs-host), and the @databricks/appkit User-Agent stamping.
• pnpm -r typecheck, lint, format clean; pnpm install --frozen-lockfile green (appkit importer synced in pnpm-lock.yaml).
• Verified live against standard + Reyden serverless warehouses: JSON + arrow (INLINE attachment and EXTERNAL_LINKS), OBO (.obo.sql), warehouse auto-start, metric views.
• Verified end-to-end on a deployed Databricks App: an SP-default query and an OBO (asUser, user_api_scopes: [sql]) query both execute against the SQL warehouse.

Notes

• Metric views run entirely on the migrated services (they compile to SQL); no separate migration needed.
• Regenerated the workspace-client API docs to match the new types (included).

This pull request and its description were written by Isaac.

[github-actions]

📦 Bundle size report

Compared against bundle-size-baseline.json (main).

@databricks/appkit

npm tarball (packed): 1.2 MB (+15 KB) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist	raw	gzip
JS (runtime)	1.2 MB (+15 KB)	453 KB (+5.9 KB)
Type declarations	458 KB (+2.2 KB)	166 KB (+896 B)
Source maps	2.5 MB (+27 KB)	848 KB (+10 KB)
Other	11 KB	3.7 KB
Total	4.2 MB (+45 KB)	1.4 MB (+17 KB)

Per-entry composition (own code — deps external (as shipped))

Entry	Initial (gz)	Lazy (gz)	Total (gz)	node_modules (min)	Own code (min)
.	104 KB (+1.8 KB)	2.5 KB	106 KB (+1.8 KB)	external	336 KB (+4.5 KB)
./beta	100 KB (+1.9 KB)	486 B (+1 B)	101 KB (+1.9 KB)	external	302 KB (+4.6 KB)
./testing	44 KB (+1.7 KB)	32 KB (-41 B)	76 KB (+1.7 KB)	external	219 KB (+4.4 KB)
./tsdown	520 B	0 B	520 B	external	813 B
./type-generator	25 KB (+1.6 KB)	0 B	25 KB (+1.6 KB)	external	71 KB (+4.4 KB)

Chunks:

Entry	Chunk	Load	Size (gz)
.	index.js	initial	99 KB
.	utils.js	initial	4.6 KB
.	remote-tunnel-manager.js	lazy	2.5 KB
./beta	beta.js	initial	82 KB
./beta	stream-manager.js	initial	5.9 KB
./beta	modular.js	initial	4.8 KB
./beta	service-context.js	initial	4.2 KB
./beta	databricks.js	initial	3.3 KB
./beta	client.js	initial	596 B
./beta	index.js	initial	20 B
./beta	supervisor-api.js	lazy	193 B
./beta	databricks.js	lazy	178 B
./beta	index.js	lazy	115 B
./testing	manifest.js	initial	30 KB
./testing	index.js	initial	10 KB
./testing	wide-event-emitter.js	initial	2.9 KB
./testing	index.js	lazy	28 KB
./testing	remote-tunnel-manager.js	lazy	2.5 KB
./testing	utils.js	lazy	1.8 KB
./tsdown	index.js	initial	520 B
./type-generator	index.js	initial	25 KB

@databricks/appkit-ui

npm tarball (packed): 397 KB (+94 B) — gzipped download (dist + bin; excludes release-only docs/NOTICE).

dist	raw	gzip
JS (runtime)	429 KB	146 KB
Type declarations	266 KB (+217 B)	98 KB (+130 B)
Source maps	853 KB	285 KB
CSS	16 KB	3.2 KB
Total	1.5 MB (+217 B)	533 KB (+130 B)

Per-entry composition (consumer bundle — deps bundled, peerDeps external)

Entry	Initial (gz)	Lazy (gz)	Total (gz)	node_modules (min)	Own code (min)
./js	5.3 KB	49 KB	55 KB	208 KB	14 KB
./js/beta	2.1 KB	0 B	2.1 KB	0 B	5.0 KB
./react	433 KB	49 KB	481 KB	1.3 MB	177 KB
./react/beta	4.9 KB	0 B	4.9 KB	0 B	12 KB

Chunks:

Entry	Chunk	Load	Size (gz)
./js	index.js	initial	5.2 KB
./js	chunk	initial	120 B
./js	apache-arrow	lazy	49 KB
./js/beta	beta.js	initial	2.1 KB
./react	index.js	initial	431 KB
./react	tslib	initial	2.1 KB
./react	apache-arrow	lazy	49 KB
./react/beta	beta.js	initial	4.9 KB

[github-actions]

🤖 AppKit PR bot

🔬 Run evals

Start an eval for this PR from the evals-monitor app: Go to Evals Monitor →

📦 Try this PR's app template

Scaffolds a new app from this PR's SDK build. Run it in any folder (requires the GitHub CLI — gh auth login — and the Databricks CLI):

gh run download 38918766917 -R databricks/appkit -n appkit-template-0.88.1-pr.69dce07-analytics-migration-sdk-562 -D appkit-pr-562 \
  && unzip -o "appkit-pr-562/appkit-template-0.88.1-pr.69dce07-analytics-migration-sdk-562.zip" -d "appkit-pr-562" \
  && databricks apps init --template "appkit-pr-562"

The template pins @databricks/appkit and @databricks/appkit-ui to tarballs built from this branch, so the scaffolded app runs against this PR's code.