Skip to content

Build managed Codex model catalogs offline from harness metadata - #612

Open
lilly-luo wants to merge 4 commits into
david/AIGTWY-4573-model-pickerfrom
codex/offline-managed-model-catalog
Open

lilly-luo wants to merge 4 commits into
david/AIGTWY-4573-model-pickerfrom
codex/offline-managed-model-catalog

Conversation

@lilly-luo

Copy link
Copy Markdown
Collaborator

Summary

Stacked on david/AIGTWY-4573-model-picker (one additional commit).

Replace the static-model catalog's authenticated /ai-gateway/codex/v1/models fetch with client-side construction, inspired by databricks-eng/universe#2591694:

  • Extract full native presets using the active Codex binary's debug models --bundled in an isolated temporary home, and validate the generated catalog with the same binary before writing configuration.
  • Preserve GPT-specific prompts, capabilities, and unknown schema fields; resolve known gateway/legacy GPT aliases to native metadata.
  • For non-GPT models without native metadata, use the GPT-5.2 ordinary-tool template with conservative defaults (text-only, 32K context, reasoning none, native search disabled). Apply explicit capability overrides for known GLM and Kimi models.
  • Show only the admin-selected unique names, in admin order. Hidden native-ID aliases exist only for selected GPTs to support smart routing.
  • Forward the catalog to both the smart-routing app-server and TUI, and use the static list instead of stale discovery results for routing candidates.
  • Block configuration with an actionable error if extraction/validation fails or requested GPT metadata is absent. Never fall back to discovery or reuse a stale catalog. Provider-service configurations continue to bypass static catalogs.

Validation

  • Regression run: 2217 passed, 42 skipped, 1 deselected:
    uv run --frozen pytest -q --ignore=tests/test_e2e_user_agent.py --deselect=tests/test_claude_smart_routing_v2.py::TestV2Launch::test_restores_model_captured_immediately_before_switch
  • ruff check ., ruff format --check src/ tests/, ty check src/, and git diff --check pass.
  • Verified offline extraction and generated-catalog validation using installed Codex 0.154.0, covering GPT aliases, GLM/Kimi, and an arbitrary non-GPT service.

The full-suite attempt exposed an unrelated Claude route display-name assertion (reproduced on the unchanged base branch) and two host-dependent test_e2e_user_agent.py failures: the installed/managed Claude and Codex configurations prevented requests reaching the tests' localhost capture servers. The passing regression run excludes that E2E file and the confirmed baseline assertion.

No dependency or lock-file changes.


This PR was created with GitHub MCP.

david-siqi-liu and others added 4 commits September 13, 2026 23:23
Land the CodingAgentConfig parsing and the version and watermark helpers on their own, ahead of any
change to launch or configure behavior, so the data-shape change is reviewable in isolation. Nothing
here is called from the launch path yet; the stacked launch-apply PR wires it in.

- managed_config: read the extended config shape (enabled_agents with per-agent AgentModels: names /
  model_service_location / provider, default_model, default_alias_models, http_headers; spend_tiers
  with a budget_policy fallback; update_time). Add managed_config_is_newer and managed_update_time, a
  spec_version forward-compat gate (a newer or malformed spec_version keeps the last-known-good
  cache), and the UCODE_MANAGED_CONFIG_STUB local-file read for pre-server testing.
- mcp_servers and skills each parse to a names-or-location selector: an absolute names list of UC
  FQNs, or a unity_catalog_location (a catalog.schema) whose contents are discovered, enforced
  either/or. Both are UC mcp-services / skills only; there is no per-server type model. managed_setup
  serializes and validates the same shape (names are 3-part FQNs, location a 2-part catalog.schema,
  exactly one set), so serialize then normalize round-trips.
- state: add get/set_applied_managed_update_time, the applied-version watermark the launch gate reads.
- managed_export / managed_setup: parse and export updates; spec_version is server-owned on export so
  a supplied config cannot override the envelope value.
- databricks: switch the managed-config update mask to spend_tiers; rename the user-facing "Unity AI
  Gateway" strings to "Unity Gateway".

Co-authored-by: Isaac <no-reply@databricks.com>
With the config parsed (the stacked parse PR), apply the managed configuration to every enabled
agent at launch, but only when it changed since the last apply (an update_time watermark) or on
--refresh, so an unchanged launch never re-writes the OS-managed files or prompts for a password.
Also remove the post-configure probe validation, wire the managed MCP servers and skills, and rename
user-facing strings.

- Gate the launch apply on update_time. refresh_managed_config always fetches; a launch re-applies
  the managed configuration (the only step that writes the OS-managed files and can prompt for a
  password) only when update_time is newer than the applied watermark, or on ug <agent> --refresh,
  and records the new watermark. An unchanged launch skips the apply and never prompts. ug configure
  always applies and records the watermark. The "was updated; re-applying it" note prints only on a
  genuine version change, not on a no-op --refresh. The per-tool launch write is wrapped in
  suppressed_managed_writes() so the apply-all owns the single OS write.
- Apply the managed config's mcp_servers and skills to each enabled agent. Both are UC-services-only
  selectors: register the absolute names (as UC mcp-services) or download the named skills, plus
  everything discovered under a unity_catalog_location (a schema's mcp-services via list_mcp_services,
  or its skills), unioned. Nothing on the launch path turned the workspace's published servers or
  skills into agent registrations before, so they never showed up.
- `ug configure` wires the managed config's MCP servers and skills at configure time (not only at
  launch), and suppresses the interactive MCP/skills/plugins prompt when the managed config already
  defines them (and never installs anything under --dry-run), so a managed workspace is set up
  without asking the developer to configure MCP.
- Stop ug configure from validating agents with a probe message. Remove the post-configure
  validation and the --skip-validate flag: it is not configure's job to test models, the probe also
  reverted a freshly-written config on failure, and the gateway enforces model access regardless.
  The now-unused validate_all_tools / provider_permission_error / resolve_managed_for_tool helpers
  are removed; validate_tool stays for headless and e2e callers.
- Drop the "Using managed config - launching <agent>" banner and the redundant post-panel "Starting
  <agent>" line from the bare-ug launch, so bare ug matches ug <agent>.
- When a config we previously applied is gone (deleted, or the feature turned off), the launch
  does not auto-revert the OS-managed files; it notes that Unity Gateway managed configurations are
  still applied and points the developer to `ug revert`, but only when the config is definitively
  gone (NOT_FOUND or the feature is off), not on a transient fetch failure. Keyed on the applied
  watermark, so a workspace that never had a config still just uses the developer's own settings.
- Rename the product string "CLI Managed Configuration" to "managed configuration" (lowercase)
  across user-facing lines, tighten the budget-tier recommendation line, and remove the
  --skip-validate docs.

Co-authored-by: Isaac <no-reply@databricks.com>
Resolve the managed config's per-agent inputs into each agent's own config shape: the model
allow-list or discovery location into Claude's /model picker and Codex's model catalog, and the
custom http_headers into each agent's outbound requests. Stacked on the launch-apply PR (#593).

- managed_resolve: add managed_static_models, managed_model_service_location, and
  managed_custom_headers; extend managed_state_overrides to layer claude_static_models /
  codex_static_models, the per-agent model_service_location, and {tool}_custom_headers into state.
  For codex a discovery location alone does not count as supplying models (no reader for it), so
  discovery still runs; for claude it does.
- claude: a static list becomes availableModels + enforceAvailableModels + a modelPicker (exactly
  the named ids, non-Claude included); a discovery location turns on gateway discovery instead.
  Ownership markers prune only ug-owned picker keys on a static-to-discovery transition, and clear
  only when the OS-managed file was written. Managed http_headers append to ANTHROPIC_CUSTOM_HEADERS,
  with ug's fixed headers winning on conflict; values that would break the header serialization
  (newlines, and commas for Gemini) are dropped.
- codex: a static list is written as a model_catalog_json catalog of full ModelPresets; a fetch
  failure preserves an existing catalog (or falls back to discovery when none exists yet), and a
  successful but empty result falls back to discovery.
  Managed http_headers merge into the provider http_headers, ug's fixed headers winning.
- gemini: managed http_headers merge into GEMINI_CLI_CUSTOM_HEADERS, ug's fixed headers winning.
- README: document the Claude picker, the Codex catalog, and the ucode-models.json managed file.

Co-authored-by: Isaac <no-reply@databricks.com>
@david-siqi-liu
david-siqi-liu force-pushed the david/AIGTWY-4573-model-picker branch 11 times, most recently from 329ebdd to 040523f Compare September 15, 2026 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants