feat(agentx): display physical cache-tier token sources

[cquil11]

Summary

• ingest vllm:prompt_tokens_cached_by_source from AgentX server-metric artifacts
• combine fresh prefill with physical HBM, CPU, NVMe, and connector-defined cache-hit tiers without double counting
• retain the existing logical vLLM fallback and SGLang source mapping for older runs
• display tier-specific colors and English/Simplified Chinese labels on the point-detail chart
• bump the derived chart-series version so existing rows can be recomputed

Validation

• bun run typecheck
• bun run lint
• bun run fmt
• bun --cwd packages/db vitest run src/etl/compute-chart-series.test.ts
• bun --cwd packages/app vitest run src/components/inference/agentic-point/time-series-math.test.ts src/lib/api-route-catalog.test.ts
• bun --cwd packages/app vitest run src/lib/zh-copy.test.ts
• bun run --cwd packages/app test:e2e:component --spec cypress/component/agentic-token-source-chart.cy.tsx

Live validation

Both official DeepSeek-V4 AgentX sweeps are green and have been backfilled into the PR-specific child DB: B200 offloading and GB300 1P/1D. Production is untouched.

Preview points, checked in the deployed browser and API:

• Native DRAM c8
• Simple NVMe c14
• Native DRAM+NVMe c14
• NIXL P2P + MooncakeStore c256

All four stored Prometheus CSVs match the authoritative artifacts byte-for-byte. Raw JSON, request traces, server logs, and version-17 chart payloads are linked. P2P covers all 20 engines / five endpoints; its chart separates P2P and external-cache hits. The child branch br-mute-band-ainhq3nj expires October 8, 2026. Its compute was temporarily increased to 2 CU for the 307 MB compressed P2P blob import. Cold-cache verification found that the original 0.25 CU also cannot serve the uncached P2P chart query; the preview child now uses 1 CU with five-minute auto-suspend. The uncached request passed at that size.

Known display caveat: the pre-existing CPU cache hit summary badge can still show aggregate external hits for NVMe/Mooncake configurations. Use the new cumulative source-breakdown chart for physical tier attribution. The P2P run also canceled 435 outstanding requests at its timed drain deadline; zero request errors and 1,740 completed requests do not mean every submitted request completed.

Compatibility

No relational migration is required. The existing JSONB chart_series payload is versioned from v15 to v17. Historical rows without the new vLLM metric keep their current source breakdown.

---

Note

Medium Risk
Changes versioned chart-series aggregation (affects stored metrics display after backfill) and extends benchmark-siblings payload/caching; UI and query changes are additive with logical fallbacks for older vLLM rows.

Overview
Adds physical vLLM KV-cache tier attribution to the AgentX prompt-token source pipeline and surfaces it in the UI and sibling navigator.

Data / ETL: Chart-series extraction (v16–v17) reads vllm:prompt_tokens_cached_by_source when present, keeps only fresh prefill (local_compute / miss) from the logical prompt_tokens_by_source metric, and maps device/CPU/host/disk/connector labels into stacked buckets (HBM, CPU offload, NVMe, custom tiers) so cached tokens are not double-counted. Older blobs without the new metric keep the logical breakdown; SGLang behavior is unchanged. CHART_SERIES_VERSION bumps to 17 for backfill.

AgentX UI: The stacked prompt-token chart gets tier-specific colors and English / Simplified Chinese labels (plus a localized card title). Cypress covers tier rendering on /zh routes.

Sibling navigator: The benchmark-siblings API/cache (v4) adds kv_offloading, p90_intvty, and p90_ttft. Chips show physical offload tiers (e.g. DRAM, NVMe) instead of off=ON, and points on both P90 interactivity and TTFT Pareto frontiers get a ring + legend (shared generic paretoFront* helpers).

Smaller fixes: Agentic eval ETL treats isl/osl of 0 as unspecified (null). API docs and data-pipeline.md document the new breakdown behavior.

^{Reviewed by Cursor Bugbot for commit aaa6784. Bugbot is set up for automated code reviews on this repo. Configure here.}

Canonical host source compatibility

Recognizes the current vLLM host source as the existing CPU/DRAM cache-hit bucket, while retaining cpu for earlier artifacts. Chart-series version 17 enables recomputation of previously ingested rows. Tests cover both labels and preserve prompt-token totals without double counting.

Validation: 5,163 local unit tests passed; fixture-backed Cypress component and integration smoke suites passed; typecheck, lint, formatting, and pre-commit checks passed. The preview deployment now serves both successful DeepSeek-V4 SWA sweeps from the child DB; metric artifacts and all four point pages were checked.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
inferencemax-app	Ready	Preview	Sep 9, 2026 11:06pm UTC

[cquil11]

Preview test environment is ready.

• Preview: https://inferencemax-app-git-feat-cache-tier-toke-788211-semianalysisai.vercel.app
• Neon branch: pr-944-cache-tier-token-sources (br-divine-cake-ai4cq1m7), cloned from production and expiring 2026-09-04
• Ingested source run: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/33169766010
• Ingest result: 25 benchmark points and 25 trace-replay sidecars; database verification passed

Representative points:

• NVMe / vllm-simple: https://inferencemax-app-git-feat-cache-tier-toke-788211-semianalysisai.vercel.app/inference/agentic/440534 — Fresh, HBM Cache Hit, NVMe Offload Cache Hit
• Native DRAM+NVMe: https://inferencemax-app-git-feat-cache-tier-toke-788211-semianalysisai.vercel.app/inference/agentic/440549 — Fresh, HBM Cache Hit, CPU Offload Cache Hit
• DRAM / Mooncake: https://inferencemax-app-git-feat-cache-tier-toke-788211-semianalysisai.vercel.app/inference/agentic/440541 — Fresh, HBM Cache Hit, External Cache Hit
• Chinese NVMe view: https://inferencemax-app-git-feat-cache-tier-toke-788211-semianalysisai.vercel.app/zh/inference/agentic/440534

Verified the deployed API response, English and Chinese page routes, browser rendering, and deployment runtime logs (zero 5xx responses during verification).

[cquil11]

Added in commit 68a8d4fd:

• Sibling chips now use physical KV tiers: DRAM, NVMe, or DRAM+NVMe; no-offload points have no suffix and the legacy off=ON text is gone.
• Points on both the P90 interactivity/throughput and P90 TTFT/throughput Pareto frontiers receive a double orange outline, with a legend below the selector.
• The deployed 25-point run marks three intersection points: NVMe c=12, DRAM c=9, and DRAM+NVMe c=7.

Verified on the deployed English and Chinese pages. All PR checks are green.

Preview: https://inferencemax-app-git-feat-cache-tier-toke-788211-semianalysisai.vercel.app/inference/agentic/440549