Skip to content

feat(admin): tabbed cost analytics + all-in platform cost from the AWS bill - #1235

Merged
philmerrell merged 4 commits into
developfrom
claude/cost-analytics-tabs-9f8cc9
Sep 22, 2026
Merged

philmerrell merged 4 commits into
developfrom
claude/cost-analytics-tabs-9f8cc9

Conversation

@philmerrell

Copy link
Copy Markdown
Contributor

Why

The cost dashboard stacked four blocks down one column and shared a two-column grid between the donut and the trend chart, so every visualisation rendered at half the console's width. Three concrete costs:

  • The model-usage legend ran past the fold — you could not see the fleet without scrolling past the chart.
  • The top-users table lost its Tier and Quota used columns to horizontal scroll.
  • The trend chart's date axis sat on a 45-degree rotation and still dropped labels.

Now that the admin nav lives in the sidenav (#1232), that width is available.

What

One tab per visualisation. The period KPIs stay pinned above the strip — every tab is a drill-down of those four numbers — and the Session Cost Anatomy lookup moves out of the page header into the Conversations tab, beside the rows it links to.

Tab Contents Badge
Model Usage (default) pie/bar breakdown, Pie preselected model count
Cost Trends daily cost + requests line chart day count
Top Users sortable user table rows loaded
Conversations session-ID lookup + expensive-conversations scan rows, hidden until scanned

Model Usage leads because the donut answers the first question an admin brings to this page — what the money went to — and it is the only panel that is complete on load with no further fetch or scan.

Three decisions worth the reviewer's time:

  • The strip is the documented pill idiom, not an underline. app-conventions.md rules underline tabs out explicitly: the brand token is a fixed colour with no dark variant, so a brand-coloured underline all but disappears on the dark surface. Raised white pill on a recessed shell, inline-flex not flex, with role="tablist"/tab/tabpanel, roving focus and arrow keys; below sm it collapses to a <select>.
  • Panels are destroyed on switch, not hidden. A hidden canvas cannot size itself, so a [hidden] panel would come back to a 0×0 chart. Everything that must survive a switch (the loaded rows) already lives in AdminCostStateService, not in the panel.
  • The model card's column split flips with the view. A donut is constrained by its shorter side, so extra width past a point is empty canvas; the horizontal bars carry long us.anthropic.* labels and earn every pixel. Donut 2/5 with a roomy legend beside it, bars 3/5. The legend's share bar is pie-only — in bar view the chart already is that baseline, and the column it costs is what truncated the model ids next to it.

Two defects found on the way

  • Whole-dollar axis labels printed $4 $4 $3 $2 $2 $1 $1 $0 whenever the axis stepped by under a dollar — seven labels for four values against eight distinct gridlines. Pre-existing, but a full-width chart makes it the first thing you read. Both charts now take the precision from the full tick array Chart.js hands the callback, so the column stays uniform instead of mixing $1 with $1.50.
  • Neither chart destroyed its Chart.js instance. That cost nothing while the cards lived for the life of the page. Now that they mount and unmount, every tab switch would have leaked one.

Also

Admin content is capped at max-w-[100rem] and centred. That is a sprawl guard, not the max-w-7xl reading-width cap this shell exists to drop: it is wider than any laptop the console is used on (verified non-binding at a 1600px viewport, binding at 2600px), and only stops a 20-column cost table reaching 3000px on an ultrawide, where a row's rank and its dollars end up a head-turn apart.

Card radii on this page move to rounded-2xl to match the documented token. All five components touched are used only by this page.

Verification

Clicked through all four tabs against real dev data — local SPA on :4200, worktree app-api on :8000 against dev DynamoDB, signed in as a real admin. Loaded the conversations scan (25 rows, badge updated from hidden to 25). Confirmed the mobile <select> switches panels and that the segmented control stops stretching at 375px with no horizontal page overflow.

  • Light and dark, both levers (class="dark" removed and prefers-color-scheme emulated).
  • 375px / 1600px / 2600px.
  • Badge contrast measured through a canvas normalise: 6.51–9.63:1 light, 4.74–5.04:1 dark — all AA.
  • Full SPA suite green: 296 files / 3701 tests.

🤖 Generated with Claude Code

philmerrell and others added 2 commits September 21, 2026 21:25
…width

The dashboard stacked four blocks down one column and shared a two-column
grid between the donut and the trend chart, so each visualisation rendered at
half the console's width. The donut's legend ran past the fold, the top-users
table lost its tier and quota columns to horizontal scroll, and the trend
chart's date axis sat on a 45-degree rotation and still dropped labels.

One tab per visualisation instead: Model Usage (default), Cost Trends, Top
Users, Conversations. The period KPIs stay pinned above the strip — every tab
is a drill-down of those four numbers — and the Session Cost Anatomy lookup
moves out of the page header into the Conversations tab, beside the rows it
links to. Model Usage leads because the donut answers what the money went to,
and it is the only panel complete on load with no further fetch or scan.

Tab strip is the documented pill idiom (raised white pill, recessed shell),
not an underline: the brand token has no dark variant, so a brand-coloured
underline all but disappears on the dark surface. Panels are destroyed on
switch rather than hidden, because a hidden canvas cannot size itself and a
[hidden] panel would come back to a 0x0 chart; the loaded rows already live in
AdminCostStateService, not in the panel.

Two defects found on the way:

- Whole-dollar axis labels printed "$4 $4 $3 $2 $2 $1 $1 $0" whenever the axis
  stepped by under a dollar — seven labels for four values against eight
  distinct gridlines. Both charts now read the precision off the full tick
  array Chart.js hands the callback, so the column stays uniform rather than
  mixing "$1" with "$1.50".
- Neither chart destroyed its Chart.js instance. That cost nothing while the
  cards lived for the life of the page; now that they mount and unmount, every
  tab switch leaked one.

Also caps admin content at max-w-[100rem] and centres it. That is a sprawl
guard, not the max-w-7xl reading-width cap this shell exists to drop: it is
wider than any laptop the console is used on, and only stops a 20-column cost
table reaching 3000px on an ultrawide.

Verified against dev data across all four tabs, light and dark, at 375 /
1600 / 2600px. Badge contrast 6.51-9.63:1 light and 4.74-5.04:1 dark. Full
SPA suite green (296 files / 3701 tests).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The dashboard measured Bedrock token spend and measured it well — against
prod's September 1-21 bill our ledger came within 0.50% of Cost Explorer's
model SKUs ($1,076.03 vs $1,081.44). It could not see anything that was not a
token, which on prod was 38.8% of the bill: $685.96 of $1,767.41. Amazon
Bedrock AgentCore alone was $201.54/month, the second-largest line item in
the account and on no screen anywhere. With 1,807 active users, "Avg
Cost/User" was reporting $0.60 against a true all-in $0.98 — understating
unit economics by 1.64x.

A daily EventBridge tick runs a Lambda that makes one Cost Explorer call per
period and writes PLATFORM#* rows into the EXISTING system-cost-rollup table
(own PK namespace — no new table, no GSI, no migration). A new Platform tab
and the two headline cards read those rows.

Scheduled and never live, because Cost Explorer bills $0.01 per request: a
screen that queried it on load would cost ~$10/month per thousand page loads
to display costs. ce:GetCostAndUsage is granted to the Lambda role and
deliberately NOT to app-api's, so no request path can spend billing dollars
however often it is called. Steady state is ~$0.02/day.

Two sources, never added twice. Inference stays on our own ledger (per user
and per session, where CE is per-account only); infrastructure comes from CE.
CE's own inference figure rides along ONLY to reconcile the two — adding it
to a total would count every token twice. The Platform tab surfaces that
comparison as a pricing regression test: a widening delta means the rates in
curated-models.ts have drifted from what AWS actually charged, which
CLAUDE.md names as a live risk.

Attribution is a service allowlist, not tags. applyStandardTags already puts
Project/Environment/Version on every resource, but no cost allocation tag is
activated in either account, so GROUP BY TAG Project returns one Project$
bucket holding the whole bill. Activation is an org-management-account action
(a Control Tower linked account gets AccessDeniedException listing them) and
is not retroactive. Meanwhile the accounts are shared — bsu-prod-backend runs
an Aurora cluster we do not provision — so services are bucketed
inference/platform/excluded, and excluded rows are persisted and DISPLAYED
rather than dropped: a total is only trustworthy if its exclusions are
visible.

Opt-in, against this repo's usual default-on posture, because this flag reads
the account's billing data, needs an IAM action an SCP may deny, and costs
money per call. Only the literal "true" enables; an unset workflow variable
arrives as an empty string and leaves it off. While off the construct
produces zero resources and the tab explains that rather than rendering a
$0.00 that would read as "infrastructure is free".

Known limitation, stated in the UI and the spec: infrastructure is measured
per AWS account, and dev-ai hosts five deployments, so dev's figures cover
all of them. Prod is a dedicated account and is sound. Activating the Project
tag is the fix; the row shape does not change when it lands.

Also fixes two icons that rendered wrong. Utility classes on <ng-icon> do not
apply: its :host width/height rule is UNLAYERED so it outranks Tailwind's
@layer utilities size-*, and its colour rule sits in @layer ng-icon
(registered after Tailwind's layers) resolving to currentColor. Measured
size-5 rendering at 16px and text-state-warning-600 resolving to black, while
the same class on a plain span resolved correctly. Colour now lives on a
wrapper and dimension on ng-icon's own [size] input; the status glyph pairs
-700/dark:-400, measured 5.03 light and 8.52 dark.

Verified by running the sync Lambda against dev-ai end to end: August came
back 519.02 platform + 13.56 inference + 203.74 excluded = $736.32, matching
the bill measured independently. Tests: 35 handler, 9 CDK, 233 backend cost,
plus full suites green — 9,631 backend, 896 CDK, 3,701 SPA.

See docs/specs/platform-cost-visibility.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@philmerrell philmerrell changed the title feat(admin): split cost analytics into tabs and give the charts full width feat(admin): tabbed cost analytics + all-in platform cost from the AWS bill Sep 22, 2026
@philmerrell

Copy link
Copy Markdown
Contributor Author

Follow-up commit: all-in platform cost (e06c881b)

The tabs work uncovered a bigger gap, so this PR now also closes it.

The dashboard was seeing ~61% of the bill

Measured against prod, Sept 1–21:

Amount Share
Inference (Bedrock model SKUs) $1,081.44 61.2%
Infrastructure $685.96 38.8%
Total $1,767.41

Amazon Bedrock AgentCore alone is $201.54/month — the second-largest line item in the account, on no screen anywhere. (It's also the line that would have surfaced the idle-reaper regression, #827.)

With 1,807 active users, "Avg Cost/User" was reporting $0.60 against a true all-in $0.98 — understating unit economics by 1.64×.

Our pricing tables are accurate to 0.50%

Prod ROLLUP#MONTHLY for 2026-09 says totalCost = $1,076.03; Cost Explorer's Bedrock Edition SKUs say $1,081.44. A $5.41 gap on a thousand dollars of spend.

That result drove the design: inference stays on our own ledger (per user, per session — CE is per-account only), and CE is used only for the 39% we never measured. CE's own inference figure rides along purely to reconcile, surfaced as a pricing regression test — a widening delta means curated-models.ts has drifted from what AWS charged, which CLAUDE.md flags as a live risk. It is never added to a total; doing so would count every token twice.

Design decisions worth a reviewer's attention

  • Scheduled, never live. CE bills $0.01/request. A screen querying it on load would cost ~$10/month per thousand page loads to display costs. One daily tick, ~$0.02/day, results in DynamoDB. ce:GetCostAndUsage is on the Lambda role and deliberately not on app-api's, so no request path can spend billing dollars.
  • No new infra. PLATFORM#* rows go in the existing system-cost-rollup table — no new table, no GSI, no migration.
  • Opt-in, against this repo's usual default-on posture: the flag reads billing data, needs an IAM action an SCP may deny, and costs money per call. Only the literal true enables. While off, zero resources and the tab explains itself rather than rendering a $0.00 that reads as "infrastructure is free".
  • Service allowlist, not tags. applyStandardTags already tags everything, but no cost allocation tag is activatedGROUP BY TAG Project returns one Project$ bucket holding the whole bill. Activation is an org-management-account action (dev-ai gets AccessDeniedException just listing them) and is not retroactive. And the accounts are shared: bsu-prod-backend runs an Aurora cluster we don't provision. So services bucket into inference / platform / excluded, and excluded rows are displayed, not dropped — a total is only trustworthy if its exclusions are visible.

⚠️ Known limitation, stated in the UI

Infrastructure is measured per AWS account, and dev-ai hosts five deployments (beta-boisestateai-dev, boisestateai-v2-dev, bsu-agentcore, dev-boisestateai-v2, nightly-mv) — so dev's platform figure and its reconciliation cover all of them. That's why dev shows a 24% reconciliation delta and prod would show 0.5%. Prod is a dedicated account and is sound apart from the excluded Aurora. Activating the Project tag fixes it; the row shape doesn't change when it lands.

The one thing this PR can't do for you: activating Project as a cost allocation tag needs org-management-account billing access. Since activation isn't retroactive, it's worth doing early regardless of when this merges.

Also fixed: two icons that rendered wrong

Utility classes on <ng-icon> don't apply, and I measured why: its :host{width:var(--ng-icon__size,1em)} is unlayered, so it outranks Tailwind's @layer utilities size-*; its colour rule sits in @layer ng-icon, registered after Tailwind's layers, resolving to currentColor. Measured size-5 rendering at 16px and text-state-warning-600 resolving to black, while the identical class on a plain <span> resolved correctly. Colour now lives on a wrapper, dimension on ng-icon's own [size] input. The status glyph uses -700/dark:-400 — measured 5.03 light / 8.52 dark.

Verification

Ran the sync Lambda end to end against dev-ai. August returned 519.02 platform + 13.56 inference + 203.74 excluded = $736.32 — exactly the account total measured independently. Then drove the UI against those real rows: all-in card, reconciliation warning, 19-service table, excluded footer.

Tests: 35 handler · 9 new CDK · 233 backend cost · full suites green — 9,631 backend (0 failures), 896 CDK, 3,701 SPA.

Spec: docs/specs/platform-cost-visibility.md

🤖 Generated with Claude Code

philmerrell and others added 2 commits September 22, 2026 07:05
…count

An account is not an application. This stack is open source, so a deployer may
share an account with other workloads — and ours does: dev-ai hosts FIVE
deployments of this stack plus unrelated apps, and both accounts run a
bsu-*-backend Aurora cluster we do not provision. prod-ai happens to hold
exactly one deployment today, but that is luck, not design, and no fork should
inherit an assumption that only holds for us.

The sync now asks for this deployment's own resources first, filtering on the
Project tag applyStandardTags already writes. Its value is the stack's
projectPrefix, so there is nothing to configure and a fork gets it free.

Cost Explorer will not filter by a cost allocation tag until that tag is
ACTIVATED in the payer account — an Organizations member gets AccessDenied
merely listing them, and activation is not retroactive. So the scope is
resolved at runtime rather than assumed: query scoped, and fall back to
account-wide if it comes back empty. Verified against prod that an inactive
tag returns HTTP 200 with zero groups and $0.00 rather than an error, which is
exactly what the probe keys on. We deliberately do NOT call
ListCostAllocationTags — that is the call a linked account is denied, and a
linked account is the topology most likely to need the fallback.

Every row records which answer the deployer got. `scope: "deployment"` means
the figures are this stack's; `scope: "account"` means they are a ceiling, and
the UI says so in three places that all disappear once the tag is live: a
"Whole account" badge, a panel naming the exact fix and the tag value to
activate, and the reconciliation copy leading with scope instead of blaming
pricing drift. Rows written before scoping existed carry no scope and default
to "account" — the pessimistic default is deliberate, because defaulting the
other way would relabel an account-wide figure as this app's cost with nothing
to reveal the error.

Also sets propagateTags: SERVICE on the Fargate service. Fargate bills per
TASK and tasks do not inherit a service's tags without it; the live dev
service reported NONE, which would have dropped ~17% of prod's infrastructure
(~$118/month) out of the tagged scope while everything still looked healthy.
That is the kind of gap that makes a scoped total quietly wrong rather than
obviously broken.

Tag coverage audited across 1,519 dev resources before relying on it:
ECR/ELB/Cognito/SQS 100%, DynamoDB 98%, KMS 96%, CloudWatch 91%, and the two
largest lines confirmed individually — our NAT gateway and our AgentCore
runtime both carry Project=dev-boisestateai-v2, while other stacks' carry
theirs or none.

Verified end to end against dev: the sync probed the tag, logged an actionable
warning naming the fix, fell back, and still produced usable data
(scope=account, $321.28 platform). The dashboard renders the badge, the
warning panel and the scope-first reconciliation copy in both themes; badge
contrast 5.03 light.

Tests: 45 handler (10 new for scoping), 11 CDK (2 new), 235 backend cost
(2 new), full suites green — 898 CDK, 3,701 SPA.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"These figures cover the whole account" describes a condition and leaves the
reader to hunt for what to do about it — and the instruction that mattered was
buried two paragraphs down in the smallest text on the card. The heading now
states the action: activate Project as a cost allocation tag in the payer
account.

Three changes, all about making it land:

- Moved above the figures it qualifies. A caveat printed after the numbers is
  one people read second, if at all.
- Heading carries the fix, with the tag name in a code chip; the "whole
  account" condition moves to the supporting line where it belongs.
- Where / Tag / Then rows replace the small-print paragraph, so the payer
  account requirement, the exact tag value already on every resource, and the
  not-retroactive warning each stand on their own instead of running together.

Card is now a state-warning tint rather than a white card with a warning
border, which is what makes it read as an instruction at a glance. Contrast
measured in both themes: icon 4.85 light / 9.53 dark, heading 17.11 / 16.42,
body 9.94 / 11.15, code chips 17.75 / 18.89. The icon stays at -700 rather
than -600 precisely because it now sits on its own tint, where -600 measures
2.87 and fails.

Still conditional, so it disappears the moment the tag is activated and never
becomes furniture. Also drops a "see below" that the move made wrong.

SPA suite green (296 files / 3,701 tests).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@philmerrell
philmerrell merged commit 6d6528e into develop Sep 22, 2026
6 checks passed
@philmerrell
philmerrell deleted the claude/cost-analytics-tabs-9f8cc9 branch September 22, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant