[PowerX] expose measurement coverage and verify publication / 展示实测指标覆盖并核验发布结果

.github/workflows/apply-run-overrides.yml

@@ -5,6 +5,8 @@ on:
     branches: [main, master]
     paths:
       - 'packages/db/src/etl/run-overrides.ts'
+      - 'packages/db/src/etl/power-p90-backfills.ts'
+      - 'docs/data/power-p90-backfill.json'
       - '.github/workflows/apply-run-overrides.yml'
   workflow_dispatch:
     inputs:

.github/workflows/ingest-results.yml

@@ -75,6 +75,7 @@ jobs:
           INGEST_RUN_ATTEMPT: ${{ steps.artifacts.outputs.merge-run-attempt }}
           INGEST_ARTIFACTS_PATH: ${{ github.workspace }}/artifacts
           INGEST_REPO: SemiAnalysisAI/InferenceX
+          POWER_PUBLICATION_MANIFEST: ${{ github.workspace }}/power-publication.json
           UNMAPPED_ENTITIES_OUTPUT: ${{ github.workspace }}/unmapped-entities.json
         run: bun run admin:db:ingest:ci
 
@@ -94,7 +95,20 @@ jobs:
           VERCEL_INVALIDATE_SECRET: ${{ secrets.VERCEL_INVALIDATE_SECRET }}
         run: |
           curl -sSf -X POST "https://inferencex.semianalysis.com/api/v1/invalidate" \
-            -H "Authorization: Bearer $VERCEL_INVALIDATE_SECRET" || true
+            -H "Authorization: Bearer $VERCEL_INVALIDATE_SECRET"
+
+      - name: Verify PowerX source, database and public API
+        env:
+          DATABASE_WRITE_URL: ${{ secrets.DATABASE_WRITE_URL }}
+        run: bun packages/db/src/verify-power-publication.ts power-publication.json
+
+      - name: Retain PowerX publication receipt
+        if: always()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
+        with:
+          name: power-publication-${{ steps.artifacts.outputs.source-run-id }}
+          path: power-publication.json*
+          if-no-files-found: warn
 
       - name: Check for unmapped entities
         if: always()

docs/d3-charts.md

@@ -79,7 +79,9 @@ Power axes use the same Pareto directions as other metrics while **Optimal Only*
 
 Power-boundary views show only the measurements forming that boundary by default. **Show all measurements** reveals the remaining dots without changing the curves, axis domains or zoom. This display preference is independent of **Optimal Only**, defaults off, and is shared through `i_allpoints=1`. Historical rings remain attached to visible historical points; tables and data exports retain their existing selection rules.
 
-Measured power as a percentage of TDP has no preferred direction, so it always uses the upper boundary; its optimal switch is hidden. Charts preserve both display preferences when switching metrics. Energy per token retains its existing Pareto behavior. Power boundaries disable gradient strategy labels and the performance ruler.
+Measured power as a percentage of TDP uses the same lower-is-better power-demand direction as watts. Dividing watts by one hardware's positive, constant TDP preserves its Pareto membership. Optimal Only therefore works on both metrics and retains the saved preference when switching between them. A lower percentage across different chips is not, by itself, an energy-efficiency comparison. Energy per token retains its existing Pareto behavior.
+
+Upper boundaries use monotone interpolation between unique-X vertices, including after zoom, and disable gradient strategy labels and the performance ruler. The same grouping applies to unofficial-run overlays; unrelated dates and runs never share a curve.
 
 ## Gradient Roofline Labels
 

docs/data-pipeline.md

@@ -491,3 +491,37 @@ requires a known, matching run attempt and leaves the row unchanged if those
 source values differ. Database recovery rejects the mismatch before writing.
 P90 replays require numeric `power_valid: 1`, schema version 2, and the exact
 original average power, including when checking an already-applied correction.
+
+### Power Audit Provenance (`power_invalid_reasons`, `power_audit`)
+
+Producers (`aggregate_power.py`) annotate every aggregate result row with two optional provenance fields alongside the `power_valid` verdict:
+
+- **`power_invalid_reasons`** — array of snake_case reason-code strings explaining a withheld verdict (emitted when `power_valid == 0`), e.g. `sampling_gap_exceeded`, `expected_gpu_count_mismatch`.
+- **`power_audit`** — compact measurement-window audit object with optional fields: `window_start_unix`, `window_end_unix`, `expected_gpu_count`, `observed_gpu_count`, `sample_count`, `max_sample_gap_s`, `producer_sha`, `exporter_image_sha256`, `source`, `observed_gpu_ids`. `source` is a safe relative path inside the original artifact bundle, and device identifiers may be indices on older collectors. Present on valid and invalid rows alike.
+
+`mapBenchmarkRow()` narrows them defensively (`extractPowerInvalidReasons` / `extractPowerAudit`): reason codes must match `/^[a-z][a-z0-9_]*$/` (≤ 64 chars, deduplicated, capped at 32), audit numerics must be finite (counts: non-negative safe integers), shas collapse to `null` unless a non-empty string ≤ 128 chars, and unknown audit keys are dropped. A failed `benchmark_outcome.status` is rejected as a performance point, retaining its original artifact as evidence. An empty result maps to `undefined`, so the dedicated `benchmark_results.power_invalid_reasons` / `power_audit` JSONB columns (migration 015, mirroring the `workers` precedent from migration 006) store SQL NULL — never `[]` or `{}`. Legacy artifacts without the fields flow through every layer as NULL/undefined.
+
+Reads are **permanently tolerant**: `queries/benchmarks.ts` selects the columns as `to_jsonb(br) -> 'power_invalid_reasons'` (and `lb` on the matview branch) rather than bare column references. A bare reference fails during query planning until the next ingest workflow applies the migration, because migrations run in the ingest workflows rather than at Vercel deploy. The key lookup degrades to NULL while the column is missing and is byte-identical once it exists, making deploy order irrelevant.
+
+### PowerX publication receipts
+
+The normal CI importer writes `POWER_PUBLICATION_MANIFEST` when configured. Each
+8K/1K point records the mapped, override-adjusted metric contract, all configuration
+dimensions, original source run/attempt, structured audit, and input file SHA-256.
+Reused sweeps keep their original source identity. Failed or unmapped explicit 8K/1K
+results and database errors remain in the manifest; they cannot pass verification.
+
+After ingestion and cache invalidation, `bun packages/db/src/verify-power-publication.ts
+power-publication.json` compares every expected point against the exact database
+run attempt and public `runId=…&exactRun=true` response. It checks missing values and
+withheld telemetry as well as numbers; legacy missing measurements remain missing.
+A `matched` receipt establishes transport fidelity, not collection coverage. An
+empty receipt says `no_8k1k_points`, never that power coverage was validated.
+The workflow retains both the input manifest and verification receipt. Cache
+invalidation errors fail the workflow instead of being swallowed. Imported P75/P90
+ledger edits trigger the existing reviewed override workflow.
+
+The dashboard availability panel uses scoped points before Y-metric filtering,
+including visible unofficial overlays. It distinguishes schema-2 validation,
+other validated data, missing verdicts, withheld measurements, unavailable metrics,
+and non-applicable separate-pool metrics without filling missing values.

packages/app/cypress/component/gpu-graph.cy.tsx

@@ -1,4 +1,5 @@
 import GPUGraph from '@/components/inference/ui/GPUGraph';
+import { chartDefinitions } from '@/components/inference/metric-registry';
 import { InferenceContextsProvider } from '@/components/inference/InferenceContext';
 import { useState } from 'react';
 import { mountWithProviders } from '../support/test-utils';
@@ -549,6 +550,8 @@ describe('GPU comparison power envelopes', () => {
               chartType: latency ? 'e2e' : 'interactivity',
               y_measuredAvgPower_roofline: latency ? 'lower_left' : 'lower_right',
               y_measuredJPerOutputToken_roofline: latency ? 'lower_left' : 'lower_right',
+              y_measuredPowerPercentTdp_roofline:
+                chartDefinitions[latency ? 1 : 0].y_measuredPowerPercentTdp_roofline,
             })}
           />
         </div>
@@ -625,20 +628,27 @@ describe('GPU comparison power envelopes', () => {
       .and('contain.text', 'not efficiency frontiers');
   });
 
-  it('localizes the %TDP measurement toggle without changing the saved Optimal Only preference for energy', () => {
+  it('applies %TDP Pareto filtering and localizes its independent measurement toggle', () => {
     mountWithProviders(
       <PathnameContext.Provider value="/zh/inference">
         <PowerComparison />
       </PathnameContext.Provider>,
     );
     cy.contains('button', 'Percent TDP').click();
-    cy.get('#gpu-hide-non-optimal').should('not.exist');
+    cy.get('#gpu-hide-non-optimal').should('have.attr', 'data-state', 'checked');
+    cy.get('#gpu-power-curves .dot-group').should('have.length', 2);
+    cy.get('#gpu-power-curves .roofline-path').should('not.exist');
+    cy.get('[data-testid="power-curve-description"]').should('contain', '只有一个点');
+    cy.get('#gpu-show-all-measurements').should('not.exist');
+    cy.get('#gpu-hide-non-optimal').click({ force: true });
     cy.get('#gpu-power-curves .dot-group').should('have.length', 6);
     cy.contains('显示全部测量点').should('be.visible');
+    cy.get('#gpu-show-all-measurements').should('have.attr', 'data-state', 'unchecked');
     cy.get('#gpu-show-all-measurements').click({ force: true });
     cy.get('#gpu-power-curves .dot-group').should('have.length', 12);
     cy.get('#gpu-power-curves .roofline-path').should('have.length', 2);
     cy.get('[data-testid="power-curve-description"]').should('contain', '不代表能效 Pareto 前沿');
+    cy.get('#gpu-hide-non-optimal').click({ force: true });
     cy.contains('button', 'Energy').click();
     cy.get('#gpu-show-all-measurements').should('not.exist');
     cy.get('#gpu-hide-non-optimal').should('have.attr', 'data-state', 'checked');

packages/app/cypress/component/inference-chart-controls.cy.tsx

@@ -74,12 +74,12 @@ describe('Modeled system-power table', () => {
  */
 function mountWithPowerGroupsUnlocked() {
   cy.window().then((win) => win.localStorage.setItem('inferencex-feature-gate', '1'));
-  mountWithProviders(<InferenceChartControls showXAxisMode />, { inference: {} });
+  mountWithProviders(<InferenceChartControls showXAxisMode />, { inference: {}, unofficial: {} });
 }
 
 describe('Inference ChartControls', () => {
   beforeEach(() => {
-    mountWithProviders(<InferenceChartControls showXAxisMode />, { inference: {} });
+    mountWithProviders(<InferenceChartControls showXAxisMode />, { inference: {}, unofficial: {} });
   });
 
   afterEach(() => {
@@ -320,7 +320,10 @@ describe('Inference ChartControls', () => {
 
   it('keeps benchmark and chart settings in one row when history comparison is omitted', () => {
     cy.viewport(1280, 900);
-    mountWithProviders(<InferenceChartControls hideGpuComparison />, { inference: {} });
+    mountWithProviders(<InferenceChartControls hideGpuComparison />, {
+      inference: {},
+      unofficial: {},
+    });
     cy.get('[data-testid="x-axis-mode-selector"]').should('not.exist');
     cy.get('fieldset')
       .should('have.length', 2)
@@ -344,7 +347,7 @@ describe('Inference ChartControls', () => {
 
   it('keeps primary controls visible while secondary controls collapse on mobile', () => {
     cy.viewport(390, 844);
-    mountWithProviders(<InferenceChartControls showXAxisMode />, { inference: {} });
+    mountWithProviders(<InferenceChartControls showXAxisMode />, { inference: {}, unofficial: {} });
 
     cy.get('#model-select').should('be.visible');
     cy.get('[data-testid="inference-secondary-controls"] > button')
@@ -362,7 +365,7 @@ describe('Inference ChartControls', () => {
 
   it('shows secondary controls by default on desktop', () => {
     cy.viewport(1280, 900);
-    mountWithProviders(<InferenceChartControls showXAxisMode />, { inference: {} });
+    mountWithProviders(<InferenceChartControls showXAxisMode />, { inference: {}, unofficial: {} });
 
     cy.get('[data-testid="inference-secondary-controls"] > button').should('not.be.visible');
     cy.get('[data-testid="yaxis-metric-selector"]').should('be.visible');
@@ -374,7 +377,7 @@ describe('Inference ChartControls', () => {
       <PathnameContext.Provider value="/zh/inference">
         <InferenceChartControls showXAxisMode />
       </PathnameContext.Provider>,
-      { inference: {} },
+      { inference: {}, unofficial: {} },
     );
     // The count is derived from actual non-default settings, not merely present controls.
     cy.get('[data-testid="inference-secondary-controls"] > button')
@@ -386,6 +389,7 @@ describe('Inference ChartControls', () => {
 describe('Inference ChartControls cost metrics', () => {
   beforeEach(() => {
     mountWithProviders(<InferenceChartControls showXAxisMode showTcoBasis />, {
+      unofficial: {},
       inference: {
         selectedYAxisMetric: 'y_costh',
         selectedModel: Model.Qwen3_5,
@@ -398,6 +402,7 @@ describe('Inference ChartControls cost metrics', () => {
 
   it('hides the TCO basis toggle for other models and scenarios', () => {
     mountWithProviders(<InferenceChartControls showXAxisMode showTcoBasis />, {
+      unofficial: {},
       inference: {
         selectedYAxisMetric: 'y_costh',
         selectedModel: Model.DeepSeek_V4_Pro,
@@ -411,6 +416,7 @@ describe('Inference ChartControls cost metrics', () => {
     cy.get('[data-testid="yaxis-metric-selector"]').should('exist');
     cy.get('[data-testid="tco-basis-toggle"]').should('not.exist');
     mountWithProviders(<InferenceChartControls showXAxisMode showTcoBasis />, {
+      unofficial: {},
       inference: {
         selectedYAxisMetric: 'y_costh',
         selectedModel: Model.Qwen3_5,
@@ -424,6 +430,7 @@ describe('Inference ChartControls cost metrics', () => {
   for (const selectedYAxisMetric of ['y_tpPerGpu', 'y_tpPerMw'] as const) {
     it(`hides TCO for ${selectedYAxisMetric} even with visible TPU hardware`, () => {
       mountWithProviders(<InferenceChartControls showXAxisMode showTcoBasis />, {
+        unofficial: {},
         inference: {
           selectedYAxisMetric,
           selectedModel: Model.Qwen3_5,
@@ -438,6 +445,7 @@ describe('Inference ChartControls cost metrics', () => {
 
   it('hides TCO for a cost metric when no TPU hardware is visible', () => {
     mountWithProviders(<InferenceChartControls showXAxisMode />, {
+      unofficial: {},
       inference: {
         selectedYAxisMetric: 'y_costh',
         selectedModel: Model.Qwen3_5,
@@ -489,6 +497,7 @@ describe('Inference ChartControls cost metrics', () => {
 describe('Inference ChartControls infrastructure tokens per dollar', () => {
   beforeEach(() => {
     mountWithProviders(<InferenceChartControls showXAxisMode />, {
+      unofficial: {},
       inference: { selectedYAxisMetric: 'y_tokensPerDollarR' },
       globalFilters: {},
     });
@@ -512,6 +521,7 @@ describe('Inference ChartControls infrastructure tokens per dollar', () => {
 describe('Inference ChartControls with GPUs selected', () => {
   it('shows the date range picker when GPUs are selected', () => {
     mountWithProviders(<InferenceChartControls showXAxisMode />, {
+      unofficial: {},
       inference: {
         selectedGPUs: ['h100'],
         selectedDateRange: { startDate: '', endDate: '' },
@@ -523,6 +533,7 @@ describe('Inference ChartControls with GPUs selected', () => {
 
   it('leaves the optional date range unflagged for a selected current config', () => {
     mountWithProviders(<InferenceChartControls showXAxisMode />, {
+      unofficial: {},
       inference: {
         selectedGPUs: ['h100'],
         selectedDateRange: { startDate: '', endDate: '' },
@@ -537,6 +548,7 @@ describe('Inference ChartControls with GPUs selected', () => {
 
   it('leaves the date range unflagged when exact comparison entries are pinned', () => {
     mountWithProviders(<InferenceChartControls showXAxisMode />, {
+      unofficial: {},
       inference: {
         selectedGPUs: ['b200_sglang', 'b200_vllm'],
         selectedDateRange: { startDate: '', endDate: '' },
@@ -551,6 +563,7 @@ describe('Inference ChartControls with GPUs selected', () => {
 describe('Inference ChartControls with hideGpuComparison', () => {
   it('hides GPU config selector when hideGpuComparison is true', () => {
     mountWithProviders(<InferenceChartControls hideGpuComparison />, {
+      unofficial: {},
       inference: {},
     });
 
@@ -567,6 +580,7 @@ describe('Inference axis selector — Chinese Agentic controls', () => {
         <InferenceChartControls showXAxisMode />
       </PathnameContext.Provider>,
       {
+        unofficial: {},
         inference: { selectedSequence: Sequence.AgenticTraces, selectedXAxisMode: 'interactivity' },
       },
     );
@@ -586,7 +600,7 @@ describe('Inference axis selector — Chinese Agentic controls', () => {
 
 describe('Axis option help', () => {
   beforeEach(() => {
-    mountWithProviders(<InferenceChartControls showXAxisMode />, { inference: {} });
+    mountWithProviders(<InferenceChartControls showXAxisMode />, { inference: {}, unofficial: {} });
   });
 
   for (const searchable of [true, false]) {