Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ on:
- "mkdocs.yml"
- "pyproject.toml"
- "scripts/check_docs_i18n.py"
- "scripts/check_rendered_code_docs.py"
- "src/blueprinting/**"
- "uv.lock"
pull_request:
branches: [main]
Expand All @@ -18,6 +20,8 @@ on:
- "mkdocs.yml"
- "pyproject.toml"
- "scripts/check_docs_i18n.py"
- "scripts/check_rendered_code_docs.py"
- "src/blueprinting/**"
- "uv.lock"
workflow_dispatch:

Expand Down Expand Up @@ -61,6 +65,9 @@ jobs:
- name: Build documentation
run: uv run --no-dev --extra docs mkdocs build --strict

- name: Check rendered code documentation
run: uv run --no-dev --extra docs python scripts/check_rendered_code_docs.py

- name: Upload documentation preview artifact
if: github.event_name == 'pull_request' || github.ref != 'refs/heads/main'
uses: actions/upload-artifact@v7
Expand Down
86 changes: 67 additions & 19 deletions .github/workflows/quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,12 @@ jobs:
run: uv run pytest -m baseline_regression tests/regression

test-suite:
name: Full test suite
name: Full test suite (Python ${{ matrix.python-version }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.12"]
steps:
- name: Check out repository
uses: actions/checkout@v6
Expand All @@ -53,35 +57,56 @@ jobs:
enable-cache: true

- name: Install Python
run: uv python install 3.12
run: uv python install ${{ matrix.python-version }}

- name: Install locked dependencies
run: uv sync --locked

- name: Lint supported architecture boundary
run: >-
uv run ruff check
src/blueprinting/__init__.py
src/blueprinting/__main__.py
src/blueprinting/cli
src/blueprinting/schema
src/blueprinting/workload
src/blueprinting/mapping
src/blueprinting/system
src/blueprinting/synthesizer
src/blueprinting/analysis
src/blueprinting/validation
src/blueprinting/application
src/blueprinting/workbench
tests/synthesizer
tests/analysis
tests/application
tests/regression
tests/workbench
src/blueprinting
tests
examples/calculon_calibration.py
scripts

- name: Check source formatting
run: >-
uv run ruff format --check
src/blueprinting
tests
examples/calculon_calibration.py
scripts

- name: Compile runtime type contracts (without mypy)
run: uv run python scripts/check_type_contracts.py

- name: Run tests
run: uv run pytest

static-typing:
name: Optional static typing layer
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v6
with:
fetch-depth: 0

- name: Install uv
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
enable-cache: true

- name: Install Python
run: uv python install 3.12

- name: Install optional static-typing dependencies
run: uv sync --locked --group typing

- name: Type-check Blueprinting source
run: uv run mypy src/blueprinting

package-contract:
name: Base wheel contract
runs-on: ubuntu-latest
Expand All @@ -104,3 +129,26 @@ jobs:

- name: Verify wheel contents and size
run: uv run python scripts/check_wheel_contract.py dist/*.whl

performance-data-contract:
name: Optional performance-data contract
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v6
with:
fetch-depth: 0

- name: Install uv
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
enable-cache: true

- name: Install Python
run: uv python install 3.12

- name: Install locked dependencies with performance-data support
run: uv sync --locked --extra performance-data

- name: Run performance-data provider tests
run: uv run pytest tests/analysis/test_cost_model_providers.py
8 changes: 6 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,9 +173,13 @@ interconnect 与 system profile 位于 `src/blueprinting/system/`;canonical
`src/blueprinting/synthesizer/`;cost/evidence analysis 进入 `src/blueprinting/analysis/`。不得新建平行表示栈。
- `SystemProfile` 是当前有限的 compute/memory/network evidence-bearing adapter,不得被描述成已经实现的完整
`ArchitectureBlueprint`;`src/blueprinting/types/system/` 只服务 legacy calculator,新代码不得依赖它。
- `blueprinting.compiler` Python path 已硬切删除;历史 `compiler.*` canonical codec tag 作为 wire identity 保留,
未经迁移 ADR 不得改写。
- Canonical codec tag 必须位于与领域 ownership 一致的 `blueprinting.*` namespace;不得新增历史 package-derived
tag 或兼容 alias。Pre-graduation 阶段 nested type identity 不携带独立版本号,五层 IR root 统一使用 `0.0.0`;
首次 schema increment 必须有 ADR、真实 migration、产物再生成方案和 conformance gate。
- IR 对象默认 frozen;语义字段使用 typed dataclass/enum/ID,不使用自由字典代替 contract。
- 普通应用代码不得依赖 authoring decorator;canonical schema/dialect 作者只从 `blueprinting.schema.authoring`
使用 `record/adt/variant`,pass/target extension 作者只从 `blueprinting.synthesizer.passes.authoring`
使用 `derivation/relation/claim`。Codec registry、manifest 与 pass registry 是内部实现,不得从 package root 转发。
- 所有公共 derivation/transformation 和 verifier 必须有 positive、negative、round-trip 与 lineage 测试。
- Python 最低版本为 3.10;不得使用只在更高版本解析的语法,除非先更新 packaging contract。
- 修改后至少运行相关 pytest 与 Ruff;文档修改运行双语一致性检查和 `mkdocs build --strict`。
Expand Down
27 changes: 13 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,15 +77,15 @@ For development and documentation tooling:
pip install -e ".[dev,docs]"
```

Static source analysis is an independent optional layer: `pip install -e ".[typing]"`. Runtime type-contract checks do not require mypy.

## Build the current Transformer workload blueprint

```python
from blueprinting.synthesizer.lowering import (
DistributeTransformerTrainingPass,
PlanTransformerTrainingPass,
)
from blueprinting.synthesizer.frontend import build_transformer_model_ir, synthesis_session_for
from blueprinting.synthesizer.passes import PassManager, PassPipeline
from blueprinting.synthesizer.stages.distributed.passes import DistributeTransformerTrainingPass
from blueprinting.synthesizer.stages.portable_plan.passes import PlanTransformerTrainingPass
from blueprinting.mapping import TransformerTrainingMappingSpec
from blueprinting.workload import TransformerModelSpec, TransformerTrainingWorkloadSpec

Expand All @@ -101,11 +101,11 @@ result = PassManager().run(
),
source,
session=synthesis_session_for(model, workload, mapping),
)
).or_raise()

portable_plan = result.ir
for checkpoint in result.checkpoints:
print(checkpoint.pass_name, checkpoint.ir.digest)
print(checkpoint.record.pass_name, checkpoint.ir.digest)
```

The adapter reads the retained model/execution JSON presets in `data/`, then separates workload facts from the
Expand Down Expand Up @@ -139,14 +139,8 @@ TP/PP/DP strategy exploration, and a read-only performance-evidence lab. The evi
Vidur Phi-2/A100 records and compares exact GEMM samples with the analytical roofline on identical workload facts.
Analysis runs outside the UI event loop, and failed candidates remain visible as structured diagnostics.

The existing Calculon Streamlit tools remain isolated as an optional legacy interface. Floating-point analysis is available in the primary NiceGUI workbench:

```bash
uv sync --extra legacy-ui
uv run streamlit run streamlit_app.py
```

Calculon remains an adjacent calibration utility and does not participate in the Blueprinting product analysis path.
Floating-point analysis is available directly in the primary NiceGUI workbench.

## Repository layout

Expand All @@ -159,7 +153,7 @@ src/blueprinting/synthesizer/ # canonical IR, exact-work dialects, and verified
src/blueprinting/analysis/ # evidence protocols, cost resolution, and projections
src/blueprinting/application/ # framework-neutral analysis services and reports
src/blueprinting/validation/ # external baselines and strict regression gates
src/blueprinting/workbench/ # NiceGUI workbench and legacy presentation adapters
src/blueprinting/workbench/ # NiceGUI workbench and presentation adapters

data/evidence/ # optional external evidence, excluded from the base package
tests/ # domain, derivation, application, and regression contracts
Expand All @@ -176,8 +170,13 @@ explicit system, deployment mapping, and evidence snapshot. External oracles rem
pytest
ruff check src/ tests/ examples/calculon_calibration.py
ruff format --check src/ tests/ examples/calculon_calibration.py
uv run python scripts/check_type_contracts.py
# Optional static layer:
uv sync --locked --group typing
uv run mypy src/blueprinting
uv run python scripts/check_docs_i18n.py
uv run mkdocs build --strict
uv run python scripts/check_rendered_code_docs.py
```

## License
Expand Down
4 changes: 2 additions & 2 deletions data/examples/3072_t4_p64_d12_mbs4_full.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"num_procs": 3072,
"tensor_par": 4,
"pipeline_par": 64,
"pipeline_par": 48,
"data_par": 12,
"sequence_par": true,
"tensor_par_net": 0,
Expand All @@ -24,4 +24,4 @@
"optimizer_offload": false,
"training": true,
"zero": 1
}
}
2 changes: 1 addition & 1 deletion data/examples/3072_t4_p64_d12_mbs4_full.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
num_procs = 3072
tensor_par = 4
pipeline_par = 64
pipeline_par = 48
data_par = 12
sequence_par = true
tensor_par_net = 0
Expand Down
64 changes: 46 additions & 18 deletions data/validation/baseline_regression_contract.json
Original file line number Diff line number Diff line change
@@ -1,13 +1,41 @@
{
"schema": "blueprinting.baseline-regression-contract.v1",
"schema": "blueprinting.baseline-regression-contract.v0",
"training": {
"baseline": "Calculon / SeqSel Table 5",
"report_schema": "blueprinting.calculon-calibration-experiment.v0",
"oracle": {
"name": "Calculon",
"package_version": "0.1.0",
"source_digest": "c72cf8a0a0fc9f1fb9813a2248747d6242bbc664b665abe4b5fc6b6b18f5927b"
},
"inputs": {
"models": {
"megatron-22B.json": "fd62296fa7370d0e84291003fb1057f4131553c973f65954afaecf0dea1c3b25",
"gpt3-175B.json": "fabfc66b4a57d3c357d410232a9805b004bbee8f23abbc505c0d23594c208c11",
"turing-530B.json": "2e855371e0abe718b346ad3f876436d0f61953ba6e2c828e68be664596b83667",
"megatron-1T.json": "5d5c2c6678b044897b1ea70fbaf2ab4ded83c643c3998b7e9e26ce57ae2967da"
},
"executions": {
"megatron-22B_full.json": "375e22afecdb4b24cf354b89ccce080c59e097ab8c85b5e73ecc142a120ef005",
"megatron-22B_seqsel.json": "94fc3a6b946fbcd19601410789690e68b36417032473f4cf65bd97e14d97fb74",
"gpt3-175B_full.json": "25ccadd0e39ba81f544ea3b0415189ea42102543d550cf725d8ae15ae99dbd81",
"gpt3-175B_seqsel.json": "99bd010ac34b52578a461aa877d208c65e777932e300b191d7c9e29d38cea6a8",
"turing-530B_full.json": "157ccecce654eb875b6a1682fca6bb2a10309961bc50382db56cfff54bd23bfe",
"turing-530B_seqsel.json": "2d6516a1ce9664951608239838572db5a0129ef794f90d4c46f5924a3221891a",
"megatron-1T_full.json": "24f738a12c5b1fc64b0c0c36bdfc70237032388bc0d383e0aac0d66ebb5b14e6",
"megatron-1T_seqsel.json": "7ba4278645e3ed2dbf26c4dddc007364a56fe495200061183a641de9c0dec9e4"
},
"systems": {
"a100_80g.json": "afdd153ff952babe2102c23c6eba390f67e624eddbd4159f61015a1486a556bc"
}
},
"evidence_revision": "eb1eb9fcc4a6e414e85b0252c23ea9ad2730aae2",
"case_count": 8,
"budgets": {
"workload_max_absolute_error_percent": 1e-9,
"calculon_mean_absolute_error_percent": 1e-9,
"calculon_max_absolute_error_percent": 1e-9,
"breakdown_max_absolute_error_percent": 1e-9,
"paper_mean_absolute_error_percent": 3.7,
"paper_max_absolute_error_percent": 8.9,
"memory_max_absolute_error_bytes": 1.0
Expand All @@ -18,28 +46,28 @@
"paper_mean_absolute_error_percent": 3.654361437637248,
"paper_max_absolute_error_percent": 8.874452316395821,
"portable_digests": {
"seqsel-tab5/megatron-22B/full": "00517ca9fb4df346eebe6d4aa63543ca666a7af4",
"seqsel-tab5/megatron-22B/seqsel": "8eb389c417a9b33124997c21afd5c18597a39404",
"seqsel-tab5/gpt3-175B/full": "2ce66153c5c81ee97c6fa1e2bc14b9e2c0d844ff",
"seqsel-tab5/gpt3-175B/seqsel": "5c888d51fd8d5ffb5bb515ff797a9776a12cea6c",
"seqsel-tab5/turing-530B/full": "99c4b79b949fa4f58ffbd1cc98661d3b2101f5d6",
"seqsel-tab5/turing-530B/seqsel": "4a33b75f901df8c5d4948a8f44d501e8f695674e",
"seqsel-tab5/megatron-1T/full": "0e203dc354d91a45d8860c15aee999d57dbb5bbf",
"seqsel-tab5/megatron-1T/seqsel": "e7c5e86b0f7e9f9f8f3f47124fc643cf4a4d8e09"
"seqsel-tab5/megatron-22B/full": "5a96a71f806f1aede2be8280be8b8436bea3c51d",
"seqsel-tab5/megatron-22B/seqsel": "02857f01085962db40c46d18288d9a61680389b0",
"seqsel-tab5/gpt3-175B/full": "5d8df3e92f712d42df1c8c1c97fffbbb3b5cc9eb",
"seqsel-tab5/gpt3-175B/seqsel": "362754f4f37894aa7bd50eb8e8b7d0319a748b61",
"seqsel-tab5/turing-530B/full": "b3fa239a4ef8a29609d5afcdb58b66ac479efd60",
"seqsel-tab5/turing-530B/seqsel": "126da397b345f17b828170a70e811be98398421f",
"seqsel-tab5/megatron-1T/full": "db115adc9fbe30965631fc3122b1976c751c9b9a",
"seqsel-tab5/megatron-1T/seqsel": "a27fd7b19ea9a120427cbfe6b522d13e4e3a6fc9"
}
}
},
"inference": {
"baseline": "Vidur Phi-2 / A100 / TP1 raw component-profile alignment",
"validation_claim": "drift-detection-not-accuracy-validation",
"fixture": "data/validation/vidur/phi2_a100_tp1",
"fixture_manifest_sha256": "24511af4979e32fdc0d1c610960f93e92ac816a2a5b30f30cf6015d2492b2c33",
"fixture_manifest_sha256": "89d507931b931c3bb1e84277a7bac637028ba3f8e80c842d1c2b554714c689a6",
"source_repository": "https://github.com/microsoft/vidur",
"source_revision": "8383d2935bc62723a212090baa9f98ada206fc14",
"license_file": "LICENSE.vidur",
"license_sha256": "7df20dcdf9197e9945c14858d41c60f11b52b93e5b69e2b63416b874d598d322",
"baseline_revision": "6a583f1c90969bd82ef82b8cf6dd26bcef914909",
"model_digest": "d67e201b24cc428ac0d3330d7fd597cfe90dcdf6",
"baseline_revision": "49f6d006968ffd30b465b3a67741a58053eb9e2b",
"model_digest": "5d48c3b4d4e8bae69e16b02675a0969f9bb81aca",
"case_count": 3,
"budgets": {
"minimum_semantic_component_coverage": 0.75,
Expand All @@ -54,8 +82,8 @@
"system_evidence_component_max_absolute_error_percent": 99.50036630036631,
"cases": {
"phi2-a100-tp1/decode/b1-c33": {
"distributed_digest": "3aaaddf8d05e49da9d675acc2b1dc0e252c4f1fc",
"portable_digest": "d96d85895fa4f19655b2dc0592b5dc402e1bfb0d",
"distributed_digest": "5427664e6e99885931f5f85f584e024287f37a88",
"portable_digest": "a5f111748cca9799c5878b1414fc63b4b749e53c",
"component_coverage": 0.75,
"baseline_comparable_block_seconds": 0.000147,
"system_evidence_comparable_block_seconds": 0.00010824697435897437,
Expand All @@ -64,8 +92,8 @@
"system_evidence_component_max_absolute_error_percent": 99.50036630036631
},
"phi2-a100-tp1/decode/b1-c129": {
"distributed_digest": "727acb5e4a18f54570180de6dfe7c6f28b634b92",
"portable_digest": "ce6e6d1fd1233134300445391f4cd33a4ae7dff3",
"distributed_digest": "c1fb781cb833250bbd1ed95c88f85b949a35a61a",
"portable_digest": "020a2870ec38c774feb4d33ef0fb250f6a22dff3",
"component_coverage": 0.75,
"baseline_comparable_block_seconds": 0.00015000000000000001,
"system_evidence_comparable_block_seconds": 0.00010879564102564103,
Expand All @@ -74,8 +102,8 @@
"system_evidence_component_max_absolute_error_percent": 99.50036630036631
},
"phi2-a100-tp1/prefill/b1-c128": {
"distributed_digest": "56e34687555489beae3d40b66deea30e529bde10",
"portable_digest": "71d41b98eb84dfc57ee3ec0e4614ce1a52622aef",
"distributed_digest": "7fd7bde9661c03f66806246744a4c268b2e69b61",
"portable_digest": "e87aaed848653b5e13bbe5048c828169b344499b",
"component_coverage": 0.75,
"baseline_comparable_block_seconds": 0.0002015,
"system_evidence_comparable_block_seconds": 0.00023554310256410256,
Expand Down
2 changes: 1 addition & 1 deletion data/validation/vidur/phi2_a100_tp1/manifest.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema": "blueprinting.vidur-validation-slice.v1",
"schema": "blueprinting.vidur-validation-slice.v0",
"source": {
"repository": "https://github.com/microsoft/vidur",
"revision": "8383d2935bc62723a212090baa9f98ada206fc14",
Expand Down
4 changes: 2 additions & 2 deletions docs/assets/architecture/implemented-derivation-path.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading