diff --git a/.agents/docs/2026-08-21-freestanding-outstanding-four.md b/.agents/docs/2026-08-21-freestanding-outstanding-four.md index 21d04373..2a9fddbd 100644 --- a/.agents/docs/2026-08-21-freestanding-outstanding-four.md +++ b/.agents/docs/2026-08-21-freestanding-outstanding-four.md @@ -39,6 +39,9 @@ payload: /c/Users/runneradmin/.mcpp/registry/data/xpkgs/xim-x-llvm/*/ ## 1. `mcpp test` 与 `mcpp build` 对 feature 源的判断不一致 +> **已修复(mcpp 2026.8.21.1)。** ⚠️ 而修法既不是路线 A 也不是路线 B —— 是第三次 +> 尝试的判据,它当初被以一个**错误的理由**放弃了。见 1.3.1。 + ### 1.1 现象与位置 同一个包、同一份清单,两条命令编译的源文件集合不同: @@ -82,11 +85,38 @@ base `sources` 里,而 dev 依赖那条轨道的「逐测试 main 探测」需 |---|---| | DROP 与 `!` 排除在两种模式下都跑 | **弄坏 gtest**:mcpp 自己的单测链接失败,`ld returned 1 exit status` | | test 模式只删 glob 字符串、不加 `!` 排除 | 无效:`src/kal/**` 从来不在 `bc.sources` 里,删字符串什么都没删,文件仍由 base glob `src/**` 匹配 | -| 按「该 glob 是否也出现在 base `sources`」判别 | 无效:gtest 的 base 是一条**能匹配到**那个文件的 glob,不是同一个字符串;字符串相等判别不到 | +| 按「该 glob 是否也出现在 base `sources`」判别 | 记录为无效,理由是「gtest 的 base 是一条能匹配到那个文件的 glob,不是同一个字符串」。**这条记录是错的** —— 见 1.3.1 | 三次都已撤销。**不留半个修复**:把一个只在部分情形下正确的门放进已发布的 引擎,比一个有文档的缺陷更糟。 +### 1.3.1 ⚠️ 第三次尝试的判据是对的,被放弃的理由是错的 + +去读索引真正携带的描述符 `pkgs/c/compat.gtest.lua`: + +```lua +sources = { "*/googletest/src/gtest-all.cc", + "*/googletest/src/gtest_main.cc" }, -- 第 71–73 行 +features = { ["main"] = { sources = { "*/googletest/src/gtest_main.cc" } } }, -- 第 90 行 +``` + +两处**逐字节相同**。所以「字符串相等判别不到」这句话与事实不符,而当初把它写进 +文档时没有去读那个文件 —— 判据被一个凭印象写下的理由否掉了。 + +真正让第三次尝试失效的是**判据被施加的时机**:membership 是拿 `bc.sources` 去比 +的,而 `drop()` 已经先一步把那条 glob 从里面删掉了,于是这个测试只可能为假。 + +⭐ **修法**:在 `drop()` 之前把 base 的 glob 集合快照下来,并且只在 test 模式使用 +它 —— + +* glob 同时出现在 base `sources` 与 feature 里 ⇒ 这是一个**门**(gtest 那一族), + test 模式下保持可见,逐测试 main 探测仍然剪得掉它; +* glob 只出现在 feature 里 ⇒ 这是一个**提供者**(riscv-virt-rt 那一族), + test 模式下也要 `!` 排除。 + +⚠️ 而 `!` 排除才是整个机制,删 glob 字符串不是:`src/kal/**` 从来就不在 +`bc.sources` 里(该包根本没声明 `sources`),它的文件由推导出的 `src/**` 匹配。 + ### 1.4 正确的修法 三次失败指向同一个结论:**判据不是「这条 glob 字符串在不在 base 里」,而是 @@ -129,14 +159,19 @@ openkal = { sources = ["src/kal/**"] } # 只经 feat 必须同时满足,**任缺其一即视为未修复**: -1. `mcpplibs/riscv-virt-rt` 不带 feature 时 `mcpp test` 与 `mcpp build` - 编出的目标文件集合相同; -2. mcpp 自己的 `mcpp test unit/test_manifest` 仍然链接并通过(gtest 那一族); -3. 两条断言进 e2e,用两个最小包分别表达上面两族,而不是依赖生态包。 +1. ✅ `mcpplibs/riscv-virt-rt` 不带 feature 时 `mcpp test` 不再编译 + `src/kal/**`,两条命令都以 0 退出(此前 test 死于 + `'openkal/abort.h' file not found`); +2. ✅ mcpp 自己的 92 个单测仍然链接并通过(gtest 那一族); +3. ✅ `tests/e2e/138_feature_sources_gate_vs_provider.sh`,两个最小包分别表达 + 上面两族,不依赖任何生态包。 ⚠️ 第 3 条是重点。这个缺陷之所以活到今天,是因为**没有任何测试同时覆盖两族**; 只修不测,下一次同样的改动会再次在两者之间摇摆。 +⭐ 那条 e2e 已按「回归测试必须先失败」验证过:拿修复前的引擎跑它,它以正是本节 +开头那句症状失败;拿修复后的引擎跑它,它通过。 + ### 1.6 副作用 修好之后,`mcpplibs/riscv-virt-rt` CI 里那句 @@ -237,10 +272,22 @@ libcxx-headers = "22.1.8" # 与产出导出表的那个 libc++ 同版本 **步骤 2:新建 `xim:libcxx-headers`。** -内容是 `include/c++/v1` 与 `share/libc++`,实测 xz 压缩后 **0.9 MB**。它与 -宿主无关,因此**一个包服务全部五个宿主**,而「加进载荷」要给每个缺失的宿主 +内容是 `include/c++/v1` 与 `share/libc++`。复测于 llvm 22.1.8 的载荷: + +| | | +|---|---| +| `include/c++/v1` | 16 MB | +| `share/libc++` | 624 KB | +| 打包后(`tar` + `xz -9`) | **943,180 字节** | + +它与宿主无关,因此**一个包服务全部五个宿主**,而「加进载荷」要给每个缺失的宿主 各加一次。 +⚠️ **这个包尚未上传,而不上传是有意的。** 它今天可以打出来,但没有任何东西验证过 +它在 Windows 上确实让 `std-freestanding` 工作 —— 步骤 1 还没做。把一个未经验证的 +产物先镜像出去,只是让「未经验证的东西」传播得更快。这与 `mcpplibs/qemu-x86` 那边 +定下的顺序是同一条:**先跑通,再发布**。 + **步骤 3:实现层的选择成为可声明的。** 接口层不变,配置层按检测到的实现分支: @@ -285,78 +332,144 @@ libcxx-headers = "22.1.8" # 与产出导出表的那个 libc++ 同版本 --- -## 3. openarch:trap 与 cpu 已实现,目录树重整为三层 +## 3. openarch:三台机器,一个包两个门面,后端由 feature 选择 + +> **本节已由 0.4.0 落地。** 下面记录做成了什么、以及两处本方案原先判断错了的地方。 + +### 3.1 已完成(0.4.0) -### 3.1 已完成(0.3.1) +四个接口 —— contexts、页表项、trap、per-CPU 与屏障 —— 覆盖**三个**指令集: +riscv64、aarch64、x86_64。一份探针源码在三台机器上构建并运行,输出逐字节相同。 -`openarch.trap` 与 `openarch.cpu` 两个接口连同两个后端已实现,探针仍是一份 -源码、在两台机器上输出逐字相同。 +⭐ **第三台机器是把门槛变成证据的那一步,而本方案原先低估了它。** §4 把 x86_64 +当作「模拟器载荷问题」,并把它排在最后;实际上 riscv64 与 aarch64 都是弱内存序、 +定长指令的 load/store RISC 机器,**一个同时适配两者的接口可能是因为它对,也可能 +是因为它们像**,而在这两台机器上再怎么测也分不开这两种情况。x86_64 两样都不是: +变长指令;total store order(四条屏障里三条不需要任何指令);中断机制是 256 个门 +的表;控制台由 `out` 到达,没有任何指针能命名它。 -⭐ **门槛产出的第三条发现:`trap_frame` 必须带 `instr_len`。** 第一版没有它, -理由是「两台机器的陷入指令都是四字节」。实测 rv64gc: +它挖出了三条,而每一条都是两台 RISC 机器合起来也看不见的: + +1. **`MAIR_EL1` 那个决定不再是 aarch64 的例外。** 两台机器时是一比一,「本层拥有 + 属性寄存器」还可以被称作 aarch64 的权宜。x86_64 的 `PWT`/`PCD`/`PAT` 三个分散 + 的位同样构成 `IA32_PAT` 的索引 —— 现在是二比一,方向反了过来。 + ⚠️ **而且它的规则更严。** 未编程的 `MAIR_EL1` 字段读作最严格的类型,过早的 + aarch64 映射只是慢而正确;`IA32_PAT` 的复位值在索引 1 上是 **write-through**, + 过早的设备映射是被缓存的 —— 写在程序没有选择的时刻到达设备,不触发任何异常。 +2. **`pc` 在每台机器上并不指同一件事。** 两台 RISC 机器都报告出错指令的地址; + x86_64 把异常分为 *fault*(如此)与 *trap*(报告**下一条**的地址),而 `int3` + ——`instr_len` 正是为跨过它而存在的断点 —— 是 trap。后端做归一化,于是 + `f->pc += f->instr_len` 在三台机器上都恢复到同一处。 +3. **接口的一条承诺在这台机器的页表项里无法表达。** riscv 用 `U` 限定 `X`, + aarch64 有独立的 `PXN`/`UXN`;x86_64 只有一个覆盖全部特权级的 `NX`,该规则改由 + `CR4.SMEP` 提供。 + +⭐ **0.3.x 的第三条发现仍然成立:`trap_frame` 必须带 `instr_len`。** 实测 rv64gc: ``` cause:3 desc=breakpoint epc:0x800001f2 cause:2 desc=illegal_instruction epc:0x800001f6 ← 无限重复 ``` -`0x800001f2` 不是四字节对齐——C 扩展是 `rv64gc` 的一部分,汇编器发的是两字节 -的 `c.ebreak`,handler 按 4 推进就落进下一条指令中间。aarch64 只有一种指令 -宽度,永远暴露不了这一条。后端知道答案而调用方推导不出来,这正是「该放进 -frame」的定义。 - -其余两处结构性差异: - -* **陷入的形状**。riscv 一个入口加一个原因寄存器;aarch64 十六个槽,而「哪个 - 槽跑了」这半信息在槽跳走之后任何寄存器里都读不到。本层因此接管 aarch64 的 - 向量表,理由与接管 `MAIR_EL1` 一致。⚠️ 每个槽必须在**不破坏被陷入上下文** - 的前提下记下自己的编号——`mov x9, #N` 会毁掉 x9,而那是被中断代码的寄存器。 -* **屏障**。riscv 一条 `fence`、操作数是两个集合的叉积;aarch64 三条含义不同 - 的指令。接口暴露的是两者都能回答的四个问题,而 `complete` 单列是因为 `dmb` - 与 `dsb` 的区别在写设备寄存器时是正确性问题。 +`0x800001f2` 不是四字节对齐 —— C 扩展是 `rv64gc` 的一部分,汇编器发的是两字节的 +`c.ebreak`。aarch64 只有一种指令宽度,永远暴露不了这一条。 -### 3.2 目录树:接口层、ABI 与后端在树上分开 +### 3.2 目录树:混合式的根,以及它顺带修好的一条 ``` openarch/ -├── mcpp.toml [workspace] +├── mcpp.toml [package] openarch 兼 [workspace] +├── src/ C++ 门面 —— 模块 mcpplibs.openarch 再导出四个 +├── tests/ 两个门面必须一致的地方 ├── abi/ 契约 —— 只有头文件,不依赖任何东西 │ ├── mcpp.toml openarch-abi -│ └── include/openarch/ -│ ├── abi.h 后端要实现的东西 -│ └── pte_encode.h 纯编码器,宿主可调用 -├── spec/ 契约之上的 C++ 模块 -│ ├── mcpp.toml openarch → 依赖 openarch-abi -│ ├── src/ context/trap/cpu/pte.cppm -│ └── tests/ 编码器的宿主断言 -├── backends/ 每个指令集一个包 -│ ├── riscv64/mcpp.toml openarch-riscv64 → 依赖 openarch-abi -│ └── aarch64/mcpp.toml openarch-aarch64 → 依赖 openarch-abi -└── examples/switch/ 一份探针源码,在每台机器上运行 +│ └── include/ +│ ├── mcpplibs/openarch.h C 门面,全部 +│ └── openarch/ +│ ├── types.h 宽度,写一次、断言一次 +│ ├── abi.h 后端要实现的东西 +│ └── pte_encode.h 纯编码器,宿主可调用 +├── backends/ 每个指令集一个包,都 provides "openarch-backend" +│ ├── riscv64/ aarch64/ x86_64/ +└── examples/switch/ 一份探针源码 +``` + +⚠️ **本方案原先写的是「根是 `[workspace]`,接口在 `spec/` 成员里」,那个形状是错 +的,而错在一条它自己没有预见的地方。** portability 作业在仓库根跑 +`mcpp build --target riscv64-none-elf`,虚拟 workspace 会**对所有成员扇出** —— +于是 aarch64 汇编被喂给 riscv 汇编器: + +``` +unrecognized instruction mnemonic, did you mean: sra, srl? ``` -⭐ **`spec/` 拥有全部模块、没有一条指令;`backends/` 拥有指令、不导出模块。** -两条都由 CI 断言,而不是交给目录名——0.3.1 之前后端在 `src/arch//`、 -与规范同包,分层是一个由路径撑着的约定。 +混合式的根(同时是 `[package]` 与 `[workspace]`)修好了它:根现在是接口包,构建 +它只拉入该 target 的后端。这个形状同时也是消费者只写一行依赖的原因 —— 虚拟 +workspace 会让 `openarch = "0.4.0"` 不得不指名成员目录。 -**拆分逼出了一个接口变化,而那个变化本身是对的。** 模块实现单元必须与它实现 -的模块同包,所以在 `pte`/`trap`/`cpu` 的边界变成 C ABI 之前,拆包不可能。收益 -不是整洁:边界是 C ABI 的规范可以由**不是 C++ 模块**的东西实现——一份汇编、 -一个厂商的二进制,或者同一指令集在另一个特权级上的第二个后端。riscv 正需要 -最后这一种。 +### 3.2.1 两个门面 -⚠️ **`abi/` 单独成包是依赖图逼出来的。** 第一版把头放在 `spec/`、后端依赖 -`spec/`,而 `spec/` 用 cfg 拉后端以免消费者写自己的架构。mcpp 拒绝: +消费者写一行依赖,然后二选一: +```c +#include /* C,以及想要 C 名字的 C++ */ ``` -error: dependency cycle through package 'openarch' - while computing its build-cache key +```cpp +import mcpplibs.openarch; // 四个模块,再导出 ``` -放弃自动选后端会让每个消费者写下架构;把头复制进每个后端会造出两份必须一致 -而没有机制保证一致的文件。契约不属于任何一侧。 +⭐ 两者是**一个库的两种拼写**,不是两份互相对齐的声明:模块的 `trap_frame` +**就是** `::arch_trap_frame`(`using`,不是同形体),枚举由契约的枚举量*定义而来* +—— `illegal = ARCH_TRAP_ILLEGAL`。`tests/faces.cpp` 检查的是**推导**而不是一致性, +后者是更弱的东西:「两边都是 2」今天成立、明天可能不成立,唯一维持它的是有人同时 +改两处。 + +### 3.2.2 后端由 feature 选择 + +三个曾由一个机制回答的问题被分开了: + +| 消费者要什么 | 清单里写什么 | +|---|---| +| 本 target 的后端 | `openarch = "0.4.0"` | +| 指定某一个 | `default-features = false, features = ["backend-riscv64"]` | +| **自己实现** | `default-features = false, features = ["backend-external"]` + 一个 `provides = ["openarch-backend"]` 的包 | + +`backend-external` 不指名任何包,而是 *require 能力*;图里没有提供者时构建在 +configure 阶段停下并说明,而不是在链接期报出一个改过名的符号。这与 +`std-freestanding` 的分配器同形,于是生态里「一个可被替换的默认实现」只有一种 +写法而不是两种。 -**消费者写的东西一字未变**:`openarch = "0.3.1"`,后端由目标解析。 +⚠️ **`backend-auto` 刻意不 require 该能力,而第一版让它 require 了。** feature 是 +可加的,而 `requires` 是无条件的 —— 哪怕满足它的 `feature-deps` 是 target 条件化 +的。于是本包自己的宿主测试无法构建: + +``` +error: no package provides capability 'openarch-backend' required by 'openarch' +``` + +宿主目标没有后端是**关于目标的事实**,不是消费者能处理的错误。 + +### 3.2.3 类型集中到一处 + +`openarch/types.h` 定义 `arch_u32`/`arch_u64`/`arch_uptr` 并**断言它们的宽度**。 +此前每处用点各自拼出 `unsigned long long`,顶上一段注释解释为什么不是 +`unsigned long` —— 一条被描述而从未被检查的规则。它唯一一次被违反(`1UL << 53`) +是靠运气发现的:那个移位恰好在 `constexpr` 里,编译器被迫求值。 + +⚠️ `arch_uptr` **不**断言为八字节。页表项在每台机器上都是 64 位(包括 32 位机器), +指针不是,而 `riscv32-none-elf` 是本仓库打算到达的目标。断言指针是八字节会在今天 +测过的每台机器上通过,而那正是 openkal 在 `fs.h` 里犯过的错。 + +### 3.2.4 ⚠️ CI 从 0.3.0 起一直是红的,而我此前报告过它是绿的 + +两处,都是我写的断言把「意图」和「它实际匹配的模式」搞混了: + +1. **「探针不按架构分支」这条太宽。** 它 grep `__riscv|__aarch64__`,而探针**必须** + 在恰好一处指名架构 —— 陷入指令,`ebreak` / `brk #0` / `int3` 是同一个想法的三种 + 拼写,没有可移植的第四种。trap 接口在 0.3.0 落地时这条断言就开始失败,按它自己 + 的字面是对的、按它的意图是错的,而它一直红到 0.3.1 因为没有人去读那些 run。 + 收窄为:**一个**条件块,块内除指令外别无他物。 +2. portability 作业的扇出问题,见 §3.2。 ### 3.3 仍未做:时钟,以及它是否属于这一层 @@ -380,7 +493,67 @@ S 模式那份是否只能经 SBI。两个答案决定接口归属,而调研比 ## 4. x86_64 裸机目标 -### 4.1 卡在哪里 +> **本节的阶段 4 已由 mcpp 2026.8.21.1 提前落地,而落地过程推翻了本节的一个前提。** +> 阶段 1–3(`xim:qemu-x86` 的构建与收录)仍然未做,分析依旧成立。 + +### 4.0 ⚠️ 「不是代码,是模拟器载荷」这句话是错的 + +本节原先断言目标行本身没有工作量,只差一个模拟器。实测下来目标行需要**引擎 +代码**,而原因是 clang 的属性、不是指令集的属性。 + +clang 由 triple 选工具链。它为 arm / aarch64 / riscv 备有 *BareMetal* 工具链, +直接以 `ld.lld` 链接;**它没有 x86_64 的**,于是裸 x86_64 triple 的每一种写法都 +落到通用 GCC 工具链上 —— 而后者的链接器是**宿主的 `g++`**: + +``` +g++: error: unrecognized command-line option '-fuse-ld=/…/llvm/22.1.8/bin/ld.lld' +``` + +对 `x86_64-none-elf`、`x86_64-unknown-none-elf`、`x86_64-unknown-none`、 +`x86_64-elf`、`x86_64-none-none`、`x86_64-unknown-unknown` 逐一实测,结果一致; +`-fuse-ld=lld` / `--ld-path=` / `--gcc-toolchain=` / `-B` 逐一实测,均不改变结果。 +唯一能改变它的是把 `linux` 放进 OS 位 —— 那会给一次裸机链接带来**八条宿主 `-L`**。 + +两种结果都不可接受:经宿主 `g++` 会让这一行只在 Linux 宿主上成立(而 macOS 与 +Windows 宿主根本没有能产 ELF 的 `g++`);宿主搜索路径出现在 freestanding 链接 +上,正是引擎要守住的封闭性。 + +**解法**:ISA 档表新增 `lldEmulation` 列;置位时引擎直接用 `ld.lld` 驱动链接。 +标志的词汇随工具一起改变 —— `-Map=` 而非 `-Wl,-Map=`。⚠️ 仅属于驱动的标志是 +**丢弃**而非翻译,而第一次尝试漏了两个: + +``` +ld.lld: error: unknown argument '-nostdlib++' +ld.lld: error: unknown argument '-Wl,--disable-new-dtags' +``` + +⚠️ riscv 与 aarch64 两行该列**留空**。它们的驱动本就到得了 lld,为了让三行看起来 +一致而改动一条可用的链接,正是引入回归的方式。 + +第二列 `extra` 承载 `-mno-red-zone`,也不是偏好:System V 的 128 字节红区在有 OS +的机器上安全,是因为内核为中断切了栈;裸机上处理器把中断帧压进红区,被中断的叶 +函数恢复后局部变量已被覆盖 —— 不触发异常、没有诊断,而且只在中断恰好落在叶函数 +内部时发生。 + +### 4.0.1 ⭐ 探针能跑起来,靠的是 multiboot 的 a.out kludge + +QEMU 的 multiboot 装载器**只接受 32 位 ELF**: + +``` +qemu-system-x86_64: Cannot load x86-64 image, give a 32bit one. +``` + +而 ELF 的 class 是整个文件的属性,x86-64 代码产不出 ELF32。走的是 multiboot 的 +另一条路 —— flag 位 16 的 a.out kludge,头里自带装载地址,装载器根本不解析 ELF。 + +⭐ 让这条路的算术成立的是 `SIZEOF_HEADERS`:装载器算的起始文件偏移是 +`header_addr - load_addr`,所以这个差必须等于 multiboot 头在文件里的真实偏移。 +把镜像起点写成 `0x100000 + SIZEOF_HEADERS`,ELF 头恰好占满 `load_addr` 与 +`header_addr` 之间的字节,偏移与地址保持同余、链接器不插填充,差值**按构造**就是 +偏移。按平常写法(`. = 0x100000`)`.multiboot` 落在文件偏移 0x1000 而地址 +0x100000,`load_addr` 就得是 0xFF000 —— 落在写入会被丢弃的 legacy BIOS 窗口里。 + +### 4.1 阶段 1–3 卡在哪里 不是代码,是模拟器载荷。`qemu-riscv` 描述符里写明了本索引的收录门槛:为它 服务的五个宿主目标(linux x64/arm64、darwin x64/arm64、win32 x64)从**同一个 @@ -447,10 +620,14 @@ QEMU 可以从源码构建,而且**只构建需要的目标**能把代价压下 `openxlings/xim-pkgindex`。DT_NEEDED 闭包按 `qemu-arm` 的做法**实测**后决定 `deps` 是否为空,而不是从兄弟描述符抄结论。 -**阶段 4:目标表加 `x86_64-none-elf` 行。** +**阶段 4:目标表加 `x86_64-none-elf` 行。✅ 已完成(mcpp 2026.8.21.1)。** 与 `aarch64-none-elf` 同样是零 libc 档:索引里没有 x86 的裸机 C 库,而第一批 -消费者——UEFI 应用与 openarch 的第三个后端——都不需要。 +消费者 —— UEFI 应用与 openarch 的第三个后端 —— 都不需要。 + +⚠️ 这一阶段本来排在最后,理由是「等模拟器」。实际顺序反了过来:目标行先落地, +openarch 的第三个后端因此写得出来,而**第三台机器正是把门槛从「适配」变成 +「抽象」的那一步**(见 §3.1)。模拟器仍然缺,CI 的那一行用 apt 装并注明了原因。 ### 4.4 判据 @@ -468,10 +645,15 @@ QEMU 可以从源码构建,而且**只构建需要的目标**能把代价压下 | | 项 | 状态 | 建议 | |---|---|---|---| | 1 | Windows libc++(第 2 节) | 未做。⭐ 结论已从「加进载荷」**改为独立成包**,理由见 2.5 | 先做 2.1 的测量,再按三步落地。**我们自己能做完** | -| 2 | feature 源不一致(第 1 节) | 未做,三次修法被证伪 | 需要 glob 展开 + 两族的 e2e | -| 3 | openarch 的 trap / cpu(第 3 节) | ✅ **0.3.1 已完成**,目录树重整为 abi/spec/backends 三层 | — | +| 2 | feature 源不一致(第 1 节) | ✅ **已修复**(2026.8.21.1)。⚠️ 修法就是第三次尝试的判据 —— 它当初被一个**没有去读文件就写下的理由**否掉了 | — | +| 3 | openarch(第 3 节) | ✅ **0.4.0 已完成**:混合式的根、两个门面、feature 选后端、三个指令集 | — | | 4 | openarch 的时钟归属(3.3) | 未做 | ⭐ 先作为**调研**:两种启动方式各一个最小探针,答案决定接口归属 | -| 5 | x86_64 裸机(第 4 节) | 未做 | 阶段 1 先做 linux x64 供 openkal-uefi 用;阶段 2 用**临时 PR 的 CI 矩阵**跑通五条腿再谈收录 | +| 5 | x86_64 裸机(第 4 节) | ✅ **阶段 4 已完成**(目标行 + 引擎的直连链接);阶段 1–3 未做 | `xim:qemu-x86` 仍缺。阶段 1 先做 linux x64 供 openkal-uefi 用;阶段 2 用**临时 PR 的 CI 矩阵**跑通五条腿再谈收录 | + +⚠️ **本方案排序里有一处判断错了,值得记下来。** 第 5 项原先排在最后,理由是它 +「卡在模拟器载荷」。实际做下来,它是**唯一一项改变了对已完成工作之信心**的: +第三台机器挖出了三条两台 RISC 机器合起来也看不见的东西(§3.1)。一个「被外部 +依赖卡住」的条目和一个「价值低」的条目在列表上看起来一样,而它们不是一回事。 ⚠️ 排期时要当外部依赖的只剩两处,比本文第一版少了一处:`xim:qemu-x86` 需要 进 `openxlings/xim-pkgindex`;QEMU 的五宿主构建若沿用 xPack 的脚本,需要与那个 diff --git a/docs/13-baremetal.md b/docs/13-baremetal.md index c87defc4..f95c8d48 100644 --- a/docs/13-baremetal.md +++ b/docs/13-baremetal.md @@ -13,30 +13,79 @@ covers the hosted link model this chapter departs from. ## Overview A freestanding target is a target whose `os` field is `none`. The target table -at `src/toolchain/triple.cppm` carries three of them: +at `src/toolchain/triple.cppm` carries four of them: | Triple | Tier | C library | |---|---|---| | `riscv64-none-elf` | verified | `xim:picolibc-riscv` | | `riscv32-none-elf` | verified | `xim:picolibc-riscv` | | `aarch64-none-elf` | preview | none — the zero-libc tier | +| `x86_64-none-elf` | preview | none — the zero-libc tier | `verified` means an image has been built **and run** for the row. `preview` means it builds and has been observed to run, but is not yet covered by the engine's own emulator jobs. -⚠️ The third row's C library column is empty, and that is a statement rather -than an omission: no aarch64 build of picolibc exists in the package index, and -the first consumer of that row — the `openarch` layer of machine mechanism — -references no C library symbol. An empty column means exactly what -`[target.].sysroot = ""` means in a manifest, so a project targeting it +⚠️ The last two rows' C library column is empty, and that is a statement rather +than an omission: no aarch64 or x86_64 build of picolibc exists in the package +index, and the first consumer of both rows — the `openarch` layer of machine +mechanism — references no C library symbol. An empty column means exactly what +`[target.].sysroot = ""` means in a manifest, so a project targeting one begins on the zero-libc tier without asking. A project that wants a C library on -that target declares one, which is also how it would choose a different one. +those targets declares one, which is also how it would choose a different one. Such a target needs no per-host cross toolchain. clang and lld are cross-compilers by construction — one binary emits every target it was built with — so the target table pins `llvm@22.1.8` on every host, and any machine -that can install the LLVM payload can produce an image for any of the three. +that can install the LLVM payload can produce an image for any of the four. + +### The x86_64 row is not four strings + +⚠️ **A target row is normally an entry in two tables and nothing else. This one +needed engine code, and the reason is a property of clang rather than of the +instruction set.** + +clang selects a toolchain from the triple. It has a *BareMetal* toolchain for +arm, aarch64 and riscv, which links with `ld.lld` directly; it has none for +x86_64, so every spelling of a bare x86_64 triple falls through to the generic +GCC toolchain — whose linker is the **host's `g++`**: + +``` +g++: error: unrecognized command-line option '-fuse-ld=/…/llvm/22.1.8/bin/ld.lld' +``` + +Measured for `x86_64-none-elf`, `x86_64-unknown-none-elf`, `x86_64-unknown-none`, +`x86_64-elf`, `x86_64-none-none` and `x86_64-unknown-unknown`, and unchanged by +`-fuse-ld=lld`, `--ld-path=`, `--gcc-toolchain=` or `-B`. The one thing that +does change it is putting `linux` in the OS position, which makes clang link +directly and adds eight host `-L` paths to a bare-metal link. + +Neither outcome is acceptable: routing through a host `g++` makes the row work +on a Linux host and nowhere else, and host search paths on a freestanding link +are the hermeticity this engine exists to keep. So the row carries a fifth +column, `lldEmulation`, and when it is set the engine drives the link with +`ld.lld` itself. The flag vocabulary changes with the tool — `-Map=` rather than +`-Wl,-Map=`, `-m elf_x86_64` rather than `--target=` — and the driver-only flags +(`-nostdlib++`, the loader tag) are dropped rather than translated. + +The column is empty for the riscv and aarch64 rows. Their driver already reaches +lld, and changing a working link to make three rows look alike is how a +regression is introduced. + +### `-mno-red-zone` is part of the target, not a preference + +The System V x86-64 ABI reserves 128 bytes below `rsp` that a leaf function may +use without adjusting the stack pointer, because on a hosted system nothing else +writes there. On bare metal the processor pushes an interrupt frame at `rsp` — +into the red zone — and the interrupted leaf resumes to find its locals +overwritten. There is no fault and no diagnostic, and it happens only when an +interrupt arrives inside a leaf. + +There is no bare-metal x86_64 program for which the red zone is safe, so the +flag is a property of the row rather than something a project remembers. It +reaches the command line through a new `extra` column in the ISA-profile table, +which exists because `-march`/`-mabi`/`-mcmodel` could not express it. RISC-V +and aarch64 have no equivalent, which is why the column did not exist before. Three things a bare-metal build requires are not properties of the ISA, and mcpp does not attempt to derive them: which startup object and libraries to diff --git a/docs/zh/13-baremetal.md b/docs/zh/13-baremetal.md index 3cd782d9..494d970e 100644 --- a/docs/zh/13-baremetal.md +++ b/docs/zh/13-baremetal.md @@ -11,26 +11,66 @@ ## 概述 freestanding 目标是 `os` 字段为 `none` 的目标。`src/toolchain/triple.cppm` -的目标表中有三个: +的目标表中有四个: | Triple | 档位 | C 库 | |---|---|---| | `riscv64-none-elf` | verified | `xim:picolibc-riscv` | | `riscv32-none-elf` | verified | `xim:picolibc-riscv` | | `aarch64-none-elf` | preview | 无 —— 零 libc 档 | +| `x86_64-none-elf` | preview | 无 —— 零 libc 档 | `verified` 意味着该行的镜像被构建**并被运行**过。`preview` 意味着它构建得出、 也被观察到能运行,但尚未纳入引擎自己的模拟器作业。 -⚠️ 第三行的 C 库列为空,这是声明而非遗漏:包索引里不存在 aarch64 的 picolibc -构建,而这一行的第一个消费者 —— 机器机制层 `openarch` —— 一个 C 库符号都不引用。 -空列在这里的含义与清单里 `[target.].sysroot = ""` 完全一致,因此面向它的 -工程无需声明即处在零 libc 档。想要 C 库的工程自行声明一个,而那也正是它换用另一份 -C 库的做法。 +⚠️ 后两行的 C 库列为空,这是声明而非遗漏:包索引里不存在 aarch64 与 x86_64 的 +picolibc 构建,而这两行的第一个消费者 —— 机器机制层 `openarch` —— 一个 C 库符号 +都不引用。空列在这里的含义与清单里 `[target.].sysroot = ""` 完全一致, +因此面向它们的工程无需声明即处在零 libc 档。想要 C 库的工程自行声明一个,而那也 +正是它换用另一份 C 库的做法。 这类目标不需要逐宿主的交叉工具链。clang 与 lld 在构造上就是交叉编译器 —— 一个二进制发射它构建时支持的全部目标 —— 因此目标表在每个宿主上都钉 -`llvm@22.1.8`,任何能安装 LLVM 载荷的机器都能为这三个中的任何一个产出镜像。 +`llvm@22.1.8`,任何能安装 LLVM 载荷的机器都能为这四个中的任何一个产出镜像。 + +### x86_64 这一行不止是四个字符串 + +⚠️ **一个目标行通常就是两张表里的两条记录,不含引擎代码。这一行需要引擎代码, +而原因是 clang 的属性而非指令集的属性。** + +clang 由 triple 选择工具链。它为 arm、aarch64、riscv 备有 *BareMetal* 工具链, +直接以 `ld.lld` 链接;它没有 x86_64 的,于是裸 x86_64 triple 的每一种写法都落到 +通用 GCC 工具链上 —— 而后者的链接器是**宿主的 `g++`**: + +``` +g++: error: unrecognized command-line option '-fuse-ld=/…/llvm/22.1.8/bin/ld.lld' +``` + +对 `x86_64-none-elf`、`x86_64-unknown-none-elf`、`x86_64-unknown-none`、 +`x86_64-elf`、`x86_64-none-none`、`x86_64-unknown-unknown` 逐一实测,结果一致; +`-fuse-ld=lld`、`--ld-path=`、`--gcc-toolchain=`、`-B` 逐一实测,均不改变结果。 +唯一能改变它的是把 `linux` 放进 OS 位,而那会给一次裸机链接带来八条宿主 `-L`。 + +两种结果都不可接受:经宿主 `g++` 会让这一行只在 Linux 宿主上成立;宿主搜索路径 +出现在 freestanding 链接上,正是本引擎存在的意义所要守住的封闭性。因此该行带了 +第五列 `lldEmulation`,置位时引擎直接用 `ld.lld` 驱动链接。标志的词汇随工具一同 +改变 —— `-Map=` 而非 `-Wl,-Map=`,`-m elf_x86_64` 而非 `--target=` —— 而仅属于 +驱动的标志(`-nostdlib++`、加载器标签)是被丢弃而不是被翻译。 + +riscv 与 aarch64 两行的该列为空。它们的驱动本就到得了 lld,为了让三行看起来一致 +而改动一条可用的链接,正是引入回归的方式。 + +### `-mno-red-zone` 是目标的属性,不是偏好 + +System V x86-64 ABI 在 `rsp` 之下保留 128 字节,叶函数可以不调整栈指针直接使用, +因为在有操作系统的机器上没有别的东西会写那里。裸机上不存在做这件事的一方:处理器 +在 `rsp` 处压入中断帧 —— 正压进红区 —— 被中断的叶函数恢复后发现自己的局部变量已被 +覆盖。既不触发异常也没有诊断,而且只在中断恰好落在叶函数内部时发生。 + +不存在红区安全的裸机 x86_64 程序,所以这个标志是行的属性,而不是工程需要记住的 +事情。它经 ISA 档表新增的 `extra` 列到达命令行 —— 该列之所以存在,是因为 +`-march`/`-mabi`/`-mcmodel` 表达不了它。RISC-V 与 aarch64 没有对应物,这也正是 +此前不需要这一列的原因。 裸机构建需要的三样东西并不是 ISA 的属性,mcpp 也不试图推导它们:选哪个启动对象 与哪些库、哪份链接脚本描述这台机器的内存、以及如何执行产出的镜像。这三样随 diff --git a/mcpp.toml b/mcpp.toml index 7138431f..e5ada811 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.8.20.3" +version = "2026.8.21.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 15d24c04..8b8ca8ec 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -59,6 +59,16 @@ struct CompileFlags { std::filesystem::path ccBinary; // gcc / clang (derived; cl.exe = same) std::filesystem::path arBinary; // ar / llvm-ar / lib.exe (empty → PATH) std::filesystem::path ldBinary; // link.exe (SeparateLinker dialects only) + // ⚠️ THE LINK IS DRIVEN BY THIS BINARY INSTEAD OF BY THE COMPILER, AND + // WHEN IT IS SET THE WHOLE FLAG VOCABULARY CHANGES WITH IT. + // + // Empty everywhere except a freestanding target whose row carries an + // `lldEmulation` — today `x86_64-none-elf`, where clang delegates the link + // to the host's `g++`. `ld`/`ldC` then hold LINKER flags rather than driver + // flags: `-Map=` and not `-Wl,-Map=`, `-m elf_x86_64` and not `--target=`. + // A caller that mixes the two gets a diagnostic from the linker naming a + // flag the reader never wrote. + std::filesystem::path ldDriver; std::string sysroot; // --sysroot=... (for ninja ldflags) std::string bFlag; // -B (for ninja ldflags) bool staticStdlib = true; @@ -1327,8 +1337,22 @@ CompileFlags compute_flags(const BuildPlan& plan) { // RISC-V firmware image with an x86-64 PT_INTERP baked in, which // links clean and reports success. Measured, on this very change, // before this line existed. - std::string fsLd = isClangWithCfg ? " --no-default-config" : ""; - fsLd += mcpp::freestanding::link_flags(*spec, in, ninjaEsc); + // ⚠️ TWO SHAPES, AND WHICH ONE IS A PROPERTY OF THE TARGET'S ROW. + // + // With an `lldEmulation` the compiler driver is bypassed entirely, + // because for that target it would hand the link to a host `g++` + // that cannot take our linker's path — see the column's comment in + // mcpp.freestanding.target. `--no-default-config` is then not + // needed either: the payload's clang config file is a DRIVER + // config, and nothing here runs the driver. + std::string fsLd; + if (!spec->lldEmulation.empty()) { + f.ldDriver = in.lld; + fsLd = mcpp::freestanding::link_flags_direct(*spec, in, ninjaEsc); + } else { + fsLd = isClangWithCfg ? " --no-default-config" : ""; + fsLd += mcpp::freestanding::link_flags(*spec, in, ninjaEsc); + } f.ld = fsLd + user_ldflags; f.ldC = f.ld; // Nothing hosted survives: these carry payload/sysroot/-B flags @@ -1337,6 +1361,22 @@ CompileFlags compute_flags(const BuildPlan& plan) { f.bFlag.clear(); f.ldRuntimeFallback.clear(); f.linkage = "static"; + // ⚠️ WHEN THE LINKER IS DRIVEN DIRECTLY, EVERY REMAINING DRIVER + // FLAG IS AN ERROR RATHER THAN A NO-OP. Measured on the first + // attempt, where two survived the freestanding replacement and + // reached `ld.lld`: + // + // ld.lld: error: unknown argument '-nostdlib++' + // ld.lld: error: unknown argument '-Wl,--disable-new-dtags' + // + // The first is the per-role C++ runtime contract; the second is the + // loader tag, which names a dynamic-section entry an image with no + // loader does not have. Both are meaningful to a compiler driver + // and meaningless here, so they are cleared rather than translated. + if (!f.ldDriver.empty()) { + for (auto& r : f.ldStdlibByRole) r.clear(); + for (auto& r : f.ldStdlibCByRole) r.clear(); + } } } diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index c755f865..8055e0d3 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -633,6 +633,11 @@ std::string emit_ninja_string(const BuildPlan& plan) { flags.ldBinary.empty() ? std::string("link.exe") : escape_ninja_path(flags.ldBinary))); } + // The linker itself, for a freestanding target whose compiler driver would + // hand the link to a foreign one. Bound only when set, so a graph that does + // not use it has no variable naming a tool it never runs. + if (!flags.ldDriver.empty()) + append(std::format("ld_driver = {}\n", escape_ninja_path(flags.ldDriver))); // `$mcpp` is needed by stage_file in EVERY configuration (dyndep or not), // so the binding cannot live inside the `if (dyndep)` below. append(std::format("mcpp = {}\n", escape_ninja_path(mcpp_exe_path()))); @@ -1152,6 +1157,19 @@ std::string emit_ninja_string(const BuildPlan& plan) { "$ld /nologo /DLL /OUT:$out $implib_flag $def_flag " "$in $ldflags $unit_ldflags", "SHARED"); + } else if (!flags.ldDriver.empty()) { + // ⚠️ THE LINKER, NOT THE DRIVER, AND THE OBJECTS COME LAST. + // + // A compiler driver accepts objects anywhere on the line and sorts + // them out; `ld` resolves left to right, so a library named before + // the object that needs it contributes nothing and the link fails + // on a symbol that is present. This shape is only reached by a + // freestanding target whose row carries an `lldEmulation`. + link_rule("cxx_link", + "$ld_driver $ldflags $unit_ldflags -o $out $in", "LINK"); + link_rule("cxx_archive", std::string(dial.archiveCmd), "AR"); + link_rule("c_link", + "$ld_driver $c_ldflags $unit_ldflags -o $out $in", "LINK"); } else { link_rule("cxx_link", "$cxx $in -o $out $ldflags $unit_ldflags", "LINK"); @@ -2008,14 +2026,26 @@ std::string emit_ninja_string(const BuildPlan& plan) { // in every graph that builds a static library. if (lu.kind != LinkUnit::StaticLibrary) tail.runtimeFallback = flags.ldRuntimeFallback; - tail.loaderTag = lu.loaderTagFlag; + // ⚠️ NOT ON A DIRECT LINK. The tag selects between `DT_RPATH` + // and `DT_RUNPATH`, entries of a dynamic section; an image with no + // loader has neither, and `ld.lld` rejects the flag's `-Wl,` form + // outright. + if (flags.ldDriver.empty()) tail.loaderTag = lu.loaderTagFlag; // The link map — per unit, because it is named after the artifact. if (!fsObjcopy.empty() - && (lu.kind == LinkUnit::Binary || lu.kind == LinkUnit::TestBinary)) - tail.dependencies += mcpp::freestanding::map_flag( - lu.output, [](const std::filesystem::path& q) { - return escape_ninja_path(q); - }); + && (lu.kind == LinkUnit::Binary || lu.kind == LinkUnit::TestBinary)) { + // ⚠️ `-Map=` OR `-Wl,-Map=`, DECIDED BY WHO IS BEING SPOKEN TO. + // With `ldDriver` set the link is `ld.lld` itself, and `-Wl,` + // is a driver's way of saying "pass this on" — handed to the + // linker it is an unknown option, on the one target where the + // map matters most. + auto esc = [](const std::filesystem::path& q) { + return escape_ninja_path(q); + }; + tail.dependencies += flags.ldDriver.empty() + ? mcpp::freestanding::map_flag(lu.output, esc) + : mcpp::freestanding::map_flag_direct(lu.output, esc); + } if (auto unit = tail.render(); !unit.empty()) out_line += " unit_ldflags =" + unit + "\n"; } @@ -2276,6 +2306,33 @@ std::expected NinjaBackend::build(const BuildPlan& plan "compile_commands.json was not updated: {}", cdb.error().message)); } + // ⚠️ A SHARED LIBRARY ON A TARGET WHOSE LINK IS DRIVEN BY THE LINKER, SAID + // IN WORDS RATHER THAN AS A MISSING NINJA RULE. + // + // The direct-linker path defines `cxx_link`, `c_link` and `cxx_archive` and + // no shared-library rule, because a target with no loader has nothing to + // load one. Without this check the failure is + // + // ninja: error: build.ninja:88: unknown build rule 'cxx_shared' + // + // which names an internal rule to somebody who wrote `kind = "shared"` in a + // manifest. Defining the rule would be worse: `ld.lld -shared` succeeds and + // produces an object nothing on that machine can load. + if (!flags.ldDriver.empty()) { + for (auto const& lu : plan.linkUnits) { + if (lu.kind != LinkUnit::SharedLibrary) continue; + return std::unexpected(BuildError{std::format( + "target '{}' is a shared library, and '{}' has no dynamic " + "loader to load one.\n" + " A freestanding image is linked statically because there is " + "no other option: nothing on that machine resolves a symbol at " + "run time.\n" + " Use `kind = \"lib\"` for a static library, or `kind = " + "\"bin\"` for the image itself.", + lu.targetName, plan.toolchain.targetTriple)}); + } + } + // A distribution contract that could not be honored is reported, never // silently downgraded — the whole point of the model (INV-1/INV-4 in // .agents/docs/2026-08-02-issue336-pr142-analysis.md). Emitted here rather diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index d6bd7aa2..c6881e61 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -4535,6 +4535,45 @@ prepare_build(bool print_fingerprint, // doubly-listed gtest_main.cc cannot land twice. auto& bc = pkg.manifest.buildConfig; if (!bc.featureSources.empty()) { + // ⭐ WHETHER A FEATURE *GATES* A SOURCE OR *PROVIDES* IT, AND + // THE ANSWER IS WRITTEN IN THE MANIFEST ALREADY. + // + // Two families of package reach this code and they want + // opposite things under `mcpp test`: + // + // gtest lists `*/googletest/src/gtest_main.cc` in + // base `sources` AND under `features.main`. + // The package provides the file unconditionally; + // the feature is a gate over it. The + // dev-dependency track's per-test main detection + // must still SEE it in order to prune it per + // test, so an inactive gate must not make it + // vanish. + // + // riscv-virt-rt names `src/kal/**` under `features.openkal` + // and nowhere else. The package does not provide + // those files at all without the feature — the + // headers they include arrive through that + // feature's `[feature-deps]` — so compiling them + // fails on `'openkal/abort.h' file not found`. + // + // The discriminator is membership in base `sources`, evaluated + // BEFORE the drop below removes it. A glob in both places is a + // gate; a glob in one place is a provider. + // + // ⚠️ THIS IS THE FOURTH ATTEMPT, AND THE THIRD WAS ABANDONED ON + // A MISTAKEN READING. It was recorded as failing because + // "gtest's base entry is a glob that MATCHES the file rather + // than the same string". Measured against the descriptor the + // index actually carries, the two entries are byte-identical + // (`compat.gtest.lua` lines 71 and 90). The criterion was + // sound; what it was applied to was not — the earlier attempt + // compared against `bc.sources` AFTER `drop()` had already + // removed the entry, so the membership test could only ever be + // false. + std::set baseGlobs(bc.sources.begin(), bc.sources.end()); + baseGlobs.insert(pkg.manifest.modules.sources.begin(), + pkg.manifest.modules.sources.end()); if (!includeDevDeps) { // glob → owned by at least one ACTIVE feature? std::set activeNow(active.begin(), active.end()); @@ -4558,6 +4597,30 @@ prepare_build(bool print_fingerprint, pkg.manifest.modules.sources.push_back("!" + g); } } + else { + // `mcpp test`. The gate that build mode applies wholesale is + // applied here only to the globs the package provides + // NOWHERE ELSE, which leaves gtest's doubly-listed source + // visible and stops riscv-virt-rt's feature-only sources + // from being compiled without their feature. + // + // ⚠️ THE `!` EXCLUSION IS THE WHOLE MECHANISM, NOT THE GLOB + // REMOVAL. `src/kal/**` is never IN `bc.sources` — the + // package declares no `sources` at all and its files are + // matched by the inferred `src/**`. Erasing the string + // erases nothing; only an exclusion gates. + std::set activeNow(active.begin(), active.end()); + std::map gated; + for (auto& [f, globs] : bc.featureSources) + for (auto& g : globs) + gated[g] = gated[g] || activeNow.contains(f); + for (auto& [g, isActive] : gated) { + if (isActive || g.starts_with("!")) continue; + if (baseGlobs.contains(g)) continue; // a gate, not a provider + bc.sources.push_back("!" + g); + pkg.manifest.modules.sources.push_back("!" + g); + } + } std::set activeSet(active.begin(), active.end()); auto add = [](std::vector& v, const std::string& g) { if (std::ranges::find(v, g) == v.end()) v.push_back(g); diff --git a/src/freestanding/linkline.cppm b/src/freestanding/linkline.cppm index 23aee76e..54b31b52 100644 --- a/src/freestanding/linkline.cppm +++ b/src/freestanding/linkline.cppm @@ -97,6 +97,49 @@ inline std::string link_flags(const Spec& s, const LinkInputs& in, return out; } +// The same link, expressed for `ld.lld` itself rather than for a driver. +// +// Used when the target's row carries an `lldEmulation` — see the column's +// comment in mcpp.freestanding.target for the measurement that made this +// necessary. The two lines are deliberately NOT one function with a flag: a +// driver line and a linker line share no vocabulary, and writing them together +// would mean a conditional at every token. +// +// ⚠️ WHAT CLANG WOULD HAVE ADDED, AND WHAT IS DELIBERATELY NOT REPRODUCED. +// Measured on llvm 22.1.8 for the two rows whose driver does reach lld: it +// passes `-Bstatic`, `-m `, `-X`, and three `-L` paths under +// `lib/clang-runtimes//lib` and `lib/clang/22/lib/`. The `-L`s +// name directories THAT DO NOT EXIST in this payload — it ships no bare-metal +// runtime at all — so reproducing them would add three paths to nothing. +// `-X` discards local symbols, which changes what `nm` reports about an image; +// a link that silently drops symbols is not the same link. +inline std::string link_flags_direct(const Spec& s, const LinkInputs& in, + const std::function& esc) +{ + std::string out; + out += " -m " + std::string(s.lldEmulation); + out += " -Bstatic"; + // No loader exists, so an image must not name one. The driver line reaches + // this through `-static`; here it is said to the linker. + out += " --no-dynamic-linker"; + if (!in.sysrootLib.empty()) + out += " -L" + esc(in.sysrootLib); + if (!in.linkerScript.empty()) + out += " -T " + esc(in.linkerScript); + return out; +} + +// The map flag, for a linker rather than a driver: no `-Wl,` to route it +// through. +inline std::string map_flag_direct(const std::filesystem::path& artifact, + const std::function& esc) +{ + auto m = artifact; m += ".map"; + return " -Map=" + esc(m); +} + // The link map, for the question only a map can answer on a bare-metal target: // why a section is where it is, and why something did or did not get pulled in. // diff --git a/src/freestanding/target.cppm b/src/freestanding/target.cppm index 0f69b1ba..77440229 100644 --- a/src/freestanding/target.cppm +++ b/src/freestanding/target.cppm @@ -44,6 +44,68 @@ struct Spec { // own multilib convention (`/`), carried here so a wrapper // package can name its layout without re-deriving the profile. std::string_view libdir; + // ⚠️ FLAGS THIS ISA REQUIRES THAT THE FOUR COLUMNS ABOVE CANNOT EXPRESS. + // + // The table had three flag columns because three were enough for RISC-V and + // aarch64, where everything a freestanding build needs is an ISA profile, a + // calling convention and a code model. x86_64 is the first row that needs a + // fourth thing, and it is not a preference: see `-mno-red-zone` below. + // + // Empty for every row that does not, which is every row but one. + std::span extra; + + // ⚠️ NON-EMPTY MEANS "THIS TARGET'S LINK CANNOT GO THROUGH THE COMPILER + // DRIVER", AND THAT IS A PROPERTY OF CLANG RATHER THAN OF THE ISA. + // + // Every freestanding link is driven by clang, which selects a toolchain + // from the triple and, for the rows that have one, ends up invoking + // `ld.lld` directly. Measured on llvm 22.1.8, for every spelling of a bare + // x86_64 triple — `x86_64-none-elf`, `x86_64-unknown-none`, `x86_64-elf`, + // `x86_64-none-none` — clang instead invokes the HOST'S `g++` as the + // linker driver: + // + // g++: error: unrecognized command-line option + // '-fuse-ld=/…/llvm/22.1.8/bin/ld.lld' + // + // clang has a BareMetal toolchain for arm, aarch64 and riscv and none for + // x86_64, so that triple falls through to the generic GCC toolchain, whose + // linker IS gcc. No flag changes it: `-fuse-ld=lld`, `--ld-path=`, + // `--gcc-toolchain=`, `-B` were each measured and each left `g++` in + // place. Only putting `linux` in the OS position does — and that brings + // eight host `-L` paths onto a bare-metal link, which is the hermeticity + // this engine exists to keep. + // + // So for such a row the engine links with `ld.lld` itself and this column + // carries the emulation name that clang would have passed. Empty for the + // riscv and aarch64 rows: their driver already reaches lld, and changing a + // working link to prove a point is how a regression gets introduced. + std::string_view lldEmulation; +}; + +// The flags each row's `extra` column points at. Named arrays rather than +// inline braces because a `span` must refer to storage that outlives it, and a +// temporary array in an aggregate initialiser does not. +inline constexpr std::string_view kX86_64NoneExtra[] = { + // ⚠️ WITHOUT THIS, EVERY TRAP HANDLER ON THIS TARGET CORRUPTS THE FUNCTION + // IT INTERRUPTED, AND NOTHING REPORTS IT. + // + // The System V x86-64 ABI reserves 128 bytes below `rsp` — the red zone — + // which a leaf function may use without adjusting the stack pointer, + // because on a hosted system nothing else writes there: the kernel switches + // to its own stack for interrupts, and signal frames are placed clear of + // it. + // + // On bare metal nothing does that for you. The processor pushes an + // interrupt frame at `rsp` — into the red zone — and the leaf function + // resumes to find its locals overwritten. There is no fault and no + // diagnostic; the value is simply wrong, and only sometimes, because it + // depends on whether an interrupt happened to arrive inside a leaf. + // + // This is a property of the target rather than of a project, which is why + // it is here: there is no bare-metal x86_64 program for which the red zone + // is safe. RISC-V and aarch64 have no equivalent, which is why the column + // did not exist until this row. + "-mno-red-zone", }; // ⚠️ Defaults, not the only possibility. `rv64gc/lp64d` is what qemu `virt` @@ -78,6 +140,31 @@ inline constexpr Spec kTable[] = { // The engine reads this column only when a sysroot exists, so an invented // value would be a value nothing could ever check. { "aarch64-none-elf", "armv8-a", "aapcs", "small", "" }, + // ⚠️ `x86-64` WITH A HYPHEN, WHICH IS THE ONE PLACE THIS TRIPLE'S TWO + // SPELLINGS DIVERGE. The triple segment is `x86_64` with an underscore and + // the `-march` value is `x86-64` with a hyphen; deriving one from the other + // is a substitution that looks harmless and produces `unknown target CPU`. + // + // `x86-64` is the baseline, not a modern microarchitecture level. A row + // that named `x86-64-v3` would produce images that fault on hardware and + // emulators older than roughly 2015, for a target whose whole audience is + // people who do not control what they run on. A project that knows its + // machine raises it in its own manifest. + // + // `sysv` is the only ABI this target has, and unlike aarch64's `aapcs` it + // is also the driver's default. It is written out because this table is the + // place the decision is made, not the place it is inherited. + // + // `small` places code and data in the low 2 GiB, which is where a + // `-kernel` image loaded by an emulator runs. A higher-half kernel — one + // linked at 0xFFFFFFFF80000000 — selects `-mcmodel=kernel` in its own + // manifest; that is a linker-script decision and cannot be a default, + // because the two are wrong for each other rather than merely suboptimal. + // + // ⚠️ THE LIBDIR IS EMPTY FOR THE SAME REASON aarch64's IS: this row + // resolves no C library, so a value here could never be checked. + { "x86_64-none-elf", "x86-64", "sysv", "small", "", kX86_64NoneExtra, + "elf_x86_64" }, }; // The single read point. Returns nullopt for anything that is not a known @@ -110,6 +197,10 @@ inline std::vector compile_flags(const Spec& s) { out.emplace_back(std::string("-mabi=") + std::string(s.mabi)); if (!s.mcmodel.empty()) out.emplace_back(std::string("-mcmodel=") + std::string(s.mcmodel)); + // Whatever the three columns above could not say. Emitted before + // `-ffreestanding` so the ordering of this function's output stays a + // function of the table rather than of the row. + for (auto flag : s.extra) out.emplace_back(flag); out.emplace_back("-ffreestanding"); // ⚠️ No C++ standard library headers. Not a preference — the toolchain's // libc++ headers are built for the HOST: `#include ` resolves to diff --git a/src/toolchain/triple.cppm b/src/toolchain/triple.cppm index b9204e22..c159d1db 100644 --- a/src/toolchain/triple.cppm +++ b/src/toolchain/triple.cppm @@ -160,6 +160,27 @@ inline constexpr TargetInfo kKnownTargets[] = { // has actually booted under it, claiming `verified` would be claiming the // measurement rather than reporting it. { "aarch64-none-elf", "preview", "bare","llvm@22.1.8","", true }, + // ⚠️ THIS ROW EXISTS SO THAT A THIRD MACHINE CAN DISAGREE WITH THE FIRST + // TWO, WHICH IS THE ONLY THING THAT TELLS AN ABSTRACTION FROM A HABIT. + // + // riscv64 and aarch64 are both load/store RISC machines with a weak memory + // model and a fixed instruction width, so an interface that fits both may + // fit because it is right or because they are alike. x86_64 is neither: it + // has variable-length instructions, a total-store-order memory model under + // which three of openarch's four barriers need no instruction at all, and + // an interrupt mechanism that is a table of gates rather than a base + // register. What survives all three is an abstraction. + // + // ⚠️ The tier is `preview` for the same reason aarch64's is, and the reason + // is stricter than it sounds: `verified` here means an image was built AND + // RUN. `xim:qemu-x86` does not exist yet — the index carries no + // `qemu-system-x86_64` — so nothing on this row has booted. Claiming + // `verified` would be claiming a measurement that has not been made. + // + // The sysroot column is empty, the zero-libc tier, for the reason given + // above `aarch64-none-elf`: the first consumer is `openarch`, which + // references no C library symbol. + { "x86_64-none-elf", "preview", "bare","llvm@22.1.8","", true }, }; inline std::span known_targets() { return kKnownTargets; } diff --git a/src/version.cppm b/src/version.cppm index 831d6fa7..5eee0183 100644 --- a/src/version.cppm +++ b/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.8.20.3"; +inline constexpr std::string_view MCPP_VERSION = "2026.8.21.1"; } // namespace mcpp diff --git a/tests/e2e/137_x86_64_zero_libc_target.sh b/tests/e2e/137_x86_64_zero_libc_target.sh new file mode 100755 index 00000000..099fe9ee --- /dev/null +++ b/tests/e2e/137_x86_64_zero_libc_target.sh @@ -0,0 +1,179 @@ +#!/usr/bin/env bash +# requires: llvm unix-shell +# `x86_64-none-elf`: the third bare-metal architecture, and the first whose +# LINK the compiler driver refuses to perform. +# +# ⚠️ WHAT THIS ROW ADDED THAT THE OTHER TWO DID NOT NEED +# +# `riscv64-none-elf` and `aarch64-none-elf` are four strings in a table and +# nothing else: clang has a BareMetal toolchain for both, so it drives their +# links itself and reaches `ld.lld`. It has none for x86_64, so that triple +# falls through to the generic GCC toolchain — whose linker is the HOST'S `g++`: +# +# g++: error: unrecognized command-line option +# '-fuse-ld=/…/llvm/22.1.8/bin/ld.lld' +# +# Measured for every spelling of a bare x86_64 triple, and unfixable by any flag +# (`-fuse-ld=lld`, `--ld-path=`, `--gcc-toolchain=`, `-B` were each tried). Only +# putting `linux` in the OS position makes clang link directly, and that brings +# eight host `-L` paths onto a bare-metal link. +# +# So this row carries an `lldEmulation` and the engine links it with `ld.lld` +# itself. Three of the checks below exist for that path specifically: that the +# host toolchain is not consulted, that driver-only flags do not survive onto a +# linker's command line, and that the map is still produced. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +"$MCPP" new x86probe > /dev/null +cd x86probe +rm -f tests/*.cpp 2>/dev/null || true + +# ⚠️ NO `[target.*]` SECTION. Like `aarch64-none-elf`, the zero-libc tier is +# this row's own property rather than something the project asks for. +cat > mcpp.toml <<'EOF' +[package] +name = "x86probe" +version = "0.1.0" + +[build] +target = "x86_64-none-elf" +EOF + +# The console on this machine is not memory: x86 reaches a serial port through +# `out`, which is why the other two probes' `volatile unsigned*` has no +# counterpart here. +cat > src/main.cpp <<'EOF' +namespace { +inline void outb(unsigned short port, unsigned char v) { + asm volatile("outb %0, %1" :: "a"(v), "Nd"(port)); +} +} +extern "C" void kmain() { + for (const char* p = "x86 ok\n"; *p; ++p) outb(0x3F8, static_cast(*p)); + for (;;) { asm volatile("hlt"); } +} +asm(".section .text.entry,\"ax\",@progbits\n" + ".globl _start\n" + "_start:\n" + " movq $__stack_top, %rsp\n" + " call kmain\n" + "1: hlt\n" + " jmp 1b\n"); +EOF + +cat > link.ld <<'EOF' +ENTRY(_start) +SECTIONS { + . = 0x100000; + .text : { *(.text.entry) *(.text*) } + .rodata : { *(.rodata*) } + .data : { *(.data*) } + .bss : { *(.bss*) *(COMMON) } + . = ALIGN(16); . = . + 0x4000; __stack_top = .; +} +EOF + +cat > build.mcpp <<'EOF' +import mcpp; +int main() { mcpp::link_script("link.ld"); return 0; } +EOF + +# ── 1. It builds, which for this row means the link did not go through gcc ── +"$MCPP" build > build.log 2>&1 || { cat build.log; echo "x86_64-none-elf did not build"; exit 1; } + +ELF=$(find target/x86_64-none-elf -type f -name x86probe | head -1) +[ -n "$ELF" ] || { cat build.log; echo "no image was produced"; exit 1; } + +READELF=$(find "$HOME/.mcpp/registry/data/xpkgs/xim-x-llvm" -name llvm-readelf 2>/dev/null | head -1) +NM=$(find "$HOME/.mcpp/registry/data/xpkgs/xim-x-llvm" -name llvm-nm 2>/dev/null | head -1) +[ -x "$READELF" ] && [ -x "$NM" ] || { echo "the LLVM payload has no readelf/nm to check with"; exit 1; } + +# ── 2. It is x86-64, and it starts where the script says ──────────────────── +"$READELF" -h "$ELF" | grep -q "X86-64" || { + "$READELF" -h "$ELF" | head -12; echo "the image is not x86-64"; exit 1; } +"$READELF" -S "$ELF" | grep -E '\.text' | grep -q '0000000000100000' || { + "$READELF" -S "$ELF" | head -8 + echo "the linker script's load address did not reach the image"; exit 1; } + +# ── 3. Nothing is left undefined, and nothing names a loader ──────────────── +UNDEF=$("$NM" -u "$ELF" | wc -l) +[ "$UNDEF" -eq 0 ] || { + "$NM" -u "$ELF"; echo "the image expects $UNDEF symbols nothing provides"; exit 1; } + +# ⚠️ An image with no operating system under it must not name an interpreter. +# The direct-linker path says `--no-dynamic-linker` where the driver path said +# `-static`; a PT_INTERP here would be that substitution having been dropped, +# and it is exactly the defect the `--no-default-config` comment in flags.cppm +# records for the riscv row: it links clean and reports success. +if "$READELF" -l "$ELF" | grep -q "INTERP"; then + "$READELF" -l "$ELF" | head -12 + echo "a bare-metal image names a dynamic loader"; exit 1 +fi + +# ── 4. The link map was produced, through a linker-native flag ────────────── +# `-Wl,-Map=` is how a DRIVER is asked for a map. Handed to `ld.lld` it is an +# unknown option, so the map's existence is evidence that the flag was rendered +# for the tool actually being invoked. +[ -f "$ELF.map" ] || { ls -la "$(dirname "$ELF")"; echo "no link map beside the image"; exit 1; } + +# ── 5. The host toolchain was not consulted ───────────────────────────────── +# The load-bearing assertion for this row. A link that had fallen back to the +# driver would name `g++` — and would then work on this machine and on no +# other, which is the failure this row exists to prevent. +rm -rf target +"$MCPP" build --verbose > verbose.log 2>&1 || { cat verbose.log; exit 1; } +grep -q "ld.lld" verbose.log || { + grep -iE "link|ld" verbose.log | tail -5 + echo "the link did not go through ld.lld"; exit 1; } + +# ── 6. The ISA profile is the measured one ────────────────────────────────── +# `-march=x86-64` with a HYPHEN, where the triple segment has an underscore; +# substituting one for the other produces `unknown target CPU`. And +# `-mno-red-zone`, without which any trap handler on this target corrupts the +# leaf function it interrupted, with no diagnostic. +grep -q -- "-march=x86-64" verbose.log || { + grep -o -- '-march=[a-z0-9_-]*' verbose.log | sort -u + echo "the x86_64 ISA profile is not the measured one"; exit 1; } +grep -q -- "-mno-red-zone" verbose.log || { + echo "the red zone was left enabled on a target that takes interrupts"; exit 1; } + +# ── 7. A shared library on this target is refused in words ───────────────── +# +# ⚠️ THE DIRECT-LINKER PATH DEFINES NO SHARED-LIBRARY RULE, AND WITHOUT A CHECK +# THE FAILURE NAMES AN INTERNAL ONE: +# +# ninja: error: build.ninja:88: unknown build rule 'cxx_shared' +# +# to somebody who wrote `kind = "shared"` in a manifest. Defining the rule would +# be worse than omitting it: `ld.lld -shared` succeeds and produces an object +# nothing on a machine with no loader can load. +cd "$TMP" +mkdir -p shared/src && cd shared +cat > mcpp.toml <<'EOF' +[package] +name = "shr" +version = "0.1.0" + +[build] +target = "x86_64-none-elf" + +[targets.shr] +kind = "shared" +EOF +echo 'int f() { return 1; }' > src/lib.cpp +if "$MCPP" build > shared.log 2>&1; then + cat shared.log; echo "a shared library was built for a target with no loader"; exit 1 +fi +grep -q "no dynamic loader" shared.log || { + cat shared.log + echo "the refusal does not name the situation"; exit 1; } +if grep -q "unknown build rule" shared.log; then + cat shared.log + echo "the failure names an internal ninja rule instead of the cause"; exit 1 +fi + +echo "PASS: x86_64-none-elf builds on the zero-libc tier, linked by ld.lld directly" diff --git a/tests/e2e/138_feature_sources_gate_vs_provider.sh b/tests/e2e/138_feature_sources_gate_vs_provider.sh new file mode 100755 index 00000000..f976d537 --- /dev/null +++ b/tests/e2e/138_feature_sources_gate_vs_provider.sh @@ -0,0 +1,147 @@ +#!/usr/bin/env bash +# requires: unix-shell +# A feature that GATES a source and a feature that PROVIDES one, under both +# `mcpp build` and `mcpp test`. +# +# ⚠️ THIS DEFECT SURVIVED FOR MONTHS BECAUSE NO TEST COVERED BOTH FAMILIES, AND +# THREE ATTEMPTED FIXES EACH BROKE THE ONE THEY DID NOT COVER. +# +# Two shapes of package reach the same code in `prepare_build`, and under +# `mcpp test` they want opposite things: +# +# the GATE family gtest lists `*/googletest/src/gtest_main.cc` in base +# `sources` AND under `features.main`. The package +# provides the file unconditionally; the feature is a +# switch over it. The dev-dependency track's per-test +# main detection has to SEE it to prune it per test, so +# an inactive gate must not make it disappear. +# +# the PROVIDER family riscv-virt-rt names `src/kal/**` under +# `features.openkal` and nowhere else. Those files are +# not part of the package without the feature — the +# headers they include arrive through that feature's +# `[feature-deps]` — so compiling them fails on a header +# that was never meant to be there. +# +# The engine gated the whole exclusion on "is this a test build", which is +# right for the first family and wrong for the second. Fixing it by removing +# the gate broke gtest; fixing it by removing only the glob string was a no-op, +# because a provider's glob is not in base `sources` at all and its files are +# matched by the inferred `src/**`. +# +# Both families are asserted here so that the next change to this code cannot +# satisfy one at the other's expense. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +# ── The PROVIDER family ───────────────────────────────────────────────────── +# +# `src/gated/**` is named only by the feature, and the file includes a header +# that does not exist. With the feature inactive both commands must ignore it; +# a command that compiles it fails on the missing header, which is exactly the +# symptom the real package produced. +mkdir -p provider/src/gated provider/tests +cat > provider/mcpp.toml <<'EOF' +[package] +name = "provider" +version = "0.1.0" + +# ⚠️ NO `[build] sources`, WHICH IS THE POINT. The base set is the inferred +# `src/**`, which matches `src/gated/only_with_feature.cpp` — so removing the +# feature's glob STRING from a list it was never in gates nothing. +[features] +default = [] +gated = { sources = ["src/gated/**"] } +EOF +cat > provider/src/lib.cpp <<'EOF' +int provider_value() { return 7; } +EOF +cat > provider/src/gated/only_with_feature.cpp <<'EOF' +// The header arrives with the feature's dependencies and does not exist +// without it. Compiling this file without the feature is the defect. +#include +int gated_value() { return 1; } +EOF +cat > provider/tests/basic.cpp <<'EOF' +extern int provider_value(); +int main() { return provider_value() == 7 ? 0 : 1; } +EOF + +cd provider +"$MCPP" build > build.log 2>&1 || { cat build.log; echo "provider: build compiled a feature-only source"; exit 1; } +"$MCPP" test > test.log 2>&1 || { cat test.log; echo "provider: TEST compiled a feature-only source — the defect this file exists for"; exit 1; } + +if find target -name '*only_with_feature*' | grep -q .; then + find target -name '*only_with_feature*' + echo "provider: an object was produced for a source the feature did not activate" + exit 1 +fi + +# With the feature ON it must be reached — otherwise the check above would pass +# for a build that simply ignores feature sources entirely. +if "$MCPP" build --features gated > gated.log 2>&1; then + cat gated.log + echo "provider: the feature did not bring its own source in" + exit 1 +fi +grep -q "a_header_the_feature_would_have_brought.h" gated.log || { + cat gated.log + echo "provider: activating the feature failed for some other reason"; exit 1; } +cd .. + +# ── The GATE family ───────────────────────────────────────────────────────── +# +# The same glob in base `sources` and under a feature. `mcpp build` excludes it; +# `mcpp test` keeps it visible, which is what the dev-dependency track's +# per-test main detection depends on. +mkdir -p gate/src gate/tests +cat > gate/mcpp.toml <<'EOF' +[package] +name = "gate" +version = "0.1.0" + +# ⚠️ `src/gated_main.cpp` IS IN BOTH LISTS, EXACTLY AS gtest'S DESCRIPTOR HAS IT. +# The package provides the file; the feature is a switch over it. +[build] +sources = ["src/lib.cpp", "src/gated_main.cpp"] + +[features] +default = [] +main = { sources = ["src/gated_main.cpp"] } +EOF +cat > gate/src/lib.cpp <<'EOF' +int gate_value() { return 11; } +EOF +cat > gate/src/gated_main.cpp <<'EOF' +int gated_main_marker() { return 42; } +EOF +cat > gate/tests/basic.cpp <<'EOF' +extern int gate_value(); +int main() { return gate_value() == 11 ? 0 : 1; } +EOF + +cd gate +"$MCPP" build > build.log 2>&1 || { cat build.log; echo "gate: build failed"; exit 1; } +if find target -name '*gated_main*' | grep -q .; then + find target -name '*gated_main*' + echo "gate: build compiled a gated source with the feature inactive" + exit 1 +fi + +rm -rf target +"$MCPP" test > test.log 2>&1 || { cat test.log; echo "gate: test failed"; exit 1; } +# ⚠️ THE ASSERTION IS THAT IT IS STILL THERE. A fix that excluded every +# inactive feature glob under `mcpp test` would remove it — and would then +# break gtest, whose per-test main detection needs to see the file in order to +# prune it. Measured once as `ld returned 1 exit status` across mcpp's own +# unit tests. +find target -name '*gated_main*' | grep -q . || { + echo "gate: a source the package provides unconditionally vanished under \`mcpp test\`" + echo " (this is the shape that broke gtest)" + exit 1; } +cd .. + +echo "PASS: a feature that gates a source and a feature that provides one behave correctly under both commands"