diff --git a/.agents/docs/2026-08-27-openkal-ecosystem-design-plan.md b/.agents/docs/2026-08-27-openkal-ecosystem-design-plan.md new file mode 100644 index 00000000..d6bbe3ef --- /dev/null +++ b/.agents/docs/2026-08-27-openkal-ecosystem-design-plan.md @@ -0,0 +1,1958 @@ +# openkal 生态:接口判据、面的补全,与端口层的组合 + +2026-08-27 · 设计方案 + 分批清单(**待 review,尚未实施**) + +配套文档:[`2026-08-27-openkal-native-path-three-issues.md`](2026-08-27-openkal-native-path-three-issues.md) +—— 那份是**三个 issue 的缺陷分析与 mcpp 引擎侧的修复**;本份是 **openkal 及其生态 +仓库的设计/优化升级**,两份不重复。 + +--- + +## 0. 一句话 + +> **openkal 提供面向内核的通用原子能力,组合由端口层做一次,既有生态继承它。** +> +> 于是「openkal 该不该有 X」这个问题,永远先问「X 能不能由已有的原子能力组合出来」, +> 而不是问「有没有程序需要 X」。 + +本文按这条把 27 条运行期失败重排,得到:**四个接口该加**(两小、一中、一大)、 +**七条落在端口层**(其中可复用的那几条归 §2.7 的 `openkal-kit`)、**两条是应用自己的事**。 + +⚠️ 第四个(`openkal.space`,§2.4)是**复查掉两条错误理由之后**才进来的 —— 见 §1.3, +那两条错误本身比结论更值得读。 + +--- + +## 1. 判据:三次替换,最后一条是规范自带的 + +⚠️ **前两条都判错过,记录在此,因为犯错的方式值得记。** + +| # | 判据 | 判错了什么 | +|---|---|---| +| 1 | 「是不是通用内核能力」 | ⚠️ 要人判断。**漏掉了就绪与终端**;把 `net` 判成「等第二个消费者」 | +| 2 | 「传统体系下已可移植的程序用到的,必须有」 | ⚠️ 好用的**筛**,但把 `poll` 误判进来 —— 可移植程序用它是因为 POSIX 给了它,不是因为它不可约 | +| 3 | **clause 10 基数 + 原子性 + 端口层组合** | ⭐⭐ **规范自带,且裁决了前两条的冲突** | + +### 1.1 最终判据(三问,按顺序) + +**问一:一个程序里有几个?**(SPEC clause 10 的基数表) + +| | 一个程序里几个实现 | 语法上表现为 | 契约形式 | +|---|---|---|---| +| openarch | 一个,即处理器 | —— | concepts | +| **openkal** | **每个接口一个** | **「我的 X」**:我的标准输出、我的进程、我的时钟、我的网络栈 | **C ABI** | +| openhal | 多个,即那些设备 | **「哪一个 X」**:哪块声卡、哪个摄像头、哪个 UART | concepts | + +⭐ **判别一个能力归谁,只需问它的名字里是「我的」还是「哪一个」。** +这条与 backend 是 MMIO 驱动还是 ALSA 封装**完全无关** —— 这正是「不关心具体 +backend 实现」的意思。 + +**问二:能不能由已经命名的原子能力组合出来?** +能 ⇒ 它是**端口层**的事,不是 openkal 的。 +判据在仓库里已有一句话形态(`okm_syscall.c:836`,解释 `openkal.random` 为何必须存在): + +> **Entropy is not derivable from the other interfaces** … Neither bypassing the +> layer nor inventing entropy was acceptable, so the layer gained an interface. + +**问三:它的形状是不是某个 OS 的形状?** +是 ⇒ 拆到原子为止。`socket`+`setsockopt`+`sockaddr` 是 POSIX 的形状; +「连到一个端点,得到一个流」才是原子。 + +### 1.2 ⚠️ 一处我说过头的,必须更正:`fork` 不是「硬天花板」 + +前几轮我写过「`fork` 永远不可能,是通用性的硬天花板」。**按判据二,这话是错的。** + +clause 7.1 禁止的是 **`kal_fork` 这个操作**(复制地址空间与执行状态不能在每个环境上 +忠实完成)。它**没有**禁止**地址空间的原子能力**: + +``` +创建一个地址空间 · 复制一个地址空间 · 在其中启动一个上下文 +``` + +有了这三样,端口层**可以**把 `fork` 组合出来 —— Fuchsia 没有 `fork` 却有 +`zx_process_create`/`zx_vmar_map`/`zx_thread_start`,seL4 亦然。 + +⚠️ **本节初稿把第三样写成「以给定寄存器状态启动」,那是错的** —— 那会逼 openkal +认识 `arch::context`,而 clause 10 不允许(C ABI ↔ concepts)。正确的形状与解法 +见 §2.4。 + +按三问过一遍: + +| 问 | 答 | +|---|---| +| 几个? | 一个地址空间管理器 ⇒ **openkal** ✔ | +| 可导出? | ❌ 只有环境能造地址空间 ✔ | +| OS 的形状? | ❌ Fuchsia/seL4/Linux 三种内核都有这个形状,**而 `fork` 才是被绑定的那一个** ✔ | + +⇒ ⭐ **它通过,而且它比 `openkal.net` 更「内核」。⇒ 列入本方案,见 §2.4。** + +### 1.3 ⚠️ 我为「不做」给过三条理由,两条是错的 + +初稿把它记成「候选但不在本批」,理由三条。逐条复查: + +| # | 理由 | 复查 | +|---|---|---| +| 1 | 「面上最大的一次扩张」 | ❌ **算错了** —— 分解对了是 **5 个操作**,比 `openkal.net` 的 6 个还少(§2.4) | +| 2 | 「无 MMU 目标必然缺席」 | ❌ **这根本不是反对理由,它就是 openkal 的设计** —— clause 6.1:实现不提供的接口以**链接期缺席**表达。`openkal.fs` 在 opensbi 上就是缺席的。**由后端选择与实现,正是可组合设计的意思** | +| 3 | 「今天没有消费者」 | ⚠️ 这是**排期**理由,不是设计理由。⭐ 而且它反过来成立:见下 | + +⭐⭐ **理由 3 反过来是「现在就要设计」的论据:`kal_process_spawn` 与地址空间原子能力 +在语义上重叠。** 前者接一个**路径**,由环境完成加载;后者让程序自己造进程。 +两者的关系 —— spawn 是原子之上的复合,还是一个独立接口 —— **必须一次决定**, +否则将来会出现「造一个进程有两条路」,而 clause 8 不允许事后改。 + +⇒ **设计现在做,实现按排期。** 这与「简洁」不冲突:简洁是**面的大小**,不是 +**决定的推迟**。 + +--- + +## 2. openkal(规范包)—— 四个接口,一条记录 + +⚠️ **§2.8 是对本章的深度 review**(推翻了 §2.1/§2.3/§2.4 的部分内容); +**§2.9 是完整性验收** —— 面够不够用,以及唯一一个真实的原子缺口; +**§2.10 用四问回扫**,推翻了 §2.8 ⓹ 的结论并删掉了 §2.8 ⓸。 +**§2.11 是第二次 review**,采纳截止时间方案并补上它的真实代价。 +**§2.12 是六个接口的头文件草案**,**§2.13 是规范/ABI 级 review(六处发现)**, +**§2.14 是 clause 6.3 / clause 11 的增补文本**。 +⭐ **只读一节的话读 §2.13** —— 写草案时撞出来的问题都在那里。 + +⚠️ **每一条的形状都不是它在 POSIX 里的形状。这是本章的要点。** + +### 2.1 `openkal.terminal`(小)—— 独立接口,不是往 stream 里加 + +**依据**:「raw 模式」在不同资源上表现不同(终端 vs 普通文件),正是 clause 6.4 的 +形状 —— 它当时裁定 `seek` 因此属于 `openkal.fs`(资源是 descriptor)而**不**属于 +`openkal.stream`。同理,终端控制属于一个资源为「交互式流」的接口。 + +**入口已经存在**:`KAL_STREAM_PROP_INTERACTIVE`。 + +**面(约 2 操作 + 1 询问)**: + +``` +kal_terminal_set_raw(stream, int on) 不要解释我的输入（行编辑） +kal_terminal_set_echo(stream, int on) 不要回显 +kal_terminal_size(stream, &cols, &rows) 显示多大；不知道就说不知道 +kal_terminal_props +``` + +**不是 `termios`** —— 那是一个带六十个标志位的 POSIX 结构。 + +**各环境**:Linux `tcsetattr`;Windows Console API;裸机 UART **没有行编辑** ⇒ +「关掉它」是**无事可做即完成**(`okm_opt.h` 已有这类先例:释放一个不可能取得的句柄); +尺寸报「不知道」。 + +⚠️ **未决**:`set_raw` 与 `set_echo` 是两个操作还是一个「输入处理模式」? +POSIX 把它们放在一个结构里是历史,不是必然。**倾向两个** —— 它们可以独立变化。 + +### 2.2 `kal_process_channel`(小)—— `openkal.process` 的一次加法 + +**缺口**:`kal_spawn_streams` 已经收流句柄(`process.h:17`),缺的是**取得一个跨得过 +spawn 边界的流**。 + +``` +kal_process_channel(struct kal_stream* parent_end, + struct kal_stream* child_end) +``` + +⭐ **比「给我子进程的 stdout」更原子**:调用方自己决定把 `child_end` 装到子进程的 +哪一个流上,于是 `popen("r")`、`popen("w")`、双向捕获、`2>&1` 都由端口层组合出来。 + +⚠️ **进程内的**流对**不属于这里** —— 可由缓冲 + `kal_task_wait/wake` 导出 ⇒ 是库。 +这条存在的理由**只有**「跨地址空间」。 + +**依据**:clause 8 允许 *"A revision may add declarations"*。 + +### 2.3 `openkal.net`(大)—— 与 `openkal.fs` 同构 + +**依据三条,全部现成**: + +- **clause 10**:NIC(openhal,多个)→ 网络栈(openkal,一个),与 + 块设备 → `openkal.fs` **同构**。fs 已经在规范里,net 的理由与它逐字相同。 +- **clause 3.4**:已经点名 `openkal.net`,并指出它与 fs 的差别正是 **half-closure**。 +- **clause 3.4**:拒绝「解析无边界的名字方案」⇒ **DNS 不在里面**。 + +**面(约六个操作)**: + +``` +kal_net_connect(endpoint) → stream 连到一个端点 +kal_net_listen(endpoint) → listener 接受入站 +kal_net_accept(listener) → stream +kal_net_shutdown(stream, dir) 半关闭 —— clause 3.4 点名的那件事 +kal_net_close_listener(listener) +kal_net_props +``` + +**不在里面**(每一条都是 POSIX 的形状而非原子能力): +地址族与 `sockaddr` 家族 · `setsockopt` 的选项空间 · 非阻塞标志(就绪归端口层) · +`sendmsg/recvmsg` · **名字解析**。 +⇒ endpoint 是**结构化的地址 + 端口**,不是字符串;`getaddrinfo` 是它之上的库。 + +⚠️⚠️ **两处必须动笔前定,不能边写边定(clause 8:接口发布后不可更改):** + +1. **数据报要不要?** 流与数据报是两种资源(clause 6.4 的形状)。 + ⭐ **连锁后果是真的**:若 v1 只做流,DNS 只能走 TCP。这不是细节。 +2. **谁来定形?** 按 [[second-instance-exposes-the-interface]],需要**两个形态不同的 + 实现**才知道分解对不对 —— 一个宿主内核(openkal-linux over sockets)+ 一个 + BSP over openhal(lwIP)。⚠️ **同一个作者的两份实现等于零证据** + ([[openkal-portable-program-findings]])。 + +### 2.4 `openkal.space`(中)—— 地址空间的原子能力 + +**为什么它是原子的而不是 `fork`**:clause 7.1 拒绝 `fork` 的理由是「复制地址空间**与 +执行状态**不能在每个环境上忠实完成」。⭐ 拆开看,不能忠实完成的是**执行状态**那一半 +(任意寄存器状态的恢复),而**地址空间**那一半在任何有 MMU 的环境上都是一次操作。 + +#### 面(七个操作) + +``` +kal_space_create(struct kal_space* out) 一个空的空间 +kal_space_clone(struct kal_space src, struct kal_space* out) 复制内存与句柄 +kal_space_grant_dir(struct kal_space, struct kal_dir, + const char* name, kal_uintptr len) ← kal_fs_preopen 的逆 +kal_space_grant_stream(struct kal_space, kal_uintptr stream, int slot) +kal_space_start(struct kal_space, void (*entry)(void*), void* arg, + void* stack_top, struct kal_task* out) 在其中启动一个上下文 +kal_space_destroy(struct kal_space) +kal_space_props +``` + +⇒ **7 个操作**(含两个 grant),与 `openkal.net` 的 6 个同量级。 +§1.3 那条「最大的一次扩张」仍然是算错了。 + +#### ⭐⭐ 关键:`start` 只收 (entry, arg, stack),**不收寄存器状态** + +这是让它能被一份**可移植 C ABI**表达的那一步。 + +「以任意寄存器状态启动」需要 openkal 认识 `arch::context` —— 而 clause 10 规定 +openkal 是 **C ABI**、openarch 是 **concepts**,二者不能互相引用。 +Fuchsia 撞的是同一堵墙:`zx_thread_start(thread, entry, stack, arg1, arg2)` 只有 +入口和栈,**所以 Fuchsia 上没有真 `fork`**。 + +⭐ **出路在端口层,不在接口**:`fork` 前 `setjmp`,子空间里从 `entry` 起来之后 +`longjmp` 回去。⇒ 子进程只需要从**一个已知函数**起步,而 `jmp_buf` 指向的那段栈 +在**被复制的空间里地址相同**,所以跳得回去。UML 一类的实现就是这么做的。 + +⇒ **接口保持原子且与架构无关,复合发生在端口层。** 这正是「原子能力 + port 组合」 +这条原则第一次真正吃重的地方。 + +#### 各后端的答案(clause 6.1:不提供 = 不导出符号)——**初步网络调研** + +| backend | 形状 | 可行性 | +|---|---|---| +| openkal-linux | `clone(CLONE_VM=0)` / `mmap` | ✅ | +| **openkal-windows** | ⭐ `RtlCloneUserProcess`(ntdll)→ `NtCreateUserProcess`;内核侧 `MmInitializeProcessAddressSpace` 带 **COW 标志**,子进程**地址空间完全相同且从同一位置继续** | ⚠️ **形状完全对上**,但**未文档化**;不复制全部句柄,CSRSS 连接需另行处理 | +| **openkal-macos** | `fork()`(BSD)可用;Mach 的 `task_create(parent, inherit_memory, &child)` **形状正是本接口** | ⚠️ `fork` 可用但 Apple 强烈不建议(框架在 fork 后不安全);`task_create` 在现代 XNU 上受 SIP/entitlement 限制 ⇒ **需实测** | +| openkal-opensbi / uefi | — | ❌ 不提供 —— **缺席即答案**,与 `openkal.fs` 在 opensbi 上一样 | + +⭐⭐ **两条结论:** + +1. **Windows 那条形状比预期准得多** —— `RtlCloneUserProcess` 正是「克隆地址空间 + + 从同一处继续」,连 COW 都在内核里做。⇒ 「Windows 没有 fork 原语」这句常识 + **在 NT 层面是错的**;错的是 Win32 API 面没有暴露它。 +2. ⚠️ 但两者都带**未文档化 / 受限**的性质。⭐ 而这**不阻塞定形** —— + clause 6.1 让「不提供」成为一个合法答案。**缺席即答案这条机制,正是让规范可以 + 先定形、实现按各平台可行性推进的原因。** + +⚠️ 以上为**初步网络调研,未实测**。`RtlCloneUserProcess` 的未文档化状态对一个必须 +长期可用的包是真实风险,落地前要评估。 + +来源:[huntandhackett/process-cloning](https://github.com/huntandhackett/process-cloning) · +[Abusing Windows' Implementation of Fork()](https://billdemirkapi.me/abusing-windows-implementation-of-fork-for-stealthy-memory-operations/) · +[GNU Mach: Task Creation](https://www.gnu.org/software/hurd/gnumach-doc/Task-Creation.html) + +#### 三处待定,现给出裁决与理由 + +**① 继承什么 —— ⭐⭐ 不需要开关,两个入口天然分开** + +初稿把它当成「一个开关」,又改成「显式传句柄」,**两次都不对**。正确的形状是: +**`create` 与 `clone` 是两个不同的入口,各自的名字已经说出了它给什么。** + +| 需求 | 入口 | 得到什么 | +|---|---|---| +| `fork`(要继承全部) | `kal_space_clone(src, &dst)` | ⭐ 复制**内存与句柄** —— 一个空间**持有**它的句柄,克隆自然带上 | +| 沙箱(要一无所有) | `kal_space_create(&s)` + 显式 `grant` | 只有明确交过去的 | + +⇒ **零开关、零歧义,而且每个入口的名字就是它的语义。** + +#### 这条缝规范里已经有另一半 —— clause 7.11 的形状 + +`process.h` 自己写下了这个缺口: + +> A stream handle of zero denotes that the program inherits … **which is what an +> environment without a general mechanism for passing handles can always provide.** + +而且它已经为此留了属性位:**`KAL_PROCESS_PROP_STREAM_PASSING`**。 +⇒ **openkal 早就把「跨 spawn 边界传句柄」建模成一个随实现变化的能力,只是把它限定在了流上。** + +⭐⭐ **更准确地说,缺的那一半是 `kal_fs_preopen` 的逆。** 把接收侧与供给侧并排看: + +| 接收侧(**今天已有**) | 供给侧 | +|---|---| +| `kal_env_arg` / `arg_count` | spawn 的 `argv` ✔ | +| `kal_env_var` / `var_at` | spawn 的 `envp` ✔ | +| 三个标准流 | `kal_spawn_streams` ✔ | +| **`kal_fs_preopen`** | ⭐ **缺失** | + +**恰好一行。** 而 SPEC clause 7.11 正是为这个形状写的: + +> An interface that **reports** a property … and offers **no way to set it** is +> incomplete … The absence was not visible in the specification text; it became +> visible when such programs were compiled above an implementation. + +⇒ **这不是一个新机制,是一个已有机制缺了逆向的那一半** —— 与 `kal_fs_set_modified` +在 0.5 被补上的理由逐字相同。 + +#### 形状:grant 属于 `openkal.space`,而不是 `openkal.process` + +⚠️ clause 8 规定「可以增加声明,不得改动已有的」,而 `kal_process_spawn` 有十个参数、 +`kal_spawn_streams` 的布局按 clause 5.3 不可变 ⇒ **不能往 spawn 上加参数。** + +⭐⭐ 而 `openkal.space` **本来就是两段式的**(`create` 然后 `start`),grant 天然落在中间: + +``` +kal_space_create(&space) +kal_space_grant_dir(space, dir, name, len) ← kal_fs_preopen 的逆 +kal_space_grant_stream(space, stream, slot) ← kal_spawn_streams 的逐个形式 +kal_space_start(space, entry, arg, stack_top, &task) +``` + +**子进程通过已有的 `kal_fs_preopen` 收到它们 —— 接收侧一行不改。** + +⇒ 这一处同时满足了五件事: + +| | | +|---|---| +| clause 8 | ✔ 不动任何已有声明,`kal_process_spawn` 原样保留 | +| **spawn 与 space 独立** | ✔ spawn =「环境按名字加载,普通继承」;space =「我自己搭子进程的世界」 | +| 一致性 | ✔ 与 `kal_fs_preopen` 逐字对称,名字都一样 | +| 能力式 | ✔ 无环境权威,交出去的都是明写的 | +| 不提供的实现 | ✔ 整个 `openkal.space` 缺席,clause 6.1 | + +#### ⭐ 「受限 fork」这个限制随之消失 + +初稿说「`kal_file` 传不过去 ⇒ 端口层只能给一个受限的 fork」。 +⭐ 用 `clone` 就没有这个问题:**一个空间持有它的句柄,克隆复制它们**, +已打开文件的偏移因此是共享的 —— 正是 POSIX `fork` 的语义。 +Linux 的 `fork` 复制 fd 表、能力系统克隆 CSpace,两边都自然。 + +⚠️ **但这要成为规范的一句话**:「空间持有它的句柄;`clone` 复制它们」。 +⭐ 而 Windows 那条调研正好给出了它必须是**属性**的理由 —— +`RtlCloneUserProcess` *"does not necessarily copy all … file handles"* +⇒ 加一位 **`KAL_SPACE_PROP_CLONE_HANDLES`**。 + +⚠️ **仍未决**:`grant_stream` 的 `slot` 与 `kal_spawn_streams` 的三个字段是否统一编号。 +**倾向统一**,否则同一件事有两套编号。 + +**② 与 `spawn` 的关系 —— 独立,而且理由很硬** + +「若是复合,加载器就得从环境搬到端口层」的意思是:今天 `kal_process_spawn` 收一个 +**路径**,由**环境**完成打开文件、解析 ELF/PE/Mach-O、映射段、重定位、 +构造 argv/env/auxv 栈、跳入口 —— 这一整套就是「加载器」。 +若把 spawn 定义成 `space` 原子之上的复合,程序就得自己做这一套。 + +⭐⭐ **决定性理由:可执行格式是环境的属性。** +openkal-windows 上是 PE,openkal-linux 上是 ELF,macos 上是 Mach-O。 +加载器若在端口层,端口层就得认识**所有格式** —— **这正是 openkal 存在要消除的 +那个 N×M**。 + +⇒ **两者独立。`spawn` 保持「环境完成加载」,`space` 只管地址空间。** +这句话要写进规范文本,否则将来两条造进程的路会各长一半。 + +**③ COW —— 判据是「用户可不可感知」,而它恰好可感知一次** + +性能不是语义。⭐ **真正可感知的只有一处:失败在什么时候发生。** + +| | clone 时 | 后续写入时 | +|---|---|---| +| 立即复制 | 可能失败(ENOMEM) | 不会 | +| COW + overcommit | 成功 | ⚠️ **可能失败**(Linux OOM) | + +⇒ 这是**语义差别**,不是性能差别,而且它正是 clause 3.1 所说的「模拟让调用者 +静默地错」那一类。 + +**裁决**:语义上要求 **「成功即完成」**;`kal_space_props` 出一位说明 +**本实现是否可能把失败推迟到后续写入**。⭐ 这正是 clause 6.2 属性字的用法 —— +程序**适应**一个属性,而不是调用它。 + +⚠️ **诚实**:Linux 默认 overcommit 下保证不了 ⇒ 那一位在 openkal-linux 上会是 +「可能推迟」。**这不是缺陷,这正是属性字该说的话。** + +#### 它解锁的不只是 `fork` + +- 沙箱:在一个新空间里跑不受信任的代码 +- ⭐ **一个以 openkal 为 ABI 的操作系统的用户态自己造进程** —— 今天 `spawn` 假定 + 环境里有加载器,而一个 OS 的 userland 需要自己加载 + +### 2.5 SPEC clause 11 增补(零代码) + +六条,每条注明是**边界**还是**未决**: + +| 条目 | 类别 | 理由 | +|---|---|---| +| 进程复制(`fork`) | **边界(操作)** | `kal_fork` 永不(clause 7.1)。⭐ 但 `openkal.space`(§2.4)让端口层把它组合出来 —— **这一行必须同时说出这一点**,否则一句「不会有 fork」会把地址空间那组能力一起埋掉 | +| 网络 | **未决** | 本版不定义;形状草案见 §2.3 | +| 就绪 / 多路复用 | **边界** | 可由 task + stream 导出 ⇒ 属于端口层 | +| 终端控制 | **未决** | 本版不定义;形状草案见 §2.1 | +| 权限 / 属主 | **边界** | 预设身份模型;Windows 只有只读位 ⇒ 不通用 | +| 符号链接的创建/读取 | **边界** | 文件系统**格式**的属性;`std::filesystem` 本就设计成允许失败 | + +⚠️ **「边界」不等于「永远」**:第一行同时是边界与候选,写清楚这一点是本节存在的 +理由 —— 一句「不会有 fork」会把地址空间那组能力一起埋掉。 + +### 2.6 规范包的配套 + +- `SURFACE.txt` 增新接口的名字(⚠️ 它是 clause 9 的**唯一**依据) +- `tools/check-surface.sh` —— ⚠️ [[link-error-is-the-mechanism-not-the-defect]] 记着: + 不带 `--complete` 时它只查**多出**的名字,所以一个非规范实现能通过。 + **新增接口必须同时让 `--complete` 那一侧变红**才算写完。 +- conformance 套件增对应用例 + +--- + +### 2.7 ⭐ `openkal-kit` —— 规范之外的组合库(同仓库,独立包) + +**动机**:判据二把一批东西判给了「端口层」—— 就绪、进程内流对、`fork` 助手、DNS。 +⚠️ 但「端口层」今天只有一个,而且它组合出的是 **POSIX**。 +⇒ **一个原生 openkal 应用想同时等两个流,今天没有答案** —— +要么自己写一遍 thread-per-source,要么把整个 musl 拉进来。 + +#### 定位 + +| | | +|---|---| +| 位置 | ⭐ **openkal 仓库内,与 `include/`、`src/` 同级的 `kit/`** —— 发现性最好 | +| 形态 | **独立的 `mcpp.toml`**,包名 `mcpplibs/openkal-kit`,模块 `openkal.kit.*` | +| 规范地位 | ⭐ **不进 SPEC.md,非规范** —— 它的价值恰恰在于**可以演进**,而 clause 8 让规范不能 | +| 测试 | ✔ 有自己的测试集,进 openkal 仓库的 CI | +| 启用 | `openkal` 包的一个 feature(`[feature-deps.kit]`),不用的不链接 | + +#### ⭐⭐ 它结构上不可能被误认成规范的一部分 + +clause 10 规定 **openkal 的契约形式是 C ABI**。 +⇒ **kit 刻意不是 C ABI** —— 它是 C++ 模块 + `namespace kal::kit`,不导出任何 +`kal_` 开头的 C 符号。 + +这一条同时解决了 `SURFACE.txt` 的约束:*"exports no other name beginning with +`kal_`"* —— C++ 修饰名是 `_ZN3kal3kit…`,**不以 `kal_` 开头** ⇒ +`tools/check-surface.sh --complete` 不会把它读成一个不合规的实现。 + +⭐ **契约形式本身成了「是不是规范」的判据**,而不是靠一句声明。 + +#### 内容(全部由原子组合,零新接口) + +| 模块 | 由什么组合出来 | +|---|---| +| `openkal.kit.wait` | 等 N 个流之一就绪 = `kal_task_wait/wake` + 阻塞读 + 预读缓冲 | +| `openkal.kit.pipe` | 进程内流对 = 环形缓冲 + `wait/wake` | +| `openkal.kit.spawn` | `fork` 助手 = `openkal.space` 的 `clone` + `setjmp/longjmp` | +| `openkal.kit.name` | 端点名字解析(DNS)= `openkal.net` 之上;clause 3.4 明确把它推到接口之外 | + +#### 消费者两类 + +**原生 openkal 应用** · **openkal-musl**(§3.7 的就绪实现直接用 `openkal.kit.wait`) + +⭐ **它顺带回答了一个此前没答的问题**:判据二说「可导出的归端口层」, +但没说**归哪个**端口层。答案是:**归一个所有端口层共用的库** —— +否则「同一个组合写第二遍」会在第二个端口层出现时发生。 + +⚠️ **未决:它与 openkal-musl 的边界。** musl 的 fd 表在 musl 里,而就绪的预读缓冲要挂 +在 fd 上 ⇒ **kit 给的是「等一组流」,fd 那一层的适配仍在 musl。** 这条边界要在写之前画清。 + +### 2.8 ⭐⭐ 深度 review:七处问题,三处是设计错误 + +判据五条:**对上**(原子 · 可组合 · 最小 · 一致)、**对下**(backend 容易实现)。 +逐个接口压测的结果如下。⚠️ 前三条推翻了 §2.1–§2.4 的部分内容。 + +#### ⓵ ⭐⭐⭐ `kal_space_create` 要删掉 —— 两条独立的理由 + +**对下:Linux 上几乎无法实现。** `fork` 给的是**克隆**;要造一个**空**的用户地址空间, +只能 spawn 一个 stub 程序再往里写 —— 那需要一个 stub 二进制,是不能接受的形状。 + +**对上:它逻辑上也没有意义。** `kal_space_start(space, entry, …)` 的 `entry` 是一个 +**本空间的**函数指针。在一个**空**空间里,那段代码根本不在里面。 +⭐ **两条独立的理由指向同一个裁决 —— 这通常说明裁决是对的。** + +⇒ **`openkal.space` 只剩 `clone` / `start` / `destroy` / `props`。** +而沙箱(要一无所有)由 **spawn + grant** 满足,不由 space 满足。 + +⚠️ **连带结论:`space` 实际上只服务 `fork` 一个用例。** 它值不值四个操作, +应当在定形时重新问一次 —— 本文倾向值得,因为 `fork` 之外还有「OS 用户态自己造进程」。 + +#### ⓶ ⭐⭐⭐ 三个接口都产生**拥有的流**,而 core 的流是**借来的** + +`openkal.stream` 的模型是借用(clause 6.7 / clause 11:*"Every handle in the core +interfaces is borrowed"*),所以它**没有 close**。而: + +| 接口 | 产生 | 需要 close 吗 | +|---|---|---| +| `openkal.fs` | `kal_file` | ✔ 已有 `kal_fs_close_file` | +| `kal_process_channel` | 两个流 | ⚠️ **缺** | +| `kal_net_connect` / `accept` | 流 | ⚠️ **缺** | + +⚠️⚠️ **channel 缺 close 不是小事**:父进程 spawn 之后不关掉 `child_end`, +读端**永远等不到 EOF** —— 这是管道的经典陷阱,而接口不提供关闭就无法避免。 + +⭐ **同一个问题第三次出现,说明该收口的是规则而不是补丁。** +clause 11 自己写着:*"Owned handles arrive with `openkal.fs`, and the rules for +their release … are deferred to that interface."* —— **那笔延期到期了。** + +**裁决**:沿用 `openkal.fs` 已确立的形状 ——**谁给的谁收**,每个产生拥有句柄的接口 +自带它的 close(`kal_process_channel_close` / `kal_net_close`)。 +❌ **不**给 core 的 `kal_stream` 加通用 close:那会改变 core 接口的语义, +而 clause 3.2 的理由(core 一旦扩张就要求每个实现)同样适用于给 core 加操作。 + +⇒ channel 变 2 个操作,net 变 7 个。 + +#### ⓷ ⭐⭐ `terminal` 的两个 set 无法恢复原状 + +`set_raw(s, 0)` 恢复到**什么**?一个 TUI 退出时必须把终端还原成**它进来时的样子**, +否则用户的 shell 坏掉。我给的两个 setter **没有保存原状的地方**。 + +⇒ 改成 clause 7.11 要求的**成对**形状,而且顺带从 2 个操作变回 2 个: + +``` +kal_terminal_get_mode(stream, kal_uintptr* mode) +kal_terminal_set_mode(stream, kal_uintptr mode) +kal_terminal_size(stream, &cols, &rows) +kal_terminal_props +/* 位: KAL_TERM_LINE_EDIT KAL_TERM_ECHO */ +``` + +⭐ 与 `kal_fs_file_info` / `kal_fs_set_modified` **同构**,而且 save/restore 是 +`get` 一次、退出时 `set` 回去 —— 一个 TUI 本来就这么写。 + +⚠️ **裸机上仍然成立**:UART 没有行编辑 ⇒ `get_mode` 报「都关着」,`set_mode` 是 +无事可做即完成。 + +#### ⓸ ⭐⭐ 「诞生时收到的东西」缺一个名字 —— `kal_grant` + +`argv` / `envp` / 三个流 / preopen 是**同一类东西**(openkal.env 的定义就是 +*"the parameters a program receives at inception"*),而它们今天在**四个地方各自表达**。 + +把它具体化成一个对象,§2.4 的 grant 问题与沙箱问题一起解决: + +``` +kal_grant_create(&g) +kal_grant_add_arg / add_var / add_stream / add_dir +kal_grant_spawn(g, base, path, path_len, &proc) 按名字启动一个程序 +kal_grant_start(g, space, entry, arg, stack, &task) 在一个克隆的空间里启动 +kal_grant_close(g) +``` + +⭐⭐ **它同时做到三件事**: +1. `kal_process_spawn` 的**十个参数塌成三个**(base/path/grant) +2. **grant 同时服务 spawn 与 space** —— 沙箱走 `grant_spawn`,fork 走 `clone`+`grant_start` +3. 与 `kal_fs_preopen` / `kal_env_*` 的接收侧**逐字对称**(clause 7.11 的逆) + +⚠️ **代价**:`kal_process_spawn` 保留不动(clause 8),于是**有两条 spawn 路**。 +⭐ 这次是**有意且一次性**的,规范可以明说哪条是正道 —— 但**这是一个大决定, +必须单独确认**,它触及 `openkal.process` 的地位。 + +⚠️ **对下检查**:实现要能「暂存一组句柄再一次性交付」—— +Linux 用 `posix_spawn_file_actions` 或 fork+dup2 ✔;Windows 用 +`PROC_THREAD_ATTRIBUTE_HANDLE_LIST` ✔;裸机不提供 process ⇒ 不适用 ✔。 + +#### ⓹ ⭐⭐ net 的数据报问题,实质是「DNS 能不能用」 + +`openkal.kit.name`(DNS)需要 UDP。若 v1 只做流,DNS 只能走 TCP —— +而不少 resolver 不接受 TCP-only。⇒ **「要不要数据报」实际上等于「net 落地之后 +有没有人能用域名」。** + +⚠️ 加上数据报,net 从 7 个操作变成 ~11 个 —— **这就不小了**。 +⇒ ⭐ **这是 net 最大的设计张力,必须专门决策,不能顺带。** 本文不预设答案。 + +#### ⓺ ⭐ 窗口尺寸变化只能轮询 —— 可接受,但必须写下来 + +openkal 没有信号 ⇒ 没有 SIGWINCH。TUI 只能在事件循环里轮询 `kal_terminal_size`。 +⭐ 在一个本来就有事件循环的程序里这是自然的,**但 TUI 作者会先去找通知机制** +⇒ README 必须直说。 + +#### ⓻ ⭐ listener 不是流,而 `kit.wait` 只等流 —— **组合路径成立** + +一个服务器要同时等 listener 与若干连接。listener 不是 `kal_stream`, +`kit.wait` 表面上等不了它。 +⇒ 但 kit 可以为 listener 起一个辅助 task 做阻塞 `accept` ✔。 + +⭐⭐ **这条值得单记:它是「组合发生在库里」这条路第一次被真实用例压测,而它通过了。** +接口不必为组合的方便而变形。 + +#### 修正后的规模 + +| 接口 | 操作数 | 变化 | +|---|---|---| +| `openkal.terminal` | 4 | 形状改(成对),数量不变 | +| `openkal.process` 加法 | **2**(channel + close) | +1 | +| `openkal.space` | **4** | −3(删 `create` 与两个 grant) | +| `openkal.grant` | **7** | ⭐ 新增,但吸收了 space 的 grant 与 spawn 的参数表 | +| `openkal.net` | **7 或 ~11** | +1(close);数据报未决 | + +--- + +### 2.9 ⭐⭐ 完整性验收:面够不够用 + +**方法**:每条需求走一遍「需求 → 组合路径 → 缺哪个原子」。不看接口漂不漂亮, +只看**组合得出来还是组合不出来**。 + +#### 结论 + +> **面基本够用。全部需求里只有一个组合不出来,而它是一个真实的原子缺口:** +> **⭐⭐⭐ 一个阻塞中的读,如何结束。** + +#### 缺口的证据链(实测两处头文件) + +| 实测 | 结果 | +|---|---| +| `task.h:51` `kal_task_wait(word, expected, **timeout_ns**)` + 属性位 `KAL_TASK_PROP_WAIT_TIMEOUT` | ✅ **主循环可以带超时醒来** | +| `stream.h:26` `kal_stream_read(s, buf, len)` | ⚠️ **没有截止时间,阻塞不可打断** | + +⇒ `openkal.kit.wait` 的实现(每源一个辅助 task 做阻塞读)**对短命程序成立** +(TUI 退出时 `kal_exit` 收走一切),**对长跑程序漏 task** —— +每关一个连接漏一个,服务器跑一天漏一万个。 + +⚠️ 同一个缺口还挡住另外两处:`select`/`poll` 的 **timeout 参数**, +以及「**等多个子进程之一退出**」(`kal_process_wait` 也是不可打断的阻塞)。 + +#### 按三问判它 + +| 问 | 答 | +|---|---| +| 可导出? | ❌ **不可** —— 一个已经阻塞在环境里的调用,只有环境能让它返回 | +| 几个? | 一个 ⇒ openkal | +| OS 的形状? | ❌ POSIX 用信号/`pthread_cancel`、Windows 用 `CancelIoEx`、Fuchsia 用 handle close —— **三种形状**,而「等待要有尽头」是它们共同的那个概念 | + +⇒ **它通过,而且是本轮唯一一个通过的新原子。** + +#### 两个候选形状 —— ⚠️ 必须裁决,本文不预设 + +**(a) 带截止时间的等待** + +``` +kal_stream_read_until(stream, buf, len, deadline_ns) → kal_io_result +kal_net_accept_until (listener, deadline_ns, &stream) +``` + +| | | +|---|---| +| ✅ 一次解决三件事 | 超时读 · 辅助 task 可退出 · `select` 的 timeout | +| ✅ 一致性 | 与 `kal_task_wait` 的 `timeout_ns` **形状逐字相同**(ns,零=无超时) | +| ✅ 对下容易 | Linux `poll`+`read` · Windows overlapped/`WaitForSingleObject` · 裸机 UART 轮询+timer —— **每个环境都给得出** | +| ⚠️ 代价 | 不是「立刻取消」,辅助 task 最多再等一个 deadline | +| ⚠️ 位置 | `openkal.stream` 在 **core** 里,而 clause 3.2 的理由(core 一旦扩张就要求**每个**实现)同样适用于加操作 ⇒ ⭐ 应当**自成一个可选接口**(clause 6.2:「实现可能没有的 operation 自成接口」),资源是流/listener | + +**(b) 关闭唤醒**:在 ⓶ 新增的 `kal_net_close` / `kal_process_channel_close` 的**语义**里 +规定「关闭使阻塞在该流上的读返回」。 + +| | | +|---|---| +| ✅ 不加任何新操作 | 只写一句语义 | +| ⚠️ 只覆盖**拥有的**流 | 借来的 stdin 仍然打不断(可接受:`kal_exit` 收尾) | +| ⚠️ 解决不了 timeout | `select(timeout)` 仍然没有答案 | +| ⚠️⚠️ **对下更难** | Linux 上 `close(fd)` **不保证**唤醒 blocked reader ⇒ 实现得自己配一个内部 eventfd + poll。**接口简单了,实现复杂了** | + +⭐ **本文倾向 (a)**:它对下更容易(实现本来就要 poll),而且多解决一件事(timeout)。 +但这是一个需要拍板的决定。 + +#### 其余需求:全部可组合 ✔ + +| 需求 | 组合路径 | | +|---|---|---| +| `fork` | `space.clone` + `setjmp/longjmp` + `grant` | ✔ | +| 子进程 + 捕获输出 | `channel` + `grant_spawn` | ✔ | +| 进程内 pipe | kit:环形缓冲 + `task.wait/wake` | ✔ | +| socket 服务器 | `net.listen/accept` + `kit.wait` | ✔(需缺口) | +| 终端 UI | `terminal.get/set_mode` + `size` 轮询 | ✔ | +| `std::thread` / `atomic::wait` | `task.start` / `task.wait/wake` | ✔ **正好对上** | +| `std::chrono` / `sleep_for` | `time` | ✔ | +| `std::random_device` | `random` | ✔ | +| `` | `fs`(部分操作允许失败,标准如此设计) | ✔ | +| 时区数据库 | 读 tzdata 文件 ⇒ `fs` | ✔ | +| `dlopen` / 动态链接 | 读文件 `fs` + `openkal.exec` 的可执行内存 + 重定位(库) | ✔ **exec 够用** | +| 等多个子进程之一 | kit + 辅助 task | ✔(需缺口) | +| cwd / `chdir` | 端口层在 preopen 之上模拟(`okm_getcwd.c` 已有) | ✔ | + +#### 两条已识别、本轮不必,但其中一条会影响定形 + +| | 三问 | 处置 | +|---|---|---| +| **共享内存**(两个空间共用一段) | 通过(不可导出 · 一个 · 通用:shm/section/VMO) | ⚠️⚠️ **本轮不做,但 `space` 定形时必须留一句话** —— 将来它一出现,`kal_space_clone` 就要回答「共享的那段怎么办」 | +| **文件锁**(多进程协调) | 通过,但裸机没有 | 记入 clause 11,今天无消费者 | + +⚠️ **内存映射文件不是缺口,是性能**:`mmap(fd)` 的用途(动态链接、大文件) +都能用 `read` + `openkal.exec` 组合出来,只是慢。 + +--- + +### 2.10 ⭐⭐ 用四问回扫:数据报的结论是反的,而 grant 是我过度设计 + +**判据(四问,顺序即优先级)**: + +1. **是不是最小的原子能力?** +2. **是不是所有内核都有?** +3. **是不是只能由 openkal 给,而不能由已有原子组合出来?** +4. **是不是通用(不与某个 OS 的形状绑定)?** + +> 能由原子组合出来的,一律去 `openkal-kit`。只有**必须**的才入规范。 +> 且**对上对下双向**检查,并与 openkal 已有的语义/风格一致。 + +#### ⓵ 数据报:⭐⭐⭐ 我的结论是反的 + +初稿说「加上数据报 net 从 7 变 11,就不小了」。**用四问过一遍,这个反对理由不成立。** + +| 问 | 答 | +|---|---| +| 最小原子? | ✔ `open` / `send_to` / `recv_from` / `close` / `props` —— **5 个** | +| 所有内核都有? | ✔ 而且 ⭐ **比流更普遍** —— 一个 IP 栈实现 UDP 只要几百行,TCP 要几千行 | +| 只能 openkal 给? | ✔ **消息边界、无连接、可能乱序** —— 这些**不可能**由字节流组合出来 | +| 通用? | ✔ | + +⭐⭐ **而「变成 11 个操作」这个担心,是因为我把它们当成了一个接口。** +按 clause 6.4(我自己在 §2.3 就引用过它):**流与数据报是两种资源** —— +「定位适用于文件而不适用于连接;半关闭适用于连接而不适用于文件」是同一条推理。 + +⇒ **它们应当是两个接口:** + +``` +openkal.net 连接（流） connect / listen / accept / shutdown / close / props 6 +openkal.datagram 数据报 open / send_to / recv_from / close / props 5 +``` + +⭐ **拆开之后,对上对下都更好:** + +| | | +|---|---| +| **对下** | 一个裸机 BSP 可以**只提供 datagram**(UDP 好写得多)⇒ clause 6.1 的缺席机制正好用上 | +| **对上** | ⭐ **DNS 不再被「TCP-only」卡住** —— `kit.name` 只依赖 `openkal.datagram` | +| **风格** | 与 `fs` / `process` / `task` 一样**按资源分**,不是按主题分 | + +⇒ **§2.8 ⓹ 那条「必须专门决策」的张力消失了。答案是拆,不是选。** + +⚠️ **DNS 的 resolver 地址从哪来?** POSIX 读 `/etc/resolv.conf`(ambient 配置)。 +⭐ openkal 上应当**由调用方显式传入** —— 这与能力式模型一致(程序不该有 ambient 的 +resolver),而且它可组合(从 `kal_env_var` 或一个 preopen 的文件读,由**程序**决定)。 +⇒ **不需要新原子。** + +#### ⓶ ⭐⭐ 同一把尺子回扫,`kal_grant` 是我过度设计 + +§2.8 ⓸ 引入 `kal_grant` 对象(7 个操作),理由两条: +**(1)** 把 preopen 交给一个新空间 · **(2)** 让 `kal_process_spawn` 的十个参数塌成三个。 + +用四问过: + +| 部分 | 只能 openkal 给? | 判定 | +|---|---|---| +| `add_dir`(交出一个目录句柄) | ✔ 不可组合 | **必须** | +| `add_stream` | ✔ | **必须**(spawn 已有,形式不同) | +| `add_arg` / `add_var` | ❌ **argv/env 只是字节,spawn 已经收它们** | **便利,不是原子** | +| 「塌掉参数表」 | ❌ 纯便利 | **不是理由** | + +⇒ ⭐ **grant 对象把「没办法传的」和「已经有办法传的」放进了同一个对象。** +按「只有必须的才入规范」,它的正当理由只剩 `add_dir` 一条。 + +⚠️ 而**与 openkal 风格一致**这条也指向同一边:`kal_process_spawn` 本来就是十个参数 +的长参数表,**再加一个参数与既有风格一致**;引入一个 grant 对象是**一个新概念**。 + +⇒ **退回最小形状 —— 一个新声明,一个新参数:** + +```c +struct kal_preopen { struct kal_dir dir; const char* name; kal_uintptr len; }; + +int kal_process_spawn_with(struct kal_dir base, const char* path, kal_uintptr path_len, + const char** argv, const kal_uintptr* argv_lens, kal_uintptr argc, + const char** envp, const kal_uintptr* envp_lens, kal_uintptr envc, + const struct kal_spawn_streams* streams, + const struct kal_preopen* grants, kal_uintptr grant_count, + struct kal_process* out); +``` + +⚠️ 重复十个参数**是丑的**,但它是**最小的**:一个声明、零新对象、零新概念, +且 `kal_process_spawn` 原样保留(clause 8)。 +⭐ 这是「最小 + 一致」压过「好看 + 便利」的一次,而那正是这四问的排序。 + +#### ⓷ ⭐⭐⭐ 连带:`openkal.space` 的两个 grant 也不需要了 + +§2.8 ⓵ 删掉 `kal_space_create` 之后,`space` 只剩 `clone` —— 而 +**`clone` 已经复制了全部句柄**(§2.4)。于是: + +| 用例 | 走哪条 | 还需要 space 的 grant 吗 | +|---|---|---| +| `fork`(继承全部) | `kal_space_clone` | ❌ clone 全带 | +| 沙箱(只给指定的) | `kal_process_spawn_with` | ❌ 不走 space | + +⇒ ⭐ **`kal_space_grant_dir` / `kal_space_grant_stream` 整个删掉。** + +#### 修正后的规模(与 §2.8 对照) + +| 接口 | §2.8 | **§2.10** | 变化 | +|---|---|---|---| +| `openkal.terminal` | 4 | 4 | — | +| `openkal.process` 加法 | 2 | **3** | +1(`spawn_with`) | +| `openkal.space` | 4 | **4** | 形状变(去 grant,回到 clone/start/destroy/props) | +| `openkal.grant` | 7 | **0** | ⭐ **删除** —— 便利不是原子 | +| `openkal.net`(连接) | 7 或 ~11 | **6** | ⭐ 张力消失 | +| `openkal.datagram` | — | **5** | ⭐ 新增,独立接口 | +| 截止时间(§2.9) | — | **2** | 待裁决 | + +⇒ **总计 24 个操作,分在六个可选接口里**,而 §2.8 的版本是 24 个分在五个里 —— +**数量持平,而每个接口都更小、更可独立提供、且没有一个是「便利」。** + +⭐⭐ **这一节的价值不在数字,在于它证明了那四问是可执行的**:同一把尺子, +一次推翻了我的一个反对理由(数据报),一次扫掉了我的一处过度设计(grant)。 +**判据要能否定作者自己,才算判据。** + +--- + +### 2.11 ⭐⭐ 第二次深度 review:采纳 (a),但理由和代价都要重写 + +**(a) 带截止时间的等待,已采纳。** 但本轮 review 找到一个**更好的候选**, +算清之后它仍然输 —— 输在哪里,比结论更值得记。 + +#### ⓵ ⭐⭐⭐ 考虑过而未采纳:就绪通知(`notify`) + +与其「让阻塞的读能超时退出」,不如**让它根本不必阻塞**: + +``` +kal_stream_notify(stream, kal_u32* word) 该流可读时，递增 *word 并 wake 它 +``` + +主循环于是 `kal_task_wait(word, seen, timeout)` —— ⭐ **一个 task 等 N 个源, +辅助 task 整个不需要。** + +| | `read_until`(a) | `notify` | +|---|---|---| +| 操作数 | ⚠️ 每个可无限阻塞的操作 +1(见 ⓶) | ⭐ **1 个覆盖全部可等待的东西** | +| 辅助 task | 仍需 N 个(只是能退出了) | ⭐ **不需要** | +| ⭐ 预读缓冲(§3.7) | ⚠️ **仍然需要** | ⭐ **整个消失**(通知不消耗数据) | +| 与既有原子咬合 | 平行的第二套超时 | ⭐⭐ **正好接上 `kal_task_wait/wake`** | +| §2.8 ⓻ 的 listener | 靠 kit 变通 | ⭐ 接口层面解决 | +| 对下:裸机 | 轮询 + timer | ⭐ **中断 → wake,更自然** | +| **对下:Linux** | ⭐ `poll(fd, ms)` + `read`,**十行、无状态、无线程** | ⚠️⚠️ **要一个进程级 epoll + 一个后台线程** | + +⇒ ⭐⭐ **它在五项上更好,输在最后一项 —— 而最后一项是 clause 7.1。** + +> **clause 7.1(Naturalness)**:实现不应被迫构建一个兼容层。 + +一个**隐藏的后台轮询线程**,正是「重建一套事件机制」。而 `read_until` 在 Linux 上 +就是驱动本来就要做的那件事(`poll` 然后 `read`)。 + +⇒ **按 clause 7.1 采纳 (a)。** 并按规范自己的体例(**clause 6.3 +「Mechanisms considered and not adopted」**),把 `notify` 连同这张表写进规范 —— +⭐ **否则下一个人会重新想到它,并重新算一遍。** + +#### ⓶ ⚠️ (a) 的代价比上一版说的大 + +「哪些操作会无限阻塞」穷举一遍,每个都要一个带截止时间的版本: + +| 会无限阻塞 | 需要 | +|---|---| +| `kal_stream_read` | ✔ | +| `kal_stream_write` | ⚠️⚠️ **我上一版漏了** —— 管道满 / TCP 窗口满时会阻塞 | +| `kal_net_accept` | ✔ | +| `kal_datagram_recv_from` | ✔ | +| `kal_process_wait` | ✔ | +| `kal_task_join` | ❌ 可由 `wait/wake` 协调,不需要 | + +⇒ **`openkal.deadline` 是 5 个操作 + props = 6**,与 `openkal.net` 同样大。 +**这是 (a) 的结构性代价:每新增一个可无限阻塞的操作,这个接口就 +1。** + +⚠️ 而「同一件事两个版本」在规范里**有先例**:clause 7.8 让 +`kal_fs_open_file`(两个 flag)与 `kal_fs_open`(整个意图一个字)并存,并给了理由。 +⇒ 不是新问题,但**这个先例必须被引用**,否则它看起来像随手加的。 + +#### ⓷ ⭐⭐ `kal_stream_write` 会阻塞 ⇒ `popen` 的经典死锁,§3.3 漏了 + +kit 的 wait 只处理了**读侧**。而一个子进程往满的管道写、父进程又在等它退出 ⇒ **死锁**。 +这是 `popen` 教科书级的陷阱,而 §3.3 的两阶段实现没有提。 + +⇒ **§3.3 必须补一条**:临时文件阶段天然没有这个问题(文件不会满); +换到 `kal_process_channel` 之后**就有了** ⇒ 那一步必须同时用上 +`kal_deadline_write`。⭐ **一个「后来才出现的死锁」比一开始就有的更难查。** + +#### ⓸ ⭐⭐ `kal_space_start` 应当返回 `kal_process`,不是 `kal_task` + +一个跑在**另一个地址空间**里的执行上下文,`kal_task_join` 能 join 它吗? +跨空间的 task 句柄语义没人定过。 + +⭐ 而它本来就有名字:**一个在自己地址空间里执行的东西,就是一个进程。** + +⇒ `kal_space_start(space, entry, arg, stack_top, struct kal_process* out)`, +之后 `kal_process_wait` / `terminate` / `close` **全部复用**。 +**一致性 +1,新概念 −1,操作数不变。** + +#### ⓹ 四处小项(写下来免得落掉) + +| | 裁决 | +|---|---| +| `struct kal_endpoint` 定义在哪 | `types.h`(或共享头)—— net 与 datagram 共用,且**一个缺席时另一个仍要能用** | +| `kal_net_shutdown` 的方向 | 三个常量:read / write / both | +| `kal_terminal_size` 在非终端流上 | `kal_err_not_supported` —— 与今天 `ioctl` 的 `ENOTTY` 对应 | +| 跨接口引用类型 | ✔ **已有先例**:`kal_fs_stream(kal_file) → kal_uintptr` 已经引用了 stream。**声明在规范包、符号在实现包**,类型可见而符号缺席,正是 openkal 的模型 | + +#### 定稿规模 + +| 接口 | 操作数 | | +|---|---|---| +| `openkal.terminal` | 4 | `get_mode` / `set_mode` / `size` / `props` | +| `openkal.process` 加法 | 3 | `channel` / `channel_close` / `spawn_with` | +| `openkal.space` | 4 | `clone` / `start`(→`kal_process`) / `destroy` / `props` | +| `openkal.net` | 6 | `connect` / `listen` / `accept` / `shutdown` / `close` / `props` | +| `openkal.datagram` | 5 | `open` / `send_to` / `recv_from` / `close` / `props` | +| **`openkal.deadline`** | **6** | `read` / `write` / `accept` / `recv_from` / `wait_process` / `props` | + +⇒ **28 个操作,六个可选接口,没有一个是「便利」。** +⚠️ 比 §2.10 多 4 个,全部来自 ⓶ 的穷举 —— **穷举比估计贵,但估计会漏。** + +--- + +### 2.12 头文件草案(六个接口) + +⚠️ **草案,不是定稿。** 每份都按现有头文件的约定写:句柄是**单字 struct 传值**、 +字符串**永远带长度**、传输返回 `struct kal_io_result`、释放返回 `void`、 +只 include 兄弟头。 + +#### `openkal/timeout.h` —— ⚠️ 名字从 `deadline` 改了,理由见 §2.13 ⓷ + +```c +/* openkal.timeout --- a bound upon operations that would otherwise wait + * without end. + * + * Every operation here is the operation of the same name in another interface, + * with one argument added. Clause 7.8 already establishes that a second form of + * one operation is admissible when the first cannot state the whole of an + * intent: `kal_fs_open_file' and `kal_fs_open' stand beside each other for that + * reason, and these stand beside their originals for the same one. + * + * ⚠️ THE ARGUMENT IS A DURATION, NOT AN INSTANT, and the name of this interface + * was changed to say so. `kal_task_wait' already takes `timeout_ns' and already + * defines zero as no timeout; a second spelling of the same idea would be the + * one thing this specification most consistently refuses. + * + * An expired bound is reported as `kal_err_again' --- "the operation would + * block" --- which is what an expiry is. The error set is closed (clause 5.2) + * and required no addition. */ +#ifndef OPENKAL_TIMEOUT_H +#define OPENKAL_TIMEOUT_H +#include "types.h" +#include "stream.h" +#include "net.h" +#include "datagram.h" +#include "process.h" + +#ifdef __cplusplus +extern "C" { +#endif + +struct kal_io_result kal_timeout_read (struct kal_stream, void*, kal_uintptr len, kal_u64 timeout_ns); +struct kal_io_result kal_timeout_write(struct kal_stream, const void*, kal_uintptr len, kal_u64 timeout_ns); + +int kal_timeout_accept(struct kal_net_listener, kal_u64 timeout_ns, struct kal_stream* out); + +struct kal_io_result kal_timeout_recv_from(struct kal_datagram, void*, kal_uintptr len, + struct kal_endpoint* from, kal_u64 timeout_ns); + +int kal_timeout_wait_process(struct kal_process, kal_u64 timeout_ns, + int* status, int* terminated); + +/* The smallest bound this implementation distinguishes, in nanoseconds. A + * board whose only clock ticks at a millisecond reports 1000000; a caller that + * asks for less is not refused and does not get less. Per implementation, so a + * word rather than an enquiry (clause 6.2). */ +extern const kal_uintptr kal_timeout_granularity_ns; + +#ifdef __cplusplus +} +#endif +#endif +``` + +#### `openkal/terminal.h` + +```c +/* openkal.terminal --- what an interactive stream does with what is typed at + * it. The resource is a stream for which `kal_stream_props' reports + * KAL_STREAM_PROP_INTERACTIVE; every operation here reports + * kal_err_not_supported for any other. + * + * A separate interface rather than operations upon `openkal.stream', for the + * reason clause 6.4 gives when it places positioning in `openkal.fs': the + * behaviour varies between the RESOURCES of the stream interface, and an + * implementation could neither claim these honestly nor withhold them usefully. + * + * ⚠️ THE PAIR IS get/set AND NOT two setters. A program that turns line editing + * off must be able to put back what was there, and a setter alone gives it + * nothing to put back --- it would restore a default, and the terminal a user + * returns to is then not the one they had. Clause 7.11 states the general rule; + * this is an instance of it. */ +#ifndef OPENKAL_TERMINAL_H +#define OPENKAL_TERMINAL_H +#include "types.h" +#include "stream.h" + +/* Positions in the mode word. A position that has not been assigned reads as + * zero, so a program compiled against a later specification behaves correctly + * against an earlier implementation (clause 6.2). */ +#define KAL_TERM_LINE_EDIT ((kal_uintptr)1u << 0) /* the environment assembles lines */ +#define KAL_TERM_ECHO ((kal_uintptr)1u << 1) /* the environment shows what is typed */ + +#ifdef __cplusplus +extern "C" { +#endif + +int kal_terminal_get_mode(struct kal_stream, kal_uintptr* mode); +int kal_terminal_set_mode(struct kal_stream, kal_uintptr mode); + +/* The size of the display, in character cells. An environment that does not + * know --- a serial line has no way to ask --- reports kal_err_not_supported + * and leaves both outputs untouched. + * + * ⚠️ THERE IS NO NOTIFICATION. openkal has no signals, so a program learns of a + * change by asking again. A program with an event loop already has somewhere to + * ask from; one without does not need to know. */ +int kal_terminal_size(struct kal_stream, kal_uintptr* cols, kal_uintptr* rows); + +kal_uintptr kal_terminal_props(struct kal_stream); /* enquiry: varies per resource */ + +#ifdef __cplusplus +} +#endif +#endif +``` + +#### `openkal/net.h` —— 连接 + +> ⚠️ **本草案已被 §7.5 ⓶ 取代**:交出的是被拥有的 `kal_net_conn`,流经 `kal_net_stream` 从它借。下面保留的是当时提出的形状。 + + +```c +/* openkal.net --- a connection, which is a stream with a peer and a way to be + * half-closed. Clause 3.4 records why this is not merged with `openkal.fs': + * positioning applies to a file and not to a connection, half-closure to a + * connection and not to a file. + * + * ⚠️ NAME RESOLUTION IS NOT HERE. Clause 3.4 excludes it in terms: an + * implementation shall not be required to parse an unbounded set of name + * schemes. An endpoint is an address and a port; turning "example.com" into one + * is a library above `openkal.datagram'. */ +#ifndef OPENKAL_NET_H +#define OPENKAL_NET_H +#include "types.h" +#include "stream.h" + +struct kal_net_listener { kal_uintptr h; }; + +/* Directions for kal_net_shutdown. */ +#define KAL_SHUT_READ 1 +#define KAL_SHUT_WRITE 2 +#define KAL_SHUT_BOTH 3 + +/* Positions in kal_net_props. */ +#define KAL_NET_PROP_IPV6 ((kal_uintptr)1u << 0) +#define KAL_NET_PROP_HALFCLOSE ((kal_uintptr)1u << 1) + +#ifdef __cplusplus +extern "C" { +#endif + +int kal_net_connect(const struct kal_endpoint*, struct kal_stream* out); +int kal_net_listen (const struct kal_endpoint*, struct kal_net_listener* out); +int kal_net_accept (struct kal_net_listener, struct kal_stream* out); + +/* Ends transfer in one direction while the other continues. This is the + * operation that distinguishes a connection from a file, and it is why the two + * are separate interfaces. */ +int kal_net_shutdown(struct kal_stream, int direction); + +/* ⚠️ AN OWNED STREAM, UNLIKE THE THREE `openkal.stream' provides. A connection + * is obtained and must be released; the standard streams are borrowed and are + * not. Same division `openkal.fs' already draws with kal_fs_close_file. */ +void kal_net_close (struct kal_stream); +void kal_net_close_listener(struct kal_net_listener); + +extern const kal_uintptr kal_net_props; + +#ifdef __cplusplus +} +#endif +#endif +``` + +#### `openkal/datagram.h` + +```c +/* openkal.datagram --- messages with boundaries, sent without a connection. + * + * A SEPARATE INTERFACE FROM `openkal.net', for the reason clause 6.4 gives: a + * datagram and a connection are two resources, and an operation that some + * resources of an interface can never satisfy does not belong in it. A message + * boundary is not a property a byte stream has; ordering is not a property a + * datagram has. + * + * ⭐ AND IT IS THE EASIER HALF TO PROVIDE. A board that carries an IP stack + * reaches datagrams in a few hundred lines and connections in a few thousand, + * so an implementation that supplies only this one is ordinary rather than + * deficient --- clause 6.1 already expresses that by absence. */ +#ifndef OPENKAL_DATAGRAM_H +#define OPENKAL_DATAGRAM_H +#include "types.h" + +struct kal_datagram { kal_uintptr h; }; + +#define KAL_DGRAM_PROP_IPV6 ((kal_uintptr)1u << 0) +#define KAL_DGRAM_PROP_BROADCAST ((kal_uintptr)1u << 1) + +#ifdef __cplusplus +extern "C" { +#endif + +/* A local endpoint of zero port asks the environment to choose one. */ +int kal_datagram_open(const struct kal_endpoint* local, struct kal_datagram* out); + +/* A message is sent whole or not at all; a partial send is not a result this + * interface produces. */ +struct kal_io_result kal_datagram_send_to(struct kal_datagram, const void*, kal_uintptr len, + const struct kal_endpoint* to); + +/* Reports one message and who sent it. A message longer than the buffer is + * truncated and the excess is lost, which is what the medium does. */ +struct kal_io_result kal_datagram_recv_from(struct kal_datagram, void*, kal_uintptr len, + struct kal_endpoint* from); + +void kal_datagram_close(struct kal_datagram); + +extern const kal_uintptr kal_datagram_props; + +#ifdef __cplusplus +} +#endif +#endif +``` + +#### `openkal/space.h` + +> ⚠️ **本草案已被 §7.5 ⓵ 取代**:收敛成 `kal_space_start` 一个操作,`kal_space` 类型不存在。下面保留的是当时提出的形状。 + + +```c +/* openkal.space --- an address space, and a context executing in one. + * + * ⚠️ THIS IS NOT `fork'. Clause 7.1 refuses to require the duplication of an + * address space AND ITS EXECUTION STATE; what is here is the first half alone. + * A context started in a cloned space begins at a function the caller names, + * not at the instruction the caller was executing --- which is what lets this + * be stated in a C application binary interface at all. A library above this + * interface reaches `fork' by saving its own execution state before the clone + * and restoring it in the new context; that is composition, and it belongs + * above this line rather than in it. + * + * ⚠️ THERE IS NO `create'. An empty address space contains no code, so the + * entry function a caller would name is not in it. A program that wants a + * child with only what it grants uses kal_process_spawn_with. */ +#ifndef OPENKAL_SPACE_H +#define OPENKAL_SPACE_H +#include "types.h" +#include "process.h" + +struct kal_space { kal_uintptr h; }; + +/* Positions in kal_space_props. */ +/* Whether a clone carries the handles the original holds. An environment whose + * cloning primitive copies memory and not handles reports zero here, and a + * library above it cannot reach POSIX fork semantics. Measured: this is the + * case on at least one environment. */ +#define KAL_SPACE_PROP_CLONE_HANDLES ((kal_uintptr)1u << 0) +/* Whether the copy may be completed lazily, so that a store to cloned memory + * may fail after the clone reported success. A program that cannot tolerate a + * deferred failure adapts; it cannot ask for the other behaviour. */ +#define KAL_SPACE_PROP_DEFERRED_COPY ((kal_uintptr)1u << 1) + +#ifdef __cplusplus +extern "C" { +#endif + +int kal_space_clone(struct kal_space* out); /* clones the CALLER's space */ + +/* Starts a context in the given space. The result is a process: a context with + * an address space of its own is what that word means, and `kal_process_wait', + * `kal_process_terminate' and `kal_process_close' therefore apply unchanged. */ +int kal_space_start(struct kal_space, void (*entry)(void*), void* arg, + void* stack_top, struct kal_process* out); + +/* Releases the space. A space whose context is running is not released until + * that process ends; the two handles are independent and either may be closed + * first. */ +void kal_space_destroy(struct kal_space); + +extern const kal_uintptr kal_space_props; + +#ifdef __cplusplus +} +#endif +#endif +``` + +#### `openkal/process.h` 的三处增补 + +```c +/* --- added --- */ + +/* One directory a started program shall receive among its preopens. The layout + * is frozen (clause 5.3). */ +struct kal_preopen { + struct kal_dir dir; + const char* name; + kal_uintptr len; +}; + +/* Starts a program that receives exactly the directories named. The program + * reads them back through kal_fs_preopen, which is the operation this one is + * the inverse of --- clause 7.11. + * + * ⚠️ A SECOND DECLARATION RATHER THAN AN ARGUMENT ADDED TO THE FIRST, because + * clause 8 forbids altering an existing one. kal_process_spawn remains, and a + * program that does not grant directories keeps using it. */ +int kal_process_spawn_with(struct kal_dir base, + const char* path, kal_uintptr path_len, + const char** argv, const kal_uintptr* argv_lens, kal_uintptr argc, + const char** envp, const kal_uintptr* envp_lens, kal_uintptr envc, + const struct kal_spawn_streams* streams, + const struct kal_preopen* grants, kal_uintptr grant_count, + struct kal_process* out); + +/* A pair of streams of which one end crosses a spawn boundary. The parent + * holds `mine'; `theirs' is what it places in a kal_spawn_streams. + * + * ⚠️ BOTH ENDS ARE OWNED. A parent that does not release `theirs' after the + * spawn never observes the end of input on `mine' --- the classic deadlock of + * this arrangement, and the reason an interface that hands out these streams + * must also take them back. */ +int kal_process_channel(struct kal_stream* mine, struct kal_stream* theirs); +void kal_process_channel_close(struct kal_stream); + +/* --- added positions in kal_process_props --- */ +#define KAL_PROCESS_PROP_CHANNEL ((kal_uintptr)1u << 3) +#define KAL_PROCESS_PROP_GRANT_DIR ((kal_uintptr)1u << 4) +``` + +#### `openkal/types.h` 的两处增补 + +```c +/* A byte. The three fixed widths above are the ones openkal's OPERATIONS use; + * an address is the first datum openkal must state byte by byte. */ +#if defined(__UINT8_TYPE__) +typedef __UINT8_TYPE__ kal_u8; +#elif defined(_MSC_VER) +typedef unsigned char kal_u8; +#endif + +/* Where a connection or a message goes. The layout is frozen (clause 5.3). + * + * ⚠️ HERE AND NOT IN net.h, because `openkal.datagram' uses it and either + * interface may be provided without the other. + * + * ⚠️ BYTES AND A LENGTH, not a tagged union of families. The length is what + * distinguishes them, and it is a VALUE rather than a layout --- so the set of + * lengths this specification defines may grow while clause 5.3 holds the + * structure fixed. That is the same evolution rule clause 6.2 gives a property + * word, applied to a value instead of to a bit. + * + * An implementation refuses a length it does not know (kal_err_invalid) rather + * than reading it as one it does. Twenty-four bytes is chosen so that an + * address with a scope identifier fits without a second type. */ +struct kal_endpoint { + kal_u8 addr[24]; /* network order --- the bytes of the address itself */ + kal_uintptr addr_len; /* 4 = IPv4 16 = IPv6 20 = IPv6 with a scope + * identifier; further values are defined by later + * revisions and refused, not misread, by an + * implementation that does not know them. */ + kal_u32 port; /* a number, in host order: this interface does not + * ask a caller to perform a protocol's byte order + * conversion. */ +}; +``` + +--- + +### 2.13 ⭐⭐ 规范 / ABI 级整体 review:六处发现 + +写草案的过程本身是一次 review。逐条: + +#### ⓵ ⭐⭐ 错误集是**封闭**的,而超时需要一个错误 —— 而它已经有了 + +`types.h` 写着:*"The set is closed: an implementation maps its environment's +error values onto these and **does not extend them**."* ⇒ **不能加 `kal_err_timeout`。** + +⭐ 而 `kal_err_again` 的定义正是 *"the operation would block"* —— **一次超时的到期, +说的就是这件事。** ⇒ **复用它,零改动。** + +⚠️ 但必须在规范文本里写死:在带超时的操作上,`kal_err_again` **只**表示到期。 +否则「到期」与「本来就会阻塞」在一个不阻塞的调用上会同读数。 + +#### ⓶ ⭐⭐ `types.h` 里**没有字节类型** + +现有四个类型是 `kal_uintptr` / `kal_u32` / `kal_u64` / `kal_i64` —— +它们是**操作的**类型。而一个地址必须**逐字节**陈述。 +⇒ **加 `kal_u8`**,并在注释里说明它与那三个的区别(见 §2.12)。 + +⚠️ 用 `unsigned char` 而不加类型也可以,但 types.h 的注释说得很清楚: +「一个决定写在八个地方」正是它存在的理由。**加类型与既有理由一致。** + +#### ⓷ ⭐⭐⭐ `deadline` 这个名字与 `kal_task_wait` 不一致 ⇒ 改名 `timeout` + +`kal_task_wait(word, expected, **timeout_ns**)` —— **相对时长**,且 +**零表示不设超时**。 + +⚠️ 我原来叫它 `deadline` 并想用**绝对时刻**。⭐ **两处都要改**: +名字改成 `openkal.timeout`,参数改成相对的 `timeout_ns`,零的含义与 `task_wait` 相同。 + +> **同一个概念在一份 ABI 里必须只有一种拼法。** 这是本规范最一贯的一条, +> 而我差点在它上面留下第二种。 + +⚠️⚠️ **代价必须写下来**:零被「不设超时」占用了 ⇒ **无法表达「立即返回,不等」** +(即 `poll(timeout=0)` 的非阻塞探测)。 +⭐ 这是「逐字一致」与「表达力」的真实冲突,而一致性优先。 +调用方要非阻塞探测,传 `1`(纳秒)—— 丑,但**不引入第二种拼法**。 +⇒ 这一条应当写进 clause 6.3「考虑过而未采纳」。 + +#### ⓸ ⭐ `props` 有两种形式,而我差点全用错 + +现有头文件的分法是**严格的**: + +| 形式 | 用在 | 例子 | +|---|---|---| +| **函数**,取资源 | 属性**随资源**变化 | `kal_stream_props(s)` —— 终端与文件答案不同 | +| **`extern const kal_uintptr`** | 属性**随实现**变化 | `kal_task_props` / `kal_fs_props` / `kal_process_props` | + +⇒ 六个新接口的归属: + +| 接口 | 形式 | 理由 | +|---|---|---| +| `terminal` | **函数** | 一个 pty 能报尺寸,一条串口不能 —— **同一实现两种答案** | +| `net` / `datagram` / `space` | **`extern const`** | IPv6 支持、克隆是否带句柄 —— 是实现的性质 | +| `timeout` | **`extern const`** | 粒度是时钟的性质。⭐ 而且它不是位字段而是一个**数值**(`kal_timeout_granularity_ns`)—— 与其他 props 形状不同,要在规范里说明 | + +#### ⓹ ⚠️ `kal_endpoint` 布局冻结 ⇒ 三个字段永远加不进去 + +clause 5.3:结构体布局不可变。⇒ IPv6 的 **scope id**(链路本地地址需要)、 +**flow label**、以及任何非 IP 的地址族,**定稿之后永远加不进去**。 + +⭐ **我的裁决是不带**,理由写在草案注释里:携带一个协议族的每个字段, +就变成了那个协议族的接口而不是一个地址。 +⚠️ **但这是一条不可逆的决定,必须由你确认** —— 链路本地 IPv6(`fe80::/10`) +没有 scope id 是**用不了**的,而嵌入式网络里它并不罕见。 + +#### ⓺ ⭐ 三处小的但会咬人的 + +| | | +|---|---| +| `kal_space_clone` 克隆的是**调用者自己的**空间 | 没有第二个空间可克隆(要有就得先有 `create`,而 §2.8 ⓵ 删掉了它)⇒ **签名里没有 src 参数**,草案已改 | +| `space` 与 `process` **两个句柄同时存在** | 谁先释放?⇒ 草案定为**独立,任一先释放皆可**,并写进注释 | +| 32 位目标上 `kal_u64` 参数占两个寄存器 | `kal_timeout_recv_from` 有 5 个参数 ⇒ riscv32 上会溢出到栈。**性能,不是正确性** ⇒ 记录,不改 | + +#### 符号规模 + +SURFACE.txt 从 **~40** 增到 **~68**。`check-surface.sh` 的成本不变(它是逐行比对)。 +⚠️ **但每个后端都要重新回答一次「提供还是不提供」** —— 而那正是 clause 6.1 +要求的、且今天没有任何机制强制的事。⭐ **`--complete` 那一侧必须在新接口落地时 +一起被使用**,否则一个后端「忘了不提供」会以链接期缺席的形式混过去。 + +--- + +### 2.13' ⭐⭐⭐ `kal_endpoint` 的可扩展性:四个方案,第四个不需要保留字节 + +§2.13 ⓹ 把它记成「不可逆,要确认」。**再想一轮,它不必是不可逆的。** + +#### ❌ 方案一:保留字节 + +```c +struct kal_endpoint { kal_u8 addr[16]; kal_uintptr addr_len; kal_u32 port; + kal_u32 reserved[N]; /* must be zero */ }; +``` + +| ⚠️ | | +|---|---| +| **「必须为零」没有人保证** | C 不自动清零。一个忘了初始化的调用方,会在 reserved 被启用的**那一天**突然改变行为 | +| **N 取多少是一个猜** | 取少不够,取多每个 endpoint 都在浪费 | +| **它与 openkal 每一条规则相反** | clause 7.8 要求「说出整个意图」;保留字段是「我还没想好」的具体化 | +| 先例是反面的 | `sockaddr_storage` / `OVERLAPPED` 的保留字段被创造性地使用过 | + +⇒ **保留字节能工作,但它把一个「以后再说」冻进了 ABI。** + +#### ❌ 方案二:不透明字节 + 长度 + +调用方无法构造一个 endpoint ⇒ 需要构造函数 ⇒ 又回到接口里;且丢掉类型检查。 + +#### ⚠️ 方案三:版本化的类型 + +clause 5.3 冻结的是**已有类型的布局**,新增一个类型是允许的(clause 8)。 +⇒ 将来出 `kal_endpoint2` + `kal_net_connect2`。 +**诚实且有先例**(clause 7.8 让 `kal_fs_open_file` 与 `kal_fs_open` 并存), +⚠️ 但每次演进要把操作复制一遍。**留作最后手段。** + +#### ⭐⭐⭐ 方案四:变长地址 —— **可扩展的是取值,不是布局** + +关键在于把 scope id 看成**地址的一部分**,而不是 endpoint 的**另一个字段**。 + +```c +struct kal_endpoint { + kal_u8 addr[24]; /* 网络序，就是地址本身的字节 */ + kal_uintptr addr_len; /* 4=IPv4 16=IPv6 20=IPv6+scope 其余由 props 定义 */ + kal_u32 port; /* ⭐ 主机序数值，见下 */ +}; +``` + +⭐⭐ **它不是「保留字节」,区别是决定性的:** + +| | 保留字节 | 变长地址 | +|---|---|---| +| 那些字节今天的含义 | ❌ **未定** —— 谁也不知道该填什么 | ✔ **地址的长度就是它的长度**,超出 `addr_len` 的字节不被读 | +| 扩展时改的是 | ⚠️ **布局的解释**(clause 5.3 的边缘) | ✔ **`addr_len` 的取值集合** —— 一个值,不受 clause 5.3 约束 | +| 旧调用方 + 新实现 | ⚠️ 取决于它有没有清零 | ✔ `addr_len=16` 照常工作 | +| 新调用方 + 旧实现 | ⚠️ 静默地错 | ⭐ `addr_len=20` 被拒(`kal_err_invalid`)—— **一个诚实的失败** | + +⭐ **而这个形状 openkal 到处都在用**:每一个字符串都是 `const char* + kal_uintptr len`。 +**地址是同一形状**,不是一个新概念。 + +⭐ 并且它与 clause 6.2 的属性字**同构**:位可以增长,未分配的读作零; +这里是**取值**可以增长,不认识的被拒绝。⇒ **同一条演进规则的第二个实例。** + +#### 数组开多大 + +| 用途 | 字节 | +|---|---| +| IPv4 | 4 | +| IPv6 | 16 | +| IPv6 + scope id | **20** | +| 非 IP(Bluetooth BD_ADDR 6 / CAN 4 / mesh 8) | ≤ 8 | + +⇒ **24**。`24 + 8 + 4` 对齐后 40 字节 —— 传指针,不进寄存器,代价可忽略。 + +⚠️ **地址族怎么区分?** 一个 6 字节地址是 Bluetooth 还是别的? +⭐ **今天不需要区分**:clause 10 规定**每个接口一个实现** ⇒ 一个程序里只有一个网络栈, +而**那个栈知道它的地址是什么**;`kal_net_props` 说明它支持哪些长度。 +⇒ 一个同时支持 IP 与非 IP 的栈是将来的问题,**而 24 字节的余量让将来有得选** +(例如用 `addr_len` 的高位编码族)。**这正是不必现在就不可逆的意思。** + +#### ⭐ 顺带一处必须定死的:字节序 + +| 字段 | 序 | 理由 | +|---|---|---| +| `addr[]` | **网络序** | 它就是地址本身的字节,不是一个数值 | +| `port` | ⭐ **主机序数值** | 它是一个数,而 openkal 不让调用方做协议的字节序转换 —— 那是实现的事 | + +⚠️ **不定死的话每个实现会各猜一次**,而两边猜反了的症状是「连到了错的端口」, +在一个能连通的网络里可能很久才被发现。 + +#### 裁决 + +**采纳方案四。** ⇒ §2.13 ⓹ 那条「不可逆,要确认」**降级为一条普通的定形决定** —— +不带 scope id 的是**今天的取值集合**,不是这个类型的能力上限。 + +⚠️ 仍然不可逆的只剩一条:**`addr[24]` 这个数字**。24 是从上表推的, +若将来出现更长的地址族,只能走方案三。⭐ 但那时的代价是**一个新类型**, +而不是**一个已经发布的类型说不出话**。 + +--- + +### 2.14 SPEC 增补文本(clause 6.3 / clause 11) + +#### clause 6.3「Mechanisms considered and not adopted」增两条 + +> **Readiness notification.** An interface reporting that a stream may be read, +> by waking a word as `kal_task_wake` does, was considered as the remedy for a +> context that would otherwise wait without end. It composes better than the +> bound this specification adopted: one operation covers every waitable thing, +> a single context may await many sources, and a library above it needs no +> read-ahead buffer because a notification consumes nothing. +> +> It was not adopted because of what it asks of an implementation. On an +> environment whose readiness is discovered by polling a set of descriptors, an +> implementation would have to maintain that set and a context of its own to +> watch it — a mechanism reconstructed rather than a facility conveyed, which +> clause 7.1 excludes. The bound this specification adopts asks the same +> environment only for what it already does at the point of the call. +> +> **An instant rather than a duration.** `openkal.timeout` states a duration +> because `kal_task_wait` does. An instant would not accumulate drift when a +> caller retries in a loop, and was considered for that reason; it was not +> adopted because it would give one specification two spellings of one idea. +> A caller that requires an instant computes the remaining duration from +> `kal_time_monotonic`, and the cost of doing so falls on the caller that has +> the requirement rather than on every implementation. + +#### clause 11「Matters this version does not settle」增六条 + +> 4. **Networking.** Not defined by this version. `openkal.net` and +> `openkal.datagram` are anticipated by clause 3.4 and are separate for the +> reason clause 6.4 gives. +> 5. **Readiness.** Awaiting one of several sources is not an operation of this +> specification. It is reached above the interface, from `openkal.task` and a +> bound upon each wait; clause 6.3 records the alternative that was weighed. +> 6. **Terminal control.** Not defined by this version. `KAL_STREAM_PROP_INTERACTIVE` +> reports that a stream is one; no operation acts upon that fact. +> 7. **Permission and ownership.** Not defined, and not a deferral: a permission +> presupposes an identity, and the environments this specification targets do +> not agree that one exists. A C library above openkal reports the absence as +> the error its own surface defines. +> 8. **Creation and reading of links.** Not defined, and not a deferral, for the +> reason clause 6.4 gives: whether a filesystem has links is a property of the +> format rather than of the environment. `KAL_FS_PROP_LINKS` reports it; +> resolution follows one where the property is claimed. +> 9. **Duplication of the calling image.** `fork` is refused by clause 7.1 and +> that refusal stands. ⚠️ It is refused as an OPERATION: the atomic +> capabilities from which a library may compose it — cloning an address space, +> starting a context in one — are a candidate for a later version and are not +> excluded here. A sentence reading "openkal will not have fork" would bury +> them, and this clause exists so that it is not written. + +⚠️ **第 9 条的最后一句是本节最重要的一句** —— 它防的是我自己犯过两次的那个错 +(§1.2 / §1.3)。 + +--- + +## 3. openkal-musl(端口层)—— 七条,是本轮工作量的主体 + +### 3.1 ⓪ 定位 PC=0(**先做,而且是定位不是修**) + +⭐ **排第一不是因为最严重,是因为它让其余每一条的读数都不可信** —— 一个在早期 +初始化就 SIGSEGV 的程序,后面 26 条失败都是它的下游。 + +**已由源码收窄的两条**(不必再查): + +- `okm_syscall.c` 的 `default:` 返回 `-ENOSYS`(71 个 `case` + 一个 default) + ⇒ **不是未知系统调用走空指针** +- 端口层一共只有**两处**弱引用(`grep '__attribute__((weak))' port/src` = 2 行: + `okm_phdr.c:39` 的 `__ehdr_start`、`okm_syscall.c:31` 的 `kal_random_fill`), + **两处都判空了** ⇒ 不是 [[link-error-is-the-mechanism-not-the-defect]] 第二形态的复发 + +**三个仍开着的候选,各配一条决定性检查**: + +| 候选 | 检查 | +|---|---| +| `.init_array` 走过头或含 0 项(`okm_start.c:179` 的循环**逐项不判空**) | `readelf -x .init_array `;临时打印循环的 `a` 范围 | +| 程序自己 `dlsym` 得 0 就调用(静态 musl 的 `dlsym` 恒返回 0) | `nm \| grep dlsym`;`gdb` 在 `dlsym` 下断 | +| ⭐ **终端初始化**:`ioctl` 只答 `TCGETS`,`TCSETS`/`TIOCGWINSZ` 都 `-ENOTTY` ⇒ TUI 拿到失败后的路径 | **先排除这一条** —— 崩溃位置与它重合 | + +⚠️ **判据的单位是一整行输出**:`bt` 空的时候 `info registers rip rsp` 与 +`x/8gx $rsp` 不空。⚠️ **不要凭报错跳到修法** —— +[[reasons-written-from-memory-kill-good-fixes]];这里连报错都没有,只有一个空栈。 + +### 3.2 ① `copy_file_range` —— **只需这一个** + +⭐ 实测 libc++ 源码(`libcxx/src/filesystem/operations.cpp`): +`copy_file` 对 `copy_file_range` 的回落名单**含 ENOSYS**(`:314`), +对 `sendfile` 的**只认 `EINVAL`** ⇒ ENOSYS 走不到 `fstream` 回落。 + +⇒ **实现 `copy_file_range` 一个就够,`sendfile` 是多余的**(实现了 range 就永远到不了 +sendfile)。实现成 `kal_stream_read`→`kal_stream_write` 循环。 +⚠️ 短写不是成功结果([[openkal-spec-and-linux-impl]]),循环必须写全或报错。 + +**量级**:~40 行。 + +### 3.3 ② 替换 `popen.c` —— 替换源码,不重建机制 + +`okm_spawn.c` 的开篇已经论证过这个动作: + +> openkal has neither operation … What openkal has instead is **the composite** … +> so the replacement is a **translation of arguments rather than a reconstruction +> of a mechanism** — and it is shorter than the code it replaces. + +`posix_spawn.c` 已经在排除表里(`mcpp.toml:118`),`popen.c` **不在** ⇒ 它还在用 +`pipe2` ⇒ ENOSYS。 + +**两阶段**: + +| 阶段 | 实现 | 语义差 | +|---|---|---| +| 现在(`kal_process_channel` 落地前) | `kal_fs_open`(`CREATE\|EXCLUSIVE\|TRUNCATE`)开临时文件当子进程 stdout,`kal_process_wait` 后 seek 回 0 | ⚠️ 不能并发交错;需要可写文件系统 | +| `channel` 落地后 | 直接用 `kal_process_channel` | 语义补齐 | + +⚠️⚠️ **这不是 `pipe2`,不要假装是。** 三处语义差必须写进 README。 +⭐ **正因为差,才必须实现成 `popen` 而不是实现成 `pipe2`** —— 一个假的 `pipe2` 会让 +self-pipe 的调用方**静默地永远等下去**,这正是 +[[c-library-configured-by-what-is-beneath]] 里 futex 那条踩过的形状。 + +ⓘ **musl 的 `system()` 走 `posix_spawn`,所以它今天就能用** —— 「不能 shell out」的 +范围比它看起来窄,值得在回复里说清楚。 + +**量级**:~120 行。 + +### 3.4 ③ `last_write_time(dir)` —— clause 7.12 的保留名 + +clause 7.12 的 `"."` 就是为这件事存在的(*"a program holding a directory has no way +to ask an operation about that directory"*)。⇒ `stat`/`statx`/`newfstatat` 对目录路径 +应走 `kal_fs_open_dir` + `kal_fs_info(dir, ".")`,而不是按文件路径走 `kal_fs_open`。 + +**量级**:~20 行。 + +### 3.5 ④ `umask` 去静默 —— 本轮最硬的一条正确性缺陷 + +**实测**:`g_umask` 只被 `SYS_umask` 自己读写(`okm_syscall.c:388` 定义、`:937` 使用), +**从不作用于任何创建**;而 `kal_fs_open` 本来就没有 mode 参数。 + +⇒ `umask(077)` 返回旧值、报成功、**下一次创建不受影响**。这正是 `okm_opt.h` 开篇 +明令禁止的那一种: + +> ⚠️ THE ONE WAY THIS COULD GO WRONG IS NOT PRESENT: **nothing below reports +> SUCCESS having done nothing.** + +⭐ **它比「0600 落成 0666」那条更硬** —— 后者要论证具体值,这条不需要: +**一个有返回值的调用,它的效果不存在。** + +**处置**:删掉 `g_umask`,让 `SYS_umask` 落进 `default:` 的 `-ENOSYS`。 +⚠️ musl 的 `umask()` 会因此返回 -1/ENOSYS,调用方多数不检查 —— **这是对的**: +它此前得到的成功是假的。README 记一条。 + +**量级**:~10 行(删)。 + +### 3.6 ⑤ ⭐⭐ 「永远没有」变链接错误 —— 零设计成本,回报最大 + +> ⚠️⚠️ **本节的「零设计成本」是错的,实施后已由 §7.6 更正。** 机制有一个前提, +> 而这份文档把它当成了已经满足:`-ffunction-sections` + `-Wl,--gc-sections` +> **只有 `cfg(os = "linux")` 一个 target 设了**。另外三个 target 不收集未引用的 +> section,于是同一份排除表在宿主上绿、在 mingw 上把构建打死。 +> 下面保留的是当时的推理。 + +**问题**:今天 `okm_syscall.c` 用同一个 `-ENOSYS` 回答了两件性质不同的事。 + +| | 例子 | 正确的回答时机 | +|---|---|---| +| **这个后端不提供** | core-only 后端上的 `open` | **运行期 ENOSYS** ✔ `okm_opt.h` 那道缝做对了 | +| **openkal 根本没有这个接口** | `fork` / `socket` / `poll` / `chmod` | ⭐ **链接期** —— clause 6.2 的表就是这么规定的 | + +第二类是**永久的、与后端无关的**事实。ENOSYS 把它伪装成了运行期条件,于是 +`mcpplibs/tinyhttps`(**mcpp 自己索引里的包**)在 openkal 上**构建成功**, +跑到第 27 个测试才失败。 + +**做法**:`mcpp.toml` 的 `sources` 增排除项。**机制全部现成**: + +- `!` 排除语法已在用(今天已排除 10 个 musl 源) +- cflags 已有 `-ffunction-sections -fdata-sections`,ldflags 已有 `-Wl,--gc-sections` + (`mcpp.toml:240`)⇒ ⭐ **没被引用的不会失败,被引用的才失败** —— 粒度正好 + +**候选排除集合**(⚠️ 需一轮闭包实测): + +``` +!musl/src/network/** socket 全族 + getaddrinfo +!musl/src/select/** poll / select / epoll / eventfd / signalfd +!musl/src/process/fork.c 以及 vfork / _Fork +!musl/src/unistd/pipe*.c pipe / pipe2 +!musl/src/stat/chmod.c chmod / fchmod / fchmodat +!musl/src/unistd/symlink*.c symlink / symlinkat +``` + +⚠️ **闭包是这条的唯一风险**:musl 的 `network/` 内部互相引用 +(`getaddrinfo` → `socket`),排一个可能牵出一串。**这是实现问题,不是设计问题** —— +判据是「排完之后 `mcpp build` 全绿」。 + +⚠️ **`timerfd`/`eventfd` 要不要一起排?** 它们在 `select/` 之外。 +⭐ 建议**一起排**:一个「创建了却永远不会就绪」的 fd 比 ENOSYS 更坏。 + +**判据(两向,缺一不可)**: + +| 方向 | 断言 | +|---|---| +| 正 | 一个引用 `socket` 的程序**链接失败并指名 `socket`** | +| 反 | 一个**不**引用它的程序**正常链接** —— 否则就是把所有人都打死了 | + +### 3.7 ⑥ 就绪 `poll`/`select` —— 端口层设计,不动 spec + +**为什么在这里而不在 openkal**(⭐ 更准确地说是在 §2.7 的 `openkal.kit.wait` 里,由所有端口层共用):可由 `openkal.task`(`kal_task_wait/wake`, +`okm_opt.h:135`,端口的 `SYS_futex` 就走它)+ `openkal.stream` 的阻塞读组合出来 —— +每源一个辅助 task 做阻塞读,读到就唤醒主 task。**可导出 ⇒ 是库。** + +⚠️ **代价是真的,必须写下来**:「就绪」不等于「已读」,而 `kal_stream` 读了退不回去 +⇒ **端口层必须为每个可 poll 的 fd 做预读缓冲**。 + +- 它已经拥有 fd 表(`okm_desc`,`okm.h`)⇒ 缓冲有地方放 +- stdio 本来就在做预读 ⇒ 不外来 +- ⚠️ 但 `read` 必须改为**先取缓冲**,这动的是热路径 + +⚠️ **未测的两条**: +(a) 预读缓冲对 `poll` 语义的覆盖度 —— 能不能撑住 ftxui 的用法; +(b) 一个 `poll` 过、随后被传给别人(dup / 交给子进程)的 fd 怎么办。 + +**量级**:大,**需要一次设计**,是本批唯一需要设计而非实现的一条。 + +### 3.8 ⑦ `private_include_dirs`(等 mcpp) + +`port/include/features.h:70` 自己写下了第一优解和它为什么做不到: + +> ⓘ **THIS IS THE SECOND-BEST REMEDY.** The first would be for a package to +> distinguish the directories it is **built from** from the directories it +> **publishes**. Measured 2026-08-22: **mcpp cannot express it.** + +mcpp 侧改动见配套文档 §4。此处只需在 `mcpp.toml` 把三个内部目录移过去: + +```toml +include_dirs = ["port/include", "musl/include"] +private_include_dirs = ["musl/src/include", "musl/src/internal", + "musl-generated/internal"] +``` + +⚠️ **顺序必须与 `include_dirs` 交错保持声明序** —— `features.h` 那段实测说得很清楚: +挪到后面,musl 自己的构建会先找到公共 `` 而失败。 + +⭐ **判据不是「`hidden` 不再泄漏」,是「那三个目录不出现在消费者的命令行上」** —— +前者会随着包侧再打一个补丁而变绿,而缺陷还在。 + +### 3.9 ⑧ POSIX 验收套件 —— **长期回报最高的一条** + +**缺口**:openkal 的 conformance 验的是 **openkal 自己的 `kal_*` 面**; +**没有任何东西验 openkal-musl 重建出来的 POSIX 面。** + +⇒ 本轮五条缺陷它抓五条(⓪ / copy_file / popen / last_write_time / umask), +**在用户之前**。 + +**设计要点**: + +| | 要求 | +|---|---| +| 分母 | ⭐ 判据是 `67 passed / 94`,不是「套件跑过了」([[criterion-whose-no-is-also-silence]]) | +| 三态 | 每个用例声明**应当成功** / **应当失败且 errno 是 X** / **不适用于本后端**。⚠️ 只有两态会让「没测成」和「不支持」同读数 | +| 矩阵 | 每 backend × 每 arch,**真跑**(qemu),不是只链接 —— 本轮全部三类问题都在链接之后 | +| 种子 | ⭐ **报告者的 94 个用例**,他已经在 issue 里提出愿意提供 | + +⚠️ **它必须能变红。** 一个新增的、故意不被实现的用例必须让套件失败,否则这个套件 +就是 [[openkal-spec-and-linux-impl]] 记的那个「第一版 surface 检查器是空转的」。 + +--- + +## 4. openkal-llvm-runtime —— `__config_site` 与实际能力对账 + +### 4.1 实测的三行(§配套文档 §6.7 表 C) + +| 开关 | generic | 实际 | 处置 | +|---|---|---|---| +| `_LIBCPP_HAS_RANDOM_DEVICE` | 1 | ✅ 符合 | **0.1.3 已修**(报告者测的是 0.1.1) | +| `_LIBCPP_HAS_FILESYSTEM` | 1 | ⚠️ 部分 | **保持 1** —— `` 每个操作都有 `error_code` 重载,标准把它设计成允许失败;Windows FAT 上 `create_symlink` 同样失败 | +| `_LIBCPP_HAS_TERMINAL` | 1 | ❌ **不符** | ⭐ **改成 0**,直到 `openkal.terminal` 落地 | + +⭐ **为什么 FILESYSTEM 留 1 而 TERMINAL 改 0** —— 这不是尺度不一,是标准的态度不同: + +> **标准把这件事设计成「可以失败」的 ⇒ 声明有,失败时报错是对的。** +> **标准假定它「一定成功」/ 用开关表达有没有 ⇒ 没有就必须写 0。** + +### 4.2 ⭐ 对账要成为一条 CI 判据,而不是一次检查 + +`__config_site` 是**一份对下层能力的声明**,而今天**没有任何东西核对它**。 +三行里有两行曾与实际不符,而它们都是**编译期**写死的 —— 程序因此把 `` +和终端支持整个编进去,再在运行期一条一条撞 ENOSYS。 + +⇒ 每个开关配一个**最小探针**,与 §3.9 的验收套件同批跑: +`_LIBCPP_HAS_TERMINAL` ⇒ 一个 `tcgetattr`+`tcsetattr` 的探针; +`_LIBCPP_HAS_RANDOM_DEVICE` ⇒ 构造一个 `std::random_device` 并取一个值。 + +⚠️ 未测:`_LIBCPP_HAS_LOCALIZATION` / `UNICODE` / `WIDE_CHARACTERS` / +`TIME_ZONE_DATABASE` 四行,本轮没有验过。 + +--- + +## 5. openkal-linux 与其它 backend + +### 5.1 实测:openkal-linux 是**满的** + +| backend | abort | stream | memory | env | time | fs | process | task | random | exec | +|---|---|---|---|---|---|---|---|---|---|---| +| **openkal-linux 0.5.4** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| openkal-macos | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | +| openkal-windows | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | +| openkal-opensbi | ✅ | ✅ | ✅ | ✅ | ✅ | — | — | — | — | — | +| openkal-uefi | ✅ | ✅ | ✅ | — | — | — | — | — | — | — | + +⭐⭐ **十个接口一个不缺 ⇒ 本轮的缺口不在 backend,在接口集合本身。** +这条读数把「让 openkal-linux 更完整」整条路排除掉了,值得单独记。 + +### 5.2 新接口落地时,每个 backend 的答案 + +⚠️ **一个接口新增,五个 backend 都要**明确回答**「提供还是不提供」** —— +clause 6.1 的缺席是靠**不导出符号**表达的,不是靠不写。 + +| 接口 | linux | macos | windows | opensbi | uefi | +|---|---|---|---|---|---| +| `openkal.terminal` | ✅ termios | ✅ termios | ✅ Console API | ⚠️ UART:关行编辑=无事可做即完成;尺寸=不知道 | 同左 | +| `kal_process_channel` | ✅ pipe2 | ✅ | ✅ CreatePipe | ❌ 无 process ⇒ 整个 `openkal.process` 本就不提供 | ❌ | +| `openkal.net` | ✅ sockets | ✅ | ✅ Winsock | ⚠️ 需 BSP 带 lwIP ⇒ **默认不提供** | ⚠️ UEFI 有 TCP protocol,**未测** | + +### 5.3 openkal-linux 的一条待议:`openat(..., 0666)` 硬编码 + +`src/fs.cpp:127` 硬编码 `0666`,`:236` 硬编码 `0777`。 + +⚠️ **在 openkal 没有权限模型的前提下,这不是缺陷** —— 实现必须选一个值。 +⭐ 真正的缺陷是端口层的 `umask` 静默(§3.5)。 + +⚠️ **未测**:报告者报的「文件 0600 落成 0777」我复现不出机制 —— +`openat(...,0666)` 给不出 0777。**权限被放宽这件事成立;放宽到哪个值未定。** +⇒ §3.9 的验收套件里应当有一条用例把它量出来。 + +--- + +## 6. 分批与依赖 + +| 批 | 内容 | 仓库 | 前置 | +|---|---|---|---| +| **P0** | ⓪ 定位 PC=0 | musl / linux | — | +| **P0** | ⑤ **「永远没有」变链接错误** | musl(manifest) | — | +| **P0** | ④ `umask` 去静默 | musl | — | +| **P1** | ① `copy_file_range` · ③ `last_write_time(dir)` | musl | ⓪ | +| **P1** | ② `popen`(临时文件阶段) | musl | ⓪ | +| **P1** | `_LIBCPP_HAS_TERMINAL` → 0 | llvm-runtime | — | +| **P1** | clause 11 增补六条 | openkal | — | +| **P2** | ⑧ **POSIX 验收套件** | musl + CI | P0/P1(否则读数全是 ⓪ 的下游) | +| **P2** | ⑥ 就绪 `poll`/`select` | musl | 需设计 | +| **P2** | ⑦ `private_include_dirs` | musl | **mcpp 侧 C** | +| **P3** | `openkal.terminal` + 五个 backend | openkal + 5 | 设计定形 | +| **P3** | `kal_process_channel` + backend | openkal + 3 | 设计定形 | +| **P4** | `openkal.net` + 两个形态不同的实现 | openkal + 2 | ⚠️ 数据报与定形者两处未决 | +| **P4** | `openkal.space` + linux 实现 | openkal + linux | ⭐ **设计与 P3 同批做**,因为它决定 `spawn` 怎么被理解 | +| **P4** | ⭐ **句柄的跨空间传递**(独立立项) | openkal | ⚠️ `space` 只是第一个撞上它的;在它有答案前 `kal_space_start` 只传流 | +| **P2–P3** | ⭐ `openkal-kit`(非规范,openkal 仓库内的独立包,§2.7) | openkal 仓库 | 就绪/流对随 §3.7 一起落;`fork` 助手随 `space` 落 | + +⭐ **P0 三条零设计成本,而 ⑤ 的回报最大** —— 它把发现从运行期移到链接期, +让「范围」这件事对每一个用户可见,而不只是对报告者。 + +⚠️ **P2 的验收套件必须排在 P0/P1 之后** —— 在 ⓪ 修好之前跑它,读数全是 ⓪ 的下游 +([[openkal-portable-program-findings]]:一条流水线按顺序失败时,后面每一条缺陷都被 +藏住,而且第一条会被当成唯一一条)。 + +--- + +## 7. mcpp 侧的配合点 + +完整清单见配套文档 §6.9/§6.10.3。与本文直接耦合的只有两条: + +| # | 内容 | 本文哪里用到 | +|---|---|---| +| **C** | `[build] private_include_dirs` | §3.8 | +| **D** | 目标侧报告在「由图供给的层收窄了上层的面」时说出来 | §3.6 的运行期一侧 —— ⚠️ 引擎不得硬编码 POSIX 名 ⇒ 由包说 | + +--- + +## 7.5 ⭐⭐ 实施回填:写实现改掉了两处接口设计 + +本节是**落地之后**回填的。§2.12 的头文件草案里有两处,**读的时候没看出问题,写第一份 +实现时第一次运行就暴露了**。两处都是**改规范而不是改实现**。 + +### ⓵ `openkal.space` 从两个操作收敛成一个 + +草案是 `kal_space_clone` + `kal_space_start`,调用者手里拿着一个 space 句柄。 + +⚠️ **本规范面向的环境里没有一个把这对当原语。** Linux 的 `clone` 是**一个动作**: +复制地址空间**并且**在副本里开始执行,没有「只做前一半」的形式。要求实现把两者分开, +它只能照样启动一个上下文、把它停在某个等待原语上、再自建一条信令通道告诉它跑什么 —— +clause 7.1 对此的判词是:**是规范的形状有错,不是实现有错。** + +⭐ 分开的形式还有一个它答不上来的问题:调用者 clone 之后、start 之前改了自己的内存, +子上下文看见的是哪一份?一个操作没有这个问题。 + +⇒ `kal_space_start(entry, arg, stack_top, out)`,`kal_space` 类型与 +`kal_space_clone` / `kal_space_destroy` 一并删除。被否掉的形式记进 clause 6.3。 + +### ⓶ `openkal.net` 交出的是**被拥有的连接**,流是从它借的 + +草案里 `kal_net_connect` 直接交出一个 `kal_stream` 并声明它是被拥有的。 + +⚠️ **那在 clause 7.2 下无法实现。** 规范要求「实现不得把已释放的句柄当作有效」,而 +流句柄就是环境的传输操作所接受的东西 —— 在描述符系统上是一个**关掉就会被复用的数字**, +没有地方放代际。被拥有的句柄放得下,借来的流放不下。 + +⭐ `openkal.fs` 早就回答过同一个问题:文件被拥有,流经 `kal_fs_stream` 从它借,并随 +文件一起释放。`openkal.net` 现在就是这个形状,措辞也照抄 —— 因为是同一个安排。 + +更直白的症状是第一次跑就出现的: + + FAIL: the bytes read are the bytes written + FAIL: the peer observes end of input after a half-closure + +Linux 实现把连接打进了句柄方案,而 `kal_stream_read` 把那个打包过的字当描述符用。 + +⇒ 新增 `struct kal_net_conn` 与 `kal_uintptr kal_net_stream(struct kal_net_conn)`; +connect / accept / peer / local / shutdown / close 全部改收连接; +`kal_timeout_accept` 交出的也是连接。 + +### ⓷ `_LIBCPP_HAS_TERMINAL` 的结论与 §4.1 相反 + +§4.1 说把它设成 `0`。⭐ **那只是「对一个坏掉的 port 的正确描述」** —— 它门控的是 +`isatty`,而 `isatty` 之所以不工作,是端口层用 TCGETS 回答了 musl 用 TIOCGWINSZ 问的 +问题。修端口层才是修法,声明本来就该是真的。 + +⇒ 端口层已修;`__config_site` 的声明保持 `1`,并在 openkal-llvm-runtime 的 CI 里加了 +一条**对账**判据:判据是**关系**(pty 与 pipe 必须不同,且与系统 C 库同向), +两个方向都会红 —— 声明 1 而端口答不出、声明 0 而端口其实答得出,都判失败。 + +--- + +## 7.6 ⭐⭐ 实施回填:⑤ 的排除表从十八条缩到五条 + +§3.6 说这条「零设计成本,回报最大」。实施之后两条都要更正。 + +### 前提只在四个 target 里的一个成立 + +机制靠的是「没被引用的定义会被丢掉」,而那要 `-ffunction-sections` + +`-Wl,--gc-sections`。⚠️ **只有 `cfg(os = "linux")` 的 ldflags 里有**,macOS 与 +Windows 那两节**根本没设 ldflags**。于是宿主上排除是隐形的,mingw 上对象被保留: + + musl/src/legacy/daemon.c:18: undefined reference to `fork' + +⚠️ 我当时的实测只跑了宿主一行,并据此在清单注释里写下「这两个 flag 已经在位」—— +对我测的那个 target 为真,对其余三个为假。 + +### 闭包:一次问清,而不是跟着失败一条条加 + +排除一族只有在 musl 自己不从别处引用它时才成立。逐族查了目录之外的引用者: + +| 族 | 目录外引用者 | 闭包终点 | +|---|---|---| +| `timerfd` `eventfd` `signalfd` `inotify` `epoll` | **0** | — ⇒ 排除 | +| `select` | `network/res_msend.c` | 随 network | +| `network` | `syslog.c` `nscd_query.c` `mq_notify.c` | → `getpwnam` | +| `fork` | `forkpty.c` `wordexp.c` `daemon.c` `linux/clone.c` | → **`pthread_create`** | + +⭐ **fork 的闭包终点是线程实现,network 的终点是口令函数。** 排任一个都会带走一个 +必需的或普通的设施 ⇒ 两个都不排。 + +⚠️ 先前是跟着失败一条条加(lchmod→fchmodat、sem_open→link、daemon/wordexp/ +forkpty→fork),那是错的方法:这个问题在源码上一遍就能问清。 + +### 结论:不是拒绝,是还不可能 + +排除 network 族要等**每个 target 都收集未引用的 section**(ELF/PE 用 +`--gc-sections`,macOS 用 `-dead_strip`)—— 那是三条链接线的改动,可以单独测, +不该挂在这一批里。 + +⭐ **代价照直说**:今天一个用 socket 的程序仍然能构建、仍然在运行期失败,也就是 +`tinyhttps` 在第 27 个测试上遇到的那件事。留下的五条自带理由 —— 一个创建出来却 +永远不会就绪的 fd,比一个创建不出来的更坏 —— 且它们的闭包在每个 target 上都是空的。 + +### 另一条:可选接口的转发者必须弱引用 + +`pipe` 落到 `kal_process_channel` 上之后,裸机后端(不提供 `openkal.process`)上 +**每一个**程序都链接失败,不管它要不要 pipe。同一条规则四行之外就写在 +`kal_random_fill` 上。⇒ 判据改成符号类别:`w` 而不是 `U`。 + +--- + +## 8. 未决与我可能错的地方 + +1. **§2.3 `openkal.net` 的数据报问题未决**,而它有连锁后果(DNS)。**不能边写边定。** +2. **§3.7 的预读缓冲未测** —— 能否撑住 ftxui 的用法,以及 dup/传递后的 fd 语义。 +3. **§3.6 的排除闭包未测** —— musl `network/` 内部互相引用,可能牵出一串。 +4. **§5.3 权限放宽到哪个值未定** —— 我复现不出 0777。 +5. **§4.2 四行 libc++ 开关未验**(LOCALIZATION / UNICODE / WIDE_CHARACTERS / + TIME_ZONE_DATABASE)。 +6. ⚠️⚠️ **§1.2/§1.3 是我连续说错两次后的更正。** 第一次把 `fork` 写成「永远不可能 + 的硬天花板」—— 判据二(能不能由原子能力组合出来)本来就该问到这一步,我没问。 + 第二次把它降为「候选但不做」,给的三条理由里**两条是错的**:面的大小算错了 + (5 个操作,比 net 少),而「无 MMU 目标必然缺席」**根本不是反对理由 —— + 它就是 clause 6.1 的机制本身**。 + ⭐ 教训:**「这个目标做不到」在一个可组合的规范里不是排除理由,是缺席的答案。** +7. **§2.4 的三处未决(继承语义 / 与 spawn 的关系 / COW 是否承诺)必须动笔前定** —— + clause 8 不允许事后改,而其中第一条的两个需求方向相反。 +8. **我没有真的跑过一次报告者那样规模的工作区** —— 本文的实测都在等价的小工程上做的。 + +--- + +## 9. 相关记忆 + +[[link-error-is-the-mechanism-not-the-defect]] · [[c-library-configured-by-what-is-beneath]] · +[[openkal-spec-and-linux-impl]] · [[openkal-portable-program-findings]] · +[[second-instance-exposes-the-interface]] · [[criterion-whose-no-is-also-silence]] · +[[reasons-written-from-memory-kill-good-fixes]] · [[build-program-advisory-channel]] diff --git a/.agents/docs/2026-08-27-openkal-native-path-three-issues.md b/.agents/docs/2026-08-27-openkal-native-path-three-issues.md new file mode 100644 index 00000000..ae0a65ad --- /dev/null +++ b/.agents/docs/2026-08-27-openkal-native-path-three-issues.md @@ -0,0 +1,1426 @@ +# 目标侧被解析出来了,只发给了一个编译单元 + +2026-08-27 · 三个 issue 的深度分析 + 优化方案(**待 review,尚未实施**) + +三个 issue: +[mcpp#514](https://github.com/mcpp-community/mcpp/issues/514) · +[openkal-linux#13](https://github.com/mcpplibs/openkal-linux/issues/13) · +[openkal-musl#13](https://github.com/mcpplibs/openkal-musl/issues/13) + +前置:[`2026-08-25-the-two-layer-predicate-family.md`](2026-08-25-the-two-layer-predicate-family.md) · +[`2026-08-26-cross-target-implies-graph.md`](2026-08-26-cross-target-implies-graph.md) · +[`2026-08-24-target-side-architecture.md`](2026-08-24-target-side-architecture.md) + +--- + +## 0. 一句话 + +> **mcpp 已经算出了「这个目标的 C++ ABI 头文件集合是哪 18 个目录」,并且把它准确地 +> 交给了 `std` 模块那一个编译单元。图里其余每一个单元,只有恰好站在一条依赖边上的 +> 才拿得到。** + +`prepare.cppm:7207`: + +```cpp +// ⚠️ AND THE HEADERS THIS PACKAGE ITSELF IS BUILT AGAINST. +for (auto& d : pkg.publicUsage.includeDirs) + flags += " -isystem " + mcpp::xlings::shq(d.string()); +``` + +这段注释把道理写得完全正确 —— 「std 模块源码在任何重要的意义上都是这个包的一个翻译 +单元,它到达 C 库头文件的方式和其余单元一样」。**它只差一句:其余单元并不都能到达。** + +于是 `std` 是 openkal 味的(对),依赖包的 BMI 是载荷味的(错),任何同时导入两者的 +TU 在第一个碰到双方都声明了的名字的模板实例化处炸掉 —— 这正是报告者贴的 +`reference to 'space' is ambiguous`。 + +⭐ 这与 2026.8.25.1/.2 修的七条、2026.8.26.1 修的那条是**同一个形状**:一个已经被 +正确回答的问题,答案没有送到所有该读它的地方。区别只在这次不是「谓词问错了」,是 +**「答案只发给了一个人」**。 + +⚠️ **但这句话只覆盖 mcpp#514。** 三个 issue 里有一半不是缺陷,而是报告者被路由到了 +一条回答另一个问题的路径上 —— **§1.5 先回答「他要什么、为什么会走到这里、是谁的 +缺陷」,§6.5 给通用视角的完整性补充。** 只读修复清单会修错。 + +> ⭐ **只想看结论的读者直接去 §6.10** ——「三侧各做什么」,含判据的最终形态 +> (clause 10 基数 + 原子性)、三个接口的形状草案、mcpp 的七条、以及给报告者的 +> 三类答复。中间各章是它的依据。 +> +> ⭐ **openkal 及其生态仓库的设计/优化方案已独立成篇** —— +> [`2026-08-27-openkal-ecosystem-design-plan.md`](2026-08-27-openkal-ecosystem-design-plan.md)。 +> 本篇是三个 issue 的缺陷分析 + **mcpp 引擎侧**的修复;那篇是 **openkal / openkal-musl / +> openkal-llvm-runtime / 各 backend** 的接口判据与分批清单。两篇不重复。 + +--- + +## 1. 五条结论 + +| # | 结论 | 判据强度 | +|---|---|---| +| **A1** | 提供目标侧层的包,其 `publicUsage` 到不了兄弟依赖包的单元 | ⭐ **本机实测**(§2.2) | +| **A2** | **新发现**:编译侧仍在用 `!crossTargetFlag.empty()` 当「系统来自图」,是 #511 修的那条在**下一行**的孪生兄弟;`e2e 295` 只断言 `ldflags`,看不见它 | ⭐⭐ **本机实测**(§2.3) | +| **A3** | 跨 home 发现:`find_sibling_package` 无条件回落 `~/.xlings`;`active_home_xpkgs` 是 `mcpp::home::root()` 的第二份拷贝 | 读源码 + 本机存在性实测(§2.4) | +| **B** | 依赖缓存键描述了**编译器**,没有描述**它被指向的头文件集合**;同一台机器上 `std` 缓存把这件事做对了 | ⭐ **磁盘上的两个 JSON 实测**(§3.1) | +| **C** | openkal-musl 的 `hidden` 泄漏是**引擎缺口**,不是包的缺陷 —— 那份头文件自己写下了第一优解和它为什么做不到 | 读源码,一行(§4) | + +openkal-linux#13 不在这张表里:它**不是一条缺陷**,是三类东西被列在了一张表上。见 §5。 + +--- + +## 1.1 判据来源标注 + +本文严格区分四种依据,凡未实测的推断都标了「**未测**」: + +| 记号 | 含义 | +|---|---| +| ⭐ 实测 | 我在本机跑出来的,命令与输出都在文中 | +| 读源码 | 从 `origin/main@078631d` 的源码直接读出的事实 | +| 报告方所测 | issue 里贴的、我没有复现的 | +| **未测** | 我的推断 | + +实测环境:`origin/main@078631d`(`2026.8.26.2`)自举构建 · +x86_64-linux-gnu · llvm@22.1.8 · gcc@16.1.0 · `MCPP_HOME=~/.mcpp`。 + +--- + +## 1.5 诉求是什么,以及这条路为什么会长成这样 + +⚠️ **本章先于逐条修复,因为三个 issue 里有一半不是缺陷,而按缺陷去修会修错。** + +### 1.5.1 诉求(报告者自己的话,#492 评论) + +> 「我这边要同时出 x86_64 和 aarch64 两份静态二进制」 +> 「当时满脑子能出静态产物就行」 +> 「we're switching our workspace to llvm + musl static builds via this path and +> retiring our `build-static.sh` workaround」 + +⇒ **诉求是一个链接属性**:双架构、全静态、无 `PT_INTERP` 的 Linux 可执行文件。 + +应用本身是一个**完全普通的 POSIX C++23 CLI**:7 成员 workspace、~2k 编译单元、 +ftxui 终端 UI、tinyhttps 本地回调服务器、mbedtls、要 shell out、94 个用例的测试套。 + +⚠️ **它不是一个可移植性诉求。** 他没有要求这份源码跑到裸机、UEFI 或 supervisor 上。 + +### 1.5.2 他被路由到哪一格,以及为什么 + +`x86_64-linux-musl` 这**一个目标**在实测矩阵里是四格 +([`2026-08-26-the-support-matrix-measured.md`](2026-08-26-the-support-matrix-measured.md) §2/§3): + +| | **gcc** | **llvm** | +|---|---|---| +| **payload**(kernel-abi = Linux 直连) | ✅ `xim:musl-gcc` 自带 sysroot,libstdc++,**完整 POSIX** | ❌ ① 密闭性拒绝 —— clang 载荷不带 musl | +| **graph**(kernel-abi = openkal) | ❌ ⑤ 没有 `openkal-gcc-runtime` | ✅ libc++,**POSIX 子集** | + +⭐⭐ **两个可用格在反对角线上,而编译器那一维他没得选。** +GCC 16 的 modules 有未修的 ICE(#491 正文:BMI 读回期段错误,cc1plus 堆内 tree +指针损坏),clang 22 编同一代码库全绿。选定 clang 之后,`linux-musl` 只剩右下那格。 + +⇒ **他没有"选择"openkal —— 矩阵里只有那一格是亮的。** + +而右下那格的第三维是 `kernel-abi = openkal`,openkal SPEC clause 7.1 明写这一层 +**故意不具备 POSIX 的形状**(没有地址空间复制、没有全局路径命名空间、没有描述符), +因为它要能被实现在裸机、UEFI、supervisor 上。 + +⭐ **一句话:他要的是一个链接属性,拿到的是一层可移植性抽象。** +在 Linux 上,静态 musl 本来自带 fork/pipe/socket/chmod/symlink —— 因为 musl 直接 +对 Linux 说话。经由 openkal 之后这些被那一层的设计目标筛掉了,**而这笔交易他没有 +参与,也没有任何一处告诉过他。** + +### 1.5.3 构建过程从头到尾没有说过这件事 + +目标侧报告打的是三行,**全对**: + +``` +kernel-abi openkal (openkal-linux@0.5.4, graph) +c-abi musl (openkal-musl@0.3.5, graph) +c++-abi libc++ (openkal-llvm-runtime@0.1.3, graph) +``` + +**没有一行说「因此 fork / socket / chmod / symlink 会在运行期报 ENOSYS」。** +他是在 94 个用例跑出 27 个失败的时候知道的 —— 也就是**依赖解析期已知的事实, +被推迟到运行期才显形**。这与 [[recorded-field-with-no-decision-reader]] 是同一族: +问题问对了、答案也对了,只是没送到需要它的那一侧。 + +⚠️ **也是为什么 #491/#492 的撤回值得回看一眼。** 那个 PR 提的正是「llvm 家族的 +payload musl 路线」,即上表左下 → 右上那一格。它被 openkal 路线取代了,而两者 +**回答的不是同一个问题**: + +| | 回答的问题 | +|---|---| +| openkal 路线 | 一份源码怎么到达**多个环境** | +| #492 路线 | clang 怎么产出**静态 musl 产物** | + +第二个问题至今没有第二个答案。 + +### 1.5.4 那么,是谁的缺陷?—— 四类 + +| 条目 | 类别 | 归属 | +|---|---|---| +| §2 目标侧 include 集合到不了兄弟依赖包(A1) | **缺陷** | mcpp 引擎。与 openkal 无关,任何 graph 供给的目标侧都中 | +| §2.3 编译侧谓词(A2) | **缺陷** | mcpp 引擎。`mcpp build --target <宿主自己>` 就中,连依赖都不用 | +| §2.4 跨 home 发现(A3) | **缺陷** | mcpp 引擎(密闭性) | +| §3 缓存键缺头文件集合(B) | **缺陷** | mcpp 引擎(正确性:静默错产物) | +| §4 `hidden` 泄漏(C) | **缺陷,但根因在引擎** | 表现在 openkal-musl,成因是 mcpp 不能表达「编译用/发布用」 | +| §5.2 PC=0 跳转 | **缺陷** | openkal-musl 或 openkal-linux(位置未定) | +| §5.3 `copy_file_range`/`popen`/`last_write_time(dir)` | **缺陷** | openkal-musl 端口层:**可表达而未表达** | +| §5.5 权限静默放宽 | **缺陷** | openkal-linux + 端口层;根在 openkal 无权限模型 | +| §5.4/§5.6 `fork`/`pipe2`/socket/symlink | **不是缺陷** | openkal 在做它声明要做的事(clause 7.1) | +| §1.5.2 反对角线为空 | **不是缺陷,也不是边界** | **矩阵空缺** —— 没人做,而不是不能做 | + +⭐ **最后一行是本文最重要的一行。** 前面九行加起来修完,报告者仍然在 openkal 上, +仍然没有 `fork`。让他拿到他真正要的东西的,是最后一行。 + +--- + +## 2. mcpp#514 §A —— 目标侧的头文件集合只到达一个单元 + +### 2.1 现状:三条互不相交的路径 + +一个编译单元的 include 集合今天由三处拼出来,**互不知情**: + +| 来源 | 代码 | 到达谁 | +|---|---|---| +| 工具链载荷的头文件 | `hostflags.cppm:199-217` → 全局 `cxxflags` 规则 | 每一个单元 | +| 根清单的 `[build] include_dirs` | `flags.cppm:482` → 全局 `cxxflags` 规则 | 每一个单元(**含依赖包**,见 §2.2 的旁证) | +| 依赖包发布的 `publicUsage` | `prepare.cppm:3490` 定点 → `privateBuild` → `scanner.cppm:1100` → **逐单元** | **只有这条边的消费者** | + +openkal 那 18 个目录走第三条。而 `nlohmann.json` 不依赖 `openkal-llvm-runtime` —— +它是图里的兄弟,不是下游。所以它一个都拿不到。 + +⭐ **这不是传播漏了一条边,是这个事实放错了模型。** +`publicUsage` 描述的是「一个库对它的使用者的要求」,而目标的 C++ ABI 头文件集合是 +**整张图的属性** —— `mcpp.targetside` 那个模块的开篇写的就是这句话: + +> The fix is not a better guess. It is to resolve once, after the graph is known, +> and to have every consumer read that one value. + +目标侧已经 resolve 了(`prepare.cppm:6713`),`std` 模块读了它(间接地,通过 +provider 的 `publicUsage`),**编译边没有读**。 + +### 2.2 ⭐ 实测 ①:20 行复现,不需要 openkal + +``` +work/ + mcpp.toml [dependencies] abiprov = { path = "abi" } + mcpplibs.cmdline = "0.0.1" + src/main.cpp import abiprov; import mcpplibs.cmdline; + abi/mcpp.toml provides = ["mcpp:c++-abi=libc++"] + [build] include_dirs = ["abi-include"] +``` + +`mcpp build` 后读 `compile_commands.json`,问每一行「命令里有没有 `abi-include`」: + +``` +DEP cmdline.cppm abi-include on line: False ← 兄弟依赖包 +DEP options.cppm abi-include on line: False +DEP parse.cppm abi-include on line: False +ABIPKG abiprov.cppm abi-include on line: True ← provider 自己 +ROOT main.cpp abi-include on line: True ← 消费者 +``` + +⭐ **一个声明了 `mcpp:c++-abi` 的包,它发布的头文件目录到不了兄弟依赖包。** +报告里的现象在没有 openkal、没有 musl、没有交叉的情况下就成立了。 + +⚠️ 旁证:同一次实测里,根清单的 `[build] include_dirs = ["hdrs"]` **确实**出现在了 +`cmdline.cppm` 的命令行上(它走全局规则)。所以「依赖包看不到根的 include」这个直觉 +是错的 —— 看不到的恰恰是走 `publicUsage` 那条路的,也就是目标侧走的那条。 + +### 2.3 ⭐⭐ 实测 ②(新发现):编译侧的谓词还没修 + +`hostflags.cppm:199`: + +```cpp +const bool graphSuppliesTarget = !tc.crossTargetFlag.empty(); + +if (bypassCfg && !graphSuppliesTarget) { ...载荷 libc++ 头... } +if (!trustCfg && !graphSuppliesTarget && ...) { ...载荷 glibc / linux-headers 头... } +``` + +这**正是** [`2026-08-26-cross-target-implies-graph.md`](2026-08-26-cross-target-implies-graph.md) +分析的、`2026.8.26.1`(#511)在链接侧改成 `plan.targetSide.cAbi.prebuilt()` 的那个 +谓词。`git log -- src/toolchain/hostflags.cppm` 的最后一次改动是 #486 —— +**编译侧从未跟上**。 + +同一台机器、同一个编译器、同一个目标,只差写不写 `--target`: + +``` +$ mcpp build +$ mcpp build --target x86_64-unknown-linux-gnu +``` + +`build.ninja` 的 `ldflags` 行(#511 修过的那条):**逐 token 相同 ✔** +`cxxflags` 行:显式那条**少了六个 token**: + +``` +< --no-default-config +< -nostdinc++ +< -isystem /xim-x-llvm/22.1.8/include/c++/v1 +< -isystem /xim-x-llvm/22.1.8/include/x86_64-unknown-linux-gnu/c++/v1 +< -isystem /xim-x-glibc/2.44/include +< -isystem /xim-x-linux-headers/5.11.1/include +``` + +⭐⭐ **三个后果,一个比一个重:** + +1. **头文件来自一个库,目标文件链自另一个库。** 链接线保留了载荷的 + libc++/glibc(#511 修对了),编译线把它们全丢了 —— clang 回落到自己的默认搜索。 + 本机有系统头就悄悄编过去,没有就报一个指向载荷的错。 +2. ⚠️ **`--no-default-config` 一起消失了。** 于是任何带 `--target` 的构建都从 + 「一切显式」切换成「这台机器的 cfg 说了算」,而 `post_install.cppm:266` 自己写着 + 那个 cfg 是「per-machine, per-install-path artifact」。 + ⭐ **这解释了报告者的 workaround 为什么有效** —— + 他手写的 `x86_64-unknown-linux-musl-clang++.cfg` 之所以被读到,正是因为 + `--target` 路径把 `--no-default-config` 丢掉了。报告里那句 + 「必须叫 `-clang++.cfg` 这个名字」也随之解释:那是 clang 自己的 + `-.cfg` 默认查找规则,而不是 mcpp 的机制。 +3. **`e2e 295` 看不见它。** 那个测试的不变式写的是「命名宿主自己的目标什么都不改变」, + 而它只取 `grep -m1 '^ldflags'`。同一个不变式在下面一行不成立,而测试不看那一行。 + +### 2.4 第二个皱褶:跨 home 发现 + +报告者说「机器上存在别的 mcpp home 时,依赖单元会拿到**它们的**宿主工具链 include +集合」。两处机制: + +**(a) `xlings.cppm:935`** —— `find_sibling_package` 找不到时无条件回落 +`$HOME/.xlings/data/xpkgs`: + +```cpp +// Also check ~/.xlings/data/xpkgs/ (xlings global home) as fallback. +const char* home = std::getenv("HOME"); +if (home) { auto xlingsXpkgs = path(home)/".xlings"/"data"/"xpkgs"; ... } +``` + +`probe.cppm:448` 用它找 `linux-headers`,结果直接进每一条编译命令的 `-isystem`。 +⭐ 本机 `~/.xlings/data/xpkgs` **存在**(实测,含 `fromsource-x-*` 等 30+ 包)—— +这条回落在这台机器上是活的。 + +**(b) `xlings.cppm:836`** —— `active_home_xpkgs()` 自己重新推导了一遍 home: + +```cpp +if (const char* h = getenv("MCPP_HOME")) home = h; +else if (const char* u = getenv("HOME")) home = path(u)/".mcpp"; +``` + +而 `mcpp.home::root()` 的答案有**三档**(`$MCPP_HOME` → 自包含安装 +`/..` → `$HOME/.mcpp`)。自包含安装下两者给出不同的 home。 +⚠️ `home.cppm` 的开篇写的正是这件事:「Every path under the mcpp home must be +derived from here. Before #311 this logic existed in three places … the copies +drifted」。**这是第四份拷贝。** + +⚠️ **未测**:报告里「`--target musl` 时那些行仍然出现」这一点,与 §2.3 的实测冲突 +(`--target` 非空 ⇒ 载荷行被抑制)。最可能的解释是那次构建走了 **fast path 重放** +—— `.build_cache` 不记录 home,整项目指纹也不记录(`fingerprint.cppm` 11 个字段里没有 +任何一个是 home 或载荷路径,且 `normalize_driver_output` 会**故意**把 `/home/` 路径 +抹掉),于是换了 home 也命中同一个 `target///`,ninja 重放的是上一次生成 +的、写着另一个 home 绝对路径的 `build.ninja`。这也解释了「选中的味道会在 +`resolution.json` 里跨次留存」。**这条要一次实测才能定案**,命令在 §7。 + +### 2.5 修复方案 + +#### A1 —— 目标侧的使用要求是全图的,不是一条边的 + +在 `prepare.cppm` 目标侧解析完成之后(`resolvedTargetSide` 已定,`~6713`),收集 +**从图里供给任一 `mcpp:` 层的包**的 `publicUsage`,并入**每一个**包的 +`privateBuild`: + +```cpp +// 目标侧的头文件集合是这次构建的属性,不是某条依赖边的属性。 +// 它已经被算出来了 —— prepare.cppm:7207 把同一个集合交给了 std 模块。 +UsageRequirements targetSideUsage; // 四个字段: +for (idx : layerProviderPackageIndices) // includeDirs + merge(targetSideUsage, packages[idx].publicUsage); // includeDirsAfter +for (auto& p : packages) // cflags / cxxflags + appendUnique(p.privateBuild, targetSideUsage); // 已在的会被去重 +``` + +四条设计判断: + +1. **只并入 `privateBuild`,不并入 `publicUsage`。** 它对全图可见,不需要再传播; + 写进 `publicUsage` 会让它进入被打包的库的使用要求,那是另一个层次的承诺。 +2. **追加在末尾。** 包自己的头文件仍然先被搜到;目标侧只需要排在**驱动默认目录之前**, + 而驱动的默认目录永远在最后。 +3. **provider 自己是 `layerProviderPackageIndices` 的成员**,所以它不需要特例 —— + `appendUnique` 让「它本来就有」成为无操作。 +4. ⭐ **`prepare.cppm:7207` 那一段随之变成这条规则的一个实例。** 不要保留两份: + std 模块的 flags 应当从同一个 `targetSideUsage` 取,否则「哪些目录是目标侧的」 + 这个决定又有了两处推导 —— 这个仓库为这个形状付过四次代价(#233/#240/#242/#344)。 + +⚠️ **一个必须同时回答的问题**:`layerProviderPackageIndices` 里要不要含 +`Origin::Graph` 以外的层?**不要**。载荷/xpkg 供给的层由 `hostflags` 那条路负责, +两条路同时发会让顺序变成两处决定。判据就是 `Layer::fromGraph()`。 + +#### A2 —— 编译侧读 `targetSide`,和链接侧读的是同一个值 + +`hostflags.cppm` 拿不到 `plan.targetSide`(它是 `mcpp.toolchain` 层,不能 import +`mcpp.build.plan`)。⇒ 通过 `HostFlagOptions` 传一个 `bool cAbiPrebuilt`, +在 `flags.cppm` 那一处从 `plan.targetSide.cAbi.prebuilt()` 填,和链接侧**同一个表达式**: + +```cpp +hopt.cAbiPrebuilt = plan.targetSide.cAbi.prebuilt(); // flags.cppm，与 1245/1484/1547 同源 +... +const bool graphSuppliesTarget = !opt.cAbiPrebuilt; // hostflags.cppm:199 +``` + +⚠️ **不要在 hostflags 里重新推导。** 那正是 #486 的开篇列出的三处分歧的成因。 +`hostflags` 的其它调用方(std 模块预编译、build.mcpp 宿主编译)必须一并给出这个值, +否则 `std.pcm` 和导入它的单元会用两套头文件 —— 这就是 e2e 181 抓过的形状。 + +⚠️ **`--no-default-config` 要单独拉出来。** 它今天被 `bypassCfg && !graphSuppliesTarget` +连坐,而它与「目标侧从哪来」无关:cfg 是 per-machine 的不可复现产物,**任何** +mcpp 构建都不该读它。改成无条件发(clang 且有 cfg 时)。 + +#### A3 —— home 发现收敛到一个答案 + +1. `active_home_xpkgs()` 改为 `mcpp::home::root() / "registry" / "data" / "xpkgs"`。 +2. `find_sibling_package` 的 `~/.xlings` 回落:**去掉**,或改为只在 + `mcpp::home::root()` 与该路径同源时才走。 + ⚠️ 这条回落写于跨 home 还不是问题的时候;删掉它可能让某些机器上的 + `linux-headers` 探测从「找到别人的」变成「找不到」—— + 而 `probe.cppm:453` 的 verbose 分支已经为「找不到」准备好了诊断。 + **这个方向是对的:找不到会说话,找错了不会。** + +#### 判据(A) + +| # | 判据 | 形态 | +|---|---|---| +| A-1 | §2.2 那个 20 行工程里,`cmdline.cppm` 的命令行含 `abi-include` | 新 e2e,断言 CDB 的行而不是 grep 全文 | +| A-2 | `e2e 295` 同时比对 `^cxxflags` 与 `^ldflags`,两条都是恒等式 | **改现有测试**,一行 | +| A-3 | `--no-default-config` 在两种拼写下都在 `cxxflags` 里 | 并入 A-2 | +| A-4 | 单测:`targetSideUsage` 只收 `fromGraph()` 的层 | `test_targetside.cpp` 的表可以直接扩 | + +⚠️ **A-2 是这批里最重要的一条**,因为它是**恒等式**:不需要期望值表,在每台宿主上 +都成立,而且它会在下一次有人只修一层时立刻变红。 + +--- + +## 3. mcpp#514 §B —— 键描述了编译器,没描述它被指向的头文件 + +### 3.1 ⭐ 磁盘上的两个 JSON + +同一台机器,同一个 `build-cache/v1`: + +``` +~/.mcpp/build-cache/v1/pkg/mcpplibs/cmdline@0.0.1//entry.json + inputs.toolchain = { compiler, compiler_version, driver_identity, + target_triple, target_implied_flags, + stdlib, stdlib_version } ← 七个字段，没有一个是头文件集合 + +~/.mcpp/build-cache/v1/std//std-module.json + std_build_commands = [ "... -isystem ... -isystem ... " ] ← 整条命令行，含头文件集合 +``` + +⭐⭐ **两个缓存,一台机器,两种「输入相同」的定义。** `std` 那个是对的 —— +`stdmod.cppm` 的注释说它「一直就是正确的身份」。依赖那个少一根轴。 + +而且这根轴是**故意**被抹掉的一半:`normalize_driver_output`(`probe.cppm:151`) +会把 `/home/`、`/tmp/`、`/var/` 开头的路径整段替换掉,**这是对的** —— 它让缓存 +跨 home 可共享。错的是**没有任何别的东西**接着说出「跨的这两个 home 里,glibc 是 +2.39 还是 2.44、linux-headers 是哪一版、cfg 文件写了什么」。 + +### 3.2 报告的三个方向,机制各不相同 + +| 方向 | 报告的现象 | 机制 | +|---|---|---| +| 1 | 装了 cfg workaround 之后,被污染的 BMI 仍然被复用 | ⭐ **成立且已定位**:cfg 文件是 `--target` 路径下每条编译命令的输入(§2.3),而键里没有它。加了 cfg 键不动 ⇒ 命中 ⇒ 从不重编 | +| 3 | 两次**宿主**构建,glibc 头来自不同 home(2.39 vs 2.44),命中同一条目 | ⭐ **成立且已定位**:`driver_identity` 抹掉路径 + 键无 sysroot 轴 ⇒ 同键。混合 BMI 让 clang 前端 SIGSEGV 在 `ASTReader::FindExternalVisibleDeclsByName` —— **反序列化崩溃而不是可读诊断**,因为 BMI 之间没有互校验 | +| 2 | musl 构建填了缓存后,宿主 `mcpp build` 复用了 musl 味的 BMI | ⚠️ **按报告的字面意思不成立**:`target_triple` **在**键里(上面的 JSON 实测),`--target x86_64-linux-musl` 会改写 `tc.targetTriple`(`prepare.cppm:2480`,clang 分支)。⇒ 两次构建的 dep 键必然不同。**要么是另一条路径把它重新调味了,要么这条是从症状推出来的。§7 给了定案命令** | + +⚠️ 我把方向 2 单独拎出来说,是因为 **[[recorded-field-with-no-decision-reader]] 的 +教训是双向的**:一条被推出来而没被测的因果,会让修复落在一个已经为真的谓词上。 +`dep-bmi-cache-cross-version-poisoning` 那次我把根因写错过两次,第二次靠 grep 生成物 +才定死。 + +### 3.3 ⭐ A 修好之后,B 自己消失一半 + +`cache_key::fill_package_config` 已经把 `pkg.privateBuild.includeDirs` 折进 +`includes` 轴(前缀 `priv:`,相对 `` 归一化)。 + +⇒ **A1 把目标侧目录并进每个包的 `privateBuild` 的那一刻,每个依赖包的键就自动带上了 +目标侧的身份。** 不需要为 openkal 这条路加任何新轴。 + +这也是判断 A1 那个形状对不对的一个独立证据:**一个正确的建模会让下游的键自己变对。** + +### 3.4 剩下的一半:载荷侧的头文件集合 + +A1 覆盖不到「同一个 triple、同一个 clang、不同的 glibc/linux-headers 载荷」。这需要 +一根显式的轴。最小形状: + +```cpp +// BuildAxes 新增一个字段，A 轴（toolchain）内： +std::string targetHeaderSet; // 头文件搜索集合的摘要 +``` + +内容取**已经解析好的**那三处,不要重新推导: + +- `resolve_clang_driver(tc).compile_tokens(...)` +- `resolve_link_model(tc).compile_tokens(...)` +- clang cfg 文件的**内容摘要**(存在时),因为它是命令的一部分而不在命令里 + +⭐ **取 token 而不是取路径**:`stdmod` 已经证明了这个取法是对的(它折进整条命令行), +而且 token 里的 `` 路径可以用 `fill_package_config` 已有的归一化函数处理, +保持跨 home 可共享 —— 这正是 §3.1 说「抹路径是对的」的那个性质。 + +⚠️ **不要 bump `kCacheEpoch`。** 那个常量的注释规定了它的判据:「Bump ONLY when a +change makes previously written entries **unusable**」。加一根轴让旧条目**miss**, +不让它们不可用;而且 `inputs_match`(`bmi_cache.cppm:211`)对 +`inputs.toolchain` 是整对象比较,旧条目自然不匹配。`cache gc` 的大小统计也仍然有效。 + +⚠️ **`std` 缓存不需要改** —— 它已经对了。但它和 dep 缓存现在会因为**不同的理由** +失效,`mcpp cache list` 的输出会短暂地看起来不一致。这是正确的,值得在 +`docs/` 里写一句。 + +#### 判据(B) + +| # | 判据 | 形态 | +|---|---|---| +| B-1 | 单测:两个只在 `targetHeaderSet` 上不同的 `BuildAxes` 产生不同的 `key_hex` | `test_cache_key.cpp` 已有表 | +| B-2 | ⭐ e2e:`--cache global` 下,**写一个 cfg 文件**再构建,`build-cache/v1/pkg/...` 下多出一个目录 | ⚠️ **不能写进真载荷**;测试要在临时 `MCPP_HOME` 里做 | +| B-3 | A1 落地后,含 `mcpp:c++-abi` provider 的图与不含的图,同一个依赖包落在两个键上 | e2e,判据是**目录名**不是日志行 | + +⚠️ **B-2/B-3 的判据必须是磁盘上的条目,不是 CLI 打的 `Cached` 那一行。** +`dep-build-cache-scoping` 那次,「假状态行骗了三个月」。 + +--- + +## 4. openkal-musl#13 —— 包不能区分「我从哪儿编」和「我发布什么」 + +### 4.1 那份头文件自己写下了答案 + +`openkal-musl/port/include/features.h:70`: + +> ⓘ **THIS IS THE SECOND-BEST REMEDY.** The first would be for a package to +> distinguish the directories it is **built from** from the directories it +> **publishes**. Measured 2026-08-22: **mcpp cannot express it** — publicUsage +> takes privateBuild's include directories entire. Moving the two directories +> into per-glob flags places them AFTER include_dirs on the command line, and +> musl's own build then finds the public `` before the internal one +> and fails with `unknown type name hidden`. + +引擎侧就是一行,`prepare.cppm:4162`: + +```cpp +pkg.publicUsage.includeDirs = pkg.privateBuild.includeDirs; +pkg.publicUsage.includeDirsAfter = pkg.privateBuild.includeDirsAfter; +``` + +⇒ **issue 报的是 openkal-musl 的行为,缺陷在 mcpp。** 包侧现在的做法(`hidden` 在 +C++ 下给 `extern "C"`、C 下清空;`weak` 清空;`weak_alias` 删除)已经是第二优解, +而且被三位不同的消费者各校准过一次(`restrict` / 链接性 / compiler-rt 的 `weak`)。 +**不要动它。** + +### 4.2 方案:`[build] private_include_dirs` + +```toml +[build] +include_dirs = ["port/include", "musl/include"] # 编译 + 发布 +private_include_dirs = ["musl/src/include", "musl/src/internal", + "musl-generated/internal"] # 只编译，不发布 +``` + +- 语义:进 `privateBuild.includeDirs`,**不**进 `publicUsage.includeDirs`。 +- 顺序:与 `include_dirs` **交错保持声明顺序**,因为 features.h 那段实测说得很清楚 + —— 把这两个目录挪到后面,musl 自己的构建就先找到公共 `` 而失败。 + ⭐ 所以这不能实现成「private 的都排在后面」,必须是**同一个有序列表,带一个 + 发布位**。 +- ⚠️ 这是 `[build]` 的一个新键。按 + [[new-capability-key-floor-measured]]:`mcpp.toml` 里不认识的键会让整份 manifest + 加载失败还是被忽略?**发布前必须实测索引 latest 的行为**,并且 openkal-musl 的 + `min_mcpp` 下限要写下界。 + +### 4.3 为什么这值得做,而不是「包自己绕过去」 + +`OKM_MUSL_INTERNAL` 那条(见 [[c-library-configured-by-what-is-beneath]])解决的是 +**宏**的泄漏,它成立是因为 `defines` 不传播。**目录**没有对应的机制,所以三个 +musl-internal 目录仍然出现在每一个消费者的命令行上 —— `hidden` 只是**已经被发现的 +那一个**名字。musl 的 `src/include` 覆盖层里还有 `__syscall`、`__libc`、`a_cas` 等 +一批名字,下一个消费者会撞上下一个。 + +⭐ **判据不是「`hidden` 不再泄漏」,是「那三个目录不出现在消费者的命令行上」。** +前者会随着包侧再打一个补丁而变绿,而缺陷还在。 + +#### 判据(C) + +| # | 判据 | +|---|---| +| C-1 | 单测:声明了 `private_include_dirs` 的包,其 `publicUsage.includeDirs` 不含这些目录,`privateBuild.includeDirs` 含且**顺序与声明一致** | +| C-2 | e2e:消费者的 CDB 行里没有 provider 的私有目录,provider 自己的行里有 | +| C-3 | 生态:openkal-musl 改用新键后,一个把 `hidden` 用作普通标识符的消费者能编过 —— **这条要等 mcpp 发布并进索引** | + +--- + +## 5. openkal-linux#13 —— 三类东西,一张表 + +report 里那张 ENOSYS 表把三种性质完全不同的东西列在了一起。**逐条实现会做错至少三条。** + +### 5.1 分类 + +| POSIX 面 | 类别 | 依据 | +|---|---|---| +| `copy_file_range` / `sendfile` | **① 端口层可解,不动 spec** | 语义就是「从一个流读、往另一个流写」,`kal_stream_read/write` 齐全 | +| `popen` / 子进程输出捕获 | **① 端口层可解,不动 spec** | 见 §5.3 | +| `last_write_time(dir)` 报 "Is a directory" | **① 端口层缺陷** | `kal_fs_open_dir` + `kal_fs_info` 都在;`.` 是保留名(clause 7.12),目录自己可以被问 | +| `fork` | **② 设计边界,不是缺陷** | SPEC clause 7.1 明写:复制地址空间不能在每个环境上忠实完成 | +| 文件权限位 / `chmod` | **③ 真缺口,而且当前行为是静默放宽** | 见 §5.5 | +| `symlink` / `symlinkat` | **③ 真缺口** | `KAL_FS_PROP_LINKS` 与 `kal_node_link` 都在,**没有任何操作能用它们** | +| socket / bind / listen | **③ 真缺口** | SPEC clause 3.4 已经点名了 `openkal.net`,见 §5.6 | +| **PC=0 的跳转** | **⓪ 唯一无歧义的缺陷** | 见 §5.2 | + +### 5.2 ⓪ NULL stub —— 先修这个,因为跳到 0 的程序说不出任何话 + +⭐ **这条排第一不是因为最严重,是因为它让其余每一条都测不准。** +一个在早期初始化就 SIGSEGV 的程序,后面 26 个失败用例的读数都是它的下游。 + +我能从源码确定的**收窄**: + +- `okm_syscall.c` 的 `default:` 分支返回 `-ENOSYS`(实测 `grep`,68 个 + `case SYS_*` + 一个 default)。⇒ **不是未知系统调用走空指针。** +- 端口层一共只有**两处**弱引用(`grep '__attribute__((weak))' port/src` = 2 行): + `okm_phdr.c:39` 的 `__ehdr_start` 与 `okm_syscall.c:31` 的 `kal_random_fill`, + **两处都判空了**。⇒ 不是 [[link-error-is-the-mechanism-not-the-defect]] 第二形态 + 的复发。 + +⇒ 三个仍然开着的候选,每个配一条**决定性**检查: + +| 候选 | 决定性检查 | +|---|---| +| `.init_array` 走过了头 / 含 0 项(`okm_start.c:179` 的循环**逐项不判空**) | `readelf -x .init_array `;并在循环里临时打印 `a` 的范围 | +| 程序自己 `dlsym` 得到 0 就调用(静态 musl 的 `dlsym` 恒返回 0) | `nm | grep dlsym`;`gdb` 在 `dlsym` 下断 | +| 某个 `kal_*` 被以函数指针形式取址后置零(不是弱引用,是数据) | `gdb`:`x/4gx $sp` 取返回地址 → `info symbol` | + +⚠️ **不要凭报错信息跳到修法。** [[reasons-written-from-memory-kill-good-fixes]]、 +[[link-error-is-the-mechanism-not-the-defect]] 都是同一天里在同一个仓库上踩的: +「报错信息不是规范」。这里连报错信息都没有 —— 只有一个空栈。 + +⚠️ **判据的单位是一整行输出。** 空栈 + PC=0 时,`bt` 是空的,但 +`info registers rip rsp` 与 `x/8gx $rsp` 不是。 + +### 5.3 ① 端口层可解的四条 —— 一个都不用动 spec + +**(a) `copy_file_range` / `sendfile`。** 在 `okm_syscall.c` 里加两个 case,实现成 +`kal_stream_read` → `kal_stream_write` 的循环。⚠️ 短写不是成功结果 +([[openkal-spec-and-linux-impl]]),循环必须写全或报错。 + +**(b) `popen` —— ⭐ 替换源码,不要重建机制。** + +这正是 `okm_spawn.c` 开篇已经论证过的动作: + +> musl starts a program by duplicating itself … openkal has neither operation … +> What openkal has instead is **the composite** … so the replacement is a +> **translation of arguments rather than a reconstruction of a mechanism** —— +> and it is shorter than the code it replaces. + +`posix_spawn.c` 已经在 `mcpp.toml:118` 的排除表里。`popen.c` **不在**,所以它还在 +用 `pipe2` ⇒ ENOSYS。而 `kal_process_spawn` 的 `struct kal_spawn_streams` +(`process.h:17`)允许调用方指定三个流 —— **子进程的 stdout 可以是父进程给的任意流**。 + +⇒ 把 `!musl/src/stdio/popen.c` 加进排除表,写 `port/src/okm_popen.c`:用 +`kal_fs_open`(`CREATE|EXCLUSIVE|TRUNCATE`)开一个临时文件当子进程的 stdout, +`kal_process_wait` 之后把它 seek 回 0 交给 `FILE*`。 + +⚠️⚠️ **这不是 `pipe2`,不要假装是。** 语义差三处,而且都必须写进包的 README: +不能与子进程并发交错;不能用于 self-pipe;`popen(..., "w")` 方向要单独处理。 +⭐ **正因为差,才必须实现成 `popen` 而不是实现成 `pipe2`** —— +一个假的 `pipe2` 会让 self-pipe 的调用方**静默地永远等下去**,而这正是 +[[c-library-configured-by-what-is-beneath]] 里 futex 那条踩过的: +「⚠️ 答案由被重建的那张面决定」。 + +ⓘ 顺带:musl 的 `system()` 走的是 `posix_spawn`,所以 **`std::system` 今天就能用**。 +报告里「不能 shell out」的范围比它看起来窄,值得在回复里说清楚。 + +**(c) `last_write_time(dir)`。** clause 7.12 的保留名 `"."` 就是为这件事存在的 +(「a program holding a directory has no way to ask an operation about that +directory」)。端口层的 `stat`/`statx` 对目录路径应当走 +`kal_fs_open_dir` + `kal_fs_info(dir, ".")`,而不是 `kal_fs_open` 文件路径。 + +**(d) symlink 的**报告**方式。** 端口层已经在 +`okm_syscall.c:688/699` 对 `kal_node_link` 返回 `-ENOSYS`。这是对的,保留; +真正缺的是**创建/读取**,那属于 ③。 + +### 5.4 ② `fork` 是边界不是缺陷 —— 诊断而不是实现 + +SPEC clause 7.1 与 `okm_spawn.c` 的开篇都把 `kal_fork` 这个**操作**写死了。 + +⚠️⚠️ **本节初稿写的是「openkal 不会长出 `fork`,这是通用性的硬天花板」——那句话错了。** +clause 7.1 禁止的是**这个操作**,没有禁止**地址空间的原子能力**(创建地址空间 / +映射内存 / 以给定寄存器状态在其中启动上下文)。有了那三样,端口层**可以**把 `fork` +组合出来 —— Fuchsia 没有 `fork` 却有 `zx_process_create`/`zx_vmar_map`/`zx_thread_start`。 + +⇒ 正确的说法是:**`fork` 需要一组 openkal 目前没有的原子能力,而那组能力是一个 +将来的候选,不是一条永久的拒绝。** 为什么不在本批(三条理由)见 +[`2026-08-27-openkal-ecosystem-design-plan.md`](2026-08-27-openkal-ecosystem-design-plan.md) §1.2。 + +**本节其余结论不变**:今天没有那组能力,而报告者那一处应当改用 `posix_spawn` —— +因为**他那份代码在传统体系里也不可移植**(Windows 没有 `fork`)。 + +⚠️ 但报告者读到的是 `"fork failed: Function not implemented"` —— 一个不区分 +「本实现没有」和「本模型没有」的答案。⇒ 值得做的是**让这条边界可读**: +openkal-musl 的 README 增一节「这个 C 库不提供什么,以及改用什么」,把 +`fork`→`posix_spawn`、`pipe2`→`popen`、`socket`→(无)列成一张表。 + +⭐ **一条不能实现的调用,唯一能改善的是它被理解的速度。** + +### 5.5 ③ 权限位 —— 当前行为是**静默放宽**,这是唯一被明令禁止的失败模态 + +读源码: + +- `openkal/include/openkal/fs.h` —— `kal_fs_open` 的 flags 只有 6 位 + (READ/WRITE/CREATE/EXCLUSIVE/TRUNCATE/APPEND),**没有任何权限参数**; + `kal_node_info` 只有一个 `writable` 布尔。 +- `openkal-linux/src/fs.cpp:127` —— `openat(..., f, 0666)` 硬编码; + `:236` —— `mkdirat(..., 0777)` 硬编码。 + +⇒ 程序请求 `0600` 拿到 `0666 & ~umask`。⚠️ **这不是「缺一个特性」,这是 +`okm.h` 自己划的唯一禁区**: + +> ⚠️ THE ONE WAY THIS COULD GO WRONG IS NOT PRESENT: **nothing below reports +> SUCCESS having done nothing.** + +一个要求私有的文件被创建成组内/全局可读,而调用方**收到了成功**。密钥、token、 +会话文件都走这条路。⭐ **这条的严重性高于表里其它任何一条,而它在报告里排第四。** + +**最小的、加性的、clause 允许的一步**(如果一定要动 spec): + +- `kal_fs_props` 加一个位 `KAL_FS_PROP_PERMISSIONS`。clause 6.2 明写属性字就是 + 为这类问题准备的,而且「a position that has not been assigned reads as zero, so + that a program compiled against a later specification behaves correctly against + an earlier implementation」—— **向后兼容是这个机制自带的**,clause 8 也允许 + 「A revision may add declarations」。 +- **不加操作。** `chmod` 是一个 operation,clause 6.2 说 operation 的缺席应当由 + **独立接口**表达(链接期缺席),而不是塞进 `openkal.fs` —— 因为一个 FAT 分区上的 + 实现永远满足不了它,那正是 clause 6.4 禁止的。 + +⭐ 而在 spec 动之前,**端口层就能把静默去掉**:`O_CREAT|O_EXCL` 且请求模式比 +环境能给的更严格时,openkal-musl 至少要能被问出真相。具体形态需要一次设计讨论, +本文不预设。 + +### 5.6 ③ socket —— spec 已经预留了名字,但现在不做 + +SPEC clause 3.4 里 `openkal.net` **已经被点名**: + +> An earlier draft replaced `openkal.fs` and **`openkal.net`** with a single +> interface … Positioning applies to a file and not to a connection; +> **half-closure applies to a connection and not to a file.** + +⇒ 加一个 `openkal.net` 是 spec **预期内**的演进(clause 3.2:新接口一律放在 core +之外;clause 6.5:`openkal.exec` 已经给了「可选接口在依赖解析期决定」的先例)。 + +**但现在不该做**,三条理由,都来自这个生态自己的记录: + +1. [[openkal-portable-program-findings]]:**两个实现意见一致等于零证据** —— + 同一个作者、同一次阅读。一个从**一个 CLI 的需求**设计出来的网络接口,会把那个 + CLI 的形状(一个本地回调服务器)当成接口的形状。 +2. [[second-instance-exposes-the-interface]]:第二个实例才暴露接口是否完整。 + `openkal.net` 需要至少两个环境(比如 Linux 与 一个非 POSIX 环境)各写一份, + 才知道 half-closure、地址族、非阻塞该怎么切。 +3. clause 3.2 的不对称性:**放错位置的代价不对称**。接口一旦定下就不能改 + (clause 8:「shall not alter existing ones」;clause 5.3:结构布局不可变)。 + +⇒ 回复 issue 的正确形状是:**说明范围,而不是承诺路线图**。 +openkal 的 README/SPEC 值得增一节「本版不提供什么」,把 socket、pipe、fork、 +权限位、符号链接创建列进去,并说明每条是「边界」还是「未决」。 +⚠️ SPEC clause 11 已经有「Matters this version does not settle」这一节 —— +**这五条就应该进那一节**,那是它们的既有位置,不需要新机制。 + +--- + +## 6. 批次与优先级 + +| 批 | 内容 | 面 | 依赖 | +|---|---|---|---| +| **P0** | **A2** 编译侧谓词 + `--no-default-config` 无条件化 + `e2e 295` 扩到 `cxxflags` | mcpp | 无 | +| **P0** | **⓪** openkal-musl 的 NULL 跳转定位(不是修,是定位) | openkal-musl | 无 | +| **P1** | **A1** 目标侧使用要求全图化 + `prepare.cppm:7207` 收敛为它的实例 | mcpp | A2(否则两处顺序互相干扰) | +| **P1** | **A3** home 发现收敛到 `mcpp::home::root()` | mcpp | 无 | +| **P2** | **B** `targetHeaderSet` 轴(A1 之后剩下的那一半) | mcpp | A1 | +| **P2** | **C** `[build] private_include_dirs` | mcpp | 无 | +| **P2** | **①** copy_file_range/sendfile、popen 替换、`last_write_time(dir)` | openkal-musl | ⓪ | +| **P3** | **②** README 的「不提供什么」表 | openkal-musl / openkal-linux | ① | +| **P3** | **③** 权限位的静默去掉 + SPEC clause 11 增补五条 | openkal | 需要单独讨论 | +| **不做** | `openkal.net`、`fork`、`pipe2` | — | 见 §5.4 / §5.6 | + +⚠️ **A2 排在 A1 前面是有理由的**:A1 往命令行上加 `-I`,而 A2 决定载荷的 +`-isystem` 在不在。先加 `-I` 会让「载荷的行还在不在」这个问题被新加的目录遮住, +到时候读不出是哪一层在起作用 —— [[second-copy-of-a-decision-written-without-reading-the-first]]: +**一层盖住一层,修完要看下一层读数。** + +⚠️ **P0 那条 mcpp 改动会改变每一个带 `--target` 的构建的编译线。** 按 +[[everything-except-docs-goes-through-pr]],七个 openkal 仓库的 CI 都要在 +`MCPP_SOURCE_REF` 下现场构建验一遍,判据是 `under review:` 那一行,不是「七个全绿」。 + +--- + +## 6.5 通用完整性补充 —— 六条,都不专为这个场景 + +⚠️ 本章刻意**不**从「让报告者的程序跑起来」出发。判据是:**换一个用户、换一个应用, +这六条是否仍然成立。** 逐条给了这个反问的答案。 + +### G1 —— 一个层只有一个实现,它就还不是层 + +`mcpp.targetside` 的架构规则写着「mcpp 硬编码层名,永不硬编码实现」,理由是 +「生态的组合是 2×N×M 而包是 2+N+M」。可 `kernel-abi` 这一层今天在 Linux 上 +**N = 1**:所有 graph 路线都是 openkal。 + +两个空缺,都在矩阵里已被记录: + +1. **`openkal-gcc-runtime`**(填 ⑤)—— 让 gcc 也能走 graph。矩阵文档已记为生态空缺。 +2. **`linux-musl` 的 llvm 实现**(填 ①)—— 一个包,`provides` 只写两层: + + ```toml + provides = ["mcpp:c-abi=musl", "mcpp:c++-abi=libc++"] + # kernel-abi 不由它供给 —— 留空即"传统栈上这层没有名字" + ``` + + ⭐ **引擎已经能表达这个组合,不需要 #492 那种 `isLlvmMusl` 分支。** + `TargetSide` 的注释为这个形状写好了(*"kernelAbi HAS NO NAME ON A TRADITIONAL + STACK … 这个字段在 picolibc 裸机构建里读 `—`"*),而 `2026.8.25.1` 把 + `system_from_graph()` 拆成可以单独问 `cAbi.prebuilt()`,正是为了让 + 「一层来自图、另一层来自载荷」成为可表达的排布。 + ⇒ **这是包的工作,不是引擎的工作。** + + ⚠️ **未测**:需要真跑一次才能确认那个组合的 flag 路径通(尤其 `-B`/startup + objects 与 `--gcc-toolchain` 的来源)。#492 撤回时留下的设计文档 + `.agents/docs/2026-08-23-llvm-musl-target-design.md` 里有一半答案。 + +**换个用户还成立吗?** 成立。任何「要 clang + 静态 musl」的项目今天都只有 openkal +一条路,而这个组合与可移植性无关 —— 它是绝大多数「发一个能拷到任何 Linux 上跑的 +二进制」的诉求的形状。 + +### G2 —— 一个层收窄了它上面那层的面,这件事必须在构建期可读 + +今天:openkal 的缺席在 **链接期**表达(clause 6.1),这对直接调 `kal_*` 的消费者是 +对的。但 C 库**转发**它时,链接期缺席被翻译成了**运行期 ENOSYS** —— +这在分层上是正确的([[c-library-configured-by-what-is-beneath]]:一层用"不存在" +表达的缺席,到上一层变成"一个有定义的错误"),**代价是程序在运行期才发现依赖解析期 +就已知的事。** + +三个形态,按代价排序: + +| 形态 | 改动 | 局限 | +|---|---|---| +| openkal-musl 的 README 增「本 C 库在什么配置下不提供什么」表 | 零 | 要人去读 | +| build.mcpp 的 advisory 通道打一行 | 零引擎、零 spec | ⚠️ **缓存命中不重跑** ⇒ 提示会出现一次就消失([[build-program-advisory-channel]]),不能是唯一载体 | +| C 库声明它重建出来的 POSIX 面,mcpp 在目标侧报告里多打一行 | 新词汇 | **建议先不做** —— 等第二个消费者,理由同 §5.6 | + +⚠️ **引擎不得硬编码 POSIX 设施名**(架构规则四),所以这条无论如何都必须由包来说。 + +**换个用户还成立吗?** 成立,而且更普遍:任何 `Origin::Graph` 的层都可能收窄上层的面, +openkal 只是第一个。 + +### G3 —— 「能构建」不是「能用」,缺的是一整层验收 + +- openkal 的 conformance 套件验的是 **openkal 自己的 `kal_*` 面**。 +- **没有任何东西验 openkal-musl 重建出来的 POSIX 面。** + +⇒ 一套「POSIX 验收」套件,跑在 **每个 backend × 每个 arch** 上,会在用户之前抓住本轮 +的 ⓪(NULL 跳转)、①(`copy_file_range`/`popen`/`last_write_time`)、 +§5.5(权限静默放宽)—— **五条里的五条**。 + +⭐ **这是六条里回报最高的一条**:它把「67/94」从**用户的读数**变成 **CI 的读数**。 + +⚠️ 判据要带分母([[criterion-whose-no-is-also-silence]]): +`67 passed / 94` 是读数,「套件跑过了」不是 —— 一个什么都没发现的套件也会报成功。 +⚠️ 且必须**真跑**(qemu),不能只链接:本轮全部三类问题都在链接之后。 + +**换个用户还成立吗?** 成立。这是「第二个实例才暴露接口是否完整」 +([[second-instance-exposes-the-interface]])的验收版本。 + +### G4 —— 引擎表达力的三个缺口 + +前两个是本文 §2/§4 的 A1 与 C。第三个是新的,而且最深: + +**c. 包的构建不能条件于"已解析的目标侧"。** + +`openkal-musl/port/src/okm.h` 自己写下了这条: + +> ⚠️ THE TARGET IS A PROXY FOR THE IMPLEMENTATION, AND AN IMPERFECT ONE. … +> **mcpp conditions on the target and not on which package satisfies a +> capability; when it can, this block is where that would be read instead.** + +于是 `OKM_HAS_FS/PROCESS/TASK` 只能用 `cfg(os = "none")` 当代理 —— 而代理是错的: +一块板**可以**带真正的文件系统。 + +⭐ **这条可以做,而且比看上去便宜。已核三点:** + +1. 目标侧扫描只读 **manifest 的 `provides` + 已激活的 feature + 已解析的 tc**; + feature 激活在 `prepare.cppm:5603`,工具链在 `:5218` —— 都早于依赖 build.mcpp(`:6308`)。 +2. `directives.cppm` 的 **16 行指令表里没有任何一行能改 `provides`** + ⇒ build.mcpp 不可能改变扫描的输入。 +3. ⇒ **把目标侧解析从 `:6713` 前移到 `:6308` 之前是安全的**,之后每个包的 + build.mcpp 都能被告知"我下面是谁"。 + +⚠️ **未测**:前移会改变 `tsd::Inputs` 里 `tc` 的成熟度(`tc` 在 `:5218`–`:6713` 之间 +被读写过 39 处)。要先枚举哪些字段进了 `Inputs`,再判断它们在 `:6308` 时是否已定。 + +**换个用户还成立吗?** 成立。「C 库要被它下面那层配置」是分层系统的通例,不是 openkal 的特例。 + +### G5 —— 头文件集合必须进缓存键与指纹 + +本文 §3。通用形式:**任何"头文件集合不能由 (编译器, 三元组) 推出"的目标** —— +交叉、显式 sysroot、graph 供给 —— 都暴露在这条上。今天只有 `std` 缓存把它做对了。 + +**换个用户还成立吗?** 成立,且今天就在生效:§3.2 方向 3 是**两次宿主构建**,连交叉都不是。 + +### G6 —— 「换编译器」可能同时换掉平台接口,诊断要按两根轴说话 + +矩阵文档的建议 ③ 已经写了一半:「① 拒绝时指出可用的替代编译器,两向判据」。 + +⚠️ **另一半没写**:在 `linux-musl` 上,从 llvm 换到 gcc 不只换了编译器,还把 +`kernel-abi` 从 openkal 换成了 Linux 直连、`c++-abi` 从 libc++ 换成了 libstdc++ —— +**三样东西一起动了**。一条只说「用 gcc 也能构建这个目标」的建议,会让下一个人 +在不知情的情况下跨过同一条缝(方向相反)。 + +判据:该建议必须同时说出**换过去之后那三行目标侧报告长什么样**。 + +**换个用户还成立吗?** 成立 —— `x86_64-windows-gnu` 在两张表里 c-abi 一个是 `gnu` +一个是 `musl`,矩阵文档已经把那一格标成「一个目标名,两个 C 库」。 + +### 优先级:哪些解阻塞,哪些是完整性 + +| | 解报告者的阻塞 | 通用完整性 | +|---|---|---| +| **G1.2**(llvm-musl 包) | ⭐⭐ 直接解掉 openkal-linux#13 的**全部 27 条** —— 因为不再经过 openkal | ⭐⭐ | +| **G3**(POSIX 验收) | 间接:让剩下的缺陷在 CI 而不是在他那里显形 | ⭐⭐⭐ | +| A1/A2/A3/B(§2/§3) | ⭐ 解 mcpp#514 | ⭐⭐⭐ | +| C(§4) | ⭐ 解 openkal-musl#13 | ⭐⭐ | +| ⓪/①(§5.2/§5.3) | ⭐ 若他继续留在 openkal 上 | ⭐⭐ | +| G2/G4/G6 | — | ⭐⭐ | + +⚠️ **G1.2 与 §5 不是二选一。** 即使他换到 llvm-musl 包,openkal 路线仍然要修 —— +它服务的是**另一批用户**(裸机、UEFI、跨平台一份源码),而那批用户不会撞到 `fork`。 +⭐ 反过来说:**§5 那些条目之所以被当成"openkal 的缺陷",恰恰是因为一个不需要 +可移植性的用户被放到了可移植性抽象上。** 把他放回去之后,那些条目回到它们本来的 +优先级 —— 除了 ⓪ 和 §5.5,那两条在任何用户身上都是缺陷。 + +--- + +## 6.6 什么可以进 openkal —— 四问 + +判据不是我提的,`okm_syscall.c:836` 解释 `openkal.random` 为什么必须存在时已经写下了: + +> ⚠️ **AND IT IS WHY `openkal.random` HAD TO EXIST. Entropy is not derivable from +> the other interfaces** … Neither bypassing the layer nor inventing entropy was +> acceptable, so the layer gained an interface. + +把它和 clause 6.4 / 7.1 合起来,是四问: + +1. **只有环境能给吗?** 能由已命名的东西(流 / 任务 / 内存 / 进程)组合出来的,是**库**。 +2. **在形态不同的环境里都以可辨认的形式存在吗?** 预设了某个 OS 的模型(用户身份、 + 全局路径命名空间、地址空间复制)的,是**那个 OS 的形状**。 +3. **是不是「某些资源永远满足不了」的操作?**(clause 6.4) +4. **实现它会不会逼某个环境去构建兼容层?**(clause 7.1) + +### 按四问过一遍 + +| 候选 | 判定 | 理由 | +|---|---|---| +| **symlink** | ❌ 不进 | 文件系统**格式**的属性(FAT 没有,NTFS 语义还不同),不是内核能力。`KAL_FS_PROP_LINKS` 那个属性位是对的;加操作正撞 clause 6.4 | +| **permission / chmod** | ❌ 不进 | 预设**身份模型**。裸机/UEFI 没有用户,Windows 是 ACL,WASI 是能力模型 ⇒ 不通用。⭐ **缺陷是「静默」,不是「缺 chmod」**(见 §6.8 ②) | +| **pipe(进程内)** | ❌ 不进 | = ring buffer + `okm_task_wait/wake`,而这两个已经有(`okm_opt.h:135`,端口的 `SYS_futex` 就走它)⇒ **可导出 ⇒ 是库** | +| **pipe(跨到 spawn 的程序)** | ⚠️ 候选,但**不是现在** | 只有环境能做。⭐ 但它不是 `pipe2`,而是 `kal_process_spawn` 缺「给我一个连到子进程 stdout 的流」—— **已有接口的一次加法**(clause 8 允许)。而临时文件已能覆盖绝大多数用法(§5.3b),⇒ 等一个临时文件解决不了的用例 | +| **net** | ✅ 通过,**不是现在** | 网络栈确实不可由已有接口导出;clause 3.4 已点名 `openkal.net`。但 clause 8 定死后不能改 ⇒ 不该由一个程序的需求定形(§5.6) | +| **就绪/多路复用(poll)** | ⚠️ **新候选,见 §6.8 ①** | 这一条比上面任何一条都更接近「只有环境能给」,而且它今天一个都没有 | + +⭐ **结论:上一版列的四个可选接口里,三个不该进,第四个不该现在进。** + +### 6.6.1 ⭐⭐ 一条更硬的判据,它替换上面四问的**范围结论** + +上面四问要人去判断「是不是通用内核能力」—— 而那是一次判断,判断会错。把判断交给 +**已经存在的可移植程序**: + +> **传统体系下已经可移植的程序用到的东西,openkal 体系必须有。** +> +> 否则 openkal 让这些程序**更不可移植**,而不是更可移植 —— 它们在 Windows/macOS/Linux +> 上靠三条 `#ifdef` 分支就能走,到了 openkal 上没有第四条可写。 + +这条判据是从 openkal 自己的目的推出来的,不是外加的:openkal 存在是为了让一份源码 +到达多台机器。**一个已经到达三台机器的程序,不该在第四台上失去能力。** + +| 设施 | Windows | macOS | Linux | 可移植程序已经在用? | ⇒ openkal | +|---|---|---|---|---|---| +| 网络 socket | ✅ Winsock | ✅ | ✅ | ✅ **`mcpplibs/tinyhttps@0.2.8` 就是**(`#ifdef _WIN32` 两支) | ✅ **该有** | +| 就绪等待 | ✅ `WSAPoll` | ✅ | ✅ | ✅ 同上(它用 `poll`) | ✅ **该有** | +| 管道 + 子进程输出 | ✅ `CreatePipe`/`CreateProcess` | ✅ | ✅ | ✅ | ✅ **该有**(spawn 那半个已经有) | +| 终端控制(raw / 尺寸) | ✅ Console API | ✅ termios | ✅ | ✅ **ftxui 就是** | ✅ **该有** | +| **`fork`** | ❌ **没有** | ✅ | ✅ | ❌ 用了它的程序**本来就不可移植** | ❌ **不该有** | +| **mode 位 / `chmod`** | ❌ 只有只读位 | ✅ | ✅ | ❌ | ❌ **不该有** | +| **symlink** | ⚠️ 要权限 | ✅ | ✅ | ⚠️ 标准库把它设计成**允许失败** | ❌ **不该有** | +| `copy_file` | 标准库层面已可移植 | | | ✅ | ❌ 端口层做 | + +⚠️ **这张表与 §6.6 四问的结论不同,差在两处,而且是这条判据赢:** + +1. 四问把 `net` 判成「通过但等第二个消费者」。这条判据说**它已经有第二个消费者了** + —— mcpp 自己索引里的 `tinyhttps`,它的两支 `#ifdef` 就是两个独立实现, + 而且不是同一个作者读同一份规范写出来的 + ([[openkal-portable-program-findings]] 那条「两个实现一致等于零证据」在这里**不适用**)。 +2. 四问完全没看到**就绪原语**与**终端控制**。这条判据把它们提到与 net 同级, + 而它们比 net 小得多。 + +⭐ **推论:`__config_site` 那份开关清单本身就是一份现成的核对表** +(`_LIBCPP_HAS_TERMINAL` / `_LIBCPP_HAS_FILESYSTEM` / `_LIBCPP_HAS_THREADS` / …)—— +libc++ 已经替我们列出了「一个标准库需要下层提供什么」,而 §6.7 表 C 显示今天有两行 +与现实不符。**核对它,比逐条讨论接口更该先做。** + +### 6.6.2 ⭐⭐ 「永远不会有」与「这个后端没有」是两件事,今天用同一个答案回答 + +`okm.h` 论证 ENOSYS 正当性的那段是对的 —— 但它论证的是**第二件事**: + +> A C library may [answer unsupported], because POSIX's surface HAS one — ENOSYS +> — and every caller of `open` already handles a failure. + +`open` 在一个 core-only 后端上失败,是**接口存在而这个后端不提供** ⇒ ENOSYS 正确, +`okm_opt.h` 那道缝处理得也正确。 + +但 `fork` / `socket` / `poll` / `chmod` 是**另一件事:openkal 根本没有这个接口, +因此没有任何后端会提供它**。对这一类,ENOSYS 把一个**永久的、与后端无关的**事实 +伪装成了一个运行期条件。 + +⭐ **正确的答案是链接错误**,而这正是 clause 6.2 的表已经规定的: + +| 时机 | 机制 | 回答什么 | +|---|---|---| +| 链接 | 未定义符号 | 是否用了它**不提供**的接口 | + +⇒ **openkal-musl 应当把这些 POSIX 面从构建里排除掉**,让引用它们的程序链接失败并 +被指名。 + +**机制全部现成,是一次 manifest 编辑,没有新机器:** + +- `mcpp.toml` 的 `sources` 已经有 `!` 排除语法,**今天已经排除了 10 个 musl 源** +- cflags 已有 `-ffunction-sections -fdata-sections`,ldflags 已有 `-Wl,--gc-sections` + (`mcpp.toml:240`)⇒ **没被引用的不会失败,被引用的才失败**,粒度正好 + +**效果**:`tinyhttps` 在 openkal 上**链接失败并指名 `socket`**,而不是构建成功、 +测试跑到第 27 个才知道。⭐ 这把发现从**运行期**移到**链接期**,零设计成本。 + +⚠️ **未测**:排除集合的**闭包**要跑一轮才能定 —— musl 的 `network/` 内部互相引用 +(`getaddrinfo` → `socket`),排一个可能牵出一串。这是实现问题,不是设计问题。 + +--- + +## 6.7 能力支持表(实测) + +⚠️ 判据全部来自源码与磁盘,不是推的。每张表标了取数方式。 + +### 表 A —— openkal 接口 × 实现(取自各仓库 `src/` 的文件集合 + `provides`) + +| backend | abort | stream | memory | env | time | fs | process | task | random | exec | +|---|---|---|---|---|---|---|---|---|---|---| +| openkal-linux 0.5.4 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | +| openkal-macos | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | +| openkal-windows | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — | +| openkal-opensbi | ✅ | ✅ | ✅ | ✅ | ✅ | — | — | — | — | — | +| openkal-uefi | ✅ | ✅ | ✅ | — | — | — | — | — | — | — | + +⭐ **openkal-linux 是完整的 —— 十个接口一个不缺。** 所以本轮的缺口**不在 backend**, +在**接口集合本身**。这条读数很重要:它把「让 openkal-linux 更完整」这条路排除掉了。 + +### 表 B —— openkal-musl 重建出来的 POSIX 面 + +判据:`port/src/okm_syscall.c` 的 `case SYS_*` 集合(**71** 个)对 +`musl/src/**/*.c` 引用到的 syscall 集合(**289** 个)。⭐ 分母在这里。 + +| 设施 | 状态 | 判据 | +|---|---|---| +| 文件 I/O、目录、重命名、删除 | ✅ | `open/openat/read/write/lseek/getdents64/mkdirat/renameat2/unlinkat` 均在表 | +| stat / 时间戳 | ✅ | `stat/fstat/newfstatat/statx/utimensat` 在表 | +| 线程、futex、TLS | ✅ | `okm_thread.c` 直接走 `kal_task_*`;`SYS_futex → __okm_futex` | +| 匿名内存 | ✅(受限) | `mmap` 仅 `MAP_ANON\|MAP_PRIVATE` 且 `addr==0 && fd<0`,否则 `-ENOSYS` | +| 时间、睡眠 | ✅ | `clock_gettime/nanosleep/clock_nanosleep` 在表 | +| 起子进程 + 等待 | ✅ | `okm_spawn.c` 替换了 `posix_spawn.c`;`wait4` 在表。⇒ **`system()` 可用** | +| 熵 | ✅ | `SYS_getrandom → kal_random_fill`(弱引用 + 判空) | +| **`fork` / `clone` / `vfork`** | ❌ ENOSYS | 不在表 —— clause 7.1 的**设计边界** | +| **pipe / pipe2 / mkfifo** | ❌ ENOSYS | 不在表 | +| **poll / ppoll / select / pselect6** | ❌ ENOSYS | 不在表 —— 见 §6.7 ① | +| **epoll / eventfd / signalfd / timerfd** | ❌ ENOSYS | 不在表 | +| **socket 全族** | ❌ ENOSYS | 不在表 | +| **chmod / fchmodat / chown** | ❌ ENOSYS | 不在表 | +| **symlink / symlinkat** | ❌ ENOSYS | 不在表;`readlinkat` 在表但对 `kal_node_link` 返回 `-ENOSYS` | +| **copy_file_range / sendfile** | ❌ ENOSYS | 不在表 | +| **`umask`** | ⚠️ **成功且无效** | `g_umask` 只被 `SYS_umask` 自己读写,**从不作用于任何创建**(`okm_syscall.c:388/937`) | +| **终端控制** | ⚠️ 只答"是不是终端" | `ioctl` 仅 `TCGETS` 返回 0;`TCSETS`/`TIOCGWINSZ` → `-ENOTTY` | +| **信号 handler** | ❌ ENOSYS(**诚实**) | `rt_sigaction` 对 `SIG_DFL/SIG_IGN` 返 0,真实 handler 返 `-ENOSYS`(`:969-970`) | +| **uid/gid** | ⚠️ 虚构 | `getuid/geteuid/getgid/getegid` 一律返回 **1000**(`:935`) | + +### 表 C —— libc++ 的特性声明 vs 实际(读 `openkal-llvm-runtime/llvm-generated/*/__config_site`) + +| 开关 | generic | freestanding | 实际 | +|---|---|---|---| +| `_LIBCPP_HAS_THREADS` | 1 | 1 | ✅ 真的有 | +| `_LIBCPP_HAS_MONOTONIC_CLOCK` | 1 | 1 | ✅ | +| `_LIBCPP_HAS_RANDOM_DEVICE` | **1** | 0 | ✅ **0.1.3 已修**(报告者测的是 0.1.1) | +| `_LIBCPP_HAS_FILESYSTEM` | 1 | 0 | ⚠️ **部分** —— symlink / copy_file / permissions 运行期 ENOSYS | +| `_LIBCPP_HAS_TERMINAL` | 1 | 0 | ⚠️ **不符** —— 见表 B 终端控制 | +| `_LIBCPP_PSTL_BACKEND_STD_THREAD` | 是 | 是 | ⚠️ 线程有,**就绪原语一个没有** | +| `_LIBCPP_HAS_LOCALIZATION` / `UNICODE` / `WIDE_CHARACTERS` / `TIME_ZONE_DATABASE` | 1 | 1 | **未测** | + +⭐⭐ **这张表本身就是一个缺口的证据:`__config_site` 是一份对下层能力的声明,而 +今天没有任何东西核对它。** 三行里有两行与实际不符,而它们都是**编译期**就写死的 +—— 程序因此把 `` 和终端支持整个编了进去,再在运行期一条一条撞 ENOSYS。 + +⇒ **回答「能力是不是不完整」:是,而且不完整的位置不在 backend(表 A 满), +在接口集合(表 B)与声明和现实的落差(表 C)。** + +--- + +## 6.8 其他问题(报告者未列,本轮实测发现) + +**① ⭐⭐ 就绪 / 多路复用原语一个都没有。** +`poll`/`ppoll`/`select`/`pselect6`/`epoll_*`/`eventfd2`/`signalfd4`/`timerfd_*`/`pipe2` +**全部** ENOSYS(逐个查过 `case SYS_*`)。⇒ **任何事件循环都建不起来。** +这比"没有 socket"更根本 —— 一个不联网的 TUI 也要等"输入就绪或超时"。 +⭐ 按 §6.6 的四问,这一条最接近「只有环境能给」:等待多个来源之一变为就绪, +不能由 `kal_stream_read`(阻塞、单源)组合出来。**如果 openkal 只增一个接口, +候选应该是它,而不是 net。** +⚠️ **未测**:是否有环境无法提供它(裸机上"就绪"是否有意义)。这决定它是不是通用能力。 + +**② ⭐⭐ `umask()` 成功且无效。** `g_umask` 只被自己读写,从不作用于任何创建 +(`kal_fs_open` 也没有 mode 参数)。⇒ `umask(077)` 返回旧值、报成功、**下一次创建 +不受影响**。这正是 `okm_opt.h` 开篇明令禁止的那一种: +*"nothing below reports SUCCESS having done nothing"*。 +⭐ **它比 §5.5 那条更硬** —— §5.5 要论证"0600 被放宽"的具体值,这条不需要: +**一个有返回值的调用,它的效果不存在。** + +**③ ⭐ 终端控制不存在,而 `KAL_STREAM_PROP_INTERACTIVE` 这个位存在。** +`ioctl` 的注释自己写着:*"Everything a terminal can be asked to do beyond that is +not an operation openkal has."* ⇒ TUI 进不了 raw mode、拿不到窗口尺寸。 +⚠️ **未测**:这与 §5.2 的 PC=0 是否相关。但"早期终端 UI 初始化"这个位置与它重合, +**定位 PC=0 时应当先排除这一条**。 + +**④ `sigaction` 是诚实的**(我一度以为它静默吞掉,读源码后是我错了):真实 handler +返回 `-ENOSYS`。⇒ SIGWINCH / SIGINT 不可用,但程序会被告知。 + +**⑤ `getuid` 族返回硬编码 1000。** 一个被虚构的身份。⚠️ 与 ② 合起来,任何做 +"按权限/属主判断"的代码都在一个虚构的世界里运行,而且**全部报成功**。 + +--- + +## 6.9 综合改动清单:改动量、影响面、通用性、风险 + +⚠️ 改动量是**量级估计**,不是实测行数。影响面按"会不会改变现有构建的命令行/产物"分。 + +### mcpp 引擎侧 + +| # | 改动 | 文件 | 量级 | 影响面 | 通用性 | 风险 | 判据 | +|---|---|---|---|---|---|---|---| +| **A2** | 编译侧读 `targetSide`,`--no-default-config` 无条件化 | `hostflags.cppm` + `flags.cppm` + 3 处调用方 | ~40 行 | ⚠️ **改变每一个带 `--target` 的构建的编译线** | ⭐⭐⭐ 与 openkal 无关 | ⚠️ **中** —— 三个调用方必须同批给值,否则 `std.pcm` 与导入者头文件不一致 | e2e 295 扩到 `cxxflags`(恒等式) | +| **A1** | 目标侧使用要求全图化 + `:7207` 收敛 | `prepare.cppm` | ~60 行 | 只影响 graph 供给目标侧的构建 | ⭐⭐⭐ | 低 —— 追加+去重,现有边不动 | §2.2 的 20 行工程 | +| **A3** | home 发现收敛到 `mcpp::home::root()` | `xlings.cppm` | ~15 行 | ⚠️ 可能让某些机器上 `linux-headers` 从"找到别人的"变"找不到" | ⭐⭐⭐ 密闭性 | 低(失败会说话) | §7 ① | +| **B** | `targetHeaderSet` 轴 | `cache_key.cppm` + `prepare.cppm` | ~50 行 | **全部缓存条目 miss 一次**(不失效) | ⭐⭐⭐ | 低 —— 不 bump epoch,`inputs_match` 天然拒旧 | `test_cache_key.cpp` | +| **C** | `[build] private_include_dirs` | `types/toml/prepare` | ~80 行 | 新键,老 mcpp 的行为待测 | ⭐⭐ | ⚠️ **中** —— 见 §4.2 的 floor 问题 | 单测 + e2e | +| **G4c** | 目标侧解析前移到依赖 build.mcpp 之前 | `prepare.cppm` | ~30 行(移动) | 依赖 build.mcpp 的环境多几个变量 | ⭐⭐⭐ | ⚠️ **中高** —— `tc` 在 `:5218–:6713` 间被读写 39 处,前移要逐字段核 | 未定 | + +**引擎侧合计**:6 处、~275 行、4 个文件为主。⭐ **其中 A1/A2/A3/B 四条与 openkal +无关**,是任何交叉/graph 目标都吃的正确性问题。 + +### openkal 生态侧 + +| # | 改动 | 仓库 | 量级 | 影响面 | 通用性 | 风险 | +|---|---|---|---|---|---|---| +| **⓪** | 定位并修 PC=0 | musl 或 linux | 未知 | 全部程序 | ⭐⭐⭐ | 先定位,**不猜** | +| **①a** | `copy_file_range`/`sendfile` | musl port | ~40 行 | `std::filesystem::copy_file` | ⭐⭐⭐ | 低 | +| **①b** | 替换 `popen.c`(临时文件流) | musl port | ~120 行 | shell out | ⭐⭐ | ⚠️ 语义差三处,必须写进 README | +| **①c** | 进程内 `pipe2`(ring + task_wait) | musl port | ~150 行 | 自管道/线程唤醒 | ⭐⭐ | ⚠️ **与 spawn 的会合是本轮唯一要设计的地方** | +| **①d** | `last_write_time(dir)` 走 `"."` | musl port | ~20 行 | `std::filesystem` | ⭐⭐⭐ | 低 | +| **②** | 去掉 `umask` 的静默 | musl port | ~10 行 | 权限语义 | ⭐⭐⭐ | 低 —— 最小是删掉 `g_umask`,让它也 ENOSYS | +| **C'** | 用 `private_include_dirs` 收掉 3 个内部目录 | musl | ~5 行 | 所有消费者 | ⭐⭐ | 依赖引擎 C | +| **③** | clause 11 增补:net / pipe / 权限 / symlink / 终端 / 就绪 | openkal SPEC | 文档 | — | ⭐⭐⭐ | 零 | +| **④** | POSIX 验收套件 | musl + CI | 大 | — | ⭐⭐⭐ | 零(只增读数) | + +⭐ **注意:生态侧没有一条是「加 openkal 接口」。** 按 §6.6 的四问过完之后, +27 条里除 socket 外全部落在端口层。 + +### 架构与稳定性评估 + +| 维度 | 评估 | +|---|---| +| **架构方向** | ⭐ 全部改动都在**收敛**方向:A1/A2 把"同一个决定的第二处推导"删掉,A3 把 home 的第四份拷贝删掉,C 让包能说出它一直想说的话。**没有一条是加机制。** | +| **对既有生态的冲击** | A2 最大(改每个 `--target` 构建的编译线)。⚠️ 七个 openkal 仓库必须在 `MCPP_SOURCE_REF` 下现场构建验一遍 | +| **可回退性** | A1/A3/B/C 都是加法或收敛,回退=revert。⚠️ **A2 不是** —— 它会让一些今天"靠 cfg 文件恰好能编"的构建变成显式,**这些构建在 A2 之后可能需要真的声明它们的依赖** | +| **稳定性净收益** | ⭐⭐ B 消除的是**静默错产物**(混合 BMI 让 clang 前端 SIGSEGV);② 消除的是**静默错权限**。这两条的当前状态都是"报成功而结果是错的",属于最坏的一类 | +| **通用性** | 引擎六条里五条与 openkal 无关;生态九条里七条与本报告者的具体应用无关 | + +### 能不能解决报告者的问题,以及怎么解决 + +| 他的 27 条失败 | 修完之后 | 靠哪一条 | +|---|---|---| +| `fork` | ❌ 仍然不行 | **应用改用 `posix_spawn`**;`system()` 今天就能用 | +| 子进程输出捕获 | ✅ | ①b | +| `pipe2`(自管道) | ✅ | ①c | +| socket / 本地 HTTP 回调服务 | ❌ 仍然不行 | 需要 `openkal.net` —— **明确列为不解决** | +| 事件循环(poll) | ❌ 仍然不行 | §6.8 ①,**新候选接口** | +| `chmod` / 权限 | ⚠️ 从"静默错"变成"明确失败" | ② | +| `symlink` | ⚠️ 保持 ENOSYS,但被文档化 | ③ | +| `copy_file` | ✅ | ①a | +| `last_write_time(dir)` | ✅ | ①d | +| 终端 UI 崩溃 | ⚠️ **取决于 ⓪ 的定位结果** | ⓪(+ §6.8 ③) | +| `std::random_device` | ✅ **已解决** | 0.1.3 | + +⭐⭐ **结论:走 openkal 路线,他的应用仍然跑不起来 —— 因为 socket 与 poll 两条不在 +可修范围内,而它们是那个应用的核心(本地回调服务 + 终端事件循环)。** + +⇒ **能解决他问题的只有两条路,而且都不是"修缺陷":** + +1. **openkal 增 `net` + 就绪原语两个接口** —— 按 §6.6 的四问,`net` 通过但需要 + 第二个消费者定形;就绪原语是**本文新提的候选**,未测其通用性。 +2. **§1.5.2 那张矩阵的反对角线** —— 给 llvm 一条 payload musl 路线。他的应用不改一行。 + +⚠️ **两条都不该由这一个应用的需求来推动决定。** 而本文列的 15 条改动,无论选哪条路 +都要做 —— 它们修的是**引擎的正确性**与**端口层已经能表达却没表达的东西**, +与路线选择无关。 + +--- + +## 6.10 结论:三侧各做什么 + +### 6.10.0 判据的最终形态 + +三条判据在讨论中依次替换,**最后一条是规范自带的**,前两条只作为它的快速筛: + +| | 判据 | 地位 | +|---|---|---| +| 四问 | 「是不是通用内核能力」 | ⚠️ 要人判断,**判错过两处**(漏掉就绪与终端;把 net 判成"等第二个消费者") | +| 传统体系已可移植 | 「三个平台上可移植程序已经在用的,openkal 必须有」 | ⭐ 好用的筛,但**会把 `poll` 误判进来**(可移植程序用它是因为 POSIX 给了,不是因为不可约) | +| **clause 10 基数 + 原子性** | 「一个程序里几个实现?」+「能不能由已命名的东西组合出来?」 | ⭐⭐ **规范自带,且两处冲突都由它裁决** | + +加上一条形状约束,它决定接口**长什么样**而不只是**有没有**: + +> ⭐⭐ **openkal 的接口是面向内核的通用原子能力,不与任何一个内核的形状绑定。 +> 组合由上层做。** + +这条不是新增的原则 —— clause 3.4 已经在用它:*"The stream is therefore the shared +currency of the specification and **not its common entrance**"*,以及它拒绝那个 +「把名字映射到资源」的统一接口时给的三条理由。 + +### 6.10.1 openkal(spec)—— 三个接口,而且都要按原子性重新定形 + +⚠️ **每一条的形状都不是它在 POSIX 里的形状。** 这是本节的要点。 + +#### ① `openkal.net`(大)—— 与 `openkal.fs` 同构,但**不是 BSD socket** + +规范里的依据全部现成: +- clause 10:NIC(openhal,多个)→ 网络栈(openkal,一个),与 块设备 → `openkal.fs` **同构** +- clause 3.4:已经点名 `openkal.net`,并指出它与 fs 的差别正是 **half-closure** +- clause 3.4:拒绝解析无边界的名字方案(⇒ **DNS 不在里面**) + +⇒ 原子面大约是**六个操作**,而不是 BSD 的三十个: + +``` +kal_net_connect(endpoint) → stream 建立到一个端点的连接 +kal_net_listen(endpoint) → listener 接受入站 +kal_net_accept(listener) → stream +kal_net_shutdown(stream, dir) 半关闭 —— clause 3.4 点名的那件事 +kal_net_close_listener(listener) +kal_net_props 能力字 +``` + +**不在里面**(每一条都是 POSIX 的形状而不是内核的原子能力): +地址族与 `sockaddr` 家族、`setsockopt` 的选项空间、非阻塞标志(就绪归端口层)、 +`sendmsg/recvmsg`、**名字解析**。 +⇒ 端点是**结构化的地址+端口**,不是字符串;`getaddrinfo` 是端点之上的库。 + +⚠️ **两处未决,必须在动笔前定,不能边写边定:** +1. **数据报要不要?** 流与数据报是两种东西(clause 6.4 的形状)。若 v1 只做流, + 则 DNS 只能走 TCP —— 这是一个真实的连锁后果,不是细节。 +2. **谁来定形?** clause 8 规定接口一旦发布不可更改。⭐ 按 + [[second-instance-exposes-the-interface]],**需要两个形态不同的实现** + (一个宿主内核 + 一个 BSP over openhal)才知道分解对不对。 + +#### ② `openkal.terminal`(小)—— 按 clause 6.4 独立成接口 + +「raw 模式」**在不同资源上表现不同**(终端 vs 普通文件),这正是 clause 6.4 裁定 +`seek` 属于 `openkal.fs` 而非 `openkal.stream` 的那个形状。⇒ 独立接口,资源是一个 +交互式流,入口是已有的 `KAL_STREAM_PROP_INTERACTIVE`。 + +原子面约 **2 操作 + 1 询问**:「不要解释我的输入」(raw)、「不要回显」、「显示多大」。 +**不是 `termios`** —— 那是一个带六十个标志位的 POSIX 结构。 + +裸机上:没有行编辑 ⇒ 关掉它是**无事可做即完成**(`okm_opt.h` 已有这类先例); +尺寸报「不知道」。 + +#### ③ `openkal.process` 的一次加法(小)—— 跨 spawn 边界的通道 + +`kal_spawn_streams` 已经收流句柄,缺的是**取得一个跨得过 spawn 边界的流**。 + +``` +kal_process_channel(void) → (parent_stream, child_stream) +``` + +⭐ **比「给我子进程的 stdout」更原子**:调用方自己决定把 `child_stream` 装到子进程 +的哪一个流上,于是 `popen("r")`、`popen("w")`、双向捕获都由上层组合出来 —— 正是 +「上层自己组合」。 + +⚠️ **进程内的**流对不属于这里(可由缓冲 + `kal_task_wait/wake` 导出 ⇒ 是库)。 +这条的存在理由**只有**「跨地址空间」。 + +#### ④ 不进 openkal 的,以及为什么(记录下来,免得再讨论一次) + +| | 为什么不进 | +|---|---| +| `fork` | clause 7.1;且 Windows 没有 ⇒ 用它的程序本来就不可移植 | +| `pipe`(进程内) | 可由 `kal_task_wait/wake` + 缓冲导出 ⇒ 库 | +| **就绪 `poll`/`select`** | ⭐ 同上,可由 task+stream 导出。**「传统体系已可移植」那条判据在这里判错了** | +| `chmod` / mode 位 | 预设身份模型;Windows 只有只读位 ⇒ 不通用 | +| `symlink` | 文件系统**格式**的属性;`std::filesystem` 本来就设计成允许失败 | +| DNS / `getaddrinfo` | clause 3.4 明确拒绝「解析无边界的名字方案」⇒ 端点之上的库 | + +#### ⑤ 文档(零代码) + +clause 11「本版不解决」增补:网络、就绪、终端、权限、符号链接、进程复制 —— 六条, +每条说明是**边界**(fork/权限/symlink)还是**未决**(net/terminal/channel)。 + +### 6.10.2 openkal 生态(实现层) + +| # | 内容 | 仓库 | 量级 | +|---|---|---|---| +| ⓪ | **定位并修 PC=0** | musl / linux | 未知,**先定位** | +| ① | `copy_file_range`(**只需这一个**,见下) | musl port | ~40 行 | +| ② | 替换 `popen.c`(临时文件流;`channel` 落地后改用它) | musl port | ~120 行 | +| ③ | `last_write_time(dir)` 走 clause 7.12 的 `"."` | musl port | ~20 行 | +| ④ | **`umask` 去静默** —— `g_umask` 只被自己读写,删掉它让它也 ENOSYS | musl port | ~10 行 | +| ⑤ | ⭐ **「永远没有」变链接错误**:排除 `net/`、`fork.c`、`pipe*.c` 等 | musl **manifest** | 编辑 + 一轮闭包实测 | +| ⑥ | **就绪 `poll`/`select`**(含预读缓冲)—— 端口层设计,不动 spec | musl port | 大,**需设计** | +| ⑦ | 用 `private_include_dirs` 收掉三个 musl 内部目录 | musl | ~5 行,等 mcpp | +| ⑧ | ⭐ **POSIX 验收套件**(每 backend × 每 arch,真跑) | musl + CI | 大,**回报最高** | +| ⑨ | `__config_site` 与实际能力**对账** | llvm-runtime | 见下 | + +**关于 ①**:libc++ 的 `copy_file` 对 `copy_file_range` 的回落名单**含 ENOSYS** +(`operations.cpp:314`),对 `sendfile` 的**只认 EINVAL** ⇒ 实现 +`copy_file_range` 一个就够,两个都做是多余的。 + +**关于 ⑨**:`_LIBCPP_HAS_TERMINAL 1` 与 `_LIBCPP_HAS_FILESYSTEM 1` 今天与实际不符 +(§6.7 表 C)。⭐ **在 `openkal.terminal` 落地前,`HAS_TERMINAL` 应当是 0** —— +声称有然后运行期失败,比声称没有更糟。 + +### 6.10.3 mcpp(引擎) + +与 openkal 的路线选择**完全无关**,无论 openkal 怎么长都要做: + +| # | 内容 | 量级 | 关键风险 | +|---|---|---|---| +| **A2** | 编译侧读 `targetSide`;`--no-default-config` 无条件化 | ~40 行 | ⚠️ 改每个 `--target` 构建的编译线;三个调用方同批 | +| **A1** | 目标侧使用要求全图化;`:7207` 收敛为它的实例 | ~60 行 | 低 | +| **A3** | home 发现收敛到 `mcpp::home::root()`;删 `~/.xlings` 回落 | ~15 行 | 低 | +| **B** | 缓存键增 `targetHeaderSet` 轴 | ~50 行 | 低,不 bump epoch | +| **C** | `[build] private_include_dirs` | ~80 行 | ⚠️ 新键的 floor 待测 | +| **G4c** | 目标侧解析前移到依赖 build.mcpp 之前 | ~30 行 | ⚠️ `tc` 39 处读写,逐字段核 | +| **D**(新) | ⭐ 目标侧报告增一行:**由图供给的层收窄了上层的面时说出来** | 未定 | 引擎不得硬编码 POSIX 名 ⇒ 由包说 | + +### 6.10.4 应当提示报告者做什么 + +⭐ **给可判定的答复,不是"我们在看"。** 他的 27 条分三类,三类的处置不同: + +**第一类 —— 我们修,他不用改代码(5 条)** +`copy_file` · `last_write_time(dir)` · 子进程输出捕获 · `umask` 语义 · `random_device`(**0.1.3 已修**,他测的是 0.1.1) + +**第二类 —— 他要改,而且在传统体系里也该改(3 条)** + +| 他现在写的 | 改成 | 为什么这不是将就 | +|---|---|---| +| `fork` | `posix_spawn` / `std::system` | ⭐ **Windows 没有 `fork`** —— 这份代码在传统体系里也不可移植 | +| `std::filesystem::create_symlink` 假定成功 | 失败则降级 | Windows 上没有权限时同样失败;标准把它设计成可失败 | +| `permissions()` 假定成功 | 同上 | Windows 只有只读位 | + +**第三类 —— 今天没有绕法,要等接口(2 条)** +终端 UI(等 `openkal.terminal` + 端口层就绪)· 本地 HTTP 服务(等 `openkal.net`) + +⇒ **在这两条落地前,他的程序在 openkal 上跑不完整。这不是缺陷,是范围。** +他有三个知情选择,而**今天他一个都看不见**: + +1. 等第三类接口(时间由 spec 定形决定,不由缺陷修复决定) +2. 把这两处换成 openkal 面上能表达的(不现实) +3. 走 §1.5.2 矩阵的另一格(payload musl),放弃跨平台换取立即可用 + +⭐ **让这三个选择可见,是 6.10.2 ⑤ 那一条(链接错误)零成本做到的** —— +`tinyhttps` 会在链接期被指名,而不是跑到第 27 个测试。 + +**他能贡献的两样,他自己已经提过:** +- ⭐ **94 个 POSIX 用例** —— 正是 6.10.2 ⑧ 缺的那一层的现成种子 +- 双架构 musl libc++ 制品 —— 若走 §1.5.2 那一格,这是缺的那一块 + +--- + +## 7. 定案用的四条命令 + +三条待测项,每条给出**能给出否定读数**的形态。 + +**① §2.4 的 fast-path 重放假说**(未测): + +```sh +# 同一个工程，两个 home，只看它是否重新 prepare +MCPP_HOME=$A mcpp build --target x86_64-linux-musl +grep -c "$A" target/*/*/build.ninja # 期望 >0 +MCPP_HOME=$B MCPP_VERBOSE=1 mcpp build --target x86_64-linux-musl 2>&1 | grep "scanning module sources" +grep -c "$A" target/*/*/build.ninja # 若仍 >0 且上一行为空 ⇒ 重放了 $A 的图 +``` + +⚠️ 判据是**两条同时成立**:没有 `scanning module sources`(说明没重新 prepare) +**且** `build.ninja` 里仍是 `$A` 的路径。只看其中一条,两种不同的机制会同读数。 + +**② §3.2 方向 2**(dep 缓存跨目标复用): + +```sh +mcpp build --target x86_64-linux-musl --cache global +mcpp build --cache global +ls -d ~/.mcpp/build-cache/v1/pkg//@/*/ | wc -l # 期望 2；若为 1 ⇒ 报告成立 +python3 -c "import json,glob;[print(f, json.load(open(f))['inputs']['toolchain']['target_triple']) for f in glob.glob('...entry.json')]" +``` + +⚠️ 判据是**条目目录数 + 每个条目里的 `target_triple`**,不是 CLI 的 `Cached` 行。 + +**③ ⓪ NULL 跳转**: + +```sh +gdb --args ./prog +(gdb) run +(gdb) info registers rip rsp +(gdb) x/8gx $rsp # 返回地址在这里，bt 空的时候它不空 +(gdb) info symbol <上面读到的地址> +readelf -x .init_array ./prog | grep -c ' 00000000 00000000' +``` + +**④ A-2 的恒等式**(应当加进 `e2e 295`): + +```sh +diff <(tr ' ' '\n' /dev/null || true "$MCPP" build --dev - BUILT=$(find target -type f \( -name 'mcpp' -o -name 'mcpp.exe' \) | head -1) - [ -n "$BUILT" ] || { echo "::error::mcpp did not build"; exit 1; } + # ⚠️ EXACTLY ONE, not "at least one". Two would mean the delete above + # missed a tree and the ambiguity this step exists to remove is back. + # + # ⚠️ NO `mapfile`. This job also runs on macos, whose `/bin/bash` is + # 3.2 and has no such builtin — a bashism here would fail on one row + # of the matrix for a reason unrelated to what the row tests. + find target -type f \( -name 'mcpp' -o -name 'mcpp.exe' \) > /tmp/mcpp-built.txt + COUNT=$(wc -l < /tmp/mcpp-built.txt | tr -d ' ') + if [ "$COUNT" -ne 1 ]; then + echo "::error::expected exactly one built mcpp, found $COUNT" + sed 's/^/ /' /tmp/mcpp-built.txt + exit 1 + fi + BUILT=$(cat /tmp/mcpp-built.txt) BUILT=$(cd "$(dirname "$BUILT")" && pwd)/$(basename "$BUILT") echo "MCPP_UNDER_TEST=$BUILT" >> "$GITHUB_ENV" "$BUILT" --version diff --git a/CHANGELOG.md b/CHANGELOG.md index fed795b8..4819456e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,100 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [2026.8.27.1] — 2026-08-27 + +目标侧被解析出来了,只发给了一个编译单元。完整分析见 +[`.agents/docs/2026-08-27-openkal-native-path-three-issues.md`](.agents/docs/2026-08-27-openkal-native-path-three-issues.md)。 + +⭐ **与 `2026.8.25.x`/`2026.8.26.1` 是同一族的下一层。** 那两批修的是「谓词问错了」 +与「答案没接到决定上」;这一批里,答案**接上了一个消费者,而它有五个**。 + +### 修复 + +- **⭐⭐ 编译侧的谓词,是 `2026.8.26.1` 在链接侧修掉的那条的孪生兄弟。** + + `hostflags.cppm` 问的是 `!crossTargetFlag.empty()` ——「命令行上有没有 + `--target=`」——而它的注释写的是「目标侧来自图」。同一台机器、同一个编译器、 + 同一个目标,只差写不写 `--target`,编译线少了**六个 token**: + + ``` + --no-default-config -nostdinc++ + -isystem /include/c++/v1 + -isystem /include//c++/v1 + -isystem /include + -isystem /include + ``` + + ⇒ 头文件来自一个库,目标文件链自另一个库。两侧现在读同一个 + `plan.targetSide.cAbi.prebuilt()`。⚠️ `--no-default-config` 从这个条件里 + **拆了出来无条件发** —— 它不是载荷头文件集合的一部分,而 cfg 文件按 + `post_install.cppm` 自己的说法是「per-machine, per-install-path artifact」。 + + ⚠️ `e2e 295` 写的就是这条恒等式,而它只比对 `^ldflags`,所以恒等式在**下一行** + 不成立而测试看不见。现在两条都比。 + +- **⭐⭐ 载荷目录名是 LLVM 词汇,而查找用的是 mcpp 词汇 —— 而且失配是静默的。** + + `include//c++/v1` 与 `lib/` 由 LLVM 的构建写下,带的是 + `x86_64-unknown-linux-gnu`;`tc.targetTriple` 是 mcpp 的 + `x86_64-linux-gnu`。两者只在三元组是**探测来的**时候恰好相同。查找是 + `if (exists) push_back`,所以找不到就什么也不发生 —— 而那个目录里只有一个文件, + `__config_site`,它的缺席产生的报错读起来像载荷坏了。两种拼法现在都试。 + +- **⭐⭐ 由图供给的目标侧,只到达了一个编译单元(mcpp#514 §A)。** + + 提供 `mcpp:` 层的包发布的是**整个目标**编译时所依据的头文件集合,而它今天以 + `publicUsage` 的形态**沿依赖边**传播。于是根与 provider 自己的单元拿得到,而 + **兄弟依赖包**拿不到 —— `nlohmann.json` 不在 `openkal-llvm-runtime` 的下游, + 它在它旁边。结果是一次构建里两种口味的 BMI,任何同时导入两者的 TU 在第一个 + 模板实例化处炸掉(`reference to 'space' is ambiguous`)。 + + ⭐ 目标侧解析之后,`fromGraph()` 的层的 `publicUsage` 并入**每一个**包的 + `privateBuild`;`std` 模块的命令行也改读同一个集合,不再自己推一遍。 + +- **⭐⭐ 缓存键描述了编译器,没有描述它被指向的头文件集合(mcpp#514 §B)。** + + A 轴上的每一项都在描述**编译器**,没有一项描述它编译时所依据的**库** —— + 而两者是分开安装的。`driverIdentity` 按设计也覆盖不了它: + `normalize_driver_output` **故意**抹掉路径,好让一个条目能被两个 home 共享。 + 新增 `targetHeaderSet` 轴,取自已经解析好的 `linkmodel`,并**分两档相对化**: + `` 与 ``。⚠️ 只做 `` 一档不够 —— 最常见的那台机器走 + `CLibMode::Sysroot`,它唯一的编译 token 是 `--sysroot=/registry/subos/default`, + 在 HOME 底下而不在 store 底下,于是每个条目都会带上这台机器的 home。分得开什么: + 两个载荷(路径里带版本号)⇒ 两个键;一个 home 下的两个 subos ⇒ 两个键; + 两个 home 下同名的 subos ⇒ 仍是一个键(由整工程指纹的第 11 项区分)。 + ⚠️ 不 bump `kCacheEpoch` —— 旧条目是 miss 而不是不可用。 + +- **⭐⭐ 请求的版本和载荷目录的版本是两套词汇,而每个查找都按请求那套拼。** + + RuntimeBinding 带的是**声明的**版本(`glibc@2.44`),而 xlings 把载荷目录按这个 + 请求**解析成**的版本命名(`2.44.2`)。索引在同一序列内挪动一次包,所有按声明版本 + 拼目录名的查找就同时失效 —— 在每一台**新**机器上,在任何已存在的机器上都不出现。 + + 两个查找点各自拼过一遍,所以修好一个还剩另一个,而第二处的失败**根本不提版本**: + glibc 的 include 目录只是没被加上,用户读到的是 libstdc++ 头文件里的 + `features.h: No such file`。⇒ 收敛到一个 `payload_dir_for_version`。 + + ⭐ **判据是「精化」,不是「按目录序挑一个」**:`2.44.2` 的版本**分量**以请求的分量 + 开头。`2.4` 回答不了 `2.44`(逐分量比,不是逐字符)。两个载荷都精化同一个请求时 + 返回**空** —— 「这个请求的解析结果」得是唯一一个才配叫答案,而按目录序挑正是这里 + 每个调用者都拒绝做的猜测。反方向(拿更旧的载荷回答更新的请求)不接受。 + +- **⭐ home 发现有第四份拷贝,而且会伸到别的 home 里去。** + + `active_home_xpkgs()` 自己重推了一遍 home(漏掉自包含安装那一档); + `find_sibling_package` 找不到时**无条件回落** `~/.xlings/data/xpkgs`。 + 后者意味着一次密闭构建可以从**另一棵树**取载荷,而结果直接进每条编译命令的 + `-isystem`。前者改为 `mcpp::home::root()`,后者删除 —— 找不到会说话,找错了不会。 + +### 新增 + +- **`[build] private_include_dirs`** —— 指出 `include_dirs` 中在本包边界处停住的 + 条目。`publicUsage` 此前整份接过 `privateBuild` 的目录,于是一个内嵌了带内部头 + 覆盖层的库(musl 的 `src/include` 定义 `hidden`/`weak`/`weak_alias`)会把那些宏 + 发给每一个消费者。⚠️ 它是 `include_dirs` 的**子集**而不是第二个列表:两类目录的 + 相对顺序是承重的,而两个 TOML 数组表达不了一个顺序。 + ## [2026.8.26.2] — 2026-08-26 已经解析出的答案,没有被用来做决定。完整分析见 diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index c4997c7c..ff02455a 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -176,6 +176,7 @@ module_extensions = [".ixx"] # Extra extensions used by module INTERFACES ( build_program_timeout = 1800 # Seconds a build.mcpp may run; 0 = no limit (§ below) include_dirs = ["include", "third_party/include"] # Header search paths include_dirs_after = ["*"] # Header dirs searched AFTER system dirs (-idirafter) +private_include_dirs = ["vendor/src/include"] # Of `include_dirs`, the ones a consumer must NOT get c_standard = "c11" # Standard for C source files (default c11) cflags = ["-DFOO=1"] # Extra C compile flags cxxflags = ["-DBAR=2"] # Extra C++ compile flags (do not put -std=... here) @@ -190,6 +191,44 @@ jobs = "auto" # Concurrent compiles: a positive number, or " bmi_schedule = "auto" # Module-edge scheduling: auto (= off) | on | off (§ below) ``` +`private_include_dirs` names the entries **of `include_dirs`** that stop at this +package's own boundary: this package compiles with them, and a consumer never +receives them. + +Almost every package publishes exactly the set it is built from, which is why +`include_dirs` alone was enough for a long time. The shape where the two differ +is a package that vendors a library with an *internal header overlay*. musl +reaches its own declarations through `src/include`, whose headers define +`hidden`, `weak` and `weak_alias` — names that mean something only to musl's own +sources. Publishing that directory hands those macros to every consumer, and a +consumer that uses `hidden` as an ordinary identifier stops compiling for a +reason it has no way to see. + +```toml +[build] +# ⚠️ The relative ORDER of the two kinds is load-bearing: the internal overlay +# must precede the public headers for this package's own build. That is why +# this is a SUBSET of `include_dirs` rather than a second list — two arrays +# cannot express one order. +include_dirs = ["port/include", "musl/src/include", "musl/include"] +private_include_dirs = ["musl/src/include"] +``` + +Entries take the same `*` glob convention as `include_dirs`, and are matched +after expansion — so a glob may name exactly the directories it expands to. An +entry that is not among this package's `include_dirs` withholds nothing and is +reported as such rather than passing in silence. + +**On an older engine the key is ignored, never fatal.** Measured on 2026.8.26.2: +in a dependency's manifest it is accepted silently, and in a root manifest it +warns — `[build] has unsupported key 'private_include_dirs' (ignored)` — and the +build continues. So a package may adopt the key without waiting for its +consumers to upgrade; those on an older engine simply keep receiving the +directory as they did before. The one place this does **not** hold is a +published `xim` descriptor's `target_cfg` block, where an unrecognised sub-key +is a hard error that fails the whole manifest — do not put this key there until +the index floor names an engine that knows it. + `include_dirs_after` (#249) lists header directories that are searched **after** the toolchain's system directories (emitted as `-idirafter` on GCC/Clang, as trailing `/I` under the MSVC dialect, and as plain `-I` for NASM diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index e6b3af77..cbf38566 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -166,6 +166,7 @@ module_extensions = [".ixx"] # 模块**接口**额外使用的扩展名(见 build_program_timeout = 1800 # build.mcpp 的运行上限(秒);0 = 不限(见下节) include_dirs = ["include", "third_party/include"] # 头文件搜索路径 include_dirs_after = ["*"] # 排在系统目录之后搜索的头文件目录(-idirafter) +private_include_dirs = ["vendor/src/include"] # `include_dirs` 中不发布给消费者的那些 c_standard = "c11" # C 源文件的标准(默认 c11) cflags = ["-DFOO=1"] # 额外 C 编译参数 cxxflags = ["-DBAR=2"] # 额外 C++ 编译参数(不要放 -std=...) @@ -178,6 +179,35 @@ jobs = "auto" # 并发编译数:正整数,或 "auto"(见下 bmi_schedule = "auto" # 模块边调度:auto(= 关)| on | off(见下节) ``` +`private_include_dirs` 指出 **`include_dirs` 中**在本包边界处停住的那些条目: +本包用它们编译,消费者永远收不到。 + +绝大多数包发布的就是它编译时用的那一套,所以长期以来只有 `include_dirs` 就够了。 +两者不同的形状只有一种 —— 一个包**内嵌了带内部头覆盖层的库**。musl 通过 +`src/include` 到达它自己的声明,而那些头定义了 `hidden`、`weak`、`weak_alias`, +这些名字只对 musl 自己的源码有意义。把那个目录发布出去,等于把这些宏交给每一个 +消费者;而一个把 `hidden` 当普通标识符用的消费者会编不过,且看不出原因。 + +```toml +[build] +# ⚠️ 两类目录的**相对顺序**是承重的:本包自己构建时,内部覆盖层必须排在公共头之前。 +# 这正是它被设计成 `include_dirs` 的**子集**而不是第二个列表的原因 —— +# 两个数组表达不了一个顺序。 +include_dirs = ["port/include", "musl/src/include", "musl/include"] +private_include_dirs = ["musl/src/include"] +``` + +条目支持与 `include_dirs` 相同的 `*` glob 约定,并在**展开之后**比对 —— +所以一个 glob 可以恰好指名它展开出的那些目录。若某条目不在本包的 `include_dirs` +里,它什么也没扣下,mcpp 会把这件事说出来而不是让它悄悄通过。 + +**旧引擎会忽略这个键,而不会因此失败。** 在 2026.8.26.2 上实测:出现在依赖的清单里 +时被静默接受;出现在根清单里时给一条警告 —— `[build] has unsupported key +'private_include_dirs' (ignored)` —— 构建照常继续。所以一个包可以先用上这个键, +不必等消费者升级;还在旧引擎上的消费者只是像以前一样继续收到那个目录。**唯一不成立 +的地方**是已发布的 `xim` 描述符的 `target_cfg` 块:那里不认识的子键是硬错误,会让 +整份清单加载失败 —— 在索引下限指向认识它的引擎之前,不要把这个键写进那里。 + `include_dirs_after`(#249)列出**排在工具链系统目录之后**搜索的头文件目录 (GCC/Clang 发射为 `-idirafter`;MSVC 方言退化为排在末尾的 `/I`,NASM 汇编 单元退化为普通 `-I`——两者都没有对应 flag,也都没有需要保护的系统头搜索链)。当目录是解压后的源码 tarball 根目录、且其中的文件名会与标准头冲突时, diff --git a/mcpp.toml b/mcpp.toml index 298dbc79..a592ecf8 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.8.26.2" +version = "2026.8.27.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index eb3e4546..653c217e 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -291,6 +291,15 @@ std::vector host_base_flags(const mcpp::toolchain::Toolchain& tc, // undefined __cxa_* / __gxx_personality_v0. opt.cfgBypass = mcpp::toolchain::HostFlagOptions::CfgBypass::LinuxOnly; opt.clangStdlibSelect = true; + // ⭐ `cAbiPrebuilt` is left at its default (true), and that is a statement + // rather than an omission: `tc` here is always HOST-targeting (see this + // function's header), so the helper's C library is the payload's whatever + // the project's target side turns out to be. + // + // ⚠️ It also corrects a latent defect. The predicate this replaced was + // `!tc.crossTargetFlag.empty()`, and a host toolchain resolved for a cross + // build could carry one — in which case the helper lost the payload's own + // headers for a reason that had nothing to do with it. // binutils -B so the driver finds ld/as (GCC; musl and MinGW ship their own). opt.binutilsPrefix = !mcpp::toolchain::is_musl_target(tc) && !mcpp::toolchain::is_mingw_target(tc); diff --git a/src/build/cache_key.cppm b/src/build/cache_key.cppm index 8de9119f..e9b0ea32 100644 --- a/src/build/cache_key.cppm +++ b/src/build/cache_key.cppm @@ -56,6 +56,7 @@ import mcpp.manifest; import mcpp.modgraph.scanner; import mcpp.toolchain.detect; import mcpp.toolchain.fingerprint; +import mcpp.toolchain.linkmodel; import mcpp.toolchain.triple; import mcpp.freestanding.target; // the flags a bare-metal triple implies @@ -97,6 +98,36 @@ struct BuildAxes { // Measured on exactly that upgrade. Empty for hosted targets, so nothing // else's key moves. std::vector targetImpliedFlags; + // ⭐⭐ THE HEADER SET THE DRIVER IS POINTED AT — not the driver. + // + // Everything else on this axis describes the COMPILER. Nothing described + // the LIBRARY it compiles against, and the two are separately installed: + // one clang payload sits above whichever `xim:glibc` and + // `xim:linux-headers` the home happens to carry. + // + // ⚠️ AND `driverIdentity` CANNOT COVER IT, BY DESIGN. + // `normalize_driver_output` deliberately strips `/home/`, `/tmp/` and + // `/var/` paths out of `clang --version`, which is what lets one entry be + // shared between two homes. Correct — and it means two homes carrying the + // same clang above DIFFERENT glibc payloads produce the same + // `driver_identity`, the same key, and one entry serving two header sets. + // + // Measured (mcpp#514 §B, third direction): two host builds resolving glibc + // 2.39 and 2.44 hit one entry, and the mixed BMIs then crash the clang + // frontend outright — SIGSEGV in + // `ASTReader::FindExternalVisibleDeclsByName` during deserialization — + // rather than producing a readable diagnostic. BMIs carry no cross-check + // against each other. + // + // ⭐ THE `std` CACHE ON THE SAME MACHINE HAS ALWAYS HAD THIS. Its identity + // folds in `std_build_commands`, the whole command line, `-isystem` rows + // included. Two caches, one machine, two notions of "the same inputs"; the + // dependency one was the short one. + // + // Store-relative, for the same reason the include dirs below are: the key + // must survive a different MCPP_HOME, and what matters is WHICH payloads, + // not where the home is. + std::vector targetHeaderSet; std::string stdlibId; std::string stdlibVersion; // B @@ -163,7 +194,8 @@ BuildAxes build_axes(const mcpp::toolchain::Toolchain& tc, const mcpp::manifest::Manifest& rootManifest, std::string_view cppStandardFlag, const std::vector& dialectFlags, - std::string_view macosDeploymentTarget); + std::string_view macosDeploymentTarget, + const std::filesystem::path& storeRoot = {}); // Axes E from one PackageRoot. `storeRoot` is stripped off absolute include // dirs so the key survives a different MCPP_HOME (the payload paths are @@ -217,6 +249,7 @@ nlohmann::json to_json(const BuildAxes& b, const PackageAxes& p) { {"driver_identity", b.driverIdentity}, {"target_triple", b.targetTriple}, {"target_implied_flags", b.targetImpliedFlags}, + {"target_header_set", b.targetHeaderSet}, {"stdlib", b.stdlibId}, {"stdlib_version", b.stdlibVersion}, }; @@ -264,6 +297,7 @@ std::string key_hex(const BuildAxes& b, const PackageAxes& p) { put(s, "driver", b.driverIdentity); put(s, "triple", b.targetTriple); put_list(s, "targetflags", b.targetImpliedFlags); + put_list(s, "hdrset", b.targetHeaderSet); put(s, "stdlib", b.stdlibId); put(s, "stdlibv", b.stdlibVersion); // B @@ -302,7 +336,8 @@ BuildAxes build_axes(const mcpp::toolchain::Toolchain& tc, const mcpp::manifest::Manifest& rootManifest, std::string_view cppStandardFlag, const std::vector& dialectFlags, - std::string_view macosDeploymentTarget) + std::string_view macosDeploymentTarget, + const std::filesystem::path& storeRoot) { BuildAxes b; b.compilerId = std::string(tc.compiler_name()); @@ -349,6 +384,67 @@ BuildAxes build_axes(const mcpp::toolchain::Toolchain& tc, b.stdlibId = tc.stdlibId; b.stdlibVersion = tc.stdlibVersion; + // ⭐ THE HEADER SET, TAKEN FROM THE RESOLVERS THAT PRODUCE THE COMMAND + // LINE — not derived a second time. + // + // `mcpp.toolchain.linkmodel` is already the single answer to "where does + // this toolchain's C library and libc++ come from"; every consumer of a + // compile line reads it. Asking it here means the key describes the header + // set that will actually be used, and cannot drift from it. + // + // ⚠️ RELATIVE, IN TWO TIERS — and `` ALONE IS NOT ENOUGH. + // + // The absolute form names this machine's home, and the whole point of + // `normalize_driver_output` stripping paths is that one entry can serve + // two homes carrying the same payloads. Leaving these absolute undoes it. + // + // ⚠️ MEASURED, AND `` MISSED THE COMMON CASE. This developer's own + // toolchain resolves `CLibMode::Sysroot`, whose only compile token is + // + // --sysroot=/home//.mcpp/registry/subos/default + // + // — under the HOME and not under ``, so a store-only rule left the + // home in the key and every entry stopped being shareable. `` is the + // second tier for exactly this, mirroring the ``/`` pair + // `fill_package_config` already uses for the same reason. + // + // ⭐ WHAT THIS AXIS DOES AND DOES NOT SEPARATE, stated so the next reader + // does not have to re-derive it: + // * payload-supplied headers — `/xim-x-glibc/2.44/include` — carry + // the VERSION in the path, so two payloads are two keys. This is the + // measured mcpp#514 §B case and the one this axis exists for. + // * two subos on one home — `/subos/default` vs `/subos/foo` + // — are two keys, which they were not before this axis existed. + // * the SAME subos name in two homes stays one key even if the two hold + // different C libraries. Unchanged from before, and the same trade + // `normalize_driver_output` already makes; the whole-project + // fingerprint's field 11 is what separates those. + { + const auto storeStr = storeRoot.generic_string(); + // `/registry/data/xpkgs` → `/registry`. Derived rather than + // passed because the two must be the same home by construction: a + // second parameter could be given a different one, and then the tiers + // would describe two machines. + const auto homeStr = storeRoot.empty() + ? std::string{} + : storeRoot.parent_path().parent_path().generic_string(); + const mcpp::toolchain::PathEscape relativize = + [&](const std::filesystem::path& p) -> std::string { + auto str = p.generic_string(); + if (!storeStr.empty() && str.starts_with(storeStr)) + return "" + str.substr(storeStr.size()); + if (!homeStr.empty() && str.starts_with(homeStr)) + return "" + str.substr(homeStr.size()); + return str; + }; + for (auto& t : mcpp::toolchain::resolve_clang_driver(tc) + .compile_tokens(relativize)) + b.targetHeaderSet.push_back(t); + for (auto& t : mcpp::toolchain::resolve_link_model(tc) + .compile_tokens(relativize)) + b.targetHeaderSet.push_back(t); + } + b.cppStandard = rootManifest.package.standard; b.cppStandardFlag = std::string(cppStandardFlag); b.dialectFlags = dialectFlags; diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 53921d38..f0651a98 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -593,6 +593,11 @@ CompileFlags compute_flags(const BuildPlan& plan) { mcpp::toolchain::HostFlagOptions hopt; hopt.cfgBypass = mcpp::toolchain::HostFlagOptions::CfgBypass::Always; hopt.macosDeploymentTarget = macosDeploymentTarget; + // ⭐ THE SAME EXPRESSION THE LINK SIDE ASKS, twenty lines further down + // (`plan.targetSide.cAbi.prebuilt()`). Reading one value at both sites + // is what makes it impossible for them to disagree — which they did, + // from #511 until now, because only the link side was corrected. + hopt.cAbiPrebuilt = plan.targetSide.cAbi.prebuilt(); compile_toolchain_flags = mcpp::toolchain::render_tokens( mcpp::toolchain::host_compile_tokens(plan.toolchain, hopt, ninjaEsc)); } else { diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index b7e60d9c..7c385064 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -2781,9 +2781,44 @@ prepare_build(bool print_fingerprint, std::optional> hostTcCache; auto host_tc_for_build_program = [&]() -> std::expected< std::pair, std::string> { + // ⭐⭐ A HOST TOOLCHAIN'S C LIBRARY IS THE PAYLOAD'S, WHATEVER THE + // PROJECT'S TARGET SIDE IS. + // + // `build.mcpp` is compiled AND RUN on the machine doing the build. Its + // C library therefore comes from the compiler payload — even for a + // project whose TARGET takes its C library from the dependency graph. + // The two are different machines and this function's whole job is to + // keep them apart. + // + // ⚠️ AND THE NATIVE BRANCH BELOW RETURNS THE MAIN `tc`, WHICH CARRIES + // THE OTHER ANSWER. `build_program.cppm`'s own header states the + // invariant — "`tc` is always a HOST-targeting toolchain" — and for + // every field but this one the native branch satisfied it, because on a + // native build the compiler IS the host compiler. `cAbiPrebuilt` is the + // first field where "same compiler" and "same target side" come apart. + // + // ⚠️ AN INVARIANT, NOT A BUG FIX FOR ANY MEASURED FAILURE. It was + // written while chasing a `features.h: No such file` on openkal-musl's + // CI and it is NOT that failure's cause: measured on `origin/main` and + // on this branch, the gcc std module carries zero `-isystem`/ + // `-idirafter` rows either way — that toolchain reaches its C library + // through the specs the post-install fixup rewrites, and the real + // defect was in resolving WHICH glibc payload those specs name. + // + // Kept because the invariant is worth being true: a helper compiled and + // run on the build machine must not inherit the target's C-library + // origin, and the next field that comes apart would find no rule here. + // + // ⇒ Stated once, so every consumer (the std module build, + // `host_base_flags`) gets it without asking. + auto as_host = [](mcpp::toolchain::Toolchain t) { + t.cAbiPrebuilt = true; + return t; + }; if (overrides.target_triple.empty()) - return std::pair{explicit_compiler, *tc}; - if (hostTcCache) return *hostTcCache; + return std::pair{explicit_compiler, as_host(*tc)}; + if (hostTcCache) + return std::pair{hostTcCache->first, as_host(hostTcCache->second)}; if (!tcSpec || *tcSpec == "system" || tcSpecIsMsvc) { // ⭐ A READABLE REFUSAL THAT HAD NO CODE, so the target matrix // recorded four identical `other` cells for it. The sentence was @@ -2828,7 +2863,7 @@ prepare_build(bool print_fingerprint, mcpp::ui::info("Resolved", std::format( "host toolchain for build.mcpp: {}", htc->label())); hostTcCache = std::pair{frontend, *htc}; - return *hostTcCache; + return std::pair{hostTcCache->first, as_host(hostTcCache->second)}; }; // Resolve dependencies: walk the **transitive** graph from the main @@ -4146,6 +4181,28 @@ prepare_build(bool print_fingerprint, return dirs; }; + // The same expansion for `private_include_dirs`, so a private entry may be + // a glob and still name exactly the directories it expands to. + auto expandPrivateIncludeDirs = + [&](const std::filesystem::path& packageRoot, + const mcpp::manifest::Manifest& manifest) + { + std::vector dirs; + for (auto const& inc : manifest.buildConfig.privateIncludeDirs) { + if (inc.is_absolute()) { + auto n = inc; + n.make_preferred(); + appendUniquePath(dirs, std::move(n)); + continue; + } + for (auto& dir : mcpp::modgraph::expand_dir_glob( + packageRoot, inc.generic_string())) { + appendUniquePath(dirs, dir); + } + } + return dirs; + }; + auto makePackageRoot = [&](const std::filesystem::path& packageRoot, const mcpp::manifest::Manifest& manifest) @@ -4159,7 +4216,53 @@ prepare_build(bool print_fingerprint, pkg.privateBuild.includeDirsAfter = expandIncludeDirsAfter(packageRoot, manifest); pkg.privateBuild.cflags = manifest.buildConfig.cflags; pkg.privateBuild.cxxflags = manifest.buildConfig.cxxflags; - pkg.publicUsage.includeDirs = pkg.privateBuild.includeDirs; + // ⭐⭐ NOT `= privateBuild` ANY MORE — a package may now say which of + // its include directories stop at its own boundary. + // + // This line took the whole set for as long as the two were the same + // set, which they are for almost every package. The one shape where + // they are not is a package that vendors a library with an internal + // header overlay: musl's `src/include` adds `hidden`, `weak` and + // `weak_alias` for musl's own sources, and publishing it hands those + // names to every consumer. See BuildInputs::privateIncludeDirs. + // + // ⚠️ THE FILTER IS APPLIED AFTER GLOB EXPANSION, so a private entry may + // itself be a glob and still name exactly the directories it expands + // to. Comparing the unexpanded spellings would let `musl/src/*` be + // published because it is not literally equal to `musl/src/include`. + { + const auto privateExpanded = + expandPrivateIncludeDirs(packageRoot, manifest); + for (auto const& d : pkg.privateBuild.includeDirs) + if (std::ranges::find(privateExpanded, d) == privateExpanded.end()) + pkg.publicUsage.includeDirs.push_back(d); + + // ⚠️ AN ENTRY THAT WITHHOLDS NOTHING IS REPORTED, because the way + // it fails is the very defect this key exists to prevent: a + // directory the author believes is private stays published, and + // nothing about the build looks different until a consumer trips + // over a name months later. + // + // A WARNING AND NOT AN ERROR, for consistency with `include_dirs` + // itself: that key silently ignores a glob matching nothing, and a + // conditional manifest can legitimately name a directory that + // exists on one platform only. Refusing here would be stricter + // than the list this one filters. + for (auto const& want : privateExpanded) { + if (std::ranges::find(pkg.privateBuild.includeDirs, want) + != pkg.privateBuild.includeDirs.end()) + continue; + mcpp::diag::warning("manifest", std::format( + "package '{}': `private_include_dirs` names '{}', which is " + "not among this package's `include_dirs`.\n" + " It withholds nothing — `private_include_dirs` says " + "which entries OF `include_dirs`\n" + " stop at this package's boundary, and an entry that " + "is not one of them is published\n" + " exactly as before.", + manifest.package.name, want.generic_string())); + } + } pkg.publicUsage.includeDirsAfter = pkg.privateBuild.includeDirsAfter; pkg.linkUsage.ldflags = manifest.buildConfig.ldflags; return pkg; @@ -6453,6 +6556,17 @@ prepare_build(bool print_fingerprint, mcpp::targetside::TargetSide resolvedTargetSide; + // What the packages supplying the target side's layers publish: the header + // directories and interface flags the whole build is compiled against. + // + // ⭐ ONE SET, TWO READERS, and that is deliberate: it is merged into every + // package's `privateBuild` (so every compile edge sees it) and handed to + // the `std` module's own command line (which is one more translation unit + // of the same build). Before this existed, only the second reader was + // written, and it derived the set itself — which is how the two could + // describe different worlds. + mcpp::modgraph::UsageRequirements targetSideUsage; + // ── THE TARGET SIDE, RESOLVED ONCE ─────────────────────────────────────── // // HERE AND NOT EARLIER, AND THAT IS THE WHOLE POINT. @@ -6494,7 +6608,13 @@ prepare_build(bool print_fingerprint, // keeping the first acceptable one, two suppliers resolved by graph // traversal order — an order the author neither writes nor can predict — // and the loser's `[build]` section still reached the command line. - struct Candidate { tsd::Provider p; bool direct; }; + // ⭐ `index` — WHICH PACKAGE this candidate is, not just its name. + // + // Needed once resolution is done: a layer supplied from the graph + // publishes an include set the WHOLE build must see (see + // `targetSideUsage` below), and reaching that package by name would be + // a second lookup of something already in hand. + struct Candidate { tsd::Provider p; bool direct; std::size_t index = 0; }; std::map> byLayer; std::vector requirements; @@ -6511,7 +6631,8 @@ prepare_build(bool print_fingerprint, return false; }; - for (auto const& pkg : packages) { + for (std::size_t pkgIndex = 0; pkgIndex < packages.size(); ++pkgIndex) { + auto const& pkg = packages[pkgIndex]; const auto pkgId = pkg.manifest.package.version.empty() ? pkg.manifest.package.name : std::format("{}@{}", pkg.manifest.package.name, @@ -6574,10 +6695,17 @@ prepare_build(bool print_fingerprint, auto same = std::find_if(slot.begin(), slot.end(), [&](const Candidate& c){ return c.p.name == p.name; }); if (same != slot.end()) { - if (same->p.interfaceName.empty() && !p.interfaceName.empty()) - same->p = p; + // ⚠️ `index` MOVES WITH `p` AND NOT ON ITS OWN. The two + // describe one package, and this branch is reached only + // from the same `pkgIndex` today — a package carrying both + // spellings — so they cannot differ yet. Tying them keeps + // it that way if a second package ever reaches here. + if (same->p.interfaceName.empty() && !p.interfaceName.empty()) { + same->p = p; + same->index = pkgIndex; + } } else { - slot.push_back({ p, is_direct(p.name) }); + slot.push_back({ p, is_direct(p.name), pkgIndex }); } } @@ -6711,6 +6839,114 @@ prepare_build(bool print_fingerprint, } resolvedTargetSide = tsd::resolve(in); + + // ⭐⭐ RECORDED ON THE TOOLCHAIN THE MOMENT IT IS KNOWN, because three + // producers of a compile line need it and only one of them can see + // `resolvedTargetSide`. + // + // `flags.cppm` reads `plan.targetSide` directly; the std module build + // (`mcpp.toolchain.stdmod`) and the build.mcpp host helper cannot — + // they are in the toolchain layer and take a `Toolchain`. Giving them a + // second way to derive the answer is exactly the shape this release + // exists to remove, so the answer travels on the value they already + // share. + // + // ⚠️ HERE AND NOT LATER: `ensure_built` runs at :7368 and every compile + // line is assembled after it. A std BMI built against a different C + // library than its importers is what e2e 181 catches. + if (tc) tc->cAbiPrebuilt = resolvedTargetSide.cAbi.prebuilt(); + + // ── The target side's include set is a property of the BUILD ───────── + // + // ⭐⭐ IT WAS ALREADY COMPUTED, AND IT REACHED EXACTLY ONE TRANSLATION + // UNIT. + // + // A package that supplies a target-side layer publishes the headers the + // whole target is built against — libc++'s, the C library's, the + // architecture's. Those travel today as an ordinary `publicUsage`, + // which propagates ALONG DEPENDENCY EDGES. So a workspace member that + // depends on the provider receives them and a SIBLING DEPENDENCY + // PACKAGE does not: `nlohmann.json` is not downstream of + // `openkal-llvm-runtime`, it is beside it. + // + // The result is two flavours of BMI in one build — `std` compiled over + // the target's libc++ (correct: the block at :7232 hands it exactly + // this set) and the dependency packages compiled over the payload's. + // Any unit importing both fails at the first template instantiation + // that touches a declaration present in both header sets: + // + // istream:1245: error: reference to 'space' is ambiguous + // note: candidate … xim-x-llvm/…/__locale:321 + // note: candidate … openkal-llvm-runtime/…/__locale:302 + // + // mcpp#514. Reproduced in twenty lines with no openkal at all: a path + // package declaring `provides = ["mcpp:c++-abi=libc++"]` and one + // `include_dirs` entry reaches the root and its own units, and reaches + // no sibling dependency package. + // + // ⭐ THE FIX IS THE ONE `mcpp.targetside` OPENS WITH: resolve once, + // after the graph is known, and have every consumer read that one + // value. A `publicUsage` describes what a library asks of ITS USERS; a + // target side is beneath everything. Modelling the second as the first + // is what made it edge-scoped. + // + // ⚠️ ONLY LAYERS THE GRAPH SUPPLIES. `Layer::fromGraph()` is the whole + // condition. A payload-supplied layer already reaches every unit + // through `mcpp.toolchain.hostflags`, and emitting it twice would put + // the ordering of one decision in two places. + { + std::set layerProviderIndices; + auto note_layer = [&](tsd::CapLayer which, const tsd::Layer& resolved) { + if (!resolved.fromGraph()) return; + auto it = byLayer.find(static_cast(which)); + if (it != byLayer.end() && !it->second.empty()) + layerProviderIndices.insert(it->second.front().index); + }; + note_layer(tsd::CapLayer::CompilerRuntime, resolvedTargetSide.compilerRuntime); + note_layer(tsd::CapLayer::KernelAbi, resolvedTargetSide.kernelAbi); + note_layer(tsd::CapLayer::CAbi, resolvedTargetSide.cAbi); + note_layer(tsd::CapLayer::CxxAbi, resolvedTargetSide.cxx); + + for (auto idx : layerProviderIndices) { + if (idx >= packages.size()) continue; + auto const& provider = packages[idx]; + appendUniquePaths(targetSideUsage.includeDirs, + provider.publicUsage.includeDirs); + appendUniquePaths(targetSideUsage.includeDirsAfter, + provider.publicUsage.includeDirsAfter); + appendUniqueFlags(targetSideUsage.cflags, + provider.publicUsage.cflags); + appendUniqueFlags(targetSideUsage.cxxflags, + provider.publicUsage.cxxflags); + } + + // Into `privateBuild` and NOT into `publicUsage`. + // + // It is visible to the whole graph already, so it needs no further + // propagation; and writing it into `publicUsage` would fold the + // target side into the usage requirements of any library this + // build packages — a promise about a different machine. + // + // ⚠️ APPENDED, so a package's own directories keep coming first. + // The target side only has to precede the DRIVER's own defaults, + // and those are always searched last. + if (!targetSideUsage.includeDirs.empty() + || !targetSideUsage.includeDirsAfter.empty() + || !targetSideUsage.cflags.empty() + || !targetSideUsage.cxxflags.empty()) { + for (auto& p : packages) { + appendUniquePaths(p.privateBuild.includeDirs, + targetSideUsage.includeDirs); + appendUniquePaths(p.privateBuild.includeDirsAfter, + targetSideUsage.includeDirsAfter); + appendUniqueFlags(p.privateBuild.cflags, + targetSideUsage.cflags); + appendUniqueFlags(p.privateBuild.cxxflags, + targetSideUsage.cxxflags); + } + } + } + if (auto why = tsd::check_layering(resolvedTargetSide)) { refusal::record(refusal::Code::LayerOrdering); return std::unexpected(*why); @@ -7204,9 +7440,25 @@ prepare_build(bool print_fingerprint, // The two differ, and the difference is not cosmetic --- a package's own // build path carries directories that exist for its .cpp files and that // shadow the library's headers when a module is compiled against them. - for (auto& d : pkg.publicUsage.includeDirs) + // + // ⭐⭐ AND IT IS `targetSideUsage`, NOT THIS PACKAGE'S `publicUsage`. + // + // The two are the same set whenever one package supplies every layer, + // which is the arrangement this block was written for — so reading the + // package directly was correct and stayed correct until a second + // provider appeared. `openkal-llvm-runtime` supplies the C++ runtime + // while `openkal-musl` supplies the C library, and the std module needs + // both: libc++'s own headers reach ``, which is the C + // library's. + // + // ⭐ Reading the assembled set also makes this site and every compile + // edge read ONE value. Deriving it here a second time is the shape + // #233/#240/#242/#344 each cost a release, and the same set has to + // reach both or the `std` BMI describes a different world than the + // units importing it — which is mcpp#514 exactly. + for (auto& d : targetSideUsage.includeDirs) flags += " -isystem " + mcpp::xlings::shq(d.string()); - for (auto& d : pkg.publicUsage.includeDirsAfter) + for (auto& d : targetSideUsage.includeDirsAfter) flags += " -idirafter " + mcpp::xlings::shq(d.string()); // And the definitions, for the same reason as the directories: a C // library's headers show a different library depending on which feature @@ -7214,7 +7466,7 @@ prepare_build(bool print_fingerprint, // its own translation units see. Measured: without them the module // reaches musl's and stops on `clockid_t', a name that header // declares only under the macro the package carries. - for (auto& f : pkg.publicUsage.cxxflags) + for (auto& f : targetSideUsage.cxxflags) flags += " " + mcpp::xlings::shq(f); tc->stdModuleFlags = flags; break; @@ -8067,7 +8319,11 @@ prepare_build(bool print_fingerprint, *tc, m->cppStandard.experimental, mcpp::manifest::dialect_flags(m->buildConfig)), mcpp::platform::macos::deployment_target( - m->buildConfig.macosDeploymentTarget)); + m->buildConfig.macosDeploymentTarget), + // The GLOBAL registry root — the same one `fill_package_config` + // relativizes against below, so both halves of the key describe + // payload paths the same way. + storeRoots.empty() ? std::filesystem::path{} : storeRoots.front()); // Sources belonging to each package, package-root-relative and sorted. std::vector> pkgSources(packages.size()); diff --git a/src/manifest/toml.cppm b/src/manifest/toml.cppm index 64206b97..81a8822f 100644 --- a/src/manifest/toml.cppm +++ b/src/manifest/toml.cppm @@ -350,6 +350,12 @@ std::expected parse_string(std::string_view content, if (auto v = doc->get_string_array("build.include_dirs_after")) { for (auto& s : *v) m.buildConfig.includeDirsAfter.emplace_back(s); } + // [build].private_include_dirs — of `include_dirs`, the ones a consumer + // must NOT receive. See BuildInputs::privateIncludeDirs for why it is a + // subset of that list rather than a second ordered list. + if (auto v = doc->get_string_array("build.private_include_dirs")) { + for (auto& s : *v) m.buildConfig.privateIncludeDirs.emplace_back(s); + } // [targets.*] — M5.0: now optional. If absent, defer to auto-inference (in load()). // [profile.] — bundled build settings. @@ -1241,6 +1247,7 @@ std::expected parse_string(std::string_view content, "allow_host_libs", "bmi_schedule", "build_program_timeout", "c_standard", "cache", "cflags", "cxxflags", "cxx_runtime", "default-profile", "defines", "dialect_cxxflags", "flags", "include_dirs", "include_dirs_after", + "private_include_dirs", "jobs", "ldflags", "macos_deployment_target", "module_extensions", "profile", "sources", "static_stdlib", "target", }; diff --git a/src/manifest/types.cppm b/src/manifest/types.cppm index 02809261..031d35a7 100644 --- a/src/manifest/types.cppm +++ b/src/manifest/types.cppm @@ -192,6 +192,38 @@ struct BuildInputs { std::vector includeDirs; // relative to package root // #249: emitted as -idirafter (searched after the toolchain's system dirs) std::vector includeDirsAfter; + // ⭐⭐ WHICH OF `includeDirs` A CONSUMER MUST NOT RECEIVE. + // + // `publicUsage` has always taken `privateBuild`'s include directories + // ENTIRE, so a package is built from exactly the set it publishes. For + // almost every package those are the same set. For one that vendors a + // library with an internal header overlay they are not, and the difference + // reaches every consumer. + // + // `mcpplibs/openkal-musl` states the case in its own source + // (`port/include/features.h`), having found it three times: + // + // ⓘ THIS IS THE SECOND-BEST REMEDY. The first would be for a package to + // distinguish the directories it is BUILT FROM from the directories it + // PUBLISHES. Measured 2026-08-22: mcpp cannot express it. + // + // musl's build reaches its own declarations through `src/include`, whose + // headers define `hidden`, `weak` and `weak_alias` — names that mean + // something only to musl's own sources. Publishing that directory hands + // those macros to every consumer, and which consumer breaks on which name + // was discovered one at a time: a C++ one on `restrict`, then on linkage; + // a C one (compiler-rt) on `weak`, which it writes itself. + // + // ⚠️ A SUBSET OF `includeDirs`, NOT A SECOND LIST, AND THE REASON IS ORDER. + // The relative order of the two kinds is load-bearing: moving musl's + // internal directories after the public ones makes musl's OWN build find + // the public `` first and fail with `unknown type name hidden` + // (measured, same file). Two arrays in TOML cannot express one order, so + // `includeDirs` stays the single ordered list and this one says which of + // its entries stop at the package boundary. An entry here that is not in + // `includeDirs` withholds nothing, and is reported as such rather than + // passing in silence. + std::vector privateIncludeDirs; // What the `std` module source of a package that IS a standard library // needs on its command line. // @@ -220,6 +252,9 @@ inline void append(BuildInputs& dst, const BuildInputs& src) { dst.includeDirsAfter.insert(dst.includeDirsAfter.end(), src.includeDirsAfter.begin(), src.includeDirsAfter.end()); + dst.privateIncludeDirs.insert(dst.privateIncludeDirs.end(), + src.privateIncludeDirs.begin(), + src.privateIncludeDirs.end()); dst.stdModuleFlags.insert(dst.stdModuleFlags.end(), src.stdModuleFlags.begin(), src.stdModuleFlags.end()); diff --git a/src/manifest/xpkg.cppm b/src/manifest/xpkg.cppm index ca70844b..eb526114 100644 --- a/src/manifest/xpkg.cppm +++ b/src/manifest/xpkg.cppm @@ -228,6 +228,7 @@ namespace mcpp::manifest { inline constexpr std::string_view kKnownXpkgKeys[] = { "cflags", "c_standard", "cxxflags", "defines", "deps", "features", "flags", "generated_files", "import_std", "include_dirs", "include_dirs_after", + "private_include_dirs", "language", "ldflags", "linux", "macosx", "modules", "provides", "requires", "runtime", "scan_overrides", @@ -1258,6 +1259,22 @@ synthesize_from_xpkg_lua(std::string_view luaContent, } cur.consume('}'); } + else if (key == "private_include_dirs") { + // Of `include_dirs`, the entries that stop at the package + // boundary. See BuildInputs::privateIncludeDirs. + if (!cur.consume('{')) { + return std::unexpected(ManifestError{ + "expected '{' after `private_include_dirs =`", + m.sourcePath, 0, 0}); + } + cur.skip_ws_and_comments(); + while (!cur.eof() && cur.peek() != '}') { + auto s = cur.read_string(); + if (!s.empty()) m.buildConfig.privateIncludeDirs.emplace_back(s); + cur.skip_ws_and_comments(); + } + cur.consume('}'); + } else if (key == "include_dirs_after") { // #249: header dirs searched AFTER the toolchain's system dirs // (-idirafter). Use for extracted-tarball roots that contain @@ -1387,13 +1404,15 @@ synthesize_from_xpkg_lua(std::string_view luaContent, // unknown-key policy is a separate question — #263). std::vector* pathDst = sub == "include_dirs" ? &cc.inputs.includeDirs + : sub == "private_include_dirs" ? &cc.inputs.privateIncludeDirs : sub == "include_dirs_after" ? &cc.inputs.includeDirsAfter : nullptr; if (!dst && !pathDst) { return std::unexpected(ManifestError{ std::format("unknown target_cfg key '{}' (expected " "cflags/cxxflags/ldflags/sources/defines/" - "flags/include_dirs/include_dirs_after)", sub), + "flags/include_dirs/private_include_dirs/" + "include_dirs_after)", sub), m.sourcePath, 0, 0}); } if (!cur.consume('=') || !cur.consume('{')) { diff --git a/src/platform/xlings/xlings.cppm b/src/platform/xlings/xlings.cppm index e6164a46..0f8a5e84 100644 --- a/src/platform/xlings/xlings.cppm +++ b/src/platform/xlings/xlings.cppm @@ -19,6 +19,7 @@ import mcpp.pm.index_contract; import mcpp.pm.index_snapshot; import mcpp.platform; import mcpp.log; +import mcpp.home; export namespace mcpp::xlings { @@ -169,6 +170,35 @@ namespace paths { find_home_tool(std::string_view tool, std::string_view requiredRelPath = {}); + // ⭐⭐ WHICH INSTALLED PAYLOAD DIRECTORY ANSWERS A REQUESTED VERSION. + // + // A request and a resolution are two vocabularies for one fact. A + // RuntimeBinding carries the DECLARED version (`glibc@2.44`); xlings names + // the payload directory after what that request RESOLVED to (`2.44.2`). + // They coincide until the index moves a package within a series, and then + // every lookup that spells the directory by the declared version stops + // finding it. + // + // ⚠️ STATED ONCE HERE BECAUSE IT HAS TWO CALLERS AND THEY FAILED + // SEPARATELY. `post_install`'s toolchain fixup and `probe`'s compile-side + // payload discovery each spelled it themselves; fixing one left the other, + // and the second failure did not name a version at all — it read + // + // bits/os_defines.h:39: fatal error: features.h: No such file + // + // because the glibc include directory had simply not been added. + // + // ⭐ A REFINEMENT, NOT A DIRECTORY-ORDER PICK. `2.44.2` is what the request + // `2.44` resolved to: its version COMPONENTS begin with the requested ones. + // `2.4` does not answer `2.44`, because the comparison is per component and + // not per character. And when two payloads both refine the request, this + // returns nothing — "the resolution of this request" has to be one payload + // to be an answer at all, and picking by directory order is the guess every + // caller here refuses to make. + std::optional + payload_dir_for_version(const std::filesystem::path& packageRoot, + std::string_view version); + // index data root: env.home / "data" std::filesystem::path index_data(const Env& env); @@ -834,15 +864,22 @@ find_sibling_tool(const std::filesystem::path& compilerBin, } std::optional active_home_xpkgs() { - std::filesystem::path home; - if (const char* h = std::getenv("MCPP_HOME"); h && *h) { - home = h; - } else if (const char* u = std::getenv("HOME"); u && *u) { - home = std::filesystem::path(u) / ".mcpp"; - } else { - return std::nullopt; - } - auto xpkgs = home / "registry" / "data" / "xpkgs"; + // ⭐⭐ `mcpp::home::root()`, NOT A FOURTH DERIVATION OF IT. + // + // This function used to resolve the home itself — `$MCPP_HOME`, else + // `$HOME/.mcpp` — which is two of the three answers `mcpp.home` gives. The + // one it left out is SELF-CONTAINED MODE: a release tarball or + // `xlings install mcpp` puts the binary at `/bin/mcpp` and the + // unpacked tree IS the home. On such an install this function answered + // `$HOME/.mcpp`, so payload discovery reached a DIFFERENT home than + // everything else in the process — and what payload discovery produces is + // `-isystem` rows on every compile command. + // + // ⚠️ `mcpp.home`'s own header opens with this: "Every path under the mcpp + // home must be derived from here. Before #311 this logic existed in three + // places … the copies drifted." This was the fourth copy, and it drifted + // the same way. + auto xpkgs = mcpp::home::root() / "registry" / "data" / "xpkgs"; std::error_code ec; if (!std::filesystem::exists(xpkgs, ec)) return std::nullopt; return xpkgs; @@ -932,16 +969,75 @@ find_sibling_package(const std::filesystem::path& compilerBin, if (auto found = find_package_in_xpkgs(*xpkgs, packageName, requiredRelPath)) return found; - // Also check ~/.xlings/data/xpkgs/ (xlings global home) as fallback. + // ⚠️⚠️ THE `~/.xlings` FALLBACK IS GONE, AND ITS REMOVAL IS THE POINT. + // + // It used to read: "Also check ~/.xlings/data/xpkgs/ (xlings global home) + // as fallback." That was written when one machine had one home. It means + // that a build whose `MCPP_HOME` names one tree can take a payload out of + // ANOTHER, and what this function's callers do with the result is put it on + // every compile command: + // + // probe.cppm:448 linux-headers → -isystem /…/include + // + // ⇒ a hermetic build reaching outside its own sandbox for headers, with no + // diagnostic, and with the resulting objects looking exactly like objects + // built from the payload that was actually pinned. Reported as the second + // half of mcpp#514, where a project's dependency units carried `-isystem` + // rows naming a home the build was not using. + // + // ⭐ THE DIRECTION IS THE SAFE ONE. Not finding a payload is reported — + // `probe.cppm` already has the verbose branch for it, and a glibc build + // that then fails at `` says so at the first compile. + // Finding the WRONG one says nothing at all, and this codebase has paid + // for that shape before (`e2e-inherit-toolchain-corrupts-real-payloads`, + // `dev-overlay-poisons-a-released-version`). + return std::nullopt; +} + +std::optional +payload_dir_for_version(const std::filesystem::path& packageRoot, + std::string_view version) { + // ⚠️ AN EMPTY REQUEST IS NOT A REQUEST FOR EVERYTHING. `packageRoot / ""` + // is `packageRoot` itself, which IS a directory — so without this the + // exact-match branch below would hand back the package root and every + // caller would treat that container as a payload. Both callers today + // reject an empty version before arriving here; this is the invariant + // stated where it holds rather than at each of them. + if (version.empty()) return std::nullopt; + std::error_code ec; - const char* home = std::getenv("HOME"); - if (home) { - auto xlingsXpkgs = std::filesystem::path(home) / ".xlings" / "data" / "xpkgs"; - if (xlingsXpkgs != *xpkgs && std::filesystem::exists(xlingsXpkgs, ec)) - return find_package_in_xpkgs(xlingsXpkgs, packageName, requiredRelPath); + auto exact = packageRoot / std::string(version); + if (std::filesystem::is_directory(exact, ec)) return exact; + + auto components = [](std::string_view v) { + std::vector parts; + std::size_t start = 0; + while (start <= v.size()) { + auto dot = v.find('.', start); + if (dot == std::string_view::npos) { + parts.emplace_back(v.substr(start)); + break; + } + parts.emplace_back(v.substr(start, dot - start)); + start = dot + 1; + } + return parts; + }; + const auto wanted = components(version); + + std::optional only; + std::error_code dec; + for (auto it = std::filesystem::directory_iterator(packageRoot, dec); + !dec && it != std::filesystem::directory_iterator{}; + it.increment(dec)) { + if (!it->is_directory(dec)) continue; + const auto have = components(it->path().filename().string()); + if (have.size() <= wanted.size()) continue; + if (!std::equal(wanted.begin(), wanted.end(), have.begin())) continue; + if (only) return std::nullopt; // two refinements are not an answer + only = it->path(); } - - return std::nullopt; + return only; } std::filesystem::path index_data(const Env& env) { diff --git a/src/toolchain/hostflags.cppm b/src/toolchain/hostflags.cppm index 594c90eb..fabbe241 100644 --- a/src/toolchain/hostflags.cppm +++ b/src/toolchain/hostflags.cppm @@ -76,6 +76,44 @@ struct HostFlagOptions { // Must agree across the std BMI and everything that imports it — clang // rejects a module built for a different deployment target outright. std::string macosDeploymentTarget; + + // ⭐⭐ DOES THE TARGET'S C LIBRARY COME FROM A DIRECTORY THAT EXISTED + // BEFORE DEPENDENCY RESOLUTION? — `plan.targetSide.cAbi.prebuilt()`, READ + // rather than derived. + // + // This function used to ask `!tc.crossTargetFlag.empty()`: is there a + // `--target=` on the command line. Its own comment said what it meant to + // ask — "AND NOT WHEN THE TARGET SIDE COMES FROM THE GRAPH" — and those + // are different questions. A project that names its host's own target + // while depending on nothing answers yes to the first and no to the + // second. + // + // ⚠️ THE LINK SIDE OF THIS DEFECT WAS FIXED IN 2026.8.26.1 (#511) AND THIS + // SIDE WAS NOT. Measured on 2026.8.26.2, same machine, same compiler, same + // target, differing only in whether it was spelled out: + // + // $ mcpp build ldflags: identical ✔ + // $ mcpp build --target x86_64-unknown-linux-gnu cxxflags: SIX tokens gone + // + // --no-default-config -nostdinc++ + // -isystem /include/c++/v1 + // -isystem /include//c++/v1 + // -isystem /include + // -isystem /include + // + // ⇒ headers from one library, objects linked from another. On a machine + // with system headers it compiles against /usr/include and links the + // payload — the silent ABI mix; on one without, it fails naming the + // payload. + // + // ⭐ e2e 295 states the invariant ("naming the host's own target changes + // nothing") and compared only `^ldflags`, so the identity held one line + // above the line where it did not. It now compares both. + // + // Default true = "prebuilt", which is the zero-dependency case and what + // every caller that has no graph (the std module build, the build.mcpp + // host helper) means. + bool cAbiPrebuilt = true; }; // Host-compile flags as argv tokens, in the order the string channels have @@ -196,11 +234,39 @@ std::vector host_compile_tokens(const Toolchain& tc, // // mingw's own header asked for ``, and the payload's libc++ was // still ahead of the sysroot that had just been pointed at the right place. - const bool graphSuppliesTarget = !tc.crossTargetFlag.empty(); + // + // ⭐ READ, NOT DERIVED — see HostFlagOptions::cAbiPrebuilt for the + // measurement that replaced `!tc.crossTargetFlag.empty()` here. This site + // and `flags.cppm`'s link side now ask one question of one value, so they + // cannot disagree. + const bool graphSuppliesTarget = !opt.cAbiPrebuilt; if (bypassCfg && !graphSuppliesTarget) { for (auto& t : dm.compile_tokens(esc, opt.clangStdlibSelect)) out.push_back(t); + } else if (bypassCfg) { + // ⭐⭐ THE BYPASS IS NOT PART OF THE PAYLOAD'S HEADER SET, AND IT WAS + // BEING SUPPRESSED WITH IT. + // + // The payload's `-isystem` rows describe a C library this target does + // not use, so the branch above is right to withhold them. The cfg + // bypass is a different statement: `post_install.cppm` calls that file + // "a per-machine, per-install-path artifact", and reading it makes the + // command line depend on what happened to be installed when the + // payload landed. + // + // ⚠️ Measured on 2026.8.26.2: `mcpp build --target ` + // dropped `--no-default-config`, so clang read `bin/clang++.cfg` and + // the build silently inherited that machine's install. It is also what + // made a hand-written `-clang++.cfg` a working workaround for + // mcpp#514 — a workaround that only exists because this token went + // missing. + // + // Emitted here rather than moved into `compile_tokens`: that vector's + // rendering is part of the std module's cache identity, and reordering + // it would invalidate every user's std BMI for no behavioural gain. + // Nothing that used to be emitted moves; this path emitted nothing. + out.push_back("--no-default-config"); } // Unconditional on macOS, cfg or no cfg. clang refuses to load a module diff --git a/src/toolchain/linkmodel.cppm b/src/toolchain/linkmodel.cppm index 95ccde5b..d3eb7f29 100644 --- a/src/toolchain/linkmodel.cppm +++ b/src/toolchain/linkmodel.cppm @@ -20,6 +20,7 @@ export module mcpp.toolchain.linkmodel; import std; import mcpp.platform; import mcpp.toolchain.model; +import mcpp.toolchain.triple; export namespace mcpp::toolchain { @@ -304,13 +305,57 @@ ClangDriverModel resolve_clang_driver(const Toolchain& tc) { dm.llvmRoot = tc.binaryPath.parent_path().parent_path(); auto libcxxInclude = dm.llvmRoot / "include" / "c++" / "v1"; dm.cxxIncludes.push_back(libcxxInclude); + + // ⭐⭐ THE PAYLOAD'S DIRECTORIES ARE NAMED IN LLVM'S VOCABULARY, AND THIS + // LOOKUP USED mcpp'S. + // + // `include//c++/v1` and `lib/` are written by the LLVM + // build, so they carry LLVM's spelling — `x86_64-unknown-linux-gnu`. + // `tc.targetTriple` is mcpp's — `x86_64-linux-gnu`. The two coincide only + // when the triple was PROBED (`-dumpmachine` answers in LLVM's form), and + // they come apart the moment a target is named: prepare rewrites + // `targetTriple` to mcpp's canonical spelling for a retargetable driver. + // + // ⚠️ AND THE MISS IS SILENT. Both lookups are `if (exists) push_back`, so + // a directory that is not found simply does not appear. What is in the one + // that goes missing is a single file — `__config_site` — which is why the + // failure this produces reads `'__config_site' file not found` from inside + // libc++'s own `__config`, names no decision mcpp made, and has been + // mistaken for a broken payload more than once in this codebase. + // + // Measured 2026-08-27, same machine, same compiler, same target: + // + // mcpp build include/x86_64-unknown-linux-gnu/c++/v1 ✔ + // mcpp build --target x86_64-linux-gnu (absent) + // + // ⭐ BOTH SPELLINGS ARE TRIED, and that is not a heuristic: they are two + // vocabularies for one fact, and which one a given payload used is a + // property of how it was built rather than of anything mcpp decides. The + // LLVM spelling is tried first because it is the one an LLVM payload + // writes. if (!tc.targetTriple.empty()) { - auto targetInclude = dm.llvmRoot / "include" / tc.targetTriple / "c++" / "v1"; - if (std::filesystem::exists(targetInclude)) - dm.cxxIncludes.push_back(targetInclude); - auto targetLib = dm.llvmRoot / "lib" / tc.targetTriple; - if (std::filesystem::exists(targetLib)) - dm.libDirs.push_back(targetLib); + std::vector spellings; + if (auto t = mcpp::toolchain::triple::parse(tc.targetTriple)) { + auto llvmForm = t->llvm_triple(); + if (!llvmForm.empty()) spellings.push_back(std::move(llvmForm)); + } + if (std::ranges::find(spellings, tc.targetTriple) == spellings.end()) + spellings.push_back(tc.targetTriple); + + for (auto const& spelling : spellings) { + auto targetInclude = dm.llvmRoot / "include" / spelling / "c++" / "v1"; + if (std::filesystem::exists(targetInclude)) { + dm.cxxIncludes.push_back(targetInclude); + break; + } + } + for (auto const& spelling : spellings) { + auto targetLib = dm.llvmRoot / "lib" / spelling; + if (std::filesystem::exists(targetLib)) { + dm.libDirs.push_back(targetLib); + break; + } + } } return dm; } diff --git a/src/toolchain/model.cppm b/src/toolchain/model.cppm index 2111db2b..99f59248 100644 --- a/src/toolchain/model.cppm +++ b/src/toolchain/model.cppm @@ -121,6 +121,29 @@ struct Toolchain { // throw, and wrong when something can. bool targetCxxRuntime = false; + // ⭐⭐ DOES THE TARGET'S C LIBRARY COME FROM A DIRECTORY THAT EXISTED + // BEFORE DEPENDENCY RESOLUTION? — `TargetSide::cAbi.prebuilt()`, recorded + // here so the three producers of a compile line read one value. + // + // ⚠️ THIS IS NOT `targetCxxRuntime` AND THE DIFFERENCE IS THE ONE + // 2026.8.25.1 WAS ABOUT. That field says a package supplies a C++ RUNTIME; + // this one says where the C LIBRARY comes from. A pure C program over + // openkal has no C++ runtime and its C library still comes from the graph, + // and a backend that implements openkal ON TOP OF Linux takes its kernel + // interface from the graph while its C library stays the payload's. The + // two come apart in both directions. + // + // ⚠️ RECORDED, NOT DERIVED. `mcpp.targetside` answers it once, after the + // dependency graph exists; every consumer reads this. The predicate it + // replaced on the compile side was `!crossTargetFlag.empty()` — "is there a + // `--target=` on the command line" — which is true for a project that names + // its host's own target and depends on nothing. See + // `HostFlagOptions::cAbiPrebuilt` for the measurement. + // + // Defaults to true (prebuilt): a build whose graph supplies nothing, and + // every host-targeting toolchain, means exactly that. + bool cAbiPrebuilt = true; + // ⭐⭐ THE `--target=` A RETARGETABLE DRIVER HAS TO BE GIVEN, OR EMPTY. // // Non-empty only when the user asked for a cross AND the resolved compiler diff --git a/src/toolchain/post_install.cppm b/src/toolchain/post_install.cppm index 6004c87b..5b3fda0b 100644 --- a/src/toolchain/post_install.cppm +++ b/src/toolchain/post_install.cppm @@ -410,20 +410,30 @@ select_glibc_payload_lib(const std::filesystem::path& glibcRoot, runtimeId)); } - auto payload = glibcRoot / std::string(version); - std::error_code ec; - if (!std::filesystem::is_directory(payload, ec)) { + // ⭐ ONE RESOLVER, TWO CALLERS. `payload_dir_for_version` also answers + // `probe`'s compile-side discovery; see its own header for why a request + // and a resolution are two vocabularies, and why a unique component-wise + // refinement is an answer while a directory-order pick is not. + // + // ⚠️ THE FIRST VERSION OF THIS FIX SPELLED IT HERE, and that left the other + // caller: the toolchain then installed and the compile line came out + // without the glibc include directory, which reads as + // `features.h: No such file` from inside libstdc++'s own headers. + auto payload = mcpp::xlings::paths::payload_dir_for_version( + glibcRoot, version); + if (!payload) { return std::unexpected(std::format( - "selected RuntimeBinding {} requires payload '{}', but it is not " - "installed; mcpp will not fall back to another directory entry", - runtimeId, payload.string())); + "selected RuntimeBinding {} requires payload '{}', but no installed " + "payload is its resolution; mcpp will not fall back to another " + "directory entry", + runtimeId, (glibcRoot / std::string(version)).string())); } - auto lib = payload_lib_dir_with_loader(payload); + auto lib = payload_lib_dir_with_loader(*payload); if (lib.empty()) { return std::unexpected(std::format( "selected RuntimeBinding {} payload '{}' is stale/incomplete: no " "dynamic loader was found under lib64/ or lib/", - runtimeId, payload.string())); + runtimeId, payload->string())); } return lib; } diff --git a/src/toolchain/probe.cppm b/src/toolchain/probe.cppm index 3fe41a57..e2496113 100644 --- a/src/toolchain/probe.cppm +++ b/src/toolchain/probe.cppm @@ -374,16 +374,28 @@ payload_root_for_binding(const std::filesystem::path& compilerBin, const auto name = binding.substr(0, at); const auto version = std::string(binding.substr(at + 1)); - std::error_code ec; + // ⭐ THE DIRECTORY IS NAMED AFTER WHAT THE REQUEST RESOLVED TO, NOT AFTER + // THE REQUEST. `payload_dir_for_version` is the one answer to that, shared + // with the toolchain post-install fixup — see its header. + // + // ⚠️ THIS SITE'S FAILURE DOES NOT NAME A VERSION. The fixup at least says + // which payload it wanted; here the include directory is simply never + // added, and what the user reads comes from inside libstdc++: + // + // bits/os_defines.h:39: fatal error: features.h: No such file + // + // Measured 2026-08-27 on openkal-musl's CI after the fixup alone was fixed. // Compiler siblings: <...>/xpkgs/xim-x-/ if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin)) { - auto root = *xpkgs / std::format("xim-x-{}", name) / version; - if (std::filesystem::exists(root, ec)) return root; + if (auto root = mcpp::xlings::paths::payload_dir_for_version( + *xpkgs / std::format("xim-x-{}", name), version)) + return *root; } // Active home. if (auto xpkgs = mcpp::xlings::paths::active_home_xpkgs()) { - auto root = *xpkgs / std::format("xim-x-{}", name) / version; - if (std::filesystem::exists(root, ec)) return root; + if (auto root = mcpp::xlings::paths::payload_dir_for_version( + *xpkgs / std::format("xim-x-{}", name), version)) + return *root; } return std::nullopt; } diff --git a/src/toolchain/stdmod.cppm b/src/toolchain/stdmod.cppm index 9b9c85ef..c60cfd71 100644 --- a/src/toolchain/stdmod.cppm +++ b/src/toolchain/stdmod.cppm @@ -249,6 +249,12 @@ std::expected ensure_built( HostFlagOptions hopt; hopt.cfgBypass = HostFlagOptions::CfgBypass::Always; hopt.clangStdlibSelect = true; + // ⚠️ THE STD MODULE IS A TRANSLATION UNIT OF THIS BUILD, so it must see the + // same header set every unit importing it sees. Taken from the toolchain + // rather than re-derived: prepare records the resolved answer there once, + // and a std BMI built against a different C library than its importers is + // the failure e2e 181 exists for. + hopt.cAbiPrebuilt = tc.cAbiPrebuilt; std::string sysroot_flag = render_tokens(host_compile_tokens(tc, hopt, shellEsc)); diff --git a/src/version.cppm b/src/version.cppm index 46a2ab27..bc5da918 100644 --- a/src/version.cppm +++ b/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.8.26.2"; +inline constexpr std::string_view MCPP_VERSION = "2026.8.27.1"; } // namespace mcpp diff --git a/tests/e2e/295_naming_the_host_target_changes_nothing.sh b/tests/e2e/295_naming_the_host_target_changes_nothing.sh index d15bc1f2..acdcb0c6 100755 --- a/tests/e2e/295_naming_the_host_target_changes_nothing.sh +++ b/tests/e2e/295_naming_the_host_target_changes_nothing.sh @@ -59,7 +59,26 @@ implicit_target() { # toolchain spec → the triple it would use anyway | jq -r '.data.triple.toolchain // empty' | tr -d '\r' } -ldflags_of() { # extra args… → the ldflags line, or nothing +# ⚠️⚠️ TWO LINES, AND FOR A LONG TIME THIS TEST READ ONLY ONE. +# +# The identity is about THE BUILD, and a build has a compile line as well as a +# link line. `2026.8.26.1` corrected the link side; the compile side kept +# asking `!crossTarget.empty()` and kept getting it wrong, and this test could +# not see that because it compared `^ldflags` alone. +# +# Measured on 2026.8.26.2 — same machine, same compiler, same target, differing +# only in whether it was spelled out — the compile line lost SIX tokens: +# +# --no-default-config -nostdinc++ +# -isystem /include/c++/v1 +# -isystem /include//c++/v1 +# -isystem /include +# -isystem /include +# +# ⇒ headers from one library and objects linked from another, silently, on any +# machine that happens to have system headers. +line_of() { # channel, extra args… → that line of build.ninja, or nothing + local channel="$1"; shift rm -rf target "$MCPP" build "$@" >/dev/null 2>&1 || true local f; f="$(find target -name build.ninja 2>/dev/null | head -1)" @@ -69,11 +88,28 @@ ldflags_of() { # extra args… → the ldflags line, or nothing # examine, a build that produced no link line, killed the script before it # could say so. It exited 1 with no output at all. if [ -n "$f" ]; then - grep -m1 '^ldflags' "$f" || true + grep -m1 "^$channel" "$f" || true fi return 0 } +# `--target=` is expected on the explicit side and only there — it is the one +# token that names which machine. `-fprebuilt-module-path=` names the build +# directory, which differs because the fingerprint does; that is the mechanism +# working, not a difference in what is compiled. +# +# ⚠️ NOT ANCHORED WITH `^`, AND WINDOWS IS WHY. The ninja channel QUOTES a token +# whose path needs it, so the same flag arrives as +# `"-fprebuilt-module-path=C$:\Users\..."` — an anchored pattern misses it and +# the comparison then fails on the one token this filter exists to remove. +# Measured on windows-x86_64, where it turned a green invariant into a red one +# for a reason that had nothing to do with what was being compiled. +normalise() { + printf '%s\n' "$1" | tr ' ' '\n' \ + | grep -v -- '--target=' | grep -v -- '-fprebuilt-module-path=' \ + | grep -v '^$' | sort +} + fail=0 checked=0 @@ -99,8 +135,10 @@ for tc in gcc llvm; do continue fi - implicit="$(ldflags_of)" - explicit="$(ldflags_of --target "$ht")" + implicit="$(line_of ldflags)" + explicit="$(line_of ldflags --target "$ht")" + implicit_cxx="$(line_of cxxflags)" + explicit_cxx="$(line_of cxxflags --target "$ht")" if [ -z "$implicit" ] || [ -z "$explicit" ]; then # ⚠️ EARNED, NOT ASSUMED. One of the two produced no build.ninja at all, @@ -119,20 +157,32 @@ for tc in gcc llvm; do fi checked=$((checked+1)) - # `--target=` itself is expected on the explicit side and only - # there: it is the one token that names which machine, and the identity is - # about everything else. - a="$(printf '%s\n' "$implicit" | tr ' ' '\n' | grep -v '^--target=' | sort)" - b="$(printf '%s\n' "$explicit" | tr ' ' '\n' | grep -v '^--target=' | sort)" - if [ "$a" = "$b" ]; then - echo " ok $tc@$ver: naming $ht changes nothing" - else - echo "FAIL: $tc@$ver: naming $ht changed the link line" - diff <(printf '%s\n' "$a") <(printf '%s\n' "$b") \ - | grep -E '^[<>]' | head -8 | sed 's/^/ /' - fail=1 - fi + for channel in ldflags cxxflags; do + if [ "$channel" = ldflags ]; then + lhs="$implicit"; rhs="$explicit" + else + lhs="$implicit_cxx"; rhs="$explicit_cxx" + fi + # ⚠️ BOTH SIDES MUST HAVE CONTENT. Two empty strings compare equal, and + # a comparison that passes on nothing is the false green this file's + # other guard already exists for. + if [ -z "$lhs" ] || [ -z "$rhs" ]; then + echo "FAIL: $tc@$ver: no $channel line to compare" + fail=1 + continue + fi + a="$(normalise "$lhs")" + b="$(normalise "$rhs")" + if [ "$a" = "$b" ]; then + echo " ok $tc@$ver: naming $ht changes nothing ($channel)" + else + echo "FAIL: $tc@$ver: naming $ht changed the $channel line" + diff <(printf '%s\n' "$a") <(printf '%s\n' "$b") \ + | grep -E '^[<>]' | head -8 | sed 's/^/ /' + fail=1 + fi + done done if [ "$checked" = 0 ]; then diff --git a/tests/e2e/304_the_target_side_reaches_every_unit.sh b/tests/e2e/304_the_target_side_reaches_every_unit.sh new file mode 100755 index 00000000..de1b95f3 --- /dev/null +++ b/tests/e2e/304_the_target_side_reaches_every_unit.sh @@ -0,0 +1,175 @@ +#!/usr/bin/env bash +# requires: unix-shell jq +# A layer the GRAPH supplies is beneath every unit, including a sibling +# dependency package that has never heard of it. +# +# ⭐⭐ THE SET WAS ALREADY COMPUTED AND REACHED ONE TRANSLATION UNIT. +# +# A package that supplies a target-side layer publishes the headers the whole +# target is built against. Those travelled as an ordinary `publicUsage`, which +# propagates ALONG DEPENDENCY EDGES — so the root and the provider's own units +# received them and a SIBLING dependency package did not. `nlohmann.json` is +# not downstream of `openkal-llvm-runtime`; it is beside it. +# +# The result is two flavours of BMI in one build: `std` compiled over the +# target's libc++ (correct — prepare hands it exactly this set) and the +# dependency packages compiled over the payload's. Any unit importing both +# fails at the first template instantiation touching a declaration present in +# both header sets: +# +# istream:1245: error: reference to 'space' is ambiguous +# +# mcpp#514 §A. +# +# ⚠️ NO openkal, NO CROSS, NO NETWORK. A path package declaring one capability +# and one `include_dirs` entry reproduces it, which is the point: this is not a +# property of openkal but of how a target side was modelled. +# +# ⭐ THE CRITERION IS THE CDB ROW, NOT A GREP OVER THE TREE. `compile_commands.json` +# says what each unit is actually compiled with; a grep over `build.ninja` would +# also match the global rule and pass for the wrong reason. +set -e + +MCPP="${MCPP:-mcpp}" +work="$(mktemp -d)" +trap 'rm -rf "$work"' EXIT +cd "$work" + +mkdir -p src abi/src abi/abi-include abi/abi-internal dep/src dep/dep-include + +# The provider: supplies a target-side layer, publishes one directory, and +# keeps one to itself. +# +# ⭐⭐ THE TWO KEYS OF THIS RELEASE MEET IN THIS ONE PACKAGE, AND THAT IS THE +# REAL-WORLD SHAPE. `openkal-musl` declares `provides = ["mcpp:c-abi=musl"]` +# AND carries an internal header overlay. A layer's directories reach EVERY +# unit, so publishing the overlay by mistake would put musl's `hidden`/`weak` +# macros not merely on its consumers but on every translation unit of the +# build — a strictly wider blast radius than the leak `private_include_dirs` +# was written for. +# +# The two compose correctly by construction (the target side is assembled from +# the provider's `publicUsage`, which is the filtered set), and "by +# construction" is what an assertion is for. +cat > abi/mcpp.toml <<'EOF' +[package] +name = "abiprov" +version = "0.1.0" +provides = ["mcpp:c++-abi=libc++"] + +[build] +include_dirs = ["abi-internal", "abi-include"] +private_include_dirs = ["abi-internal"] +EOF +printf 'export module abiprov;\nexport int abiprov_v() { return 1; }\n' > abi/src/abiprov.cppm +printf '#pragma once\n' > abi/abi-include/marker.h +printf '#pragma once\n' > abi/abi-internal/private.h + +# The sibling: a dependency package that does NOT depend on the provider. +cat > dep/mcpp.toml <<'EOF' +[package] +name = "sibling" +version = "0.1.0" + +[build] +include_dirs = ["dep-include"] +EOF +printf 'export module sibling;\nexport int sibling_v() { return 2; }\n' > dep/src/sibling.cppm +printf '#pragma once\n' > dep/dep-include/own.h + +cat > mcpp.toml <<'EOF' +[package] +name = "reachprobe" +version = "0.1.0" + +[dependencies] +abiprov = { path = "abi" } +sibling = { path = "dep" } +EOF +cat > src/main.cpp <<'EOF' +#include +import abiprov; +import sibling; +int main() { std::printf("%d\n", abiprov_v() + sibling_v()); } +EOF + +"$MCPP" build >/dev/null 2>&1 || { + echo "FAIL: the probe project did not build" + "$MCPP" build 2>&1 | tail -20 | sed 's/^/ /' + exit 1 +} + +[ -s compile_commands.json ] || { echo "FAIL: no compile_commands.json"; exit 1; } + +# ⚠️ A DENOMINATOR. A CDB with no sibling row would make every assertion below +# vacuously true, which is the false green this criterion has to rule out. +siblings="$(jq -r '[.[] | select(.file | test("/dep/src/"))] | length' compile_commands.json)" +if [ "$siblings" -lt 1 ]; then + echo "FAIL: the CDB has no sibling-package row — nothing was checked" + exit 1 +fi + +missing="$(jq -r ' + [ .[] + | select(.file | test("/dep/src/")) + | { f: .file, has: ((.arguments // (.command | split(" "))) | map(test("abi-include")) | any) } + | select(.has | not) + | .f + ] | .[]' compile_commands.json)" + +if [ -n "$missing" ]; then + echo "FAIL: the target side did not reach $siblings sibling unit(s):" + printf '%s\n' "$missing" | sed 's/^/ /' + exit 1 +fi + +# And the reverse direction: a package that supplies NO layer must not have its +# own directory pushed onto everyone. Otherwise this test would pass for a +# build that simply gives every include dir to every unit. +leaked="$(jq -r ' + [ .[] + | select(.file | test("/abi/src/")) + | { f: .file, has: ((.arguments // (.command | split(" "))) | map(test("dep-include")) | any) } + | select(.has) + | .f + ] | .[]' compile_commands.json)" + +if [ -n "$leaked" ]; then + echo "FAIL: a non-layer package's directory reached a sibling — the fix is too wide:" + printf '%s\n' "$leaked" | sed 's/^/ /' + exit 1 +fi + +# ⭐⭐ AND THE LAYER'S *PRIVATE* DIRECTORY REACHES NOBODY BUT ITSELF. +# +# A layer reaches every unit, so publishing an internal overlay by mistake is +# wider here than anywhere else: it would land on every translation unit of the +# build rather than on the provider's consumers. +private_leak="$(jq -r ' + [ .[] + | select((.file | test("/abi/src/")) | not) + | { f: .file, has: ((.arguments // (.command | split(" "))) | map(test("abi-internal")) | any) } + | select(.has) + | .f + ] | .[]' compile_commands.json)" + +if [ -n "$private_leak" ]; then + echo "FAIL: the layer's PRIVATE directory reached units outside the package:" + printf '%s\n' "$private_leak" | sed 's/^/ /' + exit 1 +fi + +# The control: private is not the same as dropped. The provider's own units +# must still be built with it, or the assertion above passes for a build that +# simply lost the directory. +own_private="$(jq -r ' + [ .[] + | select(.file | test("/abi/src/")) + | select((.arguments // (.command | split(" "))) | map(test("abi-internal")) | any) + ] | length' compile_commands.json)" +if [ "$own_private" -lt 1 ]; then + echo "FAIL: the provider lost its own private directory — the check above proves nothing" + exit 1 +fi + +echo "OK: the graph-supplied target side reaches every unit ($siblings sibling units), its private directory reaches only its own $own_private unit(s), and nothing else spreads" diff --git a/tests/e2e/305_built_from_is_not_published.sh b/tests/e2e/305_built_from_is_not_published.sh new file mode 100755 index 00000000..427e5cbe --- /dev/null +++ b/tests/e2e/305_built_from_is_not_published.sh @@ -0,0 +1,129 @@ +#!/usr/bin/env bash +# requires: unix-shell jq +# A package can say which of its include directories stop at its own boundary. +# +# ⭐⭐ `publicUsage` TOOK `privateBuild`'s DIRECTORIES ENTIRE, so a package was +# built from exactly the set it published. For almost every package those are +# the same set; for one that vendors a library with an internal header overlay +# they are not, and the difference reaches every consumer. +# +# `mcpplibs/openkal-musl` wrote the case down in its own source, having found +# it three times (port/include/features.h): +# +# ⓘ THIS IS THE SECOND-BEST REMEDY. The first would be for a package to +# distinguish the directories it is BUILT FROM from the directories it +# PUBLISHES. Measured 2026-08-22: mcpp cannot express it. +# +# musl reaches its own declarations through `src/include`, whose headers define +# `hidden`, `weak` and `weak_alias` — names meaningful only to musl's sources. +# Publishing that directory hands those macros to every consumer; which +# consumer breaks on which name was discovered one at a time. +# +# ⭐ THE CRITERION IS THE DIRECTORY, NOT THE SYMPTOM. Asserting that `hidden` +# no longer collides would go green again the moment the package patched that +# one macro, while the leak stayed. This asserts that the directory is not on +# the consumer's command line — and, separately, that a consumer using the name +# as an ordinary identifier compiles, which is what the user actually reported. +set -e + +MCPP="${MCPP:-mcpp}" +work="$(mktemp -d)" +trap 'rm -rf "$work"' EXIT +cd "$work" + +mkdir -p src lib/src lib/pub lib/internal lib/gen/one lib/gen/two + +cat > lib/mcpp.toml <<'EOF' +[package] +name = "vendored" +version = "0.1.0" + +[build] +# ⚠️ ORDER IS LOAD-BEARING and is why `private_include_dirs` is a SUBSET of +# this list rather than a second list: the internal overlay must precede the +# public headers for the package's OWN build, and two TOML arrays cannot +# express one order. +# +# ⭐ `gen/*` IS A GLOB ON BOTH LINES. The filter is applied AFTER expansion, so +# the glob withholds exactly the directories it expands to. Comparing the +# unexpanded spellings would be the obvious implementation and would publish +# every one of them, because `gen/*` is not literally equal to `gen/one`. +include_dirs = ["internal", "gen/*", "pub"] +private_include_dirs = ["internal", "gen/*"] +EOF +printf 'export module vendored;\nexport int vendored_v() { return 7; }\n' > lib/src/vendored.cppm +printf '#pragma once\n#define VENDORED_PUBLIC 1\n' > lib/pub/pub.h +# The internal overlay: a macro that is meaningful to the package and a +# perfectly ordinary identifier to everyone else. +printf '#pragma once\n#define hidden __attribute__((visibility("hidden")))\n' > lib/internal/overlay.h +printf '#pragma once\n' > lib/gen/one/one.h +printf '#pragma once\n' > lib/gen/two/two.h + +cat > mcpp.toml <<'EOF' +[package] +name = "boundaryprobe" +version = "0.1.0" + +[dependencies] +vendored = { path = "lib" } +EOF +# `hidden` as a local variable — the exact shape openkal-musl#13 reported. +cat > src/main.cpp <<'EOF' +#include +#include +import vendored; +int main() { + int hidden = 1; + std::printf("%d %d\n", vendored_v() + hidden, VENDORED_PUBLIC); +} +EOF + +if ! "$MCPP" build >/dev/null 2>&1; then + echo "FAIL: the consumer did not build — the private overlay is still reaching it" + "$MCPP" build 2>&1 | tail -20 | sed 's/^/ /' + exit 1 +fi + +[ -s compile_commands.json ] || { echo "FAIL: no compile_commands.json"; exit 1; } + +# ⚠️ DENOMINATORS ON BOTH SIDES. With no provider row or no consumer row the +# assertions below are vacuously true. +prov="$(jq -r '[.[] | select(.file | test("/lib/src/"))] | length' compile_commands.json)" +cons="$(jq -r '[.[] | select(.file | test("/src/main"))] | length' compile_commands.json)" +if [ "$prov" -lt 1 ] || [ "$cons" -lt 1 ]; then + echo "FAIL: CDB has provider=$prov consumer=$cons rows — nothing was checked" + exit 1 +fi + +has_dir() { # file-pattern, dir-pattern → "true"/"false" + jq -r --arg f "$1" --arg d "$2" ' + [ .[] | select(.file | test($f)) + | ((.arguments // (.command | split(" "))) | map(test($d)) | any) ] + | any' compile_commands.json +} + +fail=0 +# The provider is built from BOTH — the private directory is private, not unused. +[ "$(has_dir '/lib/src/' '/lib/internal')" = true ] || { + echo "FAIL: the provider lost its own private directory"; fail=1; } +[ "$(has_dir '/lib/src/' '/lib/pub')" = true ] || { + echo "FAIL: the provider lost its own public directory"; fail=1; } +# The consumer gets the public one and NOT the private one. +[ "$(has_dir '/src/main' '/lib/pub')" = true ] || { + echo "FAIL: the public directory did not reach the consumer"; fail=1; } +[ "$(has_dir '/src/main' '/lib/internal')" = false ] || { + echo "FAIL: the private directory leaked to the consumer"; fail=1; } + +# ⭐ AND THE GLOB WITHHOLDS WHAT IT EXPANDS TO — both directories, by name. +# Checking `gen` alone would pass for an implementation that matched the +# unexpanded spelling and published `gen/one` and `gen/two` anyway. +for g in one two; do + [ "$(has_dir '/lib/src/' "/lib/gen/$g")" = true ] || { + echo "FAIL: the provider lost 'gen/$g', which its own glob names"; fail=1; } + [ "$(has_dir '/src/main' "/lib/gen/$g")" = false ] || { + echo "FAIL: 'gen/$g' leaked to the consumer — the glob was compared unexpanded" + fail=1; } +done + +[ "$fail" = 0 ] || exit 1 +echo "OK: built-from and published are separable, globs included (provider=$prov consumer=$cons rows)" diff --git a/tests/unit/test_cache_key.cpp b/tests/unit/test_cache_key.cpp index e579bf83..20051e55 100644 --- a/tests/unit/test_cache_key.cpp +++ b/tests/unit/test_cache_key.cpp @@ -367,3 +367,92 @@ TEST(CacheKey, AHostedTargetHasNoTargetImpliedFlagsEitherWay) { EXPECT_TRUE(a.targetImpliedFlags.empty()) << a.targetImpliedFlags.size(); EXPECT_EQ(a.targetImpliedFlags, b.targetImpliedFlags); } + +// ⭐⭐ THE HEADER SET THE DRIVER IS POINTED AT IS PART OF THE IDENTITY. +// +// Everything else on axis A describes the COMPILER. Nothing described the +// LIBRARY it compiles against — and the two are separately installed: one clang +// payload sits above whichever `xim:glibc` and `xim:linux-headers` the home +// carries. `driverIdentity` cannot cover it by design, because +// `normalize_driver_output` strips paths so that one entry CAN be shared +// between two homes. +// +// Measured (mcpp#514 §B): two host builds resolving glibc 2.39 and 2.44 hit one +// entry, and the mixed BMIs crashed the clang frontend during deserialization +// rather than producing a diagnostic. +TEST(CacheKey, TheHeaderSetIsPartOfTheKey) { + mcpp::manifest::Manifest m; + m.package.standard = "c++23"; + + auto a = ck::build_axes(freestanding_tc(false), m, "-std=c++23", {}, ""); + auto b = a; + // The axis is filled from the resolvers in a real build; here it is set + // directly, because what this asserts is that the KEY reads it — not how + // linkmodel discovers a payload, which is that module's own test. + b.targetHeaderSet = { "-isystem/xim-x-glibc/2.44/include" }; + a.targetHeaderSet = { "-isystem/xim-x-glibc/2.39/include" }; + + EXPECT_NE(ck::key_hex(a, pkg()), ck::key_hex(b, pkg())); +} + +// ...and it travels in entry.json, so a suspected wrong hit can be read field +// by field rather than guessed at. `is_cached` compares `inputs.toolchain` as +// one object, which is also why an entry written before this axis existed is a +// miss without needing `kCacheEpoch` to move. +TEST(CacheKey, TheHeaderSetIsRecordedInTheEntry) { + mcpp::manifest::Manifest m; + m.package.standard = "c++23"; + auto a = ck::build_axes(freestanding_tc(false), m, "-std=c++23", {}, ""); + a.targetHeaderSet = { "-isystem/xim-x-glibc/2.44/include" }; + auto j = ck::to_json(a, pkg()); + ASSERT_TRUE(j["toolchain"].contains("target_header_set")); + EXPECT_EQ(j["toolchain"]["target_header_set"][0], + "-isystem/xim-x-glibc/2.44/include"); +} + +// ⭐⭐ AND THE PATHS IN IT ARE RELATIVE — INCLUDING THE ONES OUTSIDE ``. +// +// ⚠️ THE TWO TESTS ABOVE SET THE AXIS DIRECTLY, so neither of them runs the +// relativization, and a store-only rule passed both while leaving this +// developer's own home in every key. The toolchain that showed it is the +// ordinary one: `CLibMode::Sysroot`, whose single compile token is +// +// --sysroot=/registry/subos/default +// +// — under the HOME and not under ``. An absolute key there is not +// WRONG, it is unshareable: two homes holding the same payloads stop hitting +// each other's entries, which is the property `normalize_driver_output` gives +// up path information to preserve. +// +// ⭐ THE CRITERION IS THE ABSENCE OF THE HOME, not the presence of a tag. +// Asserting `starts_with("")` would still pass if the rest of the string +// carried the absolute path behind it. +TEST(CacheKey, TheHeaderSetCarriesNoAbsoluteHome) { + const std::filesystem::path home = "/home/somebody/.mcpp/registry"; + const std::filesystem::path store = home / "data" / "xpkgs"; + + mcpp::toolchain::Toolchain tc; + tc.compiler = mcpp::toolchain::CompilerId::GCC; // no cfg ⇒ Sysroot mode + tc.version = "16.1.0"; + tc.driverIdent = "gcc-16.1.0"; + tc.targetTriple = "x86_64-linux-gnu"; + tc.sysroot = home / "subos" / "default"; + + mcpp::manifest::Manifest m; + m.package.standard = "c++23"; + const auto a = ck::build_axes(tc, m, "-std=c++23", {}, "", store); + + // A denominator: with an empty axis every assertion below is vacuous, and + // an empty axis is itself the bug in the other direction. + ASSERT_FALSE(a.targetHeaderSet.empty()); + for (auto const& t : a.targetHeaderSet) + EXPECT_EQ(t.find(home.generic_string()), std::string::npos) << t; + + // Two subos on ONE home are still two keys — the relativization must not + // flatten the distinction it exists to preserve. + auto tc2 = tc; + tc2.sysroot = home / "subos" / "musl"; + const auto b = ck::build_axes(tc2, m, "-std=c++23", {}, "", store); + EXPECT_NE(a.targetHeaderSet, b.targetHeaderSet); + EXPECT_NE(ck::key_hex(a, pkg()), ck::key_hex(b, pkg())); +} diff --git a/tests/unit/test_hostflags.cpp b/tests/unit/test_hostflags.cpp index e5ac8d54..36e49fdf 100644 --- a/tests/unit/test_hostflags.cpp +++ b/tests/unit/test_hostflags.cpp @@ -1,4 +1,5 @@ #include +#include import std; import mcpp.platform; @@ -295,3 +296,72 @@ TEST(GraphRuntimeFlags, UnparseableTripleTakesNone) { EXPECT_TRUE(mcpp::toolchain::graph_runtime_compile_flags( graph_tc("not-a-triple-at-all")).empty()); } + +// ⭐⭐ `--no-default-config` IS NOT PART OF THE PAYLOAD'S HEADER SET, AND WAS +// BEING SUPPRESSED WITH IT. +// +// The payload's `-isystem` rows describe a C library a graph-supplied target +// does not use, so withholding them is right. The cfg bypass is a different +// statement: `post_install.cppm` calls that file "a per-machine, +// per-install-path artifact", and reading it makes the command line depend on +// what happened to be installed when the payload landed. +// +// Measured on 2026.8.26.2: `mcpp build --target ` dropped the +// token, so clang read `bin/clang++.cfg`. That is also what made a hand-written +// `-clang++.cfg` a working workaround for mcpp#514 — a workaround that +// only existed because this token went missing. +TEST(HostFlags, TheCfgBypassSurvivesAGraphSuppliedTargetSide) { + // ⚠️ A FIXTURE, NOT THE MACHINE'S OWN TOOLCHAIN. The first draft used a + // synthetic `Toolchain` with no `binaryPath`, so `resolve_clang_driver` + // reported no cfg and the whole test SKIPPED — a check that asserts + // nothing while reporting success, which is the one failure mode a test + // must not have. `resolve_clang_driver` only asks whether a sibling + // `.cfg` EXISTS, so two empty files are a complete fixture. + namespace fs = std::filesystem; + // ⚠️ A FIXED NAME AND `remove_all` FIRST, NOT A PROCESS ID. The first + // draft reached for `::getpid()` and ``, which do not exist under + // MSVC — it built on the machine it was written on and failed on Windows + // CI, which is the only place that half of this project is visible. + // gtest runs a binary's tests serially, so one name is enough. + const auto root = fs::temp_directory_path() / "mcpp_hostflags_cfg_fixture"; + fs::remove_all(root); + fs::create_directories(root / "bin"); + fs::create_directories(root / "include" / "c++" / "v1"); + { std::ofstream(root / "bin" / "clang++"); } + { std::ofstream(root / "bin" / "clang++.cfg"); } + struct Cleanup { + fs::path p; + ~Cleanup() { std::error_code ec; fs::remove_all(p, ec); } + } cleanup{root}; + + auto tc = tc_for(CompilerId::Clang); + tc.binaryPath = root / "bin" / "clang++"; + ASSERT_TRUE(mcpp::toolchain::resolve_clang_driver(tc).hasCfg) + << "the fixture did not produce a cfg — the assertions below would be vacuous"; + + const auto has = [](const std::vector& v, std::string_view f) { + return std::ranges::find(v, f) != v.end(); + }; + + HostFlagOptions prebuilt; + prebuilt.cfgBypass = HostFlagOptions::CfgBypass::Always; + prebuilt.cAbiPrebuilt = true; + + HostFlagOptions fromGraph = prebuilt; + fromGraph.cAbiPrebuilt = false; + + const auto a = mcpp::toolchain::host_compile_tokens( + tc, prebuilt, mcpp::toolchain::no_escape); + const auto b = mcpp::toolchain::host_compile_tokens( + tc, fromGraph, mcpp::toolchain::no_escape); + + // The bypass is emitted on BOTH sides: the cfg is a per-machine, + // per-install-path artifact, and reading it makes the command line depend + // on what happened to be installed when the payload landed. + EXPECT_TRUE(has(a, "--no-default-config")); + EXPECT_TRUE(has(b, "--no-default-config")); + // ...while the payload's own C++ headers stay withheld from the graph side, + // which is the distinction this splits apart. + EXPECT_TRUE(has(a, "-nostdinc++")); + EXPECT_FALSE(has(b, "-nostdinc++")); +} diff --git a/tests/unit/test_post_install.cpp b/tests/unit/test_post_install.cpp index 5f084730..61d56483 100644 --- a/tests/unit/test_post_install.cpp +++ b/tests/unit/test_post_install.cpp @@ -1,7 +1,9 @@ #include +#include import std; import mcpp.toolchain.post_install; +import mcpp.platform.xlings; import mcpp.config; import mcpp.toolchain.registry; import mcpp.platform; @@ -190,3 +192,150 @@ TEST(PostInstallFixup, APackageWithNoFixupReportsNothingToReport) { << "a toolchain with no fixup reported a degradation: " << result->skippedReason; } + +// ⭐⭐ THE VERSION THAT WAS ASKED FOR AND THE VERSION THAT WAS INSTALLED ARE +// TWO VOCABULARIES FOR ONE FACT. +// +// A RuntimeBinding carries the DECLARED identity; xlings names the payload +// directory after what the request RESOLVED to. The two come apart the moment +// the index moves a package within a series — `xim:glibc@2.44` resolving to +// `2.44.2`. +// +// Measured 2026-08-27 on every CI machine with a cold cache, on `main` as +// readily as on any branch: +// +// error: selected RuntimeBinding glibc@2.44 requires payload +// '…/xim-x-glibc/2.44', but it is not installed +// $ ls …/xim-x-glibc/ → 2.44.2 +namespace { + +// A payload is "installed" for this purpose when it has a lib dir with a +// loader in it — that is what select_glibc_payload_lib returns. +std::filesystem::path make_glibc_payload(const std::filesystem::path& root, + std::string_view version) { + auto lib = root / std::string(version) / "lib64"; + std::filesystem::create_directories(lib); + std::ofstream(lib / "ld-linux-x86-64.so.2"); + return lib; +} + +struct GlibcRootFixture { + std::filesystem::path root; + explicit GlibcRootFixture(std::string_view name) + : root(std::filesystem::temp_directory_path() / name) { + std::error_code ec; + std::filesystem::remove_all(root, ec); + std::filesystem::create_directories(root); + } + ~GlibcRootFixture() { + std::error_code ec; + std::filesystem::remove_all(root, ec); + } +}; + +} // namespace + +TEST(GlibcPayload, TheExactVersionIsPreferred) { + GlibcRootFixture fx{"mcpp_glibc_exact"}; + auto want = make_glibc_payload(fx.root, "2.44"); + make_glibc_payload(fx.root, "2.44.2"); + auto got = mcpp::toolchain::select_glibc_payload_lib(fx.root, "glibc@2.44"); + ASSERT_TRUE(got.has_value()) << got.error(); + EXPECT_EQ(*got, want); +} + +TEST(GlibcPayload, ARequestResolvesToItsOneRefinement) { + GlibcRootFixture fx{"mcpp_glibc_refine"}; + auto only = make_glibc_payload(fx.root, "2.44.2"); + auto got = mcpp::toolchain::select_glibc_payload_lib(fx.root, "glibc@2.44"); + ASSERT_TRUE(got.has_value()) << got.error(); + EXPECT_EQ(*got, only); +} + +// ⚠️ PER COMPONENT, NOT PER CHARACTER. `2.4` is not a request that `2.44` +// answers — a prefix match on the string would say it is, and would then hand +// a build the wrong C library without saying anything. +TEST(GlibcPayload, AStringPrefixIsNotARefinement) { + GlibcRootFixture fx{"mcpp_glibc_strprefix"}; + make_glibc_payload(fx.root, "2.44"); + auto got = mcpp::toolchain::select_glibc_payload_lib(fx.root, "glibc@2.4"); + EXPECT_FALSE(got.has_value()); +} + +// ⭐ AND THE REFUSAL STILL STANDS WHEN THERE IS NO ONE ANSWER. "The resolution +// of this request" has to be a single payload to be an answer at all; two +// refinements are not a menu to pick from. +TEST(GlibcPayload, TwoRefinementsAreRefusedRatherThanChosenBetween) { + GlibcRootFixture fx{"mcpp_glibc_ambiguous"}; + make_glibc_payload(fx.root, "2.44.1"); + make_glibc_payload(fx.root, "2.44.2"); + auto got = mcpp::toolchain::select_glibc_payload_lib(fx.root, "glibc@2.44"); + EXPECT_FALSE(got.has_value()); +} + +// ⭐⭐ THE RESOLVER ITSELF, because it has TWO callers and they failed +// separately. The first version of this fix lived inside the toolchain fixup; +// `probe`'s compile-side payload discovery spelled the same lookup its own way +// and kept missing — and ITS failure names no version at all: +// +// bits/os_defines.h:39: fatal error: features.h: No such file +// +// because the glibc include directory is simply never added. Asserting on the +// shared function is what makes both call sites covered by one test. +TEST(PayloadDirForVersion, ExactWinsOverRefinement) { + GlibcRootFixture fx{"mcpp_pdfv_exact"}; + std::filesystem::create_directories(fx.root / "2.44"); + std::filesystem::create_directories(fx.root / "2.44.2"); + auto got = mcpp::xlings::paths::payload_dir_for_version(fx.root, "2.44"); + ASSERT_TRUE(got.has_value()); + EXPECT_EQ(got->filename(), "2.44"); +} + +TEST(PayloadDirForVersion, OneRefinementIsTheAnswer) { + GlibcRootFixture fx{"mcpp_pdfv_one"}; + std::filesystem::create_directories(fx.root / "2.44.2"); + auto got = mcpp::xlings::paths::payload_dir_for_version(fx.root, "2.44"); + ASSERT_TRUE(got.has_value()); + EXPECT_EQ(got->filename(), "2.44.2"); +} + +TEST(PayloadDirForVersion, TwoRefinementsAreNotAnAnswer) { + GlibcRootFixture fx{"mcpp_pdfv_two"}; + std::filesystem::create_directories(fx.root / "2.44.1"); + std::filesystem::create_directories(fx.root / "2.44.2"); + EXPECT_FALSE(mcpp::xlings::paths::payload_dir_for_version(fx.root, "2.44")); +} + +// ⚠️ PER COMPONENT, NOT PER CHARACTER — a string prefix would hand a build the +// wrong C library and say nothing. +TEST(PayloadDirForVersion, AStringPrefixIsNotARefinement) { + GlibcRootFixture fx{"mcpp_pdfv_strprefix"}; + std::filesystem::create_directories(fx.root / "2.44"); + EXPECT_FALSE(mcpp::xlings::paths::payload_dir_for_version(fx.root, "2.4")); +} + +TEST(GlibcPayload, NothingInstalledIsStillRefused) { + GlibcRootFixture fx{"mcpp_glibc_empty"}; + auto got = mcpp::toolchain::select_glibc_payload_lib(fx.root, "glibc@2.44"); + EXPECT_FALSE(got.has_value()); +} + +// ⚠️ AN EMPTY REQUEST IS NOT A REQUEST FOR EVERYTHING. +// +// `packageRoot / ""` is `packageRoot`, and that IS a directory — so the +// exact-match branch would hand the CONTAINER back and every caller would +// treat it as a payload. Both callers reject an empty version before arriving, +// which is exactly why nothing would have caught it here. +TEST(PayloadDirForVersion, AnEmptyRequestIsNotTheContainer) { + auto root = std::filesystem::temp_directory_path() + / "mcpp_payload_empty_request"; + std::filesystem::remove_all(root); + std::filesystem::create_directories(root / "2.44.2"); + + EXPECT_FALSE(mcpp::xlings::paths::payload_dir_for_version(root, "")); + // The denominator: a real request against the same tree still resolves, + // so an empty result above is the guard and not an unreadable directory. + EXPECT_TRUE(mcpp::xlings::paths::payload_dir_for_version(root, "2.44")); + + std::filesystem::remove_all(root); +}