Skip to content

Commit 5c3abfb

Browse files
fix(toolchain): narrow the sweep, and stop spawning cl.exe to ask a yes/no (#444)
Both found by reviewing the 2026.8.16.3 diff after tagging it. **The sweep could delete another version mid-install.** It walked the whole family directory and removed any version directory containing no regular files. But an install populates a version directory over time -- which is why package_fetcher tracks completeness with a marker file rather than by existence -- so a DIFFERENT version being extracted right now is briefly indistinguishable from a leftover skeleton. Two mcpp processes against one MCPP_HOME is ordinary on a shared or self-hosted runner, and this feature exists for long-lived installs on exactly those machines. `.trash-*` is still swept family-wide: that name is only ever written by this code, so deleting one is safe whoever else is running. The file-less-skeleton rule now applies to the single version the command names. **`msvc_available_here()` ran a compiler to answer a yes/no.** It wanted to know whether a usable toolset exists and called the full `installation_at()`, which spawns cl.exe for its banner to identify the version. That question is asked on every build at the MSVC-ABI gate, so a machine with several installed toolsets paid several subprocess spawns per build -- and those are precisely the machines this predicate was added for. `installation_at(..., identifyVersion = false)` skips the banner. The layout still has exactly ONE implementation; this is a parameter, not a second copy of the path arithmetic. The new test was run against the family-wide sweep and fails there. Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com>
1 parent e4f6c1e commit 5c3abfb

4 files changed

Lines changed: 102 additions & 25 deletions

File tree

CHANGELOG.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,30 @@
33
> 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。
44
> 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)
55
6+
## [Unreleased]
7+
8+
### 修复
9+
10+
- **卸载后的清扫会波及**别的版本**,而那可能正在被另一个进程解压。**
11+
12+
`sweep_parked_payloads` 原来把整个 family 目录扫一遍,把**任何**没有文件的
13+
版本目录删掉。但安装是**逐步**往版本目录里写文件的(所以 package_fetcher
14+
用标记文件而不是"目录在"来判断装完没有),于是**另一个正在解压的版本**在那
15+
短暂窗口里和"残骨架"长得一模一样 —— 共用一个 `MCPP_HOME` 的机器上
16+
(自托管 runner、共享开发机)两个 mcpp 进程同时跑是常态。
17+
18+
`.trash-*` 照旧全扫(那个名字只有这段代码会写);"没有文件的骨架"这一条
19+
收窄成**只扫这条命令点名的那一个版本**
20+
21+
- **`msvc_available_here()` 每次构建都要为每个已装 payload 起一次 cl.exe。**
22+
23+
它只想知道"这儿有没有能用的 toolset",却走了完整的 `installation_at()`,
24+
那里面会跑一次 cl 拿 banner 来定版本。而这个判据在**每次构建**的 MSVC ABI
25+
门上都会被问到 —— 正好是装了多个 toolset 的机器最慢。
26+
27+
`installation_at(..., identifyVersion=false)` 跳过 banner。**布局仍然只有
28+
一份实现** —— 不是再抄一遍路径拼接。
29+
630
## [2026.8.16.3] — 2026-08-16
731

832
### 修复

src/toolchain/lifecycle.cppm

Lines changed: 28 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -330,20 +330,33 @@ export bool remove_payload_tree(const std::filesystem::path& root,
330330
// Best-effort and run before a lifecycle operation rather than after one: by
331331
// the next command the process that held the bytes is normally gone, so this
332332
// is where they actually get freed.
333-
export void sweep_parked_payloads(const std::filesystem::path& pkgRoot) {
333+
export void sweep_parked_payloads(const std::filesystem::path& pkgRoot,
334+
const std::filesystem::path& skeleton) {
334335
std::error_code ec;
335336
if (!std::filesystem::is_directory(pkgRoot, ec)) return;
337+
338+
// `.trash-*` is a name only this code writes, so deleting any of them is
339+
// safe regardless of who else is running.
336340
for (auto& e : std::filesystem::directory_iterator(pkgRoot, ec)) {
337-
if (!e.is_directory(ec)) continue;
338-
if (e.path().filename().string().starts_with(".trash-")) {
339-
std::filesystem::remove_all(e.path(), ec);
340-
} else if (!any_regular_file(e.path())) {
341-
// A version directory with no files in it is the skeleton a
342-
// removal could not delete because something held a directory
343-
// open. A real install always has files, so this cannot eat one.
341+
if (e.is_directory(ec)
342+
&& e.path().filename().string().starts_with(".trash-"))
344343
std::filesystem::remove_all(e.path(), ec);
345-
}
346344
}
345+
346+
// The file-less skeleton, on the other hand, is swept for ONE named
347+
// version — the one this command is about — and never for whatever else
348+
// happens to be sitting in the family directory.
349+
//
350+
// "No files in it" is not a safe thing to conclude about someone else's
351+
// directory: an install populates a version directory over time (it is
352+
// why package_fetcher tracks completeness with a marker file rather than
353+
// by existence), so a concurrent `toolchain install` of a DIFFERENT
354+
// version is briefly indistinguishable from a skeleton, and sweeping the
355+
// whole family would delete it mid-extraction. Two mcpp processes against
356+
// one MCPP_HOME is ordinary on a shared or self-hosted runner.
357+
if (!skeleton.empty() && std::filesystem::is_directory(skeleton, ec)
358+
&& !any_regular_file(skeleton))
359+
std::filesystem::remove_all(skeleton, ec);
347360
}
348361

349362
// The first entry that is still there after a failed removal. The error code
@@ -763,10 +776,11 @@ export int toolchain_install(const mcpp::config::GlobalConfig& cfg,
763776
// A previous `remove` may have parked a held payload beside this one;
764777
// by now whatever held it has exited, so free the bytes before adding
765778
// another few hundred MB.
766-
sweep_parked_payloads(
767-
mcpp::xlings::paths::xim_tool(mcpp::config::make_xlings_env(cfg),
768-
pkg.ximName, pkg.ximVersion)
769-
.parent_path());
779+
{
780+
auto vdir = mcpp::xlings::paths::xim_tool(
781+
mcpp::config::make_xlings_env(cfg), pkg.ximName, pkg.ximVersion);
782+
sweep_parked_payloads(vdir.parent_path(), vdir);
783+
}
770784
auto payload = fetcher.resolve_xpkg_path(pkg.target(), /*autoInstall=*/true, &progress);
771785
mcpp::log::verbose("toolchain", std::format("main install result: {}",
772786
payload ? ("ok → " + payload->root.string()) : payload.error().message));
@@ -1015,7 +1029,7 @@ export int toolchain_remove(const mcpp::config::GlobalConfig& cfg,
10151029
mcpp::ui::error(std::format("{} is not installed", spec));
10161030
return 1;
10171031
}
1018-
sweep_parked_payloads(installDir.parent_path());
1032+
sweep_parked_payloads(installDir.parent_path(), installDir);
10191033
if (!remove_payload_tree(installDir, ec)) {
10201034
// Say that the payload is now BROKEN, not merely that removal
10211035
// failed. `remove_all` deletes what it can before it stops, so a

src/toolchain/msvc.cppm

Lines changed: 21 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -93,8 +93,13 @@ std::optional<MsvcInstallation> detect_installation();
9393
// anywhere, which is what makes the managed path testable off Windows. The
9494
// cl banner simply stays unparsed there and `display_version()` falls back
9595
// to the declared version.
96+
// `identifyVersion = false` skips running cl.exe for its banner. Use it when
97+
// the question is only "is there a usable toolset here" -- `msvc_available_here`
98+
// asks that on every build, and spawning a compiler per installed payload to
99+
// answer it is a latency regression on exactly the machines that have several.
96100
std::optional<MsvcInstallation> installation_at(const std::filesystem::path& vsRoot,
97-
std::string_view toolsVersion);
101+
std::string_view toolsVersion,
102+
bool identifyVersion = true);
98103

99104
// Parse a cl.exe banner into (version, arch). Token-based so localized
100105
// banners work: first "d.d.d[.d]" run is the version, arch is the arm64/x64/
@@ -514,7 +519,8 @@ namespace {
514519
// on a Linux CI runner.
515520
std::optional<MsvcInstallation>
516521
installation_from_tools_dir(const std::filesystem::path& vsRoot,
517-
const std::filesystem::path& tools);
522+
const std::filesystem::path& tools,
523+
bool identifyVersion = true);
518524

519525
// Capture cl.exe's banner. cl prints it (plus a usage complaint) when run
520526
// bare; the exit status is irrelevant — parse whatever came out.
@@ -526,7 +532,8 @@ std::string capture_cl_banner(const std::filesystem::path& cl) {
526532

527533
std::optional<MsvcInstallation>
528534
installation_from_tools_dir(const std::filesystem::path& vsRoot,
529-
const std::filesystem::path& tools) {
535+
const std::filesystem::path& tools,
536+
bool identifyVersion) {
530537
MsvcInstallation inst;
531538
inst.vsRoot = vsRoot;
532539
inst.vsProduct = product_from_vs_root(vsRoot);
@@ -558,22 +565,25 @@ installation_from_tools_dir(const std::filesystem::path& vsRoot,
558565
// Version identification: banner is authoritative; tolerate failure
559566
// (clVersion stays empty and display_version() falls back to the
560567
// tools-dir version). Off Windows that failure is the normal case.
561-
if (auto parsed = parse_cl_banner(capture_cl_banner(inst.clPath))) {
562-
inst.clVersion = parsed->first;
563-
if (!parsed->second.empty()) inst.arch = parsed->second;
568+
if (identifyVersion) {
569+
if (auto parsed = parse_cl_banner(capture_cl_banner(inst.clPath))) {
570+
inst.clVersion = parsed->first;
571+
if (!parsed->second.empty()) inst.arch = parsed->second;
572+
}
564573
}
565574
return inst;
566575
}
567576

568577
} // namespace
569578

570579
std::optional<MsvcInstallation> installation_at(const std::filesystem::path& vsRoot,
571-
std::string_view toolsVersion) {
580+
std::string_view toolsVersion,
581+
bool identifyVersion) {
572582
if (toolsVersion.empty()) return std::nullopt;
573583
auto tools = vsRoot / "VC" / "Tools" / "MSVC" / std::string(toolsVersion);
574584
std::error_code ec;
575585
if (!std::filesystem::is_directory(tools, ec)) return std::nullopt;
576-
return installation_from_tools_dir(vsRoot, tools);
586+
return installation_from_tools_dir(vsRoot, tools, identifyVersion);
577587
}
578588

579589
std::optional<MsvcInstallation> detect_installation() {
@@ -708,7 +718,9 @@ bool msvc_available_here([[maybe_unused]] const std::filesystem::path& pkgsDir)
708718
if (!std::filesystem::is_directory(root, ec)) return false;
709719
for (auto& v : std::filesystem::directory_iterator(root, ec)) {
710720
if (!v.is_directory(ec)) continue;
711-
if (installation_at(v.path(), v.path().filename().string())) return true;
721+
// identifyVersion=false: this asks IF a toolset is here, never which.
722+
if (installation_at(v.path(), v.path().filename().string(),
723+
/*identifyVersion=*/false)) return true;
712724
}
713725
return false;
714726
#else

tests/unit/test_toolchain_lifecycle.cpp

Lines changed: 29 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,33 @@ TEST(ToolchainRemove, AnOrdinaryPayloadIsJustDeleted) {
4747
std::filesystem::remove_all(dir, ec);
4848
}
4949

50+
TEST(ToolchainRemove, TheSweepDoesNotTouchAnotherVersionThatHasNoFilesYet) {
51+
// An install populates a version directory over time, so a DIFFERENT
52+
// version being extracted right now is briefly indistinguishable from a
53+
// leftover skeleton. Two mcpp processes against one MCPP_HOME is ordinary
54+
// on a shared or self-hosted runner, and deleting someone else's
55+
// half-extracted toolchain is not a cleanup.
56+
//
57+
// So the file-less sweep applies to the ONE version this command names.
58+
auto pkgRoot = std::filesystem::temp_directory_path()
59+
/ std::format("mcpp-sweep3-{}", std::chrono::steady_clock::now()
60+
.time_since_epoch().count());
61+
auto mine = pkgRoot / "14.44.35207"; // the one being removed
62+
auto theirs = pkgRoot / "14.52.36629"; // someone else, mid-extract
63+
std::filesystem::create_directories(mine / "bin");
64+
std::filesystem::create_directories(theirs / "VC" / "Tools");
65+
66+
sweep_parked_payloads(pkgRoot, mine);
67+
68+
EXPECT_FALSE(std::filesystem::exists(mine)) << "the named skeleton survived";
69+
EXPECT_TRUE(std::filesystem::exists(theirs))
70+
<< "swept a version this command was not about — that is someone "
71+
"else's install being extracted";
72+
73+
std::error_code ec;
74+
std::filesystem::remove_all(pkgRoot, ec);
75+
}
76+
5077
TEST(ToolchainRemove, TheSweepDeletesParkedPayloadsAndNothingElse) {
5178
// The sweep runs before every install and remove, so it must be precise:
5279
// `.trash-*` goes, an installed version directory beside it stays.
@@ -58,7 +85,7 @@ TEST(ToolchainRemove, TheSweepDeletesParkedPayloadsAndNothingElse) {
5885
std::filesystem::create_directories(pkgRoot / "14.44.35207" / "bin");
5986
std::ofstream{pkgRoot / "14.44.35207" / "bin" / "cl.exe"} << "keep me";
6087

61-
sweep_parked_payloads(pkgRoot);
88+
sweep_parked_payloads(pkgRoot, {});
6289

6390
EXPECT_FALSE(std::filesystem::exists(pkgRoot / ".trash-14.44.35207-1"))
6491
<< "parked payload was not swept";
@@ -100,7 +127,7 @@ TEST(ToolchainRemove, TheSweepAlsoClearsAFileLessSkeleton) {
100127
std::filesystem::create_directories(pkgRoot / "14.52.36629" / "bin");
101128
std::ofstream{pkgRoot / "14.52.36629" / "bin" / "cl.exe"} << "a real one";
102129

103-
sweep_parked_payloads(pkgRoot);
130+
sweep_parked_payloads(pkgRoot, pkgRoot / "14.44.35207");
104131

105132
EXPECT_FALSE(std::filesystem::exists(pkgRoot / "14.44.35207"))
106133
<< "the empty skeleton survived the sweep";

0 commit comments

Comments
 (0)