[wasm-split] Precompute ownership info (NFC) by aheejin · Pull Request #8986 · WebAssembly/binaryen

aheejin · 2026-08-11T21:00:21Z

Given a module element name, many parts of the code queries for its owning modules (where the module element has to be placed) or secondary modules using that module element. This adds OwnershipTracker, which precomputes and manages that information. All calls to getOwner or getUsingSecondaries that required computations iterating on all secondary modules which can be as many as thousands, has been replaced with a call that simply returns prcomputed information.

For the Jul 2026 version of the applications received from the Dart team, #9007 and this combined reduce the running time of wasm-split by 25% for acx_gallery (30.6s -> 22.7s) and by 40% for essentials (225.1s -> 134.4s).

Suggested in #8832 (comment).

Previously we removed module elements one by one within a loop. But because `Module` stores a module element in both a map and a vector, removing a single module element using `removeModuleElement` is O(N), because it needs to shift all vector elements after it: https://github.com/WebAssembly/binaryen/blob/302396a676433152a32375a81d71e74687c97a1b/src/wasm/wasm.cpp#L1970-L1979 This removes module elements in bulk using `removeModuleElements`, which does the shifting only once. https://github.com/WebAssembly/binaryen/blob/302396a676433152a32375a81d71e74687c97a1b/src/wasm/wasm.cpp#L2004-L2018 Combining with #8986, acx_gallery's running time improved by 50.3% (30s -> 15s), and essentials by 60.8% (230s -> 90s). (for Jul 2026 version) I guess the main reason for the running time increase in #8441 was this O(N) `removeModuleElement` called within a loop after all.

tlively · 2026-08-13T18:48:19Z

+// In the initial building phase, we just directly add to a UsedName struct.
+// After OwnershipTracker is constructed, we all its insert() method to update
+// owner modules and using secondary modules correctly.


How do these two phases relate to the general process of splitting described in the comment at the top of the file?

All of shareImportableItems (and computeUsedNames called from shareImportableItems and construction of OwnershipTracker inside computeUsedNames) correspond to this paragraph:

// 4. Export globals, tags, tables, and memories from the primary module and // import them in the secondary modules. If possible, move those module // items instead to the secondary modules.

The reason ADD_ITEM does two different things is, before we create OwnershipTracker, we build UsedName for each module, and used.field.insert(val); is done in this phase. After OwnershipTracker is created, we add more constraints. But unlike the straightforward first scanning phase this constraints can add one module item to multiple modules, and we need to keep track of which one should be the owner and such. So in this second phase we use tracker->insert.

So if I understand correctly, the first pass collects the used names and the second pass computes the owners.

What I'm not understanding yet is why we need two passes for this. Can't we just insert used names into the OwnershipTracker in a single pass and have it automatically update the owner to be the primary module as soon as it sees that two different modules have used the same item?

The reason I used the two-phase thing was the first phase was using ParallelFunctionAnalysis:

binaryen/src/ir/module-splitting.cpp

Lines 704 to 728 in dae272c

// Given a module, collect names used in the module

auto scanModule = [&](Module& module) {

UsedNames used;

ModuleUtils::ParallelFunctionAnalysis<UsedNames> nameCollector(

module, [&](Function* func, UsedNames& used) {

if (!func->imported()) {

NameCollector(used).walk(func->body);

}

});

for (auto& [_, funcUsed] : nameCollector.map) {

used.globals.insert(funcUsed.globals.begin(), funcUsed.globals.end());

used.memories.insert(funcUsed.memories.begin(), funcUsed.memories.end());

used.tables.insert(funcUsed.tables.begin(), funcUsed.tables.end());

used.tags.insert(funcUsed.tags.begin(), funcUsed.tags.end());

used.dataSegments.insert(funcUsed.dataSegments.begin(),

funcUsed.dataSegments.end());

used.elementSegments.insert(funcUsed.elementSegments.begin(),

funcUsed.elementSegments.end());

}

NameCollector collector(used);

return used;

};

and if we use the OwnershipTracker from the beginning we can't use ParallelFunctionAnalysis anymore.

But is using ParallelFunctionAnalysis really faster? Several months ago I tried to use ParallelFunctionAnalysis for other functions that were not using it and it actually slowed the program down. I just tried to remove it from scanModule, and it apparently makes at least Dart applications on my local machine faster.

I uploaded the removing of ParallelFunctionAnalysis as a separate PR to make diffs clearer: #9007
After this we can remove this two phase and use the tracker from the beginning.

Eliminated the two phase and use tracker.insert for all cases, on top of #9007: f03574f

This reduces the running time of acx_gallery from 25s to 23s. (Will update the PR description after running essentials too)

tlively · 2026-08-14T23:23:34Z

+#define ADD_ITEM_TO_TRACKER(FIELD, VAL)                                        \
+  tracker.insert(VAL, owner, &OwnershipTracker::FIELD, &UsedNames::FIELD)


This macro implicitly using whatever owner happens to be in scope at the use site makes it harder to understand what's going on. Can we pass the owner in explicitly?

Also it's probably possible to use some template overloading magic to do something like tracker.insert<Memory>(segment->memory, owner) without using macros.

I tried it. It looks we need an additional template<typename T> insert wrapper method: 8912e36
Do you prefer this?

(And in case you rather prefer the macro, f03574f added the owner explicitly to it)

Thanks, I guess I don't have a strong preference either way. It's nice to avoid macros at the call sites, but it's also nice to avoid that messy helper function. Maybe the body of the helper function can be generated with a local macro that generates a single if constexpr case? That way we would get the best of both worlds.

But also happy to leave this to your preference if you have one.

Like this? 716760a

tlively · 2026-08-14T23:31:35Z

+// In the initial building phase, we just directly add to a UsedName struct.
+// After OwnershipTracker is constructed, we all its insert() method to update
+// owner modules and using secondary modules correctly.


So if I understand correctly, the first pass collects the used names and the second pass computes the owners.

What I'm not understanding yet is why we need two passes for this. Can't we just insert used names into the OwnershipTracker in a single pass and have it automatically update the owner to be the primary module as soon as it sees that two different modules have used the same item?

Given a module element name, many parts of the code queries for its owning modules (where the module element has to be placed) or secondary modules using that module element. This adds `OwnershipTracker`, which precomputes and manages that information. All calls to `getOwner` or `getUsingSecondaries` that required computations iterating on all secondary modules which can be as many as thousands, has been replaced with a call that simply returns prcomputed information. For the Jul 2026 version of the applications received from the Dart team, this reduces the running time of wasm-split by 17% for acx_gallery (30s -> 25s) and by 33% for essentials (230s -> 153s). Suggested in #8832 (comment).

Co-authored-by: Thomas Lively <tlively@google.com>

Co-authored-by: Thomas Lively <tlively123@gmail.com>

Previously we removed module elements one by one within a loop. But because `Module` stores a module element in both a map and a vector, removing a single module element using `removeModuleElement` is O(N), because it needs to shift all vector elements after it: https://github.com/WebAssembly/binaryen/blob/302396a676433152a32375a81d71e74687c97a1b/src/wasm/wasm.cpp#L1970-L1979 This removes module elements in bulk using `removeModuleElements`, which does the shifting only once. https://github.com/WebAssembly/binaryen/blob/302396a676433152a32375a81d71e74687c97a1b/src/wasm/wasm.cpp#L2004-L2018 Combining with #8986, acx_gallery's running time improved by 50.3% (30s -> 15s), and essentials by 60.8% (230s -> 90s). (for Jul 2026 version) I guess the main reason for the running time increase in #8441 was this O(N) `removeModuleElement` called within a loop after all.

aheejin · 2026-08-17T23:39:40Z

Oh no, landing the stacked PR this PR depends on automatically force-pushes all PRs up the chain. What a horrible UI. I guess I can't use this as is. (Maybe it's fine for this PR because most comments have been already addressed, but in general)

tlively · 2026-08-18T02:43:30Z

+  }
+
+    INSERT_ITEM(Table, tables)
+    else INSERT_ITEM(Memory, memories) else INSERT_ITEM(Global, globals) else INSERT_ITEM(


No need for the elses since only the code for the relevant type will be generated anyway.

Done. By the way clang-format did that horrible thing... Removing else made clang-format come to senses.

tlively · 2026-08-18T02:47:22Z

+    }
+  }
+
+  void build(const std::vector<std::unique_ptr<Module>>& secondaries) {


Can this be made a constructor? (Although, actually, I don't see where this is called. Can it be removed?)

That was used in the first phase when we did the two phase thing and I forgot to remove it. Removed.

tlively · 2026-08-18T02:50:11Z

+#define ADD_ITEM_TO_TRACKER_TO_TRACKER(FIELD, VAL)                             \
+  tracker.insert(VAL, owner, &OwnershipTracker::FIELD, &UsedNames::FIELD)
+


This is no longer used.

tlively · 2026-08-18T03:32:54Z

+    return empty;
+  }
+
+  bool useEmpty(Name name, const std::unordered_map<Name, ItemInfo>& map) {


isUnused might be a clearer name.

aheejin requested a review from tlively August 11, 2026 21:00

aheejin requested a review from a team as a code owner August 11, 2026 21:00

aheejin mentioned this pull request Aug 12, 2026

[wasm-split] Remove unnecessary global exports #8832

Merged

aheejin mentioned this pull request Aug 12, 2026

[wasm-split] Remove module elements in bulk (NFC) #8990

Open

tlively reviewed Aug 13, 2026

View reviewed changes

tlively reviewed Aug 14, 2026

View reviewed changes

aheejin changed the base branch from main to wasm_split_no_parallel August 15, 2026 03:19

Base automatically changed from wasm_split_no_parallel to main August 17, 2026 23:38

aheejin and others added 9 commits August 17, 2026 16:38

Update src/ir/module-splitting.cpp

7d0d87d

Co-authored-by: Thomas Lively <tlively@google.com>

Remove usedToSecondary map

8d9455b

FIELD->field, VAL->val

86f57f4

Rename a few variables

89b8e08

More comments

620886f

Apply suggestions from code review

47b0a61

Co-authored-by: Thomas Lively <tlively123@gmail.com>

Eliminate two-phase approach in computeUsedNames

f3b6900

Use template instead of ADD_ITEM macro

d117127

aheejin force-pushed the wasm_split_owning_modules branch from 8912e36 to d117127 Compare August 17, 2026 23:38

aheejin added 2 commits August 18, 2026 00:20

Use macros in insert()

716760a

clang-format

3fbc940

tlively reviewed Aug 18, 2026

View reviewed changes

aheejin added 3 commits August 18, 2026 05:30

Remove 'else'

7b4c577

Remove OwnershipTracker::build

b0837ed

useEmpty -> isUnused

0b1c0dc

	// Given a module, collect names used in the module
	auto scanModule = [&](Module& module) {
	UsedNames used;
	ModuleUtils::ParallelFunctionAnalysis<UsedNames> nameCollector(
	module, [&](Function* func, UsedNames& used) {
	if (!func->imported()) {
	NameCollector(used).walk(func->body);
	}
	});

	for (auto& [_, funcUsed] : nameCollector.map) {
	used.globals.insert(funcUsed.globals.begin(), funcUsed.globals.end());
	used.memories.insert(funcUsed.memories.begin(), funcUsed.memories.end());
	used.tables.insert(funcUsed.tables.begin(), funcUsed.tables.end());
	used.tags.insert(funcUsed.tags.begin(), funcUsed.tags.end());
	used.dataSegments.insert(funcUsed.dataSegments.begin(),
	funcUsed.dataSegments.end());
	used.elementSegments.insert(funcUsed.elementSegments.begin(),
	funcUsed.elementSegments.end());
	}

	NameCollector collector(used);

	return used;
	};

		#define ADD_ITEM_TO_TRACKER(FIELD, VAL) \
		tracker.insert(VAL, owner, &OwnershipTracker::FIELD, &UsedNames::FIELD)

		#define ADD_ITEM_TO_TRACKER_TO_TRACKER(FIELD, VAL) \
		tracker.insert(VAL, owner, &OwnershipTracker::FIELD, &UsedNames::FIELD)

Uh oh!

Conversation

aheejin commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aheejin Aug 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aheejin Aug 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aheejin commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

aheejin commented Aug 11, 2026 •

edited

Loading

aheejin Aug 15, 2026 •

edited

Loading

aheejin Aug 15, 2026 •

edited

Loading

aheejin commented Aug 17, 2026 •

edited

Loading