Repository navigation
Method splitting - #270
Draft
andreaTP wants to merge 11 commits into
Draft
Method splitting#270andreaTP wants to merge 11 commits into
andreaTP wants to merge 11 commits into
Conversation
A Wasm function that compiles to more than HUGE_METHOD_LIMIT bytes of JVM bytecode is past the size HotSpot will compile, so it is never JITed and runs in the JVM's own bytecode interpreter no matter how hot it gets. Nothing warns about it. Real modules hit this hard: QuickJS's JS_CallInternal is 56671 bytes and is among the hottest code in the program. That ceiling is the one dispatch methods are already sized against a few lines above, so this reuses the same constant and the same knob rather than inventing a second notion of "too big to JIT". Split such functions into JIT-eligible pieces, in three escalating steps: * Linear chunking cuts at labels where the operand stack is provably empty (ASM's own data-flow analysis, not a guess) and no branch spans the cut. * Switch splitting additionally lets a br_table's targets become chunk boundaries, resolving far targets through per-chunk stubs. A single switch's targets all share one source position, which otherwise makes every candidate look span-blocked. * Trampoline splitting handles what neither can: bodies dominated by a loop. A loop's back-edge is a goto, and a goto cannot cross a method boundary, so no position inside a loop is ever a legal linear cut -- and measured on real code that is 97.5% of JS_CallInternal and 98.2% of printf_core. Instead of cutting around loops, inter-chunk control flow stops using gotos: every local lives in a long[] frame, each chunk returns the index of the next chunk, and a driver loop in the entry method dispatches. A back-edge is then just another PC value. Because a chunk stores PC=DONE on entry, an ordinary Wasm return stays an ordinary areturn and no opcode emitter changes. A function that cannot be split safely is left exactly as it was: compiled to a single method, still not JITable, and silent. That case is a missed optimisation, not a failure, so it must not be pushed to the interpreter and must not fail a build running with InterpreterFallback.FAIL. The threshold is clamped to stay well under the 65535-byte ceiling on a JVM method's Code attribute, so raising endive.hugeMethodLimit cannot lead the planner to emit a chunk the class writer then rejects. Also routes functions with very many try_table handlers to the interpreter rather than letting ASM's exception-table handling stall the build. compiler-tests: 34110 pass.
Trampoline chunks boxed and unboxed every local in the function on every transition. That is the wrong cost to pay exactly where trampolining is most valuable: a dispatch-loop case body typically touches a handful of locals, and JS_CallInternal has 48, so each pass through the loop moved all 48 twice. The long[] frame is the single source of truth and the JVM slots are only a per-chunk cache of it, so a chunk need only load the locals it reads or writes and store back the ones it writes. A local the chunk never mentions keeps its value in the frame untouched. Written locals are loaded too, not just stored: a write that happens on only one path through the chunk would otherwise leave the slot undefined, and the shared epilogue stores unconditionally. Loading first makes the worst case a store of the value that was already there. Only LOCAL_GET, LOCAL_SET and LOCAL_TEE touch local slots (LOCAL_TEE dups the stack value and delegates to LOCAL_SET), so the read/write sets are exact rather than approximated, and no dataflow fixpoint is needed. QuickJS throughput improves 76ms -> 33ms per iteration, taking the total gain over an unsplit build to 8.2x (270ms -> 33ms), reproducible across runs.
The planner reserved bytes for every local in the function when sizing each chunk's prologue and epilogue. That was written when chunks synced all of them; they now sync only the locals they touch, so the reserve sat far above the real cost and shrank the budget enough to split functions more finely than needed. Reserve optimistically while seeding, then cost each resulting chunk exactly from its own touched/written sets. A chunk that still comes out too big is subdivided at the last candidate that fits -- packing the leading piece full rather than halving it -- instead of costing the whole function its split. Because subdividing can strand a branch that used to be internal to the chunk, the branch-lands-on-a-chunk-start repair is re-run after every subdivision, so both invariants hold together. Sizing alone is not enough, though. Whether planning succeeds also depends on where the boundaries happened to land: different cuts make different branches cross chunks, and if one of those lands on a position with a non-empty operand stack -- which can never begin a chunk -- the function was written off as unsplittable. Widening the budget shifted the boundaries and, on Boxedwine, gained one function while losing two for exactly that reason. So a failure at one budget is not proof the function cannot be split: retry at progressively tighter budgets, which lands the cuts differently. Boxedwine splits 20 functions with 11 left interpreted, against 18/13 before. QuickJS is unchanged at 17 split, 0 interpreted.
…aries A chunk boundary is a method boundary, so a label with anything pending on the operand stack could not begin a chunk -- and a branch landing on such a label disqualified the whole function. That was the single most common reason splitting still failed, and it made planning fragile: moving the boundaries changed which branches crossed chunks, so a function could stop splitting purely because the seed budget shifted. Such labels are now usable. The verifier already guarantees the stack at a branch matches its target, so a transition spills those values into slots after the PC in the long[] frame and the destination reloads them before its first instruction. Spilling also leaves the stack empty at every exit, which is what keeps one shared epilogue valid for all of them. ASM's analysis already gives the exact depth and types at each label, so no new analysis is needed. References still cannot begin a chunk -- they have no representation in a long[] -- and depth is capped. Boxedwine now splits 21 functions with 10 interpreted (was 20/11); func 4413, 28028 bytes, splits for the first time. QuickJS is unchanged at 17/0.
A function with any try_table was declined outright. It need not be: a JVM exception-table entry names a protected range and its handler, and the only real requirement is that they -- and the visitTryCatchBlock declaring them -- end up in the same method. So each try_table contributes a region no chunk boundary may fall inside, exactly like the existing branch-span rule. Linear and switch chunking still decline try_table; only the trampoline takes it, since that is where the remaining oversized functions are. Functions whose protected ranges blanket the body have nowhere left to cut and still fall back, which is the right answer: measured on Boxedwine, eight of its try_table functions have 85-98% of the body inside protected ranges, leaving runs of only 61-270 instructions. Boxedwine now splits 22 functions with 9 interpreted (was 21/10); func 2068 splits for the first time. QuickJS is unchanged at 17/0, 577 chunks.
Splitting needs asm-analysis, which nothing in endive used before, and it
arrives on the bridge's exec-maven-plugin classpath only transitively from
build-time-compiler. The plugin contributes asm and asm-tree itself, nearer the
root of its graph, so those win: with the bump to 9.11.1 the classpath ended up
running 9.11.1 analysis code against 9.10.1 core, and died resolving
LimitExceededException, which exists only in the newer.
Pin asm, asm-tree, asm-analysis and asm-commons there to ${asm.version}.
Reactor and compiler-tests pass on JDK 27 with ASM 9.11.1: 34111 tests.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This continues #256 by @kevodwyer, which splits Wasm functions too large for HotSpot to JIT-compile into smaller methods. That PR was closed and its fork is no longer available, so I've picked the commits up here. They are unchanged and keep @kevodwyer as author. Thank you for this work: on my machine, AWFY Richards runs about 40× faster on CPython, 6× on QuickJS and 5× on Lua.
On top of the original commits, I followed up on the review comment about making this a self-contained step that can be switched off:
MethodSplitter;Compileronly calls into it.MachineFactoryCompiler.builder(module).withMethodSplitting(true). With the flag off, the generated classes are identical tomain.try_tablehandlers to the interpreter, since it changes behaviour even with the flag off; it can come as its own PR. I also left out the tuning and debug system properties and the benchmark harness.compiler/METHOD_SPLITTING.mddescribes how the splitting works.I'll keep driving this towards a merge. @kevodwyer, your review and any feedback would be very welcome, and so is help if you'd like to keep working on it.