ship darwin x86_64 natives

[Random-Scientist]

Needs testing (waiting for docker...)
Fixes #64

[Random-Scientist]

For some reason rustc hangs while building the rapier3d fork

[ryanhcode]

do the darwin x86_64 natives actually build? I went without them because I stopped being able to build them after a certain nightly version a while ago iirc

[ryanhcode]

yep, hanging for me too on rapier compilation.

[Random-Scientist]

I tried updating to latest nightly, using this simba pr for some portable_simd changes

[patch.crates-io]
simba = { git = "https://github.com/aabizri/simba", rev = "cd656c434b55c59744d7c181cd8f44d1dabd0e8d" }

It builds on all the other platforms but still hangs when building for darwin x86_64.
Cloning your rapier fork and building it locally (x86_64 darwin host & target) everything is fine on latest nightly/release mode. Really strange.

[ryanhcode]

saw that simba PR but it wasn't merged so we pinned to the out-of-date nightly to avoid more git dependencies. Huh, really concerning if the rapier fork builds fine but it hangs with sable_rapier. Possibly zigbuild or another one of our configs is the thing causing it to hang?

[Random-Scientist]

I reran the build with RUSTC_LOG=trace and it gets just past codegen, then stops outputting anything after this. Is this an LLVM bug???

 INFO rustc_interface::passes Post-codegen
 Ty interner             total           ty lt ct all
     Adt               :  35573 18.8%,  5.3%   6.7%  0.3%  0.0%
     Array             :   3545  1.9%,  0.9%   0.4%  0.6%  0.0%
     Slice             :   1521  0.8%,  0.5%   0.2%  0.0%  0.0%
     RawPtr            :   1781  0.9%,  0.0%   0.0%  0.0%  0.0%
     Ref               :  90990 48.2%,  8.6%  29.3%  0.1%  0.0%
     FnDef             :  23346 12.4%,  0.6%   1.2%  0.0%  0.0%
     FnPtr             :   2257  1.2%,  0.1%   0.6%  0.0%  0.0%
     UnsafeBinder      :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
     Placeholder       :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
     Coroutine         :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
     CoroutineWitness  :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
     Dynamic           :   1170  0.6%,  0.0%   0.5%  0.0%  0.0%
     Closure           :   4399  2.3%,  0.3%   1.0%  0.0%  0.0%
     CoroutineClosure  :      0  0.0%,  0.0%   0.0%  0.0%  0.0%
     Tuple             :   8893  4.7%,  0.5%   2.5%  0.0%  0.0%
     Bound             :     80  0.0%,  0.0%   0.0%  0.0%  0.0%
     Param             :    587  0.3%,  0.0%   0.0%  0.0%  0.0%
     Infer             :   1298  0.7%,  0.7%   0.0%  0.0%  0.0%
     Alias             :  12992  6.9%,  3.6%   1.1%  0.0%  0.0%
     Pat               :    292  0.2%,  0.0%   0.0%  0.0%  0.0%
     Foreign           :      1  0.0%,  0.0%   0.0%  0.0%  0.0%
                   total 188725        21.1%  43.5%  1.1%  0.0%
 GenericArgs interner: #137977
 Region interner: #13118
 Const Allocation interner: #3830
 Layout interner: #2761

[Random-Scientist]

really concerning if the rapier fork builds fine but it hangs with sable_rapier. Possibly zigbuild or another one of our configs is the thing causing it to hang?

I think it may be the difference between x86_64_unknown_linux and x86_64_apple_darwin host rustc, which is not encouraging...

[Random-Scientist]

ideally someone would reproduce this on a real linux host using cargo-zigbuild on the rapier fork directly and grab some stack traces/take a spindump of what rustc is up to

[Random-Scientist]

i've got gdb in the container now, it seems like LLVM is just taking a really really long time to perform codegen (or has a loop somewhere). I'll try overriding the opt-level down a step, since slower natives are preferable to none. Really strange that this builds fine locally.

backtraces taken from after it hangs always contain
llvm::SelectionDAGISelLegacy::runOnMachineFunction(llvm::MachineFunction&)

[ryanhcode]

hm, intriguing. thank you for the investigation effort!

[Random-Scientist]

It builds fine when I remove codegen-units = 1 from the release profile. It seems like darwin just uses a huge number of codegen items for some reason? not entirely sure where this comes from but it builds in a sane amount of time now.

[Random-Scientist]

So... if i change to codegen-units = 2 everything magically works...

Going to run the full gradle task and mark this ready for review if it succeeds.

I also enabled thin LTO since LTO is off by default in release mode and usually translates to a nice performance win. (it would be nice if someone not on darwin could run the benchmarks before/after the change).

[ryanhcode]

could we increase codegen units just for darwin natives? to reduce the penalty for non-darwin natives

[Random-Scientist]

Done

[ryanhcode]

right then. If you stage all the changes except for the natives, I can squash merge and the workflow will build the natives and push them to main, and I'll include them in the next patch.

[ryanhcode]

still weirded out by the fact 1 codegen unit breaks darwin, but we'll go for it haha

[ryanhcode]

thank you so much for the help here, would've taken me ages to figure this out!