[wasm][coreCLR] Switch to preemptive GC mode before DetachThread in RuntimeThreadShutdown

[pavelsavara]

Summary

On wasm (CoreCLR interpreter), a thread can reach RuntimeThreadShutdown → Thread::DetachThread(TRUE)
while still in cooperative GC mode. DetachThread requires preemptive mode and asserts it:

ASSERT FAILED
	Expression: !PreemptiveGCDisabled()
	Location:   src/coreclr/vm/threads.cpp:877
	Function:   DetachThread
Aborted(native code called abort())

This is a Debug/Checked-only _ASSERTE, so it aborts the process on Debug and Checked wasm/CoreCLR runtimes.
This change transitions the thread to preemptive before DetachThread on wasm, matching what the sibling
DestroyThread path already does.

Root cause

Managed code on wasm executes in the interpreter, which runs in cooperative GC mode. The thread transitions
into cooperative mode at the reverse-pinvoke / UnmanagedCallersOnly boundary in ExecuteInterpretedMethod
(src/coreclr/vm/prestub.cpp):

GCX_MAYBE_COOP(pInterpreterCode->Method->unmanagedCallersOnly);

[Copilot]

Pull request overview

This PR adjusts CoreCLR’s thread shutdown path on TARGET_WASM so that a thread transitions to preemptive GC mode before calling Thread::DetachThread(TRUE) in RuntimeThreadShutdown. This aligns with DetachThread’s requirement that the caller is not in cooperative mode (it asserts !PreemptiveGCDisabled()).

Changes:

• Add a #ifdef TARGET_WASM block in RuntimeThreadShutdown to enter preemptive GC mode via GCX_PREEMP_NO_DTOR() immediately before pThread->DetachThread(TRUE).
• Add a comment explaining why the wasm TLS-destructor-driven shutdown path can still be in cooperative mode and why switching is safe after resetting the frame to FRAME_TOP.

[pavelsavara]

Failure is known #130479

[jkotas]

for example when the process exits from managed code

What is the actual stacktrace how we end up here in cooperative mode?

[pavelsavara]

I think it's JS setTimeout() -> C# threadpool -> xunit last item -> Environment.Exit -> posix exit() -> emscripten thread destructor

Copilot thinks:

1. Native/JS → ExecuteInterpretedMethodFromUnmanaged(pMD, …) — prestub.cpp. These are the per-method reverse thunks generated in wasm/browser/callhelpers-reverse.cpp, e.g. CallEntryPoint (managed Main), CallJSExport, CallDelegate, CompleteTask (JS interop), class-ctor/default-ctor helpers, etc.
2. → ExecuteInterpretedMethodWithArgs — prestub.cpp
3. → ExecuteInterpretedMethod — prestub.cpp, which first asserts the caller is preemptive (if (thread->PreemptiveGCDisabled()) ReversePInvokeBadTransition();), then does the GCX_MAYBE_COOP transition above.
4. → InterpExecMethod runs the bytecode in MODE_COOPERATIVE (interpexec.cpp), toggling back to preemptive with GCX_PREEMP_NO_DTOR() only for outbound unmanaged/P-Invoke calls (interpexec.cpp).

I will try to catch the real native stack.

[pavelsavara]

This is one I can easily reproduce, but it's not the one which asserts.

RuntimeThreadShutdown(void*) (ceemain.cpp:1683)
TlsDestructionMonitor::~TlsDestructionMonitor() (ceemain.cpp:1812)
$__cxx_global_array_dtor
__cxxabiv1::(anonymous namespace)::run_dtors(void*) (cxa_thread_atexit.cpp:79)
__cxxabiv1::(anonymous namespace)::DtorsManager::~DtorsManager() (cxa_thread_atexit.cpp:102)
$__cxx_global_array_dtor
__funcs_on_exit (atexit.c:34)
(anonymous) (dotnet.native.93kurx3rql.js:565)
exitRuntime (dotnet.native.93kurx3rql.js:484)
exitJS (dotnet.native.93kurx3rql.js:6476)
abortPosix (dotnet.native.93kurx3rql.js:4135)
quitNow (dotnet.js:514)
exit (dotnet.js:502)
(anonymous) (main.js:33)

[jkotas]

Right, I would expect C runtime exit to be always called in preemptive mode. I think the problem should be fixed closer to where we are calling C runtime exit in cooperative mode.

[pavelsavara]

This is broken call with R2R into UCO which causes subsequent posix exit() but the GC mode is off-balance already.

RuntimeError: function signature mismatch
    at System.Runtime.InteropServices.JavaScript.Tests.9e6h66ejnv.wasm:0x1f0fa4
    at dotnet.native.wasm.(anonymous namespace)::CallFunc_This_RetI32_PE(unsigned long, signed char*, signed (anonymous namespace)::CallFunc_This_RetI32_PE(unsigned long, signed char*, signed char*) (callhelpers-interp-to-managed.cpp:180)
    at dotnet.native.wasm.InvokeCalliStub(unsigned long, void (*)(unsigned long, signed char*, signed char*), signed char*, signed char*, InvokeCalliStub(unsigned long, void (*)(unsigned long, signed char*, signed char*), signed char*, signed char*, Object**) (helpers.cpp:891)
    at dotnet.native.wasm.InvokeManagedMethod(MethodDesc*, signed char*, signed char*, unsigned long, InvokeManagedMethod(MethodDesc*, signed char*, signed char*, unsigned long, Object**) (helpers.cpp:1483)
    at dotnet.native.wasm.InterpExecMethod(InterpreterFrame*, InterpMethodContextFrame*, InterpThreadContext*, InterpExecMethod(InterpreterFrame*, InterpMethodContextFrame*, InterpThreadContext*, ExceptionClauseArgs*) (interpexec.cpp:3518)
    at ::ExecuteInterpretedMethod(TransitionBlock *, TADDR, void *) (prestub.cpp:2093)
    at dotnet.native.wasm.ExecuteInterpretedMethodWithArgs(unsigned long, signed char*, unsigned long, void*, unsigned ExecuteInterpretedMethodWithArgs(unsigned long, signed char*, unsigned long, void*, unsigned long) (prestub.cpp:2153)
    at ::ExecuteInterpretedMethodFromUnmanaged(MethodDesc *, int8_t *, size_t, int8_t *, PCODE) (prestub.cpp:2286)
    at Call_System_Private_CoreLib_System_Threading_ThreadPool_BackgroundJobHandler_Void_RetVoid() (callhelpers-reverse.cpp:38)
    at ::SystemJS_ExecuteBackgroundJobCallback() (callhelpers-reverse.cpp:43)

::DebugBreak() (debug.cpp:414)
Thread::DetachThread(int) (threads.cpp:877)
RuntimeThreadShutdown(void*) (ceemain.cpp:1683)
TlsDestructionMonitor::~TlsDestructionMonitor() (ceemain.cpp:1812)
$__cxx_global_array_dtor
__cxxabiv1::(anonymous namespace)::run_dtors(void*) (cxa_thread_atexit.cpp:79)
__cxxabiv1::(anonymous namespace)::DtorsManager::~DtorsManager() (cxa_thread_atexit.cpp:102)
$__cxx_global_array_dtor
__funcs_on_exit (atexit.c:34)
(anonymous) (dotnet.native.vp36jarcur.js:565)
exitRuntime (dotnet.native.vp36jarcur.js:484)
exitJS (dotnet.native.vp36jarcur.js:6534)
maybeExit (dotnet.native.vp36jarcur.js:6860)
callUserCallback (dotnet.native.vp36jarcur.js:6876)
(anonymous) (dotnet.native.vp36jarcur.js:6889)
setTimeout
safeSetTimeout (dotnet.native.vp36jarcur.js:6887)
_SystemJS_ScheduleBackgroundJob (dotnet.native.vp36jarcur.js:5391)

[pavelsavara]

Can InvokeCalliStub -> CallFunc_This_RetI32_PE catch that JS exception/trap ?

[jkotas]

RuntimeError: function signature mismatch

This should be fatal error that makes us abort immediately.