Windows process exit hooks are a bit special in that they occur after other thread have been killed and thus must be wait-free or otherwise tolerate killed threads. I ran an LLM audit specifically focusing on process exit and thread exit and it found a few issues.
LLM audit
Windows shutdown: issue list
Issues in mimalloc's Windows thread-exit and process-exit paths.
Citations are against branch dev3, HEAD 622d421f.
Every issue below has been reproduced or measured; each entry records what its
reproducer does and what it produced. Measured on Windows 11 Home
10.0.26200.8875, MSVC 14.51.36231 (x64 and x86), gcc 16.1.0 (MSYS2 mingw64),
CMake + Ninja, -DCMAKE_BUILD_TYPE=Release -DMI_BUILD_TESTS=OFF unless stated.
Reproducers are marked stock (unmodified mimalloc) or instrumented (a copy of
the tree carrying test-only hooks).
Failure models
hang — an unbounded wait at process exit. RtlExitUserProcess calls
ZwTerminateProcess(NULL, ...) to kill every other thread before any detach
handler runs, at whatever instruction each happened to be executing. By the time
mimalloc's hook runs the process is single-threaded, so a wait on a peer can
never be satisfied, and ntdll cannot mitigate a spin loop because it has no way
to know the loop is a wait.
The wedge is quiet, not hot: _mi_prim_thread_yield() is SwitchToThread(),
which relinquishes the quantum, so a spinning shutdown costs 12–20 % of one core
(measured 11.9 / 14.1 / 20.1 % across three reproducers). Nothing fires — not a
crash handler, not a CPU watchdog. Such a process is also unkillable:
NtTerminateProcess has already been called, so taskkill /F reports "There is
no running instance of the task" and Stop-Process -Force silently does
nothing. Observed wedged processes were still burning a quarter to a third of a
core many hours after being told to die; only a reboot clears them.
kill — taking an orphanable lock at process exit. Once
PEB->Ldr->ShutdownInProgress is set, RtlAcquireSRWLockExclusive and
RtlpWaitOnCriticalSection consult it and call NtTerminateProcess(-1, ...)
instead of waiting. The process does not hang — it dies mid-shutdown, skipping
every remaining detach handler: other DLLs' DLL_PROCESS_DETACH, the user's
atexit handlers, buffer flushes.
The exit code is not disturbed. RtlExitUserProcess latches the exit status
before LdrShutdownProcess runs, so the mitigation reuses it: exit(0) still
reports 0 and exit(7) still reports 7, in every kill observed across three
independent reproducers. The only symptoms are lost final output and skipped
handlers, with nothing pointing at mimalloc.
deadlock — deadlocking against the loader lock. On Windows 8+
LdrpLoaderLock is held for unload and thread/process shutdown, but not
for an attach-time DllMain reached through LoadLibrary. Probed on 26200 with
LdrLockLoaderLock(LDR_LOCK_LOADER_LOCK_FLAG_TRY_ONLY) from a helper thread:
DLL_PROCESS_DETACH, FreeLibrary and DLL_THREAD_DETACH all report the lock
held; DLL_PROCESS_ATTACH via LoadLibrary reports it not held — attach-time
serialisation moved to the load-owner / work-queue mechanism in that redesign.
The detach-path issues below are unaffected; the two init-path ones are hygiene
as a result.
bug — plain defect. hygiene — maintenance risk, no runtime consequence.
Which locks can be orphaned
Only three locks in the process cannot be orphaned — LdrpLoaderLock,
FastPebLock and RtlLockHeap(ProcessHeap) — because RtlExitUserProcess
takes them before the mass kill precisely to guarantee that. Every mi_lock_t
(an SRWLOCK, atomic.h:418-420) is fair game.
How exposed a build is depends on how many the teardown touches:
subproc.c:29-56 (_mi_meta_zalloc, _mi_meta_zalloc_aligned,
_mi_meta_rezalloc) take theap_meta_lock unconditionally — but on theap,
tld and arena creation, not on every malloc.
free.c:757-758 (mi_stat_free) takes theap_meta_lock too, but the whole
function is inside #if (MI_STAT>0), and MI_STAT is (MI_DEBUG>0 ? 2 : 0)
(types.h:82-89). In a default Release build that site does not exist.
arena.c:1354 (os_abandoned_pages_lock, in _mi_arenas_page_abandon) is
reached only for collect == MI_ABANDON (thread exit), and only for non-arena
pages.
threadlocal.c:231 (mi_thread_locals_lock) is the one lock a default
Release build reliably takes on the process-exit path outside the stats print
— see W9.
Measured with an unaided probabilistic reproducer (stockrace.c: ordinary
allocator traffic on six threads while the main thread exits; a .CRT$XLZ TLS
callback marks a completed shutdown):
| configuration |
shutdowns terminated early |
static, MIMALLOC_SHOW_STATS=1 |
2 / 230 ≈ 0.9 % (2/80 in one sweep, 0/150 in another) |
shared, MIMALLOC_SHOW_STATS=1 |
0 / 80 |
static, MIMALLOC_SHOW_STATS=1 MIMALLOC_DESTROY_ON_EXIT=1 |
19 / 60 ≈ 32 % |
That tracks the lock tracer: the default teardown acquires one lock outside the
stats print, the destroy_on_exit teardown acquires 29 across 7 sites. Orphaning
is the common case for destroy_on_exit=1, and rare-but-real otherwise.
Index
The When column says at what point in a process's life the issue can fire.
process exit covers both ExitProcess and, for the same code path, module
unload via FreeLibrary — W1 is the reason those two are
indistinguishable.
| # |
Issue |
Location |
When |
Severity |
| Root cause |
|
|
|
|
| W1 |
Process exit indistinguishable from FreeLibrary; lpReserved discarded |
prim.c:766 |
process exit |
kill |
| Unbounded spins |
|
|
|
|
| W2 |
_mi_heap_detach_theaps spins forever |
theap.c:381-412 |
mi_heap_delete; process exit w/ destroy_on_exit |
hang |
| W3 |
_mi_tld_detach_theaps spins forever, holding the loader lock |
theap.c:414-451 |
thread exit |
deadlock |
| W4 |
mi_bfield_atomic_clear_once_set busy-waits |
bitmap.c:112-129 |
mi_free; process exit w/ destroy_on_exit |
hang |
| Locks on the process-exit path |
|
|
|
|
| W5 |
Process-done entered under an SRWLOCK held across the whole teardown |
init.c:648 |
process exit |
kill |
| W6 |
_mi_theap_cached_set decrefs the cached theap |
init.c:599 |
process exit |
latent |
| W7 |
mi_theap_collect compiled in for static builds, not just debug |
init.c:605-612 |
process exit |
waste |
| W8 |
_mi_subprocs_unsafe_destroy_all — 7 lock sites, incl. the W2 spin and all of W10 |
init.c:622 |
process exit w/ destroy_on_exit |
kill + hang |
| W19 |
Shared build + destroy_on_exit truncates shutdown unconditionally |
init.c:621-622 |
process exit w/ destroy_on_exit |
bug |
| W9 |
Thread-locals teardown takes mi_thread_locals_lock |
init.c:626-627 |
process exit |
kill |
| W10 |
Stats printing takes heaps_lock, out_buf_lock, does console I/O and LoadLibrary |
init.c:633 |
process exit w/ show_stats/verbose |
kill |
| W11 |
_mi_tls_slots_done calls TlsFree — PEB lock |
init.c:638 |
process exit |
waste |
| W12 |
_mi_verbose_message takes out_buf_lock, does console I/O |
init.c:641 |
process exit w/ verbose |
kill (narrow) |
| W13 |
FLS mode calls FlsFree under the global ntdll FLS lock |
prim.c:1148-1152 |
process exit (MI_WIN_INIT=FLS) |
kill |
| Loader-lock violations |
|
|
|
|
| W14 |
LoadLibrary("psapi.dll") from inside a detach callback |
prim.c:609-616 |
process exit w/ show_stats/verbose |
deadlock |
| W15 |
AttachConsole/GetStdHandle from inside a detach callback |
prim.c:635-682 |
process exit; any error path |
deadlock |
| W16 |
_mi_deferred_free runs arbitrary user code under the loader lock |
theap.c:129 |
thread exit; process exit via W7 |
deadlock |
| W17 |
_mi_allocator_done cross-DLL call during the redirect DLL's own shutdown |
init.c:640 |
process exit |
deadlock |
| W18 |
GetModuleHandleExA from DllMain(DLL_PROCESS_ATTACH) |
prim.c:812-816 |
init |
hygiene |
| W20 |
LoadLibrary("bcrypt.dll") from DllMain(DLL_PROCESS_ATTACH) |
prim.c:718-730 |
init |
hygiene |
| Build defects |
|
|
|
|
| B1 |
MinGW + TLS_DLLMAIN + static does not compile |
prim.c:1076-1077 |
build |
bug |
| B2 |
32-bit ARM gets x86 symbol decoration |
prim.c:888 |
build |
bug |
| B3 |
Release + MI_DEBUG=ON does not compile with MSVC |
types.h:180-182 |
build |
bug |
| Hygiene |
|
|
|
|
| H1 |
Three near-identical section blocks copy-pasted |
prim.c:871-908, 977-1004, 1051-1078 |
build |
hygiene |
| H2 |
TLS registration depends on prim.obj being pulled from the .lib |
prim.c:871-908 |
build |
hygiene |
| H3 |
FLS mode carries three workarounds for bugs it causes itself |
init.c:556-561, prim.c:1140, 1148-1152 |
thread exit + process exit |
hygiene |
| H4 |
DisableThreadLibraryCalls silently has no effect for modules carrying mimalloc |
threadlocal.c:53-64 |
init; every thread create/exit |
hygiene |
mi_process_done_once branches at init.c:621 on mi_option_destroy_on_exit:
the then branch is W8, the else branch holds W9 and
W10, so those are mutually exclusive. But W8 reaches W9's and W10's work
inline anyway (subproc.c:222-224, subproc.c:243), so enabling
destroy_on_exit is strictly additive in hazard terms.
Issues
W1 — process exit indistinguishable from FreeLibrary · prim.c:766 · process exit · kill
mi_win_main (prim.c:765-777) does MI_UNUSED(reserved) on line 766, and
every call site passes lpReserved in and drops it (prim.c:821, 837, 842, 849, 855, 933, 956, 964, 1018, 1034, 1039, 1099).
The loader sets lpReserved non-NULL ((PVOID)1 in LdrpCallInitRoutine)
exactly when a detach is caused by process termination, and NULL when it is
caused by FreeLibrary. Those two cases need opposite behaviour. At
FreeLibrary the process is healthy, every thread is alive, and mimalloc
genuinely must release its memory or the unloading module leaks — the current
teardown is correct and its locking is necessary. At process exit the address
space is about to be destroyed wholesale, so none of that work has correctness
value, and every lock it takes is one a dead thread may be holding.
Because mimalloc cannot distinguish them, it runs the FreeLibrary teardown
during process exit. That is the root cause of W2 through W17: individually each
is a lock or a spin, but collectively they are one design decision — doing
unload work on a path where unload work is both pointless and unsafe.
lpReserved cannot supply the distinction in the default build. Only two of
the five hook shapes ever receive it, and the default is not one of them:
| mode |
hook |
receives lpReserved? |
MI_WIN_INIT_USE_CRT_TLS (the default, prim.c:789) |
.CRT$XLB/XLY TLS callbacks (prim.c:834, 846) + atexit |
no — always NULL |
MI_WIN_INIT_USE_RAW_DLLMAIN |
mi_dll_main_raw via _pRawDllMain (prim.c:932) |
yes |
MI_WIN_INIT_USE_TLS_DLLMAIN + MI_SHARED_LIB |
real DllMain (prim.c:1017) |
yes |
MI_WIN_INIT_USE_TLS_DLLMAIN static |
TLS callbacks (prim.c:1032, 1037) |
no |
MI_WIN_INIT_USE_FLS |
literal NULL at prim.c:1099 |
no |
Two call sites hard-code it: prim.c:821, where mi_crt_done() passes a literal
0 — and in a DLL built in the default mode that is the process-done route
(atexit(&mi_crt_done), prim.c:826-828) — and prim.c:1099, the FLS arm.
RtlDllShutdownInProgress() does discriminate correctly in every context,
including from inside a TLS callback. It is therefore the mechanism to use, not
a fallback.
Fix: resolve RtlDllShutdownInProgress once at init via
GetProcAddress(GetModuleHandle(TEXT("ntdll.dll")), "RtlDllShutdownInProgress")
and cache the pointer — documented on MS Learn though absent from the SDK
headers, and a plain read of PEB->Ldr->ShutdownInProgress, so the exit path
never re-enters the loader. Never call GetProcAddress/LoadLibrary at exit.
Expose it as an internal _mi_is_process_exiting(), then split
mi_process_done_once (init.c:591-643): the unload path keeps today's code
verbatim, and the exit path skips W6-W12 entirely, retaining only
_mi_subproc_main_done() (init.c:639), _mi_allocator_done() (init.c:640)
and os_preloading = true (init.c:642).
Reproducer (plain Win32, no mimalloc): a DLL carrying both a DllMain and a
.CRT$XLY TLS callback, unloaded by FreeLibrary in one run and left to
ExitProcess in the other. DllMain is the only hook that ever sees
lpReserved=1; every TLS callback sees NULL in both runs, while
RtlDllShutdownInProgress returns 0 and 1 respectively.
W2 — _mi_heap_detach_theaps spins forever · theap.c:381-412 · any mi_heap_delete; process exit with destroy_on_exit · hang
do { ... if (!all_detached) _mi_prim_thread_yield(); } while (!all_detached);
The loop body tries mi_lock_try_acquire(&tld->theaps_lock) (theap.c:391) for
each theap and only clears all_detached when every one succeeded; the yield is
at theap.c:409. The design assumes a contending thread will make progress and
release the lock shortly. At process exit that assumption is false: the holder
was terminated mid-critical-section and will never run again.
_mi_prim_thread_yield is SwitchToThread() (prim.c:752-754), so this is a
spin, not a sleep, and unlike an SRWLOCK acquisition it gets no
ShutdownInProgress mitigation. Measured at ~20 % of one core.
When it can fire. One caller only: heap.c:167 (mi_heap_free_theaps),
reached from mi_heap_delete (heap.c:236) and _mi_heap_force_destroy
(heap.c:243). The latter reaches mi_process_done_once only via
_mi_subprocs_unsafe_destroy_all → mi_subproc_unsafe_destroy
(subproc.c:218, 227), gated at init.c:621-623 on
mi_option_destroy_on_exit, default 0 (options.c:147). So at process exit it
needs that option; otherwise the route in is a user mi_heap_delete or
mi_heap_destroy, at any time.
The holder does not have to be dead. mimalloc itself invokes the user's
deferred-free callback while tld->theaps_lock is held (init.c:380 takes the
lock, init.c:386 → _mi_theap_collect_abandon → theap.c:129
_mi_deferred_free). A callback that blocks produces the same permanent hang in
a perfectly healthy process — that is W16 seen from this side.
Fix: bail out on _mi_is_process_exiting() (W1), or bound the retry
count and give up. Leaving a theap attached at process exit costs nothing —
the kernel reclaims the memory — whereas hanging costs the whole process.
Reproducer (stock, public API only): a worker parks inside a registered
deferred-free callback during its own thread exit, leaving tld->theaps_lock
held exactly as a killed thread would; main then calls mi_heap_delete. With an
infinite park it never returns; with a timed park it returns in the same
millisecond the lock is released — the causal control.
W3 — _mi_tld_detach_theaps spins forever, holding the loader lock · theap.c:414-451 · thread exit · deadlock
Structurally identical to W2 — mi_lock_try_acquire(&heap->theaps_lock)
at theap.c:426, yield at theap.c:447, loop closes at theap.c:450 — but it
runs where spinning is far more damaging.
This is a thread-exit issue only. One caller, init.c:400, inside
mi_thread_theaps_done, called only from _mi_thread_done (init.c:475).
mi_process_done_once never calls _mi_thread_done: its only candidate,
_mi_prim_thread_done_auto_done() (init.c:603), is an empty function in every
non-FLS mode (prim.c:800, 923, 1022, 1084), and in FLS mode the
thread-id guard at init.c:472-473 returns before mi_thread_theaps_done.
On Windows _mi_thread_done is reached only from DLL_THREAD_DETACH
(prim.c:775, prim.c:1187) and the FLS callback (prim.c:1139), so the
spinning thread holds the loader lock. theaps_lock is held by threads doing
mi_heap_delete/mi_heap_destroy (heap.c:162-185, theap.c:381-412). None of
the twelve theaps_lock acquisition sites runs loader code under the lock, so
the trigger is the generic one: any thread that stops making progress while
holding theaps_lock, of which a thread killed by ExitProcess is the canonical
case.
A spin here blocks ExitProcess itself. RtlExitUserProcess acquires the
loader lock before the mass thread kill, precisely so it cannot be orphaned —
so it waits for it. An unbounded wait inside _mi_thread_done therefore does not
merely hang the exiting thread: the whole process can no longer terminate, at
~0 % CPU, with every other thread still running normally. It also blocks every
LoadLibrary, FreeLibrary and thread creation meanwhile. Independent of
mi_option_destroy_on_exit.
Fix: as W2. The thread-exit case makes this mandatory independently of
any process-exit work.
Reproducer (instrumented — one test hook inside the existing
mi_lock(&heap->theaps_lock) block of mi_theap_attach, theap.c:299-306): an
ordinary thread parks holding heap->theaps_lock while a second thread exits and
spins on it under the loader lock; main's LoadLibraryA then never returns,
completing 16 ms after the lock is released in the timed-park control.
Separately, W2's stock reproducer blocks ExitProcess outright: 0.016 s CPU
over 10 s with peer threads still scheduling, unaffected by
MIMALLOC_DESTROY_ON_EXIT.
W4 — mi_bfield_atomic_clear_once_set busy-waits · bitmap.c:112-129 · any mi_free; process exit with destroy_on_exit · hang
while ((old & mask) == 0) { _mi_prim_thread_yield(); ... }
Loop at bitmap.c:122-125. This waits for another thread to set a bit before
clearing it — a handoff, not a lock — but the failure mode is the same: the
thread that was going to set the bit was killed before it got there. The comment
at bitmap.c:110-111 already acknowledges the cost in normal operation ("This
can incur a busy wait :-("); at shutdown it is unbounded. Measured at 14 % of
one core.
Where it is reached from. mi_bitmap_clear_once_set (bitmap.c:1426) is
called at arena.c:1415 inside _mi_arenas_page_unabandon (arena.c:1393),
which has four callers: free.c:376 and free.c:472 — the ordinary mi_free
path; arena.c:642 — the alloc/reclaim path; and arena.c:2547
(mi_heap_delete_page). It is not on the mi_theap_collect path:
mi_theap_collect_ex (theap.c:123-147) reaches _mi_theap_collect_retired,
mi_theap_visit_pages and _mi_arenas_collect → mi_arenas_try_purge
(arena.c:2398), none of which unabandons. So this sits on free(), which
anything can be doing when the process starts exiting.
The exact window. The bit is not merely "set by another thread" — it is
temporarily cleared and then re-set by a searcher:
mi_bitmap_try_find_and_claim_visit (bitmap.c:1340-1364) clears the abandoned
bit at bitmap.c:1346, calls mi_arena_try_claim_abandoned (arena.c:655),
which fails when a concurrent free already owns the page and sets
*keep_abandoned = true (arena.c:664), so the searcher must set the bit again
at bitmap.c:1358. arena.c's own comment at arena.c:660-664 says it outright:
"it is very important to set the abandoned bit again (or otherwise the unabandon
will never stop waiting)". A thread killed between bitmap.c:1346 and
bitmap.c:1358 strands the freeing thread forever — a two-instruction window on
a hot path, not a shutdown-only concern.
Fix: as W2.
Reproducer (instrumented — two ordering hooks in src/arena.c, one in the
failed-claim branch of mi_arena_try_claim_abandoned and one immediately before
the busy wait; neither touches bitmap state) stages exactly that interleaving. A
plain mi_free never returns; with a timed park it returns 15 ms after the
searcher sets the bit back.
W5 — process-done entered under a held SRWLOCK · init.c:648, libc.c:115-134 · process exit · kill
mi_process_done wraps the whole of mi_process_done_once in
mi_atomic_do_once (atomic.h:555-557), which calls _mi_atomic_once_enter
(libc.c:115-134) → mi_lock_acquire(&once->lock) at libc.c:125 — an
AcquireSRWLockExclusive — and holds it until _mi_atomic_once_release
(libc.c:136-142) at the very end.
So before any of W6-W12 gets a chance to fail, the entry itself can fail: if
another thread was inside a mi_atomic_do_once on the same object when it was
killed, this acquisition triggers the NtTerminateProcess(-1) mitigation. The
tid-based recursion guard at libc.c:121-123 only handles re-entry by the same
thread and does nothing about a dead holder.
Scope. For this once object to be orphaned, another thread must have been
inside mi_process_done / _mi_auto_process_done when it stopped.
mi_process_done is exported (mimalloc.h:207), so it is reachable, but it is
not the common case. _mi_atomic_once_enter returns at libc.c:117-118
without touching the lock once a once has completed, so the other nine
mi_atomic_do_once sites in the process are harmless at exit. The exposure is
confined to onces that have not completed, of which init.c:648's is by
construction one — and, on the stats path, prim.c:609-616's (W14).
The static bool process_done guard at init.c:595-597 is not a substitute: it
is an unsynchronised read-modify-write, so removing the once in its favour would
let two threads run mi_process_done_once concurrently.
Fix: replace the mi_atomic_do_once with a bare
mi_atomic_cas_strong_acq_rel on an atomic word — a single CAS with no wait,
which keeps the concurrency guarantee without the orphanable lock.
Reproducer (stock, public API only): a thread calls the exported
mi_process_done() and parks inside the deferred-free callback that
mi_theap_collect (init.c:610) invokes, orphaning once->lock; main then
exit(7)s. A .CRT$XLZ TLS callback standing in for later shutdown work prints
in the control run and not in the orphaned one — and the exit code is 7 either
way, in 0.5 s, not a hang.
W6 — _mi_theap_cached_set decrefs the cached theap · init.c:599 · process exit · latent
The first real statement of the teardown. Path: prim-tls.c:211-229 calls
_mi_tls_slots_init() (prim-tls.c:215), then _mi_theap_incref/
_mi_theap_decref (theap.c:364-370); a decref to zero would call
mi_theap_free_mem (theap.c:347-355) → _mi_meta_free (subproc.c:73-82).
It does not reach a lock, in any configuration, for two independent reasons:
- The decref never reaches zero.
theap->refcount is initialised to 1 by
its owner (theap.c:245) and the cache adds a second reference
(theap.c:358-362). At init.c:599 the owner reference is still
outstanding, so the decref goes 2→1. Making it fire would need a cached theap
whose owner has already released it, which the teardown ordering prevents —
init.c:396 clears the cache first.
_mi_meta_free takes no lock. subproc.c:73-82 is mi_free(p) for
MI_MEM_MALLOC memids (what _mi_meta_zalloc produces) or _mi_arenas_free
otherwise. The only theap_meta_lock in the free path is mi_stat_free at
free.c:757-758, inside #if (MI_STAT>0) — absent from every default
Release build.
The mi_atomic_do_once at prim-tls.c:215 has already completed here, so
_mi_atomic_once_enter returns at libc.c:117 before mi_lock_acquire;
W5 does not apply either.
Evidence: instrumented tracing, against a workload whose main thread's last
allocation comes from a mi_heap_new() heap so the cached theap is dynamically
allocated. The function is entered and reaches the decref, but
mi_theap_free_mem is never called and no lock is acquired — in Release static,
Release shared, MI_DEBUG, FLS, or with destroy_on_exit=1.
Fix: still pointless work at process exit — resetting the cached theap matters
only if the process continues running — so it belongs in the set skipped by
W1's exit path. It is not an independent hazard.
W7 — mi_theap_collect compiled in for static builds, not just debug · init.c:605-612 · process exit · waste
The guard is not debug-only:
#ifndef MI_SKIP_COLLECT_ON_EXIT
#if (MI_DEBUG || !defined(MI_SHARED_LIB))
mi_theap_collect(_mi_theap_default(), true /* force */);
MI_SKIP_COLLECT_ON_EXIT defaults OFF (CMakeLists.txt:47) and MI_SHARED_LIB
is added only to the shared target (CMakeLists.txt:815), so a forced collect of
every page runs at process exit in every static build — a very common
configuration. A per-statement phase tracer confirms it: present in Release
static, absent in Release shared, present in both once MI_DEBUG=ON. The work is
pointless there — collecting and purging pages hands memory back to an address
space about to be destroyed.
What it does and does not touch. mi_theap_collect (theap.c:154-156) →
mi_theap_collect_ex (theap.c:123-147) reaches _mi_theap_collect_retired
(page.c:501), mi_theap_visit_pages, and _mi_arenas_collect
(arena.c:1502). Across Release-static, Release-shared, MI_DEBUG and FLS builds — with worker
threads, extra heaps and leaked blocks to produce abandoned pages, and under
MIMALLOC_DISALLOW_ARENA_ALLOC, MIMALLOC_PAGE_RECLAIM_ON_FREE and
MIMALLOC_ARENA_MAX_OBJECT_SIZE=64 — this statement acquires zero
mi_lock_t and enters neither the bitmap.c:122 busy-wait nor the
theap.c:409 spin. Specifically:
os_abandoned_pages_lock at arena.c:1354 is inside
_mi_arenas_page_abandon, reached only for collect == MI_ABANDON (thread
exit), not MI_FORCE.
arena.c:1423 is in _mi_arenas_page_unabandon, called from mi_free
(W4), not collect.
arena.c:2511 and arena.c:2632 are on the heap-destroy path — W8;
the tracer shows arena.c:2511 in the W8 phase.
pmap->lock (page-map.c:389) is only taken when a submap is absent
(mi_page_map_ensure_submap_at, page-map.c:414-427); collect only
unregisters pages, where the submap necessarily exists.
mi_arenas_try_purge (arena.c:2398) uses mi_atomic_guard
(atomic.h:397-401) — a try-once CAS, not a spin. Orphaning it silently
disables purging for the process lifetime; it cannot hang.
The real hazard on this statement is not a lock but a callout:
mi_theap_collect_ex's first act is _mi_deferred_free at theap.c:129, which
runs arbitrary user code on the process-exit path — see W16. That is also
what makes this statement usable as the staging mechanism in W5's
reproducer.
Fix: skip on the process-exit path (W1). Keep it for the unload path,
where returning the memory is the whole point.
W8 — _mi_subprocs_unsafe_destroy_all · init.c:622 · process exit with destroy_on_exit · kill + hang
subproc.c:265-278 takes mi_subprocs_lock to walk the subprocess list, then
for each one mi_subproc_unsafe_destroy takes that subprocess's heaps_lock
(subproc.c:214) and calls _mi_page_map_unsafe_destroy (page-map.c:367-383)
— which unmaps the page map that other (already dead, but that is not knowable
here) threads may have been reading.
This is the then branch of init.c:621, gated on mi_option_destroy_on_exit,
default 0 (options.c:147). Independent of static/shared and MI_DEBUG.
It is the largest single exposure in the teardown. The lock tracer logs 29
acquisitions across 7 distinct sites for a trivial 3-thread program, identical
in static and shared: mi_subprocs_lock (subproc.c:266, 207), heaps_lock
(subproc.c:214, heap.c:194), heap->theaps_lock (theap.c:385,
heap.c:170), tld->theaps_lock (theap.c:391 — the W2 spin loop),
os_abandoned_pages_lock (arena.c:2511), and mi_thread_locals_lock
(threadlocal.c:309, 231). It re-runs all of W9 inline
(subproc.c:222-224) and calls mi_subproc_stats_print_out itself at
subproc.c:243, so it carries every hazard of W10 too. That is why it is
kill + hang rather than kill alone. As with W7, all of it is work the kernel
is about to do for free.
Measured exposure: 19/60 ≈ 32 % of stockrace.c runs had their shutdown
terminated early, against ≈ 0.9 % for the default configuration. Deterministic
reproduction is lockkill.c (W10) with MIMALLOC_DESTROY_ON_EXIT=1.
Fix: skip on the process-exit path (W1).
W19 — shared build + destroy_on_exit truncates shutdown unconditionally · init.c:621-622 · process exit with destroy_on_exit · bug
In the shared build, MIMALLOC_DESTROY_ON_EXIT=1 truncates shutdown even in
a single-threaded, fully benign run with nothing contended: the EXE's
post-mimalloc TLS DLL_PROCESS_DETACH callback never runs, and neither
mimalloc: process done nor the marker appears — while the same binary without
the option prints both. Exit code is 0 either way.
This is not W8's orphaned-lock mechanism — there is nothing to orphan here
— so _mi_subprocs_unsafe_destroy_all is destroying state that mimalloc's own
remaining teardown, or the loader's subsequent detach sequence, still needs. It
also means the shared + destroy_on_exit combination cannot be used to measure
W8's probabilistic exposure.
Fix: bisect which of mi_subproc_unsafe_destroy's steps (subproc.c:214-247)
causes the truncation. _mi_page_map_unsafe_destroy (page-map.c:367-383)
unmapping the page map is the obvious suspect, since anything still calling
free afterwards would fault.
W9 — thread-locals teardown takes mi_thread_locals_lock · init.c:626-627 · process exit · kill
Two adjacent statements in the else branch of init.c:621:
_mi_thread_locals_thread_done() (threadlocal.c:205-214) → _mi_meta_free,
and _mi_thread_locals_done() (threadlocal.c:230-241) →
mi_thread_locals_lock at threadlocal.c:231.
mi_thread_locals_lock guards the list of all threads' local blocks, so it is
held briefly by every thread that starts or stops — a realistic orphaning
candidate. It is taken in every configuration: default Release static,
default Release shared, MI_DEBUG, FLS. That makes it the one lock a default
Release build reliably acquires on the process-exit path outside the stats print,
and the reason the default configuration has any probabilistic exposure at all.
The theap_meta_lock reachable through _mi_meta_free → mi_free →
mi_stat_free (free.c:757-758) is debug-only, since that function is
inside #if (MI_STAT>0). The tracer shows threadlocal.c:231 alone in default
Release static, and free.c:758 appearing in both statements once MI_DEBUG=ON.
_mi_thread_local_free (threadlocal.c:309) takes the same lock and is reached
from W8.
Fix: skip on the process-exit path (W1).
W10 — stats printing · init.c:633 · process exit with show_stats/verbose · kill
mi_subproc_stats_print_out is the worst single statement in the teardown
because it combines four independent hazards:
heaps_lock — stats.c:512-519 → mi_subproc_stats_get
(stats.c:640-649) → mi_subproc_visit_heaps (subproc.c:303-314) walks
every heap under the subprocess lock, acquired at subproc.c:307.
- Console I/O —
_mi_prim_out_stderr (prim.c:635-682). Writing to a
console is an ALPC round-trip to conhost, which can block on a process not
participating in the shutdown, and it touches the CRT's own stdio locking.
See W15.
LoadLibrary — _mi_stats_print (stats.c:356) →
mi_process_info_print_out (stats.c:427) → mi_process_info
(stats.c:568) → _mi_prim_process_info → psapi at prim.c:611, see
W14.
out_buf_lock — options.c:372, around 35 acquisitions per stats dump.
Plus libc.c:125: the mi_atomic_do_once guarding the psapi load (W5 +
W14) is acquired live, at exit, since that once has never run before.
Any one of these is disqualifying; together they make this the statement most
likely to be the thing that actually kills a real process.
When it runs. init.c:629 gates it on mi_option_show_stats or
mi_option_verbose, both default 0. The option check precedes any lock
acquisition, so a default-configured process does not reach this statement at
all. Independent of static/shared/MI_DEBUG. Also reached unconditionally from
W8 via subproc.c:243-247.
Both API hazards were confirmed live. With
MIMALLOC_SHOW_STATS=1, MIMALLOC_VERBOSE=0, the first mimalloc output of the
whole process is emitted inside this statement, so _mi_prim_out_stderr's
cached hcon is uninitialised and GetStdHandle — and, with no stderr handle,
AttachConsole — really is first called from inside DLL_PROCESS_DETACH. And
psapi.dll is not loaded before exit in any configuration, so the LoadLibrary
at prim.c:611 genuinely happens during detach.
Fix: skip on the process-exit path (W1). Users who want stats at exit
should get them from the EXE atexit slot instead — the one hook that runs
before ExitProcess, while every thread is still alive and ordinary locking,
console I/O and even LoadLibrary are all safe.
Reproducers. lockkill.c (stock, deterministic, static and shared,
100 %): a helper thread parks inside a mi_subproc_visit_heaps visitor — which
mimalloc invokes while holding subproc->heaps_lock (subproc.c:307) — and is
SuspendThreaded there, exactly the state ExitProcess's mass kill produces.
Main exits; the stats block, mimalloc: process done, and a .CRT$XLZ TLS
callback standing in for later shutdown work all vanish (95 → 60 lines of
output) while the exit code stays 0x00000000. Against the instrumented build
the phase log ends on exactly that acquisition, where the control run continues
past the same lock address. stockrace.c is the unaided probabilistic version —
hit rates under Which locks can be orphaned.
One stockrace run under destroy_on_exit=1 wedged permanently instead —
single-threaded, accruing CPU, immune to taskkill /F, alive hours later. Its
sampled RIP sat at ntdll!NtDelayExecution+0x14, not mimalloc's
SwitchToThread spin (NtYieldExecution is at RVA 0x160910), so it is most
likely ntdll's own RtlExitUserProcess wait-for-threads loop. Recorded as an
observed stock shutdown hang of unknown origin.
W11 — _mi_tls_slots_done calls TlsFree · init.c:638 · process exit · waste
_mi_tls_slots_done (prim-tls.c:135-138 for the Win32 TLS model; also
prim-tls.c:172-177 pthreads, 194-196 fixed, 205-207 none) calls
mi_win_tls_slot_free (prim-tls.c:114-121) → TlsFree at prim-tls.c:118 →
RtlAcquirePebLock. Two calls per process. Runs in every configuration; the
TlsFree body only under MI_TLS_MODEL_WIN32, the default on Windows.
Confirmed at runtime with an instrumented TlsFree site, identically in static,
shared and FLS builds. No mi_lock_t is involved.
The PEB lock is one of the three that RtlExitUserProcess deliberately protects
from orphaning — it takes it before the mass kill for exactly this reason — so in
the normal ExitProcess sequence this will not be held by a dead thread, and no
kill is possible through it.
Still worth removing: it is pure waste (the kernel reclaims TLS indices with the
process), it is an API call on a path that should have none, and the protection
depends on going through RtlExitUserProcess, which not every shutdown path
does.
Fix: skip on the process-exit path (W1).
W12 — _mi_verbose_message · init.c:641 · process exit with verbose · kill (narrow)
options.c:524-530 → mi_vfprintf → _mi_fputs (options.c:466) → the
default output function mi_out_buf_stderr (options.c:402-405) →
mi_out_stderr, then mi_out_buf (options.c:365) → out_buf_lock, declared
at options.c:363 and acquired at options.c:372, then out through
_mi_prim_out_stderr and the same console I/O as W15. The default
output really is mi_out_buf_stderr at exit (mi_add_stderr_output,
options.c:434).
This is the last statement of the teardown, which makes it easy to overlook and
easy to remove. Two conditions bound how often it matters:
_mi_verbose_message early-returns at options.c:525 unless
mi_option_verbose is enabled, default 0. In a default process this
statement takes no lock and does no console I/O at all — the tracer's W12
phase is empty without MIMALLOC_VERBOSE=1, and shows exactly one
options.c:372 acquisition with it.
mi_out_buf returns before taking the lock once
out_len >= MI_MAX_DELAY_OUTPUT (16 KiB) — options.c:368. So
out_buf_lock is only ever taken during the first 16 KiB of
mimalloc-generated output in the process's lifetime; a chatty build saturates
the delay buffer early and stops taking the lock altogether. Console I/O via
mi_out_stderr continues regardless, so the W15 exposure remains even
when the lock is skipped.
The acquisition is at the same lock address as the ~35 inside the W10 phase,
confirming it is out_buf_lock. The window is one acquisition of a lock held
only across a memcpy, so no dedicated reproducer was built; lockkill.c
demonstrates the mechanism on the same class of lock.
Fix: skip on the process-exit path (W1).
W13 — FLS mode calls FlsFree · prim.c:1148-1152 · process exit (MI_WIN_INIT=FLS) · kill
_mi_prim_thread_done_auto_done (prim.c:1148-1152) calls FlsFree(mi_fls_key)
at prim.c:1151, added to fix the dangling-callback-after-unload problem
(issue #208).
FlsFree takes ntdll's global FLS lock, the most dangerous lock in the shutdown
sequence: RtlProcessFlsData runs at the very top of LdrShutdownProcess,
before any DLL_PROCESS_DETACH, so the FLS lock is the first lock acquired after
the mass thread kill. It is also process-global, so contention is not limited to
mimalloc's own threads.
FlsFree additionally invokes the registered callback for every thread that
still has a value, from the calling thread — so it can drive _mi_thread_done on
behalf of threads that no longer exist. mimalloc guards this with the thread-id
check at init.c:472-473, but the guard is a symptom of the mechanism being
wrong for this job.
Only affects MI_WIN_INIT=FLS, deprecated and not the default (prim.c:785-793
selects CRT_TLS, or TLS_DLLMAIN for Intel ICX/ICC). It does still build and
run, static and shared. See H3.
Reproducers. flsforeign.c (instrumented, FLS build): a worker thread
still alive at ExitProcess leaves a theap in its FLS slot, and the instrumented
trace shows FlsFree at prim.c:1151 followed by mi_fls_done (prim.c:1136)
running on the main thread for that already-terminated worker's theap —
init.c:472-473 is what stops it tearing down a foreign tld. When all workers
exit normally their slots are already NULLed by mi_fls_done and the main
thread's at init.c:560, so FlsFree finds nothing; the hazard appears
specifically with threads that outlive main. A mimalloc-free control confirms
the Win32 semantics directly: FlsFree runs the callback on the calling thread
for every thread still holding a value.
W14 — LoadLibrary("psapi.dll") from a detach callback · prim.c:609-616 · process exit with show_stats/verbose · deadlock
Inside _mi_prim_process_info (prim.c:598-629):
mi_atomic_do_once {
HINSTANCE hDll = mi_win_loadlibrary(TEXT("psapi.dll")); // prim.c:611
if (hDll != NULL) {
pGetProcessMemoryInfo = (PGetProcessMemoryInfo)(void (*)(void))GetProcAddress(hDll, "GetProcessMemoryInfo");
}
}
Called from the stats path (W10), which runs from a DLL_PROCESS_DETACH
callback — with LdrpLoaderLock held and LdrpShutdownInProgress set.
LoadLibrary is at the top of the documented forbidden list for DllMain/TLS
callbacks: it re-enters the loader recursively, and loading a new module during
shutdown means running that module's DLL_PROCESS_ATTACH after the loader has
already begun detaching everything. Being wrapped in mi_atomic_do_once it also
carries W5's lock — and that once has never completed before, so the lock
is genuinely acquired.
psapi on the CMake link line (CMakeLists.txt:717) does not make this
harmless. Nothing references a psapi symbol — mimalloc resolves
GetProcessMemoryInfo by GetProcAddress — so no import is created:
dumpbin /dependents on the stock mimalloc.dll shows no psapi.dll, and neither
does a DLL statically linking mimalloc.lib with psapi.lib explicitly on the
link line. mimalloc genuinely pulls a fresh module into the process from inside
DLL_PROCESS_DETACH.
Applies to any build in CRT_TLS (default) or RAW_DLLMAIN/TLS_DLLMAIN,
static or shared, whenever mi_option_show_stats or mi_option_verbose is set
and destroy_on_exit is off. With neither option set, psapi is never loaded.
A note on probing this. Real process exit cannot be probed for loader-lock
ownership — RtlExitUserProcess kills the probe helper thread first. W14 and
W15 are therefore demonstrated on the FreeLibrary unload path, where
mi_process_done_once runs from DLL_PROCESS_DETACH under the loader lock with
all threads alive: per W1 the same teardown code, minus only the "peer
threads are dead" hazard.
Fix: resolve the psapi entry points at init and cache them, or skip stats at
process exit (W10), which removes the reachability entirely. Doing both
is cheap.
Reproducer (stock mimalloc statically linked into a DLL, unloaded by
FreeLibrary): GetModuleHandleA("psapi.dll") returns NULL at main entry,
while mimalloc runs, and at the moment the detach handler starts — where the
loader-lock probe reports held — then non-NULL once the teardown returns, so
psapi was loaded inside it under the loader lock. Without
MIMALLOC_SHOW_STATS=1 all four readings are NULL. The same run shows
bcrypt.dll already loaded at DLL_PROCESS_ATTACH — that is W20.
W15 — AttachConsole/GetStdHandle from a detach callback · prim.c:635-682 · process exit; any error path · deadlock
_mi_prim_out_stderr calls GetStdHandle (prim.c:645, 649), and on failure
AttachConsole(ATTACH_PARENT_PROCESS) (prim.c:648) then
GetConsoleScreenBufferInfo (prim.c:652). Attaching to a console performs ALPC
to conhost, and console I/O touches the CRT's own stdio locking — neither is
appropriate from a detach callback.
Reachable from W10, W12, and from any error or warning path. The
error path cannot be removed (mimalloc must be able to report a fatal condition),
but W10 and W12 can, which is most of the exposure.
hcon is a function-local static (prim.c:640), initialised on the first
call — so these APIs run wherever the process's first stderr output happens. If
any verbose or warning message was emitted earlier, hcon is already cached and
the exit path only does WriteConsoleA/WriteFile. The dangerous case is
precisely "the first output is the exit-time stats dump", the default when only
MIMALLOC_SHOW_STATS=1 is set. The AttachConsole branch additionally needs
GetStdHandle(STD_ERROR_HANDLE) == NULL.
With AttachConsole forced to run under the loader lock, an
LdrRegisterDllNotification trace shows no module loaded by it, and the call
returned without blocking (GetConsoleWindow() still NULL). So the demonstrated
hazard is console I/O and console-API calls from a detach callback; loader
re-entry through AttachConsole specifically is a plausible risk rather than an
observed one.
Fix: remove the W10/W12 callers. For the error path, consider caching the
stderr handle at init so the exit path only ever does WriteFile.
Reproducer (instrumented, markers around prim.c:645/648/652): the host
calls FreeConsole() and SetStdHandle(STD_ERROR_HANDLE, NULL) before
FreeLibrary, forcing the first _mi_prim_out_stderr down the AttachConsole
branch inside DLL_PROCESS_DETACH. The log shows GetStdHandle,
AttachConsole and GetConsoleScreenBufferInfo all inside the teardown window;
the loader-notification log records only psapi.dll loaded there
(W14).
W16 — _mi_deferred_free under the loader lock · theap.c:129, page.c:990-999 · thread exit; process exit via W7 · deadlock
_mi_theap_collect_abandon (theap.c:150-152) calls
mi_theap_collect_ex(theap, MI_ABANDON), whose first statement is
_mi_deferred_free(theap, force) at theap.c:129 — invoking whatever callback
the user registered through mi_register_deferred_free. Since
MI_ABANDON = 2 >= MI_FORCE = 1 (theap.c:90-94), the callback is invoked with
force = true on thread exit.
Reached from _mi_thread_done (init.c:451) → mi_thread_theaps_done (call at
init.c:476, function at init.c:376-390), which on Windows runs from the
.CRT$XLY TLS DLL_THREAD_DETACH callback (registration prim.c:880-883,
mi_tls_detach prim.c:846-858 → mi_win_main prim.c:765 →
_mi_thread_done(NULL) prim.c:775) — with the loader lock held. The user's
callback is arbitrary code: it may allocate, take its own locks, or call
LoadLibrary. Nothing in mimalloc's public API warns that this can happen, so a
callback that is perfectly reasonable in every other context can deadlock the
process on Windows only, and only on thread exit. It also runs on the
process-exit path, via mi_theap_collect at init.c:610 (W7).
Two guards sit on the path, neither of which blocks it: _mi_thread_done returns
early if the theap is uninitialised (init.c:459) or on a thread-id mismatch
(init.c:473), and _mi_deferred_free has a tld->recurse guard
(page.c:993-997). A thread that never allocated is unaffected.
The .CRT$XLY route is not the only one. mi_win_main skips
_mi_thread_done when redirected (prim.c:774, && !_mi_is_redirected()), so
in the default shared build with mimalloc-redirect.dll the call arrives via
_mi_redirect_entry(DLL_THREAD_DETACH) (prim.c:1185-1188) — from the
redirector's own DllMain, also under the loader lock. Confirmed static and
shared, x64 and x86, redirected and not.
Fix: at minimum document that a deferred-free callback may be invoked with the
loader lock held on Windows. Better: suppress the call for the MI_ABANDON case
in theap.c:129 — thread teardown is not a point where user code needs to run.
That also closes the hang described in W2.
Reproducers. A probe callback reports, from inside itself, that it holds the
loader lock on the worker's thread exit. deadlock.c (stock, static link, no
instrumentation) then produces the real thing, 3/3 runs, exit=124: the
worker's thread-exit callback holds the loader lock and waits on a plain
CRITICAL_SECTION, while main holds that CRITICAL_SECTION and waits on the
loader lock inside LoadLibraryA("winmm.dll"). Nothing exotic on either side;
the only surprise is that mimalloc hands control to user code with the loader
lock held.
W17 — _mi_allocator_done cross-DLL call · init.c:640, prim.c:1198-1200 · process exit · deadlock
A call into mimalloc-redirect.dll (active when
MI_SHARED_LIB && !MI_WIN_NOREDIRECT, guard block prim.c:1165-1201, import
declared at prim.c:1191, body mi_allocator_done(); at prim.c:1199), made
from inside mimalloc's own detach handler. The redirect DLL is a separate module
with its own detach handler, which may already have run or may be running — and
its code is opaque, so nothing here can be verified.
Listed for completeness, not to argue for removal: this is the one process-exit
step worth keeping. Without it the redirector carries on routing free into
an allocator that has been torn down, which is worse than the risk of the call.
Gate it only if it proves problematic in practice.
The redirector is genuinely active by default — dumpbin /dependents on the
stock mimalloc.dll lists mimalloc-redirect.dll as a static import — and the
call returns normally: a shared-build program reports mi_is_redirected() = 1,
exits 0, and under MIMALLOC_VERBOSE=1 still prints mimalloc: process done,
which is init.c:641, i.e. after _mi_allocator_done() at init.c:640.
W18 — GetModuleHandleExA from DllMain(DLL_PROCESS_ATTACH) · prim.c:812-816 · init · hygiene
mi_current_module_is_dll() (prim.c:812-816, GetModuleHandleExA at 814) is
called from mi_crt_init (prim.c:824-831, call at 826) — which in a DLL
build runs from _DllMainCRTStartup(DLL_PROCESS_ATTACH), where
GetModuleHandle* is on the documented forbidden list for DllMain.
Safe in practice, for three separate reasons.
GET_MODULE_HANDLE_EX_FLAG_UNCHANGED_REFCOUNT means the call cannot trigger an
unload; it is on the init path, so none of the shutdown hazards apply; and on
Windows 8+ an attach-time DllMain reached through LoadLibrary does not hold
LdrpLoaderLock at all — measured on 26200, where a helper thread successfully
acquired the lock while the main thread sat inside DllMain(DLL_PROCESS_ATTACH),
though the same probe reports it held for DLL_PROCESS_DETACH/FreeLibrary
and DLL_THREAD_DETACH. The process-startup attach
(LdrpInitializeProcess, for a statically imported mimalloc.dll) cannot be
probed this way — the helper thread does not exist yet — so that case is
unmeasured.
Applies to MI_WIN_INIT=CRT_TLS only, the default: the FLS arm has its own
mi_crt_init (prim.c:1098-1102) which does not call it, and
RAW_DLLMAIN/TLS_DLLMAIN have no mi_crt_init at all. Static and shared. It
is listed because it is entirely avoidable and the codebase already has the
better primitive.
Fix: use mi_module_is_dll((PVOID)&__ImageBase). __ImageBase is supplied by
link.exe, lld-link and GNU ld, and mi_module_is_dll (prim.c:805-810) just
reads the PE header from the given base with no API call at all — exactly what
the exit path already uses (prim.c:853).
Reproducer: mi_module_is_dll and mi_current_module_is_dll copied verbatim
from prim.c:805-816 and called side by side, from DllMain(DLL_PROCESS_ATTACH)
in a DLL and from main in an EXE. __ImageBase and the GetModuleHandleExA
base are identical and both is_dll answers agree (1/1 in the DLL, 0/0 in the
EXE); a gcc build confirms __ImageBase resolves identically under GNU ld.
W20 — LoadLibrary("bcrypt.dll") from DllMain(DLL_PROCESS_ATTACH) · prim.c:718-730 · init · hygiene
_mi_prim_random_buf (prim.c:718-730) does:
mi_atomic_do_once {
HINSTANCE hDll = mi_win_loadlibrary(TEXT("bcrypt.dll")); // prim.c:722
if (hDll != NULL) {
pBCryptGenRandom = (PBCryptGenRandom)(void (*)(void))GetProcAddress(hDll, "BCryptGenRandom");
}
}
This runs during mimalloc's initialization — from DllMain(DLL_PROCESS_ATTACH)
or the .CRT$XIB init callback — and pulls in bcrypt.dll plus
bcryptprimitives.dll. A nested LoadLibrary from DllMain is a larger
forbidden-API violation than W18's GetModuleHandleExA, and unlike
W14 it is unconditional: no option gates it, because mimalloc always
needs randomness at init. Observed in every run — W14's reproducer shows
bcrypt.dll already loaded by the time DllMain(DLL_PROCESS_ATTACH) returns,
and the instrumented build logs the call at attach time.
Rated hygiene rather than deadlock for the same reason as W18: on Windows 8+ an
attach-time DllMain reached through LoadLibrary does not hold
LdrpLoaderLock. The process-startup attach path (LdrpInitializeProcess) was
not measurable and remains unknown, and on pre-Win8 loaders this is a genuine
recursive-loader-entry hazard.
Fix: mimalloc already has MI_USE_RTLGENRANDOM (prim.c:705-707), which uses
RtlGenRandom with no LoadLibrary at all. Otherwise, defer the bcrypt
resolution to the first use of the RNG rather than doing it at init. Worth an
explicit decision either way, since it is currently unconditional.
B1 — MinGW + TLS_DLLMAIN + static does not compile · prim.c:1076-1077 · build · bug
The MinGW arm of the MI_WIN_INIT_USE_TLS_DLLMAIN static branch registers
&mi_tls_attach and &mi_tls_detach in .CRT$XLB/.CRT$XLY, at prim.c:1076
and prim.c:1077. Neither symbol exists in that #elif block — the functions it
defines are mi_win_main_attach (prim.c:1032) and mi_win_main_detach
(prim.c:1037). mi_tls_attach/mi_tls_detach are defined only in the
CRT_TLS arm (prim.c:834, 846) and the RAW_DLLMAIN arm (prim.c:953,
961), both mutually exclusive with this one.
This is the mode forced for Intel ICX/ICC (prim.c:786-787, issue #1268), so
the broken configuration is not purely hypothetical, though
Intel-compiler-plus-MinGW is an unlikely pairing. The equivalent MinGW arm in
RAW_DLLMAIN mode (prim.c:1002-1003) uses the same two names and is correct,
because that branch does define them — the names were copied between blocks
without being renamed, see H1.
Real compile error with gcc 16.1.0:
gcc -c -O2 -Iinclude -Isrc -DMI_WIN_INIT_USE_TLS_DLLMAIN=1 src/prim/prim.c
src/prim/windows/prim.c:1076:93: error: 'mi_tls_attach' undeclared here (not in a function)
src/prim/windows/prim.c:1077:93: error: 'mi_tls_detach' undeclared here (not in a function)
Only this one cell of the matrix fails; CRT_TLS, RAW_DLLMAIN and FLS all
build static and shared, and TLS_DLLMAIN shared is fine because the
#elif …&& defined(MI_SHARED_LIB) arm at prim.c:1013 wins.
Fix: rename to mi_win_main_attach/mi_win_main_detach in that block, or
collapse the nine blocks per H1 so the names cannot drift again.
B2 — 32-bit ARM gets x86 symbol decoration · prim.c:888 · build · bug
The section-registration code has two MSVC branches:
#if defined(_WIN64) && defined(_MSC_VER) at prim.c:871, and
#elif defined(_MSC_VER) // 32-bit at prim.c:888 — so the effective predicate
for the second is defined(_MSC_VER) && !defined(_WIN64). The 32-bit branch adds
an extra leading underscore to every /INCLUDE: symbol (prim.c:889-892:
__tls_used, __mi_tls_callback_pre, __mi_tls_callback_post,
__mi_crt_callback_init), correct for x86 because MSVC decorates cdecl C symbols
with a leading _ there.
But _M_ARM (32-bit ARM MSVC) also satisfies that predicate — _WIN64 is
defined only for 64-bit targets, _MSC_VER is defined, and there is no _M_ARM
test anywhere in the block. ARM32 uses undecorated names. On that target the
pragmas would name symbols that do not exist, the /INCLUDE: directives would be
no-ops, and /OPT:REF would discard the callback arrays — leaving mimalloc with
no thread-exit or process-exit hooks at all, silently.
The same defect is replicated at prim.c:989-992 and prim.c:1063-1066, and in
the FLS arm's #else // 32-bit at prim.c:1114-1115. Practically a dead
platform, so this is latent rather than live. The correct predicate is
defined(_M_IX86). See H1.
The decoration rule was confirmed on the two targets available: one TU built for
x64 and x86 and dumped with dumpbin /symbols gives _mi_tls_callback_pre and
__mi_tls_callback_pre respectively. The predicate is correct for x86 — a 32-bit
build under vcvars32 compiles cleanly, has a Thread Storage Directory, and its
TLS hooks fire on thread exit — so the bug is confined to ARM32.
B3 — Release + MI_DEBUG=ON does not compile with MSVC · types.h:180-182 · build · bug
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DMI_DEBUG=ON
include/mimalloc/types.h(182): fatal error C1188: '#warning' requires '/std:c++23preview' or later
The #if MI_DEBUG && NDEBUG diagnostic at types.h:180-182 uses #warning,
which MSVC only accepts under /std:c++23preview. Since mimalloc's CMake
compiles the sources as C++ by default (MI_USE_CXX=ON), any
Release-plus-MI_DEBUG configuration fails outright rather than emitting the
intended warning. This is the configuration W7's MI_DEBUG arm needs, so
it also blocks testing that path; the workaround is
-DCMAKE_CXX_FLAGS_RELEASE="-O2 -Ob2", which drops NDEBUG.
Fix: use #pragma message for MSVC, or guard the #warning on a compiler
check.
H1 — three near-identical section blocks copy-pasted · prim.c:871-908, 977-1004, 1051-1078 · build · hygiene
Each of the three init modes (CRT_TLS, RAW_DLLMAIN, TLS_DLLMAIN)
hand-writes the same three-way toolchain split:
| target |
_tls_used |
callback arrays |
| MSVC / clang-cl, x64 & ARM64 |
/INCLUDE:_tls_used |
#pragma const_seg(".CRT$XLB") + /INCLUDE:_mi_tls_callback_pre etc. |
| MSVC / clang-cl, x86 |
/INCLUDE:__tls_used |
#pragma data_seg(...), extra leading underscore |
| MinGW / GCC |
extern const IMAGE_TLS_DIRECTORY _tls_used; + a used reference |
__attribute__((used, section(".CRT$XLB"))) |
Nine near-identical blocks, differing only in symbol names. A diff -u of the
three ranges:
- 871-908 vs 977-1004 — the only differences are the four
_mi_crt_callback_init / .CRT$XIB lines that exist solely in the CRT_TLS
block, plus one space of whitespace. Everything else is byte-identical.
- 977-1004 vs 1051-1078 — differences are indentation only, plus the
callback names mi_tls_attach/mi_tls_detach →
mi_win_main_attach/mi_win_main_detach. The #elif defined(__MINGW32__)
arms are byte-identical between the two — which is exactly B1: the
rename was applied to the two MSVC arms and missed on the MinGW one.
The x64 row covers ARM64 too, since _WIN64 is defined there, so ARM64 takes the
undecorated branch and is fine; only ARM32 is broken (B2). So B1 (wrong
names carried across a copy) and B2 (wrong predicate replicated three times) are
both direct consequences, and a future fix to one block will very likely miss the
other two.
Fix: collapse to one macro parameterised on the attach/detach/init function
names, using __pragma(comment(linker, "/INCLUDE:" MI_SYM(x))) for the MSVC
family and _Pragma for GCC/clang GNU mode, with a single decoration rule
(#if defined(_M_IX86) → _ prefix, else none).
H2 — TLS registration depends on prim.obj being linked in · prim.c:871-908 · build · hygiene
The #pragma comment(linker, "/INCLUDE:...") directives (prim.c:872-875 for
x64, 889-892 for x86) are emitted into prim.obj. A linker only processes
directives from object files it actually pulls out of a static .lib, and it
only pulls an object if something already references a symbol in it.
They live in exactly one object: dumpbin /directives on the static-lib build's
src/prim/prim.c.obj shows all four /INCLUDE: directives, and scanning the
.drectve text of all 19 objects in that build gives exactly one hit. No other TU
emits them, so the coupling is real and unenforced.
Today it always holds — _mi_prim_alloc, _mi_prim_commit, _mi_prim_thread_id
and around thirty other OS primitives live in that same TU, and every build
references them. But nothing enforces it. If that TU is ever split for tidiness,
static builds would silently lose all thread-exit and process-exit hooks: no
link error, no warning, just a mimalloc that never runs _mi_thread_done or
_mi_process_done, and very hard to attribute. (That consequence follows from
documented link.exe archive-extraction behaviour plus the measurement above;
producing it would require splitting the TU.)
Fix: record the dependency in a comment at minimum; better, add a link-time
assertion or keep a deliberate reference from a TU that is always pulled in.
H3 — FLS mode carries three workarounds for its own bugs · thread exit + process exit · hygiene
Three separate fixes, all for problems the FLS mechanism creates:
| citation |
comment |
init.c:556-561 |
the main thread's FLS value is forced to NULL immediately after process init — "…FLS cleanup happens to early for the main thread… See issue #508." |
prim.c:1140 |
the callback nulls its own slot before returning — "prevent recursion as _mi_thread_done may set it back to the main theap, issue #672" |
prim.c:1148-1152, FlsFree at 1151 |
"…prevent dangling callback pointer if statically linked with a DLL; Issue #208" — this is itself W13 |
Beyond the workarounds, FLS is structurally wrong for an allocator: its callbacks
run first in both LdrShutdownThread and LdrShutdownProcess, before every
DllMain(DLL_THREAD_DETACH), before every TLS callback, and therefore before all
C++ thread_local destructors — so user destructors that free() run after
mimalloc has torn the thread down. It can also fire on the wrong thread (via
FlsFree, demonstrated in W13, and via orphan reaping on recent Windows),
and there are only 128 FLS slots per process.
Reproducer (one source built against the default and FLS libs): a worker thread
with a C++ thread_local whose destructor calls mi_free, with mimalloc's own
teardown observed through a deferred-free callback. Under the default CRT_TLS
the destructor runs first and mimalloc's teardown second; under FLS the order
inverts. The main thread shows the correct order in both builds — the
init.c:556-561 workaround (issue #508) doing its job. The inverted free()
does not fault in this simple case; the defect is the ordering.
Fix: consider marking MI_WIN_INIT=FLS for removal. Its only unique coverage —
the main thread at process exit, and orphan reaping after unclean exits — is
already handled by the DLL_PROCESS_DETACH path (prim.c:851-857) plus
atexit, and the default .CRT$XLY placement gets the ordering right for free.
H4 — DisableThreadLibraryCalls silently has no effect · threadlocal.c:53-64 · init; every thread create/exit · hygiene
mi_thread_locals and mi_slot_fast are declared mi_decl_thread
(internal.h:38 = __declspec(thread)), instantiated at threadlocal.c:63-64
via the #else // Direct thread locals arm at threadlocal.c:53-64. On
Windows/MSVC the #if MI_TLS_MODEL_PTHREADS || defined(__APPLE__) at
threadlocal.c:44 is false, so that is the arm taken, and any module carrying
mimalloc gets a TLS directory (confirmed with dumpbin /headers: a DLL
statically linking mimalloc has one, an otherwise identical DLL without mimalloc
has none).
Windows documents DisableThreadLibraryCalls as failing for any module with
active static TLS. On Windows 11 26200 it does something worse than fail: it
returns TRUE with GetLastError() == 0, and the
DLL_THREAD_ATTACH/DLL_THREAD_DETACH notifications keep being delivered
(the control DLL returns TRUE and genuinely stops receiving them).
So any DLL that statically links mimalloc keeps receiving — and paying for — the
per-thread notifications it believes it turned off, and checking the return
value does not reveal it: a silent per-thread cost the host cannot see by any
local means. Identical whether the DLL is LoadLibrary'd or bound at load
time.
This is not load-bearing from mimalloc's side in the default mode. In
CRT_TLS mimalloc's hooks are .CRT$XLB/.CRT$XLY TLS callbacks
(prim.c:876-883), which the loader runs for every module with a TLS directory
regardless of LDRP_DONT_CALL_FOR_THREADS — DisableThreadLibraryCalls could
never have suppressed them, and mimalloc's thread teardown still runs when the
call returns TRUE. Only RAW_DLLMAIN and shared TLS_DLLMAIN, which route
through DllMain, depend on the notifications at all. Static TLS is a side effect
imposed on the host, not a mechanism mimalloc relies on.
Fix: document it in the Windows section of the readme — as "returns success but
has no effect", not "fails".
Windows process exit hooks are a bit special in that they occur after other thread have been killed and thus must be wait-free or otherwise tolerate killed threads. I ran an LLM audit specifically focusing on process exit and thread exit and it found a few issues.
LLM audit
Windows shutdown: issue list
Issues in mimalloc's Windows thread-exit and process-exit paths.
Citations are against branch
dev3, HEAD622d421f.Every issue below has been reproduced or measured; each entry records what its
reproducer does and what it produced. Measured on Windows 11 Home
10.0.26200.8875, MSVC 14.51.36231 (x64 and x86), gcc 16.1.0 (MSYS2 mingw64),
CMake + Ninja,
-DCMAKE_BUILD_TYPE=Release -DMI_BUILD_TESTS=OFFunless stated.Reproducers are marked stock (unmodified mimalloc) or instrumented (a copy of
the tree carrying test-only hooks).
Failure models
hang — an unbounded wait at process exit.
RtlExitUserProcesscallsZwTerminateProcess(NULL, ...)to kill every other thread before any detachhandler runs, at whatever instruction each happened to be executing. By the time
mimalloc's hook runs the process is single-threaded, so a wait on a peer can
never be satisfied, and ntdll cannot mitigate a spin loop because it has no way
to know the loop is a wait.
The wedge is quiet, not hot:
_mi_prim_thread_yield()isSwitchToThread(),which relinquishes the quantum, so a spinning shutdown costs 12–20 % of one core
(measured 11.9 / 14.1 / 20.1 % across three reproducers). Nothing fires — not a
crash handler, not a CPU watchdog. Such a process is also unkillable:
NtTerminateProcesshas already been called, sotaskkill /Freports "There isno running instance of the task" and
Stop-Process -Forcesilently doesnothing. Observed wedged processes were still burning a quarter to a third of a
core many hours after being told to die; only a reboot clears them.
kill — taking an orphanable lock at process exit. Once
PEB->Ldr->ShutdownInProgressis set,RtlAcquireSRWLockExclusiveandRtlpWaitOnCriticalSectionconsult it and callNtTerminateProcess(-1, ...)instead of waiting. The process does not hang — it dies mid-shutdown, skipping
every remaining detach handler: other DLLs'
DLL_PROCESS_DETACH, the user'satexithandlers, buffer flushes.The exit code is not disturbed.
RtlExitUserProcesslatches the exit statusbefore
LdrShutdownProcessruns, so the mitigation reuses it:exit(0)stillreports 0 and
exit(7)still reports 7, in every kill observed across threeindependent reproducers. The only symptoms are lost final output and skipped
handlers, with nothing pointing at mimalloc.
deadlock — deadlocking against the loader lock. On Windows 8+
LdrpLoaderLockis held for unload and thread/process shutdown, but notfor an attach-time
DllMainreached throughLoadLibrary. Probed on 26200 withLdrLockLoaderLock(LDR_LOCK_LOADER_LOCK_FLAG_TRY_ONLY)from a helper thread:DLL_PROCESS_DETACH,FreeLibraryandDLL_THREAD_DETACHall report the lockheld;
DLL_PROCESS_ATTACHviaLoadLibraryreports it not held — attach-timeserialisation moved to the load-owner / work-queue mechanism in that redesign.
The detach-path issues below are unaffected; the two init-path ones are hygiene
as a result.
bug — plain defect. hygiene — maintenance risk, no runtime consequence.
Which locks can be orphaned
Only three locks in the process cannot be orphaned —
LdrpLoaderLock,FastPebLockandRtlLockHeap(ProcessHeap)— becauseRtlExitUserProcesstakes them before the mass kill precisely to guarantee that. Every
mi_lock_t(an
SRWLOCK,atomic.h:418-420) is fair game.How exposed a build is depends on how many the teardown touches:
subproc.c:29-56(_mi_meta_zalloc,_mi_meta_zalloc_aligned,_mi_meta_rezalloc) taketheap_meta_lockunconditionally — but on theap,tld and arena creation, not on every
malloc.free.c:757-758(mi_stat_free) takestheap_meta_locktoo, but the wholefunction is inside
#if (MI_STAT>0), andMI_STATis(MI_DEBUG>0 ? 2 : 0)(
types.h:82-89). In a default Release build that site does not exist.arena.c:1354(os_abandoned_pages_lock, in_mi_arenas_page_abandon) isreached only for
collect == MI_ABANDON(thread exit), and only for non-arenapages.
threadlocal.c:231(mi_thread_locals_lock) is the one lock a defaultRelease build reliably takes on the process-exit path outside the stats print
— see W9.
Measured with an unaided probabilistic reproducer (
stockrace.c: ordinaryallocator traffic on six threads while the main thread exits; a
.CRT$XLZTLScallback marks a completed shutdown):
MIMALLOC_SHOW_STATS=1MIMALLOC_SHOW_STATS=1MIMALLOC_SHOW_STATS=1 MIMALLOC_DESTROY_ON_EXIT=1That tracks the lock tracer: the default teardown acquires one lock outside the
stats print, the
destroy_on_exitteardown acquires 29 across 7 sites. Orphaningis the common case for
destroy_on_exit=1, and rare-but-real otherwise.Index
The When column says at what point in a process's life the issue can fire.
process exitcovers bothExitProcessand, for the same code path, moduleunload via
FreeLibrary— W1 is the reason those two areindistinguishable.
FreeLibrary;lpReserveddiscardedprim.c:766_mi_heap_detach_theapsspins forevertheap.c:381-412mi_heap_delete; process exit w/destroy_on_exit_mi_tld_detach_theapsspins forever, holding the loader locktheap.c:414-451mi_bfield_atomic_clear_once_setbusy-waitsbitmap.c:112-129mi_free; process exit w/destroy_on_exitinit.c:648_mi_theap_cached_setdecrefs the cached theapinit.c:599mi_theap_collectcompiled in for static builds, not just debuginit.c:605-612_mi_subprocs_unsafe_destroy_all— 7 lock sites, incl. the W2 spin and all of W10init.c:622destroy_on_exitdestroy_on_exittruncates shutdown unconditionallyinit.c:621-622destroy_on_exitmi_thread_locals_lockinit.c:626-627heaps_lock,out_buf_lock, does console I/O andLoadLibraryinit.c:633show_stats/verbose_mi_tls_slots_donecallsTlsFree— PEB lockinit.c:638_mi_verbose_messagetakesout_buf_lock, does console I/Oinit.c:641verboseFlsFreeunder the global ntdll FLS lockprim.c:1148-1152MI_WIN_INIT=FLS)LoadLibrary("psapi.dll")from inside a detach callbackprim.c:609-616show_stats/verboseAttachConsole/GetStdHandlefrom inside a detach callbackprim.c:635-682_mi_deferred_freeruns arbitrary user code under the loader locktheap.c:129_mi_allocator_donecross-DLL call during the redirect DLL's own shutdowninit.c:640GetModuleHandleExAfromDllMain(DLL_PROCESS_ATTACH)prim.c:812-816LoadLibrary("bcrypt.dll")fromDllMain(DLL_PROCESS_ATTACH)prim.c:718-730TLS_DLLMAIN+ static does not compileprim.c:1076-1077prim.c:888Release+MI_DEBUG=ONdoes not compile with MSVCtypes.h:180-182prim.c:871-908, 977-1004, 1051-1078prim.objbeing pulled from the.libprim.c:871-908init.c:556-561,prim.c:1140, 1148-1152DisableThreadLibraryCallssilently has no effect for modules carrying mimallocthreadlocal.c:53-64mi_process_done_oncebranches atinit.c:621onmi_option_destroy_on_exit:the then branch is W8, the else branch holds W9 and
W10, so those are mutually exclusive. But W8 reaches W9's and W10's work
inline anyway (
subproc.c:222-224,subproc.c:243), so enablingdestroy_on_exitis strictly additive in hazard terms.Issues
W1 — process exit indistinguishable from
FreeLibrary·prim.c:766· process exit · killmi_win_main(prim.c:765-777) doesMI_UNUSED(reserved)on line 766, andevery call site passes
lpReservedin and drops it (prim.c:821, 837, 842, 849, 855, 933, 956, 964, 1018, 1034, 1039, 1099).The loader sets
lpReservednon-NULL ((PVOID)1inLdrpCallInitRoutine)exactly when a detach is caused by process termination, and NULL when it is
caused by
FreeLibrary. Those two cases need opposite behaviour. AtFreeLibrarythe process is healthy, every thread is alive, and mimallocgenuinely must release its memory or the unloading module leaks — the current
teardown is correct and its locking is necessary. At process exit the address
space is about to be destroyed wholesale, so none of that work has correctness
value, and every lock it takes is one a dead thread may be holding.
Because mimalloc cannot distinguish them, it runs the
FreeLibraryteardownduring process exit. That is the root cause of W2 through W17: individually each
is a lock or a spin, but collectively they are one design decision — doing
unload work on a path where unload work is both pointless and unsafe.
lpReservedcannot supply the distinction in the default build. Only two ofthe five hook shapes ever receive it, and the default is not one of them:
lpReserved?MI_WIN_INIT_USE_CRT_TLS(the default,prim.c:789).CRT$XLB/XLYTLS callbacks (prim.c:834,846) +atexitMI_WIN_INIT_USE_RAW_DLLMAINmi_dll_main_rawvia_pRawDllMain(prim.c:932)MI_WIN_INIT_USE_TLS_DLLMAIN+MI_SHARED_LIBDllMain(prim.c:1017)MI_WIN_INIT_USE_TLS_DLLMAINstaticprim.c:1032,1037)MI_WIN_INIT_USE_FLSprim.c:1099Two call sites hard-code it:
prim.c:821, wheremi_crt_done()passes a literal0— and in a DLL built in the default mode that is the process-done route(
atexit(&mi_crt_done),prim.c:826-828) — andprim.c:1099, the FLS arm.RtlDllShutdownInProgress()does discriminate correctly in every context,including from inside a TLS callback. It is therefore the mechanism to use, not
a fallback.
Fix: resolve
RtlDllShutdownInProgressonce at init viaGetProcAddress(GetModuleHandle(TEXT("ntdll.dll")), "RtlDllShutdownInProgress")and cache the pointer — documented on MS Learn though absent from the SDK
headers, and a plain read of
PEB->Ldr->ShutdownInProgress, so the exit pathnever re-enters the loader. Never call
GetProcAddress/LoadLibraryat exit.Expose it as an internal
_mi_is_process_exiting(), then splitmi_process_done_once(init.c:591-643): the unload path keeps today's codeverbatim, and the exit path skips W6-W12 entirely, retaining only
_mi_subproc_main_done()(init.c:639),_mi_allocator_done()(init.c:640)and
os_preloading = true(init.c:642).Reproducer (plain Win32, no mimalloc): a DLL carrying both a
DllMainand a.CRT$XLYTLS callback, unloaded byFreeLibraryin one run and left toExitProcessin the other.DllMainis the only hook that ever seeslpReserved=1; every TLS callback seesNULLin both runs, whileRtlDllShutdownInProgressreturns 0 and 1 respectively.W2 —
_mi_heap_detach_theapsspins forever ·theap.c:381-412· anymi_heap_delete; process exit withdestroy_on_exit· hangThe loop body tries
mi_lock_try_acquire(&tld->theaps_lock)(theap.c:391) foreach theap and only clears
all_detachedwhen every one succeeded; the yield isat
theap.c:409. The design assumes a contending thread will make progress andrelease the lock shortly. At process exit that assumption is false: the holder
was terminated mid-critical-section and will never run again.
_mi_prim_thread_yieldisSwitchToThread()(prim.c:752-754), so this is aspin, not a sleep, and unlike an SRWLOCK acquisition it gets no
ShutdownInProgressmitigation. Measured at ~20 % of one core.When it can fire. One caller only:
heap.c:167(mi_heap_free_theaps),reached from
mi_heap_delete(heap.c:236) and_mi_heap_force_destroy(
heap.c:243). The latter reachesmi_process_done_onceonly via_mi_subprocs_unsafe_destroy_all→mi_subproc_unsafe_destroy(
subproc.c:218,227), gated atinit.c:621-623onmi_option_destroy_on_exit, default 0 (options.c:147). So at process exit itneeds that option; otherwise the route in is a user
mi_heap_deleteormi_heap_destroy, at any time.The holder does not have to be dead. mimalloc itself invokes the user's
deferred-free callback while
tld->theaps_lockis held (init.c:380takes thelock,
init.c:386→_mi_theap_collect_abandon→theap.c:129_mi_deferred_free). A callback that blocks produces the same permanent hang ina perfectly healthy process — that is W16 seen from this side.
Fix: bail out on
_mi_is_process_exiting()(W1), or bound the retrycount and give up. Leaving a
theapattached at process exit costs nothing —the kernel reclaims the memory — whereas hanging costs the whole process.
Reproducer (stock, public API only): a worker parks inside a registered
deferred-free callback during its own thread exit, leaving
tld->theaps_lockheld exactly as a killed thread would; main then calls
mi_heap_delete. With aninfinite park it never returns; with a timed park it returns in the same
millisecond the lock is released — the causal control.
W3 —
_mi_tld_detach_theapsspins forever, holding the loader lock ·theap.c:414-451· thread exit · deadlockStructurally identical to W2 —
mi_lock_try_acquire(&heap->theaps_lock)at
theap.c:426, yield attheap.c:447, loop closes attheap.c:450— but itruns where spinning is far more damaging.
This is a thread-exit issue only. One caller,
init.c:400, insidemi_thread_theaps_done, called only from_mi_thread_done(init.c:475).mi_process_done_oncenever calls_mi_thread_done: its only candidate,_mi_prim_thread_done_auto_done()(init.c:603), is an empty function in everynon-FLS mode (
prim.c:800,923,1022,1084), and in FLS mode thethread-id guard at
init.c:472-473returns beforemi_thread_theaps_done.On Windows
_mi_thread_doneis reached only fromDLL_THREAD_DETACH(
prim.c:775,prim.c:1187) and the FLS callback (prim.c:1139), so thespinning thread holds the loader lock.
theaps_lockis held by threads doingmi_heap_delete/mi_heap_destroy(heap.c:162-185,theap.c:381-412). None ofthe twelve
theaps_lockacquisition sites runs loader code under the lock, sothe trigger is the generic one: any thread that stops making progress while
holding
theaps_lock, of which a thread killed byExitProcessis the canonicalcase.
A spin here blocks
ExitProcessitself.RtlExitUserProcessacquires theloader lock before the mass thread kill, precisely so it cannot be orphaned —
so it waits for it. An unbounded wait inside
_mi_thread_donetherefore does notmerely hang the exiting thread: the whole process can no longer terminate, at
~0 % CPU, with every other thread still running normally. It also blocks every
LoadLibrary,FreeLibraryand thread creation meanwhile. Independent ofmi_option_destroy_on_exit.Fix: as W2. The thread-exit case makes this mandatory independently of
any process-exit work.
Reproducer (instrumented — one test hook inside the existing
mi_lock(&heap->theaps_lock)block ofmi_theap_attach,theap.c:299-306): anordinary thread parks holding
heap->theaps_lockwhile a second thread exits andspins on it under the loader lock;
main'sLoadLibraryAthen never returns,completing 16 ms after the lock is released in the timed-park control.
Separately, W2's stock reproducer blocks
ExitProcessoutright: 0.016 s CPUover 10 s with peer threads still scheduling, unaffected by
MIMALLOC_DESTROY_ON_EXIT.W4 —
mi_bfield_atomic_clear_once_setbusy-waits ·bitmap.c:112-129· anymi_free; process exit withdestroy_on_exit· hangLoop at
bitmap.c:122-125. This waits for another thread to set a bit beforeclearing it — a handoff, not a lock — but the failure mode is the same: the
thread that was going to set the bit was killed before it got there. The comment
at
bitmap.c:110-111already acknowledges the cost in normal operation ("Thiscan incur a busy wait :-("); at shutdown it is unbounded. Measured at 14 % of
one core.
Where it is reached from.
mi_bitmap_clear_once_set(bitmap.c:1426) iscalled at
arena.c:1415inside_mi_arenas_page_unabandon(arena.c:1393),which has four callers:
free.c:376andfree.c:472— the ordinarymi_freepath;
arena.c:642— the alloc/reclaim path; andarena.c:2547(
mi_heap_delete_page). It is not on themi_theap_collectpath:mi_theap_collect_ex(theap.c:123-147) reaches_mi_theap_collect_retired,mi_theap_visit_pagesand_mi_arenas_collect→mi_arenas_try_purge(
arena.c:2398), none of which unabandons. So this sits onfree(), whichanything can be doing when the process starts exiting.
The exact window. The bit is not merely "set by another thread" — it is
temporarily cleared and then re-set by a searcher:
mi_bitmap_try_find_and_claim_visit(bitmap.c:1340-1364) clears the abandonedbit at
bitmap.c:1346, callsmi_arena_try_claim_abandoned(arena.c:655),which fails when a concurrent free already owns the page and sets
*keep_abandoned = true(arena.c:664), so the searcher must set the bit againat
bitmap.c:1358. arena.c's own comment atarena.c:660-664says it outright:"it is very important to set the abandoned bit again (or otherwise the unabandon
will never stop waiting)". A thread killed between
bitmap.c:1346andbitmap.c:1358strands the freeing thread forever — a two-instruction window ona hot path, not a shutdown-only concern.
Fix: as W2.
Reproducer (instrumented — two ordering hooks in
src/arena.c, one in thefailed-claim branch of
mi_arena_try_claim_abandonedand one immediately beforethe busy wait; neither touches bitmap state) stages exactly that interleaving. A
plain
mi_freenever returns; with a timed park it returns 15 ms after thesearcher sets the bit back.
W5 — process-done entered under a held SRWLOCK ·
init.c:648,libc.c:115-134· process exit · killmi_process_donewraps the whole ofmi_process_done_onceinmi_atomic_do_once(atomic.h:555-557), which calls_mi_atomic_once_enter(
libc.c:115-134) →mi_lock_acquire(&once->lock)atlibc.c:125— anAcquireSRWLockExclusive— and holds it until_mi_atomic_once_release(
libc.c:136-142) at the very end.So before any of W6-W12 gets a chance to fail, the entry itself can fail: if
another thread was inside a
mi_atomic_do_onceon the same object when it waskilled, this acquisition triggers the
NtTerminateProcess(-1)mitigation. Thetid-based recursion guard at
libc.c:121-123only handles re-entry by the samethread and does nothing about a dead holder.
Scope. For this once object to be orphaned, another thread must have been
inside
mi_process_done/_mi_auto_process_donewhen it stopped.mi_process_doneis exported (mimalloc.h:207), so it is reachable, but it isnot the common case.
_mi_atomic_once_enterreturns atlibc.c:117-118without touching the lock once a once has completed, so the other nine
mi_atomic_do_oncesites in the process are harmless at exit. The exposure isconfined to onces that have not completed, of which
init.c:648's is byconstruction one — and, on the stats path,
prim.c:609-616's (W14).The
static bool process_doneguard atinit.c:595-597is not a substitute: itis an unsynchronised read-modify-write, so removing the once in its favour would
let two threads run
mi_process_done_onceconcurrently.Fix: replace the
mi_atomic_do_oncewith a baremi_atomic_cas_strong_acq_relon an atomic word — a single CAS with no wait,which keeps the concurrency guarantee without the orphanable lock.
Reproducer (stock, public API only): a thread calls the exported
mi_process_done()and parks inside the deferred-free callback thatmi_theap_collect(init.c:610) invokes, orphaningonce->lock; main thenexit(7)s. A.CRT$XLZTLS callback standing in for later shutdown work printsin the control run and not in the orphaned one — and the exit code is 7 either
way, in 0.5 s, not a hang.
W6 —
_mi_theap_cached_setdecrefs the cached theap ·init.c:599· process exit · latentThe first real statement of the teardown. Path:
prim-tls.c:211-229calls_mi_tls_slots_init()(prim-tls.c:215), then_mi_theap_incref/_mi_theap_decref(theap.c:364-370); a decref to zero would callmi_theap_free_mem(theap.c:347-355) →_mi_meta_free(subproc.c:73-82).It does not reach a lock, in any configuration, for two independent reasons:
theap->refcountis initialised to 1 byits owner (
theap.c:245) and the cache adds a second reference(
theap.c:358-362). Atinit.c:599the owner reference is stilloutstanding, so the decref goes 2→1. Making it fire would need a cached theap
whose owner has already released it, which the teardown ordering prevents —
init.c:396clears the cache first._mi_meta_freetakes no lock.subproc.c:73-82ismi_free(p)forMI_MEM_MALLOCmemids (what_mi_meta_zallocproduces) or_mi_arenas_freeotherwise. The only
theap_meta_lockin the free path ismi_stat_freeatfree.c:757-758, inside#if (MI_STAT>0)— absent from every defaultRelease build.
The
mi_atomic_do_onceatprim-tls.c:215has already completed here, so_mi_atomic_once_enterreturns atlibc.c:117beforemi_lock_acquire;W5 does not apply either.
Evidence: instrumented tracing, against a workload whose main thread's last
allocation comes from a
mi_heap_new()heap so the cached theap is dynamicallyallocated. The function is entered and reaches the decref, but
mi_theap_free_memis never called and no lock is acquired — in Release static,Release shared,
MI_DEBUG, FLS, or withdestroy_on_exit=1.Fix: still pointless work at process exit — resetting the cached theap matters
only if the process continues running — so it belongs in the set skipped by
W1's exit path. It is not an independent hazard.
W7 —
mi_theap_collectcompiled in for static builds, not just debug ·init.c:605-612· process exit · wasteThe guard is not debug-only:
MI_SKIP_COLLECT_ON_EXITdefaults OFF (CMakeLists.txt:47) andMI_SHARED_LIBis added only to the shared target (
CMakeLists.txt:815), so a forced collect ofevery page runs at process exit in every static build — a very common
configuration. A per-statement phase tracer confirms it: present in Release
static, absent in Release shared, present in both once
MI_DEBUG=ON. The work ispointless there — collecting and purging pages hands memory back to an address
space about to be destroyed.
What it does and does not touch.
mi_theap_collect(theap.c:154-156) →mi_theap_collect_ex(theap.c:123-147) reaches_mi_theap_collect_retired(
page.c:501),mi_theap_visit_pages, and_mi_arenas_collect(
arena.c:1502). Across Release-static, Release-shared,MI_DEBUGand FLS builds — with workerthreads, extra heaps and leaked blocks to produce abandoned pages, and under
MIMALLOC_DISALLOW_ARENA_ALLOC,MIMALLOC_PAGE_RECLAIM_ON_FREEandMIMALLOC_ARENA_MAX_OBJECT_SIZE=64— this statement acquires zeromi_lock_tand enters neither thebitmap.c:122busy-wait nor thetheap.c:409spin. Specifically:os_abandoned_pages_lockatarena.c:1354is inside_mi_arenas_page_abandon, reached only forcollect == MI_ABANDON(threadexit), not
MI_FORCE.arena.c:1423is in_mi_arenas_page_unabandon, called frommi_free(W4), not collect.
arena.c:2511andarena.c:2632are on the heap-destroy path — W8;the tracer shows
arena.c:2511in the W8 phase.pmap->lock(page-map.c:389) is only taken when a submap is absent(
mi_page_map_ensure_submap_at,page-map.c:414-427); collect onlyunregisters pages, where the submap necessarily exists.
mi_arenas_try_purge(arena.c:2398) usesmi_atomic_guard(
atomic.h:397-401) — a try-once CAS, not a spin. Orphaning it silentlydisables purging for the process lifetime; it cannot hang.
The real hazard on this statement is not a lock but a callout:
mi_theap_collect_ex's first act is_mi_deferred_freeattheap.c:129, whichruns arbitrary user code on the process-exit path — see W16. That is also
what makes this statement usable as the staging mechanism in W5's
reproducer.
Fix: skip on the process-exit path (W1). Keep it for the unload path,
where returning the memory is the whole point.
W8 —
_mi_subprocs_unsafe_destroy_all·init.c:622· process exit withdestroy_on_exit· kill + hangsubproc.c:265-278takesmi_subprocs_lockto walk the subprocess list, thenfor each one
mi_subproc_unsafe_destroytakes that subprocess'sheaps_lock(
subproc.c:214) and calls_mi_page_map_unsafe_destroy(page-map.c:367-383)— which unmaps the page map that other (already dead, but that is not knowable
here) threads may have been reading.
This is the then branch of
init.c:621, gated onmi_option_destroy_on_exit,default 0 (
options.c:147). Independent of static/shared andMI_DEBUG.It is the largest single exposure in the teardown. The lock tracer logs 29
acquisitions across 7 distinct sites for a trivial 3-thread program, identical
in static and shared:
mi_subprocs_lock(subproc.c:266,207),heaps_lock(
subproc.c:214,heap.c:194),heap->theaps_lock(theap.c:385,heap.c:170),tld->theaps_lock(theap.c:391— the W2 spin loop),os_abandoned_pages_lock(arena.c:2511), andmi_thread_locals_lock(
threadlocal.c:309,231). It re-runs all of W9 inline(
subproc.c:222-224) and callsmi_subproc_stats_print_outitself atsubproc.c:243, so it carries every hazard of W10 too. That is why it iskill + hang rather than kill alone. As with W7, all of it is work the kernel
is about to do for free.
Measured exposure: 19/60 ≈ 32 % of
stockrace.cruns had their shutdownterminated early, against ≈ 0.9 % for the default configuration. Deterministic
reproduction is
lockkill.c(W10) withMIMALLOC_DESTROY_ON_EXIT=1.Fix: skip on the process-exit path (W1).
W19 — shared build +
destroy_on_exittruncates shutdown unconditionally ·init.c:621-622· process exit withdestroy_on_exit· bugIn the shared build,
MIMALLOC_DESTROY_ON_EXIT=1truncates shutdown even ina single-threaded, fully benign run with nothing contended: the EXE's
post-mimalloc TLS
DLL_PROCESS_DETACHcallback never runs, and neithermimalloc: process donenor the marker appears — while the same binary withoutthe option prints both. Exit code is 0 either way.
This is not W8's orphaned-lock mechanism — there is nothing to orphan here
— so
_mi_subprocs_unsafe_destroy_allis destroying state that mimalloc's ownremaining teardown, or the loader's subsequent detach sequence, still needs. It
also means the shared +
destroy_on_exitcombination cannot be used to measureW8's probabilistic exposure.
Fix: bisect which of
mi_subproc_unsafe_destroy's steps (subproc.c:214-247)causes the truncation.
_mi_page_map_unsafe_destroy(page-map.c:367-383)unmapping the page map is the obvious suspect, since anything still calling
freeafterwards would fault.W9 — thread-locals teardown takes
mi_thread_locals_lock·init.c:626-627· process exit · killTwo adjacent statements in the else branch of
init.c:621:_mi_thread_locals_thread_done()(threadlocal.c:205-214) →_mi_meta_free,and
_mi_thread_locals_done()(threadlocal.c:230-241) →mi_thread_locals_lockatthreadlocal.c:231.mi_thread_locals_lockguards the list of all threads' local blocks, so it isheld briefly by every thread that starts or stops — a realistic orphaning
candidate. It is taken in every configuration: default Release static,
default Release shared,
MI_DEBUG, FLS. That makes it the one lock a defaultRelease build reliably acquires on the process-exit path outside the stats print,
and the reason the default configuration has any probabilistic exposure at all.
The
theap_meta_lockreachable through_mi_meta_free→mi_free→mi_stat_free(free.c:757-758) is debug-only, since that function isinside
#if (MI_STAT>0). The tracer showsthreadlocal.c:231alone in defaultRelease static, and
free.c:758appearing in both statements onceMI_DEBUG=ON._mi_thread_local_free(threadlocal.c:309) takes the same lock and is reachedfrom W8.
Fix: skip on the process-exit path (W1).
W10 — stats printing ·
init.c:633· process exit withshow_stats/verbose· killmi_subproc_stats_print_outis the worst single statement in the teardownbecause it combines four independent hazards:
heaps_lock—stats.c:512-519→mi_subproc_stats_get(
stats.c:640-649) →mi_subproc_visit_heaps(subproc.c:303-314) walksevery heap under the subprocess lock, acquired at
subproc.c:307._mi_prim_out_stderr(prim.c:635-682). Writing to aconsole is an ALPC round-trip to conhost, which can block on a process not
participating in the shutdown, and it touches the CRT's own stdio locking.
See W15.
LoadLibrary—_mi_stats_print(stats.c:356) →mi_process_info_print_out(stats.c:427) →mi_process_info(
stats.c:568) →_mi_prim_process_info→ psapi atprim.c:611, seeW14.
out_buf_lock—options.c:372, around 35 acquisitions per stats dump.Plus
libc.c:125: themi_atomic_do_onceguarding the psapi load (W5 +W14) is acquired live, at exit, since that once has never run before.
Any one of these is disqualifying; together they make this the statement most
likely to be the thing that actually kills a real process.
When it runs.
init.c:629gates it onmi_option_show_statsormi_option_verbose, both default 0. The option check precedes any lockacquisition, so a default-configured process does not reach this statement at
all. Independent of static/shared/
MI_DEBUG. Also reached unconditionally fromW8 via
subproc.c:243-247.Both API hazards were confirmed live. With
MIMALLOC_SHOW_STATS=1, MIMALLOC_VERBOSE=0, the first mimalloc output of thewhole process is emitted inside this statement, so
_mi_prim_out_stderr'scached
hconis uninitialised andGetStdHandle— and, with no stderr handle,AttachConsole— really is first called from insideDLL_PROCESS_DETACH. Andpsapi.dllis not loaded before exit in any configuration, so theLoadLibraryat
prim.c:611genuinely happens during detach.Fix: skip on the process-exit path (W1). Users who want stats at exit
should get them from the EXE
atexitslot instead — the one hook that runsbefore
ExitProcess, while every thread is still alive and ordinary locking,console I/O and even
LoadLibraryare all safe.Reproducers.
lockkill.c(stock, deterministic, static and shared,100 %): a helper thread parks inside a
mi_subproc_visit_heapsvisitor — whichmimalloc invokes while holding
subproc->heaps_lock(subproc.c:307) — and isSuspendThreaded there, exactly the stateExitProcess's mass kill produces.Main exits; the stats block,
mimalloc: process done, and a.CRT$XLZTLScallback standing in for later shutdown work all vanish (95 → 60 lines of
output) while the exit code stays
0x00000000. Against the instrumented buildthe phase log ends on exactly that acquisition, where the control run continues
past the same lock address.
stockrace.cis the unaided probabilistic version —hit rates under Which locks can be orphaned.
One
stockracerun underdestroy_on_exit=1wedged permanently instead —single-threaded, accruing CPU, immune to
taskkill /F, alive hours later. Itssampled RIP sat at
ntdll!NtDelayExecution+0x14, not mimalloc'sSwitchToThreadspin (NtYieldExecutionis at RVA0x160910), so it is mostlikely ntdll's own
RtlExitUserProcesswait-for-threads loop. Recorded as anobserved stock shutdown hang of unknown origin.
W11 —
_mi_tls_slots_donecallsTlsFree·init.c:638· process exit · waste_mi_tls_slots_done(prim-tls.c:135-138for the Win32 TLS model; alsoprim-tls.c:172-177pthreads,194-196fixed,205-207none) callsmi_win_tls_slot_free(prim-tls.c:114-121) →TlsFreeatprim-tls.c:118→RtlAcquirePebLock. Two calls per process. Runs in every configuration; theTlsFreebody only underMI_TLS_MODEL_WIN32, the default on Windows.Confirmed at runtime with an instrumented
TlsFreesite, identically in static,shared and FLS builds. No
mi_lock_tis involved.The PEB lock is one of the three that
RtlExitUserProcessdeliberately protectsfrom orphaning — it takes it before the mass kill for exactly this reason — so in
the normal
ExitProcesssequence this will not be held by a dead thread, and nokill is possible through it.
Still worth removing: it is pure waste (the kernel reclaims TLS indices with the
process), it is an API call on a path that should have none, and the protection
depends on going through
RtlExitUserProcess, which not every shutdown pathdoes.
Fix: skip on the process-exit path (W1).
W12 —
_mi_verbose_message·init.c:641· process exit withverbose· kill (narrow)options.c:524-530→mi_vfprintf→_mi_fputs(options.c:466) → thedefault output function
mi_out_buf_stderr(options.c:402-405) →mi_out_stderr, thenmi_out_buf(options.c:365) →out_buf_lock, declaredat
options.c:363and acquired atoptions.c:372, then out through_mi_prim_out_stderrand the same console I/O as W15. The defaultoutput really is
mi_out_buf_stderrat exit (mi_add_stderr_output,options.c:434).This is the last statement of the teardown, which makes it easy to overlook and
easy to remove. Two conditions bound how often it matters:
_mi_verbose_messageearly-returns atoptions.c:525unlessmi_option_verboseis enabled, default 0. In a default process thisstatement takes no lock and does no console I/O at all — the tracer's
W12phase is empty without
MIMALLOC_VERBOSE=1, and shows exactly oneoptions.c:372acquisition with it.mi_out_bufreturns before taking the lock onceout_len >= MI_MAX_DELAY_OUTPUT(16 KiB) —options.c:368. Soout_buf_lockis only ever taken during the first 16 KiB ofmimalloc-generated output in the process's lifetime; a chatty build saturates
the delay buffer early and stops taking the lock altogether. Console I/O via
mi_out_stderrcontinues regardless, so the W15 exposure remains evenwhen the lock is skipped.
The acquisition is at the same lock address as the ~35 inside the W10 phase,
confirming it is
out_buf_lock. The window is one acquisition of a lock heldonly across a
memcpy, so no dedicated reproducer was built;lockkill.cdemonstrates the mechanism on the same class of lock.
Fix: skip on the process-exit path (W1).
W13 — FLS mode calls
FlsFree·prim.c:1148-1152· process exit (MI_WIN_INIT=FLS) · kill_mi_prim_thread_done_auto_done(prim.c:1148-1152) callsFlsFree(mi_fls_key)at
prim.c:1151, added to fix the dangling-callback-after-unload problem(issue #208).
FlsFreetakes ntdll's global FLS lock, the most dangerous lock in the shutdownsequence:
RtlProcessFlsDataruns at the very top ofLdrShutdownProcess,before any
DLL_PROCESS_DETACH, so the FLS lock is the first lock acquired afterthe mass thread kill. It is also process-global, so contention is not limited to
mimalloc's own threads.
FlsFreeadditionally invokes the registered callback for every thread thatstill has a value, from the calling thread — so it can drive
_mi_thread_doneonbehalf of threads that no longer exist. mimalloc guards this with the thread-id
check at
init.c:472-473, but the guard is a symptom of the mechanism beingwrong for this job.
Only affects
MI_WIN_INIT=FLS, deprecated and not the default (prim.c:785-793selects
CRT_TLS, orTLS_DLLMAINfor Intel ICX/ICC). It does still build andrun, static and shared. See H3.
Reproducers.
flsforeign.c(instrumented, FLS build): a worker threadstill alive at
ExitProcessleaves a theap in its FLS slot, and the instrumentedtrace shows
FlsFreeatprim.c:1151followed bymi_fls_done(prim.c:1136)running on the main thread for that already-terminated worker's theap —
init.c:472-473is what stops it tearing down a foreign tld. When all workersexit normally their slots are already NULLed by
mi_fls_doneand the mainthread's at
init.c:560, soFlsFreefinds nothing; the hazard appearsspecifically with threads that outlive
main. A mimalloc-free control confirmsthe Win32 semantics directly:
FlsFreeruns the callback on the calling threadfor every thread still holding a value.
W14 —
LoadLibrary("psapi.dll")from a detach callback ·prim.c:609-616· process exit withshow_stats/verbose· deadlockInside
_mi_prim_process_info(prim.c:598-629):Called from the stats path (W10), which runs from a
DLL_PROCESS_DETACHcallback — with
LdrpLoaderLockheld andLdrpShutdownInProgressset.LoadLibraryis at the top of the documented forbidden list forDllMain/TLScallbacks: it re-enters the loader recursively, and loading a new module during
shutdown means running that module's
DLL_PROCESS_ATTACHafter the loader hasalready begun detaching everything. Being wrapped in
mi_atomic_do_onceit alsocarries W5's lock — and that once has never completed before, so the lock
is genuinely acquired.
psapion the CMake link line (CMakeLists.txt:717) does not make thisharmless. Nothing references a psapi symbol — mimalloc resolves
GetProcessMemoryInfobyGetProcAddress— so no import is created:dumpbin /dependentson the stockmimalloc.dllshows no psapi.dll, and neitherdoes a DLL statically linking
mimalloc.libwithpsapi.libexplicitly on thelink line. mimalloc genuinely pulls a fresh module into the process from inside
DLL_PROCESS_DETACH.Applies to any build in
CRT_TLS(default) orRAW_DLLMAIN/TLS_DLLMAIN,static or shared, whenever
mi_option_show_statsormi_option_verboseis setand
destroy_on_exitis off. With neither option set, psapi is never loaded.A note on probing this. Real process exit cannot be probed for loader-lock
ownership —
RtlExitUserProcesskills the probe helper thread first. W14 andW15 are therefore demonstrated on the
FreeLibraryunload path, wheremi_process_done_onceruns fromDLL_PROCESS_DETACHunder the loader lock withall threads alive: per W1 the same teardown code, minus only the "peer
threads are dead" hazard.
Fix: resolve the psapi entry points at init and cache them, or skip stats at
process exit (W10), which removes the reachability entirely. Doing both
is cheap.
Reproducer (stock mimalloc statically linked into a DLL, unloaded by
FreeLibrary):GetModuleHandleA("psapi.dll")returns NULL atmainentry,while mimalloc runs, and at the moment the detach handler starts — where the
loader-lock probe reports held — then non-NULL once the teardown returns, so
psapi was loaded inside it under the loader lock. Without
MIMALLOC_SHOW_STATS=1all four readings are NULL. The same run showsbcrypt.dllalready loaded atDLL_PROCESS_ATTACH— that is W20.W15 —
AttachConsole/GetStdHandlefrom a detach callback ·prim.c:635-682· process exit; any error path · deadlock_mi_prim_out_stderrcallsGetStdHandle(prim.c:645,649), and on failureAttachConsole(ATTACH_PARENT_PROCESS)(prim.c:648) thenGetConsoleScreenBufferInfo(prim.c:652). Attaching to a console performs ALPCto conhost, and console I/O touches the CRT's own stdio locking — neither is
appropriate from a detach callback.
Reachable from W10, W12, and from any error or warning path. The
error path cannot be removed (mimalloc must be able to report a fatal condition),
but W10 and W12 can, which is most of the exposure.
hconis a function-local static (prim.c:640), initialised on the firstcall — so these APIs run wherever the process's first stderr output happens. If
any verbose or warning message was emitted earlier,
hconis already cached andthe exit path only does
WriteConsoleA/WriteFile. The dangerous case isprecisely "the first output is the exit-time stats dump", the default when only
MIMALLOC_SHOW_STATS=1is set. TheAttachConsolebranch additionally needsGetStdHandle(STD_ERROR_HANDLE) == NULL.With
AttachConsoleforced to run under the loader lock, anLdrRegisterDllNotificationtrace shows no module loaded by it, and the callreturned without blocking (
GetConsoleWindow()still NULL). So the demonstratedhazard is console I/O and console-API calls from a detach callback; loader
re-entry through
AttachConsolespecifically is a plausible risk rather than anobserved one.
Fix: remove the W10/W12 callers. For the error path, consider caching the
stderr handle at init so the exit path only ever does
WriteFile.Reproducer (instrumented, markers around
prim.c:645/648/652): the hostcalls
FreeConsole()andSetStdHandle(STD_ERROR_HANDLE, NULL)beforeFreeLibrary, forcing the first_mi_prim_out_stderrdown theAttachConsolebranch inside
DLL_PROCESS_DETACH. The log showsGetStdHandle,AttachConsoleandGetConsoleScreenBufferInfoall inside the teardown window;the loader-notification log records only
psapi.dllloaded there(W14).
W16 —
_mi_deferred_freeunder the loader lock ·theap.c:129,page.c:990-999· thread exit; process exit via W7 · deadlock_mi_theap_collect_abandon(theap.c:150-152) callsmi_theap_collect_ex(theap, MI_ABANDON), whose first statement is_mi_deferred_free(theap, force)attheap.c:129— invoking whatever callbackthe user registered through
mi_register_deferred_free. SinceMI_ABANDON = 2 >= MI_FORCE = 1(theap.c:90-94), the callback is invoked withforce = trueon thread exit.Reached from
_mi_thread_done(init.c:451) →mi_thread_theaps_done(call atinit.c:476, function atinit.c:376-390), which on Windows runs from the.CRT$XLYTLSDLL_THREAD_DETACHcallback (registrationprim.c:880-883,mi_tls_detachprim.c:846-858→mi_win_mainprim.c:765→_mi_thread_done(NULL)prim.c:775) — with the loader lock held. The user'scallback is arbitrary code: it may allocate, take its own locks, or call
LoadLibrary. Nothing in mimalloc's public API warns that this can happen, so acallback that is perfectly reasonable in every other context can deadlock the
process on Windows only, and only on thread exit. It also runs on the
process-exit path, via
mi_theap_collectatinit.c:610(W7).Two guards sit on the path, neither of which blocks it:
_mi_thread_donereturnsearly if the theap is uninitialised (
init.c:459) or on a thread-id mismatch(
init.c:473), and_mi_deferred_freehas atld->recurseguard(
page.c:993-997). A thread that never allocated is unaffected.The
.CRT$XLYroute is not the only one.mi_win_mainskips_mi_thread_donewhen redirected (prim.c:774,&& !_mi_is_redirected()), soin the default shared build with
mimalloc-redirect.dllthe call arrives via_mi_redirect_entry(DLL_THREAD_DETACH)(prim.c:1185-1188) — from theredirector's own
DllMain, also under the loader lock. Confirmed static andshared, x64 and x86, redirected and not.
Fix: at minimum document that a deferred-free callback may be invoked with the
loader lock held on Windows. Better: suppress the call for the
MI_ABANDONcasein
theap.c:129— thread teardown is not a point where user code needs to run.That also closes the hang described in W2.
Reproducers. A probe callback reports, from inside itself, that it holds the
loader lock on the worker's thread exit.
deadlock.c(stock, static link, noinstrumentation) then produces the real thing, 3/3 runs,
exit=124: theworker's thread-exit callback holds the loader lock and waits on a plain
CRITICAL_SECTION, while main holds thatCRITICAL_SECTIONand waits on theloader lock inside
LoadLibraryA("winmm.dll"). Nothing exotic on either side;the only surprise is that mimalloc hands control to user code with the loader
lock held.
W17 —
_mi_allocator_donecross-DLL call ·init.c:640,prim.c:1198-1200· process exit · deadlockA call into
mimalloc-redirect.dll(active whenMI_SHARED_LIB && !MI_WIN_NOREDIRECT, guard blockprim.c:1165-1201, importdeclared at
prim.c:1191, bodymi_allocator_done();atprim.c:1199), madefrom inside mimalloc's own detach handler. The redirect DLL is a separate module
with its own detach handler, which may already have run or may be running — and
its code is opaque, so nothing here can be verified.
Listed for completeness, not to argue for removal: this is the one process-exit
step worth keeping. Without it the redirector carries on routing
freeintoan allocator that has been torn down, which is worse than the risk of the call.
Gate it only if it proves problematic in practice.
The redirector is genuinely active by default —
dumpbin /dependentson thestock
mimalloc.dlllistsmimalloc-redirect.dllas a static import — and thecall returns normally: a shared-build program reports
mi_is_redirected() = 1,exits 0, and under
MIMALLOC_VERBOSE=1still printsmimalloc: process done,which is
init.c:641, i.e. after_mi_allocator_done()atinit.c:640.W18 —
GetModuleHandleExAfromDllMain(DLL_PROCESS_ATTACH)·prim.c:812-816· init · hygienemi_current_module_is_dll()(prim.c:812-816,GetModuleHandleExAat814) iscalled from
mi_crt_init(prim.c:824-831, call at826) — which in a DLLbuild runs from
_DllMainCRTStartup(DLL_PROCESS_ATTACH), whereGetModuleHandle*is on the documented forbidden list forDllMain.Safe in practice, for three separate reasons.
GET_MODULE_HANDLE_EX_FLAG_UNCHANGED_REFCOUNTmeans the call cannot trigger anunload; it is on the init path, so none of the shutdown hazards apply; and on
Windows 8+ an attach-time
DllMainreached throughLoadLibrarydoes not holdLdrpLoaderLockat all — measured on 26200, where a helper thread successfullyacquired the lock while the main thread sat inside
DllMain(DLL_PROCESS_ATTACH),though the same probe reports it held for
DLL_PROCESS_DETACH/FreeLibraryand
DLL_THREAD_DETACH. The process-startup attach(
LdrpInitializeProcess, for a statically importedmimalloc.dll) cannot beprobed this way — the helper thread does not exist yet — so that case is
unmeasured.
Applies to
MI_WIN_INIT=CRT_TLSonly, the default: theFLSarm has its ownmi_crt_init(prim.c:1098-1102) which does not call it, andRAW_DLLMAIN/TLS_DLLMAINhave nomi_crt_initat all. Static and shared. Itis listed because it is entirely avoidable and the codebase already has the
better primitive.
Fix: use
mi_module_is_dll((PVOID)&__ImageBase).__ImageBaseis supplied bylink.exe, lld-link and GNU ld, and
mi_module_is_dll(prim.c:805-810) justreads the PE header from the given base with no API call at all — exactly what
the exit path already uses (
prim.c:853).Reproducer:
mi_module_is_dllandmi_current_module_is_dllcopied verbatimfrom
prim.c:805-816and called side by side, fromDllMain(DLL_PROCESS_ATTACH)in a DLL and from
mainin an EXE.__ImageBaseand theGetModuleHandleExAbase are identical and both
is_dllanswers agree (1/1 in the DLL, 0/0 in theEXE); a
gccbuild confirms__ImageBaseresolves identically under GNU ld.W20 —
LoadLibrary("bcrypt.dll")fromDllMain(DLL_PROCESS_ATTACH)·prim.c:718-730· init · hygiene_mi_prim_random_buf(prim.c:718-730) does:This runs during mimalloc's initialization — from
DllMain(DLL_PROCESS_ATTACH)or the
.CRT$XIBinit callback — and pulls inbcrypt.dllplusbcryptprimitives.dll. A nestedLoadLibraryfromDllMainis a largerforbidden-API violation than W18's
GetModuleHandleExA, and unlikeW14 it is unconditional: no option gates it, because mimalloc always
needs randomness at init. Observed in every run — W14's reproducer shows
bcrypt.dllalready loaded by the timeDllMain(DLL_PROCESS_ATTACH)returns,and the instrumented build logs the call at attach time.
Rated hygiene rather than deadlock for the same reason as W18: on Windows 8+ an
attach-time
DllMainreached throughLoadLibrarydoes not holdLdrpLoaderLock. The process-startup attach path (LdrpInitializeProcess) wasnot measurable and remains unknown, and on pre-Win8 loaders this is a genuine
recursive-loader-entry hazard.
Fix: mimalloc already has
MI_USE_RTLGENRANDOM(prim.c:705-707), which usesRtlGenRandomwith noLoadLibraryat all. Otherwise, defer the bcryptresolution to the first use of the RNG rather than doing it at init. Worth an
explicit decision either way, since it is currently unconditional.
B1 — MinGW +
TLS_DLLMAIN+ static does not compile ·prim.c:1076-1077· build · bugThe MinGW arm of the
MI_WIN_INIT_USE_TLS_DLLMAINstatic branch registers&mi_tls_attachand&mi_tls_detachin.CRT$XLB/.CRT$XLY, atprim.c:1076and
prim.c:1077. Neither symbol exists in that#elifblock — the functions itdefines are
mi_win_main_attach(prim.c:1032) andmi_win_main_detach(
prim.c:1037).mi_tls_attach/mi_tls_detachare defined only in theCRT_TLSarm (prim.c:834,846) and theRAW_DLLMAINarm (prim.c:953,961), both mutually exclusive with this one.This is the mode forced for Intel ICX/ICC (
prim.c:786-787, issue #1268), sothe broken configuration is not purely hypothetical, though
Intel-compiler-plus-MinGW is an unlikely pairing. The equivalent MinGW arm in
RAW_DLLMAINmode (prim.c:1002-1003) uses the same two names and is correct,because that branch does define them — the names were copied between blocks
without being renamed, see H1.
Real compile error with gcc 16.1.0:
Only this one cell of the matrix fails;
CRT_TLS,RAW_DLLMAINandFLSallbuild static and shared, and
TLS_DLLMAINshared is fine because the#elif …&& defined(MI_SHARED_LIB)arm atprim.c:1013wins.Fix: rename to
mi_win_main_attach/mi_win_main_detachin that block, orcollapse the nine blocks per H1 so the names cannot drift again.
B2 — 32-bit ARM gets x86 symbol decoration ·
prim.c:888· build · bugThe section-registration code has two MSVC branches:
#if defined(_WIN64) && defined(_MSC_VER)atprim.c:871, and#elif defined(_MSC_VER) // 32-bitatprim.c:888— so the effective predicatefor the second is
defined(_MSC_VER) && !defined(_WIN64). The 32-bit branch addsan extra leading underscore to every
/INCLUDE:symbol (prim.c:889-892:__tls_used,__mi_tls_callback_pre,__mi_tls_callback_post,__mi_crt_callback_init), correct for x86 because MSVC decorates cdecl C symbolswith a leading
_there.But
_M_ARM(32-bit ARM MSVC) also satisfies that predicate —_WIN64isdefined only for 64-bit targets,
_MSC_VERis defined, and there is no_M_ARMtest anywhere in the block. ARM32 uses undecorated names. On that target the
pragmas would name symbols that do not exist, the
/INCLUDE:directives would beno-ops, and
/OPT:REFwould discard the callback arrays — leaving mimalloc withno thread-exit or process-exit hooks at all, silently.
The same defect is replicated at
prim.c:989-992andprim.c:1063-1066, and inthe FLS arm's
#else // 32-bitatprim.c:1114-1115. Practically a deadplatform, so this is latent rather than live. The correct predicate is
defined(_M_IX86). See H1.The decoration rule was confirmed on the two targets available: one TU built for
x64 and x86 and dumped with
dumpbin /symbolsgives_mi_tls_callback_preand__mi_tls_callback_prerespectively. The predicate is correct for x86 — a 32-bitbuild under vcvars32 compiles cleanly, has a Thread Storage Directory, and its
TLS hooks fire on thread exit — so the bug is confined to ARM32.
B3 —
Release+MI_DEBUG=ONdoes not compile with MSVC ·types.h:180-182· build · bugThe
#if MI_DEBUG && NDEBUGdiagnostic attypes.h:180-182uses#warning,which MSVC only accepts under
/std:c++23preview. Since mimalloc's CMakecompiles the sources as C++ by default (
MI_USE_CXX=ON), anyRelease-plus-MI_DEBUGconfiguration fails outright rather than emitting theintended warning. This is the configuration W7's
MI_DEBUGarm needs, soit also blocks testing that path; the workaround is
-DCMAKE_CXX_FLAGS_RELEASE="-O2 -Ob2", which dropsNDEBUG.Fix: use
#pragma messagefor MSVC, or guard the#warningon a compilercheck.
H1 — three near-identical section blocks copy-pasted ·
prim.c:871-908,977-1004,1051-1078· build · hygieneEach of the three init modes (
CRT_TLS,RAW_DLLMAIN,TLS_DLLMAIN)hand-writes the same three-way toolchain split:
_tls_used/INCLUDE:_tls_used#pragma const_seg(".CRT$XLB")+/INCLUDE:_mi_tls_callback_preetc./INCLUDE:__tls_used#pragma data_seg(...), extra leading underscoreextern const IMAGE_TLS_DIRECTORY _tls_used;+ ausedreference__attribute__((used, section(".CRT$XLB")))Nine near-identical blocks, differing only in symbol names. A
diff -uof thethree ranges:
_mi_crt_callback_init/.CRT$XIBlines that exist solely in theCRT_TLSblock, plus one space of whitespace. Everything else is byte-identical.
callback names
mi_tls_attach/mi_tls_detach→mi_win_main_attach/mi_win_main_detach. The#elif defined(__MINGW32__)arms are byte-identical between the two — which is exactly B1: the
rename was applied to the two MSVC arms and missed on the MinGW one.
The x64 row covers ARM64 too, since
_WIN64is defined there, so ARM64 takes theundecorated branch and is fine; only ARM32 is broken (B2). So B1 (wrong
names carried across a copy) and B2 (wrong predicate replicated three times) are
both direct consequences, and a future fix to one block will very likely miss the
other two.
Fix: collapse to one macro parameterised on the attach/detach/init function
names, using
__pragma(comment(linker, "/INCLUDE:" MI_SYM(x)))for the MSVCfamily and
_Pragmafor GCC/clang GNU mode, with a single decoration rule(
#if defined(_M_IX86)→_prefix, else none).H2 — TLS registration depends on
prim.objbeing linked in ·prim.c:871-908· build · hygieneThe
#pragma comment(linker, "/INCLUDE:...")directives (prim.c:872-875forx64,
889-892for x86) are emitted intoprim.obj. A linker only processesdirectives from object files it actually pulls out of a static
.lib, and itonly pulls an object if something already references a symbol in it.
They live in exactly one object:
dumpbin /directiveson the static-lib build'ssrc/prim/prim.c.objshows all four/INCLUDE:directives, and scanning the.drectvetext of all 19 objects in that build gives exactly one hit. No other TUemits them, so the coupling is real and unenforced.
Today it always holds —
_mi_prim_alloc,_mi_prim_commit,_mi_prim_thread_idand around thirty other OS primitives live in that same TU, and every build
references them. But nothing enforces it. If that TU is ever split for tidiness,
static builds would silently lose all thread-exit and process-exit hooks: no
link error, no warning, just a mimalloc that never runs
_mi_thread_doneor_mi_process_done, and very hard to attribute. (That consequence follows fromdocumented
link.exearchive-extraction behaviour plus the measurement above;producing it would require splitting the TU.)
Fix: record the dependency in a comment at minimum; better, add a link-time
assertion or keep a deliberate reference from a TU that is always pulled in.
H3 — FLS mode carries three workarounds for its own bugs · thread exit + process exit · hygiene
Three separate fixes, all for problems the FLS mechanism creates:
init.c:556-561prim.c:1140_mi_thread_donemay set it back to the main theap, issue #672"prim.c:1148-1152,FlsFreeat1151Beyond the workarounds, FLS is structurally wrong for an allocator: its callbacks
run first in both
LdrShutdownThreadandLdrShutdownProcess, before everyDllMain(DLL_THREAD_DETACH), before every TLS callback, and therefore before allC++
thread_localdestructors — so user destructors thatfree()run aftermimalloc has torn the thread down. It can also fire on the wrong thread (via
FlsFree, demonstrated in W13, and via orphan reaping on recent Windows),and there are only 128 FLS slots per process.
Reproducer (one source built against the default and FLS libs): a worker thread
with a C++
thread_localwhose destructor callsmi_free, with mimalloc's ownteardown observed through a deferred-free callback. Under the default
CRT_TLSthe destructor runs first and mimalloc's teardown second; under FLS the order
inverts. The main thread shows the correct order in both builds — the
init.c:556-561workaround (issue #508) doing its job. The invertedfree()does not fault in this simple case; the defect is the ordering.
Fix: consider marking
MI_WIN_INIT=FLSfor removal. Its only unique coverage —the main thread at process exit, and orphan reaping after unclean exits — is
already handled by the
DLL_PROCESS_DETACHpath (prim.c:851-857) plusatexit, and the default.CRT$XLYplacement gets the ordering right for free.H4 —
DisableThreadLibraryCallssilently has no effect ·threadlocal.c:53-64· init; every thread create/exit · hygienemi_thread_localsandmi_slot_fastare declaredmi_decl_thread(
internal.h:38=__declspec(thread)), instantiated atthreadlocal.c:63-64via the
#else // Direct thread localsarm atthreadlocal.c:53-64. OnWindows/MSVC the
#if MI_TLS_MODEL_PTHREADS || defined(__APPLE__)atthreadlocal.c:44is false, so that is the arm taken, and any module carryingmimalloc gets a TLS directory (confirmed with
dumpbin /headers: a DLLstatically linking mimalloc has one, an otherwise identical DLL without mimalloc
has none).
Windows documents
DisableThreadLibraryCallsas failing for any module withactive static TLS. On Windows 11 26200 it does something worse than fail: it
returns
TRUEwithGetLastError() == 0, and theDLL_THREAD_ATTACH/DLL_THREAD_DETACHnotifications keep being delivered(the control DLL returns
TRUEand genuinely stops receiving them).So any DLL that statically links mimalloc keeps receiving — and paying for — the
per-thread notifications it believes it turned off, and checking the return
value does not reveal it: a silent per-thread cost the host cannot see by any
local means. Identical whether the DLL is
LoadLibrary'd or bound at loadtime.
This is not load-bearing from mimalloc's side in the default mode. In
CRT_TLSmimalloc's hooks are.CRT$XLB/.CRT$XLYTLS callbacks(
prim.c:876-883), which the loader runs for every module with a TLS directoryregardless of
LDRP_DONT_CALL_FOR_THREADS—DisableThreadLibraryCallscouldnever have suppressed them, and mimalloc's thread teardown still runs when the
call returns
TRUE. OnlyRAW_DLLMAINand sharedTLS_DLLMAIN, which routethrough
DllMain, depend on the notifications at all. Static TLS is a side effectimposed on the host, not a mechanism mimalloc relies on.
Fix: document it in the Windows section of the readme — as "returns success but
has no effect", not "fails".