Fix generic virtual dispatch hang after multiple AssemblyLoadContext unloads

[Copilot]

• Root cause confirmed: GenericCache.TrySet detected the flush sentinel with table.Length == 2, but the sentinel array is length 3 (2 entries + element 0 used for aux data). The sentinel was therefore never recognized, so after a flush entries were inserted into the shared sentinel table. Since every flush re-installs that same sentinel, stale entries survived flushes and generic-virtual-dispatch targets belonging to unloaded collectible AssemblyLoadContexts could be returned once native addresses were reused (hang/crash).
• Fix in src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/GenericCache.cs: added SENTINEL_TABLE_SIZE constant and an IsSentinel(table) helper based on CacheElementCount(table), used by both sentinel checks in TrySet; tightened the constructor assert.
• Regression test added: src/tests/Loader/CollectibleAssemblies/GenericVirtualMethod (payload assembly with the generic virtual method/override from the issue + runner that loads/invokes/unloads 10 times and asserts the virtual-function-pointer cache is empty after each unload).
• Validation: baseline ./build.sh clr+libs -lc release -rc checked (exit 0); test fails against unfixed CoreLib (cache entry version 2 after flush, exit 101) and passes with the fix (exit 100); other Loader/CollectibleAssemblies tests still pass.

• Fixes Generic virtual dispatch hangs after repeated collectible AssemblyLoadContext unloads on .NET 10 #132562

[azure-pipelines]

Azure Pipelines:
16 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[VSadov]

@copilot check if CastCache needs similar change for consistency.
We do not need to add a test for CastCache though as it would not cause failures.

[Copilot]

@copilot check if CastCache needs similar change for consistency.
We do not need to add a test for CastCache though as it would not cause failures.

Checked both the managed CastCache.cs and the native src/coreclr/vm/castcache.cpp. Neither has the GenericCache bug: their sentinel checks read the aux TableMask (== 1), not the array length, so they correctly recognize the 2-element sentinel. GenericCache was the odd one out because it derives the size from Array.Length.

For consistency I applied the same naming in the managed CastCache.cs only — SENTINEL_TABLE_SIZE constant plus an inlined IsSentinel(ref tableData) used by both checks in TrySet, and the ctor assert now uses the constant. TableMask == SENTINEL_TABLE_SIZE - 1 is constant-folded, so codegen is unchanged. Left the native side alone. Commit b612474; clr.corelib and clr.nativeaotruntime+clr.nativeaotlibs both build clean.

Note

This reply was generated by GitHub Copilot.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Fixes a correctness issue in the CoreLib GenericCache sentinel detection used during cache flushes, preventing post-flush inserts into the shared sentinel table and avoiding stale generic-virtual-dispatch targets surviving collectible AssemblyLoadContext unloads. Adds a regression test that repeatedly load/invokes/unloads a payload assembly and verifies the virtual-function-pointer cache is cleared after each unload.

Changes:

• Update GenericCache<TKey, TValue>.TrySet to reliably detect the sentinel table via a shared SENTINEL_TABLE_SIZE constant and IsSentinel helper.
• Apply the same sentinel-size constant + helper pattern to CastCache for consistency (and tighten the constructor assert).
• Add a new Loader/CollectibleAssemblies/GenericVirtualMethod test with a payload assembly + runner validating cache flush behavior across repeated unloads.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/GenericCache.cs	Fix sentinel detection to prevent inserts into the flush sentinel table.
src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/CastCache.cs	Align sentinel sizing/detection pattern with GenericCache (consistency + assert tightening).
src/tests/Loader/CollectibleAssemblies/GenericVirtualMethod/GenericVirtualMethod.csproj	New test project enabling process isolation and referencing payload assembly.
src/tests/Loader/CollectibleAssemblies/GenericVirtualMethod/GenericVirtualMethod.cs	New regression test exercising repeated collectible unloads and validating the dispatch cache flushes.
src/tests/Loader/CollectibleAssemblies/GenericVirtualMethod/GenericVirtualMethodUnloaded.csproj	New payload (unloaded) assembly project.
src/tests/Loader/CollectibleAssemblies/GenericVirtualMethod/GenericVirtualMethodUnloaded.cs	Payload types implementing a generic virtual override that triggers the cache path.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

src/tests/Loader/CollectibleAssemblies/GenericVirtualMethod/GenericVirtualMethod.cs:85

• VerifyVirtualDispatchCacheIsEmpty relies on several reflection lookups that can return null if internal implementation details change. As written, this would surface as a NullReferenceException rather than a clear test failure pointing at the missing field/type. Adding explicit assertions for each reflection step makes the failure mode much easier to triage.

        object cache = helpersType.GetField("s_virtualFunctionPointerCache", BindingFlags.NonPublic | BindingFlags.Static).GetValue(null);
        Array table = (Array)cache.GetType().GetField("_table", BindingFlags.NonPublic | BindingFlags.Instance).GetValue(cache);

        FieldInfo infoField = table.GetType().GetElementType().GetField("_info", BindingFlags.NonPublic | BindingFlags.Instance);
        FieldInfo versionField = infoField.FieldType.GetField("_version", BindingFlags.NonPublic | BindingFlags.Instance);

src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/GenericCache.cs:149

• IsSentinel duplicates the table layout knowledge ("+ 1" for the aux slot). Since this type already centralizes the managed/native contract in CacheElementCount, using it here keeps the sentinel definition consistent if the layout ever changes and avoids repeating the +1 rule.

            return table.Length == SENTINEL_TABLE_SIZE + 1;

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/tests/Loader/CollectibleAssemblies/GenericVirtualMethod/GenericVirtualMethod.cs:30

• This test constructs a collectible AssemblyLoadContext and then polls a fixed number of times (10) for unload/collection. Other CollectibleAssemblies tests gate on PlatformDetection.IsCollectibleAssembliesSupported and skip under GC stress because fixed-count polling is unreliable there; without those guards this can be flaky or fail on platforms/configurations that don't support collectible ALCs.

    [ActiveIssue("https://github.com/dotnet/runtimelab/issues/155: Collectible assemblies", typeof(Utilities), nameof(Utilities.IsNativeAot))]
    [ActiveIssue("https://github.com/dotnet/runtime/issues/34072", TestRuntimes.Mono)]
    [Fact]
    public static void CallGenericVirtualMethodAcrossUnloads()