Skip to content

[TrimmableTypeMap] Improve incremental build performance - #12229

Merged
jonathanpeppers merged 4 commits into
mainfrom
trimmable-typemap-incremental-builds
Jul 29, 2026
Merged

[TrimmableTypeMap] Improve incremental build performance#12229
jonathanpeppers merged 4 commits into
mainfrom
trimmable-typemap-incremental-builds

Conversation

@simonrozsival

@simonrozsival simonrozsival commented Jul 24, 2026

Copy link
Copy Markdown
Member

Summary

  • update post-trim JCWs in place so unchanged Java sources keep stable timestamps
  • give linked acw-map.txt and ApplicationRegistration.java a single post-trim owner
  • make _GenerateJavaStubs react independently to pre-trim manifest changes and post-trim Java changes
  • persist and validate the expected linked-Java output set, recovering missing or stale files
  • preserve last known-good linked JCWs when generation fails
  • make the legacy llvm-ir post-trimming assembly pipeline incremental
  • reuse persisted TypeMap and Java output lists instead of wildcard/glob discovery
  • persist and compare fully qualified Java paths so the lists compare correctly on Windows
  • skip unchanged CoreCLR shrunk-assembly copies while preserving NativeAOT's always-run remap
  • stop copying *.dll.config files next to the shrunk assemblies, since assembly configuration files are not supported

Motivation

On a Release CoreCLR dotnet new maui --sample-content app, an ordinary managed source touch caused the trimmable path to delete and recreate all 566 linked JCWs. Their new timestamps forced javac and D8 to rebuild identical output:

Scenario llvm-ir Trimmable before This PR
C# source touch 50.73s 68.32s 52.31s
Android manifest touch 5.63s 39.74s 10.97s

With the generic universal-APK no-op fix from #12230 applied, the additional
trimmable-specific no-op changes in this PR reduce repeated dotnet build
from 4.58 seconds to approximately 3.0-3.5 seconds:

Change Real time
#12230 only 4.58s
Use typemap-assemblies.txt for ILLink inputs 4.02s
Reuse post-trim Java list for FileWrites 3.93s
Persist/reuse pre-trim Java list 3.85s
Incremental CoreCLR shrunk-assembly copy ~3.0-3.5s

The manifest case also exposed an output-ownership bug. The pre-trim pass overwrote the final post-trim files while the post-trim target remained up to date:

Output Correct post-trim Overwriting pre-trim
acw-map.txt 660,756 B / 6,186 lines 4,927,847 B / 49,665 lines
ApplicationRegistration.java 341 B / 12 lines 413 B / 13 lines

The overwritten registration source additionally registered android.app.Application, and its changed timestamp triggered javac and D8.

Implementation

  • CoreCLR trimmed pre-generation no longer writes the final ACW map or application-registration source.
  • The post-trim pass uses content-based in-place updates instead of wiping linked-java.
  • _GenerateJavaStubs depends on both producer stamps: pre-trim for the manifest/shared outputs and post-trim for linked JCWs.
  • linked-java-files.txt records the expected JCW set. Missing or unexpected Java files, a missing ACW map, or a missing registration source invalidate the post-trim stamp.
  • Stale pruning is suppressed after generation errors so a partial result cannot delete the previous valid output set.
  • _PostTrimmingPipeline now has explicit inputs/outputs while a separate collector target continues to repopulate its dynamic assembly items.
  • _AddTrimmableTypeMapToLinker consumes typemap-assemblies.txt instead of wildcarding the output directory.
  • java-files.txt and linked-java-files.txt drive no-op FileWrites; migration fallbacks preserve outputs from older obj trees.
  • Both Java lists persist %(FullPath) and the post-trim output base directory is normalized to forward slashes. Otherwise the recorded relative/backslash paths never matched the globbed actual paths on Windows, so every build saw the whole expected set as _UnexpectedPostTrimJavaFile and invalidated the stamp.
  • CoreCLR uses an incremental shrunk-assembly copy target. NativeAOT retains the original always-run project-local remap/copy path.
  • The trimmable path no longer copies @(_ResolvedAssemblies->'%(Identity).config') into the shrunk locations. Optional source files cannot be modeled as a reliable one-to-one incremental transform, and the copy has no observable effect: assembly configuration files are unsupported in .NET for Android (see OneDotNet.md), and the assembly store's config_data is never assigned from the store nor handed to the runtime — its only use in assembly-store.cc / embedded-assemblies.cc is a log_debug message that always prints nullptr. Dropping it makes _RemoveRegisterAttributeCoreClr a clean one-to-one transform for every project instead of forcing config-file projects onto an always-run path.

Validation

  • Debug solution build
  • Build_WithTrimmableTypeMap_PublishTrimmed_IncrementalChangesAvoidUnnecessaryJavaWork
  • Build_WithTrimmableTypeMap_PublishTrimmed_PostTrimJavaGenerationIsIncremental
  • Build_WithTrimmableTypeMap_PublishTrimmed_DeletesStaleLinkedJavaWhenLinkedJavaShrinks
  • Execute_MissingJavaSource_DoesNotPruneExistingOutput
  • Build_WithTrimmableTypeMap_MissingJavaListPreservesGeneratedJava
  • Build_WithTrimmableTypeMap_PublishTrimmed_MissingLinkedJavaListRegenerates
  • parameterized Build_WithTrimmableTypeMap_IncrementalBuild
  • AfterILLinkAdditionalStepsIsSkippedOnSecondBuild
  • BuildBasicApplicationCheckConfigFiles, AppProjectTargetsDoNotBreak
  • API 36 arm64 emulator: both trimmable and llvm-ir apps launched and survived 100 Monkey events

Follow-up

Clean trimmable builds still carry 85 managed TypeMap DLLs through ILLink, ProcessAssemblies, compression, and packaging; 49 are empty stubs. Reducing that file fan-out is a separate, larger optimization.

The llvm-ir path in Microsoft.Android.Sdk.TypeMap.LlvmIr.targets still copies *.dll.config files for the same non-reason; that copy can be removed separately.

Related to #12184

Preserve unchanged post-trim Java outputs, prevent pre-trim generation from clobbering linked metadata, and make the legacy post-trimming pipeline incremental.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: a301fa68-0e6c-4280-b2f2-bc3d4c9ae3de
Copilot AI review requested due to automatic review settings July 24, 2026 14:54

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves incremental build performance for the Trimmable TypeMap pipeline (CoreCLR trimmed builds) by avoiding unnecessary Java churn, correcting output ownership for post-trim generated files, and making the legacy llvm-ir post-trimming pipeline incremental.

Changes:

  • Updates post-trim linked-Java generation to be incremental and recover missing outputs, with explicit tracking of the expected linked-Java output set.
  • Fixes output ownership so pre-trim generation doesn’t overwrite post-trim acw-map.txt / ApplicationRegistration.java, and makes _GenerateJavaStubs react to both pre- and post-trim producer stamps.
  • Makes llvm-ir _PostTrimmingPipeline incremental via explicit inputs/outputs and a separate collector target; adds/extends test coverage around these behaviors.
Show a summary per file
File Description
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/TrimmableTypeMapBuildTests.cs Adds incremental/recovery tests for post-trim linked-Java outputs and verifies Java toolchain targets remain skipped when appropriate.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/Tasks/GenerateTrimmableTypeMapTests.cs Adds a regression test ensuring failing in-place updates don’t prune last-known-good outputs; extends test harness to collect errors.
src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/IncrementalBuildTest.cs Extends incremental test expectations to include _PostTrimmingPipeline behavior for CoreCLR.
src/Xamarin.Android.Build.Tasks/Tasks/GenerateTrimmableTypeMap.cs Adjusts stale-Java pruning behavior to avoid deleting outputs when generation has logged errors.
src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.TypeMap.Trimmable.targets Updates stamp/input wiring and prevents pre-trim generator from overwriting post-trim-owned outputs; records linked-Java file list as a tracked write.
src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.TypeMap.Trimmable.CoreCLR.targets Adds stamp invalidation when linked-Java outputs are missing/unexpected; switches post-trim generation to in-place updates and persists expected output list.
src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.TypeMap.LlvmIr.targets Makes _PostTrimmingPipeline incremental with explicit inputs/outputs, plus a collector target; ensures Deterministic participates in the property cache.
src/Microsoft.Android.Sdk.TrimmableTypeMap/README.md Updates documentation to reflect the new incremental stamp ownership model and in-place linked-Java behavior.

Copilot's findings

Comments suppressed due to low confidence (1)

src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.TypeMap.Trimmable.CoreCLR.targets:129

  • linked-java-files.txt currently records the raw @(_PostTrimGeneratedJavaFiles) item values. To make the subsequent validation resilient across OS path separators and relative/absolute differences, it should persist a normalized form (e.g., %(FullPath)) that matches what MSBuild produces from globbing.
    <WriteLinesToFile
        File="$(_PostTrimTypeMapJavaFilesList)"
        Lines="@(_PostTrimGeneratedJavaFiles)"
        Overwrite="true"
        WriteOnlyWhenDifferent="true" />
  • Files reviewed: 8/8 changed files
  • Comments generated: 2

Reuse persisted TypeMap and Java output lists, and skip unchanged CoreCLR shrunk-assembly copies while retaining NativeAOT remapping behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: a301fa68-0e6c-4280-b2f2-bc3d4c9ae3de
Persist and compare fully qualified Java paths and normalize the post-trim output base directory for reliable Windows incrementality.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: a301fa68-0e6c-4280-b2f2-bc3d4c9ae3de
@jonathanpeppers

Copy link
Copy Markdown
Member

/review

@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Android PR Reviewer completed successfully!

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Looks good — a few optional follow-ups

I reviewed the incremental-build changes across the trimmable TypeMap targets, the GenerateTrimmableTypeMap task, the README, and the added tests. This is a careful, well-motivated change and the correctness reasoning holds up under the full-file context.

What I verified

  • Output ownership fix is sound. Gating _PreTrimTypeMapAcwMapOutputFile / _PreTrimTypeMapApplicationRegistrationOutputFile to empty on CoreCLR+PublishTrimmed correctly stops the pre-trim pass from clobbering the post-trim acw-map.txt / ApplicationRegistration.java, and the FileWrites includes are guarded so the empty paths aren't registered. Matches the byte/line evidence in the PR description.
  • In-place post-trim update. Switching CleanJavaSourceOutputDirectory to false routes post-trim through the directory-scan branch of DeleteStaleJavaSources (since priorJavaSnapshot is only set when cleaning), so unchanged JCWs keep timestamps via CopyIfStreamChanged — exactly the goal.
  • Error-safety. Hoisting the Log.HasLoggedErrors guard to the top of DeleteStaleJavaSources now protects both the snapshot and scan paths, so a partial GeneratedJavaFiles after XA4255 can't prune the last known-good set. Nicely covered by Execute_MissingJavaSource_DoesNotPruneExistingOutput.
  • Stamp recovery. _InvalidatePostTrimTrimmableTypeMapStampIfOutputsMissing correctly re-runs generation when the linked-Java list, ACW map, or registration source go missing.
  • Migration fallbacks. The !Exists(...FilesList) directory-glob fallbacks keep old obj trees from having their outputs swept by IncrementalClean before the lists are backfilled — good backward-compat thinking, and tested.
  • Test coverage is excellent — incremental no-op, managed-only change (timestamp stability + hash equality), manifest-only change, missing-list migration, and stale-output recovery are all exercised.

CI: All 44 dotnet-android checks are green (mergeable_state: blocked appears to be the pending required review, not a failing check).

Inline suggestions (both non-blocking 💡):

  1. The always-run validation target reintroduces a per-build recursive linked-java/**/*.java glob — mildly at odds with this PR's glob-elimination theme.
  2. The CoreCLR shrunk-assembly incremental copy is fully disabled for any app carrying a *.dll.config; worth a tracked follow-up to model the config copy incrementally.

Neither blocks merge. Thanks for the thorough write-up, benchmarks, and tests. 🎉

Generated by Android PR Reviewer for #12229 · 176.8 AIC · ⌖ 13.5 AIC · ⊞ 6.9K
Comment /review to run again

`_RemoveRegisterAttributeCoreClr` had to opt projects containing
`*.dll.config` files out of incremental builds, because the source and
destination config files can't be modeled as a reliable one-to-one
transform (the sources are optional, the destinations are not).

That escape hatch protected a copy that has no observable effect.
Assembly configuration files are not supported by .NET for Android at
all -- see `Documentation/guides/OneDotNet.md` -- and while the assembly
store still carries a `config_data` blob, that field is never assigned
from the store data nor handed to the runtime. Its only use in
`assembly-store.cc` / `embedded-assemblies.cc` is a `log_debug` message
that always prints `nullptr`. There is no
`mono_register_config_for_assembly` call anywhere in the tree.

Drop the config copy from the trimmable path so
`_RemoveRegisterAttributeCoreClr` is a clean one-to-one incremental
transform for every project, and remove the always-run input that
existed only to disable it.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2eda38c8-0207-46a4-8e47-1ba2535800ab
@jonathanpeppers
jonathanpeppers enabled auto-merge (squash) July 28, 2026 20:25
@jonathanpeppers jonathanpeppers added the ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable). label Jul 28, 2026
@jonathanpeppers
jonathanpeppers merged commit f295019 into main Jul 29, 2026
44 checks passed
@jonathanpeppers
jonathanpeppers deleted the trimmable-typemap-incremental-builds branch July 29, 2026 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable). trimmable-type-map

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants