[Xamarin.Android.Build.Tasks] instrumentation-only apps + dotnet run

[jonathanpeppers]

Why

You can already write an Android "app" whose only entry point is an Android.App.Instrumentation subclass, with no Activity at all. This is exactly the shape you want for a BenchmarkDotNet host, or any headless on-device harness.

dotnet test handles this today, but dotnet run did not. Instrumentation auto-detection was gated behind $(EnableMSTestRunner), and several code paths in Microsoft.Android.Run assumed a launchable <activity/> existed.

What changed

Resolving what to launch. New $(AndroidUseInstrumentation) property, defaulting to true when $(EnableMSTestRunner) is true. When set, the app launches through its <instrumentation/> even if it also declares a launchable activity. Separately, $(AndroidInstrumentation) is now resolved as a fallback when the app has no launchable <activity/>, so an instrumentation-only app needs no opt-in at all. If the app has neither, the build fails with a coded error instead of an unhandled exception.

Microsoft.Android.Run fixes. Four bugs made the instrumentation path unusable for anything but MSTest:

• No -r on am instrument -w, so status blocks were buffered instead of streamed.
• A single pidof call raced the app process start and usually lost, which silently dropped all Console.WriteLine output. Now polls until the process appears.
• am instrument exits 0 even when the instrumentation crashes. Output is now parsed for INSTRUMENTATION_FAILED, shortMsg/longMsg, and INSTRUMENTATION_CODE to produce a real exit code.
• No way to pass arguments. Trailing dotnet run -- <args> are now forwarded as am instrument extras: KEY=VALUE becomes -e KEY VALUE, and anything else is collected into a single -e args "...".

$(WaitForExit). The false path hard-coded am start; it now uses am instrument (without -w) when instrumentation is set.

Error codes. XA1042 and XA1043 existed in Resources.Designer.cs but were missing from Resources.resx. Added, along with xa1042.md / xa1043.md docs.

Notes for reviewers

Does this break dotnet test? No. $(EnableMSTestRunner) opts into $(AndroidUseInstrumentation), so instrumentation resolution is byte-for-byte identical to before. dotnet test routes through RunDotnetTestAsync, which is untouched; all the new behavior is confined to RunInstrumentationAsync.

One deliberate design constraint worth flagging: I did not add a -- end-of-options separator to RunArguments. Mono.Options treats -- as a terminator, which would push --server dotnettestcli into the unparsed list and silently break MTP. Trailing user args are picked up from the remaining list instead. The known trade-off is that user args colliding with Microsoft.Android.Run's own option names (--verbose, --package, --adb) get consumed by the tool.

The failure heuristic treats INSTRUMENTATION_CODE: 0 (Activity.RESULT_CANCELED) as a failure and -1 (RESULT_OK) as success, matching what the androidtest template does. A third-party runner that uses 0 for success would be misreported.

Still not covered: pulling BenchmarkDotNet.Artifacts/ back off the device.

Testing

New DotNetRunBenchmarkDotNet device test in MSBuildDeviceIntegration. It builds an app whose MainActivity.cs is replaced with a BenchmarkDotNet host and an Instrumentation subclass, runs dotnet run -- greeting=hello --custom-flag, and asserts on arg forwarding, benchmark results, and the instrumentation exit code.

Verified on a physical Pixel 5:

Instrumentation runner: com.xamarin.dotnetrunbenchmarkdotnet.BenchmarkInstrumentation
Running instrumentation: adb shell am instrument -w -r -e greeting 'hello' -e args '--custom-flag' ...
I DOTNET  : BENCHMARK_ARGS args=--custom-flag greeting=hello
I DOTNET  : // ***** BenchmarkRunner: Start   *****
I DOTNET  : SampleBenchmarks.Sum: Dry(Toolchain=InProcessNoEmitToolchain, ...)
I DOTNET  : Mean = 444.948 us, StdErr = 0.000 us (0.00%), N = 1
I DOTNET  : Run time: 00:00:00 (0.22 sec), executed benchmarks: 1
I DOTNET  : BENCHMARKS_COMPLETE reports=1
INSTRUMENTATION_RESULT: reports=1
INSTRUMENTATION_CODE: -1

So BenchmarkDotNet does genuinely run in-process on Android via InProcessNoEmitToolchain.

DotNetRunWaitForExit also passes. Three DotNetNewAndroidTest cases fail locally, but I confirmed they are pre-existing by reverting src/ and tests/ to the parent commit and reproducing the identical failures. They are XA0101 @(Content) build action is not supported warnings coming from microsoft.testplatform.testhost 18.4.0, unrelated to this change.

Documentation

Documentation/docs-mobile/building-apps/build-properties.md updates:

• $(AndroidInstrumentation) rewritten to describe the resolution order, exit code semantics, and argument forwarding.
• New $(AndroidUseInstrumentation) section.
• $(WaitForExit) now documents that it controls whether -w is passed to am instrument, with a table of the four activity/instrumentation combinations, and a note that false means no output and no meaningful exit code.

[Copilot]

Pull request overview

This PR improves dotnet run support for Android apps that launch via an Android.App.Instrumentation entry point (including instrumentation-only apps), aligning behavior with existing dotnet test support and making headless scenarios like on-device BenchmarkDotNet hosts workable.

Changes:

• Added $(AndroidUseInstrumentation) and updated launch resolution so instrumentation can be preferred (or used as a fallback when no launchable activity exists), with coded build errors instead of exceptions.
• Fixed Microsoft.Android.Run instrumentation execution: raw status streaming (-r), logcat streaming reliability (PID polling), failure detection via output parsing, and forwarding trailing args as am instrument extras.
• Added XA1042/XA1043 resource entries + documentation and updated build property docs accordingly, plus a new device integration test covering the instrumentation-only dotnet run flow.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/MSBuildDeviceIntegration/Tests/InstallAndRunTests.cs	Adds a device test that builds and runs an instrumentation-only BenchmarkDotNet host and validates arg forwarding + exit semantics.
src/Xamarin.Android.Build.Tasks/Tasks/GetAndroidInstrumentationName.cs	Converts missing instrumentation/name cases into coded XA1042/XA1043 errors instead of exceptions.
src/Xamarin.Android.Build.Tasks/Properties/Resources.resx	Adds XA1042/XA1043 localized message entries.
src/Xamarin.Android.Build.Tasks/Properties/Resources.Designer.cs	Updates generated resource bindings/comments for XA1042/XA1043.
src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.DefaultProperties.targets	Introduces $(AndroidUseInstrumentation) defaulting behavior (auto-true for MSTest runner).
src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.Application.targets	Updates run-argument computation to resolve/choose instrumentation appropriately and uses am instrument when $(WaitForExit)=false and instrumentation is selected.
src/Microsoft.Android.Run/Program.cs	Implements instrumentation robustness fixes: -r, logcat PID polling, output parsing for failures, and arg forwarding to extras.
Documentation/docs-mobile/TOC.yml	Adds XA1042/XA1043 docs to the documentation TOC.
Documentation/docs-mobile/messages/xa1043.md	New documentation page for XA1043.
Documentation/docs-mobile/messages/xa1042.md	New documentation page for XA1042.
Documentation/docs-mobile/messages/index.md	Adds XA1042/XA1043 entries to the message index.
Documentation/docs-mobile/building-apps/build-properties.md	Updates $(AndroidInstrumentation) docs, adds $(AndroidUseInstrumentation), and clarifies $(WaitForExit) behavior for instrumentation runs.

Files not reviewed (1)

• src/Xamarin.Android.Build.Tasks/Properties/Resources.Designer.cs: Generated file

[jonathanpeppers]

/review

[github-actions]

✅ Android PR Reviewer completed successfully!

[github-actions]

🤖 Code Review — ⚠️ Minor suggestions (no blockers)

Reviewed the diff independently before reading the PR description, then reconciled. This is a well-scoped, unusually well-documented change: the dotnet run instrumentation path fixes (streaming -r, polling for the app PID instead of a single racy pidof, non-zero exit on am instrument failure, and arg forwarding) all address real root causes, and the MSBuild resolution order ($(AndroidUseInstrumentation) → activity → instrumentation fallback → coded error) is sound and preserves dotnet test behavior byte-for-byte.

Verified & liked

• GetAndroidInstrumentationName correctly converts thrown InvalidOperationExceptions into coded errors (XA1042/XA1043/XA1048) and returns !Log.HasLoggedErrors — the right pattern for an MSBuild task.
• XA1042/XA1043 were dangling in Resources.Designer.cs without .resx entries; now backfilled with proper {0} manifest-path substitution and translator comments. New XA1048 is fully wired into Resources.resx, index.md, TOC.yml, and a dedicated doc.
• Nullable handling is clean — no ! operator, uses IsNullOrEmpty () extension, NoLaunchableActivity defaults correctly.
• The -w/-r distinction and the $(WaitForExit)==false instrumentation branch (am instrument -r, no -w) match the documented intent.

Suggestions (inline, non-blocking)

• 💡 Add fast task-level unit tests for the three new coded-error branches — currently only a device-only test exercises them.
• 💡 Note the embedded-double-quote limitation in QuoteForDeviceShell, or use ArgumentList to avoid ProcessStartInfo.Arguments re-parsing.

Notes (no action required)

• The INSTRUMENTATION_CODE: 0 → failure heuristic is a reasonable default and is already called out as a known trade-off in the PR description for third-party runners that use 0 for success.
• CI shows mergeable_state: blocked (pending review approval / required checks) rather than a code failure; nothing in the diff looks like it would break the build.

Nice work — the parsing helpers and target logic are easy to follow thanks to the inline rationale comments.

Generated by Android PR Reviewer for #12261 · 127.2 AIC · ⌖ 19 AIC · ⊞ 6.9K
_{Comment /review to run again}