Skip to content

Use a portable job count for -j in the Arm build docs and notebooks - #22585

Open
ShamSaleem wants to merge 1 commit into
pytorch:mainfrom
ShamSaleem:fix/10887-arm-nproc-macos
Open

Use a portable job count for -j in the Arm build docs and notebooks#22585
ShamSaleem wants to merge 1 commit into
pytorch:mainfrom
ShamSaleem:fix/10887-arm-nproc-macos

Conversation

@ShamSaleem

@ShamSaleem ShamSaleem commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Summary

The Arm flow is supported on macOS, but every build command in its docs and notebooks uses -j$(nproc), which does not exist there.

macOS support is not incidental to this flow. The Ethos-U getting-started tutorial states it under Prerequisites — "a Linux machine with aarch64 or x86_64 processor architecture, or a macOS™ machine with Apple® Silicon"backends/arm/scripts/utils.sh:186 rejects anything but Linux and Darwin on arm64, and backends/arm/scripts/toolchain_utils.sh:31 downloads a darwin-arm64 bare-metal toolchain.

On macOS the documented command loses its -j argument entirely:

$ cmake --build ethos_u_minimal_example -j$(nproc) -- arm_executor_runner
sh: nproc: command not found
# runs as: cmake --build ethos_u_minimal_example -j -- arm_executor_runner

That is a bare -j, so the build runs unbounded rather than at the intended core count.

This replaces those 16 sites with the portable form already established in the tree by #20436:

-j$(( $(nproc 2>/dev/null || sysctl -n hw.ncpu) + 1 ))

Scope is deliberately limited to the Arm flow, where macOS is a documented host. The remaining bare -j$(nproc) sites live in backends with no stated macOS support (QNN, CUDA, Samsung, OpenVINO, Espressif) and are left alone.

Follow-up to #21455, which converted the hardcoded -jN sites; @synath raised this pattern in review there and I said it would be a separate PR. Partial fix for #10887.

The Ethos-U tutorial is generated, so backends/arm/scripts/docgen/ethos-u/ethos-u-getting-started-tutorial.md.in and its output docs/source/backends/arm-ethos-u/tutorials/ethos-u-getting-started.md are both updated; the edited line is in the template's static text, not the $MINIMAL_EXAMPLE substitution, so re-running docgen.py reproduces the committed file.

Test plan

Docs and notebooks only — no build or runtime code changes.

Verified the replacement expression on both branches. With nproc available (16 cores):

$ echo "jobs = $(( $(nproc 2>/dev/null || sysctl -n hw.ncpu) + 1 ))"
jobs = 17

With nproc absent and sysctl -n hw.ncpu reporting 8, as on macOS:

$ env -i PATH="$stub" sh -c 'echo "jobs = $(( $(nproc 2>/dev/null || sysctl -n hw.ncpu) + 1 ))"'
jobs = 9

Confirmed no bare -j$(nproc) remains in the touched files, that the four notebooks still parse as JSON, and that the generated tutorial still matches its template with the placeholder substituted.

This PR was authored with AI assistance (Claude Code).

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani

The Arm flow supports macOS on Apple Silicon: the Ethos-U getting-started
tutorial lists it under Prerequisites, backends/arm/scripts/utils.sh accepts
only Linux and Darwin on arm64, and toolchain_utils.sh fetches a darwin-arm64
bare-metal toolchain. macOS has no nproc, so every documented build command in
this flow loses its -j argument there and falls back to an unbounded parallel
build.

Replace the bare -j$(nproc) with the portable form already used elsewhere in
the tree after pytorch#20436, which prefers nproc and falls back to sysctl on Darwin.
Follow-up to pytorch#21455, which covered the hardcoded -jN sites; raised by @synath
in review there. Partial fix for pytorch#10887.
@pytorch-bot

pytorch-bot Bot commented Sep 6, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22585

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 1 Awaiting Approval, 41 Pending

As of commit f621b24 with merge base 5cdcc02 (image):

AWAITING APPROVAL - The following workflow needs approval before CI can run:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 6, 2026
@github-actions github-actions Bot added ciflow/trunk module: arm Issues related to arm backend labels Sep 6, 2026
@pytorch-bot

pytorch-bot Bot commented Sep 6, 2026

Copy link
Copy Markdown

Workflows were awaiting approval. CI has now been triggered for the ciflow labels on this PR.

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@zingo zingo added the partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm label Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: arm Issues related to arm backend partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants