Skip to content

Build CPU + Thrust + OMP-offload from a single install when NVHPC is present - #11

Closed
hfwen0502 wants to merge 1 commit into
mainfrom
unify-nvhpc-backends
Closed

Build CPU + Thrust + OMP-offload from a single install when NVHPC is present#11
hfwen0502 wants to merge 1 commit into
mainfrom
unify-nvhpc-backends

Conversation

@hfwen0502

Copy link
Copy Markdown
Collaborator

Summary

Previously SBD_BUILD_BACKEND=auto built only CPU + Thrust, and the OpenMP target-offload backend (_core_gpu_omp_offload) had to be installed by itself in a separate venv, on the assumption it would conflict on the OpenMP runtime with the other backends.

That assumption no longer holds. On a GPU box all three extensions compile with NVHPC nvc++ and therefore link the same OpenMP runtime (libnvomp) — CPU via -fopenmp (nvc++ treats it as -mp), Thrust via -mp -cuda, offload via -mp=gpu. They co-load in one process without conflict, so a single pip install can now provide all three.

Changes

setup.py

  • auto: build CPU + Thrust + OMP-offload when NVHPC is present; CPU only (g++/clang) when it is not.
  • Add SBD_BUILD_BACKEND=all (builds all three; errors if nvc++ is missing).
  • When the CPU backend is built alongside any GPU backend, route the whole build through nvc++ up front so the CPU extension also links libnvomp.
  • macOS (clang) and Linux-without-NVHPC (g++) paths are unchanged. Backend selection is keyed solely on nvc++ availability — no GPU-hardware probe — so a GPU-less build node with the SDK still builds the GPU backends (build-here / run-on-compute-node stays supported).

README.md

  • Rewrite the install section for the single-venv unified build (one venv, one pip install).
  • Update the SBD_BUILD_BACKEND table (auto/all) and correct the backend co-load description and verify output.

Testing

Submodule pinned at the current vendor/sbd-upstream (21f73b4); no submodule change in this PR.

Linux + 8×H100 (NVHPC 26.3) macOS (clang)
auto builds _core_cpu + _core_gpu_thrust + _core_gpu_omp_offload _core_cpu
available_backends() ['cpu', 'gpu', 'gpu-omp'] (one process) ['cpu']
h2o (1e-3) energy -76.2359465468 — all three backends agree bit-for-bit -76.2359466293 (within 1e-4 Davidson tol)

On the GPU box the full SQD loop also runs end-to-end and converges.

…s present

Previously auto built only CPU + Thrust, and the OMP-offload backend had
to be installed alone in a separate venv on the assumption it conflicted
on the OpenMP runtime. On a GPU box all three extensions compile with
nvc++ and link the same runtime (libnvomp), so they co-load in one
process (available_backends() -> ['cpu', 'gpu', 'gpu-omp'], all agree
bit-for-bit).

setup.py:
- auto: build CPU + Thrust + OMP-offload when NVHPC is present; CPU only
  (g++/clang) otherwise.
- add SBD_BUILD_BACKEND=all (builds all three; errors without NVHPC).
- when CPU is built alongside any GPU backend, route the whole build
  through nvc++ up front so the CPU extension also links libnvomp.
- macOS (clang) and Linux-without-NVHPC (g++) paths unchanged.

README: rewrite the install section for the single-venv unified build
and correct the backend co-load description.
@hfwen0502 hfwen0502 closed this Sep 1, 2026
@hfwen0502

Copy link
Copy Markdown
Collaborator Author

the solution provided is wrong. Not valid. still need to separate out the runtime between omp offload and cpu/thrust.

@hfwen0502
hfwen0502 deleted the unify-nvhpc-backends branch September 1, 2026 00:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant