feat: support custom Nsight Systems profiling arguments

[cosmith-nvidia]

Allow holoscan run --nsys-args to customize Nsight Systems profiling options and automatically enable profiling:

holoscan run my_app \
  --nsys-args='--trace=nvtx --sample=none --backtrace=none --cpuctxsw=none -o profile'

Custom options replace the complete profiler option list. --nsys-profile alone retains --trace=cuda,vulkan,nvtx,osrt; an empty --nsys-args='' uses Nsight’s defaults.

Validate quoting before building and forward profiling settings into container execution. Compose profiler and application arguments as separate argument lists, fixing profiling with --run-args. Let Nsight diagnose collector permissions instead of rejecting all captures when perf_event_paranoid > 2.

Update CLI help and documentation, with regression coverage for local execution, container forwarding, empty arguments, and malformed quoting. Downstream wrappers need updated host and container CLI versions.

Validation:

• Full unit suite: 616 passed, 2 skipped.
• All repository pre-commit checks passed.
• Real NVTX-only capture with Nsight Systems 2026.4.1 and perf_event_paranoid=4; verified five ranges and five markers through SQLite export.
• Downstream GR00T N1.7 captures completed through ./isaac_os using the earlier revision with explicit --nsys-profile; exported data-flow profiling results were successfully analyzed.

Summary by CodeRabbit

• New Features
  • holoscan run --nsys-profile can profile an application after it builds, locally or in a container. CUDA, Vulkan, NVTX, and OS runtime tracing are enabled by default.
  • --nsys-args replaces the default profiler options and also enables profiling. Use --run-args to separate application arguments from profiler options.
  • Local runs accept shell-style argument splitting without shell expansion.
• Documentation
  • Added guidance on profiler options, output paths, Nsight availability, and container behavior. run-container grants profiling permissions but does not start profiling or accept --nsys-args.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Essentials
• Run ID: 34d0bd64-4916-4f6e-a227-1d9e523c17da

---

📥 Commits

Reviewing files that changed from the base of the PR and between 8b47e68 and c116620.

---

📒 Files selected for processing (4)

• CLI_REFERENCE.md
• CONFIGURATION.md
• src/holoscan_cli/commands/run.py
• tests/unit/test_lifecycle_commands.py

---

Included review availability: This review used your included allowance. 4 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour.

---

---

Walkthrough

The run command accepts custom Nsight Systems options through --nsys-args. It parses profiler and application arguments, uses them for local profiling, and forwards profiler options through container runs. The documentation describes defaults, argument handling, and Nsight availability.

Changes

Nsight Systems profiling

Layer / File(s)	Summary
Define and validate profiler options src/holoscan_cli/commands/run.py, CONFIGURATION.md, tests/unit/test_lifecycle_commands.py	--nsys-args enables profiling and is parsed with shell-style tokenization. Without a supplied value, profiling uses --trace=cuda,vulkan,nvtx,osrt. Tests cover malformed values.
Run local profiling src/holoscan_cli/commands/run.py, tests/unit/test_lifecycle_commands.py	Local execution tokenizes string commands and passes the selected profile options and application arguments to Nsight Systems. Tests cover local execution and Nsight executable lookup. The /proc/sys/kernel/perf_event_paranoid check is removed.
Forward options through container runs src/holoscan_cli/commands/run.py, tests/unit/test_lifecycle_commands.py, CLI_REFERENCE.md	Container runs forward custom options to the nested run command. Tests cover forwarding and root mode. The reference describes profiling behavior and distinguishes run-container permissions from starting profiling.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested reviewers: wyli

Merge Risk: ⚪ Minimal · up to c1166

No concrete merge-blocking regression was established. The requested profiling options reach the application run, though the effect of string-command splitting remains unverified.

Pre-merge checks | 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: adding support for custom Nsight Systems profiling arguments.

---

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}

[wyli]

LGTM — no blocking issues. This makes NVTX-only, low-overhead Nsight captures possible through the CLI, and building the profiled command as a list of arguments also fixes --nsys-profile combined with --run-args.

Minor, non-blocking suggestion: consider making --nsys-args turn on --nsys-profile, instead of rejecting it with --nsys-args requires --nsys-profile. --nsys-args does nothing without profiling, and that combination is an error today, so the change only affects commands that currently fail. It would let downstream wrappers such as ./isaac_os offer a single option:

holoscan run my_app --nsys-args='--trace=nvtx --sample=none --backtrace=none --cpuctxsw=none -o profile'

It's a one-line change: set args.nsys_profile = True during the existing up-front validation in handle_run, so container.run(nsys_profile=...) and the forwarded run --local command both pick it up. Then update the help text and docs ("implies" instead of "requires"). --nsys-profile alone would keep the --trace=cuda,vulkan,nvtx,osrt default.

Written by Devin

[tbirdso]

Thanks Connor, LGTM

As an aside, I don't know the degree to which we've verified Holoscan CLI NSight Systems support against latest Holoscan SDK application development. Might be good to discuss internally @wyli

[tbirdso]

Confirmed the URL check failure is fully external, not related to these changes.