Skip to content

feat(cli): add high-performance daemon architecture - #1000

Draft
willkill07 wants to merge 5 commits into
NVIDIA:mainfrom
willkill07:feat/high-performance-daemon
Draft

feat(cli): add high-performance daemon architecture#1000
willkill07 wants to merge 5 commits into
NVIDIA:mainfrom
willkill07:feat/high-performance-daemon

Conversation

@willkill07

Copy link
Copy Markdown
Member

Overview

Adds the authenticated NeMo Relay daemon architecture for managed multi-user deployments, including broker-directed worker lifecycle, managed MCP and hook forwarding, raw streaming transport, and managed agent configuration.

  • I confirm this contribution is my own work, or I have the right to submit it under this project's license.
  • I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

  • Adds nemo-relay daemon, daemon mcp, daemon hook, and daemon worker with authenticated identity, broker routing, worker activation, recovery, draining, and explicit pass-through behavior.
  • Adds pooled Hyper HTTP/1.1 and HTTP/2 frame transport that preserves streamed bytes, headers, trailers, cancellation, and backpressure without response aggregation.
  • Adds immutable managed Codex, Claude Code, and Pi artifacts, including Pi custom-provider endpoint forwarding through authenticated routing metadata.
  • Preserves Codex Responses HTTP fallback behavior equivalent to PR fix: force Codex Responses fallback to HTTP #994 and the existing personal-install commands.
  • Adds correctness, security, lifecycle, concurrency, streaming-fidelity, and architecture tests under crates/cli/tests/, plus an informational daemon transport benchmark and deployment documentation.

Where should the reviewer start?

Start with crates/cli/src/daemon/mod.rs, then crates/cli/src/daemon/broker/server.rs for the control/data-plane boundary and crates/cli/tests/coverage/daemon/daemon_worker_e2e_tests.rs for end-to-end streaming guarantees.

Validation:

  • just test-rust
  • cargo clippy --workspace --all-targets -- -D warnings
  • uv run pre-commit run --all-files
  • just test-pi
  • just docs
  • just docs-linkcheck
  • just daemon-transport-benchmark-smoke

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • Closes RELAY-832

@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added size:XXL PR is very large Feature a new feature lang:js PR changes/introduces Javascript/Typescript code lang:rust PR changes/introduces Rust code labels Sep 4, 2026
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown

License Diff

Compared against origin/main.

Lockfile license changes

Lockfile License Changes

Rust

Added

  • None

Removed

  • None

Updated/Changed

h2

Before:

  • h2 0.4.16 (MIT)

After:

  • h2 0.4.19 (MIT)

hyper

Before:

  • hyper 1.9.0 (MIT)

After:

  • hyper 1.11.1 (MIT)

Node

Added

  • None

Removed

  • None

Updated/Changed

  • None

Python

Added

  • None

Removed

  • None

Updated/Changed

  • None
Status output
[license-diff] selected languages: rust, node, python
[license-diff] generating current inventory
[license-diff] current: generating Rust inventory
[license-diff] current: Rust inventory complete (457 packages)
[license-diff] current: generating Node inventory
[license-diff] current: Node inventory complete (367 packages)
[license-diff] current: generating Python inventory
[license-diff] current: Python inventory complete (105 packages)
[license-diff] current inventory complete
[license-diff] checking out base ref origin/main into a temporary worktree
[license-diff] base: generating Rust inventory
[license-diff] base: Rust inventory complete (457 packages)
[license-diff] base: generating Node inventory
[license-diff] base: Node inventory complete (367 packages)
[license-diff] base: generating Python inventory
[license-diff] base: Python inventory complete (105 packages)
[license-diff] base inventory complete
[license-diff] removing temporary base worktree
[license-diff] comparing inventories
[license-diff] rendering Markdown output
[license-diff] done

@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown

Signed-off-by: Will Killian <wkillian@nvidia.com>
Signed-off-by: Will Killian <wkillian@nvidia.com>
Signed-off-by: Will Killian <wkillian@nvidia.com>
Signed-off-by: Will Killian <wkillian@nvidia.com>
@willkill07
willkill07 force-pushed the feat/high-performance-daemon branch from a6c8e52 to 834dda8 Compare September 5, 2026 01:06
Signed-off-by: Will Killian <wkillian@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Feature a new feature lang:js PR changes/introduces Javascript/Typescript code lang:rust PR changes/introduces Rust code size:XXL PR is very large

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant