Skip to content

Use shared memory for broker pipes#1052

Merged
wdcui merged 14 commits into
uliteboxfrom
wdcui/ulitebox/broker-pipe-shared-only
Jul 21, 2026
Merged

Use shared memory for broker pipes#1052
wdcui merged 14 commits into
uliteboxfrom
wdcui/ulitebox/broker-pipe-shared-only

Conversation

@wdcui

@wdcui wdcui commented Jul 19, 2026

Copy link
Copy Markdown
Member

Each broker connection establishes one sealed memfd after protocol negotiation and reuses it at offset zero for serialized pipe transfers. Pipe requests carry transfer lengths and responses carry byte counts; the host and local adapters stage data through shared memory while BrokerCore remains authoritative for pipe state.

@wdcui
wdcui force-pushed the wdcui/ulitebox/broker-pipe-shared-protocol branch from 24d5337 to e94d2f0 Compare July 19, 2026 17:33
@wdcui
wdcui force-pushed the wdcui/ulitebox/broker-pipe-shared-only branch 2 times, most recently from a4005be to b41d939 Compare July 19, 2026 18:08
Base automatically changed from wdcui/ulitebox/broker-pipe-shared-protocol to ulitebox July 19, 2026 18:18
@wdcui
wdcui force-pushed the wdcui/ulitebox/broker-pipe-shared-only branch 2 times, most recently from c10f4ed to b1cf754 Compare July 21, 2026 16:51
@wdcui wdcui changed the title Make broker pipes shared-memory only Use association shared memory for broker pipes Jul 21, 2026
@wdcui wdcui changed the title Use association shared memory for broker pipes Use shared memory for broker pipes Jul 21, 2026
wdcui added 14 commits July 21, 2026 13:26
Establish one sealed memfd after broker protocol negotiation and reuse it at offset zero for serialized pipe transfers. Remove per-pipe resource responses and bump the incompatible pipe wire schema to protocol version 2.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
Use one serve_connection API with an explicit post-negotiation association setup callback.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
Use one negotiate API with an explicit post-handshake shared-memory setup callback.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
Name the post-negotiation callback after its specific responsibility of sending shared memory.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
Name the local post-negotiation callback receive_shared_memory to mirror the host send callback.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
Reserve TestSharedMemory for byte-staging tests and name inert association resources NoopSharedMemory.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
Use one request failure model across top-level, event, and pipe handlers while preserving recoverable responses and terminal shared-memory failures.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
Verify recoverable failures allow subsequent requests and shared-memory failures abort without sending a response, replacing implementation-coupled helper assertions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
Keep the single-use shared-memory staging logic directly in read_pipe and write_pipe.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
Cover setup ordering, skipped setup paths, callback errors, and invalid shared-memory sizes while removing redundant private request tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
Exercise partial transfer counts and request/response size limits while removing redundant fake handle-error coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
The shared-memory transport update does not require a broker protocol version bump.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
Use connect_with_setup_deadline directly from retry loops instead of wrapping an already-connected stream.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
Add a minimal pipe write/read round trip to the real broker-process integration test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: ac86dd2d-0280-4d7b-87a8-5654ad1503a6
@wdcui
wdcui force-pushed the wdcui/ulitebox/broker-pipe-shared-only branch from a58de7a to 76209d9 Compare July 21, 2026 20:26
@wdcui
wdcui enabled auto-merge July 21, 2026 20:26
@wdcui
wdcui added this pull request to the merge queue Jul 21, 2026
Merged via the queue into ulitebox with commit e028ac5 Jul 21, 2026
7 checks passed
@wdcui
wdcui deleted the wdcui/ulitebox/broker-pipe-shared-only branch July 21, 2026 20:33
@github-actions

Copy link
Copy Markdown

🤖 SemverChecks 🤖 ⚠️ Potential breaking API changes detected ⚠️

Click for details
--- failure function_parameter_count_changed: pub fn parameter count changed ---

Description:
A publicly-visible function now takes a different number of parameters.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#fn-change-arity
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/function_parameter_count_changed.ron

Failed in:
  litebox_broker_host::serve_connection now takes 5 parameters instead of 3, in /home/runner/work/litebox/litebox/litebox_broker_host/src/lib.rs:51

--- failure auto_trait_impl_removed: auto trait no longer implemented ---

Description:
A public type has stopped implementing one or more auto traits. This can break downstream code that depends on the traits being implemented.
        ref: https://doc.rust-lang.org/reference/special-types-and-traits.html#auto-traits
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/auto_trait_impl_removed.ron

Failed in:
  type BrokerLocal is no longer UnwindSafe, in /home/runner/work/litebox/litebox/litebox_broker_local/src/lib.rs:42
  type BrokerLocal is no longer RefUnwindSafe, in /home/runner/work/litebox/litebox/litebox_broker_local/src/lib.rs:42

--- failure inherent_method_missing: pub method removed or renamed ---

Description:
A publicly-visible method or associated fn is no longer available under its prior name. It may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/inherent_method_missing.ron

Failed in:
  BrokerLocal::request, previously in file /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/62066dddce29519d1489a2508de199b2bf9bf1da/litebox_broker_local/src/lib.rs:95

--- failure method_parameter_count_changed: pub method parameter count changed ---

Description:
A publicly-visible method now takes a different number of parameters, not counting the receiver (self) parameter.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#fn-change-arity
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/method_parameter_count_changed.ron

Failed in:
  litebox_broker_local::BrokerLocal::negotiate takes 1 parameters in /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/62066dddce29519d1489a2508de199b2bf9bf1da/litebox_broker_local/src/lib.rs:51, but now takes 2 parameters in /home/runner/work/litebox/litebox/litebox_broker_local/src/lib.rs:61

--- failure constructible_struct_adds_field: externally-constructible struct adds field ---

Description:
A pub struct constructible with a struct literal has a new pub field. Existing struct literals must be updated to include the new field.
        ref: https://doc.rust-lang.org/reference/expressions/struct-expr.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/constructible_struct_adds_field.ron

Failed in:
  field ReadPipeResponse.read in /home/runner/work/litebox/litebox/litebox_broker_protocol/src/pipe.rs:46
  field WritePipeRequest.length in /home/runner/work/litebox/litebox/litebox_broker_protocol/src/pipe.rs:55

--- failure copy_impl_added: type now implements Copy ---

Description:
A public type now implements Copy, causing non-move closures to capture it by reference instead of moving it.
        ref: https://github.com/rust-lang/rust/issues/100905
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/copy_impl_added.ron

Failed in:
  litebox_broker_protocol::pipe::ReadPipeResponse in /home/runner/work/litebox/litebox/litebox_broker_protocol/src/pipe.rs:44
  litebox_broker_protocol::pipe::WritePipeRequest in /home/runner/work/litebox/litebox/litebox_broker_protocol/src/pipe.rs:51

--- failure struct_pub_field_missing: pub struct's pub field removed or renamed ---

Description:
A publicly-visible struct has at least one public field that is no longer available under its prior name. It may have been renamed or removed entirely.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#item-remove
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/struct_pub_field_missing.ron

Failed in:
  field data of struct ReadPipeResponse, previously in file /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/62066dddce29519d1489a2508de199b2bf9bf1da/litebox_broker_protocol/src/pipe.rs:45
  field data of struct WritePipeRequest, previously in file /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/62066dddce29519d1489a2508de199b2bf9bf1da/litebox_broker_protocol/src/pipe.rs:54

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant