fix(dapi): bound public request work

[QuantumExplorer]

Issue being fixed or feature implemented

Applies service-owned work and retention budgets to public DAPI request paths. Covers internal review items 005, 006, 022, 023, 097, 098, 099, 100, 114, 190, 191, 192, and 401.

What was done?

• Gave the Core gRPC service a separate inbound message ceiling and enforced transaction policy before forwarding.
• Validated BIP37 limits before construction and moved accepted Bloom bytes without intermediate payload clones.
• Added process-wide admission pools for transaction, header, masternode, and state-transition-wait operations.
• Resolved and capped historical replay ranges before creating subscriptions or workers.
• Rolled back combined-stream setup on planning failures and made receiver closure visible while workers are gated.
• Added absolute handoff deadlines plus item and byte budgets for staged live events.
• Bounded mempool snapshot entries, retained bytes, and delivered-identifier state.
• Added cancellation checks between backend calls and regression coverage for narrow input and accounting boundaries.

How Has This Been Tested?

• cargo test -p rs-dapi --lib — 290 tests passed.
• Focused transaction, Bloom, selector-shape, and retained-byte boundary tests passed.
• cargo clippy -p rs-dapi --all-targets --no-deps -- -D warnings passed.
• Workspace formatting and diff checks passed.

The dependency-inclusive strict Clippy command remains blocked by unrelated pre-existing warnings in rs-dpp; the affected rs-dapi targets are clean.

Breaking Changes

Requests outside the public service's documented transaction, replay, streaming, and BIP37 work budgets now fail with an explicit resource status.

Checklist:

• I have performed a self-review of my own code.
• I have added tests that prove my fix is effective or that my feature works.
• New and existing unit tests pass locally with my changes.
• I have run formatting and strict package lint checks.

Summary by CodeRabbit

• Bug Fixes

  • Improved transaction validation with clearer errors for empty, oversized, or overweight broadcasts.
  • Added protection against excessive concurrent requests and streams.
  • Improved stream cancellation when clients disconnect.

• Improvements

  • Added bounded buffering and deduplication for transaction and block-header streams.
  • Enforced limits on pending streamed data to prevent resource exhaustion.
  • Improved historical-to-live stream handoff reliability.
  • Added validation for bloom filters and request selectors.
  • Reduced the maximum accepted Core request size to 512 KiB.

[coderabbitai]

📝 Walkthrough

Walkthrough

The PR tightens Core request and transaction validation, limits platform wait concurrency, and adds bounded admission, buffering, deduplication, cancellation, and coordinated shutdown behavior across transaction, block-header, and masternode streams.

Changes

DAPI resource controls

Layer / File(s)	Summary
Core request decoding and broadcast validation packages/rs-dapi/src/server/grpc.rs, packages/rs-dapi/src/services/core_service.rs	Core decoding is capped at 512 KiB, while broadcast transactions validate wire size, decoding, emptiness, and standard weight before submission.
Platform state-transition wait admission packages/rs-dapi/src/services/platform_service/...	Pending state-transition result waits acquire bounded semaphore permits and return ResourceExhausted when capacity is exhausted.
Streaming admission and event accounting packages/rs-dapi/src/services/streaming_service/mod.rs, .../subscriber_manager.rs, .../masternode_list_stream.rs	Per-stream semaphores are initialized, retained event bytes are measured, and masternode workers stop when the response stream closes.
Block-header replay and live-stream coordination packages/rs-dapi/src/services/streaming_service/block_header_stream.rs	Block-header streams gain bounded deduplication, gated handoff deadlines, byte/count buffering limits, disconnect checks, and coordinated worker and permit lifetimes.
Transaction stream validation and lifecycle packages/rs-dapi/src/services/streaming_service/transaction_stream.rs	Transaction streams add bounded deduplication and buffering, cancellation, selector and bloom-filter validation, terminal resource errors, and replay/live permit coordination.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant subscribe_to_transactions_with_proofs_impl
  participant process_transactions_from_height
  participant transaction_worker
  Client->>subscribe_to_transactions_with_proofs_impl: subscribe with selector and bloom filter
  subscribe_to_transactions_with_proofs_impl->>process_transactions_from_height: replay historical transactions
  process_transactions_from_height->>Client: deliver bounded, deduplicated replay
  subscribe_to_transactions_with_proofs_impl->>transaction_worker: start gated live delivery
  transaction_worker->>Client: deliver buffered and live events
  Client->>transaction_worker: close response stream
  transaction_worker->>process_transactions_from_height: cancel replay

Loading

Possibly related PRs

• dashpay/platform#4153: Updates the overlapping Core gRPC decoding size-limit behavior.

Suggested reviewers: shumkov, zocolini, llbartekll

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 68.09% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: bounding public DAPI request work.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/fix-core-input-bounds

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[thepastaclaw]

🕓 Ready for review — 66 ahead in queue (commit 2551081)
Queue position: 67/72 · 2 reviews active
ETA: start ~11:31 UTC · complete ~12:00 UTC (median 28m across 30 recent reviews; 2 slots)
Queued 2h 57m ago · Last checked: 2026-07-22 19:40 UTC

[thepastaclaw]

Preliminary review — Codex only

The PR adds useful request bounds, replay caps, and admission controls, but four in-scope blockers remain in the newly bounded paths. Malformed state-transition hashes can still consume hundreds of MiB per admitted wait, block-header handoff can emit duplicate headers, transaction budget failures can disappear behind a full response queue, and combined streams release admission permits before their replay workers finish; the transaction deduplication cap also has a boundary-ordering bug.

Validated blockers were found in the Codex precheck. Sonnet is deferred until a fresh Codex revalidation clears the blocker gate.

Review provenance

• Codex reviewers: gpt-5.6-sol — general (completed), gpt-5.6-sol — rust-quality (completed)
• Verifier: gpt-5.6-sol — verifier
• Sonnet: not run (deferred by blocker gate)

🔴 4 blocking | 🟡 1 suggestion(s)

1 additional finding(s) omitted (not in diff).

🤖 Prompt for all review comments with AI agents

These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `packages/rs-dapi/src/services/platform_service/wait_for_state_transition_result.rs`:
- [BLOCKING] packages/rs-dapi/src/services/platform_service/wait_for_state_transition_result.rs:33-53: Validate the state transition hash before admitting the wait
  A state-transition transaction ID is exactly 32 bytes, but this path rejects only an empty value. The public Platform service accepts messages up to 64 MiB, and each admitted request then retains the original bytes while creating an approximately 2x hexadecimal string, an approximately 4/3x base64 string, and another hexadecimal clone in the subscription filter. A malformed request can therefore consume hundreds of MiB, while the new semaphore still permits enough concurrent waits to exhaust process memory. Enforce the fixed-size invariant before acquiring a permit or performing either encoding.

In `packages/rs-dapi/src/services/streaming_service/block_header_stream.rs`:
- [BLOCKING] packages/rs-dapi/src/services/streaming_service/block_header_stream.rs:291-303: Do not clear historical hashes as soon as the delivery gate opens
  The gate branch is biased ahead of `block_handle.recv()`, so it can run while duplicate block notifications are still queued in the subscription's 256-item channel. `flush_pending` handles only events already moved into the local vector; immediately clearing `delivered_hashes` means a queued notification for a block included in the replay is subsequently forwarded as a new header. Even without this explicit clear, the capacity branch in `forward_event` clears the full historical set when the first non-matching live block arrives because a replay may record up to 10,000 hashes while the threshold is 2,048. Retain a bounded recent overlap window until the subscription backlog has been drained instead of clearing all replay hashes at gate opening or on the first new block.

In `packages/rs-dapi/src/services/streaming_service/transaction_stream.rs`:
- [BLOCKING] packages/rs-dapi/src/services/streaming_service/transaction_stream.rs:334-376: Do not silently discard the handoff-budget error when the response queue is full
  Both transaction subscription branches discard the result of `try_send` when the staged-event budget is exceeded. A transaction replay can produce up to 4,000 response items while the response channel holds 512, so `TrySendError::Full` is expected for a slow client. The live worker then exits, but the history and mempool workers retain sender clones and continue; after buffered successful responses are consumed, the stream can close without ever delivering `ResourceExhausted`, appearing as a successful but truncated stream. Coordinate cancellation of all workers and use a delivery path that preserves the terminal status once channel capacity becomes available.
- [BLOCKING] packages/rs-dapi/src/services/streaming_service/transaction_stream.rs:776-810: Keep the stream permit while replay and mempool workers are still running
  In combined mode, the admission permit is moved only into the live transaction worker, while the historical and mempool workers are spawned separately without it. When the response receiver closes, `transaction_worker` takes its `tx.closed()` branch and releases the permit even if a child is still in a Core RPC call. The history loop checks closure only before starting a block, so a disconnect during `get_block_hash` can still proceed through both full-block requests; tasks waiting for the Core RPC semaphore can also accumulate. Repeated connect/disconnect cycles can therefore recycle the 64 stream permits while substantially more replay work remains active. The combined-mode coordinator must retain the permit until every child is canceled or completed and cancel its `JoinSet` when the stream closes. The block-header combined path has the same split ownership between its live worker and independently spawned historical worker.
- [SUGGESTION] packages/rs-dapi/src/services/streaming_service/transaction_stream.rs:130-185: Capacity rollover reports an existing transaction as newly delivered
  The delivery helpers clear their sets before checking membership. At exactly `MAX_DELIVERED_IDS_PER_STREAM`, receiving any currently tracked transaction clears the set and makes `mark_transaction_delivered` return `true`, so the duplicate is forwarded despite the method's documented contract. The block and instant-lock helpers use the same ordering. Check membership before performing capacity rollover, and apply the same ordering consistently to all three sets; a bounded FIFO or LRU window would preserve stronger deduplication than clearing the entire set.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.49%. Comparing base (4f245ff) to head (89c7fce).
⚠️ Report is 38 commits behind head on v4.1-dev.

Additional details and impacted files

@@             Coverage Diff              @@
##           v4.1-dev    #4161      +/-   ##
============================================
- Coverage     87.49%   87.49%   -0.01%     
============================================
  Files          2663     2663              
  Lines        336414   336422       +8     
============================================
+ Hits         294354   294360       +6     
- Misses        42060    42062       +2     

Components	Coverage Δ
dpp	88.44% <ø> (ø)
drive	86.24% <ø> (-0.01%)	⬇️
drive-abci	89.55% <ø> (ø)
sdk	∅ <ø> (∅)
dapi-client	∅ <ø> (∅)
platform-version	∅ <ø> (∅)
platform-value	92.90% <ø> (ø)
platform-wallet	∅ <ø> (∅)
drive-proof-verifier	49.79% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[QuantumExplorer]

Addressed the remaining blocker in 2551081. Both combined-mode paths (transactions and block headers) now spawn a coordinator that owns the stream admission permit for the life of the stream: on disconnect it cancels the stream state, shuts down and awaits the replay/mempool JoinSet, and stops the live worker before the permit is released. Focused regression tests verify the permit is only released after the replay workers are actually gone, and that the gate still opens with the permit held after replay completes.

Also aligned the history bounds with shipped clients: dropped MAX_TRANSACTION_HISTORY_BLOCKS (2,000) and MAX_HISTORICAL_HEADERS_PER_STREAM (10,000), since wallet-lib replays its whole range in a single stream and BlockHeadersProvider targets 50,000 headers per stream — the caps turned legitimate initial syncs into RESOURCE_EXHAUSTED retry loops. Replay work remains bounded without them: both fetchers stream in per-block/per-batch chunks with awaited sends into bounded channels, the Core RPC semaphore bounds work in flight, and the admission permits (now correctly held for the full stream lifetime) bound concurrency. The replay→live dedupe window keeps its previous size as a standalone constant because live events can only overlap the tail of a replay.

Also merged v4.1-dev, reconciling with the payload-validation and metric-label changes that landed there.

[coderabbitai]

🧹 Nitpick comments (1)

packages/rs-dapi/src/services/streaming_service/block_header_stream.rs (1)

124-125: 🚀 Performance & Scalability | 🔵 Trivial | ⚖️ Poor tradeoff

Avoid resolving the historical range twice on stream setup. Both setup paths do the Core lookups once in the preflight and again in the historical worker, so each subscription pays for get_block_count (and get_block_header_info for hash starts) twice. Thread the resolved (start_height, desired/count_target) into the worker instead of recomputing it.

• packages/rs-dapi/src/services/streaming_service/block_header_stream.rs#L124-L125 / #L632-L634
• packages/rs-dapi/src/services/streaming_service/transaction_stream.rs#L328 / #L977-L994

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/rs-dapi/src/services/streaming_service/block_header_stream.rs`
around lines 124 - 125, Avoid duplicate historical-range Core lookups by passing
the preflight-resolved start height and desired count/count target into the
historical workers. Update block_header_stream.rs at lines 124-125 and 632-634,
and transaction_stream.rs at lines 328 and 977-994; ensure both setup paths
reuse the resolved values instead of calling the range-resolution logic again.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/rs-dapi/src/services/streaming_service/block_header_stream.rs`:
- Around line 124-125: Avoid duplicate historical-range Core lookups by passing
the preflight-resolved start height and desired count/count target into the
historical workers. Update block_header_stream.rs at lines 124-125 and 632-634,
and transaction_stream.rs at lines 328 and 977-994; ensure both setup paths
reuse the resolved values instead of calling the range-resolution logic again.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 92cb8b12-f651-4888-806f-4a44fbc984bd

📥 Commits

Reviewing files that changed from the base of the PR and between 16a63c8 and 2551081.

📒 Files selected for processing (9)

• packages/rs-dapi/src/server/grpc.rs
• packages/rs-dapi/src/services/core_service.rs
• packages/rs-dapi/src/services/platform_service/mod.rs
• packages/rs-dapi/src/services/platform_service/wait_for_state_transition_result.rs
• packages/rs-dapi/src/services/streaming_service/block_header_stream.rs
• packages/rs-dapi/src/services/streaming_service/masternode_list_stream.rs
• packages/rs-dapi/src/services/streaming_service/mod.rs
• packages/rs-dapi/src/services/streaming_service/subscriber_manager.rs
• packages/rs-dapi/src/services/streaming_service/transaction_stream.rs