Skip to content

fix(drive): enforce aggregate query limits#4149

Merged
QuantumExplorer merged 14 commits into
v4.1-devfrom
codex/fix-distinct-sum-query-limit
Jul 22, 2026
Merged

fix(drive): enforce aggregate query limits#4149
QuantumExplorer merged 14 commits into
v4.1-devfrom
codex/fix-distinct-sum-query-limit

Conversation

@QuantumExplorer

@QuantumExplorer QuantumExplorer commented Jul 20, 2026

Copy link
Copy Markdown
Member

What

  • normalize caller limits against Drive defaults and maximums
  • carry the effective bound into storage query construction
  • reject any internal distinct execution path that reaches storage without a bound
  • add default, explicit, clamp, and disabled-limit regressions

Why

One aggregate query family did not preserve its request bound at the storage boundary. This change centralizes the effective limit and fails closed if a future caller omits it.

This covers PLATFORM-DS-CAND-386.

Checks

  • cargo test -p drive --features server query::drive_document_sum_query (29 passed)
  • cargo clippy -p drive --features server --all-targets -- -D warnings
  • cargo fmt --all -- --check
  • git diff --check

Summary by CodeRabbit

  • New Features

    • Added explicit range-sum modes for aggregate results and bounded distinct results.
    • Distinct range-sum queries without proofs now apply default limits, honor requested limits, and cap results at the configured maximum.
  • Bug Fixes

    • Improved handling of invalid or zero effective query limits with clear errors.
    • Increased reliability of SDK build and emulator test workflows through improved cleanup and retry handling.

@coderabbitai

coderabbitai Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 33d59b1d-d335-4caf-a8b2-d09d0861c5d0

📥 Commits

Reviewing files that changed from the base of the PR and between b7a3bd5 and 8aaad75.

📒 Files selected for processing (2)
  • .github/workflows/kotlin-sdk-build.yml
  • .github/workflows/tests-rs-workspace.yml

📝 Walkthrough

Walkthrough

The PR introduces explicit aggregate and bounded-distinct range-sum walk modes, applies configured no-proof limits, optionally prunes and stages Swift SDK Cargo artifacts, and hardens Kotlin emulator and macOS Rust CI workflows.

Changes

Range-sum walk modes

Layer / File(s) Summary
Range-sum walk-mode contract
packages/rs-drive/src/query/drive_document_sum_query/mod.rs, packages/rs-drive/src/query/mod.rs
Adds and re-exports RangeSumWalkMode, and replaces the boolean range-sum option with walk_mode.
Dispatcher and no-proof execution
packages/rs-drive/src/query/drive_document_sum_query/*, packages/rs-drive/src/query/drive_document_average_query/drive_dispatcher.rs
Selects aggregate or bounded-distinct modes, derives effective limits, and routes no-proof execution accordingly.
Limit-policy validation
packages/rs-drive/src/query/drive_document_sum_query/{drive_dispatcher.rs,tests.rs}
Tests default, explicit, capped, and zero-limit behavior.

Swift SDK target pruning

Layer / File(s) Summary
Conditional Cargo pruning and artifact staging
packages/swift-sdk/build_ios.sh
Optionally removes per-target Cargo outputs, stages final libraries and headers, and cleans staging artifacts on exit.

Kotlin emulator CI

Layer / File(s) Summary
Emulator unlock retry
.github/workflows/kotlin-sdk-build.yml
Retries keyguard dismissal up to three times and reports trust state when unlocking fails.

Rust workspace CI

Layer / File(s) Summary
macOS test timeout and cleanup
.github/workflows/tests-rs-workspace.yml
Increases the macOS job timeout and retries coverage-target cleanup.

Estimated code review effort: 3 (Moderate) | ~30 minutes

Sequence Diagram(s)

sequenceDiagram
  participant DocumentSumDispatcher
  participant execute_range_sum_no_proof
  participant distinct_sum_path_query
  DocumentSumDispatcher->>DocumentSumDispatcher: derive effective distinct limit
  DocumentSumDispatcher->>execute_range_sum_no_proof: pass RangeSumWalkMode
  execute_range_sum_no_proof->>distinct_sum_path_query: execute bounded distinct query
Loading

Possibly related PRs

Suggested reviewers: shumkov, llbartekll, zocolini

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title is concise and matches the main drive limit-enforcement change, even though the PR also covers distinct no-proof paths.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-distinct-sum-query-limit

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added this to the v4.1.0 milestone Jul 20, 2026
@QuantumExplorer
QuantumExplorer marked this pull request as ready for review July 20, 2026 13:26
@thepastaclaw

thepastaclaw commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator

🕓 Ready for review — 34 ahead in queue (commit 8aaad75)
Queue position: 35/72 · 2 reviews active
ETA: start ~03:52 UTC · complete ~04:21 UTC (median 28m across 30 recent reviews; 2 slots)
Queued 17h 1m ago · Last checked: 2026-07-22 19:40 UTC

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Codex + Sonnet

The fix correctly threads an effective, clamped limit from the dispatcher into RangeSumOptions.distinct_limit, which flows into distinct_sum_path_query's SizedQuery::limit — verified in code at execute_range_sum.rs:152-161 and path_query.rs:663 — and fails closed if a future caller omits the bound. No correctness defects found in the production changes; the two agents converge on a real test-coverage gap (the new test only exercises the pure normalization helper, not the storage boundary), plus two independent minor architecture nitpicks from Claude.

Review provenance

  • Codex reviewers: gpt-5.6-sol — general (failed), gpt-5.6-sol — general (failed), gpt-5.6-sol — general (superseded_coverage), gpt-5.6-sol — rust-quality (failed), gpt-5.6-sol — rust-quality (failed), gpt-5.6-sol — rust-quality (superseded_coverage), gpt-5.6-sol — general (completed), gpt-5.6-sol — rust-quality (completed)
  • Verifier: claude-sonnet-5 — final-verifier
  • Sonnet reviewers: claude-sonnet-5 — general (failed), claude-sonnet-5 — general (superseded_coverage), claude-sonnet-5 — rust-quality (failed), claude-sonnet-5 — rust-quality (superseded_coverage), claude-sonnet-5 — general (completed), claude-sonnet-5 — rust-quality (completed)

🟡 3 suggestion(s)

1 additional finding(s) omitted (not in diff).

🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `packages/rs-drive/src/query/drive_document_sum_query/drive_dispatcher.rs`:
- [SUGGESTION] packages/rs-drive/src/query/drive_document_sum_query/drive_dispatcher.rs:284-316: New regression test only covers the normalization helper, not the storage-boundary fix it's meant to protect
  `no_proof_distinct_limit_uses_the_default_and_clamps_to_the_maximum` calls `effective_no_proof_distinct_limit` directly and never exercises `execute_document_sum_request` with `SumMode::GroupByRange`/`GroupByCompound` and `prove: false`. I confirmed there is no existing no-proof integration test for this path either — the two `SumMode::GroupByRange` tests in `tests.rs` (lines ~444 and ~540) both set `prove: true`. That means the exact regression this PR fixes — the effective limit getting lost before reaching `SizedQuery::limit` in `distinct_sum_path_query` (path_query.rs:663) — could recur while every test in this diff continues to pass. The analogous count module already has this style of end-to-end no-proof test (`test_compound_range_in_summed_no_proof_uses_per_in_aggregate_fanout`, drive_document_count_query/tests.rs:1182, using `execute_document_count_request` against a live `Drive` with `prove: false`), so there's a direct template to follow: insert more matching documents than the limit, call the dispatcher with `prove: false` and `limit: None`/explicit/over-max, and assert the returned `Entries` are capped at the default/explicit/clamped count respectively. Also add a direct executor-level case asserting `distinct_limit: None` reaching `execute_range_sum_no_proof` in distinct mode returns the `InvalidLimit` error rather than reaching storage unbounded.
- [SUGGESTION] packages/rs-drive/src/query/drive_document_sum_query/drive_dispatcher.rs:25-46: u16::try_from error branch in effective_no_proof_distinct_limit is dead code
  `default_query_limit` and `max_query_limit` are both declared `u16` on `DriveConfig` (config.rs:52,72, confirmed). `effective_limit` is `requested_limit.unwrap_or(default_query_limit as u32).min(max_query_limit as u32)`, so after the `.min()` it can never exceed `u16::MAX` — `u16::try_from(effective_limit)` at line 41 can never return `Err`, and no test can exercise the "does not fit u16" error message because it's unreachable given the current types. A maintainer reading that error text would reasonably assume it's a real, testable failure path. Replace it with a direct `effective_limit as u16` and a short comment noting it's infallible because both `DriveConfig` fields feeding it are `u16`.

In `packages/rs-drive/src/query/drive_document_sum_query/mod.rs`:
- [SUGGESTION] packages/rs-drive/src/query/drive_document_sum_query/mod.rs:266-282: RangeSumOptions can still represent the invalid state this PR fails closed on at runtime
  `return_distinct_sums_in_range: bool` and `distinct_limit: Option<u16>` are independently settable, but `distinct_limit` is only meaningful when `return_distinct_sums_in_range` is true. `RangeSumOptions::default()` itself is safe (both fields default to false/None, a valid non-distinct combination — confirmed via `#[derive(Clone, Debug, Default)]` at mod.rs:267), but nothing stops a future caller from constructing the invalid pair via `RangeSumOptions { return_distinct_sums_in_range: true, ..Default::default() }`, which is only caught by the runtime `ok_or_else` check added in execute_range_sum.rs:152-156. Modeling this as `enum SumWalkMode { Aggregate, Distinct(u16) }` in place of the bool + Option<u16> pair would make the invalid state unrepresentable at compile time — exactly the kind of footgun this PR is otherwise trying to close for future callers — and would let you drop the runtime check entirely.

@codecov

codecov Bot commented Jul 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.49%. Comparing base (322c835) to head (f784d11).
⚠️ Report is 29 commits behind head on v4.1-dev.

Files with missing lines Patch % Lines
...uery/drive_document_sum_query/execute_range_sum.rs 75.00% 2 Missing ⚠️
...query/drive_document_sum_query/drive_dispatcher.rs 97.61% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##           v4.1-dev    #4149      +/-   ##
============================================
+ Coverage     87.47%   87.49%   +0.01%     
============================================
  Files          2661     2663       +2     
  Lines        336035   336414     +379     
============================================
+ Hits         293957   294354     +397     
+ Misses        42078    42060      -18     
Components Coverage Δ
dpp 88.44% <86.93%> (-0.04%) ⬇️
drive 86.24% <93.10%> (+0.07%) ⬆️
drive-abci 89.55% <96.00%> (+<0.01%) ⬆️
sdk ∅ <ø> (∅)
dapi-client ∅ <ø> (∅)
platform-version ∅ <ø> (∅)
platform-value 92.90% <ø> (ø)
platform-wallet ∅ <ø> (∅)
drive-proof-verifier 49.79% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/swift-sdk/build_ios.sh (1)

162-165: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Safeguard directory removal against an unset TARGET_DIR.

To prevent accidental deletion in the root filesystem if TARGET_DIR is empty or unset, require it to be set.

🛡️ Proposed fix
   rm -rf \
-    "$TARGET_DIR/aarch64-apple-ios" \
-    "$TARGET_DIR/aarch64-apple-ios-sim" \
-    "$TARGET_DIR/aarch64-apple-darwin"
+    "${TARGET_DIR:?}/aarch64-apple-ios" \
+    "${TARGET_DIR:?}/aarch64-apple-ios-sim" \
+    "${TARGET_DIR:?}/aarch64-apple-darwin"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/swift-sdk/build_ios.sh` around lines 162 - 165, Require TARGET_DIR
to be set before the rm -rf cleanup command in build_ios.sh, using a shell-safe
guard that exits or fails when it is unset or empty; preserve the existing
removal of the three target-specific directories once the variable is validated.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/swift-sdk/build_ios.sh`:
- Line 73: Safeguard all destructive cleanup paths in build_ios.sh against an
empty TARGET_DIR or target. At packages/swift-sdk/build_ios.sh lines 73-73,
update the rm command to require both variables via ${TARGET_DIR:?} and
${target:?}; at lines 162-165, replace each $TARGET_DIR reference with
${TARGET_DIR:?} while preserving the existing architecture-specific paths.

---

Nitpick comments:
In `@packages/swift-sdk/build_ios.sh`:
- Around line 162-165: Require TARGET_DIR to be set before the rm -rf cleanup
command in build_ios.sh, using a shell-safe guard that exits or fails when it is
unset or empty; preserve the existing removal of the three target-specific
directories once the variable is validated.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f0435fe1-1e06-4d7e-a11e-617eaadafc4f

📥 Commits

Reviewing files that changed from the base of the PR and between fc05d62 and 9470096.

📒 Files selected for processing (9)
  • .github/workflows/swift-sdk-build.yml
  • packages/rs-drive/src/query/drive_document_average_query/drive_dispatcher.rs
  • packages/rs-drive/src/query/drive_document_sum_query/drive_dispatcher.rs
  • packages/rs-drive/src/query/drive_document_sum_query/execute_range_sum.rs
  • packages/rs-drive/src/query/drive_document_sum_query/executors/range_no_proof.rs
  • packages/rs-drive/src/query/drive_document_sum_query/mod.rs
  • packages/rs-drive/src/query/drive_document_sum_query/tests.rs
  • packages/rs-drive/src/query/mod.rs
  • packages/swift-sdk/build_ios.sh

Comment thread packages/swift-sdk/build_ios.sh Outdated
@QuantumExplorer

Copy link
Copy Markdown
Member Author

Addressed the full review summary. ee3c9af adds storage-boundary no-proof regression coverage and removes the dead conversion branch through RangeSumWalkMode. b7a3bd5 guards all Swift build cleanup paths with non-empty TARGET_DIR and target assertions; shell syntax and explicit empty-variable abort checks pass.

@QuantumExplorer

Copy link
Copy Markdown
Member Author

reviewed

@QuantumExplorer
QuantumExplorer merged commit 4f245ff into v4.1-dev Jul 22, 2026
19 checks passed
@QuantumExplorer
QuantumExplorer deleted the codex/fix-distinct-sum-query-limit branch July 22, 2026 06:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants