feat: cursor pagination for transfer and audit history - #136
Open
Ayom04 wants to merge 1 commit into
Open
Conversation
Offset pagination over transfer and audit history was both unstable and expensive. The window is defined by a row count, so a transfer created (or archived) while a client is paging shifts every later page: rows get repeated or skipped. Each request also re-materialised and re-filtered the whole collection to reach the requested slice. Replace it with indexed cursor pagination on both history endpoints: - Add an append-only OrderedIndex behind transfers and the audit log. Each record gets a dense, immutable sequence number, so ordering is a total order even when many records share a millisecond, and a page seek is O(1) instead of O(offset). The audit log also gains a secondary index by resourceId. - Order by an immutable creation position rather than updatedAt, so claiming, cancelling or archiving a transfer never moves it within a page. - Cursors are HMAC-signed and bound to the sort order, the normalised filter set, the collection, and the calling API token. A cursor replayed under a different token is rejected with 403; a different filter set or sort order with 400. The actor fingerprint is keyed, so a captured cursor cannot be brute-forced back to a token. - Bound every history query: limits above the maximum are rejected rather than silently clamped, offsets beyond the scan budget are refused, and a scan examines at most PAGINATION_MAX_SCAN records. A budget-truncated page is still gap-free and resumable through its nextCursor. Offset pagination keeps working unchanged - same total/count/limit/offset fields, same default ordering - and offset responses now carry a nextCursor so clients can migrate mid-walk. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Offset pagination over transfer and audit history was both unstable and expensive. The window is defined by a row count, so a transfer created (or archived) while a client is paging shifts every later page: rows get repeated or skipped. Each request also re-materialised and re-filtered the whole collection to reach the requested slice.
Replace it with indexed cursor pagination on both history endpoints:
Offset pagination keeps working unchanged - same total/count/limit/offset fields, same default ordering - and offset responses now carry a nextCursor so clients can migrate mid-walk.
closes #132