add composite index on (release, timestamp) to job runs matview#3654
add composite index on (release, timestamp) to job runs matview#3654mstaeble wants to merge 1 commit into
Conversation
|
Pipeline controller notification For optional jobs, comment This repository is configured in: automatic mode |
|
Skipping CI for Draft Pull Request. |
|
Warning Review limit reached
More reviews will be available in 59 minutes and 51 seconds. Learn how PR review limits work. Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file). ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits. 🚦 How do rate limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
Walkthrough
ChangesAdditional non-unique index support for materialized views
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Suggested reviewers
🚥 Pre-merge checks | ✅ 19 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (19 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: mstaeble The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Every query against prow_job_runs_report_matview filters by release and sorts by timestamp DESC, but only id was indexed. This caused sequential scans across all ~527K rows for every paginated request. Add AdditionalIndexes support to PostgresView so matviews can have non-unique indexes alongside the required unique index, then add an index on (release, timestamp DESC). Staging benchmark for a typical paginated query: Before: 2,334ms (Parallel Seq Scan, 166K pages read) After: 0.13ms (Index Scan, 28 pages read) COUNT(*) for pagination: 120ms → 15ms (Index Only Scan). Index build cost: ~1.2s per refresh, negligible against the 14-minute matview refresh. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
94be1d9 to
6b430e2
Compare
|
Scheduling required tests: |
|
@mstaeble: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
Summary
AdditionalIndexesfield toPostgresViewfor non-unique indexes alongside the required unique index(release, timestamp DESC)index onprow_job_runs_report_matviewProblem
Every query against
prow_job_runs_report_matviewfilters byreleaseand sorts bytimestamp DESC, but onlyidwas indexed. This forced sequential scans across all ~527K rows for every paginated request to/api/jobs/runs.Benchmarks (staging)
With the matview data warm in shared_buffers, typical
/api/jobs/runsresponse times are 350-700ms. With the index, the database portion of the query drops from a sequential scan (~120ms warm for COUNT, longer for the paginated SELECT) to an index scan (15ms for COUNT, sub-millisecond for paginated SELECT). The index also eliminates the sort step since the data is pre-ordered.The improvement is more pronounced under cold-cache conditions (e.g. after a matview refresh evicts pages): the sequential scan rises to 2+ seconds while the index scan stays sub-millisecond.
Index build cost: ~1.2s per matview refresh — negligible against the ~14-minute refresh.
Test plan
go build ./...go vet ./...go test ./pkg/db/...sippy migrateon staging/api/jobs/runsresponse times improve with index present@coderabbitai pause
🤖 Generated with Claude Code