feat: add catalog search, ordering, and trigram indexes - #451
Conversation
7ac6427 to
b92e8f9
Compare
4111cfc to
585c381
Compare
Add a search parameter on the repository package catalog so clients can match groupId and artifactId by contains, including group:artifact. Support last_updated ordering and newest-first versions, and add pg_trgm GIN indexes for ILIKE filters. Assisted-By: Cursor
585c381 to
d67bd2b
Compare
There was a problem hiding this comment.
Review
1. Migration will block writes (needs fix)
migrations.AddIndex takes an ACCESS EXCLUSIVE lock — blocking all reads and writes on core_mavenpackage while the index builds. Use AddIndexConcurrently instead (the same pattern used in pulpcore's 0147_content_pulp_labels_gin.py):
from django.contrib.postgres.operations import AddIndexConcurrently, TrigramExtension
class Migration(migrations.Migration):
atomic = False # required for CONCURRENTLY
operations = [
TrigramExtension(),
AddIndexConcurrently(
model_name="mavenpackage",
index=GinIndex(fields=["group_id"], name="maven_pkg_group_id_trgm", opclasses=["gin_trgm_ops"]),
),
AddIndexConcurrently(
model_name="mavenpackage",
index=GinIndex(fields=["artifact_id"], name="maven_pkg_artifact_id_trgm", opclasses=["gin_trgm_ops"]),
),
]Note: atomic = False means no automatic rollback — a failed AddIndexConcurrently may leave an invalid index requiring manual DROP INDEX CONCURRENTLY IF EXISTS before re-running.
pg_trgm is a trusted extension on RDS PostgreSQL 13+ so TrigramExtension() will work fine with the application DB user — no DBA intervention needed.
2. Lint failure in CI
ruff reports I001 (unsorted imports) in pulp_maven/app/migrations/0010_mavenpackage.py:3 — pulpcore.app.util must move after the django imports. Fix with:
ruff check --fix pulp_maven/app/migrations/0010_mavenpackage.py
dkliban
left a comment
There was a problem hiding this comment.
Correction to my earlier review: I was wrong about TrigramExtension() requiring superuser on RDS. According to AWS docs, pg_trgm is a trusted extension on PostgreSQL 13+ — any user with CREATE privilege on the database can install it, no rds_superuser needed. The TrigramExtension() call in the migration is fine as-is.
The two remaining concerns stand: blocking index creation (use AddIndexConcurrently) and the lint failure.
📜 Checklist
Summary
This PR:
searchon repositorypackages/(contains on groupId/artifactId;group:artifactANDs both)last_updatedandorderingso clients can sort the catalogpg_trgmGIN indexes ongroup_idandartifact_idforILIKEfilters.letters-...last-segment suffix as a rebuild qualifier, so5.3.17.rhlw-00001and5.3.17.rhlw-00001-n0001share base version5.3.17Closes #450
Closes #459