Conversation
Allow selected npm repositories to reuse rewritten metadata for an explicit short freshness window. Bound retained data and concurrent fills, coalesce equivalent requests independently of client cancellation, and preserve origin cache restrictions. Invalidate all variants on authoritative removal or revocation and prevent older fills from republishing them. Keep the feature disabled by default and document the visibility-delay tradeoff.
Invalidate retained variants and older fills when bypassed requests observe removal or revoked access. Forced bypasses discard their previous representation even when the origin fails. Recognize PrivateLink repository prefixes only for VPC endpoint origins while preserving domain isolation in full URL keys. Separate eviction coverage from expiry using an injectable cache clock, and synchronize followers before releasing coalesced fills. Add regression coverage and clarify origin scoping in the documentation.
Separate delivery to concurrent metadata callers from cache retention so errors and oversized responses do not cause serialized refetches. Keep private responses isolated and bound the total wait across retries to one minute. Expose metadata cache outcomes through bounded metrics and preserve valid responses that expand past the decoded-input limit during rewriting. Cover concurrency, deadline, isolation, and response-size behavior with regression tests. Verified the full race-enabled integration suite, repeated concurrency regressions, expanded metadata delivery, and golangci-lint.
Add explicit npm, PyPI, Cargo, and Swift route admission to a shared cache, with origin-scoped repository wildcards and encoding-aware keys. Preserve raw Python and Cargo index responses, Swift protocol headers, and existing artifact routes. Propagate origin stream failures to cache fills and spool larger captures to temporary files, cleaning them up after the last waiter. Add HTTP coverage for cross-format freshness, truncation, repository and format selection, and spool cleanup. Document the replacement metadata-cache schema and supported boundaries.
Observe authoritative origin statuses before capturing their bodies so a truncated error response cannot leave another cached representation available. The caller still receives 502 for an incomplete body. Add regressions for all invalidating statuses and for concurrent readers sharing an oversized spool when the last writer fails. The invalidation regression failed before the fix; focused race tests pass across five runs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Repeated package resolution downloads large metadata documents and repeats URL rewriting even when nothing changes. Sampled CodeArtifact metadata endpoints lacked usable validators, so conditional requests did not avoid downloading the full document.
Add an opt-in, process-local metadata cache shared across npm, PyPI, Cargo, and Swift. Format-specific route and response checks admit metadata; existing artifact download paths retain their behavior. Equivalent concurrent reads share a complete response, and each representation has a non-sliding freshness window.
Configure this inside the existing
codeartifactstrategy. The enclosing strategy supplies the origin;"*"selects all repositories at that origin. Explicit repository lists are also supported, including PrivateLink routing./simple/project/config.jsonand canonical sparse index entriesversion.jsondocumentsContent-Version: 1.Swift extensionless version URLs can return archives and remain excluded. NuGet, Maven, Ruby, PyPI distribution sidecars, and other non-admitted routes keep their existing behavior. This replaces the earlier draft
npm-metadata-cacheblock;formats = ["npm"]provides npm-only scope. Package-policy checks still run on every request, without expanding their supported formats.All selected repositories and formats share the retained-byte budget, 1,024-entry limit, and fill concurrency limit within each strategy instance. Captures above 1 MiB spool to private temporary files. Responses too large to retain are delivered and released after the last reader, including cancellations and write failures. Incomplete streams become 502 responses and cannot populate the cache. Temporary storage and active transformations require resources beyond the retained-byte budget. The existing 64 MiB decoded-input guard for JSON rewriting remains; raw PyPI and Cargo indexes are not subject to that parsing limit.
Origin restrictions, age, and shorter freshness remain authoritative. Forced refresh discards the old representation; final 401/403/404/410/451 responses invalidate all resource variants and prevent older fills from republishing them, even if the error body is truncated. No stale-on-error fallback is used. Conditional, range, cookie, and query requests bypass reuse/storage. Downstream metadata requires revalidation through Cachew. This is an explicit short-lived freshness policy, not ETag-based origin revalidation.
The block is disabled by default. Enabling it can delay visibility of new versions, yanks, removals, or permission changes by the configured interval, and each pod warms independently. Saturated fill capacity returns 503 with
Retry-After: 1; callers and service-owned fills each have a one-minute deadline. Metrics usecache_mode=metadataandtier=metadata, with no package names or URLs as labels. README documents configuration, resource requirements, and freshness implications.Validated the candidate against real CodeArtifact origins for npm, PyPI, Cargo config, Cargo sparse indexes, and Swift listings: each cold/warm pair returned identical bodies with only one upstream request. These checks exercised the local candidate HTTP handler; they were not deployed latency benchmarks or package-manager installation tests.