Add support for 304 responses with If-None-Match (Etag) and If-Modified-Since

[pedro-psb]

Closes: #7929

📜 Checklist

• Commits are cleanly separated with meaningful messages (simple features and bug fixes should be squashed to one commit)
• A changelog entry or entries has been added for any significant changes
• Follows the Pulp policy on AI Usage
• (For new features) - User documentation and test coverage has been added

See: Pull Request Walkthrough

[pedro-psb]

I've added these [SRV-XYZ] "label comments" because there are 15 different return cases in the match-and-stream (without counting the serve vs stream variations), and if you wanna think about it, or take notes about this flow, it's a bit hard. Of course AI helps a lot, but still, naming things help us humans understand and manipulate them.

I'm fine with removing if this feels too personal.

[gerrod3]

What does SRV even stand for? Serve? The problem with acronyms is that no one ever knows what they mean. I would just choose a simple 1-2 word title to label the comments if that will be helpful.

[pedro-psb]

Yes, serve hehe I kinda find this helpful to myself, but I can see it might just cause confusion to anyone else. I'll drop it.

[gerrod3]

This looks good! Thanks!

[gerrod3]

The dawn of time, aka January 1st, 1970. 😆

[dkliban]

Two follow-ups on the caching headers — one correctness/security, one a config request — plus a note on documentation.

1. Object-storage redirects should be private, not public.
When a domain has redirect_to_object_storage set, _build_response_from_content_artifact returns a 302 to a presigned/signed object-store URL, and this PR applies Cache-Control: public, max-age=0, must-revalidate to that redirect (assert_object_storage_redirect asserts it).

A presigned URL is a bearer capability — anyone holding it can fetch the object directly from the store, bypassing the content guard. public lets a shared cache store the 302, and max-age=0, must-revalidate doesn't prevent reuse: on revalidation Pulp answers 304, so the shared cache serves its stored 302, i.e. a stale presigned URL. That's a correctness bug (the URL eventually expires → clients get 403s straight from the object store) and cross-user capability reuse.

The fix is to make the redirect path emit Cache-Control: private, max-age=0, must-revalidate instead. private forbids shared caches from storing the presigned URL while still letting the requesting client cache and revalidate. Please keep the ETag/Last-Modified/304 handling on redirects exactly as-is — that's the whole point: a client that already has the artifact can send If-None-Match and get a 304, skipping both the redirect and the re-download. Only the shared-cacheability needs to change. (The Redis content cache also stores these redirects today, so it may warrant the same "don't share the signed URL" treatment.)

2. Make the Cache-Control value configurable, defaulting to the current string.
The public, max-age=0, must-revalidate default is right for a deployment fronted by an edge that revalidates and authorizes every request to origin. Deployments with a less capable or untrusted intermediary may want a different policy (e.g. private, or a longer s-maxage for unguarded public content). Exposing the value as a setting — with today's string as the default — lets operators tune it without patching, and leaves current behavior unchanged. A per-domain override (Pulp already carries per-domain serving policy like redirect_to_object_storage) would be a natural follow-up.

3. Please add user documentation for the feature.
This is a user-visible behavior change, not just an internal optimization: every content-app response now carries ETag, Last-Modified, and Cache-Control, and the app honors If-None-Match/If-Modified-Since with 304 Not Modified. It'd be great to document, under docs/:

• The new response headers and the conditional-request / 304 behavior clients and CDNs can rely on.
• The Cache-Control policy (and the redirect/config points above, once resolved), so operators know what to expect from a shared cache in front of Pulp.
• The Last-Modified heuristic and its caveat — the docstring notes it's derived from repository-version history and can be inaccurate if a distribution is rolled back to an older version/publication, which can yield a 304 for content that actually changed. ETag is unaffected (it's the exact content hash), so this is worth calling out for clients that need certainty.

---

Disclosure: Claude was used to help draft this comment.