Add registry redirected urls to registryRedirectHosts in egress allowlist derivation

[v-abhishekbhaskar]

What are you trying to accomplish?

Since proxy-egress-enforce was enabled, jobs using a packagecloud.io or Gemfury registry fail with private_source_authentication_failure. The authenticated registry request succeeds, but both providers 302-redirect the actual file download to a signed URL on a storage host that appears in no credential field, so credentialHosts never allows it:

• packagecloud.io → d3fo0g5hm7lbuv.cloudfront.net
• *.fury.io → gemfury.s3-accelerate.dualstack.amazonaws.com and gemfury.s3-accelerate.amazonaws.com

This derives those hosts per job in registryRedirectHosts, alongside the existing ECR starport bucket, and replaces the stacked ifs there with a {credentialHost, derived} table.

Anything you want to highlight for special attention from reviewers?

Why not the static defaults: each target is one bucket shared by all of that provider's tenants, with the tenant in the URL path. Per add-egress-allowlist-domain, allowing such a host globally would expose every tenant's content to every job. Deriving per job keeps it closed for everyone who doesn't already use the registry.

Why the derivation is safe: every derived host is a constant — nothing from the credential is interpolated — and dynamic hosts are matched exactly, so a crafted credential can at worst open that one provider bucket for its own job. ECR stays outside the table because it is the only case that interpolates a credential-derived region, so it keeps its anchored regex.

Trailing dot: credential hosts are now normalised before matching, so pypi.fury.io. derives like pypi.fury.io.

How will you know you've accomplished your goal?

Reproduction of the redirect (packagecloud, public URL):

curl -sI https://packagecloud.io/github/git-lfs/packages/ubuntu/jammy/git-lfs_3.5.1_amd64.deb/download.deb
→ 302 https://d3fo0g5hm7lbuv.cloudfront.net/...?Expires=...&Signature=...

The target serves 403 to anything unsigned.

New tests in internal/handlers/egress_dynamic_hosts_test.go:

• With the matching credential, the registry and its download hosts are allowed.
• Still blocked: child hosts (evil.<host>), lookalikes (gemfuryx…, <host>.attacker.com), the non-accelerate bucket (gemfury.s3.amazonaws.com), sibling tenants (d1ii4ma7ymllif.cloudfront.net), and the shared parents (cloudfront.net, attacker.s3-accelerate…).
• Only the intended credential hosts derive anything: nothing from fury.io, pypi.fury.io.evil.com, pypifury.io, evil.packagecloud.io or packagecloud.io.attacker.com.
• Neither provider's storage is reachable by a job without that credential.

Checklist

• I have run the complete test suite to ensure all tests and linters pass.
• I have thoroughly tested my code changes to ensure they work as expected, including adding additional tests for new functionality.
• I have written clear and descriptive commit messages.
• I have provided a detailed description of the changes in the pull request, including the problem it addresses, how it fixes the problem, and any relevant details about the implementation.
• I have ensured that the code is well-documented and easy to understand.

[Copilot]

Copilot review overview

🟢 Approval recommended

The narrowly scoped derivation is exact, safely normalized, and comprehensively tested.

Review effort: Balanced
Findings: None

What changed in this PR

Adds per-job egress derivation for packagecloud and Gemfury redirect storage hosts without globally exposing multi-tenant storage.

Changes:

• Adds fixed redirect-host derivation with hostname normalization.
• Adds positive and security-boundary regression tests.

File	Description
internal/​handlers/​egress_dynamic_hosts.go	Derives provider redirect hosts from matching credentials.
internal/​handlers/​egress_dynamic_hosts_test.go	Tests allowed redirects and blocked lookalikes.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[v-robaiken]

@v-abhishekbhaskar Seems like we are doing this multiple times, it possible to make this an option in out allowlist yaml?

[v-abhishekbhaskar]

@v-robaiken I've added registry_redirect_derivations as a separate list in the yaml. I haven't added them as separate parameters in the existing list due to the following concerns-

1. The two kinds aren't interchangeable. Existing entries are unconditional and apply to every job; derivations are conditional on a credential. One shared schema makes it easy for a contributor to drop a multi-tenant host into a global list with an ignored "extra parameter". A separate key keeps the triage gate visible in the file.
2. It makes the common contribution harder, not easier. 295 entries across 565 lines today, where adding a host is one line. Map form roughly triples that and clutters the four YAML anchors (npm/bun ,  pip/uv ,  maven/gradle ,  docker/compose/devcontainers).
3. Matching semantics differ. Defaults support exact / leading-dot / glob; dynamic hosts are deliberately exact-only. A shared schema would imply globs work for derived hosts — they must not.