fix: resolve GitHub release asset API URL for private repo preset and workflow downloads by lselvar · Pull Request #2855 · github/spec-kit

lselvar · 2026-06-04T20:01:56Z

Summary

Extends the private-repo GitHub release asset fix from fix: resolve GitHub release asset API URL for private repo extension downloads #2792 (extensions) to cover presets and workflows
For private/SSO repos, browser release URLs (https://github.com/<owner>/<repo>/releases/download/<tag>/<asset>) redirect to an HTML/SSO page instead of the asset, causing BadZipFile / download failures
Extracts the shared resolution logic into a reusable resolve_github_release_asset_api_url utility in _github_http.py

Fix — four download paths are now covered

PresetCatalog.download_pack (catalog-based preset install):

Resolves browser release URLs to REST API asset URLs before downloading the ZIP

preset add --from <url> (direct URL preset install):

Same resolution applied before downloading the ZIP

workflow add <url> (direct URL workflow install):

Same resolution applied before downloading the workflow YAML

workflow add <id> (catalog-based workflow install):

Same resolution applied before downloading the workflow YAML from the catalog url

Other URLs: existing behavior unchanged.

Implementation

A new shared resolve_github_release_asset_api_url(download_url, open_url_fn, timeout) function in _github_http.py contains the resolution logic. PresetCatalog delegates to it via a _resolve_github_release_asset_api_url method (same pattern as ExtensionCatalog). The __init__.py download paths import and call it directly.

Test plan

Run focused test suite:

UV_NATIVE_TLS=true SSL_CERT_FILE=/opt/homebrew/etc/openssl@3/cert.pem UV_DEFAULT_INDEX=https://pypi.org/simple PYTHONPATH=src uv run --extra test pytest tests/test_presets.py -k 'download_pack'

Expected: 2 passed

Run full preset and workflow test suite:

UV_NATIVE_TLS=true SSL_CERT_FILE=/opt/homebrew/etc/openssl@3/cert.pem UV_DEFAULT_INDEX=https://pypi.org/simple PYTHONPATH=src uv run --extra test pytest tests/test_presets.py tests/test_workflows.py tests/test_github_http.py

Expected: all passing

Verify specify preset add works for presets hosted on a private GitHub repo using browser release URLs
Verify specify preset add --from <url> works with a GitHub release asset URL from a private repo
Verify specify workflow add <id> works for catalog workflows hosted on a private GitHub repo
Verify specify workflow add <url> works with a GitHub release asset URL from a private repo

AI disclosure: This PR was developed with Claude Code assistance.

🤖 Generated with Claude Code

… workflow downloads - Add shared `resolve_github_release_asset_api_url` utility to `_github_http.py` for reuse across preset and workflow download paths - Apply the same private-repo fix from PR github#2792 (extensions) to: - `PresetCatalog.download_pack` — ZIP downloads via catalog `download_url` - `preset add --from <url>` — ZIP downloads from a direct URL - `workflow add <url>` — workflow YAML downloads from a direct URL - `workflow add <id>` (catalog) — workflow YAML downloads via catalog `url` - For browser release URLs (`github.com/…/releases/download/…`), the asset is resolved via the GitHub REST API and downloaded with `Accept: application/octet-stream` - Direct REST API asset URLs (`api.github.com/…/releases/assets/<id>`) are downloaded directly with `Accept: application/octet-stream` - Auth is preserved end-to-end through the existing `open_url` infrastructure - Update `test_download_pack_sends_auth_header` and add `test_download_pack_accepts_direct_github_rest_asset_url` to cover both paths Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

This PR extends the existing private/SSO GitHub release asset download fix to preset and workflow installation paths by resolving github.com/.../releases/download/... URLs to GitHub REST release asset URLs (api.github.com/.../releases/assets/<id>) and downloading them with Accept: application/octet-stream to avoid HTML/SSO redirects.

Changes:

Added shared GitHub release asset URL resolution helper (resolve_github_release_asset_api_url) and used it before preset/workflow downloads.
Updated PresetCatalog.download_pack and CLI download paths to pass Accept: application/octet-stream when downloading via REST asset URLs.
Expanded preset catalog tests to cover both browser release URLs and direct REST asset URLs.

Show a summary per file

File	Description
`tests/test_presets.py`	Adds coverage for resolving browser release URLs and handling direct REST asset URLs for preset ZIP downloads.
`src/specify_cli/presets.py`	Resolves GitHub release asset URLs during `PresetCatalog.download_pack` and supports extra download headers.
`src/specify_cli/_github_http.py`	Introduces reusable URL resolution helper for GitHub release asset downloads.
`src/specify_cli/__init__.py`	Applies resolution + `Accept: application/octet-stream` to direct URL and catalog-based preset/workflow download flows.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 4/4 changed files
Comments generated: 1

Encode the tag as a path segment (using quote with safe='') when building the releases/tags/<tag> API URL. This prevents malformed URLs when tags contain reserved characters like '/' or '#'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot's findings

Comments suppressed due to low confidence (1)

src/specify_cli/init.py:3091

workflow add now resolves GitHub release browser URLs (and direct REST asset URLs) via resolve_github_release_asset_api_url and conditionally adds Accept: application/octet-stream, but there doesn’t appear to be test coverage exercising the CLI download/install flow for either workflow add <url> or catalog-based workflow add <id> when the underlying URL is a GitHub release asset. A focused regression test would help prevent future regressions for private/SSO workflow installs.

        from specify_cli._github_http import resolve_github_release_asset_api_url as _resolve_gh_asset

        _wf_url_extra_headers = None
        _resolved_wf_url = _resolve_gh_asset(source, _open_url)
        if _resolved_wf_url:
            source = _resolved_wf_url
            _wf_url_extra_headers = {"Accept": "application/octet-stream"}

        import tempfile
        try:
            with _open_url(source, timeout=30, extra_headers=_wf_url_extra_headers) as resp:
                final_url = resp.geturl()
                final_parsed = urlparse(final_url)
                final_host = final_parsed.hostname or ""
                final_lb = final_host == "localhost"
                if not final_lb:
                    try:

Files reviewed: 4/4 changed files
Comments generated: 1

…solution Adds regression tests covering: - resolve_github_release_asset_api_url unit tests (passthrough, resolution, network error, URL encoding of special chars in tags) - CLI-level 'preset add --from <github-release-url>' end-to-end flow - CLI-level 'preset add --from <api-asset-url>' direct passthrough Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

mnriem · 2026-06-05T13:49:43Z

Addressed review feedback from #4436653950: added CLI-level regression tests for preset add --from <github-release-url> covering both browser URL resolution and direct API asset URL passthrough (commit 4943ef0). Also added unit tests for resolve_github_release_asset_api_url including URL-encoding of special characters in tags.

Posted on behalf of @manfred-riem by GitHub Copilot (model: Claude Opus 4.6).

Copilot

Copilot's findings

Files reviewed: 5/5 changed files
Comments generated: 14

- ExtensionCatalog._resolve_github_release_asset_api_url now delegates to the shared helper in _github_http.py (also gains URL-encoding fix) - Remove unused 'io' import from test_github_http.py - Remove duplicate 'provides' dict keys accidentally added to test_presets.py Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

mnriem · 2026-06-05T14:07:59Z

Addressed review feedback from review 4436883432 in commit 0b7222a:

Removed unused io import from tests/test_github_http.py
Removed duplicate "provides" dict keys in tests/test_presets.py (caused by an overly broad sed replacement)
Deduplicated release URL resolution logic: ExtensionCatalog._resolve_github_release_asset_api_url() now delegates to the shared helper in _github_http.py, eliminating the duplicated implementation and also gaining the URL-encoding fix for tags with special characters

All 515 tests pass (presets + extensions + github_http).

Posted on behalf of @manfred-riem by GitHub Copilot (model: Claude Opus 4.6).

Copilot

Copilot's findings

Files reviewed: 6/6 changed files
Comments generated: 3

…ests - Pass timeout=30 to resolve_github_release_asset_api_url in both workflow add paths so worst-case latency matches the download timeout - Add CLI-level regression tests for 'workflow add <url>' covering browser URL resolution and direct API asset URL passthrough Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

mnriem · 2026-06-05T14:22:16Z

Addressed review 4436978846 in commit 49ecc49:

Timeout alignment: Both workflow URL resolution calls now pass timeout=30 to match the subsequent download timeout
Workflow CLI tests: Added CLI-level regression tests for workflow add <github-release-url> (browser URL resolution + octet-stream download) and workflow add <api-asset-url> (direct passthrough)

Full suite: 3631 passed, 2 skipped.

Posted on behalf of @manfred-riem by GitHub Copilot (model: Claude Opus 4.6).

Copilot

Copilot's findings

Files reviewed: 7/7 changed files
Comments generated: 2

- Remove unused 'import urllib.request' in preset add --from path - Add CLI test for catalog-based 'workflow add <id>' with GitHub release URL resolution Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

mnriem · 2026-06-05T14:38:08Z

Addressed review 4437067323 in commit 0745ad2:

Removed unused import urllib.request (line 705)
Added CLI test for catalog-based workflow add <id> path with GitHub release URL resolution + Accept: application/octet-stream

Full suite: 3632 passed, 2 skipped.

Posted on behalf of @manfred-riem by GitHub Copilot (model: Claude Opus 4.6).

Copilot

Copilot's findings

Files reviewed: 7/7 changed files
Comments generated: 2

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot

Copilot's findings

Files reviewed: 7/7 changed files
Comments generated: 0 new

mnriem · 2026-06-05T15:41:48Z

Thank you!

…wnloads Extends the GHES support pattern from extensions and presets (github#2855, github#3157) to the bundle manifest download path: resolve_github_release_asset_api_url now receives github_hosts=github_provider_hosts() so browser release URLs from GitHub Enterprise Server instances are resolved via /api/v3 rather than falling back to the unauthenticated download path. Also adds a contract test covering the GHES resolution path for _download_remote_manifest (analogous to the existing github.com tests). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…nloads (#3136) * fix: resolve GitHub release asset API URL for private repo bundle downloads For private/SSO-protected GitHub repos, browser release download URLs (https://github.com/<owner>/<repo>/releases/download/<tag>/<asset>) redirect to an HTML/SSO page instead of delivering the asset, causing bundle manifest downloads to fail. Extends the pattern from #2855 (presets/workflows) to cover the bundle manifest download path in _download_remote_manifest: - Resolves browser release URLs to GitHub REST API asset URLs via resolve_github_release_asset_api_url before downloading - Direct REST API asset URLs (api.github.com/repos/.../releases/assets/<id>) are passed through directly - Both cases use Accept: application/octet-stream so the API returns the binary payload rather than JSON metadata - The original catalog URL is used to determine artifact format (.zip vs YAML) since the resolved API URL does not carry the file extension Adds two CLI-level contract tests: - bundle info resolves browser release URL via GitHub tags API - bundle info passes direct API asset URL through with octet-stream Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: detect ZIP payload by magic bytes; add zip and API-asset tests Address Copilot review feedback on PR #3136: 1. Detect ZIP payloads by magic bytes (PK\x03\x04) in addition to the '.zip' URL suffix so that direct GitHub REST asset URLs — which carry no file extension — are correctly routed through the ZIP extraction path when the asset is a ZIP bundle artifact. 2. Add two new contract tests: - test_bundle_info_resolves_github_browser_release_url_zip: exercises the '.zip' browser release URL path end-to-end, verifying the tags API lookup fires, octet-stream header is used, and bundle.yml is successfully extracted from the ZIP payload. - test_bundle_info_api_asset_url_zip_detected_by_magic_bytes: verifies that a direct REST asset URL returning ZIP bytes is detected by magic and parsed correctly without a tags API call. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: improve error message, broaden ZIP magic, drop unused tmp_path Address second-round Copilot review feedback on PR #3136: - Error message: when the download fails, report the original catalog download_url so the user knows which entry to fix; include the resolved REST API URL when it differs for easier debugging. - ZIP detection: broaden the magic-bytes check from PK\x03\x04 to raw[:2] == b"PK", covering all valid ZIP variants (local-file header PK\x03\x04, empty-archive PK\x05\x06, spanned/split PK\x07\x08). - Tests: remove the unused tmp_path parameter from test_bundle_info_resolves_github_browser_release_url_zip. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: use full 4-byte ZIP signatures instead of 2-byte PK prefix Address Copilot feedback: raw[:2] == b"PK" is too broad and could misclassify any payload starting with ASCII "PK" as a ZIP, producing a confusing "not a valid bundle" error. Use the three specific 4-byte ZIP magic signatures instead: PK\x03\x04 — local file header (standard ZIP) PK\x05\x06 — end-of-central-directory (empty archive) PK\x07\x08 — data descriptor / spanning marker Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: harden _download_remote_manifest parsing and tighten tests - Promote _ZIP_SIGNATURES to module-level constant (was redefined per call) - Use PurePosixPath for URL path suffix extraction so query strings and fragments are ignored and URL paths are treated as POSIX on all OSes - Move yaml/BundleManifest imports to function top to flatten the previously nested try/except into a single handler with explicit except _yaml.YAMLError and except Exception clauses - Re-add None guard on _local_manifest_source return: the function is typed Optional[BundleManifest] and without the guard a None return propagates silently to callers that degrade gracefully rather than raising an actionable error; comment explains it is defensive not dead - Assert exact resolved asset URL in browser-URL download tests, not just the Accept header, so a regression where download uses the original URL instead of the resolved one would be caught - Add resolution-failure test: when tags API finds no matching asset the code falls back to the original URL and exits non-zero with Error: Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(bundle): pass github_provider_hosts() for GHES private release downloads Extends the GHES support pattern from extensions and presets (#2855, #3157) to the bundle manifest download path: resolve_github_release_asset_api_url now receives github_hosts=github_provider_hosts() so browser release URLs from GitHub Enterprise Server instances are resolved via /api/v3 rather than falling back to the unauthenticated download path. Also adds a contract test covering the GHES resolution path for _download_remote_manifest (analogous to the existing github.com tests). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test(bundle): remove unused ghes_entry variable from GHES contract test The dict was defined but never consumed — the test drives GHES host recognition entirely through the github_provider_hosts() patch. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(bundle): include source URL in remote manifest parse errors Thread the catalog URL (and resolved API URL when it differs) into the YAML parse, generic parse, and ZIP-extraction error paths of _download_remote_manifest so failures point at the offending source instead of an opaque temp path. Addresses PR review feedback. Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Manfred Riem <15701806+mnriem@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

lselvar requested a review from mnriem as a code owner June 4, 2026 20:01

mnriem requested a review from Copilot June 5, 2026 13:15

Copilot started reviewing on behalf of mnriem June 5, 2026 13:15 View session

Copilot AI reviewed Jun 5, 2026

View reviewed changes

Comment thread src/specify_cli/_github_http.py Outdated

mnriem requested a review from Copilot June 5, 2026 13:27

Copilot started reviewing on behalf of mnriem June 5, 2026 13:28 View session

Copilot AI reviewed Jun 5, 2026

View reviewed changes

Comment thread src/specify_cli/__init__.py

mnriem requested a review from Copilot June 5, 2026 13:55

Copilot started reviewing on behalf of mnriem June 5, 2026 13:56 View session

Copilot AI reviewed Jun 5, 2026

View reviewed changes

mnriem requested a review from Copilot June 5, 2026 14:10

Copilot started reviewing on behalf of mnriem June 5, 2026 14:10 View session

Copilot AI reviewed Jun 5, 2026

View reviewed changes

Comment thread src/specify_cli/__init__.py

Comment thread src/specify_cli/__init__.py

Comment thread src/specify_cli/__init__.py Outdated

mnriem requested a review from Copilot June 5, 2026 14:23

Copilot started reviewing on behalf of mnriem June 5, 2026 14:23 View session

Copilot AI reviewed Jun 5, 2026

View reviewed changes

Comment thread src/specify_cli/__init__.py

Comment thread src/specify_cli/__init__.py

mnriem requested a review from Copilot June 5, 2026 14:38

Copilot started reviewing on behalf of mnriem June 5, 2026 14:38 View session

Copilot AI reviewed Jun 5, 2026

View reviewed changes

Comment thread tests/test_workflows.py

Comment thread tests/test_presets.py

style: remove unused MagicMock imports from tests

f00ea71

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

mnriem requested a review from Copilot June 5, 2026 14:49

Copilot AI reviewed Jun 5, 2026

mnriem requested a review from Copilot June 5, 2026 15:04

Copilot started reviewing on behalf of mnriem June 5, 2026 15:04 View session

Copilot AI reviewed Jun 5, 2026

View reviewed changes

mnriem approved these changes Jun 5, 2026

View reviewed changes

mnriem merged commit f512b8b into github:main Jun 5, 2026
11 checks passed

lselvar mentioned this pull request Jun 23, 2026

fix: resolve GitHub release asset API URL for private repo bundle downloads #3136

Merged

4 tasks

Uh oh!

Conversation

lselvar commented Jun 4, 2026

Summary

Fix — four download paths are now covered

Implementation

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

mnriem commented Jun 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mnriem commented Jun 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mnriem commented Jun 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

mnriem commented Jun 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

mnriem commented Jun 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants