feat: sync API specs from infra and belt with Copybara - #1564
Conversation
🦋 Changeset detectedLatest commit: e1c667a The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
PR SummaryMedium Risk Overview Reviewed by Cursor Bugbot for commit e1c667a. Bugbot is set up for automated code reviews on this repo. Configure here. |
Package ArtifactsBuilt from 23e0162. Download artifacts from this workflow run. JS SDK ( npm install ./e2b-2.35.4-fetch-spec-via-copybara.0.tgzCLI ( npm install ./e2b-cli-2.15.1-fetch-spec-via-copybara.0.tgzPython SDK ( pip install ./e2b-2.34.0+fetch.spec.via.copybara-py3-none-any.whl |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9602b8a2ec
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit d253e48. Configure here.
The specs in spec/ were copied from e2b-dev/infra and e2b-dev/belt by hand and had drifted far behind. They are now imported with Copybara (copy.bara.sky, run in a pinned Docker image by scripts/fetch-spec.sh): make codegen re-fetches them at the commits pinned in spec/infra-ref and spec/belt-ref before generating, and the generated-files CI check fails if the tracked copies do not match the pins. Updating a spec is now just bumping the pin and re-running make codegen. Regenerating from the current pins picks up the accumulated spec changes (renamed request schemas, SandboxNetworkConfig, FILE_TYPE_SYMLINK, access-token auth deprecation, volume path metadata tweaks) in the generated JS and Python clients. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
spec/remove_extra_tags.py filtered the REST spec down to the tags each SDK exposes before client generation. Redocly CLI's filter-in decorator (applyTo: Operation) has identical semantics — drop operations without an allowlisted tag, drop untagged operations, drop emptied paths — so the per-SDK tag lists now live in redocly.yaml and the pipelines run `redocly bundle <api>` instead. Verified: identical generated JS output; Python client models now list fields in spec order instead of the alphabetical order imposed by the script's yaml.dump (mechanical reordering, no behavior change). pyyaml is no longer needed by codegen. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the BELT_READ_TOKEN secret requirement in the generated-files check with a short-lived app token scoped to e2b-dev/belt with contents: read, minted from the org-wide Autofixer app credentials. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cp -R only overlays the staged Copybara output onto spec/, so a file deleted or renamed upstream would linger locally with green CI. Each spec now declares the paths its upstream owns and fetch-spec.sh removes them before copying, so upstream deletions surface as git deletions while repo-owned files in spec/ stay untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fork PRs can't access the Autofixer App secret, so the belt token mint now runs only for same-repo PRs; forks fall back to the default read-only token for the public infra fetches and skip the volume-content spec via the new SKIP_VOLUME_SPEC Makefile flag. Also add redocly.yaml to the generated-files paths filter so tag-filtering changes trigger the freshness check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Rename SKIP_VOLUME_SPEC to FETCH_PRIVATE_SPECS (default 1) so the flag covers any spec sourced from a private repository, not just the volume-content one. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generalize the fork-PR special case: make fetch-specs tries every spec with whatever GitHub token is available (the public infra specs also fetch anonymously) and warns and falls back to the tracked copy in spec/ when a fetch fails, instead of gating the belt fetch behind FETCH_PRIVATE_SPECS. Direct pnpm fetch:*-spec invocations still fail loudly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The minted token is used for every fetch-spec.sh run in make codegen, not just the belt one. Public-repo git reads happen to accept out-of-scope tokens, but scope the token to both repos explicitly rather than relying on that behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both SDK generate pipelines bundled the spec to the same spec/openapi_generated.yml, so the file on disk after a full codegen was whichever SDK ran last (the python view, which lacks the auth-tagged endpoints the js-sdk keeps for the CLI's /teams calls). Each pipeline now writes spec/openapi_generated.<api>.yml, which removes the latent race under parallel make and lets the two filtered views be inspected side by side. The outputs stay gitignored; the old filename remains ignored so stale copies in existing checkouts don't show up as untracked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fetch-spec.sh removed the upstream-owned paths from spec/ before copying the Copybara staging output over, so a fetch that produced an incomplete staging tree (or a copy that failed midway) could leave spec/ missing tracked files while `make fetch-specs` claimed to fall back to them. The script now verifies every expected target exists in the staging dir before touching spec/, and swaps each target in with mv — a same- filesystem rename — instead of one big cp, so there is no window where spec/ is half-populated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The upstream-owned spec/ paths were listed twice: as origin globs in copy.bara.sky and again as TARGETS in fetch-spec.sh, which staged the Copybara output and implemented the replace semantics itself. Each workflow now declares its owned paths with destination_files and the script points --folder-dir straight at spec/, so Copybara's folder destination does the delete-and-replace natively (only after a successful fetch, preserving the no-wipe-on-failure guarantee) and the script is back to resolving pins and auth. The .copybara/ staging dir is gone along with its gitignore entry. Verified: all three fetches produce byte-identical trees, a planted stale file under an owned path is deleted while repo-owned files survive, a failed fetch (bad ref) leaves spec/ untouched, and anonymous fetching still works with no token available. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
afa391b to
95c37df
Compare
…ients The upstream specs can mark endpoints and component schemas that exist in the spec but are not implemented with x-not-implemented: true. The Redocly filter-out decorator now removes those nodes before client generation. It intentionally has no applyTo so the flag works on schemas as well as operations; a kept operation that still references a dropped schema loses that media-type entry instead of keeping a dangling $ref. No-op for the current specs, which carry no such flags. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pulls the latest infra spec: new SandboxIam workload-identity schemas on NewSandbox and a 409 response on the node-status endpoint. The upstream spec also marks the SOCKS5 egress-proxy config x-not-implemented, so the new filter-out decorator drops SandboxEgressProxyConfig and the egressProxy properties from both generated clients — the first real exercise of that filter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Upstream wrapped the egressProxy $ref in allOf so x-not-implemented is a schema keyword instead of an ignored $ref sibling (infra#3383). Purely structural: the filter-out decorator already dropped the old shape, so the generated clients are unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The regenerated envd protobufs added FILE_TYPE_SYMLINK, but the hand-written FileType mappings in the JS and Python SDKs only knew FILE and DIRECTORY, so files.list() silently dropped symlink entries and getInfo()/get_info() returned an undefined/None type for them. Also pin @redocly/cli@2.39.0 as a js-sdk devDependency (matching codegen.Dockerfile) so local generate runs can't drift from CI, and drop the stale `spec` entry from pnpm-workspace.yaml left behind by the spec/package.json removal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The specs in
spec/were copied from their source repos by hand and had drifted ~2,400 lines behind infra, so they are now imported with Copybara (copy.bara.sky, run in a pinned Docker image byscripts/fetch-spec.sh):make codegenre-fetches them at the commits pinned inspec/infra-refandspec/belt-refbefore generating, and the generated-files CI check fails if the tracked copies don't match the pins. Regenerating from the current pins picks up the accumulated spec changes in the generated JS/Python clients (renamed request schemas,SandboxNetworkConfig,SandboxIamworkload identity,FILE_TYPE_SYMLINK, access-token auth deprecation, volume path-metadata tweaks). The one handwritten SDK change follows from that: the publicFileTypeenums gain aSYMLINKmember (JS and both Python surfaces) so entries envd reports as symlinks show up infiles.list()andgetInfo()/get_info()instead of being silently skipped as unknown types. The customspec/remove_extra_tags.pytag-filtering script is replaced by Redocly CLI'sfilter-indecorator (redocly.yaml), which produces identical generated JS output; afilter-outdecorator additionally drops any operation or component schema the upstream specs markx-not-implemented: true(currently the SOCKS5SandboxEgressProxyConfig/egressProxysurface, which infra flagged as spec-only); each SDK's bundle now goes to its own gitignoredspec/openapi_generated.<api>.ymlinstead of both pipelines overwriting one shared file; Python client models now list fields in spec order instead of alphabetical (mechanical reordering only — construct models with keyword args). Spec fetches try whatever GitHub token is available and fall back to the tracked copies with a warning (the public infra specs also fetch anonymously); in CI a short-lived belt-scoped token is minted from the org-wide Autofixer GitHub App (no new secrets), so fork PRs simply fall back for the belt spec; the CI workflows also cache the Copybara image alongside the codegen image, and the previously ignoredCODEGEN_IMAGEenv is honored by the Makefile.Usage
🤖 Generated with Claude Code