Skip to content

Spike: run the VS Code E2E suite inside a dev container - #95

Draft
dnegstad wants to merge 3 commits into
mainfrom
claude/project-thread-gg1b4m
Draft

dnegstad wants to merge 3 commits into
mainfrom
claude/project-thread-gg1b4m

Conversation

@dnegstad

Copy link
Copy Markdown
Owner

Requested by David · project thread

Before: the VS Code E2E harness from #91 loads both extensions into one local window. The workspace extension's remote gate is opened by the DEVCONTAINER_DEV_CERTS_TEST_REMOTE seam, cross-host payloads are passed by reference, and the "container" is a temp dir on the runner.

After: a second level, npm run test:e2e:devcontainer, runs the workspace extension and a remote suite in a real remote extension host inside a dev container built from this checkout's feature. The cert payload crosses VS Code's real RPC, the gate opens on a real remoteName (the suite asserts the seam variable is unset), install.sh's SSL_CERT_DIR has to reach the extension host, and the final check is openssl verify inside the container using only that inherited SSL_CERT_DIR. The local seam-based level is unchanged.

This prototypes the resolver-extension alternative that AGENTS.md left as an open question, as an additional level rather than a replacement.

How: runDevcontainer.mjs stages .out/devcontainer-e2e/ with the local feature, brings it up with @devcontainers/cli (new pinned dev dependency), downloads desktop VS Code plus the VS Code server for the same commit (checked), and starts the server in the container with devcontainer exec so it gets the CLI's userEnvProbe. A ~40-line test-only resolver extension (test/vscode-e2e/resolver/, proposed resolvers API, enabled with --enable-proposed-api) maps vscode-remote://devcerts-e2e+e2e/... to the server's loopback-published port. The UI extension and resolver load locally; the workspace extension and remote-suite.cjs load from the container via vscode-remote:// development/test paths. Shared launcher pieces moved into launcherCommon.mjs. --container-only stops after devcontainer up plus an env probe and needs no VS Code download. A separate vscode-e2e-devcontainer CI job runs the full level.

Not covered: the Dev Containers extension itself, the dotnet dev-cert and reverse-sync paths, and non-Linux hosts.

Verified so far: lint, typecheck:e2e, check:e2e-load for both bundles, and --container-only against real Docker. The full VS Code run has not been executed yet (VS Code downloads are blocked in the session that built this); this PR's CI is its first run.


Generated by Claude Code

Adds a second E2E level that runs the workspace extension and a remote
suite in a real remote extension host inside a dev container built from
this checkout's feature. A test-only resolver extension answers a
devcerts-e2e authority with a VS Code server started in the container via
devcontainer exec, so the cert payload crosses VS Code's real RPC and the
remote gate opens on a real remoteName.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEkrfbZo8CXdZ2K6tBWLfn
@dnegstad dnegstad self-assigned this Oct 26, 2026
@dnegstad
dnegstad requested a balanced review from Copilot October 28, 2026 00:42

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The startup timeout leaves the devcontainer CLI process and output pipes open, bypassing cleanup and potentially hanging CI for 25 minutes instead of exiting after 60 seconds.

1 open issue
What changed in this PR

Adds a Linux devcontainer E2E testing level alongside the existing local harness, exercising certificate transfer between separate VS Code extension hosts.

Changes:

  • Adds a container launcher and test-only resolver to connect desktop VS Code to a matching server build.
  • Adds remote checks for certificate transfer, inherited feature settings, installation, and OpenSSL trust.
  • Shares launcher utilities and adds build, load-check, and CI support. The full remote run remains unverified in the PR description.
File Description
test/​vscode-e2e/​src/​suites/​certFlow.ts Exports the PEM comparison helper for reuse.
test/​vscode-e2e/​src/​remote/​suite.ts Adds remote certificate-flow and trust checks.
test/​vscode-e2e/​src/​remote/​index.ts Adds the remote suite entry point.
test/​vscode-e2e/​runTests.mjs Reuses shared launcher utilities.
test/​vscode-e2e/​runDevcontainer.mjs Stages the container and launches connected VS Code hosts.
test/​vscode-e2e/​resolver/​package.json Declares the test-only resolver extension.
test/​vscode-e2e/​resolver/​extension.js Resolves the test authority to the container server.
test/​vscode-e2e/​loadCheck.cjs Checks that both suite bundles load.
test/​vscode-e2e/​launcherCommon.mjs Shares fixture and sandbox setup.
test/​vscode-e2e/​esbuild.mjs Builds separate local and remote bundles.
test/​vscode-e2e/​devcontainer/​.devcontainer/​devcontainer.json Defines the container using the checkout’s feature.
package.json Adds the launcher command and Dev Containers CLI dependency.
package-lock.json Locks the added CLI dependency.
AGENTS.md Documents the new testing level and limitations.
.github/​workflows/​build-extensions.yml Adds a dedicated devcontainer E2E job.

Review effort: Balanced

Comment thread test/vscode-e2e/runDevcontainer.mjs Resolved
Without this the rejected promise left the child and its pipes alive, so
the launcher could hang until CI's job limit instead of failing at 60s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEkrfbZo8CXdZ2K6tBWLfn

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants