Skip to content

ci: smoke-test CDK synth to catch cloud-assembly schema mismatches#1467

Open
aidandaly24 wants to merge 1 commit into
mainfrom
ci/synth-smoke-test-schema-guard
Open

ci: smoke-test CDK synth to catch cloud-assembly schema mismatches#1467
aidandaly24 wants to merge 1 commit into
mainfrom
ci/synth-smoke-test-schema-guard

Conversation

@aidandaly24
Copy link
Copy Markdown
Contributor

Description

Follow-up to #1465. That PR fixed a customer-facing deploy break — aws-cdk-lib@2.258.0 bumped the cloud-assembly schema to 54, but the bundled CDK reader only read up to 53, so freshly-created projects failed at Synthesize CloudFormation with AssemblyVersionMismatch. No test caught it. This PR adds the guard that would have.

Why nothing caught #1465

Test layer Synthesizes? Floating aws-cdk-lib? Cadence Caught it?
Unit (240) ❌ no ❌ pinned via shrinkwrap every PR No — structurally can't
build-and-test sanity create only ✅ floating every PR No — stops before synth
PR e2e ✅ yes ✅ floating PR + authorized user Only if a PR fired in the window
Full e2e ✅ yes ✅ floating Monday cron + push to main No — next run was 3 days later

The break came from an upstream transitive publish (aws-cdk-lib@2.258.0, 2026-06-04) sliding into the generated template's floating ^2.248.0 range — not from a commit. Unit tests run against the pinned shrinkwrap tree, so they test a different dependency graph than a customer's fresh npm install resolves. The one always-on job that touches a generated project stops at create and never synthesizes.

Change

Add a credential-free cdk synth smoke test immediately after agentcore create in the always-on PR build job (Node 20.x). agentcore create already auto-installs the generated CDK app against the template's floating range — resolving the latest published aws-cdk-lib — so this step writes a manifest at the newest schema and reads it back through the bundled aws-cdk binary. It goes red the moment upstream CDK's schema outruns the bundled reader.

- name: Smoke-test CDK synth (catches cloud-assembly schema mismatch)
  if: matrix.node-version == '20.x'
  working-directory: sanitytest/agentcore/cdk
  run: |
    echo '[{"name":"default","account":"000000000000","region":"us-east-1"}]' > ../aws-targets.json
    npm run cdk -- synth --quiet
    test -f cdk.out/manifest.json
  • Credential-free: synth performs no AWS calls. (deploy --dry-run can't be used here — its STS GetCallerIdentity validation runs before synth, so it would fail on missing credentials in CI without ever reaching the schema path.)
  • Catches the whole class: exercises the aws-cdk binary reader; fix(deploy): bump @aws-cdk/toolkit-lib to read cloud-assembly schema 54 #1465's break was the toolkit-lib reader. Both lag the same upstream schema, so one synth smoke test guards either reader falling behind.

Type of Change

  • Other: CI / test-coverage hardening (no runtime code change)

Testing

Verified the guard's exact command against a freshly-created project:

  • Pre-fix binary (aws-cdk@2.1100.1) + aws-cdk-lib@2.258.0 (schema 54) → fails: Cloud assembly schema version mismatch: Maximum schema version supported is 49.x.x, but found 54.0.0 — i.e. it would have caught fix(deploy): bump @aws-cdk/toolkit-lib to read cloud-assembly schema 54 #1465.

  • Shipped binary (aws-cdk@2.1126.0, from fix(deploy): bump @aws-cdk/toolkit-lib to read cloud-assembly schema 54 #1465) + aws-cdk-lib@2.258.0passes, manifest written at schema 54.

  • Confirmed agentcore create --json auto-installs the CDK app (no --skip-install), resolving the floating aws-cdk-lib@2.258.0.

  • Confirmed the synth runs with all AWS credential env unset.

  • I ran npm run typecheck (via pre-commit hook)

  • I ran npm run lint (prettier + secretlint via pre-commit hook)

  • No src/ changes — workflow-only, no snapshots affected

Checklist

  • I have read the CONTRIBUTING document
  • My changes generate no new warnings
  • Workflow run: block uses only static literals — no untrusted github.event.* interpolation

#1465 fixed a deploy break that no test caught: aws-cdk-lib@2.258.0 bumped
the cloud-assembly schema to 54, but the bundled CDK reader only read up to
53, so freshly-created projects failed at synth with AssemblyVersionMismatch.

Unit tests run against the pinned shrinkwrap, so they never exercise the
floating aws-cdk-lib range a customer's `npm install` resolves. The existing
sanity check stops at `agentcore create` and never synthesizes. The e2e suite
does synth but only on push-to-main / a weekly cron — the upstream publish
landed between runs.

Add a credential-free `cdk synth` smoke test right after `agentcore create`
in the always-on PR build job. `create` auto-installs the generated CDK app
against the template's floating range (resolving the latest aws-cdk-lib), so
this step writes a manifest at the newest schema and reads it back through the
bundled aws-cdk binary — going red the moment upstream CDK outruns our reader.

Verified: with the pre-#1465 binary (aws-cdk@2.1100.1) against
aws-cdk-lib@2.258.0 this step fails with the exact schema mismatch; with the
shipped 2.1126.0 it passes. synth performs no AWS calls, so no credentials
are required.
@aidandaly24 aidandaly24 requested a review from a team June 5, 2026 19:40
@github-actions github-actions Bot added the size/xs PR size: XS label Jun 5, 2026
@github-actions github-actions Bot added the agentcore-harness-reviewing AgentCore Harness review in progress label Jun 5, 2026
@agentcore-devx-automation agentcore-devx-automation Bot added the claude-security-reviewing Claude Code /security-review in progress label Jun 5, 2026
@agentcore-devx-automation
Copy link
Copy Markdown
Contributor

Claude Security Review: no high-confidence findings. (run)

@agentcore-devx-automation agentcore-devx-automation Bot removed the claude-security-reviewing Claude Code /security-review in progress label Jun 5, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 5, 2026

Package Tarball

aws-agentcore-0.17.0.tgz

How to install

gh release download pr-1467-tarball --repo aws/agentcore-cli --pattern "*.tgz" --dir /tmp/pr-tarball
npm install -g /tmp/pr-tarball/aws-agentcore-0.17.0.tgz

Copy link
Copy Markdown

@agentcore-cli-automation agentcore-cli-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to merge.

Tightly-scoped CI hardening that closes a real gap exposed by #1465. Verified the mechanics:

  • agentcore create --name sanitytest --language Python --framework Strands --model-provider Bedrock --memory none --json runs CDKRenderer which performs npm install in agentcore/cdk/ (no AGENTCORE_SKIP_INSTALL set in CI), so the floating aws-cdk-lib@^2.248.0 resolves to whatever is latest on npm at PR time — exactly what's needed to surface upstream schema bumps.
  • agentcore create already writes an empty aws-targets.json via writeAWSDeploymentTargets([]); the workflow's echo overwrites it with one target so synth has work to do. The path ../aws-targets.json from sanitytest/agentcore/cdk matches what bin/cdk.ts reads via ConfigIO (configRoot = path.resolve(process.cwd(), '..')). ✓
  • deployed-state.json absence is handled (try/catch in bin/cdk.ts), Bedrock provider path skips credential strategy resolution, so no API key / AWS calls needed. ✓
  • npm run cdk -- synth --quiet runs npm run build && cdk synth --quiet — gives a free TypeScript compile check on the generated project as a bonus.
  • Gating on matrix.node-version == '20.x' is correct: the bundled aws-cdk reader is the same across Node versions, so running this on each would just add ~minutes for no extra signal.

No blocking issues.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 5, 2026

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 35.54% 11182 / 31462
🔵 Statements 34.87% 11888 / 34086
🔵 Functions 30.24% 1878 / 6209
🔵 Branches 29.31% 7134 / 24334
Generated in workflow #3505 for commit a927e55 by the Vitest Coverage Report Action

@github-actions github-actions Bot removed the agentcore-harness-reviewing AgentCore Harness review in progress label Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xs PR size: XS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants