feat: add --ouput json flag (sam deploy) - #9146
Conversation
Add --output json option to sam deploy that emits newline-delimited JSON (NDJSON) streaming output instead of human-formatted tables. Each deploy stage emits structured JSON lines in real-time as CloudFormation events arrive, preventing process timeouts during long-running deployments. Line types emitted: - changeset: one per resource change (action, logical_id, resource_type) - event: streaming CFN stack events during deploy - outputs: stack outputs after completion - result: final status line (SUCCESS/FAILED/NO_CHANGES/CHANGESET_CREATED)
Ensures stdout is pure NDJSON when --output json is used by: - Forcing no_progressbar=True on both PackageContext and DeployContext S3 uploaders, preventing upload progress lines from leaking to stderr - Raising samcli logger to WARNING level to suppress LOG.info messages
github-actions
Bot
added
area/deploy
madhavdonthula1
changed the title
![aws-sam-tooling-bot[bot]](https://avatars.githubusercontent.com/b/1290?s=60&v=4){kind=small}
| sys.stdout.flush() | ||
|
|
||
| self.describe_stack_events(stack_name, time_stamp_marker, on_failure) | ||
| self.describe_stack_events(stack_name, time_stamp_marker, on_failure, output_mode=self.output_mode) |
There was a problem hiding this comment.
[BUG] rollback_delete_stack() calls self.describe_stack_events(...) at two sites (currently around lines 830 and 848 in the base file) without passing output_mode. Every other call site was updated to propagate output_mode=self.output_mode, but the rollback/delete path was not.
When a deployment fails with --output json and --on-failure DELETE (or when a DeployFailedError is caught and rollback is triggered), the sequence in deploy_context.py is:
- NDJSON event records stream to stdout during wait_for_execute.
- A DeployFailedError is caught and a {"type": "result", "status": "FAILED", ...} line is written to stdout.
- self.deployer.rollback_delete_stack(stack_name) is invoked, which calls describe_stack_events in the default (text) mode.
- The @pprint_column_names decorator prints the table header, dashes and each event row via click.secho(...) — all of which go to stdout.
The result is that the NDJSON stream on stdout is interleaved with the human-readable table (header, ----------, columns and closing ----------), which will make consumers piping the JSON output fail to parse. This defeats the purpose of --output json on exactly the code path where machine-readable failure info is most valuable.
Fix by threading output_mode through rollback_delete_stack:
# In Deployer.rollback_delete_stack, both call sites:
self.describe_stack_events(stack_name, marker_time, FailureMode.DELETE, output_mode=self.output_mode)Since self.output_mode is already stored on the Deployer instance, no new plumbing is required — this is the same pattern that was applied to wait_for_execute on line 613 and create_and_wait_for_changeset.
…eaming Thread output_mode=self.output_mode through the two describe_stack_events calls in rollback_delete_stack, preventing table-formatted text from leaking into the NDJSON stream during failure/rollback scenarios.
There was a problem hiding this comment.
Code Review Results
Reviewed: fe8f8db..16998d9
Files: 8
Comments: 3
Comments on lines outside the diff:
[samcli/commands/deploy/command.py:338] [BUG] Only print_managed_s3_bucket_info is now gated on output != "json", but manage_stack itself is not:
s3_bucket = manage_stack(profile=profile, region=region)
if output != "json":
print_managed_s3_bucket_info(s3_bucket)manage_stack delegates to samcli.lib.utils.managed_cloudformation_stack.manage_stack, which writes free-form messages directly to stdout via click.echo ("\tCreating the required resources...", "\tSuccessfully created!", "\tUpdating the required resources...", "\tSuccessfully updated!"). On a first-time sam deploy --resolve-s3 --output json (or any run where the managed bucket stack needs to be created/updated) those plain-text lines are interleaved into the NDJSON stream on stdout and break any consumer that parses each line as JSON.
Either route the managed-stack bootstrap messages through the same JSON/text switch (e.g. thread output/output_mode into manage_stack) or suppress its stdout when output == "json" so the JSON stream stays parseable.
| return | ||
|
|
||
| if confirm_changeset: | ||
| if confirm_changeset and self.output != "json": |
There was a problem hiding this comment.
[BUG] When --output json is combined with --confirm-changeset, the entire confirmation block is silently skipped and the changeset is executed without user consent:
if confirm_changeset and self.output != "json":
click.secho(self.MSG_CONFIRM_CHANGESET_HEADER, fg="yellow")
click.secho("=" len(self.MSGCONFIRM_CHANGESET_HEADER), fg="yellow")
if not click.confirm(f"{self.MSG_CONFIRM_CHANGESET}", default=False):
returnBecause self.output != "json" gates the whole block, in JSON mode the return path is unreachable — so sam deploy --confirm-changeset --output json proceeds straight to execute_changeset without the confirmation the user explicitly requested. This is a safety regression: users can lose the ability to review a changeset before it is applied.
Two safer options:
- Fail early when the two flags are combined (mutually-exclusive validation in
command.py), or - Keep the confirmation gate but replace the interactive prompt with an auto-deny in JSON mode (e.g. emit
{"type": "result", "status": "CONFIRMATION_REQUIRED", ...}andreturn), so the deploy does not proceed silently.
At minimum, only the interactive click.confirm should be gated on self.output != "json", not the outer if confirm_changeset: and its return.
| click.echo(str(ex)) | ||
| except deploy_exceptions.DeployFailedError as ex: | ||
| if self.output == "json": | ||
| sys.stdout.write(json.dumps({"type": "result", "status": "FAILED", "error": str(ex)}) + "\n") |
There was a problem hiding this comment.
[BUG] The FAILED terminal result is written before the rollback/delete work runs:
except deploy_exceptions.DeployFailedError as ex:
if self.output == "json":
sys.stdout.write(json.dumps({"type": "result", "status": "FAILED", "error": str(ex)}) + "\n")
sys.stdout.flush()
# Failed to deploy, check for DELETE action otherwise skip
if self.on_failure == FailureMode.DELETE:
self.deployer.rollback_delete_stack(stack_name)
raiseWhen --on-failure DELETE is in effect, rollback_delete_stack then calls describe_stack_events(..., output_mode=self.output_mode), which streams additional {"type": "event", ...} records after the {"type": "result", "status": "FAILED"} line. Consumers that reasonably treat a result record as terminal will stop reading and miss the rollback/delete progress, and the emitted stream ends up looking like FAILED followed by more events with no closing record.
Emit the FAILED result after the rollback path completes (or emit a distinct intermediate record for the failure and a single terminal result at the end of the except block). That way the ordering matches the semantics claimed by the PR ("streams in real-time as CloudFormation events arrive") and there is exactly one terminal record per invocation.
When --output json and --confirm-changeset are both set, emit a structured result line with status CONFIRMATION_REQUIRED and return without executing, instead of silently skipping the confirmation and proceeding with the deploy.
Move the FAILED result line to after rollback_delete_stack finishes, ensuring the result record is always the last line in the NDJSON stream. Previously it was emitted before rollback, causing event records to appear after the terminal result line.
There was a problem hiding this comment.
Code Review Results
Reviewed: fe8f8db..9d3405b
Files: 8
Comments: 1
Comments on lines outside the diff:
[samcli/commands/deploy/command.py:338] [BUG] manage_stack() still runs unconditionally and writes plain text to stdout via click.echo, corrupting the NDJSON stream in --output json mode. Only print_managed_s3_bucket_info is gated on output != "json", but the underlying manage_stack -> manage_cloudformation_stack -> _create_stack / _update_stack path emits messages such as:
# samcli/lib/utils/managed_cloudformation_stack.py
click.echo("\tCreating the required resources...") # line 248
click.echo("\tSuccessfully created!") # line 270
click.echo("\tUpdating the required resources...") # line 280
click.echo("\tSuccessfully updated!") # line 294Any first-time sam deploy --resolve-s3 --output json (or any run that triggers an update of the SAM-managed bootstrap stack) will interleave these tab-indented text lines with the JSON records emitted later, breaking downstream jq / NDJSON consumers.
Wrap the whole manage_stack + info-print block in the same guard, or redirect stdout while creating the managed stack. For example:
if resolve_s3:
if bool(s3_bucket):
raise DeployResolveS3AndS3SetError()
if output == "json":
with open(os.devnull, "w") as devnull, contextlib.redirect_stdout(devnull):
s3_bucket = manage_stack(profile=profile, region=region)
else:
s3_bucket = manage_stack(profile=profile, region=region)
print_managed_s3_bucket_info(s3_bucket)This issue was raised in a prior review round and has not been addressed in the current diff.
Redirect stdout to /dev/null while manage_stack() runs in JSON mode,
preventing its click.echo calls ("Creating the required resources...",
"Successfully created!") from corrupting the NDJSON stream.
| raise DeployResolveS3AndS3SetError() | ||
| s3_bucket = manage_stack(profile=profile, region=region) | ||
| print_managed_s3_bucket_info(s3_bucket) | ||
| if output == "json": |
There was a problem hiding this comment.
[BUG] The --output flag is only handled in the non-guided branch. When a user runs sam deploy --guided --output json, the if guided: branch (which precedes this block) invokes guided_context.run(), which unconditionally writes decorative text via click.echo/click.secho and blocks on interactive prompts (click.prompt, click.confirm) in samcli/commands/deploy/guided_context.py (e.g., lines 128, 149, 153, 165, 186, 191) and guided_config.py. As a result:
- The NDJSON stream on stdout is corrupted by yellow-tinted prose (
"Setting default arguments for 'sam deploy'","Looking for resources needed for deployment:", managed-bucket info, etc.). - Interactive prompts hang indefinitely because a JSON-consuming caller cannot supply keyboard input.
Please either thread output into the guided flow and suppress its output there too, or reject the combination early in cli():
if guided and output == "json":
raise click.UsageError("--guided is not compatible with --output json")|
|
||
| for resource, authorization_required in auth_required_per_resource: | ||
| if not authorization_required: | ||
| if not authorization_required and self.output != "json": |
There was a problem hiding this comment.
[BUG] The auth warning is silently dropped in JSON mode with no equivalent structured event:
for resource, authorization_required in auth_required_per_resource:
if not authorization_required and self.output != "json":
click.secho(f"{resource} has no authentication.", fg="yellow")Every other user-facing signal in this method was translated into a type-tagged JSON record (changeset, event, outputs, result). Discarding this specific warning means a JSON consumer deploying an API with an unauthenticated resource receives no indication at all — the same deployment surfaces a yellow warning in text mode but nothing in JSON mode. Emit a structured event instead of suppressing:
for resource, authorization_required in auth_required_per_resource:
if not authorization_required:
if self.output == "json":
sys.stdout.write(
json.dumps({"type": "warning", "message": "no authentication", "resource": resource}) + "\n"
)
sys.stdout.flush()
else:
click.secho(f"{resource} has no authentication.", fg="yellow")…s JSON
- Raise UsageError when --guided and --output json are combined, since
guided mode requires interactive prompts incompatible with JSON output.
- Emit {"type": "warning", "message": "no authentication", "resource": ...}
instead of silently dropping the auth warning in JSON mode.
1 participant
Summary
Adds
--output jsontosam deploy. Emits newline-delimited JSON (NDJSON) that streams in real-time as CloudFormation events arrive.Before
After
{"type": "changeset", "action": "Add", "logical_id": "MyFunction", "resource_type": "AWS::Lambda::Function", "replacement": "N/A"} {"type": "event", "status": "CREATE_IN_PROGRESS", "resource_type": "AWS::Lambda::Function", "logical_id": "MyFunction", "reason": "", "timestamp": "2026-07-24T22:58:40+00:00"} {"type": "event", "status": "CREATE_COMPLETE", "resource_type": "AWS::Lambda::Function", "logical_id": "MyFunction", "reason": "", "timestamp": "2026-07-24T22:58:47+00:00"} {"type": "outputs", "stack_outputs": [{"key": "MyApiEndpoint", "value": "https://abc123.execute-api.us-west-2.amazonaws.com/Prod", "description": "API endpoint"}]} {"type": "result", "status": "SUCCESS", "stack_name": "my-app", "region": "us-west-2"}Failure output:
{"type": "result", "status": "FAILED", "error": "Failed to create/update the stack: my-app, ..."}No-execute-changeset output (returns immediately):
{"type": "changeset", "action": "Add", "logical_id": "MyFunction", ...} {"type": "result", "status": "CHANGESET_CREATED", "changeset_id": "arn:aws:cloudformation:..."}Why NDJSON instead of a single JSON blob
Deploys take 1-15 minutes. A single blob at the end causes process timeouts in agent frameworks and CI/CD pipelines. NDJSON streams output continuously, process managers see activity and don't kill it. Each line is independently
json.loads()File Changes
samcli/commands/_utils/table_print.pyMakes the
@pprint_column_namesdecorator conditional when called withoutput_mode="json", skips table header/footer printing and calls the wrapped function directly.samcli/commands/deploy/command.pyAdds
--outputClick option accepting"text"(default) or"json". Passes the value throughcli()todo_cli()toDeployContext. Suppressesprint_managed_s3_bucket_info()in JSON mode.samcli/commands/deploy/deploy_context.pyoutputformat, passes it toDeployerprint_deploy_args(), auth warnings, changeset echo, confirm prompt, success/express messagessamcli/lib/deploy/deployer.pyoutput_modeon the Deployer instancedescribe_changeset: emits{"type": "changeset", ...}per change instead of table rowsdescribe_stack_events: emits{"type": "event", ...}per event as they stream from CloudFormationwait_for_execute: emits{"type": "outputs", ...}with stack outputs instead of the outputs tablesamcli/commands/deploy/core/options.pyRegisters
"output"inADDITIONAL_OPTIONSso it appears in--help.schema/samcli.jsonRegenerated to include the new
outputparameter withenum: ["json", "text"].Test updates
tests/unit/commands/deploy/test_command.py: addsoutput="text"to alldo_cliandDeployContextassertionstests/unit/commands/samconfig/test_samconfig.py: adds"text"to deploydo_cliassertionsBenchmark Notes
Adding
--output jsontosam deploydoes not dramatically change agent performance for debugging tasks. Frontier models parse both text and JSON output effectively. The primary value is machine scripts and CI/CD pipelines that needjson.loads()cannot parse text output at all (jqexits with parse error on text mode).