Skip to content

Fix secret exposure in function and agent trace spans - #6000

Open
BillLeoutsakosvl346 wants to merge 3 commits into
stagingfrom
fix/secrets-sanitization-trace-spans
Open

Fix secret exposure in function and agent trace spans#6000
BillLeoutsakosvl346 wants to merge 3 commits into
stagingfrom
fix/secrets-sanitization-trace-spans

Conversation

@BillLeoutsakosvl346

@BillLeoutsakosvl346 BillLeoutsakosvl346 commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

What changed

  • Added an environment-aware sanitizer that restores configured {{ENV_VAR}} references in trace/display values, including nested tool calls, timing segments, errors, object keys, and URL-encoded values.
  • Preserved unresolved environment references in Function display/error source while continuing to resolve plaintext runtime code.
  • Applied sanitization at the BlockExecutor observability boundary without changing handler inputs, runtime state, handled errors, retries, or resume execution.
  • Registered the sanitizer at the workflow logging boundary so Overview final output, synthetic errors, fallbacks, final spans, and telemetry copies are also sanitized.
  • Prevented streaming completion and human-in-the-loop resume paths from reintroducing raw values into already-sanitized logs.
  • Kept raw outputs available for API responses, downstream execution, retry/resume state, and billing calculations.

Root cause

Block inputs and provider/tool outputs were logged after environment references had been resolved. Function display code also resolved environment values, and post-executor paths could overwrite sanitized logs with raw streaming, pause-state, or workflow final-output data immediately before persistence.

Impact

New trace spans and Overview workflow output show configured references such as {{AWS_SECRET_ACCESS_KEY}} instead of plaintext values. Runtime tool/function execution, API responses, downstream block behavior, billing, retries, pause/resume, and persisted runtime state remain unchanged. Existing historical logs are not rewritten.

Validation

  • Independent senior review with a second review after addressing all findings
  • 216 targeted Vitest tests across sanitizer, resolver, BlockExecutor, execution core, logging completion, trace spans, Function handler, snapshots, streaming, and HITL resume
  • App TypeScript check
  • Targeted Biome lint/format on all changed files
  • API validation contract audit
  • git diff --check

@vercel

vercel Bot commented Jul 28, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 28, 2026 6:34am

Request Review

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@greptile

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cursor review

@cursor

cursor Bot commented Jul 28, 2026

Copy link
Copy Markdown

PR Summary

High Risk
Touches security-sensitive logging across executor, streaming, HITL resume, and workflow completion; incorrect sanitization could still leak secrets or break billing/resume if runtime paths were mutated.

Overview
Prevents resolved environment secrets from leaking into traces, block logs, and workflow overview output while keeping runtime execution, API responses, billing, and pause/resume state on raw values.

A new createEnvironmentSecretSanitizer walks block config for {{ENV_VAR}} references and replaces matching plaintext (including URL-encoded variants) with placeholders in observability copies only. BlockExecutor applies it to log inputs/outputs, errors, child trace spans, and completion callbacks; LoggingSession sanitizes final output, trace spans, and completion errors; execution-core registers a workflow-scoped sanitizer from all block configs.

Function blocks keep unresolved env refs in display code (resolver no longer inlines env vars there). Streaming estimates token usage from resolved input before sanitization and stops overwriting sanitized log content with raw stream text. HITL resume rebuilds log output from sanitized sources via buildResumedOutput.

Reviewed by Cursor Bugbot for commit e73afbe. Bugbot is set up for automated code reviews on this repo. Configure here.

@greptile-apps

greptile-apps Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR stops environment-resolved secrets from being persisted in observability surfaces while leaving runtime execution values intact.

  • Adds createEnvironmentSecretSanitizer and applies it at BlockExecutor log/callback boundaries and LoggingSession completion paths.
  • Keeps Function display/error code unresolved for env refs; runtime code still resolves.
  • Stops streaming completion from overwriting sanitized log content; HITL resume rebuilds log output from the existing sanitized log copy.
  • Moves streaming token/cost fallback estimation to resolved inputs before sanitization, and skips placeholder-based estimates later.

Confidence Score: 5/5

The PR appears safe to merge; no blocking failures remain from prior Greptile threads or eligible follow-up findings.

No blocking failure remains.

Important Files Changed

Filename Overview
apps/sim/executor/utils/environment-secret-sanitizer.ts New recursive env-reference sanitizer for observability copies only, with longest-first and URL-encoded replacements.
apps/sim/executor/execution/block-executor.ts Sanitizes log inputs/outputs/errors/child spans and estimates streaming cost from resolved input before sanitization.
apps/sim/lib/logs/execution/logging-session.ts Session-level sanitizer on completion paths; cost still computed from raw spans.
apps/sim/lib/workflows/streaming/streaming.ts No longer writes raw streamed content back into already-sanitized block logs.
apps/sim/lib/workflows/executor/human-in-the-loop-manager.ts Resume updates runtime state from raw pause output and logs from the sanitized log copy via shared builder.
apps/sim/executor/variables/resolver.ts Function display code no longer inlines environment variable values.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Config["Block config + env vars"] --> Sanitizer["createEnvironmentSecretSanitizer"]
  Resolve["Resolve inputs for handlers"] --> Runtime["Runtime state / API / retries"]
  Resolve --> Logs["blockLogs + onBlockComplete"]
  Runtime --> Logs
  Sanitizer --> Logs
  Logs --> Session["LoggingSession.safeComplete*"]
  Sanitizer --> Session
  Session --> Persist["Persisted finalOutput + traceSpans"]
  Runtime -.->|not mutated| Downstream["Downstream blocks / resume state"]
Loading

Reviews (4): Last reviewed commit: "preserve streaming usage estimates" | Re-trigger Greptile

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 95d4fd1. Configure here.

@BillLeoutsakosvl346
BillLeoutsakosvl346 marked this pull request as ready for review July 28, 2026 05:44
@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@greptile

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cursor review

Comment thread apps/sim/lib/workflows/streaming/streaming.ts
@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@greptile please review the latest commit, including the workflow-overview sanitization follow-up and the streaming token-accounting correction.

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit e73afbe. Configure here.

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

Demo:

Screen.Recording.2026-07-28.at.12.00.01.AM.mov

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant