Skip to content

feat: add Workflow Insight instrumentation plugin - #632

Draft
wangyb-A wants to merge 1 commit into
mainfrom
feat/workflow-insight-plugin
Draft

feat: add Workflow Insight instrumentation plugin#632
wangyb-A wants to merge 1 commit into
mainfrom
feat/workflow-insight-plugin

Conversation

@wangyb-A

Copy link
Copy Markdown
Contributor

Summary

Adds a Workflow Insight instrumentation plugin as a new package,
packages/aws-durable-execution-sdk-python-insight/ — a port of the JS SDK's
workflowInsight() plugin (aws-durable-execution-sdk-js-insight), treated as the
reference implementation throughout. Experimental, matching the JS plugin's status.

It listens to the SDK's instrumentation hooks and emits one curated WorkflowInsight
record (schemaVersion: "1.0") per execution. The wire record keeps the JS camelCase
field names so records read identically across SDKs and land in the same stores/queries.

Behavior (mirrors the JS plugin)

  • Exporters: LambdaLogExporter default (one JSON line to the function's log group,
    carrying the name-keyed operationsByName summary) and S3Exporter (the lossless
    per-occurrence operations array; upsert-by-execution-name; none/date/
    function-name partitioning). boto3 is an extra ([s3]) since Lambda provides it.
  • Emit model: on-complete / on-failure / on-change with export coalescing —
    a newer record supersedes a pending one; exports never propagate errors into the
    execution.
  • Sampling: deterministic per-execution ARN hash; all-or-nothing per execution.
  • Content config: input/output omission or transform (redaction), include_errors
    gating operation-level error detail only, per-operation result opt-in with optional
    transform.
  • Truncation: phase 1 drops opted-in results oldest-first, phase 2 drops whole
    operations oldest-first, input/output last; per-exporter max_record_size_bytes
    measured against the exact shape each exporter emits.
  • Operation detail: top-level (default; children with parentId suppressed) vs
    full-tree; unnamed operations are dropped (JS parity).

Depends on #616 (merged)

The plugin reads InvocationInfo.execution_input / InvocationEndInfo.execution_result
introduced by #616 — the dependency floor is set to >=1.8.0 accordingly (first release
that will carry those hooks). Capability note kept in the module docstring: the operations
map is reconstructed by accumulating per-operation hooks into per-execution state (keyed
by execution ARN to isolate warm-container reuse), since Python hooks carry no
end-of-invocation operations snapshot.

Conformance validation (live, us-west-2)

Validated against the cross-SDK insight conformance suite
(aws/aws-durable-execution-conformance-tests#73, 18 requirements): 18/18 on the s3
sink and 18/18 on the cloudwatch sink
. Two known cross-SDK divergences are documented
in that suite rather than patched over here: operation ids pass through the SDK's native
blake2b[:64] format (JS uses MD5[:16]; the suite asserts ids as opaque), and the
per-operation error.name surfaces the customer error class while the record-level error
carries the SDK wrapper name (the suite asserts non-empty).

The suite's Python example handlers land in the conformance repo as a follow-up to #73
once this package is available.

Testing

  • 17 unit tests (hatch run test:all packages/aws-durable-execution-sdk-python-insight/tests/)
    covering record shaping, operations indexing, truncation phases, sampling, emit modes,
    and exporter rendering
  • hatch fmt clean; package registered in the root known-first-party
  • Live conformance runs as above (JS-parity behavior confirmed record-for-record)

Port of the JS SDK's workflowInsight() plugin as a new package,
aws-durable-execution-sdk-python-insight: listens to the SDK's
instrumentation hooks and emits one curated WorkflowInsight record
(schemaVersion 1.0, JS-identical camelCase wire format) per execution
through configurable exporters (LambdaLogExporter default with the
operationsByName summary; S3Exporter with the per-occurrence operations
array). Mirrors the JS emit model: on-complete/on-failure/on-change
scheduling with coalescing, ARN-hash sampling, content configuration
(input/output omission and transforms, include_errors, per-operation
result opt-in), two-phase truncation, top-level vs full-tree operation
detail, and unnamed-operation dropping. Uses the invocation-hook
execution_input/execution_result fields introduced in #616.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant