Skip to content

feat(plugin): operation maps on invocation hooks - #623

Closed
wangyb-A wants to merge 2 commits into
feat/plugin-invocation-input-resultfrom
plugin-parity-invocation-info
Closed

feat(plugin): operation maps on invocation hooks#623
wangyb-A wants to merge 2 commits into
feat/plugin-invocation-input-resultfrom
plugin-parity-invocation-info

Conversation

@wangyb-A

@wangyb-A wangyb-A commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Closes #617.

Part of a three-PR stack — merge in this order:

  1. feat(plugin): surface execution input and result #616execution_input / execution_result (base main)
  2. feat(plugin): operation maps on invocation hooks #623 (this PR) — the operation maps (base feat/plugin-invocation-input-result)
  3. Add hook-info field-shape conformance handlers #615 — the conformance handlers that exercise both (base plugin-parity-invocation-info)

SDK-only. Touches plugin.py, execution.py, plugin_test.py and nothing else.

What

The invocation-level plugin info objects carried no view of the execution's
operation state, so Python plugins could not see what the JS SDK exposes on
every invocation hook.

  • InvocationInfo.operations — the checkpointed operation map converted to
    OperationInfo, on invocation-start and invocation-end. Mirrors JS,
    where it sits on InvocationBaseInfo.
  • InvocationStartInfo.updated_operations — the subset named by the durable
    invocation input's UpdatedOperationIds, i.e. operations completed
    externally while the execution was suspended. Start hook only, matching JS's
    InvocationInfo.updatedOperations.

Both are kw-only with empty-map defaults, so existing plugins, hook
constructors, and positional callers are unaffected.

Notable decisions

  • The end hook re-reads the operation map rather than reusing the
    start-of-invocation snapshot. This matches JS, which overrides
    operations: toOperationInfoMap(...) at every onInvocationEnd site, and it
    is strictly more informative — the suspending invocation reports 1 operation
    at start and 2 at end.
  • Snapshot eagerly, convert lazily. Plugin invocation hook infos missing operations and updated-operations maps #617 suggested lazy construction for
    operation-heavy executions. Fully lazy was wrong: a plugin that stashes the
    info and reads it later would observe a later state than its own hook. So
    the raw map is snapshotted when the hook fires (a cheap dict copy) and only
    the per-operation OperationInfo conversion is deferred and cached. A unit
    test pins this.
  • is_replayed is left False on map entries, matching JS's
    toOperationInfoMap (isReplay: false) — these describe stored state, not
    replay events.
  • Reaching any of this already requires the plugins= parameter, which emits a
    FutureWarning.

Testing

  • Unit tests 1518 passing on this branch; hatch fmt --check clean.
  • Live plugin conformance suite (us-west-2) with the stacked handler PR
    applied: 23/23, exit 010-19 flips PASSED with no change to the
    requirement
    , and 10-1..10-18 / 10-20..10-23 are unaffected.

Known pre-existing flake (not from this change)

One earlier suite run showed 10-22 (PluginOperationChangeShape) red; an
immediate re-run on byte-identical code was 23/23 clean. The only edit on that
path here is a pure refactor (an inline dict comprehension replaced by
_to_operation_info_map, the same conversion), and 60/60 local in-process
repeats of the 10-22 handler satisfy its primary matcher. JS's handler for that
requirement is structurally identical, so it is likely cross-SDK. Tracked
separately.

Comment on lines +500 to +501
self._operations_provider = operations_provider
operations = self._snapshot_operation_infos(operations_provider)

This comment was marked as outdated.

Comment on lines +314 to +316
operations_provider=lambda: execution_state.operations,
updated_operation_ids=invocation_input.updated_operation_ids,
execution_input=input_event,

This comment was marked as outdated.

@github-actions

This comment has been minimized.

Comment on lines +256 to +258
The same value the durable handler receives as its ``event`` argument,
parsed from the execution's stored input payload. ``None`` when the
execution has no input payload, or when the info was built without one.

This comment was marked as outdated.

@github-actions

This comment has been minimized.

@wangyb-A
wangyb-A force-pushed the plugin-parity-invocation-info branch from 9cdcfe9 to 3210772 Compare August 11, 2026 05:16
@wangyb-A wangyb-A changed the title feat(plugin): operation maps and payloads on invocation hooks feat(plugin): operation maps on invocation hooks Aug 11, 2026
@wangyb-A
wangyb-A changed the base branch from main to feat/plugin-invocation-input-result August 11, 2026 05:17
@wangyb-A
wangyb-A temporarily deployed to ai-pr-review-runtime August 11, 2026 05:17 — with GitHub Actions Inactive
@wangyb-A
wangyb-A had a problem deploying to ai-pr-review-runtime August 11, 2026 05:17 — with GitHub Actions Error
"""
aws_request_id = lambda_context.aws_request_id if lambda_context else None
self._operations_provider = operations_provider
operations = self._snapshot_operation_infos(operations_provider)

This comment was marked as outdated.

Comment on lines +253 to +256
A point-in-time view of the execution's operation map: as observed at the
start of the invocation on ``on_invocation_start``, and as observed at the
end of the invocation on ``on_invocation_end``. Empty on the very first
invocation-start, before any operation has been checkpointed.

This comment was marked as outdated.

@github-actions

This comment has been minimized.

@wangyb-A
wangyb-A force-pushed the plugin-parity-invocation-info branch from 3210772 to fb8cb0f Compare August 11, 2026 05:28
@wangyb-A
wangyb-A marked this pull request as draft August 11, 2026 05:33
@wangyb-A
wangyb-A temporarily deployed to ai-pr-review-runtime August 11, 2026 06:02 — with GitHub Actions Inactive
@wangyb-A
wangyb-A temporarily deployed to ai-pr-review-runtime August 11, 2026 06:02 — with GitHub Actions Inactive

def __init__(self, provider: Callable[[], dict[str, OperationInfo]] | None) -> None:
self._provider = provider
self._lock = Lock()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex AI review

[P2] Keep invocation info deepcopy-safe. dataclasses.asdict() deep-copies non-dict fields, but this embedded lock is not deepcopyable. Plugins that serialize or copy invocation info will raise, and dispatch will silently drop the hook. Implement __deepcopy__ to materialize a lock-free mapping or expose a plain dictionary, with a regression test using asdict().

Comment on lines +496 to +497
self._operations_provider = operations_provider
operations = self._snapshot_operation_infos(operations_provider)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex AI review

[P2] Skip snapshots when no plugins are active. durable_execution always supplies the provider, so default executions still snapshot here and again at invocation end despite having no plugin executor. Since ExecutionState.operations already returns a copy and _snapshot_operation_infos copies it again, this adds four O(n) dictionary copies per invocation. Short-circuit invocation hook setup when self._plugins is empty and test that the provider is not called.

Comment on lines +707 to +709
def test_first_invocation_has_empty_maps(self):
with self.executor.run():
self._start(operations={}, updated_operation_ids=[])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex AI review

[P2] Exercise the real first-invocation path. _start hard-codes is_first_invocation=False, and production supplies ExecutionState.operations after pagination, where a normal first invocation already contains the root EXECUTION operation. This synthetic empty map neither tests its name nor validates the new wiring. Add the required end-to-end suspend/replay coverage through durable_execution, asserting the root map, UpdatedOperationIds subset, and refreshed end map; also remove the public empty-map guarantee unless execution entries are intentionally filtered.

@github-actions

Copy link
Copy Markdown
Contributor

Codex AI review

Found three issues affecting serialization, no-plugin performance, and production-path coverage. Repository code was not executed per review constraints.

Reviewed commit fb8cb0f1e28ec645cd99f04d87b99019c74a651b. Workflow run

Comment on lines +255 to +256
end of the invocation on ``on_invocation_end``. Empty on the very first
invocation-start, before any operation has been checkpointed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude AI review

The map is not empty on the first invocation-start. on_invocation_start snapshots execution_state.operations after fetch_paginated_operations has loaded the initial state, and a fresh execution's initial state contains the EXECUTION operation in STARTED status (see execution_test.py:2089-2093). So on the first invocation this map typically holds one entry — the execution operation itself — which is also consistent with how the existing on_operation_change hook builds its operations map from self.operations.

Impact: a plugin author who trusts this docstring and, e.g., counts or iterates operations on the first invocation expecting emptiness will be off by one. Fix by correcting the wording to state that the first invocation-start typically contains just the execution operation (it is empty only in the rare case where the initial state page carried no operations).

Suggested change
end of the invocation on ``on_invocation_end``. Empty on the very first
invocation-start, before any operation has been checkpointed.
end of the invocation on ``on_invocation_end``. On the first
invocation-start it typically contains just the execution operation
(mirroring ``on_operation_change``); it is empty only when the initial
state page carried no operations.

@github-actions

Copy link
Copy Markdown
Contributor

Claude AI review

Solid, well-tested change. The eager-snapshot / lazy-convert design is thread-safe (the _LazyOperationInfoMap guards resolution with a Lock, and the underlying snapshot is a copy of frozen Operation objects), the provider correctly survives ExecutionState.close() so the end hook can re-read end-of-invocation state, and all new fields/params are kw-only with defaults so existing constructors, OTel plugins, and positional callers are unaffected. I found no correctness, determinism, concurrency, or API-compatibility regressions.

One low-severity finding: the InvocationInfo.operations docstring claims the map is empty on the very first invocation-start, but the snapshot is taken after fetch_paginated_operations, so it typically contains the EXECUTION operation (consistent with the existing on_operation_change.operations map). See packages/aws-durable-execution-sdk-python/src/aws_durable_execution_sdk_python/plugin.py:255.

Residual test risk: the unit tests exercise PluginExecutor with a mock operations_provider, but there is no end-to-end test asserting that a real durable_execution run threads the actual operation map (and updated_operation_ids) through on_invocation_start/on_invocation_end to a plugin — the execution.py wiring at line 313 is only covered for presence-of-call, not content. A minor, non-blocking gap given the direct unit coverage.

Reviewed commit fb8cb0f1e28ec645cd99f04d87b99019c74a651b. Workflow run

Invocation-level plugin infos carried no view of the execution's
operation state, so Python plugins could not see what the JS SDK
exposes on every invocation hook.

Add to the invocation infos:
- InvocationInfo.operations, the checkpointed operation map converted
  to OperationInfo, on both invocation-start and invocation-end
- InvocationStartInfo.updated_operations, the subset named by the
  invocation input's UpdatedOperationIds, i.e. operations completed
  externally while the execution was suspended

Both are kw-only with empty-map defaults, so existing constructor
calls and plugins are unaffected. The end hook re-reads the map so it
reports end-of-invocation state rather than the start snapshot.

The map is snapshotted when the hook fires but converted to
OperationInfo only on first access, so an operation-heavy execution
does not pay per invocation for a view no plugin reads, while a
plugin that stashes the info still sees the state as of its hook.

Refs #617
@wangyb-A
wangyb-A force-pushed the plugin-parity-invocation-info branch from fb8cb0f to bd64ec2 Compare August 11, 2026 18:41
Addresses the Codex review comment on #623.

The thread lock in the lazy operation map made the enclosing frozen
info uncopyable: dataclasses.asdict() deep-copies any field that is not
a dict, list, tuple or dataclass, and a lock cannot be copied. Both
asdict() and deepcopy() raised TypeError, and since plugin exceptions
are swallowed a plugin that serialized an info would have its hook
silently dropped.

Drop the lock: building the map is pure and idempotent, so a race can
only duplicate work, never corrupt the result, and the assignment is
atomic. Add __deepcopy__ so the view materializes into an ordinary dict
when copied, which also keeps the provider closure out of the copy.

Adds regression tests using asdict() and deepcopy() over both hook
infos and both maps, including the not-yet-read case.

Refs #617
@wangyb-A wangyb-A closed this Aug 11, 2026
An error occurred while trying to automatically change base from feat/plugin-invocation-input-result to main August 11, 2026 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Plugin invocation hook infos missing operations and updated-operations maps

1 participant