Skip to content

perf(parser): lazy-load envelopes to reduce Lambda cold start latency by ~900ms - #8405

Open
ErezMizrahi wants to merge 1 commit into
aws-powertools:developfrom
ErezMizrahi:perf/parser-lazy-envelope-imports
Open

perf(parser): lazy-load envelopes to reduce Lambda cold start latency by ~900ms#8405
ErezMizrahi wants to merge 1 commit into
aws-powertools:developfrom
ErezMizrahi:perf/parser-lazy-envelope-imports

Conversation

@ErezMizrahi

@ErezMizrahi ErezMizrahi commented Aug 27, 2026

Copy link
Copy Markdown

Issue number: closes #8406

Summary

Changes

  • Replace eager imports of envelopes and BaseEnvelope in parser/__init__.py with a module-level __getattr__ that defers loading until those names are actually accessed
  • Add a functional test asserting that importing parse does not load any envelope modules

User experience

Before: Any Lambda function importing parse or event_parser from aws_lambda_powertools.utilities.parser caused all 16 envelope modules to load eagerly at cold start — even when no envelope was ever used.

After: Envelope modules are only imported on first access (e.g. when passing envelope=SqsEnvelope to parse()). Functions that use parse() without an envelope pay zero envelope-loading cost.

Measured impact: ~900ms reduction in Lambda INIT_DURATION for API Gateway handlers using parse() without an envelope. Profiled using PYTHONPROFILEIMPORTTIME=1 on AWS Lambda (arm64, Python 3.13, 1024MB):

# Before
import time:  30627 |   1413849 | aws_lambda_powertools.utilities.parser
import time:   6263 |    295176 |   aws_lambda_powertools.utilities.parser.envelopes
# ... 14 more envelope modules loaded unnecessarily

# After
import time:    312 |     41203 | aws_lambda_powertools.utilities.parser
# No envelope modules loaded

Public API is fully preservedfrom aws_lambda_powertools.utilities.parser import envelopes and from aws_lambda_powertools.utilities.parser import BaseEnvelope continue to work exactly as before.


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

…rt latency

Importing `parse` or `event_parser` from `aws_lambda_powertools.utilities.parser`
previously caused all 16 envelope modules to load eagerly via the package
`__init__.py`, even when no envelope was used.

This added ~900ms to Lambda cold start for any function using the parser
without an envelope (the common case for API Gateway handlers).

The fix uses Python's module-level `__getattr__` to defer loading of
`envelopes` and `BaseEnvelope` until they are actually accessed. Public
API is unchanged — existing code importing from this module continues to
work without modification.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@ErezMizrahi
ErezMizrahi requested a review from a team as a code owner August 27, 2026 15:59
@boring-cyborg

boring-cyborg Bot commented Aug 27, 2026

Copy link
Copy Markdown

Thanks a lot for your first contribution! Please check out our contributing guidelines and don't hesitate to ask whatever you need.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

@boring-cyborg boring-cyborg Bot added the tests label Aug 27, 2026
@powertools-for-aws-oss-automation powertools-for-aws-oss-automation Bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Aug 27, 2026
@sonarqubecloud

Copy link
Copy Markdown

@leandrodamascena leandrodamascena left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this. The lazy-loading direction is valuable, but the current implementation breaks an existing public import path. I am requesting changes so we preserve compatibility and make the import test reliably isolated.


def __getattr__(name: str) -> object:
if name == "envelopes":
from aws_lambda_powertools.utilities.parser import envelopes as _envelopes # noqa: PLC0415

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Erez, thank you for profiling this and documenting the cold-start impact. I really like the direction of this change.

I pulled the branch locally and tried the existing public import in a fresh Python process:

from aws_lambda_powertools.utilities.parser import envelopes

It currently ends in a RecursionError. __getattr__("envelopes") imports the same attribute from the same package, which invokes __getattr__ again. from ...parser import * fails for the same reason.

Could we load the submodule with importlib.import_module(f"{__name__}.envelopes") and cache the resolved value in globals()? I tested that approach locally and it preserves both the lazy import and the existing public API.

import sys

# Remove any previously cached parser modules to simulate a fresh import
parser_modules = [key for key in sys.modules if "aws_lambda_powertools.utilities.parser.envelopes" in key]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One concern with this test: by the time it runs, the conftest and this module have already imported envelope classes. Removing entries from sys.modules can leave references behind, so the result may depend on test order.

Could we run this check in a fresh subprocess instead? I would also cover the existing public imports for envelopes and BaseEnvelope, since those are the compatibility paths affected by this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Denotes a PR that changes 30-99 lines, ignoring generated files. tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(parser): eagerly loading all 16 envelopes on import adds ~900ms Lambda cold start latency

2 participants