perf(parser): lazy-load envelopes to reduce Lambda cold start latency by ~900ms - #8405
Conversation
…rt latency Importing `parse` or `event_parser` from `aws_lambda_powertools.utilities.parser` previously caused all 16 envelope modules to load eagerly via the package `__init__.py`, even when no envelope was used. This added ~900ms to Lambda cold start for any function using the parser without an envelope (the common case for API Gateway handlers). The fix uses Python's module-level `__getattr__` to defer loading of `envelopes` and `BaseEnvelope` until they are actually accessed. Public API is unchanged — existing code importing from this module continues to work without modification. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
|
Thanks a lot for your first contribution! Please check out our contributing guidelines and don't hesitate to ask whatever you need. |
|
leandrodamascena
left a comment
There was a problem hiding this comment.
Thanks for working on this. The lazy-loading direction is valuable, but the current implementation breaks an existing public import path. I am requesting changes so we preserve compatibility and make the import test reliably isolated.
|
|
||
| def __getattr__(name: str) -> object: | ||
| if name == "envelopes": | ||
| from aws_lambda_powertools.utilities.parser import envelopes as _envelopes # noqa: PLC0415 |
There was a problem hiding this comment.
Hi Erez, thank you for profiling this and documenting the cold-start impact. I really like the direction of this change.
I pulled the branch locally and tried the existing public import in a fresh Python process:
from aws_lambda_powertools.utilities.parser import envelopesIt currently ends in a RecursionError. __getattr__("envelopes") imports the same attribute from the same package, which invokes __getattr__ again. from ...parser import * fails for the same reason.
Could we load the submodule with importlib.import_module(f"{__name__}.envelopes") and cache the resolved value in globals()? I tested that approach locally and it preserves both the lazy import and the existing public API.
| import sys | ||
|
|
||
| # Remove any previously cached parser modules to simulate a fresh import | ||
| parser_modules = [key for key in sys.modules if "aws_lambda_powertools.utilities.parser.envelopes" in key] |
There was a problem hiding this comment.
One concern with this test: by the time it runs, the conftest and this module have already imported envelope classes. Removing entries from sys.modules can leave references behind, so the result may depend on test order.
Could we run this check in a fresh subprocess instead? I would also cover the existing public imports for envelopes and BaseEnvelope, since those are the compatibility paths affected by this change.



Issue number: closes #8406
Summary
Changes
envelopesandBaseEnvelopeinparser/__init__.pywith a module-level__getattr__that defers loading until those names are actually accessedparsedoes not load any envelope modulesUser experience
Before: Any Lambda function importing
parseorevent_parserfromaws_lambda_powertools.utilities.parsercaused all 16 envelope modules to load eagerly at cold start — even when no envelope was ever used.After: Envelope modules are only imported on first access (e.g. when passing
envelope=SqsEnvelopetoparse()). Functions that useparse()without an envelope pay zero envelope-loading cost.Measured impact: ~900ms reduction in Lambda INIT_DURATION for API Gateway handlers using
parse()without an envelope. Profiled usingPYTHONPROFILEIMPORTTIME=1on AWS Lambda (arm64, Python 3.13, 1024MB):Public API is fully preserved —
from aws_lambda_powertools.utilities.parser import envelopesandfrom aws_lambda_powertools.utilities.parser import BaseEnvelopecontinue to work exactly as before.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.