Summary
Add the FAIL API to the pynumaflow-lite SDK so that user-defined functions can explicitly mark a message as failed. A failed message is retried according to the configured retryStrategy, matching the behavior recently introduced in Numaflow's transformer/map retry mechanism.
Background
Numaflow added a retry mechanism for transformer and map UDFs in numaproj/numaflow#3570. As part of that work, the ability to fail a message from within a UDF was exposed across SDKs:
The full pynumaflow SDK already provides this API. pynumaflow-lite does not yet expose it, so users on the lite SDK cannot fail/retry a message.
Goal
Provide the FAIL API in pynumaflow-lite for the UDF types that support it in the full SDK, so a handler can return a "failed" message and have it retried per retryStrategy.
Target UDF surfaces (mirroring pynumaflow#370):
- Map
- Batch Map
- Map Stream
- Source Transformer
Acceptance Criteria
References
Summary
Add the FAIL API to the pynumaflow-lite SDK so that user-defined functions can explicitly mark a message as failed. A failed message is retried according to the configured
retryStrategy, matching the behavior recently introduced in Numaflow's transformer/map retry mechanism.Background
Numaflow added a retry mechanism for transformer and map UDFs in numaproj/numaflow#3570. As part of that work, the ability to fail a message from within a UDF was exposed across SDKs:
The full
pynumaflowSDK already provides this API.pynumaflow-litedoes not yet expose it, so users on the lite SDK cannot fail/retry a message.Goal
Provide the FAIL API in
pynumaflow-litefor the UDF types that support it in the full SDK, so a handler can return a "failed" message and have it retried perretryStrategy.Target UDF surfaces (mirroring pynumaflow#370):
Acceptance Criteria
pynumaflow-litecan mark a message as failed for the map, batch map, map stream, and source transformer types.References