fix(cli): bound how much log a single drain reads into memory [PC-4873] - #1844
Draft
robert-ursu wants to merge 1 commit into
Draft
fix(cli): bound how much log a single drain reads into memory [PC-4873]#1844robert-ursu wants to merge 1 commit into
robert-ursu wants to merge 1 commit into
Conversation
The tailer did f.read() with no limit, so one drain pulled everything written since the last poll — and the decoded str costs more again. Normally the 250 ms poll keeps that tiny, but a job that logs heavily between polls, a starved tailer, or the final drain after a long run could all pull a large log in at once. The same read-it-all-to-forward-it shape has OOMed pods on the handler side. Each pass now reads at most LOG_READ_CHUNK_BYTES and loops until caught up, so peak memory is the window rather than the backlog. Two cases only reachable once the read is bounded: - A chunk boundary landing mid-line: trimmed back to the last newline, which is also what keeps the decode safe, since \n never appears inside a multi-byte UTF-8 sequence. - A single line longer than the window: emitted as a fragment rather than held. Buffering it without bound would defeat the point, and stalling on it would wedge the tailer permanently. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
robert-ursu
force-pushed
the
feat/python-job-cancellation
branch
from
August 4, 2026 16:00
b05d195 to
1df44eb
Compare
robert-ursu
force-pushed
the
feat/bound-log-tailer-reads
branch
from
August 4, 2026 16:00
228c7c2 to
2321386
Compare
🚨 Heads up:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
JobLogTailer._draincalledf.read()with no limit, so a single pass pulled everything written since the last poll into memory — and the decodedstrcosts more again.The 250 ms poll normally keeps that tiny. It does not when:
STOP_GRACE_SECONDS),Same read-it-all-only-to-forward-it shape that has OOMed pods on the handler side (UiPath/hdens#7711).
Each pass now reads at most
LOG_READ_CHUNK_BYTES(256 KiB) and loops until caught up, so peak memory is the window, not the backlog.Two cases only reachable once the read is bounded
\nnever appears inside a multi-byte UTF-8 sequence, so cutting there can't split a character.cut == -1 → returnbecomes an infinite stall once reads are bounded).Testing
Two new tests for exactly those cases: catch-up across multiple windows with a shrunk
LOG_READ_CHUNK_BYTES, and a 500-byte line through a 64-byte window. The existing 14 tailer tests are unchanged and still pass — ordering, no-replay, partial-line hold-back, and final-drain behaviour are all preserved.Full
tests/cligreen; ruff, format, mypy clean.Jira
PC-4873
🤖 Generated with Claude Code