Skip to content

fix(cli): bound how much log a single drain reads into memory [PC-4873] - #1844

Draft
robert-ursu wants to merge 1 commit into
feat/python-job-cancellationfrom
feat/bound-log-tailer-reads
Draft

fix(cli): bound how much log a single drain reads into memory [PC-4873]#1844
robert-ursu wants to merge 1 commit into
feat/python-job-cancellationfrom
feat/bound-log-tailer-reads

Conversation

@robert-ursu

@robert-ursu robert-ursu commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Draft — third in the stack. Base is feat/python-job-cancellation (#1842), so the diff here is only the bounding work. Paired with UiPath/hdens#7711.

Summary

JobLogTailer._drain called f.read() with no limit, so a single pass pulled everything written since the last poll into memory — and the decoded str costs more again.

The 250 ms poll normally keeps that tiny. It does not when:

  • a job logs heavily between polls,
  • the tailer is starved (a stop can hold the loop for up to STOP_GRACE_SECONDS),
  • or the final drain runs after a long job.

Same read-it-all-only-to-forward-it shape that has OOMed pods on the handler side (UiPath/hdens#7711).

Each pass now reads at most LOG_READ_CHUNK_BYTES (256 KiB) and loops until caught up, so peak memory is the window, not the backlog.

Two cases only reachable once the read is bounded

  • A chunk boundary landing mid-line — trimmed back to the last newline. That is also what keeps the decode correct: \n never appears inside a multi-byte UTF-8 sequence, so cutting there can't split a character.
  • A single line longer than the window — emitted as a fragment rather than held. Buffering it defeats the point, and stalling on it would wedge the tailer permanently (the old code's cut == -1 → return becomes an infinite stall once reads are bounded).

Testing

Two new tests for exactly those cases: catch-up across multiple windows with a shrunk LOG_READ_CHUNK_BYTES, and a 500-byte line through a 64-byte window. The existing 14 tailer tests are unchanged and still pass — ordering, no-replay, partial-line hold-back, and final-drain behaviour are all preserved.

Full tests/cli green; ruff, format, mypy clean.

Jira

PC-4873

🤖 Generated with Claude Code

@github-actions github-actions Bot added test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-integrations labels Aug 4, 2026
@robert-ursu robert-ursu changed the title fix(cli): bound how much log a single drain reads into memory fix(cli): bound how much log a single drain reads into memory [PC-4873] Aug 4, 2026
The tailer did f.read() with no limit, so one drain pulled everything written
since the last poll — and the decoded str costs more again. Normally the 250 ms
poll keeps that tiny, but a job that logs heavily between polls, a starved
tailer, or the final drain after a long run could all pull a large log in at
once. The same read-it-all-to-forward-it shape has OOMed pods on the handler
side.

Each pass now reads at most LOG_READ_CHUNK_BYTES and loops until caught up, so
peak memory is the window rather than the backlog.

Two cases only reachable once the read is bounded:
- A chunk boundary landing mid-line: trimmed back to the last newline, which is
  also what keeps the decode safe, since \n never appears inside a multi-byte
  UTF-8 sequence.
- A single line longer than the window: emitted as a fragment rather than held.
  Buffering it without bound would defeat the point, and stalling on it would
  wedge the tailer permanently.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@robert-ursu
robert-ursu force-pushed the feat/python-job-cancellation branch from b05d195 to 1df44eb Compare August 4, 2026 16:00
@robert-ursu
robert-ursu force-pushed the feat/bound-log-tailer-reads branch from 228c7c2 to 2321386 Compare August 4, 2026 16:00
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown

🚨 Heads up: uipath-langchain cross-tests are FAILING 🚨

Your changes may break the uipath-langchain-python integration.

⚠️ These checks are NOT enforced by branch protection rules. Please review the failures before merging.

🔍 Inspect the failed run →

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:uipath-integrations test:uipath-langchain Triggers tests in the uipath-langchain-python repository

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant