-
Notifications
You must be signed in to change notification settings - Fork 0
fix(security): contain sandbox paths and output #931
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
seonghobae
wants to merge
15
commits into
main
Choose a base branch
from
fix/sandboxed-verify-symlink-boundary-20260811
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
faca1f1
test(security): expose sandbox symlink escape
seonghobae 856d92f
fix(security): reject sandbox-escaping symlinks
seonghobae ea6ac85
test(security): cover absolute and relative symlink escapes
seonghobae c0aefd3
docs(security): record verifier symlink boundary
seonghobae ee903b8
test(security): close verifier branch coverage
seonghobae 6f59730
test(security): expose ignored-link false positive
seonghobae ea2672a
fix(security): validate exact sandbox copy
seonghobae 46b2237
docs(security): scope validation to copied tree
seonghobae b4547a5
test(security): expose unbounded sandbox output
seonghobae 52e1281
fix(security): bound sandbox evidence streams
seonghobae 7b29a20
fix(sandbox): preserve capture ownership evidence
seonghobae ce370cf
fix(sandbox): classify path and cleanup failures
seonghobae 92990d2
fix(sandbox): validate missing-log budgets
seonghobae 4cbf3f6
test(sandbox): isolate platform capability probes
seonghobae dd2132b
test(sandbox): synchronize service readiness
seonghobae File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| # Sandboxed subprocess output resource bounds | ||
|
|
||
| ## Decision | ||
|
|
||
| The central sandbox wrappers continuously drain child stdout and stderr into fixed-size final-suffix buffers before publication-stage processing. A stream that exceeds its declared byte budget terminates the isolated POSIX process group and produces stable exit code `123`. Timeout remains `124`; service-readiness failure remains `125` unless a more specific output limit occurred. | ||
|
|
||
| The default retained budgets are: | ||
|
|
||
| - 1,048,576 bytes for each short-lived command stream; and | ||
| - 4,194,304 bytes for each backend or frontend combined service stream. | ||
|
|
||
| Configurations below 4,096 bytes or above 67,108,864 bytes are rejected before repository code executes. Every normal-path output-reader join also has a finite 30-second bound. | ||
|
|
||
| ## Why complete capture was unsafe | ||
|
|
||
| Python's `subprocess.PIPE` creates operating-system pipes for child standard streams. Waiting without concurrently reading can deadlock when a pipe fills, while `communicate()` solves that deadlock by accumulating the complete streams in parent memory. Neither behavior supplies an evidence-size ceiling. Long-running services that write directly to ordinary files similarly consume disk until the process or runner fails, and reading the complete file merely moves that unbounded allocation into parent memory. | ||
|
|
||
| The control plane therefore uses `Popen` directly, starts one reader thread per pipe immediately, reads fixed 64 KiB chunks, and retains only a locked final suffix. The first byte beyond a stream budget marks the result and kills the entire child process group created with `start_new_session=True`. Reader threads continue through EOF and are joined before bounded text is decoded or published. | ||
|
|
||
| A descendant can intentionally create a new session while retaining an inherited stdout or stderr descriptor. The original process group can then terminate while the escaped descendant keeps the pipe open. For that reason, every ordinary reader finalization passes the 30-second join bound to each capture, continues to finalize sibling readers, and then re-raises the first failure. A reader still alive after that bound produces the explicit `bounded output drain did not finish` failure instead of holding the job until its outer workflow timeout. | ||
|
|
||
| ## Rejected process-wide file limit | ||
|
|
||
| POSIX file-size resource limits apply to every regular file written by the child process. A repository verification command may legitimately create coverage databases, compiled assets, archives, package artifacts, temporary databases, or generated fixtures larger than its log budget. Applying `RLIMIT_FSIZE` to the child would therefore change application and build behavior rather than only bounding evidence. The implemented boundary constrains stdout/stderr retention and leaves ordinary repository file semantics unchanged. | ||
|
|
||
| ## Short-lived command boundary | ||
|
|
||
| `bounded_subprocess.run_bounded_command()`: | ||
|
|
||
| 1. validates a structured, nonempty argument vector and positive timeout; | ||
| 2. requires POSIX process-group termination and launches with `shell=False` and `start_new_session=True`; | ||
| 3. connects stdout and stderr to independent binary pipes; | ||
| 4. drains both pipes concurrently into separate bounded final-suffix buffers; | ||
| 5. kills the process group exactly once when either stream exceeds its budget; | ||
| 6. kills the group on timeout and joins both readers through the finite normal-path bound; and | ||
| 7. returns or raises only bounded evidence. | ||
|
|
||
| A truncation marker is included inside, not in addition to, the declared retained byte budget. Reader errors and reader-join timeouts are explicit failures. | ||
|
|
||
| ## Long-running service boundary | ||
|
|
||
| Each backend and frontend uses one combined stdout/stderr pipe and the same bounded drainer. The capture retains the final suffix in memory and writes only its bounded rendered form to the private sandbox log file when the stream closes. The evidence file therefore cannot exceed the declared service-log budget. | ||
|
|
||
| Service overflow is checked during readiness, after E2E execution, and after service shutdown. It takes precedence over an ordinary command or readiness result, but a true E2E timeout remains `124`. `tail_text()` reads no more than 65,536 bytes from the end of the already bounded file, retains the configured final line count, and publishes only that bounded suffix. Credential redaction remains a separate active integration line and is not claimed by this slice. | ||
|
|
||
| If orderly service or capture finalization raises, the wrapper makes a second | ||
| best-effort process-group kill and bounded reap before publishing resource | ||
| failure evidence. A capture error therefore cannot silently leave the backend | ||
| or frontend process group running after the ordinary shutdown path aborts. | ||
|
|
||
| A realistic regression gives the flooding backend an actual readiness URL and configures the E2E command to create a sentinel file. The overflow result must be emitted while the sentinel remains absent, proving that readiness handling cannot silently execute an ordinary E2E command before acknowledging the service evidence limit. | ||
|
|
||
| ## Security and availability properties | ||
|
|
||
| - Parent retained memory is bounded independently for stdout and stderr. | ||
| - Service evidence disk use is bounded per service. | ||
| - Child pipes are continuously drained, preventing a full pipe from blocking the child indefinitely. | ||
| - Process-group termination covers ordinary descendants that retain inherited pipe descriptors. | ||
| - A descendant that escapes the original group cannot create an unbounded reader join. | ||
| - Structured argv and `shell=False` remain unchanged. | ||
| - Environment scrubbing, timeout enforcement, process cleanup, SSRF-safe readiness polling, and machine-readable evidence remain independent controls. The bounded capture is intentionally composable with the separately reviewed credential-redaction line. | ||
| - Non-POSIX environments fail closed rather than using unmanaged capture. | ||
| - Output overflow cannot be converted into success by the child process or into an E2E execution by readiness short-circuiting. | ||
|
|
||
| MITRE CWE-770 identifies unbounded memory and other resource consumption as an availability weakness and recommends explicit minimum/maximum expectations, throttling, quotas, and safe failure when limits are reached. This implementation sets explicit per-stream ceilings, a finite finalization bound, and a stable failure result. NIST SP 800-218 supplies the secure-development framework used to define, test, and retain this control as reviewable evidence. | ||
|
|
||
| No formal CWE, NIST, or POSIX conformity is claimed. | ||
|
|
||
| ## Verification contract | ||
|
|
||
| Real subprocess tests exercise: | ||
|
|
||
| - ordinary Korean Unicode stdout and stderr; | ||
| - infinite stdout and stderr floods; | ||
| - timeout with partial output; | ||
| - final-suffix retention and one overflow callback; | ||
| - bounded persisted service evidence; | ||
| - service overflow before or during readiness/E2E, including a sentinel proof that E2E never ran; | ||
| - ordinary backend/frontend/E2E success and cleanup; | ||
| - partial UTF-8 suffix decoding; | ||
| - bounded file reads; | ||
| - unsupported-platform failure; | ||
| - invalid budgets; | ||
| - reader exceptions, stuck-reader joins, a common finite join bound, and sibling finalization after the first failure; | ||
| - retained redaction of credentials in output, commands, notes, structured JSON, and service tails; and | ||
| - deterministic result fields and exit-code precedence. | ||
|
|
||
| The exact pull-request head must additionally pass the complete central test suite, 100% production statement and branch coverage for the changed surface, production docstrings, Secret Scan, CodeQL, Semgrep, Python Security, dependency and supply-chain checks, OpenCode, Noema, CodeRabbit, independent current-head approval, and branch protection. | ||
|
|
||
| ## Limitations | ||
|
|
||
| This slice does not limit: | ||
|
|
||
| - repository workspace-copy size; | ||
| - application/build artifacts written outside standard streams; | ||
| - CPU time beyond the existing command timeouts; | ||
| - address space, process count, network traffic, or external service response size; or | ||
| - output generated by an unrelated process that does not inherit the managed service pipes. | ||
|
|
||
| The reader buffers intentionally retain the final suffix rather than the complete beginning of an oversized stream because terminal diagnostics normally contain the most actionable failure evidence. Complete oversized logs are not retained as artifacts. | ||
|
|
||
| The finite reader join converts an escaped inherited descriptor into a deterministic failure, but it does not discover or terminate arbitrary processes outside the original process group. Isolation beyond that boundary remains the responsibility of the surrounding container or runner. | ||
|
|
||
| ## Rollback | ||
|
|
||
| Rollback must restore a different proven memory-and-disk bound for every short-lived and long-running publication path. Reverting only the process-group kill, service capture, suffix reader, or finite reader join would recreate an unbounded path around the remaining controls. Before rollback, operators must demonstrate realistic flood tests, bounded retained memory and files, finite finalization, timeout behavior, cleanup and exact-head independent review. | ||
|
|
||
| ## APA 7 references | ||
|
|
||
| MITRE Corporation. (2026). *CWE-770: Allocation of resources without limits or throttling* (CWE Version 4.20). https://cwe.mitre.org/data/definitions/770.html | ||
|
|
||
| Python Software Foundation. (2026). *subprocess—Subprocess management* (Python 3.14.6 documentation). https://docs.python.org/3.14/library/subprocess.html | ||
|
|
||
| Souppaya, M., Scarfone, K., & Dodson, D. (2022). *Secure software development framework (SSDF) version 1.1: Recommendations for mitigating the risk of software vulnerabilities* (NIST Special Publication 800-218). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-218 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| # Sandboxed verification symlink boundary | ||
|
|
||
| ## Incident | ||
|
|
||
| The review verifier copied an untrusted checkout with `shutil.copytree(..., | ||
| symlinks=True)`. That preserves symbolic links rather than copying their | ||
| targets. A pull request could therefore add an absolute link, or a relative | ||
| link containing enough parent traversal, that a verification command followed | ||
| outside the temporary repository. Environment scrubbing did not close that | ||
| filesystem boundary. | ||
|
|
||
| ## Decision | ||
|
|
||
| After applying the copy ignore policy, and before running the untrusted command, | ||
| the verifier walks the exact copied tree without following directory links and | ||
| validates every symbolic link. An absolute target is rejected because the copied | ||
| link would still point at a host path. A relative target is accepted only when | ||
| its fully resolved path remains beneath the copied repository. Safe internal | ||
| relative links remain links so project semantics are preserved. Links under | ||
| ignored paths such as `node_modules` never enter the copy and are not evaluated. | ||
|
|
||
| The validation happens before the untrusted command starts. Rejection is | ||
| fail-closed with stable exit code `126`, `path_boundary_rejected=true` in the | ||
| machine-readable result, and a generic diagnostic that does not disclose the | ||
| resolved host target. It produces no verification success evidence. This is | ||
| filesystem containment, not an operating-system sandbox claim; the existing | ||
| network-mode field remains evidence metadata rather than enforcement. | ||
|
|
||
| The walk is intentionally a pre-execution copy validation, not a continuous | ||
| kernel-enforced filesystem sandbox. A command may create a new symlink after | ||
| validation. The wrapper therefore does not claim to contain a hostile process | ||
| that can mutate its copied workspace during execution; that stronger boundary | ||
| belongs to the surrounding runner or container. The control closes exposure | ||
| introduced by attacker-supplied links already present in the copied checkout. | ||
|
|
||
| ## Test-first evidence | ||
|
|
||
| `tests/test_sandboxed_verify_symlink_boundary.py` first reproduced the defect: | ||
| an escaping repository link copied successfully instead of raising. The | ||
| accepted tests require rejection of both relative traversal and absolute links, | ||
| including an absolute link back into the original checkout, while retaining a | ||
| safe repository-internal relative link. | ||
|
|
||
| ## Failure, recovery, and rollback | ||
|
|
||
| Repositories that intentionally contain absolute or escaping links must replace | ||
| them with bounded relative links before review verification. A rollback is safe | ||
| only if an independently reviewed replacement proves that no path available to | ||
| the copied command can resolve outside the copy. Dereferencing untrusted links | ||
| during the copy is not an acceptable fallback because it can read the external | ||
| target while constructing the sandbox. | ||
|
|
||
| ## APA 7th reference | ||
|
|
||
| Python Software Foundation. (2026). *shutil—High-level file operations* | ||
| (Python 3.14 documentation). Retrieved August 11, 2026, from | ||
| https://docs.python.org/3.14/library/shutil.html#shutil.copytree |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.