Skip to content

scp: fix parsing, special files and quoting - #1373

Open
ejohnstown wants to merge 3 commits into
wolfSSL:masterfrom
ejohnstown:scp-library
Open

ejohnstown wants to merge 3 commits into
wolfSSL:masterfrom
ejohnstown:scp-library

Conversation

@ejohnstown

Copy link
Copy Markdown
Contributor

Fixes to the SCP command and record parsing, the recursive send of
special files, and the remote path in the client command. Several
change behavior to match OpenSSH scp.

  • the server stops parsing options at -t/-f and honors "--"; bare
    target names keep their first character
  • behavior: an existing directory as the target receives files and
    trees inside it under their own names, as OpenSSH does
  • behavior: a FIFO, socket or device is skipped with a warning record;
    the sink writes the remaining files, then the transfer fails with
    WS_FATAL_ERROR, on the client and the server side
  • behavior: WS_SCP_ABORT goes out as a fatal (0x02) record, so an
    OpenSSH peer stops the transfer
  • files over 4 GiB are refused; the client puts "--" before a remote
    path starting with "-", refuses control characters, and no longer
    writes through SCP_from's const dst

Issue: F-15504, F-15501, F-11655, F-13320, F-15503, F-15507, F-12624, F-13319, F-12546

Copilot AI balanced review requested due to automatic review settings November 3, 2026 16:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The automated review did not complete successfully.

No open issues 🎉

What changed in this PR

This PR improves SCP parsing and transfer behavior, aligning directory destinations, skipped special files, and fatal errors with OpenSSH behavior.

Changes:

  • Correct command and record parsing, including option boundaries and buffered warning records.
  • Add special-file warnings, fatal abort records, and oversized-file rejection.
  • Preserve const destination paths and expand unit, API, and script coverage.
File Description
wolfssh/​wolfscp.h Documents callback statuses, remote paths, and warning behavior.
wolfssh/​internal.h Adds warning state and an internal command-builder test hook.
src/​wolfscp.c Implements parsing, destination, warning, size-limit, and path-copying changes.
tests/​unit.c Tests parsing boundaries, warning records, and command construction.
tests/​api.c Tests FIFO skipping, oversized files, recursive transfers, and aborts.
scripts/​scp.test Adds transfers targeting the current directory.
examples/​scpclient/​scpclient.c Simplifies the fatal-error condition without changing its behavior.

Review effort: Balanced

@ejohnstown

Copy link
Copy Markdown
Contributor Author

@claude review

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #1373

Scan targets checked: wolfssh-src, wolfssh-bugs
Not reviewed (no scan target covers them): scripts/scp.test
Coverage: 2 of 6 in-scope changed file(s) opened by the reviewer; not opened: examples/scpclient/scpclient.c, tests/api.c, tests/unit.c, wolfssh/wolfscp.h

Fenrir result: Approved ✅

No new issues found in the changed files.

Advisory only — this automated result does not count as a GitHub approval.

Review tier: Lite

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings marked 🟡 are optional suggestions and need no follow-up push.

Additional findings (outside the current diff — GitHub can't attach inline comments there):

  • 🟡 src/wolfscp.c — A recursive copy that contains a single file of 4 GiB or more now aborts the whole tree. The peer only sees "unable to get file size", so every later file is lost too. The new size check in _GetFileSize (src/wolfscp.c:3192) returns WS_BAD_FILE_E, and ScpProcessEntry maps that to WS_SCP_ABORT at src/wolfscp.c:3704. Fix: in the recursive path, skip a file too large for the word32 header the same way FIFOs are skipped, with a clear warning such as "file too large" via wolfSSH_SetScpErrorMsg and WS_SCP_CONTINUE. Keep the abort only where nothing else is left to send.

    Why this was flagged

    The trigger is a server or client sending a directory with -r where one regular file is at least 4 GiB, through the default send callback wsScpSendCallback and ScpProcessEntry (src/wolfscp.c:3686-3716). The new check at src/wolfscp.c:3192 rejects the size, and src/wolfscp.c:3701-3704 then sets WS_SCP_ABORT with the message "unable to get file size". DoScpSource handles WS_SCP_ABORT at src/wolfscp.c:745 by sending a fatal 0x02 record and finishing, so the files after the large one are never sent. The peer is also told the size could not be read, not that the file is too large. On the base branch the file went out with a truncated size, so the stream was corrupt; this change trades that for losing the rest of the tree. The same change adds a warning record (SendScpWarning, WS_SCP_CONTINUE) that skips one entry and keeps going, and FIFOs already use it, so this abort is an avoidable cost, not something the stated purpose requires.

    Verification: Trigger: a recursive (-r) send where the tree holds a regular file of 4 GiB or more. The new check at src/wolfscp.c:3192 makes _GetFileSize fail. ScpProcessEntry at src/wolfscp.c:3701-3704 then returns WS_SCP_ABORT. DoScpSource at src/wolfscp.c:745-758 then stops the whole walk, and no later entries are sent. Special files at src/wolfscp.c:3717-3733 return WS_SCP_CONTINUE with a warning, but oversized regular files don't use that path.

Comment thread src/wolfscp.c Resolved
Comment thread src/wolfscp.c Resolved
ParseScpCommand stops looking for options once -t or -f has taken the
rest of the command as the path, and skips a "--" before it. A target
with no directory part names a file in the current directory, unless
it names an existing directory, which stays the target as in OpenSSH.

- ScpCheckForRename keeps a bare name whole and uses "." as its base,
  and leaves ".", "..", and an existing directory as they are
- GetScpFileMode checks the start index and the full mode field before
  reading
- DoScpRequest returns WS_SUCCESS when data follows a finished transfer
- tests: add ParseScpCommand cases, with directory targets and path
  checks skipped on Windows, and GetScpFileMode index cases
- scp.test: copy to and from "." in both directions

Issue: F-15504, F-15501, F-11655, F-13320
@ejohnstown

Copy link
Copy Markdown
Contributor Author

On the 4 GiB file in a recursive send: it is now skipped with "scp: : file too large", like a special file, and the rest of the tree is still sent. A single-file request for one gets the same warning. Two related fixes:

  • A send callback error other than WS_WANT_READ/WRITE now reaches the peer as an error record. A missing source file used to make wolfSSH_SCP_from return WS_SUCCESS with no file; it now returns WS_FATAL_ERROR.
  • wFopenNoFollow opens non-blocking and refuses a non-regular file. A single-file request for a FIFO with no writer is now refused at once; before, it blocked the server in open().

@philljj

philljj commented Nov 16, 2026

Copy link
Copy Markdown

@wolfSSL-Fenrir-bot review max

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #1373

Scan targets checked: wolfssh-src, wolfssh-bugs
Not reviewed (no scan target covers them): scripts/scp.test
Coverage: 7 of 8 in-scope changed file(s) opened by the reviewer; not opened: wolfssh/wolfscp.h

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

Review tier: Max

Comment thread src/wolfscp.c Resolved
Comment thread src/wolfscp.c Resolved
@philljj philljj assigned ejohnstown and unassigned wolfSSL-Bot Nov 16, 2026
Comment thread wolfssh/internal.h Resolved
The default send callback skips a FIFO, socket, device, or a file of
4 GiB or more with a warning. On WS_SCP_CONTINUE, DoScpSource sends a
warning record and asks a recursive walk for its next entry; a skipped
single file or recursive root ends the transfer. As in OpenSSH scp,
both sides fail a transfer after a warning, the source with exit 1.

- wFopenNoFollow opens non-blocking and refuses a non-regular file, so
  a single-file FIFO request cannot block
- a callback error goes out as a fatal (0x02) record and stays the
  session error; mid-file the transfer is dropped instead
- ReceiveScpMessage handles one record per call and keeps the rest of
  the read; a record cut off by EOF also fails the transfer
- the warning is cut to fit the sink's message buffer, and DoScpSource
  clears scpFileSz with the other per-file state
- tests: special files, a 4 GiB + 1 file, warning and error records,
  skips and unopenable sources on each side, mid-file skips

Issue: F-15503, F-15507
As with OpenSSH scp, the client sends the remote path to the server's
shell as given, with "--" before a path starting with "-". A control
character in the path is refused with WS_BAD_ARGUMENT, since it would
end or split the command.

- wolfSSH_SCP_from works on a copy of dst, not the caller's const string
- wolfscp.h: the remote shell expands the path; callers quote it
- scpclient: test only for WS_FATAL_ERROR in the retry loop
- tests: add MakeScpCmd cases, parsed back off Windows, and check
  SCP_from leaves dst unchanged

Issue: F-12624, F-13319, F-12546

@wolfSSL-Fenrir-bot wolfSSL-Fenrir-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fenrir Automated Review — PR #1373

Scan targets checked: wolfssh-src, wolfssh-bugs
Not reviewed (no scan target covers them): scripts/scp.test
Coverage: 1 of 4 in-scope changed file(s) opened by the reviewer; not opened: tests/api.c, wolfssh/internal.h, wolfssh/wolfscp.h

Findings: 1
1 finding(s) posted as inline comments (see file-level comments below)

This review was generated automatically by Fenrir. Reported findings require changes before merge.

Review tier: Lite

Comment thread src/wolfscp.c
if (pathSz < 0)
pathSz = 0;
WSNPRINTF(msg, sizeof(msg), "scp: %.*s: %s", pathSz, path, reason);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skipped-entry warning embeds raw filename, so a newline in the name injects SCP records · SFTP/SCP path traversal

The local path goes into the 0x01 warning record as-is. A FIFO or >4 GiB file named with \n splits the record. The sink, which now reads one record at a time, then parses the rest as C/D/E records: this creates files the source never sent and desyncs the transfer.

Suggested fix: Replace every control character (bytes < 0x20 and 0x7F) in path with a printable placeholder before formatting the warning.

Related known findings (similar but distinct; listed for context, not part of this finding)

  • F-15507 (assigned): File/function: the candidate is ScpSetSkipMsg on the source side. F-15507 is _GetFileSize, a different function. Operation: the candidate formats an unsanitized local path into a 0x01 warning record with WSNPRINTF. F-15507 casts a WFTELL result to word32 without a range check. Root cause: the candidate is missing control-character neutralization in the path, which lets a newline inject protocol records. F-15507 is 32-bit truncation of the size. They are related only because the candidate's trigger

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure this is valid or something to fix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants