Skip to content

feat: surface mid-stream parse errors via ParseWithErrors - #9

Open
vaibhav8a wants to merge 1 commit into
gitleaks:masterfrom
vaibhav8a:feat/surface-parse-errors
Open

feat: surface mid-stream parse errors via ParseWithErrors#9
vaibhav8a wants to merge 1 commit into
gitleaks:masterfrom
vaibhav8a:feat/surface-parse-errors

Conversation

@vaibhav8a

Copy link
Copy Markdown

Refs gitleaks#1338.

The problem

Parse drops the error when a fragment fails to parse:

n, err := fn(file)
if err != nil {
    return          // error discarded
}

The deferred close(out) then closes the file channel exactly as it would at the end of a well-formed patch. A caller ranging over that channel cannot tell a patch that ended from one abandoned part-way through.

That is the worst shape a failure can take for a consumer that scans patch content: nothing surfaces, and the answer is the reassuring one. In gitleaks it means a secret scan silently covers only the part of the diff that parsed, and exits 0 — the report in gitleaks#1338, where a hunk header of @@ -231,18446744073709551615 +231,1 @@ failed parseRange.

What this adds

ParseWithErrors(r) (<-chan *File, <-chan error) — the same parse, with the error that ended it surfaced. The error channel is buffered, yields at most one value, and is closed when parsing finishes, so a caller that drains the files and then reads the error channel sees nil for a patch that ended normally.

Parse is unchanged for existing callers. It now delegates to ParseWithErrors and drains the error channel in a goroutine, so no caller has to change and the parse goroutine is never left blocked sending an error nobody reads. TestParseKeepsItsOldContract pins that for both a good and a malformed patch.

Why not detect it in the consumer instead

I tried that first, in gitleaks: track whether the reader ever reached EOF, on the reasoning that the parse loop only ends normally at EOF. It does not work — newParser wraps the reader in a bufio.Reader, so read-ahead pulls a small patch entirely into the buffer before the parse error is hit, and EOF is observed even on an aborted parse. The signal would have been real for large diffs and silently wrong for small ones, which is worse than none. The error has to come from where it is raised.

Tests

7 tests, all passing, plus the existing suite and -race:

  • a malformed hunk header now surfaces an error — the case from the issue;
  • a well-formed patch reports no error, and still yields its file. A signal that fires on everything is not a signal;
  • empty input is not an error;
  • the error channel always closes, so a caller reading it after draining files never blocks;
  • Parse keeps its old contract, and its goroutine finishes on both good and malformed input;
  • a failing reader propagates rather than being swallowed.

go test ./..., go test -race ./gitdiff/ and go vet are clean. gofmt reports only apply.go and testdata/apply/bin.go, which are untouched by this change and already unformatted on master.

Follow-up

Once this is released, gitleaks/sources/git.go can switch its two gitdiff.Parse call sites to ParseWithErrors and surface the error instead of reporting a clean scan. Happy to open that PR against gitleaks when you tag a version.

Parse discarded the error when a fragment failed to parse and let the
deferred close(out) end the stream, so an abandoned parse was
indistinguishable from a well-formed patch that simply ended. Consumers
scanning patch content therefore reported success over the part they had
managed to read (gitleaks#1338).

Add ParseWithErrors returning a buffered, always-closed error channel
carrying the error that ended the parse. Parse delegates to it and drains
the channel, so its contract is unchanged and the parse goroutine is never
left blocked on an unread send.

Refs gitleaks/gitleaks#1338
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant