Skip to content

Add proof serve for testing clients against other servers - #2

Merged
bbatsov merged 2 commits into
mainfrom
serve
Oct 8, 2026
Merged

bbatsov merged 2 commits into
mainfrom
serve

Conversation

@bbatsov

@bbatsov bbatsov commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

The proxy can only grade what a client sends. proof serve is for what it does with the replies: a client's tests run against it, and scenarios make it reply the way jank, Basilisp, ClojureCLR and others do where they differ from nREPL.

Scenarios are limited to things real servers (or TCP) actually do, so clients aren't asked to cope with replies nobody sends. The evaluator only knows a small Clojure subset, enough for the profile snippets, CIDER's connect code and typical client tests.

A server for client test suites needs the same handling of client
connections as the proxy: accepting them (and carrying on after running
out of file descriptors), numbering them the way the log does, recording
what's said, giving clients a moment to hang up when proof stops, and
leaving out connections that only probed the port. That now lives in
internal/clients, and the proxy keeps only the relaying.

The listener also closes a client's socket once its handler returns, and
the tests for all of the above moved along with it.
The proxy only checks what a client sends, not what it does with the
replies. proof serve is a small nREPL server for a client's own tests to
run against. On its own it behaves like nREPL 1.7.0, and gives the same
replies for the same code. Each scenario makes it behave like some other
server in one particular way, e.g. last-value sends only the value of
the last form, like Basilisp, jank and dialtone do.

Every scenario is something a server in the compatibility matrix does,
or something TCP can do to the replies (writing them a byte at a time,
or several at once). The tests make sure of that. proof's own checks
have to give each scenario the verdicts its servers get, and all of
jank's (or Basilisp's) scenarios together reproduce that server's column
of the matrix.

proof can't evaluate Clojure, so proof serve understands just enough of
it for the snippets of profiles/clojure.toml, the code CIDER sends when
it connects, and what client tests need: output, values, errors, input,
interrupts and late output from futures. Interrupting an eval, closing a
session with one running and errors in the middle of the code all play
out the way they do on nREPL. When proof stops, it grades the requests
with the client rules, just like the proxy does.

The proxy and proof serve share their options, grading and command
tests, and the verdict checks of the server checks' tests moved to
internal/check/checktest, so the scenario tests can use them too.
@bbatsov
bbatsov merged commit 496fac4 into main Oct 8, 2026
4 checks passed
@bbatsov
bbatsov deleted the serve branch October 8, 2026 05:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant