Repository navigation
A route that fails to rebuild no longer keeps its old client-certificate policy - #65
Conversation
A hot reload that can't build a route carried the previous route forward whole, client verifier included. A route republished to require client certificates kept admitting certless clients, and one whose CA bundle changed kept trusting the old CAs, until a later reload built (#62). build_route_table now carries a failing route forward only while the new spec asks for the client-certificate policy the last-good route enforces: passthrough, no client cert, or the same effective CA bundle and requireClientCert. Otherwise the SNI is tombstoned so it can't fall through to a wildcard. Mid-rotation cert/key failures under an unchanged policy still keep last-good. The policy lives on RouteTable, not Route, so the per-connection task doesn't grow. One spec decides each configured SNI even when it is rejected: only the last exact duplicate is built, and the first wildcard reserves its suffix before it is validated, so a duplicate can't serve a tombstoned SNI under another policy. symphony-server's resolveRoute keeps the requested mtls when a cert file can't be read, and substitutes an unbuildable empty CA bundle when the CA file itself can't be read, so an unknown policy on a terminating route is dropped, never kept. Fixes #62 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request ensures that when a route fails to rebuild during a hot-swap, its last-good version is only carried forward if the requested client-certificate policy remains unchanged. If the policy has changed, the route is dropped (tombstoned) to prevent an old verifier from outliving its replacement. This is implemented by introducing a ClientAuth enum and tracking client-certificate policies in the RouteTable. Additionally, the TypeScript server wrapper is updated to handle client CA resolution failures by returning an unresolvable mTLS configuration, and comprehensive integration and unit tests are added to verify these behaviors. I have no further feedback to provide as there are no review comments.
Sync package + platform + crate versions to 1.4.0. Since 1.3.0, in the order a consumer meets them: Less memory per connection (#61). Each connection's task future shrank from 42.8 KB to 3.3 KB: the HTTP/1 header-rewriting path is boxed, the copy future is pinned once, and the accepted TLS stream is passed by reference instead of being copied into every state. Parked connections measured 66.8 KB -> 27.2 KB each (-59%). A test fails if the future grows past 4 KiB. Memory is returned after connection peaks on Linux (#66). The addon's Rust allocations now use jemalloc on Linux (glibc and musl, x64 and arm64); macOS keeps the system allocator, and Node's own allocations are unchanged. Under glibc, symphony kept ~100% of its peak memory after the connection count fell, and the peak ratcheted up across cycles. With jemalloc, troughs hold ~28% of peak and stay flat across cycles, at +1.9% memory per parked connection. Build with --no-default-features for the system allocator. arm64 builds use 64 KiB jemalloc pages so one binary runs on 4, 16 and 64 KiB-page kernels. A route that fails to rebuild no longer keeps its old client-certificate policy (#65). A hot reload now carries a failing route forward only when the new spec asks for the same client-cert policy; otherwise the SNI is tombstoned rather than admitting clients under the old verifier. TLS session resumption works for resolved suspended routes, and expired IP-affinity entries are reclaimed (#57). forwardFingerprint accepts a list, so a route can forward both JA3 and JA4 (#49). Overlapping upgrades keep status.json pointing at the new process: an outgoing process no longer repoints or unlinks it, and a process that yields to a newer peer keeps watching. Benchmarking: `npm run benchmark:memory-oop` measures symphony's own memory out of process under parked, active, stalled-reader and churn-then-drop load (#60). SYMPHONY_TLS_BUFFER_LIMIT and SYMPHONY_MALLOC_TRIM_INTERVAL_MS are benchmark hooks, off by default and not a supported configuration surface. Known open measurements on #66: CPU for 4 KiB messages is still noisy (256 B messages show no difference), and trough retention with 64 KiB pages has been measured on x86 only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
⊙ Problem
When a hot reload couldn't build a route, symphony kept serving the whole previous route, including its client verifier. A route updated to require client certificates kept admitting clients without one until a later reload succeeded. Reproduced on
mainthroughsymphony-server(#62).🤔 Is a policy-aware carry-forward the right size for this?
It closes the hole for every consumer (napi and
symphony-server) and keeps mid-rotation cert/key failures serving. host-manager#254 doesn't depend on it. The "trust nothing" route from #62 item 3 is proposed on the issue, not built here.💡 Solution
requireClientCert.symphony-serverkeeps the requestedmtlswhen a cert file can't be read. A terminating route whose CA file can't be read is dropped.⚖️ Alternatives
Route— every connection clonesRouteinto its task; measured +32 B per connection (3384 → 3416 B).server.ts— embedded napi consumers keep the hole, and TS can't predict which rustls builds fail.🤔 Should an unreadable CA file drop the route?
It does: the requested policy is unknown, and keeping the route could keep trusting a removed CA. A CA file caught mid-write leaves the route offline until the next reconcile. Rename-based rotation (what host-manager does) never hits this window.
🔧 Changes
src/router.rs:ClientAuthis the policy a spec asks for, fromeffective_mtls, whichbuild_routenow shares.RouteTable.client_authrecords it per installed route. The failure arm ofbuild_route_tablecompares it before carrying forward, and drops the SNI through a sharedtombstonehelper.last_exactskips earlier exact duplicates, and the first wildcard now reserves its suffix before it is validated.src/tls.rs:sha256ispub(crate)so the policy hashes the CA bundle the way the TLS cache keys it.ts/server.ts:resolveRoutereads the CA first;UNRESOLVABLE_MTLSrepresents an unreadable CA.CLAUDE.md,README.md: the carry-forward and duplicate rules. The README no longer says a rejected protocol declaration keeps serving last-good; it never did.🤔 What about a listener-level CA?
Out of scope here. An unreadable or unbuildable listener
mtlsCA still leaves the old proxy, and its old policy, running. Failing closed there takes every route on the port down.✅ Verification
mainat their behavior checks and pass here.src/router.rs: five unit tests. A changed policy drops the route, with no wildcard fallthrough, and stays dropped on repeat. Every policy change drops a failing route. An unchanged policy keeps last-good. A listener-inherited CA compares by effect. A rejected duplicate, exact or wildcard, isn't replaced by its sibling.__test__/server.spec.ts, through a spawnedsymphony-server(a probe counts only an echoed payload): a certless client is refused after mTLS is required with an empty CA. A deleted CA file refuses its clients, and restoring it heals the route. A deleted key file keeps an mTLS route serving. That last test fails with only the Rust half applied.cargo test163 pass, clippy andcargo fmt --checkclean,npm test159 pass / 2 platform skips. Per-connection future unchanged at 3384 B.test:integrationfails in setup locally onmaintoo (needs a harper-pro build); CI runs it.Live probe output (before → after)
Fixes #62
🤖 Generated by Claude Opus 5.5 via Claude Code (devain); posted via @Devin-Holland.
🤖 Generated with Claude Code
Release target: main — symphony ships from main; it has no release branches
Related PRs: none found
Complexity: complicated
Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex,cursor-muse; blocked=domain(auth); declined=cursor-grok,cursor-kimi,muse; rounds=2; full=1 @ 5c33811
Review-Attention: read ~6m (raised: degraded review) @ 5c33811