refactor(qwp): rename sf_max_bytes to sf_max_segment_bytes - #70
Conversation
The connect-string key caps a single SF segment, and the new name says so; it also pairs cleanly with sf_max_total_bytes. The builder method follows: storeAndForwardMaxBytes() is now storeAndForwardMaxSegmentBytes(). Internal identifiers and error messages rename in step so the old spelling is gone entirely. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Review: PR #70Verdict: Approve. This is a clean, complete, behavior-preserving rename. No Critical or Moderate findings. What I verified
Minor
Summary
|
Merge main (PR #67, keep SF slot locked until manager worker quiesces, plus its periodic-durability work) into the branch that renames sf_max_bytes to sf_max_segment_bytes. The merge base is one commit behind on each side, so every conflict is this branch's rename landing on top of #67's edits; each was resolved by keeping main's semantic change and re-applying the rename. Conflict resolutions: - Sender.java (3 spots): kept main's reworded comment and its new actualSfSyncIntervalNanos argument threaded into the CursorSendEngine constructor and startOrphanDrainers, with the identifiers renamed to actualSfMaxSegmentBytes / sfMaxSegmentBytes. - SegmentRing.java: took main's version wholesale. The only line this branch changed here was a comment that #67's recover() rewrite deleted, so main's file already reflects the intended state. - SfFromConfigTest.java: kept main's four new testSfSyncInterval* methods and renamed the trailing method to testSfMaxSegmentBytesAcceptsSizeSuffixes. - SegmentRingTest.java: kept main's Javadoc wording (it matches the deterministic comparison-count assertion) and applied the rename. Verified no conflict markers or stale sf_max_bytes spellings remain, and that the merged call sites resolve against the five-argument CursorSendEngine constructor and startOrphanDrainers overload. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The rename of sf_max_bytes to sf_max_segment_bytes ships no alias, so the old spelling must reject as an unknown configuration key. Nothing asserted that. Every existing drift guard -- WsSenderConfigHonoredTest and QwpConfigKeysTest -- iterates ConfigSchema.all(), so by construction they can only catch a half-applied rename, never a whole-registry one: re-introducing the old key would make them pass just as happily. testWsConfigString_withSfMaxBytes_fails asserts the ws:: parse fails with "unknown configuration key: sf_max_bytes", using the existing assertBadConfig helper and joining the unknown-key rejection cluster next to the gorilla / init_buf_size / max_schemas_per_connection tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
storeAndForwardMaxSegmentBytes still took a parameter called maxBytes, which reads as the total cap and reintroduces exactly the ambiguity the sf_max_segment_bytes rename removes. Every sibling names its parameter after the method tail: storeAndForwardMaxTotalBytes takes maxTotalBytes, storeAndForwardDir takes dir, storeAndForwardDurability takes durability, storeAndForwardSyncIntervalMillis takes millis. The parameter is now maxSegmentBytes. Signature, the positivity guard, the exception text argument, and the field assignment all follow. No behavior changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Reviewing PR #70 at level 1 The rename itself is provably semantics-free: inverse-renaming the PR-branch CriticalNone. Moderate1. No test pins that the old key is now rejected — in-diff omission. The helper is already there — assertBadConfig("ws::addr=localhost:9000;sf_max_bytes=4096;", "unknown configuration key: sf_max_bytes");2. The exposure is wider than the PR body states. There is a real charter conflict here, so decide it explicitly rather than by omission: the table's javadoc scopes it to "keys that live in the legacy http/tcp/udp vocabulary", and 3. Parameter name left stale — in-diff, public LineSenderBuilder storeAndForwardMaxSegmentBytes(long maxBytes) {Every sibling names its parameter after the method tail: 4. Javadoc is missing "WebSocket transport only." — in-diff, /**
* Maximum bytes per segment file before rotation. Defaults to
* {@code DEFAULT_SEGMENT_BYTES}
* (4 MiB). Smaller segments mean faster trim of acked data; larger
* segments mean fewer rotations.
*/
5. New tests are misordered among themselves — in-diff, 6. Existing rejection helper not reused — in-diff, Minor7. 8. Naming style. 89 of 115 tests in this file (77%) use 9. The "which" → "that" commit is a regression, not a fix — 10. Uncovered edge in the new rejection test. 11. Connect-string non-WS message untested — 12. PR body. The Downgraded (false positives)
SummaryApprove with changes — nothing blocking. The rename is complete, mechanically inert, and CI is green on all 14 checks. One missing thing is a test pinning the removed key as rejected (finding 1): this PR's entire user-visible effect is untested, and every existing guard is schema-derived and structurally blind to it. Findings 3 and 4 are cheap and belong in a rename PR specifically — the parameter still named |
The storeAndForwardMaxSegmentBytes javadoc omitted the "WebSocket transport only" sentence that storeAndForwardDir, storeAndForwardDurability and storeAndForwardMaxTotalBytes all carry, even though the setter throws for HTTP, TCP and UDP. The javadoc now states that constraint, names sf_max_segment_bytes so a reader can connect the builder method to the connect-string key, and documents the parameter. It gives the default as a plain 4 MiB rather than as DEFAULT_SEGMENT_BYTES, a private constant that resolves to nothing in published javadoc. The paragraph is re-flowed to the surrounding width. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The two storeAndForwardMaxSegmentBytes rejection tests spelled out try/fail/catch inline, 19 lines for what the class already expresses in one call. assertThrows(String, Runnable) is the dominant idiom here, used by roughly twenty other tests in the file, and it produces a better failure message than a bare Assert.fail. The assertion moves from message equality to containment. The expected text is the whole message in both cases, so the check is as strong as it was; a mutation run with both production guards removed fails both tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
[PR Coverage check]😍 pass : 12 / 15 (80.00%) file detail
|
Upstream 5bbdfb8 (#67) rewrote the same store-and-forward recovery core this branch is built on, so this is a semantic reconciliation rather than a textual merge. 17 files conflicted; the dispositions are: Exception taxonomy. Adopt upstream's corruption/operational split whole. Positively-identified corruption (MmapSegmentCorruptionException) is quarantined; a plain MmapSegmentException is operational and aborts startup without quarantining, so a transient EMFILE/ENOMEM can never silently drop durable frames. Our rename-on-any-MmapSegmentException is gone. UnreplayableSlotException survives -- it carries a verdict none of upstream's types do, that the symbol dictionary cannot be rebuilt from any source -- and every catch site that quarantines now orders the arms SfSanitizedResidueException (retry), SfOperationalException (never quarantine), UnreplayableSlotException, SfRecoveryException / MmapSegmentCorruptionException, MmapSegmentException. Sender.build() gained the same lattice: upstream's terminal recovery types would otherwise escape it and re-brick the producer on every restart, which is the failure the constructor-time quarantine arm exists to prevent. Lineage id. Our per-producer generation stays in the segment header (HEADER_SIZE 32, VERSION 2) but is renamed lineageId throughout, because SfManifest.generation is a monotonic boundary-record counter and the two meanings cannot share a word in one package. The id has to travel in the same bytes as the frames whose symbol ids it validates, which is why it is not in the manifest. SegmentRing.recover checks agreement across the admitted chain and exposes recoveredLineageId(). Ack seeding and skipped segments. Upstream's SfManifest boundary checks subsume our skip-tally refusal and are strictly stronger: a recorded headBase/activeBase proves whether a set-aside file was load-bearing, where the tally could only count. The one gap left was legacy migration, which has no boundary to check against, so recover() refuses there when the surviving chain is based above zero and a file was set aside during that same recovery. A chain based at zero needs no check, and a chain based above zero with nothing set aside is the ordinary steady state of an ack-trimmed slot. Dropped as premise-removed: MmapSegment's inFlight holder and SegmentRing's per-file mmap-fault skip. Recovery now scans through positioned reads into a bounded buffer and maps only after validating, so no recovery byte is dereferenced through mmap and the async unsafe-access fault they guarded cannot arise. isMmapAccessFault stays for PersistedSymbolDict's append and heal paths, which are still folded over Crc32c.updateUnsafe. Also taken: sf_max_bytes -> sf_max_segment_bytes (#70), and upstream's sealedHead deque, successor-linked nextSealedAfter and binary-search findSegmentContaining, which supersede the equivalents this branch had grown independently. Two tests are disabled with the reason in their javadoc. Both tear a slot by deleting sf-initial.sfa to model an ack-driven trim; a real trim now durably advances the manifest head before unlinking, so a raw delete reads as a head segment that vanished outside the protocol and recovery fails closed before the dictionary-gap verdict they pin is reached. The guards themselves are unchanged. The local darwin-aarch64 native library was rebuilt: upstream added openRWExclusive0, fsyncDir0 and SlotLock.release0. Suite: 2943 pass, 0 fail, 4 skipped.
Tandems:
questdb/questdb#7413
https://github.com/questdb/questdb-enterprise/pull/1140
Renames the store-and-forward connect-string key
sf_max_bytestosf_max_segment_bytes: the key caps a single SF segment, and the new name says so; it also pairs cleanly withsf_max_total_bytes. The builder method follows:storeAndForwardMaxBytes()is nowstoreAndForwardMaxSegmentBytes(). Internal identifiers, error messages, and test assertions rename in step, sogit grep -iE 'sf[_-]?max[_-]?bytes|storeandforwardmaxbytes'returns zero.The key shipped only in beta-state releases (1.2.1–1.3.5, not on Maven Central), so this is a hard rename — no alias, no deprecation path.
Part of the cross-repo rename of this key (server, c/rust, go, .NET, python, enterprise, docs land in the same window). This repo sets the naming precedent; the QuestDB server PR bumps its submodule to this branch's commit and should merge right after this one.
Full core test suite is green locally, including the five affected test classes.
Note
The red "Build, test & javadoc (JDK 8)" check is the known
MmapSegmentRecoveryFaultTestInternalErrorflake tracked by #69 — pre-existing #64 code, where C2-inlined async unsafe-fault delivery escapes the converting catch. It reproduced twice on this PR with a different failing test each time, and this diff only touches that code path in one comment line, so it is unrelated to the rename. The fix is #71, which reads the recovery scan viapreadinstead of the mapping so there is no fault to catch at all — merge #71 first, then rebase/merge this PR to clear the check for good.🤖 Generated with Claude Code