feat(qwp): add table options API to name the designated timestamp column - #76
feat(qwp): add table options API to name the designated timestamp column#76jerrinot wants to merge 3 commits into
Conversation
QWP senders let callers name the designated timestamp column that the server creates when it auto-creates a missing table. The sender API exposes the hint through a Sender.TableOptions flyweight obtained via sender.table(name).tableOptions().designatedTimestamp(col), keeping Sender itself free of flat per-option methods so future create-time hints (partitioning, storage policies) can land beside it. The wire encoding sets header flag 0x20 and appends a per-table TLV options trailer (tag 0x01 = designated timestamp name) plus a u32 length footer after the table blocks. Old servers ignore the unknown flag and the trailing bytes, so frames degrade to the conventional timestamp column name; when no hint is set the emitted bytes stay byte-identical to the legacy encoding, which a golden-bytes test pins. Both the WebSocket and UDP transports carry the trailer, and store-and-forward replays the self-describing frames safely against older servers. The WebSocket client reads the X-QWP-Table-Options capability header during the upgrade and logs one warning when a configured hint reaches a server that does not advertise support. ILP senders reject the API. The hint is sticky per table, validated to 127 UTF-8 bytes, and re-declarable while a table buffer holds no rows, so pooled sender borrowers can re-declare it instead of hitting an unclearable conflict. The pooled sender guards stale handles behind its lease generation. Tests cover trailer layouts and mixed presence, UDP framing, upgrade-header parsing, stale and retained flyweight references, options-only rows, and lease rejection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jerrinot
changed the title
TableOptionsImpl.designatedTimestamp() invalidated the cached base estimate only on the first name set. QwpTableBuffer permits renaming the designated timestamp while rowCount == 0, and the cache (keyed on column count alone) folds in the trailer size, which depends on the name length. After a flush kept the buffer's name and cache alive, a rename reused the shorter-trailer estimate, so a bounded UDP sender could emit datagrams above maxDatagramSize. The fix invalidates the cache on every accepted set/change; the committedDatagramEstimate bump stays under previousName == null because a rename requires rowCount == 0, where the committed estimate is already 0. The regression test asserts committedDatagramEstimate covers the actual datagram after a short-name flush and a long-name rename (fails at 43 vs 155 bytes without the fix). A new encoder test mirrors flushPendingRowsSplit: consecutive single-table messages on a reused encoder each carry the FLAG_TABLE_OPTIONS flag and trailer, and a table without options does not inherit them. The FLAG_TABLE_OPTIONS javadoc now records the forward-compat guarantee the ungated emission relies on: pre-table-options servers validate only magic, version and payload length, and iterate exactly the header-declared table count, so unknown flag bits and trailer bytes after the last table body are ignored (verified against the server decoder at 51b235b9ca~1). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
QwpTableBuffer.setDesignatedTimestampName guarded only the rename branch on rowCount, so a first set was accepted with rows already buffered. On the UDP sender, TableOptionsImpl.designatedTimestamp then bumped committedDatagramEstimate by the trailer size without flushing or re-checking the cap. Since only commitCurrentRow checks maxDatagramSize, a subsequent flush() shipped a datagram exceeding the configured cap: IP fragmentation, or EMSGSIZE and silent data loss near the 65507-byte limit. setDesignatedTimestampName now rejects a first set while the buffer holds rows, symmetric with the rename guard. This makes the estimate bump in TableOptionsImpl unreachable, so this commit deletes it and keeps only the base-estimate invalidation. Javadocs on QwpTableBuffer and Sender.TableOptions now state the name may be set or re-declared only while the table has no buffered rows. Regression tests cover the buffer-level throw and the sender-level scenario: the late set throws, the flushed datagram carries no table-options trailer, and all packets stay within maxDatagramSize. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
[PR Coverage check]😍 pass : 135 / 138 (97.83%) file detail
|
|
M2/M3 confirmed against the diff source ( Code Review: PR #76 —
|
2 participants
Sender users could not control the designated timestamp column name of
auto-created tables, QWP servers always created
timestamp. This PR adds atable-options API to set it:
Changes
Sender.TableOptionsinterface, obtained viatableOptions()aftertable(). Options are namespaced so future create-time hints(partitioning, storage policies) land beside this one instead of on
Senderdirectly. ILP senders reject the call — ILP cannot express it.options trailer. When no hint is set, the emitted bytes are identical to
the previous encoding, pinned by a golden-bytes test.
X-QWP-Table-Optionscapabilityheader and logs one warning when the hint will not be honored.
Limitations
across table selections is documented as unsupported.
Tandem PR: questdb/questdb#7435