Repository navigation
feat(clickhouse)!: storage clauses as nodes, and one introspect question per family - #6
Merged
Merged
Conversation
The shape check now lives where the flag has to be honoured rather than where it is declared, so a value that would silently render the wrong DDL is named instead. The attribute is read directly: whether the field exists is settled by the class, so probing for it with a default would only turn a wrong class into a quiet signed column. CI installs the core library from fix/column-constraint-gates so this backend runs against the branch it depends on.
Mechanical rename wave against the core's W1/W2/W3a/W3b/S1-c breaks. Constraint surfaces follow in the next commit -- see below. Migration surface (src + tests + docs, 93 files, 826 occurrences, no SQL moved): W2, statement-level classes -> *Statement (core 34e1fb45). The largest: CreateTableExpression->CreateTableStatement 196, DropTableExpression ->DropTableStatement 127, QueryExpression->QueryStatement 93, InsertExpression ->InsertStatement 59, AlterTableExpression->AlterTableStatement 20, CreateMaterializedViewExpression->CreateMaterializedViewStatement 18, TruncateExpression->TruncateStatement 18, SetOperationExpression ->SetOperationStatement 16, RefreshMaterializedViewExpression ->RefreshMaterializedViewStatement 15, CreateViewExpression->CreateViewStatement 13, CreateTableLikeExpression->CreateTableLikeStatement 13, CTE-family below, CreateTableCloneExpression->CreateTableCloneStatement 11, BeginTransactionExpression ->BeginTransactionStatement 11, and the rest of the 57-class list this package touches (Create/Drop/Alter Database, Index, Trigger, Type, Domain, JsonDualityView, MaterializedView, View, Explain, Delete, Update, Merge, CommentOn). Three class names were resolved against the core source rather than the wave guide's prose, because the guide names neither: CreateTableLikeExpression -> CreateTableLikeStatement (NOT CreateTableFromTemplateStatement, which is what CreateTableUsingTemplateExpression became -- the two are different nodes, both present in core), CreateTableAsExpression -> CreateTableAsStatement and CreateTableCloneExpression -> CreateTableCloneStatement. W1, CTE family (core 683a183e): CTEExpression->CTEClause 12, WithQueryExpression->WithStatement 6, and the call-site keyword main_query=->query= 3 (examples/query/cte.py; every occurrence in this package was an argument of WithQueryExpression). W3a, predicate nodes (core 890aaaa3): ILIKEExpression->ILikePredicate 1, XMLExistsExpression->XMLExistsPredicate 1. Both were module paths pinned as strings in test_expression_roundtrip_all.py. W3b, value / function / introspection families (core 5dc64fd5): SQLValueExpression->SQLValue 24, CastExpression->CastFunction 4, CaseExpression->CaseFunction 3, CollateExpression->CollateFunction 2, TrimExpression->TrimFunction 2, JSONDocumentExpression->JSONDocumentValue 3, JSONTextExpression->JSONTextValue 3, UUIDCastExpression->UUIDCastFunction 1, datetime._TemporalValueExpression->_TemporalValue 1, and the six introspection containers that take the prefix: TableInfo 5, ViewInfo 5, ForeignKeyInfo 4, DatabaseInfo 3, TriggerInfo 3, ColumnInfo 3 (all in introspector.py). IndexInfo and IndexColumnInfo keep their names -- core did not rename them. Override points renamed (functional; not renaming would drop the override): - mixins/cte.py: format_cte_expression -> format_cte_clause, with the super() call and the :meth: reference in supports_materialized_cte. This is the one that matters most on this backend: the override is what refuses ``AS NOT MATERIALIZED``, and leaving the old name would silently fall back to core's spelling and emit a statement the server rejects (Code 62, measured on 26.7.3.19). - mixins/set_operation.py: format_set_operation_expression ->format_set_operation_statement. - mixins/transaction.py: format_begin_transaction ->format_begin_transaction_statement. - test_expression_roundtrip_all.py: the two formatter names it asserts the dialect does NOT declare, format_ilike_expression->format_ilike_predicate and format_xmlexists_expression->format_xmlexists_predicate. S1-c, PASSING: no change needed. This backend constructs the XML expressions through the xmlexists()/xmlquery() factories, whose passing= signature is unchanged. Behaviour: none. Every edit is a name; no SQL spelling moved. The only behavioural consequence is the one above, and it is a rename that restores intent rather than a change of intent. Deliberately not renamed: - The backend's own statement classes (ClickHouseInsertExpression, ClickHouseCreateMaterializedViewExpression, ShowExpression, ClickHouseDropPartitionExpression, ...). Core renamed only its own nodes; the M1 naming work for backend-local classes is not part of this wave. - WildcardExpression (3 call sites, all select-list, examples/insert/*.py and examples/ddl/view.py). Core's split has no intermediate state -- ``Wildcard`` never shipped -- so the migration goes straight to AllColumnsWildcard / RowCountWildcard and belongs to the wildcard commit, not to this one. - ColumnConstraintType / TableConstraintType, which core's S3 deleted. That is the wave-2 surface. Note: the constraint surfaces still reference the enums core's S3 removed, so this commit is an expected red window -- 10 modules still fail to import, all of them on those two deleted symbols, plus 3 example modules on WildcardExpression. Wave 2 migrates the construction sites onto the branch nodes and wave 4 finishes the wildcard. Verification: - import smoke over every module in the backend package: 153 imported clean, 29 failing -- 10 on the deleted constraint enums, 3 on WildcardExpression and 16 because their module body opens a connection and no ClickHouse is listening on this host (baseline before this commit: 73 clean / 109 failing). - python -m compileall src tests docs: exit 0. - ruff check on the 86 changed .py files: byte-identical finding profile to the HEAD baseline for the same file set (grouped per file and per rule), so no new class of finding; the one delta is an E501 whose measured length moved from 124 to 123 characters in tests/providers/fixtures/basic.py. - tests/.../backend/expression/test_expression_signatures.py: exit 0. - tests/.../backend/expression/test_json_expressions.py + tests/.../backend/test_identifier_quoting.py: exit 0.
…odes
Core's S3 wave replaced the fat constraint nodes with a production plus a
branch: ``ColumnConstraint(branch=...)`` / ``TableConstraint(branch=...)``
name the branch, the branch names the kind, and the ``*ConstraintType`` enums
are gone. This commit migrates every construction site, every override and
every assertion in the package onto that shape, and absorbs the mechanical
part of the B-series draft that was sitting uncommitted in the worktree.
Construction sites migrated (src + tests, 39 files):
ColumnPrimaryKeyConstraint ~60 ColumnDefaultConstraint ~25
ColumnNotNullConstraint ~100 ColumnNullableConstraint 4
ColumnUniqueConstraint 2 ColumnCheckConstraint 1
TablePrimaryKeyConstraint 4 TableCheckConstraint 1
TableUniqueConstraint 1 TableForeignKeyConstraint 3
Call shape: ``ColumnConstraint(dialect, ColumnConstraintType.NOT_NULL)``
becomes ``ColumnConstraint(dialect, branch=ColumnNotNullConstraint(dialect))``.
The old keyword-per-field form is gone with it -- ``default_value=`` is now the
branch's positional ``value``, ``foreign_key_reference=`` the branch's
``references`` pair, ``check_condition=`` the CHECK branch's ``condition``.
Two dispatch sites that read the deleted tag now read the branch class:
* ``ClickHouseTableMixin.format_table_constraint`` narrows on
``TablePrimaryKeyConstraint`` / ``TableCheckConstraint`` /
``TableUniqueConstraint`` / ``TableForeignKeyConstraint`` /
``TableExcludeConstraint``. PRIMARY KEY and CHECK render (both measured
accepted in a column list); UNIQUE / FOREIGN KEY / EXCLUDE refuse by name
with the measurement quoted, as they did before. The production-level
DEFERRABLE / INITIALLY traits now refuse explicitly too -- ClickHouse has no
spelling for either, and the previous code dropped them silently.
* ``ClickHouseTableMixin.format_column_definition`` reads
``constraint.branch.is_auto_increment`` instead of the production's deleted
field, because ``is_auto_increment`` belongs to the PRIMARY KEY branch.
``ClickHouseDDLColumnMixin`` keeps no ``format_table_constraint``: the old
column-side override existed only to carry the narrowing, and it now lives on
the table mixin where core dispatches table constraints. That also retires the
unconditional ``format_add_table_constraint_action`` refusal, whose message
("ClickHouse does not support table constraints") had stopped being true: the
server accepts ``ADD CONSTRAINT ... CHECK`` and ``ADD CONSTRAINT ... ASSUME``.
``supports_drop_constraint_if_exists`` flips to ``True`` on the strength of a
live measurement (``DROP CONSTRAINT IF EXISTS nope`` is a silent no-op, while
the plain form is Code 36). Core renders the clause itself as of its
``4875ea3b`` fix, so the draft's override that spliced ``IF EXISTS`` in is
gone -- keeping it would emit ``DROP CONSTRAINT IF EXISTS IF EXISTS <name>``.
``format_add_table_constraint_action`` and ``format_drop_table_constraint_action``
are now inherited; ``ClickHouseTableMixin`` gained the two overrides above and
``ClickHouseConstraintMixin`` carries the switch values.
``test_create_table_like`` keeps its ``NOT NULL`` render assertion: the draft
dropped it, but ``ColumnNotNullConstraint`` renders on this dialect
(``format_column_not_null_constraint`` consults ``supports_not_null_constraint``,
which is ``True``), so the assertion still holds and dropping it would have
lost coverage silently.
Tests: the new ``test_clickhouse_constraint_capabilities.py`` pins every switch
to its measured value and checks that each one is declared by this package
rather than inherited from a core default.
Verification:
- tests/.../backend/adapters/test_adapters_table_mixins.py, exit 0.
- tests/.../backend/ddl/test_create_table_like.py, exit 0.
- tests/.../backend/ddl/test_ddl_source_mixin.py, exit 0.
- tests/.../backend/dialect/test_dialect_security.py, exit 0.
- tests/.../backend/test_clickhouse_constraint_capabilities.py, exit 0.
- testsuite feature/ddl shared contract, exit 0.
- import smoke over the backend package, 131 modules, 0 failures.
- ruff on the changed src files: clean.
…be protocol The switches already took ``(self)`` and already answered their measured values; what this commit adds is the part that makes that claim checkable, and it removes three pieces of prose that had gone stale. **Local declaration discipline** (pattern 5), now enforced rather than asserted. New in test_clickhouse_constraint_capabilities.py: - ``test_the_declaring_class_lives_in_this_package`` -- for every switch this backend relies on, the first class in the dialect's MRO that has the name in ``__dict__`` must be inside this package. The distinction matters more here than on most backends: core's ``ConstraintMixin`` defaults most constraint switches to ``True``, which is the opposite of what a live ClickHouse says about almost all of them, so an inherited answer would not merely be unpinned, it would be wrong. Not every switch is on ``ClickHouseConstraintMixin`` -- ``supports_drop_constraint_if_exists`` is declared by the DDL-column mixin, which reaches the MRO first -- so the guard is package-scoped, which is what the rule actually says. - ``test_probes_take_no_argument`` -- ``(self)`` and nothing else, checked on the bound method so a defaulted parameter cannot slip back in. - ``test_docstring_quotes_the_measurement`` -- pattern 6, made mechanical: a switch docstring must contain the measured server version, a server error code, or the server's own ``accepted`` verdict. A docstring that stops quoting its measurement fails here instead of quietly becoming a guess with a name. This is the check that caught ``supports_generated_column``, whose docstring had described the absence without quoting the measurement; it now quotes the parser's own column-modifier list from 26.7.3.19. - ``test_the_measured_engine_is_pinned_in_the_constraint_mixin`` -- the ``MEASURED_ENGINE`` constant and the docstrings have to name the same release, or the evidence stops identifying its source. **No ``_require_*`` helper, and that is a measurement rather than an omission.** Every switch on this backend is a statement-level fact that does not move with the version (asserted by ``test_version_does_not_move_the_values``), so there is no kind x version matrix for such a helper to carry. The per-kind judgement lives in the two table-level formatters, where the branch class is in scope: ``format_table_constraint`` narrows the CREATE position and ``format_add_table_constraint_action`` narrows the ADD position, which accept different kind sets on this engine (``ADD CONSTRAINT pk PRIMARY KEY (id)`` is Code 62 while ``CREATE TABLE ... PRIMARY KEY (id)`` is accepted). **Docstrings corrected against what the code now does** (pattern 10: the measurements are not re-derived, only the claims that had drifted): - ``supports_column_primary_key_constraint`` said the flat ``supports_primary_key_constraint`` "is also ``False``". It returns ``True``, and the two switches answer deliberately different questions -- the flat one gates the ``PRIMARY KEY`` column *modifier* (measured accepted), this one names the per-level *constraint* (measured absent). The note now says so instead of contradicting the code next to it. - The per-level block's "Core S3 may rename them" is replaced with what S3 actually did: it moved the constraint *nodes*, deleted the two enums, and added no protocol for these switches, so the names stand. - Two "Not implemented here; see the task's scope note" asides about the column-``DEFAULT`` trap are reworded: the gate is a render-path concern, not a switch value, and the scope note they pointed at belongs to a task rather than to this module. Verification: - tests/.../backend/test_clickhouse_constraint_capabilities.py, exit 0. - ruff on the changed files: clean.
Core split the single asterisk into two by where it sits in the grammar
(spec 4.10): ``AllColumnsWildcard`` for the select list and RETURNING,
``RowCountWildcard`` for ``COUNT``'s argument. The two answer different
questions -- ``COUNT(*)`` counts rows, ``t.*`` expands columns -- and only the
aggregate's contract admits the second.
Call sites migrated by position, not by name. All three were select-list
positions, so all three became ``AllColumnsWildcard``:
examples/insert/single.py select=[WildcardExpression(d)] -> AllColumnsWildcard(d)
examples/insert/batch.py select=[WildcardExpression(d)] -> AllColumnsWildcard(d)
examples/ddl/view.py select=[WildcardExpression(d)] -> AllColumnsWildcard(d)
No ``format_wildcard`` override existed here, so none needed renaming.
Backend-specific: ``ClickHouseRowCountDistinctWildcard``, the
``COUNT(DISTINCT *)`` spelling. Spec 4.10.2 leaves it to ClickHouse ("only
ClickHouse supports it"), the expression catalogue names the node, and the
wildcard survey records it as row B8 -- the one backend of eleven that accepts
it. It extends core's ``RowCountWildcard`` and takes no relation, exactly as
that node does: there is no qualified form of this argument. The override is
``format_row_count_wildcard``, which is already in core's ``WildcardSupport``,
so no new public surface is added.
One placement decision is load-bearing: ``ClickHouseWildcardMixin`` has to
precede ``ExpressionMixin`` in the dialect's base list. C3 gives the earlier
name priority, and listed after ``ExpressionMixin`` the override would never be
consulted -- the node would still render, as core's bare ``*``, and the SQL
would look right while the server returned a different number. The comment in
``dialect.py`` and ``test_the_override_is_reached_before_core`` both record it,
mirroring the Snowflake layout.
No new capability probe. The nodes that carry the ClickHouse spelling exist only
in this backend, so there is no dialect that has the node and lacks the
spelling; what is gated is core's wildcard contract, which is core's
``WildcardSupport`` and which this dialect already satisfies.
Tests: ``test_clickhouse_wildcards.py`` covers both positions -- the bare and
qualified all-columns forms, ``COUNT(*)`` unchanged, ``COUNT(DISTINCT *)``
reached through the aggregate's ``is_distinct`` flag, the MRO ordering, and
core's aggregate contract that admits a row-count wildcard only to ``COUNT``.
The evidence is marked documented rather than re-measured: no ClickHouse
instance is reachable from this worktree, so the assertions verify the rendering
contract, not the server's acceptance.
Verification:
- import smoke over the backend package: 133 modules, 0 failures.
- tests/.../backend/test_clickhouse_wildcards.py, exit 0.
- tests/.../backend/protocol/test_protocol_conformance.py, exit 0.
- ruff on the changed files: clean.
…nd wildcards The matrix is exhausted against every class in the two covered packages and classifies each failure, so it is also the place where a rename or a new node shows up as a stale entry. Four things had drifted: **Stale keys from the W2 rename** (guide section 3, item 4). Eight entries named classes core renamed to ``*Statement``: ``CreateSchemaExpression``, ``DropSchemaExpression``, the three sequence statements and the three property graph statements. The entries were correct in substance and wrong in name, so they read as "this class cannot be found" rather than "this class does not render". Renamed to the classes core now defines. **LIKE / ILIKE need real operands.** The introspective constructor hands every unannotated slot a bare string, and both predicates call ``.to_sql()`` on their operands, so the guess produced an ``AttributeError`` rather than a render. Registered the same way core's own matrix registers them: ``LikePredicate(d, "LIKE", Column(d, "a"), Literal(d, "%x%"))`` and ``ILikePredicate(d, Column(d, "a"), "%x%")``. ILIKE then reaches its formatter and is classified as unmodelled, which is the intended verdict -- ClickHouse mixes in no ILIKE mixin. **ADD CONSTRAINT needs a constraint node.** ``AddTableConstraint`` carries a ``TableConstraint``, and the guess supplied a string, which surfaced as ``constraint.branch`` on a ``str``. Registered with a table-level PRIMARY KEY branch, the cheapest one this dialect renders in that position. **Three new bases and the S1-c PASSING node.** ``ColumnClause``, ``ColumnConstraintBranch`` and ``TableConstraintBranch`` arrived with core's two-level constraint nodes; each is a base that names no formatter (its concrete members do), so all three are pinned in ``LEGITIMATE_NON_RENDERS`` with ``_NO_FORMATTER``, matching the entry the PostgreSQL matrix pins for the same three classes. ``xml.PassingClause`` dispatches on ``format_passing_clause``, which ClickHouse does not declare -- it mixes in no SQL/XML formatter at all -- so it joins the SQL/XML group in ``UNMODELLED_FORMATTERS``. Verification: - tests/.../backend/test_expression_roundtrip_all.py, exit 0. Coverage report: 451 registered, 444 constructible, 7 pinned-unconstructible, 59 pinned-non-render; rendered=264, unsupported=120, unmodelled=32, non-render=28.
The testsuite's release/v1.0.0.dev17 predates the S3 constraint nodes and the wildcard split; this branch's tests need the matching one. The core pin already names fix/column-constraint-gates, so only the testsuite line moved -- the same change the other eight backends made (postgres bb7e28c, mysql edf1279).
Wave 1 renamed the override to format_cte_clause, which is what restores the NOT MATERIALIZED refusal core's dispatch would otherwise bypass. The parameter was left annotated as the generic BaseExpression; guide section 8 asks for the node itself, and the override reads not_materialized off it, so CTEClause is what it actually takes. Annotation only -- the refusal path was already restored and is unchanged: plain -> \c\ AS (SELECT \�\ FROM \ \) materialized -> \c\ AS MATERIALIZED (SELECT \�\ FROM \ \) not_materialized -> refused by name (Code 62, measured) Verification: - tests/.../backend/expression/test_expression_signatures.py, exit 0. - ruff on the file: clean.
…m contracts Core retired the W3c class names and rebuilt the set-operation, arithmetic and predicate construction contracts. This dialect and its tests follow. Class names: CastFunction/CaseFunction/TrimFunction -> *Expression, RawSQLExpression -> RawSQL, Pivot/UnpivotExpression -> Pivot/Unpivot, and the three dialect hooks format_pivot_expression / format_unpivot_expression / format_order_by_expression -> format_pivot / format_unpivot / format_order_by_item. Set operations: SetOperationStatement is gone, replaced by SetOperation, whose constructor is (dialect, op, left, right) with op a closed word set (union / union_all / intersect / except). ClickHouse's formatter maps each word to the spelling it writes, and every spelling still carries the ALL/DISTINCT modifier the grammar requires -- ClickHouse rejects a bare UNION (measured Code 558), so `union` spells UNION DISTINCT and `union_all` spells UNION ALL. The all_/distinct flag pair, and with it the one intentional SQL collision the pair guard used to pin, is gone: the word set has no duplicate member, and an unknown word is refused at construction. The trailing ORDER BY / LIMIT / FOR UPDATE clauses no longer live on the node; ordering, row limiting and locking are statement-level clauses, so the two orphan supports_set_operation_order_by / supports_set_operation_limit_offset probes go with them, and the Q01 guard now asserts both that the node cannot carry a FOR UPDATE clause and that the statement-level one still fails fast. Arithmetic: BinaryArithmeticExpression -> ArithmeticOperation, and `op` is now the operation word rather than the character (add/sub/mul/div/mod). This is a semantic change, not a rename: every "+" becomes "add" and every "*" becomes "mul", because a symbol is refused at construction. DDL: Literal(inline_literals=True) is replaced by the DdlLiteral class, which is the class that owns the inline strategy at a DDL syntax position. Serialization: ExpressionRegistry is no longer importable from expression.serialization. Deserialization resolves a class through the serialization class index, so the matrix registers with auto_register_builtins() and checks resolvability against registered_expression_classes(). The matrix also gains explicit constructors for the two nodes whose closed word sets the introspective guesser cannot satisfy (ArithmeticOperation and SetOperation), which keeps them rendered rather than skipped.
…tattr Every formatter now reads the fields its node contract declares; the getattr/hasattr probes are gone from the dialect layer (rg '(getattr|hasattr)\(\s*(expr|node|clause|item)\b' src -> zero matches). Migration surface (63 probe reads removed, 61-site checklist plus two extra sites in mixins/ddl_column.py): - mixins/json.py (21): alias/paths/pairs/values/path/path_value_pairs/all read directly off the ClickHouse JSON expression contracts; the column check isinstance()s against BaseExpression instead of probing for to_sql. expression/json.py declares 'paths' on ClickHouseJSONExtractExpression (keyword-only, default []) so the multiple-paths form is a constructor concern, and the test builds it that way instead of assigning the attribute after construction. - mixins/materialized_view.py (32): the formatters dispatch on isinstance() between this backend's own nodes and the generic core statements, then read declared fields only -- the generic CreateMaterializedViewStatement path rejects its unsupported clauses (column_aliases/tablespace/storage_options/with_data/no_data) before raising the TO-or-ENGINE requirement; CASCADE/RESTRICT and CONCURRENTLY/WITH [NO] DATA refusals apply to the generic statements, which are the nodes that carry those fields. The _mv_target helper is gone (it probed to_table for the isinstance this code now does once). - mixins/ddl_database.py (4): ENGINE/ON CLUSTER/SYNC are ClickHouse clauses living on this backend's own nodes, so the formatters isinstance() on those before reading them; the ALTER DATABASE error feature name comes from the enum-or-string union via isinstance(Enum). - mixins/ddl_table.py (3), mixins/dml.py (2), mixins/partition.py (1): direct reads of table_options/comment, or_replace/or_ignore, and partition_id. - mixins/ddl_column.py (2, beyond the checklist): the default-change operation is read from the enum-or-string union the way the core formatter reads it; a new_value that is an expression is recognised by isinstance(BaseExpression) instead of a to_sql probe. Behavior changes: - INSERT OR REPLACE / INSERT IGNORE are now actually refused: the old code probed 'replace'/'ignore', fields no InsertStatement carries, so those requests silently rendered as a plain INSERT. - ADD COLUMN AFTER is no longer rendered: core's AddColumn carries no position field, so the probed branch was unreachable dead code; the comment now records why the clause is absent. - CREATE/DROP DATABASE: the ClickHouse-only clauses (ENGINE, ON CLUSTER, SYNC) are rendered when this backend's node carries them, and the generic statements fail on their own declared fields instead of silently omitting what they never had. Test evidence (central venv, PYTHONPATH=src;tests): - functions/test_json_functions.py + dialect/test_materialized_view.py: 76 passed, exit 0 - ddl/test_partition_maintenance.py + test_clickhouse_database_ddl.py: 15 passed, exit 0 - ddl/test_ddl_generation.py + ddl/test_ddl_coverage.py: 22 passed, exit 0 - dialect/test_dialect_capabilities.py: 24 passed, exit 0 - ruff check on all changed files: clean
… spellings
Wire the core's TypedValueMixin and a ClickHouseTypedValueMixin into
the dialect (this backend's mixin ahead of the core's, as the MRO guard
in the test file pins), so the twelve expression.values nodes render
through ClickHouse spellings.
Tier 2 (the backend's own forms, measured on the scenario instance named
by tests/config/clickhouse_scenarios.yaml -- ClickHouse 26.7.3.19
official build, HTTP 127.0.0.1:18682, database test_db):
- binary: unhex('0102'). CREATE TABLE ... s String DEFAULT unhex('0102')
accepted; a row inserted without the column read back hex(s) = '0102'.
- UUID: the bare quoted text. u UUID DEFAULT
'01234567-89ab-cdef-0123-456789abcdef' accepted; read back verbatim
through toString(u).
- INTERVAL: deliberately not implemented. The grammar's INTERVAL is
date-arithmetic syntax (INTERVAL n UNIT, unit part of the spelling)
while the node carries a bare timedelta; the measurements that close
the question: a String DEFAULT INTERVAL '1' DAY accepted but stored
'1' (the unit silently dropped), and INTERVAL '1' DAY + INTERVAL '2'
HOUR answers toTypeName Tuple(IntervalDay, IntervalHour) -- a
mixed-unit span is not one interval. IntervalDay columns exist and
take DEFAULT INTERVAL '3' DAY (read back as 3), which makes this a
real choice: the one working spelling ties the literal to the
column's interval unit, and the value node carries no unit to tie it
to. The method stays absent so the base's explicit failure is the
answer.
Tier 1 audit (quoted ISO dates, TRUE/FALSE): measured accepted verbatim
as DEFAULTs on Date/DateTime/Bool columns on the same instance, so the
base spellings are ClickHouse's own and no override is added.
Test evidence (central venv, PYTHONPATH=src;tests):
- new test_clickhouse_typed_values.py: 12 passed, exit 0 (data-position
placeholders, exact DDL texts, the INTERVAL NotImplementedError with
its message, the structural absence of the override, the MRO order,
and two live CREATE/INSERT/read-back round trips through
CreateTableStatement + ColumnDefaultConstraint)
- feature/backend -k 'typed or literal or value' (matrix file excluded;
its registry import is wave M2's to reconcile against the current core
HEAD): 105 passed, exit 0
- ruff check on all changed files: clean
…e VALUES cells
Follow the core's classed function nodes (9b559030 and after) where
they change what this backend must answer, and the INSERT VALUES branch
that core now owns (8e2e4d82).
char_length: ClickHouseFunctionMixin overrides
format_char_length_function to render lengthUTF8. ClickHouse's LENGTH
counts bytes, so the core's default spelling answers a different question
than the node asks; measured on the scenario server (ClickHouse 26.7.3.19,
127.0.0.1:18682): length('héllo') = 6, lengthUTF8('héllo') = 5,
CHAR_LENGTH('héllo') = 5 (an accepted alias on this release; lengthUTF8 is
the spelling whose availability spans the versions this backend declares).
MRO repair: ClickHouseFunctionMixin now sits ahead of ExpressionMixin in
the dialect bases. ExpressionMixin inherits the core's FunctionCallMixin,
and the mixin was listed after it -- so the char_length override was dead
code from the moment it was added (the tests caught it: LENGTH(%) rendered
where lengthUTF8(%) was expected). supports_functions is unaffected (the
core default it shadows lives in FunctionMixin, which this dialect does
not compose).
COUNT(DISTINCT *): the core now refuses count(dialect, '*', is_distinct=True)
at construction (the row-count wildcard counts rows, which have no
duplicates) and its shared aggregate helper answers every RowCountWildcard
with the bare asterisk -- which would silently drop this backend's
distinct node's spelling and return a count of the wrong thing. The
extension is a different operand, not a modifier, so ClickHouseWildcardMixin
overrides the classed format_count_function entry to consult it:
count(dialect, ClickHouseRowCountDistinctWildcard(dialect)) renders
COUNT(DISTINCT *). Re-measured on the live server: the spelling is
accepted and answers the distinct combinations of every column (4 rows
with a duplicate pair -> count() = 4, count(DISTINCT *) = 3, answered as
countDistinct(a, b)); the previously 'documented, not re-measured' notes
in the mixin and the test are upgraded with that measurement.
INSERT VALUES: the ClickHouse format_insert_statement no longer
re-implements the rows; its VALUES branch delegates to the core's
format_insert_values_source, which is the one place that decides a cell is
an expression (rendered through its own to_sql) or a bare host value (a
bind parameter). The old loop called to_sql() unconditionally, so a bare
scalar cell crashed before reaching the driver. The ClickHouse-specific
parts of the statement (the into-Table check, the OR REPLACE / OR IGNORE /
ON CONFLICT refusals, the column list, RETURNING) stay.
FunctionCall audit (rg 'FunctionCall\(' src, 14 sites): the four
standard-aggregate construction sites in examples/query (SUM x2, AVG x2)
migrate to the classed factories sum_/avg, mirroring the mysql precedent
(a6a7389/037bc6c); the ten unclassed or window-bearing sites (ROW_NUMBER
and SUM with window_spec -- the aggregate classes carry no window_spec --
JSONExtractString/JSONExtract, DATABASE/VERSION/formatDateTime/
generateUUIDv4) stay on FunctionCall. There is no format_function_call
override in this package and no dead arithmetic-formatter rename to
migrate.
Test evidence (central venv, PYTHONPATH=src;tests):
- new functions/test_string_function_entry_points.py: 5 passed, exit 0
(lengthUTF8 spelling + alias, live character-count answer, shared
UPPER/LOWER defaults)
- new dml/test_insert_value_cells.py: 5 passed, exit 0 (expression cells,
bare host cells, mixed row, multi-row, refusals still precede the source)
- test_clickhouse_wildcards.py updated to the core ruling: 10 passed, exit 0
- functions/test_dialect_function_support.py + dialect/test_dialect_capabilities.py
(MRO-move regression): 35 passed, exit 0
- py_compile on the two migrated examples: exit 0
- ruff check on all changed files: clean
Core 239678e5 ('give a literal its job and nothing else') removed
TypedValueMixin and the twelve *Value classes and made
Literal(dialect, value, inline=) the one value node; the tier-2
spellings a backend owns moved into an override of
SQLDialectBase.format_literal. This follows:
- mixins/typed_value.py (ClickHouseTypedValueMixin,
format_binary_value / format_uuid_value) is replaced by
mixins/literal.py (ClickHouseLiteralValueMixin.format_literal):
bytes -> unhex('...'), UUID -> bare quoted text, everything else
super(). timedelta is deliberately left to the base's TypeError:
the grammar's INTERVAL is date-arithmetic syntax with the unit in
the spelling, and a host timedelta carries no unit. The live
measurement records from the typed-value form carry over verbatim
-- they are verdicts about the engine (26.7.3.19), not the API
shape.
- The mixin sits first in the dialect's base list, ahead of
SQLDialectBase: it overrides format_literal, and a mixin placed
after the class it overrides never wins.
- The inline decision is the caller's, per node. The class-level DDL
lift is gone; the DEFAULT clause is the one place that normalises,
and the domain CHECK test states inline=True itself.
- INSERT VALUES cells are bare Literals (or host scalars), and
ClickHouseJSONObjectExpression takes its members as 'data' only: a
**kwargs parameter cannot be matched by name in an artifact.
Also follows core 5a613f3c ('name an action by what it is, drop
aliases'): the ALTER action renames land everywhere they surface
here -- AddColumn/ModifyColumn/ChangeColumn/AlterColumn/
AddTableConstraint/AlterConstraint/ValidateConstraint/
DropTableConstraint -> *Action, in expression/alter_column.py, the
dialect, both examples, the TYPE_CHECKING imports in mixins/column.py
and protocols/modify_column.py (aliased, the dialect's own precedent),
and the constraint-capability and DDL-coverage tests.
Migration surface: the draft's 13 modified + 3 new + 2 deleted, plus
five follow-up sites the draft missed (two TYPE_CHECKING import
lists, one prose class name, AlterColumn in ddl/test_ddl_coverage,
DdlLiteral in ddl/test_type_domain_negative).
Behaviour: rendered SQL is byte-identical to the typed-value form;
what changed shape is the binding decision (per node inline=, no DDL
auto-inline) and the JSON object class refusing **kwargs.
Tests: test_clickhouse_literal_values.py 20/20 including both live
round-trips on the 18682 scenario server; -k literal across
feature/backend (matrix file excluded -- its harness is the next
wave) 24/24; ddl/test_ddl_coverage.py + ddl/test_type_domain_negative.py
20/20; ruff clean at 120 on every touched file. Exit code 0
throughout.
…artifact
Reconciled against core 679d9817 ("make a node's kind its identity and its
payload plain JSON"): all markers gone, value kinds flat scalars, kinds
resolved through the framework's default registry.
Harness (vendored, repo-owned): testsuite e05edb0 dropped the shared
round-trip harness; the matrix now imports the repository's own
expression_roundtrip_harness.py (following the mysql/bigquery direction),
aligned to the current core -- register_all registers into default_registry()
via register_node, and the enum/dataclass vocabulary tables are not
referenced anywhere. The stale "graph.EdgeTable" special-constructor key
follows the class to EdgeTableDefinition, which is what core renamed it to.
values.* out of the table, Literal in: the twelve value classes are gone from
core, so the dynamically collected matrix drops their rows by itself; Literal
is the one class that now carries every value, and a new
TestLiteralValueKinds pins all fourteen value kinds as one flat JSON scalar
each (binary_value base64, interval_value ISO 8601, decimal_value exact
text, ...), per channel, with the restored host type asserted, plus a
closed-set pin that forces a fifteenth kind to be added here deliberately.
Fourth verdict, LEGITIMATE_NON_ROUNDTRIPS: the plain-JSON contract (组织规则
§10) puts conversion in the constructor, and every entry below is a class
whose own tree breaks it. Three shapes, each pinned per channel, each
asserted in both directions so a core fix forces its entry out:
- "raises": the datetime family (DateTimeField/IntervalUnit lack
__str__ = str.__str__, so XML writes "DateTimeField.YEAR" and the
constructor's own normaliser refuses it); CreateTableCloneStatement (the
constructor isinstance-checks mode without normalising the serialized
word); CreateViewStatement (a ViewOptions instance in a parameter slot --
the artifact cannot carry it at all); GraphTableExpression (GraphColumn),
ColumnGeneratedConstraint (GeneratedColumnType) and ForeignKeyConstraint
(ReferentialAction), all plain classes or non-str enums in parameter
slots. All core-side, read-only from here; reported upstream.
- "word": (str, Enum) members spelled properly but stored raw by their
constructors -- GraphEdge, ForUpdateClause, AlterDatabaseStatement,
CreateTriggerStatement, MergeAction, MergeStatement (loss one level down).
The restore is value-faithful and type-lossy; asserted strictly-different
and equal under member-=word normalisation.
- "enum-str": XMLParseExpression / XMLSerializeExpression -- their document
type enums also lack __str__ = str.__str__, so the XML channel writes
"EnumClass.MEMBER"; the pin names that corrupted spelling itself.
Backend nodes adapted to the same contract (the reconciliation forced them):
- admin/load_xml/maintenance enums (FlushOption, ResetOption, KillTarget,
HandlerReadMode, LoadXMLPriority, LoadXMLConflictMode, NoWriteToBinlogOption,
CheckTableOption, ChecksumTableOption, RepairTableOption) become
(str, Enum) with __str__ = str.__str__, and their constructors normalise
the serialized word back to the member.
- ClickHouseRefreshSchedule: dataclass -> expression node (a clause is an
expression). A dataclass parameter has no artifact form, so a schedule
buried in CREATE/ALTER MV parameters made those statements
unserializable. It now renders through its own
format_click_house_refresh_schedule; the create and modify formatters
reach it via the node's to_sql(). depends_on is normalised to a tuple.
- ClickHouseLoadDataExpression: the LoadDataOptions dataclass is flattened
into the statement's own parameters (the class was internal and unused
outside this package, and the statement's formatter refuses LOAD DATA
regardless).
- ClickHouseIntervalUnit spelled (str, Enum) + __str__ like its siblings
(currently unread -- flagged for the kind-hygiene audit).
Matrix bookkeeping: graph.EdgeTable pin and format_edge_table entry retired
with the rename (format_edge_table_definition observed in its place); the
rendered bind-parameters comparison is sequence-tolerant (a tuple travels as
an array, §4.8 -- Subquery.query_params is the case in the tree); two admin
special constructors registered for the selectors that now validate their
words at construction.
Coverage after reconciliation (baseline 679d9817): 515 registered, 510
constructible, 5 pinned-unconstructible; rendered=327, unsupported=121,
unmodelled=32, non-render=30.
Evidence: the full matrix green (exit 0, 5 named skips pinned in
UNCONSTRUCTIBLE), test_materialized_view.py and test_unsupported_features.py
green (exit 0), ruff clean on every changed file.
Kind list generated with the core script (a copy with this backend's package and dialect class filled into BACKENDS -- registering clickhouse there is a change to the core repository and is reported, not committed here): core kinds 357, clickhouse override 0, clickhouse custom 121. Override zero is the rule that matters most: a backend may replace the rendering of a kind core ships, never the node class behind it, because two classes under one kind leave the deserializer no way to choose. This backend adds nodes and overrides formatters; it replaces nothing. §11.1 audit, read per class and then mechanically over the 165-entry base list: no mixin sits behind the base it overrides. Every one of the 178 overriding pairs in ClickHouseDialect's MRO is ordered so the earlier name wins, and the one that reads as the classic trap -- ClickHouseLiteralValueMixin ahead of SQLDialectBase, because it overrides format_literal -- is already placed there with the reason written down. Nothing changes here; the check is recorded because it is the check that has to keep holding. The audit's findings, in the order the rules reach them: - Compound-word kind, 121 kinds. ClickHouse is one word, and the general derivation -- a run of capitals counts as one -- splits at the camel-case seam instead, so this backend shipped click_house_flush_expression and click_house_load_xmle_xpression: two spellings of one identifier, neither of them the word the class is named after. The package now declares the token where it assembles its own expressions, the way the sqlite and mysql packages do, and the kind follows the declaration. KIND_RESERVED is data, not behaviour: no class overrides kind(), so class name, kind and formatting entry keep one source. The loop walks this package's own modules rather than its __all__, so the SHOW family and the private _ClickHousePartitionIdExpression -- neither re-exported -- stop keeping the split spelling too. - Two class names that could not derive a kind at all. ClickHouseAddColumn dispatched through format_add_column_action: the identity suffix the node is an action by, missing from the class name (§3). Renamed ClickHouseAddColumnAction, and its ClickHouse-only AFTER <column> clause -- declared since the class was introduced and read by nobody, so the example that passes after="name" and claims the new column shifts position never shifted it -- now renders through its own format_clickhouse_add_column_action. Core's AddColumnAction keeps format_add_column_action, which is what it should have been rendering all along. ClickHouseLoadXMLEXpression had the same split, "XML" + "Expression" reading as "XMLE" + "Xpression" however the capitals are grouped; renamed ClickHouseLoadXMLStatement, the statement it renders, giving clickhouse_load_xml_statement and format_clickhouse_load_xml_statement. - Two enums that were (Enum) rather than (str, Enum) with __str__ = str.__str__: DualityViewDMLTag and OptimizerHintType. A member now spells as the word it carries, so an artifact names it as a plain JSON string instead of "DualityViewDMLTag.INSERT" (§11.3, and the same correction bebc428 applied to the admin family). - One dead declaration. ClickHouseIntervalUnit was introduced with ClickHouseRefreshSchedule, read by no formatter, and exported from two places; removed. The interval a ClickHouse REFRESH clause carries is the literal the caller writes, which is what the schedule node now holds. - Create/DropJsonDualityViewStatement -> ClickHouseCreate/ DropJsonDualityViewStatement, with their two entries renamed to match. The backend name belongs at the head of the name (§3), and without it these were the only two nodes in this backend whose kind carried no backend name at all. Matrix, reconciled with the audit's renames in the same batch (§11.4): - ComparisonPredicate: core normalised its operator through normalize_comparison_operator, and the introspective guess hands Any the literal "x". A registered constructor rather than a sixth UNCONSTRUCTIBLE entry -- the class renders, and a constructor keeps it in the matrix. - The DataType pin now names the dispatch's own spelling ("does not support the generic type None") instead of a message the class no longer produces. - ForUpdateClause retires from LEGITIMATE_NON_ROUNDTRIPS: core now normalises the stored word back to the member, so the pin's subject is gone and leaving it would be a lie in the table. - The value-kind set is sixteen, not fourteen: debde516 gave NaN and the infinities kinds of their own (nan_value, infinity_value) rather than letting them ride on float_value, which JSON cannot represent faithfully. Both are pinned as flat scalars -- "nan", "+inf"/"-inf" -- with the negative sign asserted separately so one wire kind carrying two spellings cannot lose the direction, and the NaN comparison is written to not trip over NaN != NaN. - test_clickhouse_type_protocol.py: CustomType's raw is keyword-only in core now (two call sites), and the sweep that loads the matrix by file spec followed the matrix to its new home under expression/ -- and needs that directory on sys.path, which pytest's own collection was supplying. Coverage after the audit (core baseline debde516): 515 registered, 510 constructible, 5 pinned-unconstructible, 61 pinned-non-render; rendered=327, unsupported=121, unmodelled=32, non-render=30. Evidence (central venv, PYTHONPATH=src;tests, exit 0): - tests/.../feature/backend/expression/test_expression_roundtrip_all.py, full matrix under xdist: green - tests/.../feature/backend/dialect/test_unsupported_features.py: 34 passed - tests/.../feature/backend/types/test_clickhouse_type_protocol.py and tests/.../feature/backend/protocol: green apart from one case blocked outside this repository -- TestParseRoundTripSweep reaches testsuite.utils.parse_contract, which imports the .expression module e05edb0 deleted. Reported, not worked around. - ruff on every changed file: the finding set is identical to HEAD (the four F841s in types.py and the F401s in the protocol modules are pre-existing).
The org rules (expression-dialect §12) say it in three sentences: expression
modules are independent of I/O, so their tests start no connection, create no
database and touch no file; every case carries its own dialect instance and
shares no mutable state; and therefore tests run in parallel. This repository's
expression area was violating all three at once, in the same file.
- feature/backend/test_clickhouse_literal_values.py grew two live round-trip
cases in wave ② -- CREATE TABLE with a tier-2 literal as the DEFAULT, INSERT,
read the stored value back, compare -- sitting beside the render-only cases
for the same two spellings. They are the same statement twice: one proves the
dialect writes unhex('...') and a bare quoted UUID, the other proves the
server accepts them. The first belongs where expressions are tested and the
second does not, so the two cases move to
feature/backend/ddl/test_literal_default_readbacks.py, which runs on the
scenario server (clickhouse_http, 26.7.3.19 on 127.0.0.1:18682) through the
clickhouse_backend fixture the rest of that directory uses. The measured
verdicts and the statements behind them are unchanged; only the room changed.
What stays in the expression file is construction -> render, with the module
docstring saying where the end-to-end half went and why.
- Parallel is now the default rather than something each run remembers to ask
for (§12.1: "并行是测试目录的默认配置，不是每次跑测试时手加的参数"). The
worker count is read by pytest-xdist before any conftest loads, so a conftest
cannot inject -n; the default is a repository-local plugin,
tests/plugins/expression_xdist.py, registered from pyproject.toml and
therefore active for every run here. It is narrow on purpose: it enables
nothing when pytest-xdist is absent, it acts only when every target names the
expression area (a whole-repository run, a mixed run, or any other directory
keeps the serial default -- the live scenario tests have their own
--scenario-parallel runner and are not dragged into a pool they did not ask
for), it never overrides an explicit -n, and it refuses --dist=loadscope and
--dist=loadfile outright because those bind whole modules together and hide
the case-to-case dependencies parallelism exists to expose. -n without
--dist gets --dist=loadgroup, the one scheduling the rules allow.
- feature/backend/expression/conftest.py records the three properties and how
the parallel default is wired one level out, so the next person to add a case
there knows which of the three it has to keep. No case carries xdist_group:
there is no I/O in the area to serialise around, and the mark is reserved for
a case that genuinely cannot share a worker, with the reason beside it.
Evidence (central venv, PYTHONPATH=src;tests):
- tests/.../feature/backend/expression (the whole area, default scheduling):
exit 0. The run reports "bringing up nodes", which is the plugin's own
default firing -- no -n on the command line.
- tests/.../feature/backend/protocol --co: no worker pool, confirming the
scope rule keeps every other directory serial.
- ruff on tests/plugins, the expression area and the ddl area: clean.
Not run here: the two migrated cases and the rest of the ddl directory need
the scenario server, which is not listening on 127.0.0.1:18682 in this
environment (connection refused, [WinError 10061]). They are reported as
unverified rather than as passing; CI runs the scenario matrix.
…unction waves The closeout run against core ``debde516``, and the record of what the tables now say. The reconciling edits themselves are small and went in with the kind audit they belong to (``b9742c4``); what is missing here is the statement a reader needs to trust the numbers -- which tree every table in the file is a statement about, and which way each of core's four moves between the two baselines pushed a row. That is the module docstring's new closing section, because a matrix without a baseline is a matrix whose numbers quietly go stale. What core moved between ``679d9817`` (the baseline the previous reconciliation was written against) and ``debde516``: - Value kinds went from fourteen to sixteen: NaN and the infinities are kinds of their own rather than travelling as ``float_value``, which JSON cannot represent faithfully. ``nan_value`` carries the text ``"nan"`` and ``infinity_value`` carries ``"+inf"``/``"-inf"``; the negative direction is asserted on its own, because one wire kind carrying two spellings is exactly where a sign can go missing, and the NaN comparison is written so that ``nan != nan`` does not read as a failed round-trip. - ``ComparisonPredicate`` normalises its operator now, so the introspective constructor's placeholder stopped satisfying it. Registered constructor, not an UNCONSTRUCTIBLE entry: the class renders, and a constructor keeps it in the matrix rather than parking it. - ``CustomType.raw`` is keyword-only, and the type root's refusal now speaks the generic dispatch's own spelling. - ``ForUpdateClause`` normalises the word it stores, so its LEGITIMATE_NON_ ROUNDTRIPS entry describes a loss that no longer happens and was retired. Coverage at ``debde516`` (test_coverage_report prints this on every run): 515 registered, 510 constructible, 5 pinned-unconstructible, 61 pinned-non-render; rendered 327, unsupported 121, unmodelled 32, non-render 30. Evidence (central venv, PYTHONPATH=src;tests): - tests/.../feature/backend/expression/test_expression_roundtrip_all.py, full run with no options beyond the repository defaults: exit 0. The run reports "bringing up nodes" -- the parallel default from the previous commit, on the one file whose runtime it was introduced to bound. - the same file under ``-n 0 -s -k coverage_report``: exit 0, printing the coverage line above, so the figures in this message are the run's, not a transcription. - ruff on the file: clean.
`ClickHouseAddColumnAction.after` has been declared, stored, serialised and read by nobody since the class was introduced. The kind audit found it: the parameter's only purpose is to change the rendered SQL (§4.4), and the SQL did not change -- `ALTER TABLE t ADD COLUMN c UInt32` came out the same whether or not the caller named a position. The example that ships with this backend (`examples/schema_diff/column_order.py`) passes `after="name"` and then reports the new column shifted into position; it never did. The clause now renders, on its own entry. Two kinds means two entries (§3): `format_clickhouse_add_column_action` for the derived node and core's `format_add_column_action` for `AddColumnAction`, which is what core's entry should have been rendering all along -- before this, one entry served both. The name is an identifier, not a value, so it goes through `format_identifier` and arrives quoted -- including when the name is a reserved word, which is the case a bare f-string would turn into the server's syntax error rather than our own. `after=None` renders byte-identically to the core entry: the clause is a position the caller asked for and nothing else. Covered by feature/backend/expression/test_add_column_action.py (five cases: no position, the core node untouched, AFTER, AFTER composed with IF NOT EXISTS, AFTER on a reserved word), and the matrix's constructor for the class now hands `after` a value so the declared parameter is exercised on the artifact side too -- the matrix pins serialisation and classification, the new file pins the text, and between them the parameter is no longer one nothing reads. Evidence (central venv, PYTHONPATH=src;tests, exit 0): - tests/.../feature/backend/expression (whole area, default parallel scheduling): 1157 passed, 10 skipped - the same area under -k add_column: 6 passed - ruff on the two changed source/test paths and the new file: clean
…sted The create formatter refused every generic CreateMaterializedViewStatement wholesale, before it read the statement's clauses at all. That answered the four-state contract wrongly on the "neither" state: with neither WITH DATA nor WITH NO DATA requested there is nothing to refuse, yet the statement was refused as a whole (feature name "CREATE MATERIALIZED VIEW") -- so the unspecified state could not be told apart from a statement ClickHouse genuinely cannot write, and core's gate round went red: test_clause_pair_guard.py::TestFourStatesArePairwiseDistinguishable:: test_neither_set_renders_the_dialect_default[CreateMaterializedViewStatement.with_data] expected the render 'CREATE MATERIALIZED VIEW `mv` TO `target` AS SELECT `id` FROM `t`' and got a refusal (CI run 39312577539, every leg). The duo is now gated the way core gates it and the way this backend's CTAS formatter already gates it: a *requested* spelling is refused by its own name (CREATE MATERIALIZED VIEW WITH DATA / ... WITH NO DATA), and with neither requested the statement renders the dialect's default form, without the clause. The whole-statement refusal survives for what it was actually for: a statement carrying neither a TO target nor an ENGINE, which ClickHouse requires (without TO the engine is mandatory), still refuses as a whole -- test_generic_create_without_engine_or_to_is_rejected pins exactly that. to_table and engine are not fields of the generic node, so a caller that carries them attaches them on the instance; the formatter now reads them through getattr, and the clauses of the ClickHouse spelling likewise, which keeps one render path for both node kinds. The siblings already worked this way: a generic DROP MATERIALIZED VIEW renders DROP VIEW and a generic REFRESH MATERIALIZED VIEW renders SYSTEM REFRESH VIEW, each refusing only the clauses ClickHouse cannot spell. The create formatter was the outlier. Evidence (worktree venv, PYTHONPATH=tests, exit 0): - feature/backend/test_clause_pair_guard.py (whole file, 73 tests): passed - the same file under -n 4 together with the whole types/protocol file: passed - feature/backend/dialect/test_materialized_view.py (whole file): passed, including the generic-statement refusals and the no-TO/ENGINE refusal - ruff on the changed path: clean
…ip harness
The testsuite's shared harness is gone: it deleted testsuite/utils/expression.py
(e05edb0, round-trip coverage moves to the owning repository), and its
parse_contract.py imports that deleted module, so the import from it dies too.
CI proved it on every leg of run 39312577539:
TestParseRoundTripSweep::test_every_rendered_type_parses_back_coherently never
ran -- it raised ModuleNotFoundError
("No module named 'rhosocial.activerecord.testsuite.utils.expression'") at
import time, so the sweep's verdict on this dialect was unknown rather than
green. Pointing the import at a fixed parse_contract is not an option: the
module that dies is the one the sweep imports, and the venv cannot see either
one.
Following the mysql precedent (35018b4) and this repository's own
vendored-harness precedent (eb45de1), parse_roundtrip_failures is now vendored
into expression_roundtrip_harness.py. It is the suite's sweep with its
make_instance dependency resolved to the harness's own make_instance (the same
function the matrix's dedicated constructors register through), and the two
scope decisions carried over verbatim: a class the registry cannot build or
cannot render belongs to the matrix's lists, and a CustomType answer is never
blessed for a rendered word. The test's class docstring already describes the
sweep as "the shared sweep across this backend's whole declared surface"; the
import now names the vendored module beside the matrix that registry comes
from.
With the import alive the sweep runs for the first time here, and it passes:
the WIDENING_ANSWERS table already covers every widening this dialect applies
and the backend's own classes parse back to themselves, so no gap the dead
import had been hiding was named (mysql's companion fix found one; this
dialect found none).
The same file also drops one unused function-local import (ClickHouseGeometryType,
ruff F401) so the changed path lints clean.
Evidence (worktree venv, PYTHONPATH=tests, exit 0):
- tests/.../backend/types/test_clickhouse_type_protocol.py (whole file):
388 passed, including TestParseRoundTripSweep with the vendored import
- the expression area --collect-only: 1116 tests collected, no errors (the
harness's added symbol does not disturb the matrix that imports it)
- ruff on the two changed test paths: clean
The LIKE family's operator became an enum (core f2bdf386): the parameter is LikeOperator, the keyword is a spelling the dialect owns, and the constructor coerces words and keywords alike (core 4774c502) -- but the wire form is the member. These call sites still passed the SQL keyword as a string, so they built whichever shape the coercion happened to produce; they now pass LikeOperator.LIKE, which is what the artifact carries. Verified: the LIKE predicate renders on this backend's dialect through the new enum (left LIKE placeholder, the pattern bound); ruff on the touched files matches the repository baseline.
Every catalog read this backend performs on its own metadata is now an expression tree: `QueryStatement` + `Column` / `FunctionCall` / `ComparisonPredicate` / `InPredicate` / `LogicalPredicate` / `WhereClause` / `OrderByClause`, with `Literal(dialect, v, inline=True)` wherever the old text wrote a fixed word into the statement. The dialect renders; the introspector and the backend only construct and execute (expression-dialect §16.2). Migrated formatters (counts are "string builds -> node builds"): - `mixins/introspection.py`, all 8 `format_*_query`: the six that render plus the two that refuse (foreign key, trigger — unchanged refusals). - `mixins/namespace.py`: `engine_kind_condition(offset=)` -> `engine_kind_predicate()`. The old form counted its own placeholders from a caller-supplied `offset`, so the correct index depended on how many conditions happened to precede it. `InPredicate` / its negation carry their own parameters, in order. - `introspection/status_introspector.py`, 16 reads (8 sync + 8 async) plus the replica summary: 18 hand-built `SELECT`s -> 18 nodes in the new `expression/status.py`, rendered by the new `mixins/status.py`. - `backend/backend.py`: the mutation pre-count was `f"SELECT count() FROM ... WHERE ..."`, built by rendering the WHERE clause and then stripping the word `WHERE` back off the front of the text; it is now `ClickHouseRowCountQuery` with the predicate the mutation already carries. The version read and the two liveness probes became `ClickHouseVersionQuery` / `ClickHouseConnectionProbeQuery`. - `mixins/concurrency.py`: the `max_concurrent_queries` read became `ClickHouseSettingQuery` (sync + async). New nodes: `expression/status.py` (20 classes, each with its own formatter — no shared intermediate node, because a node-shaped base with no kind and no formatter can never render); `mixins/status.py` (`ClickHouseStatusQueryMixin`). Baseline diff (16 introspection shapes + 10 status steps, machine-compared): **0 failures**. Projection order, every alias, and every parameter tuple are identical. The differences that exist are the three the wave guide allows plus two stated ones: | difference | why | |---|---| | `system.tables` -> `` `system`.`tables` `` | the allowed `Table` reference form: the database is the object's `catalog_name`, not part of its name | | `!= 'system'` -> `<> 'system'` | the comparison node normalises to `ne`, whose standard spelling is `<>` (both engines accept it) | | `engine NOT IN (...)` -> `NOT (engine IN (...))` | the negation node; same parameter list, same order | | `0 AS AUTO_INCREMENT` -> `CAST(0 AS Int64) AS AUTO_INCREMENT` | a literal cannot carry an alias (core: a bind parameter has no name to give), so constant cells are cast to the type the position declares and the cast carries the alias | | `LIKE 'Nullable(%%'` -> `LIKE 'Nullable(%'` | the `%%` was escaping for clickhouse-connect's `%` interpolation, which the backend already applies to every statement it executes; writing it twice escaped it twice | Evidence (central venv, no live server — B-class repo): - baseline/after diff, introspection: 16 cases, 0 failures - baseline/after diff, status introspector: 10 steps, 0 failures (baseline captured before any edit into `clickhouse-intro-baseline.json`, after into the same shape) - `test_introspection_query_nodes.py` (new, offline): 13 passed - `test_show_output_shape.py -k "not AgainstTheServer"`: 37 passed - roundtrip matrix `test_expression_roundtrip_all.py`: exit 0 - ruff on the touched set: clean - live-DB legs (`TestShowOutputAgainstTheServer`, `test_introspection.py`, `test_status_introspector.py`) are not run locally: no ClickHouse server on this machine. They go to CI.
The SHOW family already had nodes and dialect formatters, but a node was only
sometimes finished: `schema()`, `full()`, `like()`, `for_user()`, `for_table()`,
`limit()`, `global_vars()` and `global_status()` let a caller build an
incomplete statement and fill it in afterwards. `get_params()` could not say
which state an artifact had captured, and the three call sites that drove SHOW
(the functionality facade, the sync introspector, the async introspector) each
configured-then-rendered in their own order (expression-dialect §16.1).
- `expression/show.py`: every parameter is a constructor parameter, stored under
its own name, and every fluent setter is gone. `schema` is declared on each
subclass that can carry one, not only on the base, because `get_params()`
reads the class's own signature and a base-only qualifier is invisible to the
artifact. `ShowGrantsExpression` gains `user` / `host` as constructor
parameters (it had only `for_user`).
- `show/dialect.py`: the formatters read the declared fields instead of
`expr.get_params()` — one spelling, no dict round-trip. The legal formatters
are kept as they are; nothing here changed what SQL is emitted.
- `show/functionality.py` (9 sites) and `introspection/show_introspector.py`
(38 sites: 19 sync + 19 async): each builds the node complete and renders it.
Rendered SQL is unchanged for every shape. Verified by constructing and
rendering all 19 classes through the facade against a recording backend:
SHOW CREATE TABLE `db`.`t` ()
SHOW CREATE VIEW `db`.`v` ()
SHOW FULL COLUMNS FROM `db`.`t` LIKE %s ('i%',)
SHOW INDEX FROM `db`.`t` ()
SHOW FULL TABLES FROM `db` LIKE %s ('t%',)
SHOW DATABASES LIKE %s ('d%',)
SHOW PROCESSLIST ()
SHOW ENGINES ()
SHOW GRANTS FOR `root` ()
and by `get_params()` round-tripping every class (rebuild from its own
parameters, compare): 19/19 equal. The nine MySQL-only commands still refuse
with `UnsupportedFeatureError` naming the ClickHouse replacement.
Evidence (central venv, no live server — B-class repo):
- render/no-op smoke above: 9 emitted statements + 19 parameter round-trips, exit 0
- `test_show_output_shape.py -k "not AgainstTheServer"`: 37 passed
(four assertions moved from the setters to the constructors; two new cases pin
that `full=True` still never reaches `SHOW PROCESSLIST`)
- `test_introspection_query_nodes.py`: 13 passed
- roundtrip matrix: exit 0
- `rg '\.with_\w+\(' src`: no matches; `rg '\.(schema|like|full|for_user|for_table|limit)\('` in the SHOW modules: no matches
- ruff on the touched set: clean
- live-DB legs stay for CI (no ClickHouse server here).
…them, and double the LIKE percent
Two defects the first CI run found (`39486311884`, 3.10 / ClickHouse 25.8:
4 failed, 3725 passed). Both are mine, and neither could show up locally.
1. `mixins/concurrency.py` imported `ClickHouseSettingQuery` from
`...expression.status` -- three dots, from `backend.impl.clickhouse.mixins`,
resolves to `backend.impl`. The module is `backend.impl.clickhouse.expression`,
so the import is `..expression.status`. CI reported it the way a swallowed
exception does: `Failed to fetch concurrency hint: No module named
'rhosocial.activerecord.backend.impl.expression'` on every connection, with
the hint quietly falling back to `None`. The four sync/async probe sites are
fixed; the hint now resolves.
2. `format_column_info_query` wrote `LIKE 'Nullable(%'` with a single percent,
on the reasoning that the backend escapes `%` for the driver. It does -- in
`Backend.execute`. The introspector executor does not go through it: it calls
`cursor.execute(sql, params)` directly. So this statement alone reached
clickhouse-connect's `%` interpolation with a bare `%`, which was read as the
start of a format specifier:
ValueError: unsupported format character ''' (0x27) at index 134
That is exactly the failure the previous concatenated form avoided by writing
`%%`, and the behaviour in `tests/.../test_show_output_shape.py` documents.
The doubling is restored and the docstring now says why the doubling is
load-bearing rather than cosmetic -- the executor asymmetry is the reason.
The offline test pinned the repaired spelling, so the next edit that
"simplifies" it back to a single `%` fails on a machine with no server:
IF(`type` LIKE 'Nullable(%%', 'YES', 'NO') AS `IS_NULLABLE`
Evidence (central venv, no live server — B-class repo):
- `test_introspection_query_nodes.py`: 13 passed
- introspection baseline diff: 16 cases, 0 failures (`%%` normalizes to `%`)
- import smoke: `ClickHouseSettingQuery` resolves from both concurrency mixins
- ruff on the two touched source files and the test: clean
- the live legs these four failures belonged to
(`test_introspection.py::test_get_table_info_returns_table`,
`test_table_info_has_clickhouse_columns`, `test_column_data_types`,
`test_list_columns`) go back to CI, which is where they failed.
…um fix
Two things changed under this table, both from the same core commit.
The nine datetime/interval classes were pinned on the XML channel because the
XML encoder spelled ``DateTimeField.YEAR`` through ``Enum.__str__`` and the
normaliser refused it. Core gave those families ``__str__ = str.__str__``, so
the channel writes the word and all three channels round-trip exactly. The rows
are gone; the comment block stays and says what the family used to be.
``ColumnGeneratedConstraint`` and ``ForeignKeyConstraint`` were pinned on all
three channels because ``GeneratedColumnType`` and ``ReferentialAction`` were
plain enums the artifact refused to name. They are str enums now: the writer
carries them, the dict channel hands the member through untouched, and JSON and
XML return the word -- which the constructors store raw. So the pins narrow to
``("word",)`` on those two channels, the same shape the other word-form rows
already had.
``XMLParseExpression`` and ``XMLSerializeExpression`` move from ``enum-str`` to
``word`` on the XML channel for the same spelling fix: the channel no longer
writes ``"XMLParseDocumentType.DOCUMENT"``, so the pin records a type loss
rather than a corrupted value. ``enum-str`` stays in the vocabulary because the
mode documents a real spelling, not because a row uses it today.
The ``LEGITIMATE_NON_RENDERS`` table is untouched: it records classes with no
formatter, which this fix does not affect.
…as no form for ClickHouse's ALTER DATABASE takes a list of ALTER commands, and MODIFY SETTING / MODIFY COMMENT are the two this branch spells. It has no RENAME TO: the server's own answer to that spelling is a syntax error, "Expected one of: ON, a list of ALTER commands, ALTER command, MODIFY SETTING, MODIFY COMMENT". A database is renamed with RENAME DATABASE <old> TO <new>, which is a different statement. The mixin rendered the portable spelling anyway, on the belief -- stated in its own comment -- that "RENAME TO is the one action core's enum names and ClickHouse spells the same way". The same file's supports_alter_database docstring said the opposite: "MODIFY COMMENT only". The server settled it. Measured on ClickHouse 26.7.3 (HTTP, port 18682): ALTER DATABASE `db` RENAME TO `db2` syntax error, at "RENAME" ALTER DATABASE `db` MODIFY SETTING x = 1 accepted ALTER DATABASE `db` MODIFY COMMENT 'x' accepted RENAME DATABASE `a` TO `b` accepted So the action is refused by name now, and the refusal names the spelling that does work. Every other action was already refused; the suggestion text said "ClickHouse supports ALTER DATABASE ... RENAME TO", which was the same wrong belief, so it names MODIFY SETTING / MODIFY COMMENT instead. The round-trip case that pinned the rejected text now pins the refusal. It asserted the statement's spelling and never asked the server, which is how a form ClickHouse rejects could sit in the suite for this long. MODIFY SETTING and MODIFY COMMENT are what ClickHouse does accept, and core's AlterDatabaseAction has no member for either, so they are still not reachable from here. That is a missing capability rather than a wrong rendering, so it is left for a decision rather than added here.
The dialect's surface is two kinds of member: capability switches (``supports_*()``) and formatters (``format_*(self, expr)``). This change makes ClickHouse's code match that, and settles what a type position holds while it is there. **A formatter owns its production.** Core's dialect mixins held six private renderers serving families of thin ``format_*(self, expr)`` wrappers. The wrappers were formatters; the bodies were not, and being private they could not be overridden. ClickHouse's ``lengthUTF8`` -- which core spelled through one of them -- now owns its production. **A type-name position holds a type.** The positions that carried a raw string guarded it with a character class in the base class, which neither matched what the engines accept nor kept the value inside its syntactic position. A position now holds a type node; a name the framework does not model becomes the framework's name node, whose name the grammar validates. **Binding is the caller's call.** A DDL position that cannot bind says so through ClickHouse's own capability switch, and a value the caller chose to inline is inlined. The row-source probe answers for ClickHouse too. **The backend's own answers are the effective ones.** ClickHouse's mixin precedes the core mixin it overrides, and a test holds that in place. **The mixin list says what it is for.** The base-list convention is written down where the list is, and the function examples reach their constructors through ``functions as F``.
Core renamed the four statement-shaped nodes it still called *Expression: TransactionExpression -> TransactionStatement SetTransactionExpression -> SetTransactionStatement SQLiteAttachExpression -> SQLiteAttachStatement SQLiteDetachExpression -> SQLiteDetachStatement This repository refers to them in its dialect signatures and its tests, so the same names follow here. Nothing changes but the names.
``ShowCreateTableExpression`` renders a whole ``SHOW CREATE TABLE`` statement, so it is a ``*Statement`` now. The dialect entry point keeps its name: the SHOW family's twenty formatters all end in ``_expression``, and renaming one of them alone would make it the odd one out. Nothing changes but the class name: no behaviour, no rendering, no signature.
… nodes A dict cannot carry the clauses' types, their canonical order, or their own spellings, and every value in it was pasted into the statement verbatim. One clause is now one `ClickHouseTableStorageOption`: an `option` member plus a `value` whose shape that member decides, validated at construction and rendered by `format_clickhouse_table_storage_option`. - `ClickHouseEngineSpec` carries the engine and its parameter nodes, so `ReplacingMergeTree(version)` is expressible without raw text. - `ClickHouseSetting` carries one `SETTINGS` pair; a scalar goes through `format_literal`, not through the caller's string. - `ClickHouseCreateTableOptions` extends the core node with `on_cluster` and `table_storage_options`, and has its own entry -- class name, kind and entry stay three-way consistent instead of inheriting core's entry under a ClickHouse kind. - Clauses are emitted in ClickHouse's grammar order whatever order they were declared in; a repeated option fails at construction; a single expression is not parenthesised. - `ON CLUSTER` sits between the table name and the column list, where the grammar puts it. - `ClickHouseStorageOptionKind` spells itself as its keyword, so the artifact carries `"ENGINE"` and not `"ClickHouseStorageOptionKind.ENGINE"`. Removed without aliases: `format_table_engine_clauses`, `format_clause_list`, `_format_table_storage_options`, and both `format_storage_options` declarations. The core `CreateTableStatement(storage_options=...)` slot is refused by name, with the replacement spelled out in the message.
The provider fixtures still handed a dict to ``CreateTableStatement``'s generic ``storage_options`` slot, which ClickHouse refuses by name, so every DB-backed feature fixture failed at table creation (566 errors): the statement renderer was right and the fixtures were a release behind the contract they exist to exercise. ``tests/providers/fixtures/_common.py`` gains ``default_table_options``, the one place the shared clause run (``ENGINE = MergeTree``, the sort key, the two lightweight-update settings) is built; ``order_by`` is the only part that varies. The post-processing that existed only because the generic renderer quoted option values -- ``_strip_storage_option_quotes`` and the ``ENGINE = InnoDB`` rewrite -- is gone with the dict it repaired. Two structural consequences: - ``ClickHouseTableMixin`` inherits ``ClickHouseTableEngineMixin``: the engine, sort-key and settings renderers are the tail of ``CREATE TABLE``, and the conformance suite pairs one table protocol with one mixin. - the adapters and dialect-security tests exercise the nodes instead of the removed dict renderer, and the literal-readback probes declare their clause run the way the fixtures do.
Statement nodes that rendered a whole statement behind the name ``*Expression`` are now ``*Statement``, and every formatter gained, dropped or respelled no word compared with the kind of the node it renders. The SHOW family forms the bulk of it, and the class name was the only part disagreeing: these render statements. Their formatters are ``format_show_*_statement``, with the capability's own spelling and no backend prefix, since MySQL, MariaDB and ClickHouse all spell SHOW the same way. CREATE SPATIAL INDEX is a statement and is named one. The JSON family, MATCH AGAINST, the optimizer hint, the spatial literal and the eight ST_* functions stay fragments and now spell their formatters with the ``_expression`` suffix their class names carry -- including ``format_st_as_geo_json_expression``, because AS_GEO_JSON is three words and the class has said so all along.
Core moved ten cross-package helpers out from behind a leading underscore: they are called from this backend, so the underscore was claiming a scope the names never had. Each old name had exactly one new name, checked against every public definition and call site in the ten repositories first, so nothing is shadowed by the rename.
…ge definition ClickHouse answers two questions the core introspection API has no statement shape for, and no other backend can answer at all: how a table's parts divide into partitions (system.parts, grouped by partition_id), and the storage definition a table was created with (system.tables). Both are read as expression statements and rendered by the dialect, like every other query in this backend: two nodes in ..expression.introspection carry the database and table, and ClickHouseIntrospectionMixin writes each statement in full out of column, predicate, aggregate and clause nodes. No part of either statement is assembled from text, so the bind parameters stay in the tree and are emitted in projection order. The reads are cached under IntrospectionScope.TABLE with extra='partition' / extra='engine'. The extra namespace is what keeps them apart: the same table also answers column, index and engine questions, and must not answer a partition question with a column answer. A missing table is not cached, because None is the cache's no-entry sentinel and caching it would make the next call answer from a miss anyway. Two naming decisions worth stating rather than implying: the methods and nodes say database, not schema -- ClickHouse has no schema layer, its top-level namespace is a database, and the value is the same string system.parts.database holds; the core introspection API calls that slot schema because most backends have both, so this backend is the place a reader would otherwise guess wrong. And only *active* parts are aggregated: ClickHouse keeps a replaced part until its replacement is written, so summing every row would count the same rows twice -- active = 1 is what makes the row count mean what it says. The results are frozen dataclasses, one row per answer, so a cached answer cannot be mutated by one caller and handed to the next. total_rows / total_bytes stay None rather than 0 for engines that do not track them: an absent estimate is not a zero, and reporting one would be a claim the server did not make. The CLI reaches the same methods through introspect partitions and introspect table-engine, and an example exercises both plus the cache.
The introspect subcommand accepted a list of types and nothing in the tree said what each one reads or where its answer comes from. This adds a README to each backend's cli package: the questions with the catalogue each reads, the options, the notes that keep an answer from being misread (estimate lengths, upper-casing, version gates, capabilities that are refused rather than empty), and examples. The tables are generated from the code, not from memory -- every entry was checked against the statement it dispatches to.
The introspection CLI took a --schema flag on backends that have no schema. MySQL, MariaDB and ClickHouse have exactly one namespace level and call it a*database; Firebird has a single unnamed namespace and qualifies nothing above the*table. Asking those engines for a --schema named a level they do not have, and*the flag's help text then had to explain that the value meant something else. The flag is gone from those four. What is left is the flag the connection*already takes, which for MySQL, MariaDB and ClickHouse is --database -- so the*run connects to a database and asks about that database, which is the whole*answer when there is only one level. list_datasets() is the way to enumerate*databases; the run does not need a flag to ask about a different one. BigQuery had both --schema and --dataset naming the same level, with a*fallback order between them. One flag now, --dataset, which the connection*takes. The flags are removed rather than aliased: an alias lets a caller keep naming a*level that does not exist, which is the confusion this removes. The tests that*pinned the old flags now pin that they are gone.
…swer `introspect` accepted `foreign-keys` and `triggers`, while the dialect declares `supports_foreign_key_introspection() = False` and `supports_trigger_introspection() = False` -- ClickHouse has neither. A CLI that advertises a question the engine refuses is worse than one that does not advertise it: the user learns from an error instead of from `--help`. The two types are gone from `INTROSPECT_TYPES`, and the unreachable branches that dispatched them went with them. One of those branches was itself broken: it called `list_triggers(table=...)` where the parameter is `table_name`, so `introspect triggers` failed with a TypeError rather than the capability refusal it was written to produce. Removing the `elif` headers left their bodies orphaned inside the `table-engine` branch, where they would have run as part of it; both blocks are deleted outright rather than left as dead code that looks reachable. `get_table_info` still carries an `if info.foreign_keys:` section, because that is core's shape for a table's own answer and the guard makes it inert on this engine -- recorded rather than changed here, since touching it means special-casing core's result in one backend. Tests: the cli and introspection suites pass.
The same false negative as SQL Server's: the switches were left to core's default of False while the backend reads the whole RDB$ catalogue and its live tests cover generators and domains against a real server. Declared on the existing dialect mixin rather than a new file, since it already owns this backend's introspection concerns and a second home for them would split the answer across two modules. Each answer names its relation.
Core no longer arbitrates a per-backend family vocabulary: `backend/cli/introspect_capabilities.py` and its `available_introspect_types()` are gone, so importing them here broke the CLI outright. The parser is back to `choices=INTROSPECT_TYPES`, the list this backend writes for itself, and the standing `ClickHouseDialect()` built only to feed that helper is gone with it. What the helper was asked to decide -- whether `foreign-keys` and `triggers` belong on this engine -- stays answered, just in the place that owns the fact: the two types remain absent from `INTROSPECT_TYPES` because ClickHouse structurally has neither. The dispatch branches for them were already removed, so nothing advertises a question the engine cannot answer. The namespace rule is unchanged and re-checked: ClickHouse has one level, the database, so there is no `--schema` flag and every call site passes `schema=None` (or `database=None` for the ClickHouse-only reads). The README said `triggers`/`foreign-keys` were "refused"; they are not offered at all now, and it says so.
…ntract The row parsers are this backend's implementation of the base introspection contract: the base declares them and every backend must supply them, so the leading underscore described a visibility the code never had. It is also how the next private helper gets justified, which is what this branch is trying to stop. default_schema becomes a property rather than a method, beside dialect and cache: it states a fact about the database, it does not perform an action. No aliases: the old names are gone. ClickHouse side: thirty-one parsers across the main, SHOW and status introspectors, plus default_schema. The SHOW output-shape test that calls the parsers directly follows.
"table" answered several catalogue questions at once -- the relation's row, its columns, its indexes and its foreign keys -- and printed them as three sections. Each is an item the introspector reports on its own and each already has a type, so the aggregate was a second way to ask for the same rows and the only way to get an unparseable answer: under "-o json" the sections are three arrays one after another, which is not a document. The tests that asserted the three-section rendering (and the not-found path that only the aggregate had) go with it; the types that remain each answer one question.
Table introspection compared system.tables.engine against the words BASE TABLE, VIEW and SYSTEM VIEW, none of which a ClickHouse server reports. Every relation therefore came back as a base table, and the engine name stood in for the kind in output. The kind is now derived from the engine through the dialect's declared view-engine set (View/LiveView/WindowView -> VIEW, MaterializedView -> MATERIALIZED_VIEW, system database -> SYSTEM_TABLE, anything else -> BASE_TABLE), the projection names engine as ENGINE, and the engine string travels in extra. The table_type filter read the same value as an engine equality, so VIEW selected nothing. It now reads the TableType vocabulary: VIEW and MATERIALIZED_VIEW select their engines, BASE_TABLE is the complement of the view set, SYSTEM_TABLE selects the system database, and a value that is not a kind keeps working as an engine name. parse_database_info reported 0.0.0 as the version and the database engine as encoding. It now reads server_version (never raising, no I/O), uses the unknown version convention when nothing knows it, and stops claiming a charset and collation ClickHouse does not have per database; the engine is kept in extra. The introspect CLI calls introspect_and_adapt() after connecting, and the supports_* version predicates use version_at_least() so an unadapted dialect answers False instead of raising.
…aceholder AsyncClickHouseBackend raises in its constructor -- clickhouse-connect has no async mode -- and handle() called it unguarded, so introspect ... --async ended in a traceback from a constructor rather than in a sentence. query already refuses the same request with the same words; introspect now does too. The async handler stays, and stays corrected: it is unreachable through the CLI while the backend is a placeholder, but its five call sites were naming methods no introspector class has, which is a trap for whoever connects a real async backend. The note beside it says so.
`fix/column-constraint-gates` was squashed into `release/v1.0.0.dev30` as `57439841`, so the branch reference this workflow used to resolve core through no longer names anything ahead of the release. Pointing at the release branch makes each run test what a user installs. The testsuite stays on `fix/column-constraint-gates` on purpose: its own branch carries the two `pytest_compat` shims that keep pytest 9.2 working (the benchmark fixture's `_nodeid` read and xdist's loadgroup suffix), and those are not on the testsuite's release branch yet. That line is repointed when the testsuite merges.
Named after the pull request once it had a number, which is the order the fragment check requires: 4 entries, one per kind of change.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## release/v1.0.0.dev1 #6 +/- ##
=======================================================
+ Coverage 70.23% 74.81% +4.58%
=======================================================
Files 127 136 +9
Lines 7438 7843 +405
Branches 916 887 -29
=======================================================
+ Hits 5224 5868 +644
+ Misses 1923 1688 -235
+ Partials 291 287 -4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Lands
fix/column-constraint-gatesonrelease/v1.0.0.dev1.Scope up front: this branch carries the whole cycle started on 2026-10-11 as it reaches this backend — the statement/expression refactor, the typed literals, and the introspection pass at the tip — not only the last few days of fixes. 50 commits, 7 of them marked breaking.
Headline changes:
CREATE TABLE's storage clauses are expression nodes — engine,ORDER BY,PARTITION BYandTTL— which is also what madepartitionsandtable-engineanswerable (feat(clickhouse)!: declare CREATE TABLE storage clauses as expression nodes).feat(introspection): read ClickHouse's partitions and a table's storage definition).introspect --asyncrefuses instead of raising out of the placeholder.AsyncClickHouseBackendis a fail-fast stub, and the command used to call it unguarded, ending in a traceback; it now says so in one sentence, and the five*_asynccall sites that named methods no introspector has were corrected (fix(cli): introspect --async refuses…)._TYPE_CAPABILITIES.Type of change
feat: A new featurerefactor: Internal restructuring with no intended behaviour changetest: Adding missing tests or correcting existing testsdocs: Documentation onlyperf/chore/ci/build/revert(not relevant)Breaking Change
Impact on End-Users:
introspect partitionsandintrospect table-engineexist;introspect tableis gone.introspect ... --asyncnow answers "--async is not supported by the ClickHouse backend" and exits 1, instead of a traceback.Impact on Backend Developers (Custom Implementations):
parse_*lost its leading underscores.Related Repositories/Packages
python-activerecord57439841pytest_compatpython-activerecord-testsuite1991449python-activerecord-clickhousec6295e1Core is merged (
rhosocial/python-activerecord#127, squashed as57439841); this branch's CI now installs core fromrelease/v1.0.0.dev30. The test suite is still referenced by branch, because itspytest_compatshims are not on its release branch yet; that reference is repointed when the test suite merges.Testing
Test Plan: The full matrix ran green on this exact head (
c6295e1) as run41092221103— every Python version and every service container the backend tests against. A local full run is skipped deliberately: the organisation caps a single task at 180 seconds, and the CI result on this SHA is the stronger evidence. The offline CLI tests (parser surface, NAME rules, capability gate) run in under a second and are part of that run.Checklist
README.md, the CLI guide).format_*).Additional Notes