Repository navigation
feat(mysql)!: the statement layer, typed literals, and one introspect question per family - #62
Merged
Merged
Conversation
The shape check now lives where the flag has to be honoured rather than where it is declared, so a value that would silently render the wrong DDL is named instead. The attribute is read directly: whether the field exists is settled by the class, so probing for it with a default would only turn a wrong class into a quiet signed column. CI installs the core library from fix/column-constraint-gates so this backend runs against the branch it depends on.
The shape check moved from the constructor to the formatter, so a case that still expected the constructor to raise was asserting a guarantee the code no longer makes -- and passing for the wrong reason would have left the silent signed column unguarded. Each case now checks both halves: the declaration keeps the value it was handed, and the dialect names it.
… too The same case as the other widths, which the previous commit moved onto the formatter. It had been missed here: the declaration now keeps the value it was handed, and the dialect is the one that names it.
…otocol supports_alter_constraint_enforced answers the verb's existence and is version-gated ((self), 8.0.16+); the CHECK-only narrowing it used to carry as a per-kind parameter moved into this backend's formatters, where the server's own refusal (errno 3941 names PRIMARY, UNIQUE and FOREIGN KEY) becomes a refusal by name: * format_alter_constraint_action refuses every kind but CHECK before the shared rendering runs; * _format_constraint_enforcement narrows the column path the same way; * the table path's format_table_constraint used to drop an ENFORCED request on PRIMARY KEY / UNIQUE / FOREIGN KEY silently -- it now refuses by name like its own DEFERRABLE branches do. The signature sniffing (normalize_table_constraint_type from core) went with the parameter: the core deleted it, and the probe reads no kind. Tests follow the contract: the value table gains the alter-enforcement switch, per-kind assertions became formatter-level refusals, and a string kind is the constructor's to normalise, reaching the formatter as the enum.
Mechanical adaptation to the core renames (W2+W3a+W3b+W1+S1-c), 109 files / 1130 lines, all in src+tests+docs examples: - W2 statement classes (46 of the core's 57 hit this repo): QueryStatement, InsertStatement, CreateTableStatement, DropTableStatement, the transaction five (BeginTransactionStatement, CommitStatement, RollbackStatement, SavepointStatement, ReleaseSavepointStatement), Create*/Alter*/Drop* DDL, ExplainStatement, CommentOnStatement, the property-graph three. - W2 formatter (functional, 5 hits): format_begin_transaction -> format_begin_transaction_statement (mixins/transaction.py override). - W3a: ILikePredicate, FulltextMatch, XMLExistsPredicate + format_ilike_predicate / format_xmlexists_predicate (roundtrip skip lists); OnConflictExpression docstring token corrected to the core's OnConflictClause (the old name never existed in core). - W3b: SQLValue, JSONDocumentValue, JSONTextValue, CollateFunction, CastFunction, CaseFunction, UUIDCastFunction, Wildcard, and the six introspection dataclasses -> Introspection*Info. - W1: CTEClause, WithStatement, main_query= -> query= (examples/query/cte.py). Constraint surfaces still reference the removed ColumnConstraintType / TableConstraintType enums and follow in the next commit; the Wildcard references follow in the wildcard-split commit (the intermediate `Wildcard` name no longer exists in core). Expected-red window per the wave guide. Backend-owned statement classes (MySQLInsertExpression, CreateJsonDualityViewExpression, the Show* family, SetTransaction and the GraphTable/Pivot wrappers) are outside the core's rename mapping and keep their names this wave. Evidence: full-package import smoke green (138 modules) modulo the two surfaces above; targeted runs green: expression/test_expressions_transaction.py (32 passed, exit 0), dml/test_insert_on_conflict_clauses.py + test_mysql_datetime_interval_expressions.py (sync tests passed; 4 async scenario tests error on unreachable mysql_56/57 scenario servers, environmental); compileall src+tests exit 0; ruff delta vs HEAD on the 109 changed files: zero new findings (E501 count down by 3).
Core's S3 put the kind inside the clause: ``ColumnConstraint`` and
``TableConstraint`` are productions that carry a **branch**, and the branch
is a node in its own right. ``ColumnConstraintType`` and
``TableConstraintType`` are gone -- there is no tag enum left to switch on,
and every call site in this backend now says which alternative it means.
**Construction sites** (60 files; src 33, tests 25, docs 2). The mapping,
applied by hand, not by search-and-replace, because the table-level
branches take their columns positionally:
ColumnConstraint(d, ColumnConstraintType.PRIMARY_KEY,
name=n, is_auto_increment=True)
-> ColumnConstraint(d, branch=ColumnPrimaryKeyConstraint(d, is_auto_increment=True),
name=n)
TableConstraint(d, TableConstraintType.PRIMARY_KEY, columns=[...], name=n)
-> TableConstraint(d, branch=TablePrimaryKeyConstraint(d, [...]), name=n)
... check_condition / foreign_key_reference / default_value ->
ColumnCheckConstraint / ColumnForeignKeyConstraint / ColumnDefaultConstraint
... NOT_NULL / NULL / UNIQUE tags ->
ColumnNotNullConstraint / ColumnNullableConstraint / ColumnUniqueConstraint
Zero references to the deleted enums remain in this repository, including
the prose docs and the README, which wave 1 had left naming
``ColumnConstraintType``.
**Override points.** ``MySQLTableMixin.format_table_constraint`` -- a fat
tag switch over the whole table-level family -- is gone, replaced by two
branch formatters that answer for their own alternative and delegate:
format_table_check_constraint keeps the >= 8.0.16 version gate on
the branch this backend overrides
format_table_foreign_key_constraint keeps MySQL's spaced reference body,
"REFERENCES `p` (`id`)", by going
through the shared ReferencesClause
so the MATCH / ON DELETE probes
stay consulted
Both are new public members and are registered in ``MySQLTableSupport``
(protocols.py) as the protocol-conformance sweep requires. The production
level moved with the gate that belongs to it: ``format_table_constraint``
now lives on ``MySQLConstraintMixin``, next to the probes it consults, and
its declaration was dropped from ``MySQLTableSupport`` rather than left
pointing at a mixin that no longer implements it.
**Narrowing, re-checked against the new API** (wave 3 of the plan, folded in
here because the probes, the ``_require_*`` helpers and the formatters that
call them are one unit -- splitting them would put a commit on record that
contradicts core's API). The three shape rules the plan set out all hold:
probes are ``(self)``, narrowing lives in ``_require_*`` helpers, and the
kind is read off the branch class, never off the deleted enum. Two things
needed adjusting:
- the enforcement gate had to become ``_require_constraint_enforcement``.
``_format_constraint_enforcement`` takes the *production* and is shared by
the column trait renderer and the table one, so the kind test cannot live
there; the helper is what both reach.
- ``format_table_constraint`` runs that gate **before** delegating. Core
renders the branch body first and the traits after it, but this server's
CHECK body carries its own version refusal that says nothing about the
enforcement spelling -- so without the hoist, ``CHECK ... NOT ENFORCED``
at 8.0.0 would be refused as "CHECK constraint" instead of by the
spelling it asked for, and the two refusals would collapse into one.
This restores the ordering the mixin had before the branch nodes split
the production apart.
Messages follow the shared style: the feature is the SQL spelling
(``FOREIGN KEY NOT ENFORCED``), not an enum member and not a class name,
and the suggestion restates it. All measured verdicts and their errno are
unchanged -- they are MySQL's, measured on Community Server 26.7.0 at
192.168.1.3:14686, and nothing here re-derives them.
**Behaviour changes**
- A table-level FOREIGN KEY with no local column, no referenced column or no
referenced table now raises ``ValueError`` where the old switch silently
emitted nothing at all.
- The params of a rendered table constraint are a tuple. They were a list,
from the collector the old switch used; core's branch formatters hand back
tuples and the production level normalises to one.
- ``AlterConstraint`` takes ``constraint_kind=`` (a branch class, or the
spelling the constructor matches) where it took ``constraint_type=``.
**Tests**
pytest tests/.../feature/backend/ddl \
tests/.../test_mysql_ddl_features.py \
tests/.../test_mysql_ddl_improvements.py \
tests/.../test_auto_increment_ddl.py \
tests/.../test_mysql_dialect_security.py \
tests/.../test_mysql_mixin_expressions.py \
tests/.../test_mysql_constraint_switches.py \
tests/.../test_mysql_clause_pair_guard.py \
tests/.../test_mysql_protocol_conformance.py exit 0
``test_expression_roundtrip_all.py``: the stale ``_NOT_RENDERED`` row for
``ColumnConstraint`` is deleted -- the class builds through the testsuite's
branch-aware special constructor and renders `` NOT NULL``, so the row
described an API that no longer exists. Five failures and one error remain
in that file and are **identical at 8ba84de** (verified by running the HEAD
copy of the file against the same environment): ``LikePredicate`` and
``DomainCheckConstraint`` need special constructors the testsuite does not
carry, ``PassingClause`` needs a ``_DECLARED_GAP`` row. None of them is a
constraint surface and none is this wave's to fix.
Shared contract, from the core worktree::
pytest ../../../python-activerecord-testsuite/.worktree/.../testsuite/feature/ddl/
exit 4 -- ImportError: cannot import name 'ColumnConstraintType' from
...statements.ddl_table
The testsuite's own S3 adaptation is unfinished (``feature/ddl/fixtures/
models.py`` still imports the deleted enum). Recorded as the known gap; the
testsuite is another agent's repository.
ruff ``src`` is at parity with 8ba84de, per file and per rule.
Core's wildcard split (`61a1a458`) replaced the single `Wildcard` node
with `AllColumnsWildcard` for select-list / RETURNING positions and
`RowCountWildcard` for the COUNT argument position, and the shared
formatters were renamed to match (`format_all_columns_wildcard` /
`format_row_count_wildcard`). There was never an intermediate
`Wildcard` to pass through, so every call site moves straight to its
grammar position's node.
**Call sites** (6 files; src 3, tests 3), all expression-level -- the
`COUNT(*)` occurrences that remain are raw SQL strings handed to
`backend.execute`, not expression nodes, and need no migration:
select=[Wildcard(dialect)] -> AllColumnsWildcard(dialect)
FunctionCall(dialect, "COUNT", Wildcard(dialect)) -> FunctionCall(dialect, "COUNT", RowCountWildcard(dialect))
No `format_wildcard` override existed in this backend (`WildcardSupport`
is inherited from the core protocol), so none needed renaming. No
backend-specific wildcard class is warranted here: `COUNT(t.*)` is a
postgres / snowflake affair and MySQL's `GRANT *.*` is a plain string
default on the grant expression (`on_object`), not a wildcard node --
both stay as they are.
**Tests** (live scenarios mysql_97/94/92/84/80 at 192.168.1.3; the
localhost:14681/14680 async scenarios mysql_56/57 refuse connections in
this environment, a pre-existing gap unrelated to this change):
pytest .../test_mysql_partition_explain.py -k "not mysql_56 and not mysql_57" exit 0
pytest .../test_mysql_partition_operations.py -k "not mysql_56 and not mysql_57" exit 0
pytest .../test_mysql_partition_strategy_operations.py -k "not mysql_56 and not mysql_57" exit 0
python -m compileall over the six changed files: exit 0. ruff over the
six changed files: 63 findings, byte-for-byte the same rules and counts
as the same files at 3d643d6 (pre-existing E501/B017/F821/F811/E402/F401;
the S3 migration in 3d643d6 lengthened the constraint construction lines
past 120) -- this change adds none.
Every install line in .github/workflows/test.yml (three jobs: sync, async,
scenarios) pinned the testsuite to its release branch while the core line
already pointed at fix/column-constraint-gates. The two have to move
together: the release branch's testsuite still imports the deleted
`ColumnConstraintType` (its own S3 adaptation is still in progress), so
against this backend's two-level nodes it would fail at import.
rhosocial-activerecord-testsuite @ .../python-activerecord-testsuite.git
@release/v1.0.0.dev17 -> @fix/column-constraint-gates (3 places)
Checked and already correct, so untouched: the core install line is
git+https://github.com/rhosocial/python-activerecord.git@fix/column-constraint-gates
at all three places. No other workflow installs either dependency by ref
(publish.yml and changelog-check.yml have no such line; their
`release/v1` hits are GitHub Action refs).
No test evidence: a workflow-file ref change has nothing to run against.
The import smoke over the backend package (115 modules, 0 failures;
example scripts skipped -- they connect on import) is what stands behind
the branch pairing it points at.
…atrix The core's wildcard commit (61a1a458) noted the roundtrip contract needs dedicated constructors for LIKE/ILIKE, whose operand slots the generic guess fills with plain strings -- the core registered them in its dummy2 matrix, and this file is where the same registration belongs for MySQL: two factory rows with reasons, matching what the matrix's table-exactness tests demand. xml.PassingClause joins the declared gaps: MySQL has no XML family, so the S1-c clause's formatter is absent and the dispatch refuses it like every other xml.* row. The remaining failures were environmental in appearance only: CI runs with -x, so the DomainCheckConstraint setup error (fixed testsuite-side in 11b0f6c) stopped the session before these were reached.
Core dropped the Function/Expression suffix on seven nodes and renamed two dialect hooks. Carry the same names here: RawSQLExpression is RawSQL, CastFunction is CastExpression, CaseFunction is CaseExpression, PivotExpression is Pivot and UnpivotExpression is Unpivot -- the last two with their formatters following to format_pivot and format_unpivot. The roundtrip matrix names its rows by fqn suffix and asserts them exactly in both directions, so the three renamed rows move with the classes. Two rows are added at the same time: ArithmeticOperation and SetOperation validate their op against a closed set of wire words, which the generic constructor cannot guess, so each gets the dedicated factory the other un-guessable shapes already have.
Three faces of the same change: what a DDL position is written with, what a set operation carries, and what the registry is called. A literal's inline strategy is now declared by its class rather than by a per-instance flag, so the two DDL-position literals here become DdlLiteral -- a DEFAULT and a CHECK bound value both accept no bind parameter, which is exactly what that class says. SetOperationStatement is SetOperation and carries op + left + right. The operation is a word (union / union_all / intersect / except) and the dialect writes the spelling, so the UNION ALL spelling that used to be operation='UNION' plus all_=True is one word now. An unknown word is refused where it is written rather than at render time. The roundtrip docstring pointed at ExpressionRegistry._registry as the thing discovery must not use; that name is gone, and the two registries it conflated are genuinely different -- one indexes (dialect class, kind) to a formatter, the other a kind to its expression class.
…regates BinaryArithmeticExpression is ArithmeticOperation and its op is now the *operation* rather than the character one backend's lexer happens to write: add / sub / mul / div / mod, validated as a closed set at construction so a symbol or an unknown word is refused where it is written. The dialect formatter maps each word to the character the rendered SQL takes, and the precedence table is keyed by the same words -- so every BinaryArithmeticExpression(d, '+', ...) becomes ArithmeticOperation(d, 'add', ...) and renders the same bytes. The bitwise factories are the one place where that is not a rename. ArithmeticOperation admits five words and &, |, ^, << and >> are none of them, so bit_and / bit_or / bit_xor / bit_shift_* / bit_get_bit move to BinaryExpression -- a plain binary operator taking the character the backend spells. Same SQL, and no symbol smuggled into a node whose wire form does not admit it. Standard functions stop going through FunctionCall. The MAX / MIN / AVG half of math_enhanced's max_ / min_ / avg is now MaxFunction / MinFunction / AvgFunction, so a single-argument call returns the classed node; the GREATEST / LEAST half stays on FunctionCall, because those are MySQL scalars with no standard node of their own and the generic call is the escape hatch. The aggregate and subquery examples build SUM and AVG through the sum_ / avg factories instead of a name string. A windowed aggregate is left on FunctionCall on purpose: the OVER clause is not part of the aggregate nodes' contract yet.
The standard aggregate is a real node now, so COUNT is built as CountFunction rather than through the user-defined-function escape hatch it shares spelling with.
The testsuite's fix/column-constraint-gates branch dropped
rhosocial.activerecord.testsuite.utils.expression in e05edb0 ("drop the
round-trip harness the registry replaced") with the stated direction that
round-trip coverage moves to the owning repository. This matrix imported
that module, so collection failed locally and would fail in CI, which
installs the testsuite at that branch.
- Vendor the module (recovered from testsuite 4ef2551) as
tests/.../feature/backend/expression_roundtrip_harness.py, sibling to the
matrix; pytest's basedir insertion makes it importable without touching
PYTHONPATH. Provenance recorded in the module header; fix drift here
deliberately, do not re-import from the suite.
- Adapt register_all to the split registry: core's serialization no longer
exports register_expression_class. Classes now register into
default_registry() via register_node; re-registering core's own
auto-populated classes is a no-op, and the mysql classes are what the call
adds.
- Add register_enums_and_dataclasses(package) and call it for the mysql
expression package: artifacts carry enum members under their short class
name and the whitelist resolves that name, and core auto-populates only
its own packages (core + sqlite + dummy), so the backend's vocabulary
has to be registered beside its classes.
Evidence: targeted matrix selection -k "insert or table_options or
comment" -- 16 passed, exit 0 (collection previously failed).
Clear every getattr/hasattr probe of an expression slot in src (25 sites; the acceptance scan rg "(getattr|hasattr)\(\s*(expr|node|clause|item)\b" now returns zero). Each site falls in one of two shapes: - The field is declared on the node the formatter already serves: read it by name (ddl_table table_options/comment, maintenance no_write_to_binlog/options/option, namespace catalog_name, partition subpartition_by/partitions/partitions_count/subpartition_definitions). The maintenance reads stay gated on the operation, so only the classes declaring each field are ever asked. - The formatter serves both the core node and this backend's derived node (format_insert_statement), or the slot takes either an expression or a bare host value (handler key_value, DO expressions): dispatch by isinstance on the declared base (MySQLInsertExpression / BaseExpression -- the same judgement core's format_insert_values_source makes), then read the field. Behavior changes (tightened, no default masks left): - replace+ignore and replace+on_conflict are now construction errors on MySQLInsertExpression; a plain core InsertStatement can no longer pick the flags up at all and renders a plain INSERT (previously the probe would have accepted any attribute-shaped source, and the mutual exclusion fired at render time). - HANDLER ... READ key_value and DO items that are not BaseExpression are bind parameters now; the duck-typed "anything with to_sql" branch is gone. test_admin_statement_coverage's _InspectableExpr stand-in now subclasses BaseExpression accordingly. - Removed the unused ColumnTypeSupport import from dialect.py: it was never in the class base list; the protocol is satisfied structurally by MySQLColumnTypeMixin and stays verified by the conformance tests (positive list, negative list, and test_mysql_column_type). Draft follow-up absorbed: test_mysql_partition_strategy_operations.py already called CountFunction at its call site but never imported it, so the file was NameError-red at HEAD; the import is added here. Evidence (central venv, PYTHONPATH=src;tests, all exit 0): - test_mysql_column_type.py + test_mysql_protocol_conformance.py: 380 passed - test_admin_statement_coverage.py: 79 passed (after the stand-in fix) - test_mysql_partition_expressions.py + test_mysql_mixin_expressions.py + test_mysql_ddl_coverage.py: 201 passed - roundtrip matrix -k "insert or table_options or comment": 16 passed - live mysql_97 (192.168.1.3:14686): test_replace_into.py + test_insert_ignore.py --scenarios=mysql_97: 13 passed, including the construction-time ValueError paths - ruff on all changed files: clean. Pre-existing ruff debt in the partition strategy test file (23 findings at HEAD, incl. async test names shadowing their sync twins) is unchanged and out of scope.
Wire core is TypedValue family (473b8e37 / 5ba491d5 / 1549380b) into the
MySQL dialect and answer the tier-2 kinds this engine spells its own way:
- mixins/typed_value.py adds MySQLTypedValueMixin, standing alone rather
than inheriting TypedValueMixin (the dialect composes the specific mixin
ahead of the generic one; the position is what resolves the overrides):
* format_binary_value inlines X-quoted upper-case hex (X'0102FF') and
binds the bytes at the data position;
* format_uuid_value inlines the bare quoted canonical text and binds the
UUID object at the data position -- MySQL has no UUID type and no
literal other than the text;
* format_interval_value is deliberately absent: INTERVAL is only the
keyword parameter of date arithmetic here, so the base's explicit
NotImplementedError at a DDL position is the answer (structural absence,
same fail-closed choice as sqlite is reference form).
- dialect.py composes MySQLTypedValueMixin ahead of TypedValueMixin, after
ExpressionMixin and before the generic DateTime/DQL/DML block, mirroring
the placement every backend that has landed this wave uses.
- Tier-1 is not overridden: every default spelling was measured, not
assumed -- on 8.0.46 (mysql_80, 192.168.1.3:14682) and on the server
behind mysql_97 (192.168.1.3:14686, VERSION() 26.7.0) a CREATE TABLE
carrying DEFAULT TRUE / FALSE / 42 / 1.5 / 1.25 / 'x''y' / '2026-10-17'
/ '10:30:00' / '2026-10-17 10:00:00' is accepted and reads back the same
values, so the base delegating through format_literal (and through this
dialect is backslash-doubling _escape_sql_string) is the answer.
Measured evidence recorded in both new docstrings: SELECT X'0102FF' ->
b'\x01\x02\xff' and VARBINARY(16) DEFAULT X'0102FF' reads the bytes back
(also TINYBLOB DEFAULT (X'0102FF') via the expression-default syntax);
CHAR(36) DEFAULT '<canonical uuid>' reads the text back verbatim; every
INTERVAL literal attempt (SELECT INTERVAL 1 DAY, DEFAULT INTERVAL 1 DAY
in each column shape) fails with errno 1064; the single-backslash probe
('a\b') reads back as 'a\x08', which is the measured reason the default
sql_mode doubles backslashes and NO_BACKSLASH_ESCAPES stops doubling.
New tests/.../test_mysql_typed_values.py: per-kind DDL-position lift
classes (the same class-level strategy lift core performs for
Literal -> DdlLiteral), data-position binding on the shipped classes,
tier-1 audit pinning the base spellings as reached through MySQL's
escaping (including both sql_mode branches for the backslash), MRO-order
assertions, and a live-server class re-measuring the tier-2 answers end
to end (CREATE TABLE with the DEFAULT -> INSERT -> read back, plus the
INTERVAL refusal) on every scenario the suite reaches.
Evidence (central venv, PYTHONPATH=src;tests):
- test_mysql_typed_values.py, all 7 scenarios: 50 passed, exit 0
- guide is targeted selection over feature/backend -k "typed or
literal": 130 passed, exit 0
- the wave-1 live files re-run with every server now reachable
(localhost:14680/14681 were down during wave 1, now up):
test_replace_into + test_insert_ignore + test_insert_on_conflict_
clauses + test_mysql_operational_maintenance, all scenarios: exit 0
- ruff on the four changed files: clean
…UES cells
Four follow-ups to the core waves this dialect had not digested:
1. format_char_length_function override (core 9b559030's classed node):
the base default spells LENGTH, which counts BYTES on MySQL -- measured
on 8.0.46: LENGTH('aé') answers 3 where CHAR_LENGTH('aé') answers 2 --
so MySQLFunctionMixin now maps the node is declared unit to this
engine is native spelling: char -> CHAR_LENGTH, octet -> LENGTH
(measured: LENGTH(X'0102') is 2), bit -> BIT_LENGTH (measured:
BIT_LENGTH('abc') is 24). Reading the unit is a declared-field read,
not probing; the node is docstring says the backend formatter decides
whether its spelling can honour the unit, and MySQL can honour all
three.
2. INSERT VALUES delegation (core 8e2e4d82): the ValuesSource branch of
format_insert_statement re-implemented the per-cell loop with
val.to_sql(), so a bare host-value cell raised AttributeError. The
branch now delegates to the base format_insert_values_source, which
owns the expression-vs-host-value judgement in one place. The same
duck-typed hasattr(x, "to_sql") probes in the LOAD DATA SET and
ON DUPLICATE KEY UPDATE assignment loops are now isinstance checks
against BaseExpression, the judgement core itself makes.
3. Math factory migration (core a3978e10): round_ / pow / power / sqrt /
mod / ceil / floor / trunc construct the classed nodes
(RoundFunction.for_operand & co.) instead of FunctionCall. GREATEST /
LEAST stay on FunctionCall -- core has no node for them, the escape
hatch is the point. Two behaviour notes: pow() now renders POWER
instead of POW (synonyms on MySQL; the classed node carries the
standard spelling), and a bare string or number argument is refused
with a TypeError at construction naming the factory -- it used to
surface as an AttributeError from inside the renderer, and the classed
nodes own operand lift would have *parsed* a numeric-looking string
into the number it spells, the exact guessing this backend refuses.
4. A matrix-surfaced drift: validate_collation_name still read the
options bag core deleted when CollateExpression became CollateFunction
(AttributeError from inside the renderer). The bag is gone; the one
named option left is schema qualification, and MySQL is COLLATE
grammar takes a single collation identifier, so a schema-qualified
request is now refused by name (UnsupportedFeatureError) instead of
being silently dropped -- and the unknown-collation ValueError path
the matrix row records is reachable again.
New tests/.../test_mysql_classed_functions_and_bare_values.py: the three
unit spellings, the alias, the MRO resolution and the unknown-unit
refusal; INSERT with bare/expression/typed-value cells, REPLACE over
bare cells, and a RoundFunction inside a VALUES cell.
test_mysql_math_enhanced_functions.py gains class-identity assertions
and records the POW -> POWER normalisation.
Evidence (central venv, PYTHONPATH=src;tests, exit 0):
- test_mysql_classed_functions_and_bare_values.py +
test_mysql_math_enhanced_functions.py + test_mysql_ddl_coverage.py:
90 passed
- live dml: test_load_data + test_insert_on_conflict_clauses +
test_replace_into + test_insert_ignore, all scenarios: exit 0
- matrix spot checks: -k CollateFunction (2 passed);
-k "insert or table_options or comment" (16 passed, wave-1 commit)
- ruff on all changed files: clean
…names Core is get_params() reads each __init__ parameter off the attribute of the same name and refuses to paper over a class that stored it anywhere else -- its own docstring names this exact defect: the fluent setters that shared their constructor parameters names forced the storage under a underscore prefix, and the introspection-based read now raises AttributeError instead of guessing. Every SHOW expression in this backend hit that: the matrix could neither serialize nor render any of the twenty classes (show.ShowCharsetExpression was the first row to surface it; the rest followed in the full run). - expression/show.py: every constructor parameter is stored under its own name (schema, table_name, view_name, trigger_name, full, like_pattern, session, limit, user, host). The fluent setters become with_* -- the same rename core applied to its own introspection expressions (with_schema, with_table_name, ...): the parameter owns the plain name, the setter keeps its ergonomics, neither shadows the other. for_table/for_user become with_table_name/with_user, matching core is spelling. - A subclass that supports a schema qualifier (SHOW CREATE TABLE/VIEW/ TRIGGER, SHOW COLUMNS/INDEX/TABLES/TABLE STATUS/TRIGGERS) now declares schema in its own __init__: get_params reads the class is own signature, so a schema declared only on the base was invisible to the formatter and the qualifier the caller passed was silently dropped -- SHOW CREATE TABLE with schema="db" rendered the unqualified name before this change, and renders `db`.`t` now. - show/functionality.py and introspection/show_introspector.py: the ~100 internal configure-then-render sites call the with_* setters. Evidence (central venv, exit 0): - render smoke: every schema/full/like/limit/user shape renders, and SHOW CREATE TABLE `db`.`t` now carries the qualifier - test_show_functionality.py, all scenarios: 17 passed - matrix -k show: 40 passed (all twenty classes, serialization and rendering, no rows needed -- they all render now) - introspection spot check: test_introspection_database.py --scenarios=mysql_97: 20 passed (the full introspection directory exceeds the 180 s single-run budget; the SHOW-renamed introspector paths are the same call sites verified above) - ruff on the three changed files: clean
Core 5a613f3c named each ALTER TABLE action class by what it is (AddColumn -> AddColumnAction, ModifyColumn -> ModifyColumnAction, AlterConstraint -> AlterConstraintAction, ChangeColumn -> ChangeColumnAction among the twelve) and dropped the four alias names that never defined anything. Every callsite in this repository follows the new names: - src: expression/alter_column.py (MySQLAddColumn derives AddColumnAction), protocols.py, mixins/column.py, mixins/constraint.py, examples/ddl/alter_table.py - tests: the vendored roundtrip harness, the matrix, and the constraint-switch, ddl-improvements, mixin-expression and ddl-coverage files The matrix's six constraint-action rows collapse to two: the alias names (AlterConstraintAction as an alias, AlterTableConstraint, ValidateConstraintAction as an alias, ValidateTableConstraint) no longer exist as separate registry names, so only the two classes themselves are left to register. The vendored harness's register_enums_and_dataclasses is dropped in the same collection repair: the registry the core settled has no enum or dataclass table (an enum member rides as the JSON scalar its kind names), so the old registration calls no longer resolve. The format entry points (format_add_column_action and siblings) already carried the action suffix and are unchanged. Migration surface: 13 old names scanned, 25 callsite hits across 11 files renamed, 0 left (rg-verified).
Core 239678e5 gave a literal its job and nothing else: the TypedValueMixin tier and the twelve *Value classes are gone, replaced by LiteralKind plus the one Literal(dialect, value, inline=) node whose flag answers bind-or-text per position. This dialect follows:
- mixins/literal.py replaces mixins/typed_value.py. MySQLLiteralValueMixin overrides format_literal -- the single entry point for writing a value as SQL text -- with MySQL's two tier-2 spellings only: bytes become X'0102FF' (upper-case hex) and a UUID becomes the quoted canonical text. Everything else falls through to the base's three-tier construction; timedelta gets no branch, so the base's explicit TypeError stays the answer -- measured on both servers, INTERVAL exists only as the keyword parameter of date arithmetic (DATE_ADD's), never as a literal form, and leaving the method without that branch is the structural refusal. The mixin stands first in the dialect's base list, ahead of SQLDialectBase, because that is the one base defining format_literal: a mixin placed after it never wins. The TypedValueMixin wiring is gone from dialect.py and mixins/__init__.py.
- format_char_length_function stops reading expr.unit: the base's contract says the unit records the semantics the caller asked for and never enters the SQL, and the octet and bit questions have their own dedicated nodes whose defaults (OCTET_LENGTH / BIT_LENGTH) this engine accepts as-is. The override spells CHAR_LENGTH because the base's LENGTH counts bytes here (measured on 8.0.46: LENGTH('ae-acute') answers 3 where CHAR_LENGTH answers 2).
- test_mysql_literal_values.py replaces test_mysql_typed_values.py. The old class-level inline strategy (Ddl*Value subclasses setting inline_params) is gone with its shape: a DDL position now says inline=True per node, and the same measured spellings are asserted through the per-position flag. The live-server read-back class keeps its measured records verbatim -- the architecture turn changes none of the answers. The classed-functions file's VALUES-cell case follows: a Literal cell binds behind the same placeholder a bare host value gets.
- Two test call sites that composed the retired DdlLiteral lift now carry the per-position flag: a DEFAULT value clause marks its Literal inline itself, and a CHECK condition's literals say inline=True each.
Measured records are preserved unchanged: every spelling was measured on 8.0.46 (192.168.1.3:14682) and on the 26.7.0 server at 192.168.1.3:14686. Evidence: test_mysql_literal_values.py green (exit 0); -k literal across feature/backend green (127 passed, exit 0); the two DdlLiteral sites green (15 passed, exit 0); ruff clean on every changed file (the 39 findings in three legacy files are byte-identical at HEAD and untouched here).
Core 679d9817 made the artifact plain JSON and the parameter sequence the only way a value travels, and the matrix run against it exposed four shapes in this backend that no artifact can spell. - expression/json.py: MySQLJSONObjectExpression drops its **kwargs. A parameter whose identity depends on how many extras arrived has no artifact spelling (§4.11.4) -- the deserializer matches parameters by name and refuses the kind outright. The mapping and the pair iterable are the two shapes that name every member; both are taken through the one declared `data` parameter now, and the kwargs test call site follows the pair-list shape. - Twenty backend-owned enums become `str` enums with `__str__ = str.__str__` (admin, load_xml, maintenance, partition, column, json_duality_view, optimizer_hint, table_options, charset_collation). A plain Enum is neither a JSON scalar nor a node, so serialization refused every instance that carried one; a str member *is* the word the artifact carries. - The classes whose formatters read `.value` accept the word and store the member (KILL target, FLUSH/RESET options, LOAD XML selectors, maintenance selectors and option lists, SUBPARTITION BY strategy) -- §10: wide entry, normalised storage, so a restored instance renders what the original rendered. The word a vocabulary does not name is now refused at construction (MySQLResetExpression becomes unbuildable by the generic constructor; MySQLSubpartitionClause refuses with ValueError where it refused with TypeError before). - mixins/partition.py: format_partition_clause renders the clause PartitionClause actually declares -- method and keys. It used to route every method spelling into the format_partition_by_* entries, which read `subpartition_by` / `partitions` off nodes that do not declare them; the six MySQLPartitionBy* subclasses now name their own entries and the base clause renders `PARTITION BY <method> (<keys>)`. - expression/load_data.py: LoadDataOptions becomes a clause node. A constructor parameter is a scalar or an expression instance (§10) -- those are the two forms an artifact can carry, and a dataclass is neither; the statement's parameters now carry a nested load_data_options node and the test call sites pass the dialect. Evidence (central venv, PYTHONPATH=src;tests, exit 0): 200 tests across test_mysql_mixin_expressions.py, test_json_expressions.py, test_json_functions.py, test_mysql_partition_expressions.py; smoke: every touched class round-trips all three channels and renders, and the JSON object renders JSON_OBJECT with its params. ruff on the changed files: error set identical to HEAD (all remaining findings pre-exist).
Reconciliation baseline: core 679d9817 (a node's kind its identity and its payload plain JSON). Full matrix file, 538 collected classes across core's expression package and this backend's, every table exact in both directions, exit 0. Dedicated constructors (the generic filler cannot know these shapes, so each carries an explicit factory and a reason row): - the string-function family whose operands are SQLValue forward references (ChrFunction, SpaceFunction, LeftFunction, RightFunction, OverlayExpression, SubstringExpression) -- the filler hands plain strings where value nodes belong, and Left/Right/Substring compare a Literal's value against a bound the string cannot survive on the way back; - the two extract spellings (ExtractFieldExpression, _ExtractBase) whose field must name a DateTimeField member or its word. Table adjudication, each row against the live dialect: - _UNBUILDABLE: MySQLResetExpression joins -- the constructor now normalises its option through the enum and refuses the filler's guess at construction. ValidateConstraintAction, EnumType and CustomType rows retire -- all three construct since the annotation-inheritance wave. AlterConstraintAction's reason now names the keyword-only constraint_kind. - _NOT_RENDERED: the eleven declared-result annotation classes join -- format_declared_result reads expr.source.alias, an optional field a Literal does not declare (core defect, named per row); ColumnGeneratedConstraint joins (ValueError on a string expression slot). MySQLResetExpression leaves for _UNBUILDABLE. - _DECLARED_GAP: graph.EdgeTableDefinition replaces the stale graph.EdgeTable row (core renamed the class); load_data.LoadDataOptions joins -- the clause is a node so the statement's parameters can carry it, and its rendering is inline in the statement's own formatter. - _ROUNDTRIP_DEFECT, a new table with the same shape and the same bidirectional exactness the other tables have: classes whose artifact round-trip does not deliver an equivalent instance for a reason the core worktree owns. Nineteen rows, each naming the defect: DateTimeField/IntervalUnit/XML enums carrying no `__str__ = str.__str__` (the XML channel writes EnumName.MEMBER and the read refuses it), two plain core enums the value vocabulary cannot write (ReferentialAction, GeneratedColumnType), constructors that refuse or mis-store their own artifact's word (CreateTableCloneStatement, CreateTriggerStatement, ForUpdateClause, GraphEdge), two dataclass parameters with no artifact form (GraphColumn, ViewOptions), and Subquery binding a JSON array where it constructed a tuple. The row is the accounting, not the fix: when core lands a fix the row goes stale and the exactness test says to delete it. Also: test_constructor_tables_are_exact reads the vendored harness now -- the testsuite module it imported was retired when round-trip coverage moved to the owning repositories. Evidence (central venv, PYTHONPATH=src;tests): the full test_expression_roundtrip_all.py run, exit 0; ruff on the file: clean.
Kind list generated with the core script (a copy with this backend's package and dialect class filled into BACKENDS -- registering mysql there is a change to the core repository and is reported, not committed here): core kinds 357, mysql override 0, mysql custom 162. Override zero is the rule that matters most: a backend may replace the rendering of a kind core ships, never the node class behind it, because two classes under one kind leave the deserializer no way to choose. This backend adds nodes and overrides formatters; it replaces nothing. §11.1 audit, read per class: the classes that specialise a core node derive from it (MySQLAddColumn from AddColumnAction, MySQLColumnDefinition from ColumnDefinition, MySQLInsertExpression from InsertStatement, the six MySQLPartitionBy* from PartitionClause, and each MySQL<Type> from its core type), and the concepts core has no node for stand on the bare bases. No class copies a core base's mixin list by hand, and no dialect mixin sits behind the base it overrides. The audit's one finding is the compound-word defect the core docstring names when it introduces KIND_RESERVED: `MySQL` is one word, and the generic derivation split it, so this backend shipped `my_sql_flush_expression`, `my_sqljson_object_expression`, `my_sqlst_as_text_expression` and `my_sqluuid_type` -- four spellings of one identifier, none of them the word the formatter entries use. The package now declares the token where it assembles its own expressions, the way the sqlite package does, and the kind follows the declaration: my_sql_* -> mysql_* (105 kinds) my_sqljson_* -> mysql_json_* (11 kinds) my_sqlst_* -> mysql_st_* (7 kinds) my_sqluuid_type -> mysql_uuid_type (1 kind) 124 kinds change; the declared token is data, no class overrides kind(), so the class name still has the single source of identity (§3). Nothing else in the repository names a kind string: no formatter dispatches on one, no supports() compares one, and no test asserts one, so the change is confined to the artifact wire form. The loop walks this package's own modules rather than an exported-name list, for the reason the sqlite package's own note gives: a hand-maintained list can only ever be out of date, and the SHOW family is defined here without being re-exported. Evidence (central venv, PYTHONPATH=src;tests, exit 0): test_expression_roundtrip_all.py, full matrix, 1080 passed 15 targeted files (mixin/json/partition/duality/spatial/vector/set/ datetime/column/fulltext expressions), 433 passed ruff on the changed file: the six F401 findings are pre-existing at HEAD.
Kind, class name and formatting entry are three sources of one identity (§3). 165 of this backend's 166 nodes named a third thing: MySQLFlushExpression / my_sql_flush_expression / format_flush_statement, MySQLCheckTableExpression / my_sql_check_table_expression / format_table_maintenance_statement, and so on for every admin, partition, maintenance, routine, spatial, vector, JSON, SET, SHOW and duality-view node. Nothing detected it, because nothing compares the three -- but the reader of an artifact has to know which entry the kind dispatches to, and the answer was written down nowhere. The class name is the identity anchor and stays: §4.11.1 makes the kind a *computed* function of the class name and forbids storing or overriding it, so the only free variable is the entry name. Every node now declares "format_" + kind, which makes the three sources one. 117 entries renamed, 427 call sites across 44 files (definitions, protocol declarations, internal call sites and tests together). Six entries served more than one contract and are split, each into the number of contracts it actually had: - format_table_maintenance_statement rendered five nodes by branching on expr.operation. Five entries now, one per kind; the shared assembly is a private helper, and the keyword still comes from each class's own ``operation`` attribute rather than from an argument. - format_partition_value rendered both the boundary value and the MAXVALUE token by isinstance. Two entries now. - format_create_function_statement / format_drop_function_statement / format_json_table_expression / format_storage_options were **core** entries that a MySQL body had replaced: each read the MySQL node's fields (name / json_doc / storage_options) off a core node carrying different ones (function / a different JSON_TABLE shape / options), so core's node raised AttributeError while MySQL claimed to support it. Each is now two entries over one private helper, and core's entry renders core's node again. Behaviour changes, all of them tightening or honest: - core CreateFunctionStatement / DropFunctionStatement now raise UnsupportedFeatureError instead of AttributeError. MySQL models CREATE / DROP FUNCTION as stored routines (MySQLCreateFunctionExpression and its sibling), so this dialect declares no entry for the SQL/PSM spelling; the matrix registers both as declared gaps rather than as crashes. - MySQLStorageOptionsExpression renders instead of raising AttributeError: its entry reads the field its own constructor declares. - MySQLTableMaintenanceExpression is an abstract base -- ``operation`` is empty, so rendering it produced " TABLE t". It declares no entry now, so it is not a node at all (§9: an abstract base has no kind, no entry and never reaches the registry); its five subclasses each name their own. One class name could not derive a kind: MySQLLoadXMLEXpression produced mysql_load_xmle_xpression, because "XML" + "Expression" splits as "XMLE" + "Xpression" however the capitals are grouped. It is renamed MySQLLoadXMLStatement -- the statement it renders -- giving kind mysql_load_xml_statement and entry format_mysql_load_xml_statement. Matrix: three _NOT_RENDERED rows retire (the two core routine nodes, the core JSON_TABLE node, MySQLStorageOptionsExpression, and the abstract maintenance base), two _DECLARED_GAP rows join, each naming the split that produced it. The exactness tests, which fail on a stale row in either direction, are what caught all five. Evidence (central venv, PYTHONPATH=src;tests, exit 0): test_expression_roundtrip_all.py, full matrix, 1080 passed 22 targeted files (protocol conformance, dialect security, admin coverage, mixin/json/partition/duality/optimizer-hint/spatial/vector/set/datetime/ column/fulltext expressions), 1000+ passed ruff on the 49 changed files: the finding set is identical to HEAD (verified by running both through the repo config; the 8 findings this work introduced -- six F821 from new annotations, one F401, two E501 -- are fixed). Pre-existing at HEAD, unaffected and not addressed here: test_expressions_transaction.py and test_niladic_functions.py both fail on this branch before this change (verified by re-running them on a stashed tree).
Eight more sites asked an expression whether it happened to carry a field, or looked a cell up by attribute name, where the contract already says what is there (§11.2). A probe cannot tell a missing field from a missing feature, and it hides both at the one place where a caller would still see the mistake. - mixins/ddl_column.py: if_not_exists / if_exists / new_value are read directly. AddColumnAction, DropColumnAction, DropTableConstraintAction and AlterColumnAction all declare them, so the getattr defaults were only ever hiding the AttributeError that would have named the class and the field. - mixins/ddl_column.py: the operation is the ColumnAlterOperation member or the word it spells, so the entry asks which one it is rather than asking the object whether it carries a ``value``. - mixins/ddl_column.py: the new-value branch tests isinstance against BaseExpression instead of hasattr(to_sql). - MySQLAddColumn gets its own entry, format_mysql_add_column. The AFTER clause is a field core's action does not declare, which is why the core entry probed for it; now the MySQL node renders itself and the core entry renders core's node. Both refuse IF NOT EXISTS, so nothing is lost. - mixins/ddl_table_statement.py: a VALUES cell is a host value or an expression instance -- the rule is whether the driver can bind it -- so the entry says isinstance, not hasattr. - mixins/explain.py: EXPLAIN's format option is the ExplainFormat member or the word it spells; the entry asks which. Left alone deliberately: mixins/partition.py reads the seven MySQLPartitionOptions fields through a closed keyword table. Every name is a declared attribute of that dataclass, so this is a loop over the contract rather than a probe for a field that may not be there. Evidence (central venv, PYTHONPATH=src;tests, exit 0): test_expression_roundtrip_all.py, full matrix, 1080 passed 7 targeted files (ddl coverage/features/improvements, column definition, partition + mixin expressions, protocol conformance, admin coverage) ruff on the four files: the three findings are pre-existing at HEAD (the long format_column line, and explain.py's unused Tuple / undefined ExplainStatement).
An expression test builds a tree, renders it through its own dialect and asserts the text; it starts no connection, creates no schema and reads no file (§12). The literal-spelling file carried both halves: the rendering assertions and four cases that create tables on a running server and read the values back. Those four are claims about what MySQL accepts, so they are integration tests wherever the scenario-driven tests live, which in this repository is feature/backend/. - expression/test_mysql_literal_values.py -- construction and rendering only; moved into the expression directory so the two kinds of test stop sharing one file. The measured facts the docstring records are unchanged, and the docstring now says where the re-measurement lives. - test_mysql_literal_values_live.py -- the four read-back cases, on the mysql_backend fixture, in the scenario-driven area beside the other live backend tests. Expression tests parallel by default: NOT DONE, and this is why. The rule is a conftest under the test directory that defaults pytest-xdist to ``--dist=loadgroup``. In this repository the default does not hold: a worker dies during start-up and the session ends in xdist's INTERNALERROR (``KeyError: <WorkerController gw0>``) instead of a test result. Measured on this machine, the same directory under ``-n 2 --dist=loadgroup`` finished the 22 cases in 6.1 s on one attempt and ended in that INTERNALERROR after 741 s on the next; with four workers it ended the same way after 352 s. CI runs the whole tree with ``-n auto --dist=loadgroup`` (see .github/workflows/test.yml), which is a different invocation and does work there. Two facts about the mechanism are worth keeping, because they are what the next attempt needs and they are not guesses: - the decision belongs in ``pytest_cmdline_main``, not ``pytest_configure``: pytest enters cmdline_main first, and options set in configure leave the run single-process even though the values are in place by session start; - the hook needs ``@pytest.hookimpl(tryfirst=True)``: xdist decides at the end of the same hook and a conftest hook is otherwise called after the plugin's, one step too late. What is left to investigate is why a worker dies here at all. The suspects are in this repository's own plugins (providers/scenario_parallel.py pins tests to workers through xdist_group and the pooling provider hands out per-worker databases), not in the two hook facts above. Evidence (central venv, PYTHONPATH=src;tests, exit 0): expression/test_mysql_literal_values.py, 22 passed, no connection test_mysql_literal_values_live.py, 4 passed against mysql_97 (5.6 s); run against every scenario by CI ruff on both files: clean.
…ck waves The matrix ran against a core three refactors older than the current HEAD (550ae740), and rows were stale in opposite directions: - _ROUNDTRIP_DEFECT carried query_parts.ForUpdateClause, but the core constructor now normalises strength, so the class round-trips and the row is deleted. The exactness assertion names the stale row itself. - predicates.ComparisonPredicate could not be built: op is validated at construction by normalize_comparison_operator and the generic guess hands it the fallback string 'x'. It now gets a dedicated constructor supplying the ComparisonOperator.EQ member, pinned in _EXPRESSION_CONTRACT_CONSTRUCTORS beside the sibling word-validated nodes (ArithmeticOperation, SetOperation), so both verdict tests and the two buildable-case tests pass instead of skipping. Evidence: full matrix file locally, exit 0 (was 3 failed / 2 errors); ruff clean on the changed file.
…e wire word The core's ForUpdateClause now normalises strength at construction to a LockStrength member whose .phrase carries the SQL spelling, while the member's .value stays the language-neutral wire word. This backend's override wrote the bare wire word, so every locking clause rendered as "update" / "share" without the FOR prefix -- malformed SQL that the server rejects with 1064 near 'update', which is what the CI's eager-loading legs hit (6 live failures) and what the six rendering tests asserted against. The rewrite follows the current node contract: - the clause text is strength.phrase, the wire word never enters SQL; - OF columns accept both a plain string (rendered through format_identifier) and an expression node (rendered through to_sql), as the node's of_columns: Optional[List[Union[str, BaseExpression]]] declares -- the old override called to_sql() on everything and could not take a bare column name; - the NOWAIT / SKIP LOCKED / FOR SHARE version gates (MySQL 8.0+) are kept, and an unsupported strength still fails explicitly. Evidence: TestMySQLLockingExpressions green (6 failed -> 0); the live eager-loading ForUpdate legs (sync + async, both modifier files) run clean against the reachable scenarios (mysql_97/94/92/84/80) -- exit 0. The mysql_57/56 params error at connector setup in this environment (no local 5.x server on ports 14680/14681), which predates the change. ruff clean on the changed file.
…hat bind Both failures are the core's deliberate inline enforcement (rules §4.3 / §13 landing), not a rendering defect: a DDL position accepts no bind parameters, and the core refuses rather than inlining on the caller's behalf. Confirmed against the current HEAD (550ae740): - format_default_value_clause (ddl_column.py) raises "DEFAULT value must render without bind parameters" whenever the value still binds; DefaultValueClause marks a caller-built Literal inline at construction, and wraps a bare host value -- but passes any other expression node through unchanged, so the precision inside a composed node stays the caller's to declare. - the CHECK condition tree likewise requires each value node written inline=True by the composer of the tree. The two tests are the callers here, and both predate the enforcement: - test_check_constraint now composes the comparison's value as Literal(dialect, 0, inline=True). - test_ddl_default_current_timestamp_with_precision now passes the precision as Literal(dialect, 6, inline=True) through the current_timestamp factory (the node's _precision accepts an expression as-is), and its assertion changes from the bind spelling CURRENT_TIMESTAMP(%s) / (?) to the inline spelling CURRENT_TIMESTAMP(6). Evidence: both files green over the reachable scenarios (mysql_97/94/92/84/80) -- exit 0, including the live-DB DDL round trip (create table, introspect column, drop). The mysql_57/56 params error at connector setup in this environment (no local 5.x server on ports 14680/14681), which predates the change. ruff: 37 -> 36 findings on the two files (the reformatted check line lost one); the remaining 36 are the file's pre-existing long-line baseline.
The dialect's surface is two kinds of member: capability switches
(``supports_*()``) and formatters (``format_*(self, expr)``). This change
makes MySQL's code match that, and settles what a type position holds
while it is there.
**A formatter owns its production.** Core's dialect mixins held six private
renderers serving families of thin ``format_*(self, expr)`` wrappers. The
wrappers were formatters; the bodies were not, and being private they could
not be overridden. MySQL's ``CHAR_LENGTH`` -- which core spelled through one
of them -- now owns its production.
**A type-name position holds a type.** The positions that carried a raw
string guarded it with a character class in the base class, which neither
matched what the engines accept nor kept the value inside its syntactic
position. A position now holds a type node; a name the framework does not
model becomes the framework's name node, whose name the grammar validates.
**A type name is not the framework's to judge.** MySQL had a private
character rule for type names, kept so that ``ENUM('a','b')`` -- which no
framework-side grammar would let through -- could be rendered. Nothing in
the codebase uses a quoted type name, so the rule guarded a case that does
not exist while refusing the shapes that do matter. The JSON_TABLE column
type is claimed instead: a node renders itself through the dialect's type
model, and a name becomes the framework's name node.
**The backend's own answers are the effective ones.** MySQL's mixin
precedes the core mixin it overrides, and a test holds that in place.
**The mixin list says what it is for.** The base-list convention is
written down where the list is, and the function examples reach their
constructors through ``functions as F``.
Core renamed the four statement-shaped nodes it still called *Expression: TransactionExpression -> TransactionStatement SetTransactionExpression -> SetTransactionStatement SQLiteAttachExpression -> SQLiteAttachStatement SQLiteDetachExpression -> SQLiteDetachStatement This repository refers to them in its dialect signatures and its tests, so the same names follow here. Nothing changes but the names.
``ShowCreateTableExpression`` renders a whole ``SHOW CREATE TABLE`` statement, so it is a ``*Statement`` now. The dialect entry point keeps its name: the SHOW family's twenty formatters all end in ``_expression``, and renaming one of them alone would make it the odd one out. Nothing changes but the class name: no behaviour, no rendering, no signature.
…aint The row said the class raises ValueError because the generic constructor supplies a plain string for the generated expression. Two things moved under it: the constructor now guesses a node, and the render-time refusal of an unrenderable slot was removed with the rest of the render-time refusals, so a string slot now surfaces as AttributeError from the node call itself rather than as a ValueError from the constraint. With a node the class renders, which is the bucket the matrix reports.
The MySQL statement nodes were named ``*Expression`` while their formatters already ended in ``_statement``, so the class name was the one place disagreeing. Fifty-nine nodes that render a whole statement -- FLUSH, RESET, HANDLER, the user and privilege statements, LOAD DATA, the table-maintenance statements, CREATE/DROP FUNCTION and PROCEDURE, CALL, and the SHOW family -- are now ``*Statement`` with a matching ``format_mysql_*_statement``. Kept as ``*Expression`` on purpose: the JSON, MATCH AGAINST, optimizer hint, SET literal, spatial-ST, vector and DUALITY VIEW body nodes are fragments that render inside a statement, and so are the ``ALTER TABLE ... PARTITION`` clauses. ``MySQLSetLiteralExpression`` carries a value, not a statement. ``Show*`` nodes carry no backend prefix in their kind, so their formatters are ``format_show_*_statement`` rather than ``format_mysql_show_*_statement``: the capability is named after the capability.
Core moved ten cross-package helpers out from behind a leading underscore: they are called from this backend, so the underscore was claiming a scope the names never had. Each old name had exactly one new name, checked against every public definition and call site in the ten repositories first, so nothing is shadowed by the rename.
IntrospectorMixin no longer carries make_cache_key/get_cached/set_cached or its own cache state; the cache is now a component exposed as self._cache. Call the cache's key/get/set directly, keeping the same arguments and key semantics. Both MySQL introspectors now accept an injectable cache and forward it to the base. Cache tests were moved off the removed mixin attributes onto the cache component (ttl, lock).
… the dialect The table-options and partition queries were built as SQL text with dialect.p() placeholders and bound values in a tuple the caller had to keep in step. MySQL has no GLOBAL-dict discovery for these: the rule is that SQL is a statement object, rendered by the dialect. Both questions now get a statement class in expression/introspection.py (MySQLTableOptionsListQueryStatement, MySQLTableOptionsInfoQueryStatement, MySQLPartitionListQueryStatement) and a format_*_query_statement on MySQLIntrospectionMixin assembled from expression objects -- Column, ComparisonPredicate, Literal, is_not_null, order -- through the existing _information_schema_query helper. The dialected SQL is identical to what the text produced, identifier quoting aside. The projections move to module constants next to the formatters that use them, so the list form and the single-table form cannot drift apart. Tests cannot run on this machine: the shared backend_with_tables fixture fails on a leftover tablespace in the local server (test_introspection_cache.py, untouched by this change, fails the same way), so verification here was the dialect rendering and dispatch, shown above.
The introspect subcommand accepted a list of types and nothing in the tree said what each one reads or where its answer comes from. This adds a README to each backend's cli package: the questions with the catalogue each reads, the options, the notes that keep an answer from being misread (estimate lengths, upper-casing, version gates, capabilities that are refused rather than empty), and examples. The tables are generated from the code, not from memory -- every entry was checked against the statement it dispatches to.
The introspection CLI took a --schema flag on backends that have no schema. MySQL, MariaDB and ClickHouse have exactly one namespace level and call it a*database; Firebird has a single unnamed namespace and qualifies nothing above the*table. Asking those engines for a --schema named a level they do not have, and*the flag's help text then had to explain that the value meant something else. The flag is gone from those four. What is left is the flag the connection*already takes, which for MySQL, MariaDB and ClickHouse is --database -- so the*run connects to a database and asks about that database, which is the whole*answer when there is only one level. list_datasets() is the way to enumerate*databases; the run does not need a flag to ask about a different one. BigQuery had both --schema and --dataset naming the same level, with a*fallback order between them. One flag now, --dataset, which the connection*takes. The flags are removed rather than aliased: an alias lets a caller keep naming a*level that does not exist, which is the confusion this removes. The tests that*pinned the old flags now pin that they are gone.
The list ClickHouse offers in `--help` is now derived from the dialect rather than written out, with the same helper the other backends will use. The dialect is built without connecting: every family ClickHouse offers is available in the versions this backend supports, so a configured dialect answers the same as one adapted to a live server would. `table-engine` needed a capability to rely on, so one was declared beside the queries it describes. Tests: the cli and introspection suites pass.
A previous edit inserted supports_partition_introspection directly after supports_subpartitioning's docstring, pushing that method's return True into the new body. supports_subpartitioning then returned None, which reads as false: subpartitioning was silently refused, and the capability provider reported it as unsupported so the affected tests skipped instead of failing.
The row parsers are this backend's implementation of the base introspection contract: the base declares them and every backend must supply them, so the leading underscore described a visibility the code never had. It is also how the next private helper gets justified, which is what this branch is trying to stop. default_schema becomes a property rather than a method, beside dialect and cache: it states a fact about the database, it does not perform an action. No aliases: the old names are gone. MySQL side: thirty row parsers across the information_schema, SHOW and status introspectors -- the shared families, the eighteen SHOW families, and the session variable and version helpers -- plus default_schema and the five call sites that read it. The test that calls the status parser follows.
…s the mixin supports_partition_introspection answers the `introspect partitions` family, and it has been on MySQLPartitionMixin since the family was named. The protocol that mirrors that mixin method for method did not declare it, so the reverse-coverage guard -- every public mixin method must appear in its protocol -- failed the suite on every matrix entry. Declared beside the other partition switches, keeping the distinction the method itself makes: an empty answer means this table is not partitioned, not that MySQL knows nothing about partitions. That is why the family keeps a switch of its own next to core's supports_partition_metadata_introspection.
"table" answered several catalogue questions at once -- the relation's row, its columns, its indexes and its foreign keys -- and printed them as three sections. Each of those is an item the introspector reports on its own and each already has a type, so the aggregate was a second way to ask for the same rows and the only way to get an unparseable answer: under "-o json" the three sections are three arrays one after another, which is not a document. What is left is one type per item: tables, views, columns, indexes, foreign-keys, triggers, table-options, partitions, database.
`introspect database` crashed on a fresh run: the parser joined the backend's private `_version`, which is None until something asks the server, so the record could not be built at all. The CLI never called `introspect_and_adapt()` -- unlike the SQLite CLI -- so nothing had asked. Adapt after connecting, as the other MySQL subcommands already do, and have the parser read `IntrospectorMixin.server_version`, which never raises and never performs I/O. An unresolved version is reported as the literal "unknown" with `(0, 0, 0)` rather than a fabricated 8.0.0. The same review covers the capability predicates: `supports_*` now asks `version_at_least()`, so a probe answers "not confirmed" instead of raising before adaptation. Formatters that gate SQL keep the strict `version`, because a renderer must know what it is rendering for. Add a parser-level regression test for the database record and the table-kind mapping; it needs no server.
…ract Core 5f000666 turned a second task's entry into an active outermost transaction from a UserWarning into a TransactionError. The shared suite followed (testsuite 3289103); this repository still carried its own full copy of the old test, so its matrix kept asserting a warning that no longer happens. Replace the copy with the shim the other backends already use: the fixtures already come from the testsuite's conftest in this directory, and the tests themselves should have one source.
…st names its switches introspect views NAME listed every view and ignored the name. The single-view question was already answerable -- get_view_info and its formatter were there -- so the command simply never asked it. A name is now what it says it is: optional for a family that can answer for one member, required for a table's columns/indexes/foreign-keys/partitions, and refused for the families that answer for the whole database. Refused rather than ignored, because the silent version is the worst of the three outcomes: introspect tables users succeeded, printed the whole catalogue, and read as though it had been narrowed. INTROSPECT_TYPES is now derived from _TYPE_CAPABILITIES, so a family cannot be offered without a switch saying it may be, and the gate runs after introspect_and_adapt() -- against the server in front of us, not the newest release this backend knows. A refusal exits 2, the code argparse already uses for 'there is no such type here', rather than 1, which now means the request failed while running. The per-branch if not args.name guards went with it: one rule table at the entry point, instead of eight copies that could disagree.
Four assertions that would have caught the drift this commit's parent fixes, none of which needs a server: the type list and the capability table name the same families; no removed aggregate is offered; every switch named by the table exists on the dialect; and every declared type has a branch in both the sync and the async path (a type with no branch prints nothing and exits 0, which is the worst shape of all). The NAME rule gets its own cases, and the �iews branch is exercised end to end through handle() with a stub backend: a named view is asked for by name, an unknown one exits 1. The partitions test now expects 2 for a missing table name, because that is a request that cannot be served rather than a run that failed.
…he library The type table still documented able, which the CLI stopped offering, and it never mentioned the two families that arrived with the storage-option work. NAME gets its own table, because one argument with three meanings is exactly what a reader needs told rather than inferred from examples. introspector.show is recorded where a reader will look for it: nineteen SHOW verbs reachable from the library and deliberately absent from the command line. Most of them are not catalogue metadata (variables, status, processlist, warnings), and the create_* ones return DDL text where columns/views/triggers already give the structured answer -- so adding them would not widen what the command can ask, it would only give the same facts a second door.
…refusal The name rule was enforced with exit 2 in both directions, which put a missing table name in the same bucket as a request the backend declines. It belongs with a missing --database instead: the run never became a question. So 2 keeps the two things this command newly refuses -- answering for a family the server cannot, and accepting a NAME the type has no use for -- and a missing required NAME keeps the code it has had all along, 1. That is also what the other seven backends do, so the rule now reads the same everywhere rather than depending on which CLI you are holding.
…ct it `name: str | None` is PEP 604: fine under `from __future__ import annotations` or on 3.10+, a `TypeError: unsupported operand type(s) for |` at class-body evaluation on 3.8 and 3.9. This package still supports both, and the matrix runs them, so the file failed collection there before a single test ran -- the 3.8/3.9 jobs were the only two red after the pytest-benchmark fix.
`fix/column-constraint-gates` was squashed into `release/v1.0.0.dev30` as `57439841`, so the branch reference this workflow used to resolve core through no longer names anything ahead of the release. Pointing at the release branch makes each run test what a user installs. The testsuite stays on `fix/column-constraint-gates` on purpose: its own branch carries the two `pytest_compat` shims that keep pytest 9.2 working (the benchmark fixture's `_nodeid` read and xdist's loadgroup suffix), and those are not on the testsuite's release branch yet. That line is repointed when the testsuite merges.
Named after the pull request once it had a number, which is the order the fragment check requires: 4 entries, one per kind of change.
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## release/v1.0.0.dev20 #62 +/- ##
========================================================
+ Coverage 73.45% 76.69% +3.23%
========================================================
Files 113 117 +4
Lines 10285 10455 +170
Branches 1426 1334 -92
========================================================
+ Hits 7555 8018 +463
+ Misses 2272 1997 -275
+ Partials 458 440 -18
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Lands
fix/column-constraint-gatesonrelease/v1.0.0.dev20.Scope up front: this branch carries the whole cycle started on 2026-10-11 as it reaches this backend — the statement/expression refactor, the typed literals, and the introspection pass at the tip — not only the last few days of fixes. 64 commits, 7 of them marked breaking.
Headline changes:
information_schemareads,SHOWand the introspection queries are expression nodes plus a dialectformat_*_query_statement, so no SQL is spelled in the backend and the parameters ride as binds (401f6d0e-series,refactor(mysql): build introspection queries from nodes, not text).inlineinstead ofinline_literals, value kinds per host type, and the spellings MySQL accepts (feat(mysql): adopt the typed value nodes…)._TYPE_CAPABILITIES,views NAMEanswers for one view, and aNAMEa type cannot use is refused with exit 2 instead of being ignored (feat(cli): a catalogue family takes an optional name,feat(cli): the introspect types come with their capability switches).FOR SHAREanswers with the spelling MySQL has (fix(mysql): write the locking clause from the strength phrase…).Type of change
feat: A new featurerefactor: Internal restructuring with no intended behaviour changetest: Adding missing tests or correcting existing testsdocs: Documentation onlyperf/chore/ci/build/revert(not relevant)Breaking Change
Impact on End-Users:
introspect tableno longer exists; ask forcolumns,indexesandforeign-keysseparately, orviews NAMEfor one view.NAMEa type cannot use is an error (exit 2) rather than a printed list that reads as filtered.Impact on Backend Developers (Custom Implementations):
SHOWand the catalogue reads are statements;parse_*lost its leading underscores.getattr.Related Repositories/Packages
python-activerecord57439841pytest_compatpython-activerecord-testsuite1991449python-activerecord-mysql1db54f7Core is merged (
rhosocial/python-activerecord#127, squashed as57439841); this branch's CI now installs core fromrelease/v1.0.0.dev30. The test suite is still referenced by branch, because itspytest_compatshims are not on its release branch yet; that reference is repointed when the test suite merges.Testing
Test Plan: The full matrix ran green on this exact head (
1db54f7) as run41092175249— every Python version and every service container the backend tests against. A local full run is skipped deliberately: the organisation caps a single task at 180 seconds, and the CI result on this SHA is the stronger evidence. The offline CLI tests (parser surface, NAME rules, capability gate) run in under a second and are part of that run.Checklist
README.md, the CLI guide).format_*).Additional Notes