sql: support EXCLUDE CONSTRAINTS for SQL Server sources - #38532
Draft
peterdukelarsen wants to merge 3 commits into
Draft
sql: support EXCLUDE CONSTRAINTS for SQL Server sources#38532peterdukelarsen wants to merge 3 commits into
peterdukelarsen wants to merge 3 commits into
Conversation
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
from
August 27, 2026 18:57
9163653 to
4924d5d
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
2 times, most recently
from
September 8, 2026 15:48
2172d11 to
f2b47a9
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
from
September 8, 2026 15:54
f2b47a9 to
dcb7ce4
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
2 times, most recently
from
September 8, 2026 18:15
1909bff to
7245b7a
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
2 times, most recently
from
September 9, 2026 15:49
c6e1d25 to
9360059
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
from
September 9, 2026 16:43
9360059 to
03db862
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
from
September 9, 2026 16:44
03db862 to
b0c2c4e
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
from
September 9, 2026 16:47
b0c2c4e to
373f5d9
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
2 times, most recently
from
September 9, 2026 16:56
6cba69b to
3fa1364
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
from
September 9, 2026 16:57
3fa1364 to
7f84a76
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
2 times, most recently
from
September 9, 2026 17:28
0dd0039 to
041e852
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
from
September 9, 2026 18:04
041e852 to
1403290
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
4 times, most recently
from
September 9, 2026 21:24
d2aa160 to
aeff1eb
Compare
peterdukelarsen
commented
Sep 9, 2026
| // incompatible upstream DDL via a textual scan of cdc.ddl_history | ||
| // rather than the descriptor, so an upstream ALTER COLUMN | ||
| // (including NOT NULL changes) still errors the table regardless | ||
| // of this option. |
Contributor
Author
There was a problem hiding this comment.
delete this comment
Contributor
Author
There was a problem hiding this comment.
[peter's bot] Deleted. The claim was also no longer true after this round: SQL Server schema changes are now checked structurally against the live upstream schema rather than by scanning the DDL text, so an upstream NOT NULL drop on an EXCLUDE ALL CONSTRAINTS table is a non-event. See the self-review commit.
peterdukelarsen
commented
Sep 9, 2026
| INSERT INTO t27_uniq VALUES (1, 'a'), (2, 'b'); | ||
| EXEC sys.sp_cdc_enable_table @source_schema = 'dbo', @source_name = 't27_uniq', @role_name = 'SA', @supports_net_changes = 0; | ||
|
|
||
| CREATE TABLE t27_all (id INT NOT NULL, email VARCHAR(64) NOT NULL, CONSTRAINT t27_all_pk PRIMARY KEY (id)); |
Contributor
Author
There was a problem hiding this comment.
why is this named t27? where does the 27 come from, should it just say exclude constraints rather than a number?
Contributor
Author
There was a problem hiding this comment.
[peter's bot] The number is this directory's convention: each numbered file prefixes its upstream tables with its own number so files sharing one SQL Server instance never collide (25-constraints.td uses t25_pk, 26-constraint-column-order.td uses t26_pk_order, 15-expose-progress.td uses t15_pk). Kept it for consistency. The MySQL and Postgres variants use descriptive names because those directories are not numbered.
peterdukelarsen
commented
Sep 9, 2026
| # are not recorded as Materialize keys. NOTE: SQL Server does not allow | ||
| # dropping a PRIMARY KEY while CDC is enabled, and upstream ALTER COLUMN still | ||
| # errors the table regardless of these options, so this test covers key | ||
| # recording and nullability only. |
Contributor
Author
There was a problem hiding this comment.
Can we add a test for dropping a unique constraint that's not used for the CDC table? What actually happens? Similar question for dropping a nullability constraint? Why do these fail.
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
2 times, most recently
from
September 10, 2026 19:27
aff1dd2 to
ca504d2
Compare
peterdukelarsen
removed this pull request from stack #38533
September 10, 2026 20:19
peterdukelarsen
changed the base branch from
plarsen/exclude-constraints-mysql
to
plarsen/sqlserver-constraint-validation
September 10, 2026 20:19
peterdukelarsen
added this pull request to stack #38752
September 10, 2026 20:19
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
from
September 10, 2026 20:27
ca504d2 to
607854d
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
2 times, most recently
from
September 11, 2026 18:28
a3b3fea to
abc8643
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
2 times, most recently
from
September 11, 2026 19:35
4b4387a to
c2a763a
Compare
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
from
September 11, 2026 19:58
c2a763a to
e7abdad
Compare
Extend the EXCLUDE CONSTRAINTS / EXCLUDE ALL CONSTRAINTS options on CREATE TABLE .. FROM SOURCE to SQL Server sources. Named PRIMARY KEY and UNIQUE constraints are validated against and pruned from the purified SqlServerTableDesc, so they are not recorded as Materialize relation keys, and EXCLUDE ALL CONSTRAINTS additionally marks every column nullable. NOTE: unlike Postgres and MySQL, SQL Server detects incompatible upstream DDL via a textual scan of cdc.ddl_history rather than the descriptor, so an upstream ALTER COLUMN (including NOT NULL changes) still errors the table regardless of these options, and SQL Server does not allow dropping a PRIMARY KEY while CDC is enabled. For SQL Server the options therefore control only which keys Materialize records. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018yzDA9ywnCLXweNCzqBm12
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
With constraint changes now detected in the CDC poll, show that dropping an excluded UNIQUE constraint upstream is a non-event, while dropping a non-excluded one stalls the table with the shared error and recovery hint. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
peterdukelarsen
force-pushed
the
plarsen/exclude-constraints-sqlserver
branch
from
September 11, 2026 20:04
e7abdad to
4ba24d9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Extends the
EXCLUDE CONSTRAINTS/EXCLUDE ALL CONSTRAINTSoptions to SQL Server sources, completing coverage of the three CDC source types in this stack.Description
Named
PRIMARY KEY/UNIQUEconstraints are validated against and pruned from the purifiedSqlServerTableDesc, so they are not recorded as Materialize relation keys;EXCLUDE ALL CONSTRAINTSadditionally marks every column nullable.SQL Server differs from Postgres and MySQL in two ways worth reviewer attention, both called out in code comments and the docs PR above this one:
cdc.ddl_history, not the descriptor, so an upstreamALTER COLUMN(includingNOT NULLchanges) still errors the table regardless of these options.PRIMARY KEYwhile CDC is enabled, so a planned constraint drop on SQL Server is always aUNIQUEconstraint. Dropping aUNIQUEconstraint upstream is detected by storage/sql-server: catch incompatible constraint changes in the CDC poll #38747, the PR below this one in the stack, so excluding it here makes that drop a non-event, as for Postgres and MySQL.User-visible effect: SQL Server-fed tables can exclude selected (or all) upstream constraints from being recorded as keys, keeping Materialize's key metadata correct across planned upstream constraint drops.
Verification
New
test/sql-server-cdc/27-exclude-constraints.tdcovers key pruning and introspection,EXCLUDE ALL CONSTRAINTSnullability, continued replication, validation errors, empty-list semantics,SHOW CREATE TABLEroundtrip, and the feature-flag gate. It also shows that dropping an excludedUNIQUEconstraint upstream is a non-event, while dropping a non-excluded one stalls the table with the shared error and recovery hint.🤖 Generated with Claude Code
https://claude.ai/code/session_018yzDA9ywnCLXweNCzqBm12