Build SQL dialect auto linter / add in github workflows - #263
Open
keenzarate213 wants to merge 30 commits into
Open
Build SQL dialect auto linter / add in github workflows#263keenzarate213 wants to merge 30 commits into
keenzarate213 wants to merge 30 commits into
Conversation
get latest main into feature branch
…erytime we push code in the branch
keenzarate213
marked this pull request as ready for review
August 3, 2026 15:44
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description & motivation
Adds a new CI check (
sql_dialect_linter) that catches Databricks/Snowflake SQLincompatibilities in the PR.
For each dialect, the workflow:
edu_whpackage against a dummy dbtprofile and dummy macro overrides for anything that
would require a database connection during compile (e.g.
is_incremental,get_column_values,union_relations).connection to compile; currently
bld_ef3__student_programs,bld_ef3__student_indicators,bld_ef3__student_assessments_long_results, andcfg_assessment_scores.tpdm_warehouseandfinance_warehouse, which are disabled by default, sothey get checked linted as well.
models and tests.
Also added a check for known Databricks-incompatible function/type usage that
sqlfluff can't catch on its own since it only validates syntax, doesnt detect
a given function or type actually exists. Currently checks for
try_to_date()and castingas time, both valid syntax everywhere but with noDatabricks equivalent. I added a list directly in
sql_dialect_linter.sh(
databricks_incompatible_patterns) so new ones are a one-line addition (we could also maybe store in a yaml?)Versions (sqlfluff, dbt-core, dbt-databricks, dbt-snowflake) are pinned in
.github/workflows/requirements/sql-linter.txt.Also had to add
noqa: PRS(see documntation here) workaround (scoped to a single known macro) for an unsupported sqlfluff parsing gap around Databricks'variant_explodesyntax,and also allow for caching of virtualenv and
dbt_packagesin workflow so dependencies don't getreinstalled/re-downloaded on every push in the branch.
Breaking changes introduced by this PR:
None, this is a new CI/tooling, no changes to warehouse models or logic.
PR Merge Priority:
Changes to existing files:
New files created:
.github/scripts/sql_dialect_linter.sh: compiles + lints edu_wh against a dummyDatabricks/Snowflake profile for the given dialect argument.
.github/workflows/sql_dialect_linter.yml: runs the above on PRs for both dialects..github/workflows/requirements/sql-linter.txt: pinned tool versionsTests and QC done:
edu_wh PR Review Checklist:
Make sure the following have been completed before approving this PR:
## New Featuresfor features, etc.