Repository navigation
feat: add OpenSearch SQL dialect - #202
Merged
Merged
Conversation
Adds a seventh dialect targeting the OpenSearch SQL plugin (POST _plugins/_sql) and an index-mapping type provider. Dialect (dialect/opensearch): MySQL-flavoured syntax with CONCAT(), LOCATE(), (col REGEXP 'p') = 1, DATE_ADD/DATE_SUB, ? placeholders, dot-path access to object fields and PartiQL-style EXISTS (SELECT * FROM p.nested AS v WHERE …) for exists() over nested fields. Forms the engine cannot run (array values, list indexing, size() of a list, split/join, bytes literals, filter/map/exists_one, all(), time-zone extraction, in against a non-literal list) are rejected at conversion time. Provider (opensearch): loads GET /<index>/_mapping through *opensearchapi.Client, turning object properties into nested schemas and nested properties into lists of them. Two additions to dialect.Dialect, implemented as pass-through by the six existing dialects: - SupportsSubqueryExpressions() gates filter/map/exists_one, which are written as value-position subqueries. - WriteBooleanFieldPredicate() writes a boolean column used directly as a predicate; OpenSearch needs col = true because its plugin silently matches every document for a bare column and rejects NOT col. Verified against OpenSearch 3.5 (default V2 engine) by a testcontainers integration test that executes the generated SQL. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a seventh dialect, OpenSearch SQL (the OpenSearch SQL plugin,
POST _plugins/_sql), plus an index-mapping type provider, following theadd-sql-dialectpattern.Dialect —
dialect/opensearchCONCAT(),LOCATE(),MOD(),?placeholders,DATE_ADD/DATE_SUB,CAST(x AS LONG|DOUBLE|STRING|BOOLEAN|TIMESTAMP),(DAYOFWEEK(x) + 5) % 7(Monday=0 like the other dialects).matches()→(col REGEXP 'p') = 1— the plugin'sREGEXPreturns an integer, so a bareREGEXPinWHEREfails.objectfields are dot paths (p.dims.width);has()→IS NOT NULL.nestedfields:p.variants.exists(v, v.sku == "A")→EXISTS (SELECT * FROM p.variants AS v WHERE v.sku = 'A')(the plugin's PartiQL form).inagainst a literal list →IN (…); anything else, plus array indexing,size()of a list,split/join, bytes literals,filter/map/exists_one/alland time-zone extraction, is rejected at conversion time withErrUnsupportedDialectFeaturerather than emitting SQL the engine cannot run.IndexAdvisor).Provider —
opensearchNewTypeProviderWithClient(ctx, *opensearchapi.Client)(caller owns the client),LoadTableSchema(ctx, index)viaGET /<index>/_mapping.objectproperties → nested schemas (ObjectType("products.dims")),nestedproperties → lists of nested schemas; multi-fields ignored; index names validated against OpenSearch naming rules.github.com/opensearch-project/opensearch-go/v4(onlyopensearchapiis imported).Two additions to
dialect.Dialect(pass-through in the six existing dialects, no output changes for them)SupportsSubqueryExpressions() bool— gatesfilter/map/exists_one/transformList, which are written as value-position subqueries.WriteArraySubqueryOpenis void, so this is the only way a dialect can refuse them.WriteBooleanFieldPredicate(w, writeField)— a boolean column used directly as a predicate (whole condition,AND/OR/NOToperand,CASE WHENcondition, comprehension predicate). OpenSearch writescol = true: its plugin silently matches every document for a barecoland rejectsNOT col.Verification
make lintclean;go test -short -racegreen for every package (the Docker-onlypgtestcontainer tests were skipped locally).check_testcase_coverage.py opensearch).opensearch_integration_test.gostartsopensearchproject/opensearch:3.5.0via testcontainers, loads the mapping through the provider, and executes the generated SQL for 24 expressions (comparisons, bare and negated booleans,in, regex incl. inline(?i),contains, ternary,EXTRACT, timestamp literal + arithmetic, object dot paths,has(), nestedexists, parameterized?binding). It skips when no container runtime is reachable. Findings from that run shaped the dialect:format=jsonis V1-only (the test uses the default JDBC format), bare booleans are ignored by the engine,REGEXPreturns int, and the nested-query handler rejectsNOT/!=/IN/REGEXPinside the subquery (documented in the README's OpenSearch notes).Docs
README (count, badge, comparison table, provider import, index table, placeholder list, an "OpenSearch SQL notes" section), CLAUDE.md,
docs/parameterized-queries.md, CHANGELOG[Unreleased], and theadd-sql-dialect/add-cel-featureskill references.🤖 Generated with Claude Code