Skip to content

feat: add OpenSearch SQL dialect - #202

Merged
richardwooding merged 1 commit into
mainfrom
feat/opensearch-dialect
Oct 15, 2026
Merged

richardwooding merged 1 commit into
mainfrom
feat/opensearch-dialect

Conversation

@richardwooding

Copy link
Copy Markdown
Contributor

Summary

Adds a seventh dialect, OpenSearch SQL (the OpenSearch SQL plugin, POST _plugins/_sql), plus an index-mapping type provider, following the add-sql-dialect pattern.

Dialect — dialect/opensearch

  • MySQL-flavoured syntax: CONCAT(), LOCATE(), MOD(), ? placeholders, DATE_ADD/DATE_SUB, CAST(x AS LONG|DOUBLE|STRING|BOOLEAN|TIMESTAMP), (DAYOFWEEK(x) + 5) % 7 (Monday=0 like the other dialects).
  • matches() → (col REGEXP 'p') = 1 — the plugin's REGEXP returns an integer, so a bare REGEXP in WHERE fails.
  • object fields are dot paths (p.dims.width); has() → IS NOT NULL.
  • nested fields: p.variants.exists(v, v.sku == "A") → EXISTS (SELECT * FROM p.variants AS v WHERE v.sku = 'A') (the plugin's PartiQL form).
  • in against a literal list → IN (…); anything else, plus array indexing, size() of a list, split/join, bytes literals, filter/map/exists_one/all and time-zone extraction, is rejected at conversion time with ErrUnsupportedDialectFeature rather than emitting SQL the engine cannot run.
  • Index analysis: not supported (OpenSearch indexes every mapped field itself; no IndexAdvisor).

Provider — opensearch

  • NewTypeProviderWithClient(ctx, *opensearchapi.Client) (caller owns the client), LoadTableSchema(ctx, index) via GET /<index>/_mapping.
  • object properties → nested schemas (ObjectType("products.dims")), nested properties → lists of nested schemas; multi-fields ignored; index names validated against OpenSearch naming rules.
  • New direct dependency: github.com/opensearch-project/opensearch-go/v4 (only opensearchapi is imported).

Two additions to dialect.Dialect (pass-through in the six existing dialects, no output changes for them)

  • SupportsSubqueryExpressions() bool — gates filter/map/exists_one/transformList, which are written as value-position subqueries. WriteArraySubqueryOpen is void, so this is the only way a dialect can refuse them.
  • WriteBooleanFieldPredicate(w, writeField) — a boolean column used directly as a predicate (whole condition, AND/OR/NOT operand, CASE WHEN condition, comprehension predicate). OpenSearch writes col = true: its plugin silently matches every document for a bare col and rejects NOT col.

Verification

  • make lint clean; go test -short -race green for every package (the Docker-only pg testcontainer tests were skipped locally).
  • All 65 shared test cases have an OpenSearch entry (check_testcase_coverage.py opensearch).
  • Live verification: opensearch_integration_test.go starts opensearchproject/opensearch:3.5.0 via testcontainers, loads the mapping through the provider, and executes the generated SQL for 24 expressions (comparisons, bare and negated booleans, in, regex incl. inline (?i), contains, ternary, EXTRACT, timestamp literal + arithmetic, object dot paths, has(), nested exists, parameterized ? binding). It skips when no container runtime is reachable. Findings from that run shaped the dialect: format=json is V1-only (the test uses the default JDBC format), bare booleans are ignored by the engine, REGEXP returns int, and the nested-query handler rejects NOT/!=/IN/REGEXP inside the subquery (documented in the README's OpenSearch notes).

Docs

README (count, badge, comparison table, provider import, index table, placeholder list, an "OpenSearch SQL notes" section), CLAUDE.md, docs/parameterized-queries.md, CHANGELOG [Unreleased], and the add-sql-dialect / add-cel-feature skill references.

🤖 Generated with Claude Code

Adds a seventh dialect targeting the OpenSearch SQL plugin
(POST _plugins/_sql) and an index-mapping type provider.

Dialect (dialect/opensearch): MySQL-flavoured syntax with CONCAT(),
LOCATE(), (col REGEXP 'p') = 1, DATE_ADD/DATE_SUB, ? placeholders,
dot-path access to object fields and PartiQL-style
EXISTS (SELECT * FROM p.nested AS v WHERE …) for exists() over nested
fields. Forms the engine cannot run (array values, list indexing,
size() of a list, split/join, bytes literals, filter/map/exists_one,
all(), time-zone extraction, in against a non-literal list) are
rejected at conversion time.

Provider (opensearch): loads GET /<index>/_mapping through
*opensearchapi.Client, turning object properties into nested schemas
and nested properties into lists of them.

Two additions to dialect.Dialect, implemented as pass-through by the
six existing dialects:
- SupportsSubqueryExpressions() gates filter/map/exists_one, which are
  written as value-position subqueries.
- WriteBooleanFieldPredicate() writes a boolean column used directly as
  a predicate; OpenSearch needs col = true because its plugin silently
  matches every document for a bare column and rejects NOT col.

Verified against OpenSearch 3.5 (default V2 engine) by a testcontainers
integration test that executes the generated SQL.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@richardwooding
richardwooding merged commit ec9d975 into main Oct 15, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant