[BugFix] Rewrite IS NOT NULL / IS NULL to native exists DSL in v2 filter pushdown

[mengweieric]

Summary

• Rewrites IS NOT NULL / IS NULL pushdown in the v2 FilterQueryBuilder from a serialized compounded script query to native OpenSearch exists DSL.
• Closes tracker row "Pushdown coverage - IS NOT NULL pushes script instead of exists".
• Affects all v2 SQL queries, not just vectorSearch() table-function queries.

DSL shape: before / after

Before (both predicates serialized through the script engine):

{"script": {"script": {"source": "{\"langType\":\"v2\",\"script\":\"is not null(age)\"}", ...}}}

After:

IS NOT NULL col  ->  {"exists": {"field": "col"}}
IS NULL col      ->  {"bool": {"must_not": [{"exists": {"field": "col"}}]}}

Native exists is cheaper to evaluate, supported on AOSS / serverless, and matches what the Calcite PredicateAnalyzer already emits for these predicates.

Design

New ExistsQuery lucene query class in opensearch/.../storage/script/filter/lucene/ overrides the LuceneQuery.canSupport / build pair to accept a single ReferenceExpression argument (the base class contract is binary {ref, literal}). Two instances are registered in FilterQueryBuilder.luceneQueries:

.put(BuiltinFunctionName.IS_NULL.getName(), new ExistsQuery(true /* negated */))
.put(BuiltinFunctionName.IS_NOT_NULL.getName(), new ExistsQuery(false))

Nested-field predicates are explicitly guarded: ExistsQuery.canSupport returns false when the argument is a nested() function. This mirrors the equivalent guard in PredicateAnalyzer ("OpenSearch DSL does not handle IS_NULL / IS_NOT_NULL on nested fields correctly"). When canSupport returns false, the default branch in FilterQueryBuilder.visitFunction falls through to the script query path, preserving prior behavior for that edge case.

Test plan

• ./gradlew spotlessCheck - clean
• ./gradlew :opensearch:test --tests "*FilterQueryBuilderTest*" - green
  • Replaced should_build_script_query_for_unsupported_lucene_query with two focused assertions: should_build_exists_query_for_is_not_null and should_build_must_not_exists_query_for_is_null, pinning the new DSL shape.
• ./gradlew :integ-test:integTest -Dtests.class="*ExistsPushdownIT*" - green
  • New IT ExistsPushdownIT covers both senses via _explain against TEST_INDEX_ACCOUNT, asserting "exists" is present and "script" is absent in the source builder JSON.
• ./gradlew :integ-test:integTest -Dtests.class="*AggregationIT*" - green
  • Regression check: existing where int0 IS NOT NULL and where int0 IS NULL queries continue to produce correct result-level output.

Follow-up: nested unary predicate fallback (commit aba5e89)

Addresses reviewer feedback that ExistsQuery.canSupport() returning false for nested arguments was not sufficient: FilterQueryBuilder.visitFunction() has a second dispatch that checks isNestedPredicate() when canSupport() is false, and the inherited LuceneQuery.isNestedPredicate() returns true for any function whose arg[0] is a nested reference. Unary IS NULL / IS NOT NULL could therefore still route into NestedQuery.buildNested(), which reads arg[1] and would throw.

The follow-up commit overrides ExistsQuery.isNestedPredicate() to return false, forcing the script-query fallback for nested-field predicates. Two new regression unit tests (should_fallback_to_script_for_nested_is_not_null, should_fallback_to_script_for_nested_is_null) lock in the fallback in FilterQueryBuilderTest.

• ./gradlew :opensearch:test --tests "*FilterQueryBuilderTest*" - green (including two new nested fallback tests)
• ./gradlew :integ-test:integTest -Dtests.class="*ExistsPushdownIT*" - green