[spark][core] Support more expressions in V2 filter pushdown

[JingsongLi]

Purpose

Expand Spark DataSource V2 filter conversion with direct-column support for:

• <> / not-equal predicates
• CHAR_LENGTH
• YEAR, MONTH, DAY, HOUR, MINUTE, and SECOND extraction for DATE and timestamp-without-time-zone fields
• NOT LIKE pushdown for Spark V2 NOT(STARTS_WITH), NOT(ENDS_WITH), and NOT(CONTAINS) predicates, preserving SQL null semantics
• Safe fallback behavior for the new negative string predicates in index and format visitors

When legacy timestamp mapping exposes a Paimon timestamp-without-time-zone as Spark TIMESTAMP, date/time extraction remains in Spark to preserve session-time-zone semantics.

Tests

• paimon-common predicate and JSON serde tests: 141 tests
• paimon-format: 57 ORC and Parquet filter tests
• Spark 3.5 SparkV2FilterConverterTest: 37 tests
• Spark 3.3 compatibility SparkV2FilterConverterTest: 37 tests
• Spark 3.5 PaimonPushDownTest: 22 tests
• SparkFilterConverterTest: 7 tests on Spark 3 / Scala 2.12 and Spark 4 / Scala 2.13
• Spark 4 / Scala 2.13 affected modules compile successfully
• Non-fast Spotless, Checkstyle, and compile verification for affected modules

[leaves12138]

Reviewed 6d14883. Requesting changes for one reproducible wrong-result issue: date/time extraction can incorrectly prune timestamp partitions under legacy Spark timestamp mapping when JVM and session time zones differ. Details and a minimal reproduction are in the inline comment.

Validation on the unmodified production code:

• 298 targeted common predicate/transform/JSON tests passed.
• 57 ORC/Parquet filter tests passed.
• 36 Spark 3.5 V2 filter converter tests passed.
• 36 Spark 3.3 V2 filter converter compatibility tests passed.
• Six additional Spark 3.5 boundary tests: four passed (ordinary-table legacy timestamps, negative string predicates with NULL/OR, and Unicode character length); two failed with actual missing rows in timestamp-partitioned tables, independently for ORC and Parquet.

A temporary local fallback guard made all 42 Spark 3.5 converter/boundary tests pass. That experimental production-code change has been reverted; no fix was pushed. The new negative string predicates' null handling and conservative index/format fallbacks did not reveal another blocker in the exercised paths.

[leaves12138]

Re-reviewed 345260b, including the delta from my previous review. The legacy timestamp partition-pruning issue is fixed, and I found no remaining blocking issue in this update.

The new guard correctly keeps timestamp extraction in Spark when legacy timestamp mapping is enabled, without disabling DATE or normal timestamp-without-time-zone extraction. I reran the original end-to-end reproductions with JVM time zone UTC and Spark session time zone America/Los_Angeles: both ORC and Parquet timestamp-partitioned tables now return the expected row for hour(ts) = 1.

Validation on this head:

• Spark 3.5: all 43 Scala tests passed (37 V2 converter tests plus 6 independent boundary cases, including the original missing-row reproductions).
• Spark 3.3: all 37 Scala V2 converter compatibility tests passed.
• All 7 Java SparkFilterConverterTest tests passed in each run, including the updated supported/unsupported predicate assertions.

The broader GitHub CI is still in progress; this approval reflects code review and the targeted checks above, not a claim that every CI lane has completed. No production-code workaround was applied for this validation, and no code was pushed.