Skip to content

[Bug] collect/merge_map still compare some byte[] elements and keys by identity after #9249 #9869

Description

@thswlsqls

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version
master @ bbae07d / 2.2-SNAPSHOT

Compute Engine
Engine-agnostic (core). GEOMETRY is reachable via Spark/Java API (Flink has no GEOMETRY type); the non-distinct retract from any engine emitting UPDATE_BEFORE/DELETE.

Minimal reproduce step
Two leftovers of #9249, which recognised only the BINARY_STRING family as byte[]:

  • FieldCollectAgg.needsEqualiser() (line 90) and BinaryMapKeys.isBinary() (line 42) miss GEOMETRY/GEOGRAPHY, whose values are also byte[] (InternalArray.createElementGetter). Distinct collect over ARRAY<GEOMETRY>, acc [a] + input [a, b]: 3 elements instead of 2. merge_map over MAP<GEOMETRY, INT>, {a:1} + {a:2, b:3}: 3 keys instead of 2.
  • FieldCollectAgg (line 58) builds the equaliser only when distinct is set, so retract() on a non-distinct ARRAY<VARBINARY> falls back to byte[].equals (line 217): [a, b] retract [a] leaves both elements.

What doesn't meet your expectations?
Same-content byte[] values should compare equal in every path, as #9249 established for BINARY/VARBINARY.

Anything else?
#9251 introduced GEOMETRY/GEOGRAPHY 3.5 hours before #9249 merged; #9249 was written against a tree without them and explicitly left the non-distinct retract gap for a follow-up.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions