Skip to content

[Bug] Bitmap file index drops rows for <> / NOT IN on TIMESTAMP(7..9) columns #9863

Description

@thswlsqls

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version
master @ bbae07d

Compute Engine
Any engine writing TIMESTAMP(9) / TIMESTAMP_LTZ(9) columns (Flink, Java API); Spark's microsecond default is unaffected.

Minimal reproduce step

  1. Create a table with f1 TIMESTAMP(9) and file-index.bitmap.columns = f1.
  2. Insert two rows whose f1 differ only below a microsecond, e.g. ...00.000123000 and ...00.000123400.
  3. SELECT * FROM t WHERE f1 <> TIMESTAMP '...00.000123000'.

Expected 1 row, got 0. BitmapFileIndex.getValueMapper() keys TIMESTAMP / TIMESTAMP_LTZ through Timestamp.toMicros(), so both rows share one key; Reader.visitNotIn flips that bitmap and RawFileSplitRead applies it as an exact row selection, dropping the whole bucket.

What doesn't meet your expectations?
<> / NOT IN on an indexed TIMESTAMP(7..9) column must return the same rows as an unindexed scan. #9654 fixed this for BSI (REMAIN for value predicates, exact IS NULL / IS NOT NULL); bitmap has used the same mapper since #3775; range-bitmap rejects precision > 6 at build time.

Anything else?
N/A

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions