Search before asking
Paimon version
master
Compute Engine
Any (primary-key table vector search)
Minimal reproduce step
On a primary-key table with a pk-vector index and freshly-written rows that have not been compacted yet (level-0 APPEND files present in the bucket), run a vector search with vector-index.search-mode set to full or detail.
What doesn't meet your expectations?
Rows written since the last compaction are missing from the result.
full and detail are documented to exact-scan unindexed data files, but PrimaryKeyVectorRead handed the bucket search only the files PrimaryKeyIndexSourcePolicy.shouldRead accepts (complete COMPACT level>0 outputs). Level-0 APPEND files were therefore dropped by every search mode. FAST intentionally ignores unindexed files, but full/detail should scan them.
Anything else?
No response
Are you willing to submit a PR?
Search before asking
Paimon version
master
Compute Engine
Any (primary-key table vector search)
Minimal reproduce step
On a primary-key table with a
pk-vectorindex and freshly-written rows that have not been compacted yet (level-0 APPEND files present in the bucket), run a vector search withvector-index.search-modeset tofullordetail.What doesn't meet your expectations?
Rows written since the last compaction are missing from the result.
fullanddetailare documented to exact-scan unindexed data files, butPrimaryKeyVectorReadhanded the bucket search only the filesPrimaryKeyIndexSourcePolicy.shouldReadaccepts (complete COMPACT level>0 outputs). Level-0 APPEND files were therefore dropped by every search mode.FASTintentionally ignores unindexed files, butfull/detailshould scan them.Anything else?
No response
Are you willing to submit a PR?