[python] Read native vector index ranges with bounded concurrency - #9756
Conversation
JingsongLi
left a comment
JingsongLi
left a comment
There was a problem hiding this comment.
Reviewed bd40362. Requirement fit: supported; no actionable implementation finding.
The benefit is on positional reads with remote latency. The shared per-reader semaphore covers both native callbacks and executor workers; seek/read streams remain serialized, output range order is preserved, and failed callbacks wait for submitted reads before returning. Keeping the local default at 1 avoids paying thread overhead on ordinary local reads.
All 88 focused input/native-search/filter tests passed locally. A native IVF-FLAT batch-search smoke test (4,096 × 64, 8 queries, 3 repetitions) preserved IDs, scores, read count and byte count. With injected 2 ms read latency, P50 was about 141 ms for the baseline and 42.5 ms at parallelism 4; actual concurrency stayed at 4. This confirms the adapter benefit under injected latency, not performance against a live object store.
done |
2 participants
Purpose
The native vector index reader submits multiple positional ranges in one callback, but the Python adapter reads them sequentially. Remote read latency therefore accumulates within a callback.
Use a lazy, reusable executor for thread-safe positional reads, preserving range order. The table option
vindex.read.parallelismbounds active reads per reader across native callbacks, including single-range callbacks. It defaults to 4 for remote index paths and 1 for local paths; seek/read-only streams remain serialized. A value of 1 serializes positional reads across callbacks as well.Wait for submitted reads before propagating an I/O failure, and release the executor on reader close or initialization failure. Document the option.
Tests
python -m pytest pypaimon/tests/vindex_input_test.py pypaimon/tests/vindex_vector_index_test.py pypaimon/tests/vector_search_filter_test.py -q: 88 passed.dev/cfg.ini, license-header check, andgit diff --checkpassed.Benchmark
macOS arm64, Python 3.9, paimon-vindex 0.4.0. Random seed 42, 16,384 vectors x 64 dimensions, IVF-FLAT nlist=64, nprobe=16, Top-K=10. Baseline uses the original serial positional-read adapter. Timings include reader open, resident initialization, native search, and close. Construction and result assertions are outside the timed region.
These are local-file measurements with injected per-read latency, not measurements against a live object store. Twenty repetitions per single-query variant; cells show P50 / P95 milliseconds.
For the 32 x 4 KiB range microbenchmark at 2 ms injected delay, P50 was 78.323 ms for the baseline and 78.512 / 39.954 / 20.364 / 10.403 ms for parallelism 1 / 2 / 4 / 8. This isolates range concurrency from search computation. Local-file thread overhead is why local paths default to 1.
With eight queries per batch at 2 ms injected delay, IVF-FLAT P50 fell from 142.437 ms to 41.284 ms at parallelism 4 (10 repetitions). Native IDs/scores and physical read counts/bytes matched across all IVF-FLAT variants: 18 reads and 1,100,414 bytes for one query; 57 reads and 3,660,416 bytes for eight queries.
A DiskANN batch check (eight queries, l_search=100, 2 ms injected delay, five repetitions) measured P50 of 99.307 / 194.721 / 59.499 / 38.694 ms for baseline / 1 / 4 / 8. IDs and scores matched. The baseline already reached eight simultaneous reads through native query workers; the new cap applied across callbacks. DiskANN physical read counts varied with shared-cache scheduling (about 73-77 reads per batch), so its I/O totals are not presented as identical. Comparisons use a serial reference for each latency setting because DiskANN also selects its read plan using observed I/O latency.