feat(lancedb): enhance index types and CLI integration

[XuQianJin-Stars]

Summary

Rewrites the lancedb client for LanceDB, adding support for IVF_HNSW_SQ and IVF_HNSW_PQ index types, standard filter queries, PyArrow batch insert optimization, COS/GooseFS remote storage, and full CLI integration.

What's added

• Client implementation: vectordb_bench/backend/clients/lancedb/{lancedb.py,config.py,cli.py}
• CLI commands: vectordbbench LanceDB ... / LanceDBAutoIndex / LanceDBIVFPQ / LanceDBIVFHNSWSQ / LanceDBIVFHNSWPQ
• Index type registration: vectordb_bench/backend/clients/api.py — add IVF_HNSW_SQ and IVF_HNSW_PQ to IndexType enum
• Unit tests: tests/test_lancedb_config.py — 10 offline tests covering config defaults, to_dict connection options, index_param / search_param generation, metric parsing, case-config registry, and CLI structure
• Entry point: vectordb_bench/cli/vectordbbench.py — register all new LanceDB CLI commands

Algorithms supported

Algorithm	Build params	Search params
IVF_PQ (default)	num_partitions, num_sub_vectors, nbits, sample_rate, max_iterations	nprobes, refine_factor
AutoIndex	(auto — metric only)	nprobes, refine_factor
IVF_HNSW_SQ	num_partitions, m, ef_construction	ef, nprobes, refine_factor
IVF_HNSW_PQ	num_partitions, num_sub_vectors, m, ef_construction	ef, nprobes, refine_factor
NONE (brute-force)	—	refine_factor

Key improvements

• Filter support: Implement standard prepare_filter() pattern (NumGE / StrEqual)
• Batch insert: Use PyArrow FixedSizeListArray for batch writes instead of per-row dicts
• Multi-process: Custom __deepcopy__ to solve Rust handle serialization issue
• Remote storage: storage_options for COS/GooseFS backends
• Optimize: compact_files + cleanup_old_versions support

Example

vectordbbench LanceDBIVFHNSWSQ --case-type Performance768D1M --k 10 \
    --uri /tmp/lancedb \
    --num-partitions 256 --m 16 --ef-construction 128 \
    --ef 64 --nprobes 20 --refine-factor 25

vectordbbench LanceDBIVFPQ --case-type Performance768D1M --k 10 \
    --uri s3://my-bucket/lancedb \
    --storage-options '{"aws_access_key_id":"...","aws_secret_access_key":"..."}' \
    --num-partitions 256 --num-sub-vectors 48 --nprobes 20

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: XuQianJin-Stars
To complete the pull request process, please assign xuanyang-cn after the PR has been reviewed.
You can assign the PR to them by writing /assign @xuanyang-cn in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[XuQianJin-Stars]

/assign @XuanYang-cn