Add PyLucene backend for cuVS-Lucene vector search in cuvs-bench

.pre-commit-config.yaml

@@ -97,7 +97,7 @@ repos:
         name: spotless-fmt
         entry: ci/checks/run_spotless.sh
         pass_filenames: false
-        files: ^java/(cuvs-java|cuvs-lucene)/([^/]+/)?src/.*\.java$
+        files: ^(java/(cuvs-java|cuvs-lucene)/([^/]+/)?src/.*|python/cuvs_bench/(cuvs_bench/backends/_java|tests/java)/.*)\.java$
         exclude: .*/panama/.*
         language: script
         verbose: true

ci/checks/run_spotless.sh

@@ -2,8 +2,8 @@
 # SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
 
-# pre-commit hook wrapper that runs 'spotless:apply' to format the Java sources of every Maven
-# project under java/.
+# pre-commit hook wrapper that runs 'spotless:apply' for the Java sources owned by the Maven
+# projects under java/ and the Java adapters owned by cuvs-bench.
 #
 # Most cuvs contributors do not work on the Java client and do not have Maven installed. For them,
 # running 'pre-commit run --all-files' matches every Java source file in the repo regardless of
@@ -15,11 +15,15 @@ set -euo pipefail
 
 # Keep these in sync with the spotless-fmt hook's 'files'/'exclude' entries in
 # .pre-commit-config.yaml.
-JAVA_SRC_PATTERN='^java/(cuvs-java|cuvs-lucene)/([^/]+/)?src/.*\.java$'
+JAVA_SRC_PATTERN='^(java/(cuvs-java|cuvs-lucene)/([^/]+/)?src/.*|python/cuvs_bench/(cuvs_bench/backends/_java|tests/java)/.*)\.java$'
 JAVA_SRC_EXCLUDE='.*/panama/.*'
 
 java_sources_modified() {
-  git status --porcelain --untracked-files=all -- java/cuvs-java java/cuvs-lucene |
+  git status --porcelain --untracked-files=all -- \
+    java/cuvs-java \
+    java/cuvs-lucene \
+    python/cuvs_bench/cuvs_bench/backends/_java \
+    python/cuvs_bench/tests/java |
     cut -c4- |
     grep -Ev "${JAVA_SRC_EXCLUDE}" |
     grep -Eq "${JAVA_SRC_PATTERN}"
@@ -43,6 +47,7 @@ POMS=(
   java/cuvs-lucene/pom.xml
   java/cuvs-lucene/bench/pom.xml
   java/cuvs-lucene/examples/pom.xml
+  python/cuvs_bench/java-format-pom.xml
 )
 
 for pom in "${POMS[@]}"; do

conda/recipes/cuvs-bench/recipe.yaml

@@ -63,7 +63,12 @@ requirements:
 tests:
   - package_contents:
       files:
+        - site-packages/cuvs_bench/backends/_java/PyLuceneConfiguredHnswCodec.java
+        - site-packages/cuvs_bench/config/algos/pylucene_cuvs_cagra.yaml
+        - site-packages/cuvs_bench/config/algos/pylucene_cuvs_hnsw.yaml
         - site-packages/cuvs_bench/config/datasets/datasets.yaml
+  - script:
+      - python -c "import importlib.metadata as metadata; paths = {path.as_posix() for path in (metadata.files('cuvs-bench') or ())}; required = {'cuvs_bench/backends/_java/PyLuceneConfiguredHnswCodec.java', 'cuvs_bench/config/algos/pylucene_cuvs_cagra.yaml', 'cuvs_bench/config/algos/pylucene_cuvs_hnsw.yaml'}; missing = required - paths; forbidden = sorted(path for path in paths if '/tests/java/' in '/' + path or 'PyLuceneTestSupport' in path or path.endswith('.class')); assert not missing, f'missing cuvs-bench package resources: {sorted(missing)}'; assert not forbidden, f'forbidden cuvs-bench package payload: {forbidden}'"
   - python:
       imports:
         - cuvs_bench

dependencies.yaml

@@ -640,6 +640,7 @@ dependencies:
         packages:
           - click
           - cuvs==26.10.*,>=0.0.0a0
+          - h5py>=3.8.0
           - pandas
           - pyyaml
           - requests

fern/docs.yml

@@ -221,6 +221,8 @@ navigation:
                 path: "./pages/cuvs_bench/synthesize_dataset.md"
               - page: "Backends"
                 path: "./pages/cuvs_bench/pluggable_backend.md"
+              - page: "PyLucene Backend"
+                path: "./pages/cuvs_bench/pylucene_backend.md"
               - page: "cuVS Bench Parameter Tuning Guide"
                 hidden: true
                 path: "./pages/cuvs_bench/param_tuning.md"

fern/pages/cuvs_bench/index.md

@@ -11,3 +11,4 @@ Use these guides to install cuVS Bench, run benchmark workloads, prepare dataset
 - [Usage](/user-guide/benchmarking-guide/cu-vs-bench-tool/usage): configure algorithms, run benchmarks, and inspect results.
 - [Datasets](/user-guide/benchmarking-guide/cu-vs-bench-tool/datasets): prepare benchmark datasets, ground truth, and dataset descriptors.
 - [Backends](/user-guide/benchmarking-guide/cu-vs-bench-tool/backends): understand and extend cuVS Bench backend integrations.
+- [PyLucene Backend](/user-guide/benchmarking-guide/cu-vs-bench-tool/pylucene-backend): prepare the PyLucene runtime and run cuVS-Lucene HNSW and CAGRA benchmarks.

fern/pages/cuvs_bench/install.md

@@ -49,6 +49,12 @@ Exact tags are listed on Docker Hub:
 
 **Note:** GPU containers use the CUDA toolkit inside the container. The host only needs a compatible driver, so CUDA 12 containers can run on systems with CUDA 13.x-capable drivers. GPU access also requires the NVIDIA Docker runtime from the [NVIDIA Container Toolkit](https://github.com/NVIDIA/nvidia-docker).
 
+## PyLucene backend prerequisites
+
+The optional `pylucene` backend requires runtime components that the standard cuVS Bench packages do not install automatically: JDK 22, a source-built PyLucene wrapper for Lucene 10.2.0, and version-matched cuVS Java, cuVS-Lucene, and native artifacts.
+
+See [PyLucene Backend](/user-guide/benchmarking-guide/cu-vs-bench-tool/pylucene-backend) for the pinned dependency build, validation procedure, runtime configuration, and benchmark workflows.
+
 ## Build from Source
 
 Build cuVS Bench from source when you need local benchmark executables that match a development checkout, include custom algorithm targets, or use dependencies that are not available in the pre-built packages.

fern/pages/cuvs_bench/param_tuning.md

@@ -210,6 +210,8 @@ Use FAISS IVF-PQ index on CPU
 
 ## HNSW
 
+The PyLucene accelerated HNSW path has backend-specific parameter mappings and runtime constraints. See [PyLucene Backend](/user-guide/benchmarking-guide/cu-vs-bench-tool/pylucene-backend#configure-hnsw-benchmarks) for its `m`, `ef_construction`, `direct_single_segment`, and `num_candidates` settings.
+
 ### cuvs_hnsw
 
 NVIDIA cuVS HNSW constructs a CAGRA graph on the GPU and converts it to an HNSW index for CPU-based search. The build process uses in-memory CAGRA if the estimated memory requirements fit within available host and GPU memory. If not, it falls back to ACE (Augmented Core Extraction), which enables construction for larger datasets by partitioning them. Explicitly setting ACE parameters also triggers the ACE build path. The ACE-specific parameters listed below are relevant only when ACE is being used.

fern/pages/cuvs_bench/pluggable_backend.md

@@ -194,7 +194,7 @@ class ElasticsearchBackend(BenchmarkBackend):
         dry_run=False,
     ):
         n_queries = dataset.n_queries
-        return SearchResult(
+        return [SearchResult(
             neighbors=np.zeros((n_queries, k), dtype=np.int64),
             distances=np.zeros((n_queries, k), dtype=np.float32),
             search_time_ms=0.0,
@@ -203,7 +203,7 @@ class ElasticsearchBackend(BenchmarkBackend):
             algorithm=self.algo,
             search_params=indexes[0].search_params if indexes else [],
             success=True,
-        )
+        )]
 ```
 
 ```python
@@ -219,9 +219,13 @@ get_registry().register("elasticsearch", ElasticsearchBackend)
 | Component | Description |
 | --- | --- |
 | `ConfigLoader` | Abstract class whose `load(**kwargs)` method returns `(DatasetConfig, List[BenchmarkConfig])`. Register with `register_config_loader(backend_type, loader_class)`. |
-| `BenchmarkBackend` | Abstract class whose `build(...)` method returns `BuildResult` and whose `search(...)` method returns `SearchResult`. Register with `BackendRegistry.register(name, backend_class)`. |
+| `BenchmarkBackend` | Abstract class whose `build(...)` method returns `BuildResult` and whose `search(...)` method returns `List[SearchResult]`. Register with `BackendRegistry.register(name, backend_class)`. |
 | `BackendRegistry` | Singleton registry returned by `get_registry()`. It maps backend type names to backend classes. |
 
+## PyLucene Backend
+
+The built-in `pylucene` loader expands algorithm YAML groups into one Lucene index per selected codec. The backend initializes PyLucene's process-global JVM and resolves the production `Lucene101AcceleratedHNSWCodec` and `CuVS2510GPUSearchCodec` through Lucene's service-provider interface. See [PyLucene Backend](/user-guide/benchmarking-guide/cu-vs-bench-tool/pylucene-backend) for codec behavior, dependency setup, configuration, benchmark parameters, and runtime limits.
+
 ## C++ Backend
 
 The built-in `CppGoogleBenchmarkBackend` uses `backend_type="cpp_gbench"`. Its config loader reads YAML under `config/datasets` and `config/algos`, expands parameter combinations, and validates constraints. Its backend runs the C++ benchmark executables and merges their results.