A PostgreSQL extension for high-performance approximate nearest neighbor (ANN) search.
- PRISM index (100M+ vectors) — IVF-style, with hierarchical routing
- RaBitQ quantization with theoretical error bounds for two-stage search
- SIMD-optimized distance computation (AVX2, AVX512, NEON)
- Dynamic updates — inserts, updates and deletes, with posting-list splits on demand (see PRISM index maintenance)
- Vector types compatible with pgvector's types
PRISM (Partitioned Routing Index for Similarity Matching) is a PostgreSQL-native index access method that is conceptually similar to pgvector's IVFFlat, but it is built around state-of-the-art quantization, routing, and partitioning techniques. It is implemented in C for high performance and seamless integration with PostgreSQL.
The result: an index that is significantly faster than pgvector's IVFFlat and HNSW, for both querying and index building. In fact, compared to graph-based HNSW, PRISM is faster at equivalent recall on a 100-million-vector dataset, on an instance with a quarter of the memory.
To achieve this performance, PRISM uses a hierarchical centroid tree to partition vectors into clusters, and relies on the PostgreSQL buffer cache to keep that tree warm in memory. Vectors are RaBitQ-quantized and stored in posting lists with a page layout optimized for SIMD distance computation. The RaBitQ encoding carries error bounds that let a search rule out vectors that cannot possibly be nearest neighbors, so far fewer of them need reranking against the full-precision vectors stored in the table.
Query Flow:
1. Traverse centroid tree to find nearest clusters (buffer cache)
2. Scan posting lists for candidates (buffer cache, async I/O)
3. RaBitQ two-stage filtering: estimate → error bound → rerank
4. Full-precision reranking from heap
5. Return k nearest neighbors
For detailed architecture, see docs/architecture.md.
PRISM indexes work with pgvector's vector and halfvec types and
operators, but do not depend on them. The pg_vectorsearch extension ships
its own vec32 and vec16 types, binary compatible with their pgvector
counterparts but named differently so that both extensions can coexist even
with their objects in the same schema. Binary casts between each pair mean an
existing pgvector column can be indexed as-is, with no rewrite and no copy,
and queries already written against pgvector's operators work unchanged with
a PRISM index.
Alpha-level.
See docs/architecture.md for the design.
Requirements:
- PostgreSQL 18+ (with development headers)
- C23 compiler (GCC 13+ or Clang 16+)
- Meson build system
- Optional: CBLAS, for faster RaBitQ encoding. See Development.
# Release build
meson setup builddir --buildtype=release
# Build and install to PostgreSQL
meson install -C builddirSee the development guide for build options.
CREATE EXTENSION pg_vectorsearch;
-- PLAIN keeps the vector inline. EXTERNAL (the default) toasts a value
-- over ~2 kB, and rerank then pays an extra fetch the planner does not cost.
CREATE TABLE items (
id serial PRIMARY KEY,
embedding vec32(3) STORAGE PLAIN
);
-- Load first; the index is sized from the row count at CREATE INDEX.
-- 500 rows is enough for the planner to use the index.
INSERT INTO items (embedding)
SELECT ARRAY[i::real, (i % 7)::real, (i % 5)::real]::vec32
FROM generate_series(1, 500) i;
-- Or vec32_ip_ops / vec32_cosine_ops.
CREATE INDEX ON items USING prism (embedding vec32_l2_ops);
-- 0 = auto (~0.95 recall). Raise for recall, lower for speed.
SET prism.nprobe = 40;
SELECT * FROM items
ORDER BY embedding <-> '[3,1,2]'
LIMIT 5;Defaults are tuned from large-scale benchmarks; most deployments only
ever adjust prism.nprobe.
See the tuning guide for every index parameter and GUC,
their tradeoffs, and when changing them makes sense.
If you already have a table with pgvector's vector or halfvec
columns, you can build a PRISM index on it directly. Nothing about the
table or your queries needs to change.
-- The table you already have, included so the example runs on its own.
CREATE EXTENSION vector;
CREATE TABLE documents (
id bigserial PRIMARY KEY,
body text,
embedding vector(384)
);
CREATE INDEX documents_embedding_hnsw
ON documents USING hnsw (embedding vector_l2_ops);
-- Install pg_vectorsearch next to pgvector and build the index on the
-- existing column. The column is not rewritten or copied.
CREATE EXTENSION pg_vectorsearch;
CREATE INDEX documents_embedding_prism
ON documents USING prism (embedding vec32_l2_ops);
-- Your queries keep working as before. PRISM is normally faster, so
-- the planner should prefer it over the HNSW index while both exist.
-- Verify with an EXPLAIN:
EXPLAIN (COSTS OFF)
SELECT id, body
FROM documents
ORDER BY embedding <-> (SELECT embedding FROM documents WHERE id = 42)
LIMIT 10;
-- Limit
-- InitPlan 1
-- -> Index Scan using documents_pkey on documents documents_1
-- Index Cond: (id = 42)
-- -> Index Scan using documents_embedding_prism on documents
-- Order By: (embedding <-> (InitPlan 1).col1)
-- Once you are happy with it, drop the HNSW index.
DROP INDEX documents_embedding_hnsw;Use vec32_ip_ops or vec32_cosine_ops for an HNSW index built with
vector_ip_ops or vector_cosine_ops, and the vec16_* classes for a
halfvec column.
An insert appends to whichever posting list its vector routes to, so lists
grow as rows arrive and never split on their own. Besides REINDEX, the
prism_rebalance procedure splits oversized lists in place.
See the maintenance guide for how it works and its current
limitations.
- Tuning - Index parameters, GUCs, tradeoffs, defaults
- Maintenance - Rebalancing and splitting posting lists
- Architecture - High-level design and data structures
- SIMD - SIMD build options and distance computation
- RaBitQ - Binary quantization with theoretical error bounds (SIGMOD 2024)
- SPFresh - LIRE protocol for incremental updates (SOSP 2023)
- SPANN - Billion-scale ANN with boundary replication (NeurIPS 2021)
- ScaNN for AlloyDB - Hierarchical clustering (Google)
- pgvector - Vector type and operators for PostgreSQL
- pgvectorscale - DiskANN-based vector index for PostgreSQL