v1.3.0 — Real Embeddings: OpenAI-Compatible APIs & Local ONNX Models

Added

• vector.HTTPEmbedder — adapter for any OpenAI-compatible embeddings API (OpenAI, Ollama, LM Studio, Voyage AI, llama.cpp server, vLLM), built on stdlib net/http only. EmbedBatch for one-call corpus indexing, EmbedContext/EmbedBatchContext for cancellation, dims validation with optional inference (dims = 0), bearer/custom-header auth, optional L2 normalization. Strict response validation: index permutation, empty embeddings, and inconsistent dims all error instead of silently corrupting results.
• pkg/onnx — new package running BERT-family transformer models (e.g. sentence-transformers/all-MiniLM-L6-v2) fully in-process via ONNX Runtime: no server, no API key, deterministic. Includes a pure-Go BERT WordPiece tokenizer (lowercase, NFD accent stripping, Unicode format-char removal, punctuation/CJK splitting — no Python/Rust tokenizer). Auto-detects model layout (mean-pools last_hidden_state or uses pre-pooled sentence_embedding); batch calls pad+mask so results match per-text calls.
• cmd/onnx-demo + make demo-onnx — end-to-end semantic search demo; make model downloads MiniLM pinned to a Hugging Face revision with sha256 verification.

Changed

• Dependency policy is now scoped rather than absolute: pkg/vector still imports stdlib only and builds with CGO_ENABLED=0; the third-party deps (onnxruntime_go, golang.org/x/text) are quarantined in pkg/onnx, so consumers who don't import it pay no CGo or dependency cost.
• README/AGENTS.md/CLAUDE.md updated for the new embedders and policy.

Details

// Remote: any OpenAI-compatible endpoint
e := vector.NewHTTPEmbedder("http://localhost:11434/v1", "nomic-embed-text", 0)

// Local: ONNX transformer, fully in-process
e, _ := onnx.New("model.onnx", "vocab.txt")
defer e.Close()

vecs, _ := e.EmbedBatch(docs)              // one call, padded + masked
store := vector.NewStore(vector.CosineDistance)
q, _ := e.Embed("animals that live with people")
store.Search(q, 5)                          // real semantic matches

ONNX setup: brew install onnxruntime (or set ONNXRUNTIME_SHARED_LIBRARY_PATH), then make model && make demo-onnx to see it run. MiniLM embeds at ~1ms/query after a ~200ms model load (384 dims).

Verification: 40+ new tests (race-clean), tokenizer fuzz harness (3.2M execs, 0 failures), end-to-end suite against real MiniLM artifacts, plus an adversarial multi-agent review pass — findings and certificate on #3.

🤖 Generated with Claude Code

go-vector v1.2.1

go-vector v1.2.1

Search performance — backward compatible, zero dependencies.

Performance

• Store.Search: replaced the full reflection-based sort.Slice over all
  n candidates with a bounded top-k max-heap — O(n·log k) selection instead
  of O(n·log n), and O(k) scratch memory instead of O(n).
• Cosine search: the query's self dot-product is now computed once per
  search rather than re-derived for every stored vector.
• Manhattan: branchless float32 abs, dropping the per-element float64
  round-trip (still zero-allocation).
• RandomProjections: preallocate sparse projection rows and tokenizer
  output to cut append churn during Fit/Embed.

Measured (k=10): StoreSearch10000 25.6ms → 22.0ms with scratch memory down
185KB → 62KB; SearchCosine 1.10ms → 0.99ms. Distance results are identical.

v1.2.0 — Embedder State Persistence

Added

• RandomProjections.SaveEmbedder(path) / LoadEmbedder(path) — persist and restore the embedder state (vocabulary + projection matrix) to disk via Gob serialization
• Allows building the vocabulary once, saving it, and reloading on subsequent starts without re-fitting from the corpus

Changed

• Website migrated to shared asset hub (assets.21no.de)
• Improved responsive layout and install commands on mobile
• SEO metadata, OG image, sitemap, robots.txt added to docs page

Details

SaveEmbedder and LoadEmbedder enable a two-phase initialization pattern:

emb := vector.NewRandomProjections(256)
emb.Fit(corpus)
emb.SaveEmbedder("embedder.gob")

// Later, in a new process:
emb, _ := vector.LoadEmbedder("embedder.gob")
vec, _ := emb.Embed("some text")

This avoids re-fitting the vocabulary from scratch on every restart, which is critical for odek's session search where 117+ sessions would otherwise need re-indexing.

v1.1.1 — Text Embedding, Persistence, CLI Demos

go-vector v1.1.1

Text embedding, disk persistence, and CLI demos — zero dependencies.

Text Embedding

• Embedder interface — swap backends without changing search code
• RandomProjections — sparse Johnson-Lindenstrauss projection
  • Builds vocabulary from your corpus (Fit)
  • Tokenizer: split on non-letter/digit, lowercase, min 2 chars
  • Deterministic output (seed 42), L2-normalized

rp := vector.NewRandomProjections(256)
rp.Fit(corpus)
v, _ := rp.Embed("machine learning is fascinating")

Disk Persistence

• Store.Save(path) / Store.Load(path) — gob-encoded binary
• Store.SaveJSON(path) / Store.LoadJSON(path) — human-readable JSON
• Full roundtrip: IDs, vectors, metric all preserved

store.Save("/data/vectors.db")
restored.Load("/data/vectors.db")

CLI Demos

go run ./cmd/go-vector demo     — vector store search
go run ./cmd/go-vector embed    — text embedding + similarity search
go run ./cmd/go-vector persist  — save/load roundtrip

Stats

• 40 tests · 96.8% coverage · 0 dependencies

v1.1.0 — Text Embedding and Disk Persistence

go-vector v1.1.0

Text embedding and disk persistence — still zero dependencies.

Text Embedding

• Embedder interface — swap backends without changing search code
• RandomProjections — sparse Johnson-Lindenstrauss projection
  • Builds vocabulary from your corpus (Fit)
  • Tokenizer: split on non-letter/digit, lowercase, min 2 chars
  • Deterministic output (seed 42), L2-normalized
  • ~10µs per embed at 256 output dims

rp := vector.NewRandomProjections(256)
rp.Fit(corpus)
v, _ := rp.Embed("machine learning is fascinating")

Disk Persistence

• Store.Save(path) / Store.Load(path) — gob-encoded binary
• Store.SaveJSON(path) / Store.LoadJSON(path) — human-readable JSON
• Full roundtrip: IDs, vectors, metric all preserved
• ~60MB for 10K vectors at 1536d, ~200ms save/load

Stats

• 40 tests · 96.8% coverage · 0 dependencies

v1.0.1 — Security, Performance, Docs

go-vector v1.0.1

Security hardening, performance optimizations, and comprehensive documentation.

Security

• Documented float32 overflow limits (MaxSafeDims = 1,000,000)
• Formalized clone-safety guarantees — all store outputs are deep copies
• Thread-safety guidance for concurrent Store access

Performance

• Euclidean: inlined single-pass computation — 0 allocations (was Sub + Norm, now one loop)
• Cosine: single-pass computation — computes dot and both norms in one loop
• Comprehensive benchmark suite: 11 benchmarks at 768–1536 dimensions
• All distance functions verified zero-allocation

Docs

• Rewritten README with performance benchmarks table and security checklist
• GitHub Pages landing page under /docs/ — dark theme, four sections
• Updated AGENTS.md with full conventions and performance rules

Stats

• 27 tests · 99.1% coverage · 0 dependencies

v1.0.0 — First Stable Release

go-vector v1.0.0

Zero-dependency vector similarity library for Go.

What's included

• Vector type — Dot, Norm, Normalize, Add, Sub, Scale, Equal, Clone
• Similarity metrics — Cosine Distance, Euclidean, Manhattan, Dot Product
• Vector Store — in-memory brute-force nearest-neighbor search with top-K
• 99% test coverage — 26 tests, zero dependencies

Quick start

go get github.com/BackendStack21/go-vector

import "github.com/BackendStack21/go-vector/pkg/vector"

store := vector.NewStore(vector.CosineDistance)
store.Add("cat", vector.Vector{1.0, 0.8, 0.1})
results := store.Search(vector.Vector{1.0, 0.9, 0.1}, 3)

License

MIT