Reject lossy and ambiguous numeric inputs - #27
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
This PR closes two input-conversion gaps before NumPy Vector Store's API becomes stable:
within_rowssequence now rejects booleans even when they are mixed with integers. For example,[0, True]raisesTypeErrorinstead of becoming row indexes[0, 1]through NumPy dtype coercion.TypeError. Their imaginary components are no longer silently discarded while converting to the store's realfloat32representation.The README documents both rules. The public roadmap now describes 0.7.0 as the final contract-closure release before 1.0, including the later archive-binding, compatibility-policy, and release-hardening work planned for this cycle.
Why this is needed
Both old behaviors accepted data the API never intended to support, and both could change the meaning of a call without telling the caller.
Python and NumPy deliberately treat booleans as integer-like in some conversion contexts. That meant a selector such as
[0, True]could bypass the documented rule that booleans are not row indexes. The result depended on whether the boolean appeared alone or beside an integer.NumPy also permits a complex array to be converted to a real dtype while discarding its imaginary part. A vector such as
[1 + 2j, 0]could therefore be stored and searched as[1, 0], producing valid-looking results for different data.Rejecting these cases at the public boundary is safer than trying to infer caller intent. Because they tighten accidentally accepted inputs, they belong in the pre-1.0 0.7 minor release rather than a patch release.
Implementation details
Python row-selector sequences are inspected before
np.asarray()chooses a common dtype. Boolean detection shares the existing single pass used to identify nested shapes, so valid Python selectors are not traversed an extra time. Native NumPy integer arrays keep the current vectorized validation and zero-copynp.intppath.Vector and query conversion now inspects the input array before requesting
float32. The same private conversion boundary servesadd()and all three search methods, so complex inputs fail consistently for cosine, dot-product, and Euclidean search. Query validation still runs before empty-result shortcuts, which makes the failure independent of whether the store contains rows.No public method, return type, archive field, dependency, or valid documented input changes.
User impact
Most users do not need to change anything. Existing real-valued vectors, real-valued queries, and integer row selectors behave as before.
Code that passes Python selectors containing booleans must replace them with the intended integer row indexes or fix the upstream filtering logic. Code that passes complex vectors or queries must choose an explicit real-valued representation before calling the store rather than relying on an implicit lossy cast.
Archive format version 1 remains unchanged, and existing archives require no migration.
Documentation and roadmap
Verification
uv lock --checkruff check .ruff format --check .mypy src/ benchmarks/np.intpselector validation near 43 ms; a Python integer list validated near 145 ms on the Apple M4 development machine.Release note
This is the first PR planned for 0.7.0. It intentionally does not change the package version or changelog yet; those will be prepared after the full 0.7 milestone is complete.