Skip to content

[Experimental] Implement dataset as a single type+specs - #2532

Draft
achirkin wants to merge 2 commits into
NVIDIA:mainfrom
achirkin:fea-dataset-implementation-template
Draft

[Experimental] Implement dataset as a single type+specs#2532
achirkin wants to merge 2 commits into
NVIDIA:mainfrom
achirkin:fea-dataset-implementation-template

Conversation

@achirkin

Copy link
Copy Markdown
Contributor

An experimental implementation draft of a dataset that keeps its main functionality in a single struct template.
It's parameterized by the value type, index type and the specification. The specification defines all return types and implementations for member functions. For example, the data_view() member returns an appropriate raft::mdspan for all dense datasets, but a sparse matrix view types for COO/CSR (sparse matrices).

The specs included in the example:

  • empty_spec: has nothing but the dimension of the dataset
  • mdarray_spec: implement plain (contiguous) and padded datasets via appropriate mdarrays.
  • sparse_spec: datasets backed by sparse (COO/CSR) matrices from raft
  • sq_spec: scalar compression
  • vpq_spec: PQ compression plus centroids (composed dictionary)

@achirkin achirkin added breaking Introduces a breaking change feature request New feature or request labels Aug 31, 2026
@copy-pr-bot

copy-pr-bot Bot commented Aug 31, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking Introduces a breaking change feature request New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant