The native CPU path of bitsandbytes::dequantize_4bit reshapes a 1-D shape argument to (1, shape[0]) before allocating the output (bitsandbytes/backends/cpu/ops.py, if len(shape) == 1:), so a quantize/dequantize round trip of a 1-D tensor comes back 2-D on CPU.
import torch, bitsandbytes.functional as F
A = torch.randn(4096)
q, state = F.quantize_4bit(A, blocksize=64, quant_type="nf4")
F.dequantize_4bit(q, state).shape # torch.Size([1, 4096]) on CPU
Every other implementation returns exactly shape: the CUDA backend allocates torch.empty(shape), the default and MPS backends reshape to shape, and the registered fake kernel returns torch.empty(shape) — so eager CPU also contradicts the op's fake-tensor contract under torch.compile. The smoking gun that this is a bug and not a design choice: the CPU backend is inconsistent with itself — an odd-length 1-D input (n=4095) takes the generic fallback and correctly returns (4095,), while n=4096 returns (1, 4096).
Present since the native CPU 4-bit kernel landed in #1789. Reproduced on macOS arm64 (NEON path), torch 2.13, main @ 95f9af3. Fix is a 2-line deletion — m = prod(shape[:-1]) already evaluates to 1 for 1-D inputs and the kernel writes a flat contiguous buffer; PR incoming.
The native CPU path of
bitsandbytes::dequantize_4bitreshapes a 1-Dshapeargument to(1, shape[0])before allocating the output (bitsandbytes/backends/cpu/ops.py,if len(shape) == 1:), so a quantize/dequantize round trip of a 1-D tensor comes back 2-D on CPU.Every other implementation returns exactly
shape: the CUDA backend allocatestorch.empty(shape), the default and MPS backends reshape toshape, and the registered fake kernel returnstorch.empty(shape)— so eager CPU also contradicts the op's fake-tensor contract under torch.compile. The smoking gun that this is a bug and not a design choice: the CPU backend is inconsistent with itself — an odd-length 1-D input (n=4095) takes the generic fallback and correctly returns(4095,), while n=4096 returns(1, 4096).Present since the native CPU 4-bit kernel landed in #1789. Reproduced on macOS arm64 (NEON path), torch 2.13, main @ 95f9af3. Fix is a 2-line deletion —
m = prod(shape[:-1])already evaluates to 1 for 1-D inputs and the kernel writes a flat contiguous buffer; PR incoming.