[codewithdark-git]

Summary

An audit of the whole codebase found ~70 real-world bugs, mostly on GPU and real-model paths that the CPU-only tests never exercised. This PR fixes them and adds a behavioural GPU test suite that runs for free on Kaggle.

Verified on real GPUs: Kaggle 2× Tesla T4 with torch 2.11 / transformers 5.16 / bitsandbytes 0.50 / Python 3.13. Two consecutive full runs gave 23 passed, 1 skipped (ONNX skips because optimum-onnx doesn't support transformers 5 yet). CPU suite: 232 passed. ruff and mypy are clean.

Key fixes

• Export kept the base weights and threw away fine-tuning. 4-bit models are now dequantized in memory, 8-bit dequantization applies the SCB scale, and ONNX/MLX export from a staged copy of the in-memory model.
• Security: trust_remote_code is now off by default in turbo(), model analysis, GGUF loading and the server. The CLI gets a --trust-remote-code flag.
• Loading: errors are re-raised with the real cause instead of blaming the architecture. Also fixed: the bitsandbytes availability check, CPU offload with bitsandbytes, the MoE size estimate, free GPU memory, attn_implementation being set at load time, and the default of 16-bit when there's no GPU.
• Generation: no doubled BOS token, the model's own list of end tokens is respected, stop strings are handled natively, and errors in streaming propagate instead of hanging.
• Fine-tuning: works on transformers 5 (warmup_ratio was removed), extra kwargs override the defaults, LoRA target modules are correct, EOS/pad tokens are handled, the adapter is saved, and the result includes final_loss.
• Server: endpoints no longer block the event loop, model access is serialized with a lock, and streaming follows the OpenAI format.
• GGUF: fixed the converter deadlock and Windows .exe paths. bf16/f32 output types are now real, temp files no longer land in the current directory, and imatrix is supported.
• HQQ: actual half-quadratic zero-point optimization, and layers whose width isn't a multiple of the group size are padded correctly.
• Hub/registry: private=True is respected, existing model cards are preserved, and the registry works with huggingface_hub 1.x.
• Packaging: the ONNX extra now uses optimum-onnx, transformers>=4.46, the git-URL extra is removed, and the version-aware dtype/torch_dtype helper silences the transformers 5 deprecation.

GPU testing

• tests/gpu/: an end-to-end suite on a real model that checks behaviour. It skips itself when there's no CUDA.
• python scripts/kaggle_gpu_tests.py [--slow]: packs the working tree, runs it on Kaggle's free GPUs, downloads the results, and warns if Kaggle substituted a different GPU.
• Also: scripts/modal_gpu_tests.py, scripts/run_gpu_tests.sh, and a manual or PR-label-triggered GPU E2E workflow.

Not covered yet

• bf16 and FlashAttention-2 need an Ampere-or-newer GPU; Kaggle runs only T4s on this account.
• MLX export needs a Mac.
• Multi-GPU, CPU offload, Triton kernels and the CLI end to end can all run on T4 but don't have GPU tests yet.

🤖 Generated with Claude Code

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.