Add composable scale calibration to GPTQ

[realAsma]

What does this PR do?

Type of change: new feature

Adds a backward-compatible scale_algorithm sub-config to GPTQ so its weight update can compose with Max, MSE, local Hessian, and activation-error-coupled local Hessian scale calibration. NVFP4 4/6 composes through its existing quantizer config plus MSE scale calibration.

The historical GPTQ behavior remains unchanged when scale_algorithm is omitted: GPTQ still runs Max calibration first. Fused GPTQ now rejects 4/6 explicitly because the current fused kernel assumes 448-normalized scaling while 4/6 uses 256.

This is a stacked draft based on #1976 (asma/activation-quant-aware-scale-setting), which provides local Hessian and the shared scale-calibration dispatcher.

Usage

config["algorithm"] = {
    "method": "gptq",
    "layerwise": True,
    "scale_algorithm": {
        "method": "local_hessian",
        "activation_error_coupling": True,
    },
}

Use {"method": "mse"} for MSE, {"method": "max"} for explicit Max, or combine the existing NVFP4 4/6 quantizer config with the MSE scale algorithm.

Testing

• pre-commit run --files CHANGELOG.rst modelopt/torch/quantization/config.py modelopt/torch/quantization/model_calib.py modelopt_recipes/ptq.md tests/gpu/torch/quantization/test_gptq.py tests/unit/torch/quantization/test_gptq.py
• pytest_pwd tests/unit/torch/quantization/test_gptq.py -q -x — 15 passed
• pytest_pwd tests/unit/torch/quantization/test_lsq.py -q -x — 49 passed
• pytest_pwd tests/unit/torch/quantization/test_config_validation.py -q -x — 79 passed

GPU tests were added but not run locally because every visible GPU had active compute allocations and process ownership could not be established safely.

Before your PR is "Ready for review"

Make sure you read and follow Contributor guidelines and your commits are signed (git commit -s -S).

Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded trust_remote_code=True, torch.load(..., weights_only=False), pickle, etc.).

• Is this change backward compatible?: ✅
• If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A
• Did you write any new necessary tests?: ✅
• Did you update Changelog?: ✅
• Did you get Claude approval on this PR?: N/A — draft; formal PR review not requested yet

Additional Information

The experiment launcher/config follow-up is intentionally deferred until the current GSM8K campaign completes. Layerwise-off ablations are deprioritized behind that sweep.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9e6aac81-5b2f-44c3-97ce-d36fc1aea6b7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch asma/gptq-scale-algorithms

---

_{Comment @coderabbitai help to get the list of available commands.}

[realAsma]

RB: Can you explain to me how exactly four_over_six is implemented in modelopt?

[realAsma]

🤖 Bot comment.

🐝 I’m tracing the four_over_six configuration through calibration, quantization, and packing now.

[realAsma]

🤖 Bot comment.

Blocked by Review Bee’s scope guard: GitHub reports PR #2004 is authored by realAsma, while Review Bee may only handle PRs authored by someone else. No PR changes were made.