Deprecate unused DeepSpeed features

[sfc-gh-truwase]

Summary

Tracking issue for retiring unused / unmaintained DeepSpeed features so the runtime, docs, tests, and packaging stay consistent.

Goal: remove code paths that are no longer supported, then sweep the leftover docs, extras, CODEOWNERS, and tests so they cannot be re-enabled by accident.

PRs

PR	State	Scope
#8490	Merged 2026-09-12	First sweep: compression library, 1-bit Adam/Lamb, 0-1 Adam, MiCS, Nebula (71d316d60, +169 / −8,298)
#8099	Merged 2026-07-01	ZeRO-3 elastic_checkpoint (use Universal Checkpointing). Stage 1/2 unchanged
#8493	Merged 2026-09-14	Sparse Attention
#8504	Merged 2026-09-16	sparse_gradients
#8534	Open	LoCo-Zero++ (zeropp_loco_param)
#8535	Open	Compression leftovers: 1-bit comm backends, MoQ/eigenvalue, fake-quant kernels, docs/CODEOWNERS/issue template

There is no PR for the full follow-up sweep described below (MoQ + 1-bit comm + PLD + Apex AMP as one bundle). #8535 covers the 1-bit comm and MoQ/eigenvalue leftovers. Progressive Layer Dropping and Apex AMP still have no PR.

Coverage statistics

Line counts are from git diff --numstat. Code = runtime/library sources; tests = tests/; docs = docs/ plus CODEOWNERS / issue templates / CI where noted.

Removed in #8490

Feature	Code −	Tests −	Docs −	Net
Compression library	2,430	301	455	−3,166
1-bit / 0-1 Adam optimizers	1,132	1,255	745	−3,131
MiCS ZeRO-3	650	225	34	−909
Nebula checkpoint engine	229	0	0	−229
Runtime wiring (config.py / engine.py / launchers / CODEOWNERS)	264	0	3	−208
Shared docs (config-json, training pages, nav)	0	0	513	−506
Shared tests (config dict, AutoEP MiCS gate, ZeRO)	0	62	0	−19
Helper relocation (recursive_getattr → module_utils)	0	0	0	+39
PR total				+169 / −8,298

Follow-up sweep (no dedicated PR)

Local working-tree estimate vs master when this was written: +104 / −2,814. Combined vs pre-#8490: +253 / −11,092.

Feature	Code −	Tests −	Docs −	Status
1-bit comm backends	717	676	0	Open in #8535
MoQ / eigenvalue	333	18	212	Open in #8535
Progressive Layer Dropping	40	116	155	No PR
Apex AMP / runtime wiring	383	0	0	No PR (engine amp path)
Docs / CI / packaging	24	0	113	Partially in #8535
ZeRO-3 elastic checkpoint	16	—	config-json	Done in #8099
Rejection tests	0	+52	0	Split across follow-up PRs

Leftover configs should fail at parse time (DeepSpeedConfigError or ZeRO-3 ValidationError) instead of silently training without the feature.

Features to deprecate

Training / checkpointing

• Nebula checkpoint engine (deepspeed/nebula, NebulaCheckpointEngine) — #8490
• MiCS ZeRO-3 sharding (MiCS_Init, mics_shard_size, mics_hierarchical_params_gather) — #8490
• ZeRO-3 elastic checkpoint (zero_optimization.elastic_checkpoint with stage 3; use Universal Checkpointing). ZeRO-1/2 elastic checkpoints remain. — #8099

Optimizers

• 1-bit Adam (OneBitAdam) — #8490
• 1-bit Lamb (OneBitLamb) — #8490
• 0/1 Adam (ZeroOneAdam) — #8490

Compression / quantization

• Model compression library (deepspeed.compression, compression scheduler, related tests) — #8490
• Mixture-of-Quantization (MoQ) / quantize_training / eigenvalue scheduling — #8535

Mixed precision

• Apex AMP (amp config). Use fp16, bf16, or torch_autocast. No PR.

Training schedule

• Progressive Layer Dropping (progressive_layer_drop). No PR.

1-bit communication leftovers (optimizers are gone; backends remain)

• Compressed allreduce backends (deepspeed/runtime/comm/{nccl,mpi,compressed}.py, deepspeed/runtime/compression/cupy.py) — #8535
• tests/onebit/ and remaining unit tests for those backends — #8535
• setup.py extras 1bit / 1bit_mpi and requirements/requirements-1bit-mpi.txt — #8535

Docs / repo hygiene

• Docs nav Model Compression (removed in Deprecate unused DeepSpeed features #8490)
• Docs nav Mixture-of-Quantization and Progressive Layer Dropping — MoQ nav in #8535; PLD still open
• .github/ISSUE_TEMPLATE/compression_bug_report.md — #8535
• CODEOWNERS /deepspeed/runtime/fp16/onebit/ (removed in Deprecate unused DeepSpeed features #8490)
• CODEOWNERS /deepspeed/runtime/compression/ — #8535
• Config / tutorial leftovers for MoQ, Apex AMP, PLD (bulk Nebula/one-bit optimizer/MiCS docs already removed in Deprecate unused DeepSpeed features #8490)

GitHub usage consensus (2026-09-14)

Three independent classifications (Opus, Codex, Grok) over the same GitHub pull agreed on the next deprecation set below.

Method

• Issues/PRs: unauthenticated /search/issues on 2026-09-14. Primary column is -org:deepspeedai. Relative score vs "zero_optimization" -org:deepspeedai = 1,362.
• Code: authenticated /search/code on 2026-09-14 via gh. Relative score vs "zero_optimization" -org:deepspeedai = 49,280. Code search over-counts vendored DeepSpeed copies (e.g. DeepSpeed-0.9.5/ inside other repos) and generic identifiers. Treat as ordinal rank, not user counts.
• Not consensus / do not add from this scan: inference v1/v2, Windows, ZeRO offload, pipeline, DeepCompile, Hybrid Engine, autotuning, DataStates, ZenFlow.

Consensus candidates

Feature	Issue query	Issues −org	Rel. issues	Issues in DS repo	Code query	Code −org	Rel. code	Status
Sparse Attention	"sparse_attention"	358 (158 w/ "mode")	0.26 (0.12)	86	"sparse_attention"	27,264	0.55	Done #8493
sparse_gradients	"sparse_gradients"	20	0.015	8	"sparse_gradients"	4,616	0.094	Open #8504
Elastic training	"max_train_batch_size" "micro_batch_sizes"	0	0	64 (word “elasticity”)	same pair	1,900	0.039	No PR
graph_harvesting	"graph_harvesting"	83	0.061	10	"graph_harvesting"	1,052	0.021	No PR (0 tests; config-dump noise)
Legacy curriculum_learning	"curriculum_learning"	185	0.14	27	"curriculum_learning"	15,136	0.31	No PR (homonym; superseded by Data Efficiency)
LoCo-Zero++	"zeropp_loco_param"	37	0.027	3	"zeropp_loco_param"	152	0.003	Open #8534

2-of-3 only (not unanimous; do not treat as consensus)

Data Efficiency / Random-LTD, dump_state (debug flag; "dump_state" code search is 164k unrelated hits), disable_allgather, FusedLion.

Other unused / already-deprecated candidates

• Sparse Attention (deepspeed/ops/sparse_attention, extras_require['sparse_attn']) — #8493
• sparse_gradients (config-json already calls this essentially deprecated) — #8504
• Elastic training (elasticity / max_train_batch_size + micro_batch_sizes)
• graph_harvesting
• Legacy curriculum learning (superseded by Data Efficiency)
• LoCo-Zero++ (zero_optimization.zeropp_loco_param) — #8534

Notes

• Checked items landed in #8490, #8099, or #8493.
• The original follow-up sweep is not a single open PR. #8535 takes 1-bit comm + MoQ leftovers; PLD and Apex AMP still need a PR.
• MiCS removal also dropped the AutoEP ZeRO-3 “no MiCS” compatibility gate and its unit test.
• 1-bit optimizers are gone in Deprecate unused DeepSpeed features #8490; 1-bit communication backends are in Deprecate leftovers of compression library #8535 (coalesced_collectives.py stays for ZeRO-3).
• deepspeed/compression/helper.py remains as a FutureWarning shim for DeepSpeed-Chat after Deprecate unused DeepSpeed features #8490.
• Apex AMP removal is still local / un-PRed; PyTorch torch_autocast is unchanged. Local import apex remains only to detect apex.optimizers.FusedAdam for FP16 optimizer wrapping.
• Stats methodology: Deprecate unused DeepSpeed features #8490 from git diff --numstat 71d316d60^1 71d316d60; follow-up from git diff --numstat origin/master on the working tree at the time; combined vs b726f4edb.

[hwchen2017]

1. Will the Data Efficiency library, including Random-LTD, also be part of the deprecation plan?
2. Windows support currently appears partial. Will this feature continue to be supported?

[sfc-gh-truwase]

@hwchen2017 great call, those two are candidates for deprecation as we lack bandwidth to continue support.

[hwchen2017]

Will the inference code be deprecated as well?

[sfc-gh-truwase]

Not at this time as there seems to be usage.

[sfc-gh-truwase]

GitHub usage consensus (2026-09-14)

Updated the issue body with a GitHub usage consensus section. Three independent classifications (Opus, Codex, Grok) agreed on the next deprecation set (not yet implemented, except Sparse Attention in #8493):

Feature	Issues −org:deepspeedai (rel. vs 1,362 zero_optimization)	Code −org:deepspeedai (rel. vs 49,280)
Sparse Attention	358 / 0.26 (158 / 0.12 with "mode")	27,264 / 0.55 (homonym + vendored trees)
sparse_gradients	20 / 0.015	4,616 / 0.094
Elastic training	0 / 0 on the real config pair	1,900 / 0.039
graph_harvesting	83 / 0.061 (0 tests; config-dump noise)	1,052 / 0.021
Legacy curriculum_learning	185 / 0.14	15,136 / 0.31 (ML homonym)
LoCo-Zero++ (zeropp_loco_param)	37 / 0.027 (0 tests)	152 / 0.003

Code search was authenticated GET /search/code. Counts include vendored DeepSpeed copies inside other repos; use ranks, not absolute users.

Not consensus: inference v1/v2, Windows, ZeRO offload, pipeline, DeepCompile, Hybrid Engine, autotuning, DataStates, ZenFlow.

[sfc-gh-truwase]

PR status (2026-09-16)

There is no PR for the full follow-up sweep (MoQ + 1-bit comm + PLD + Apex AMP as one bundle).

Item	PR
First sweep (compression, 1-bit Adam/Lamb/0-1 Adam, MiCS, Nebula)	Merged #8490
ZeRO-3 elastic_checkpoint	Merged earlier #8099
Sparse Attention	Merged #8493
sparse_gradients	Open #8504
LoCo-Zero++	Open #8534
1-bit comm + MoQ/eigenvalue leftovers + docs/CODEOWNERS	Open #8535
Progressive Layer Dropping	No PR
Apex AMP (amp)	No PR
Elastic training / graph_harvesting / legacy curriculum	No PR

Issue body checklists updated to match.