(Draft) [Roadmap] DeepSpeed Roadmap Q2 2026

[tohtana]

This is a draft roadmap for DeepSpeed Q2 2026. Feedback is welcome — please leave comments on this issue or join the #2026q2-roadmap channel on the DeepSpeed Slack.

New feature and enhancement

AutoEP support

AutoEP enables Expert Parallelism (EP) for major Mixture-of-Experts (MoE) models out of the box, eliminating the need for users to write model-specific parallelization code. By automatically distributing expert layers across devices, AutoEP allows users to scale MoE training with minimal configuration changes.

A prototype implementation has been validated on 8xH100, achieving ~5x throughput improvement over ZeRO-3 baselines. We will build on this work to extend AutoEP support to production readiness in Q2.

• Convergence validation: Verify training convergence matches non-EP baselines across latest MoE model architectures
• Model coverage: Add support for additional MoE architectures (e.g., Qwen-MoE)
• ZeRO-3 support: Extend AutoEP to work with ZeRO Stage 3
• AutoTP integration: Combine AutoEP with AutoTP for hybrid expert/tensor parallelism
• Benchmarking: Publish throughput, memory, and scaling efficiency numbers across model sizes and GPU counts
• Universal Checkpoint support: Enable saving and resuming from Universal Checkpoints with AutoEP

AutoTP extension

AutoTP was significantly revamped in Q1 (PR #7806), introducing a flexible, configuration-driven API for custom layer partitioning patterns. In Q2, we will extend this foundation to support a broader range of models and scales.

• HuggingFace tp_plan support: Leverage the base_model_tp_plan metadata provided by HuggingFace Transformers models to automatically derive partitioning configurations, enabling out-of-the-box TP for any model that ships with a tp_plan
• Combination with AutoEP: Support parallel folding for hybrid expert/tensor parallelism
• Universal Checkpoint support: Enable saving and resuming from Universal Checkpoints with AutoTP

AutoSP Integration

AutoSP (ICLR 2026) is a compiler-based approach that automatically applies sequence parallelism via DeepSpeed Ulysses, removing the need for manual partitioning of sequence dimensions.

• Initial integration: The initial PR (Merging AutoSP into DeepSpeed #7860) is ready
• Model coverage: Improve coverage for major model families (e.g., Qwen, Llama)
• Multimodal model support: Multimodal models involve significantly longer sequence lengths, making sequence parallelism critical for training efficiency (blog post). However, existing frameworks such as Megatron-LM do not support sequence parallelism for ViT encoders, and manually implementing it requires substantial engineering effort. AutoSP aims to automate this, enabling DeepSpeed Ulysses-based sequence parallelism for multimodal architectures out of the box.

Compiler Integration Enhancement (Optional)

• "DTensor mode" for less graph break and stable graph tracing
• DeepCompile enhancement
  • Support multi-stage optimization passes for PyTorch v2.9+
  • Compiler pass enhancement
    • AutoTP support
    • AutoEP support
  • AMD support

New Accelerator Support (Q2)

• Planning (Scope, target accelerators)

RL training specific Optimization for DeepSpeed-Inference

• Systems Design, prototyping and benchmarking

Stability (Q2)

• Performance regression test
• Enable nightly full test
  • CUDA
  • AMD
  • Intel XPU
  • Intel Gaudi
  • NPU

[nathon-lee]

hi, @tohtana
I saw the plan for RL training optimization in this issue.
Unsloth recently introduced padding-free sequence packing that significantly improves training throughput for variable-length data.
https://unsloth.ai/docs/zh/bo-ke/3x-faster-training-packing

Just wondering — has DeepSpeed considered something similar for RL training?
Or any plans to support / integrate this kind of optimization?

Thanks!

[PKUWZP]

@nathon-lee Thanks for showing the interests and providing some interesting progress on the Unsloth side! Yes, we are in the process of finalizing our Q2 roadmap, which includes the ambitious plan to enable seamless integration of Training/Inference Engines within the RL framework through DeepSpeed. We understand that it's not going to be just the one quarter work, so we will probably start with the training engine optimization. We welcome any contributions and technical discussions (which I will set up shortly after we finalize the roadmap), and you can feel free to join.

[jiosephlee]

Looking forward to Q2! Does the MoE support plans include gpt-oss? On a related note, it would be quite helpful if there is better documentation on how MoE's should be treated differently when configuring deepspeed. Going through the repository has not made this immediately clear. This would be particularly helpful as RL frameworks built upon upstream backends like DeepSpeed e.g. TRL, OpenRLHF and Megatron e.g. Slime, Nemo-RL are quite numerous.

[tohtana]

Hi @jiosephlee!
Sorry for the late response. For MoE support, here is the current (experimental) implementation.

• DeepSpeed fork
• Example

It currently doesn't support gpt-oss, but I think it should be possible to add a pattern for the model. I would appreciate it if you could help us add a preset for gpt-oss and test it.