Skip to content

Hybrid Integration for Qwen3Next, Qwen3.5, and Granite MoE models that include Mamba layers#1255

Merged
jlarson4 merged 9 commits intodev-3.x-canaryfrom
feature/hybrid-handling-for-existing-architectures
Apr 15, 2026
Merged

Hybrid Integration for Qwen3Next, Qwen3.5, and Granite MoE models that include Mamba layers#1255
jlarson4 merged 9 commits intodev-3.x-canaryfrom
feature/hybrid-handling-for-existing-architectures

Conversation

@jlarson4
Copy link
Copy Markdown
Collaborator

Description

  • Applied integration work from Prep work for Hybrid model integration #1254 to the existing architectures that use hybrid structure
  • Added a GatedDeltaNet bridge component for handling hybridization in Qwen3Next and Qwen3.5
  • Verified that existing models in these architectures function as expected & pass verification on all of their attention layers, in addition to generating coherent output

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

@jlarson4 jlarson4 merged commit 63e667a into dev-3.x-canary Apr 15, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant