Skip to content

Prep work for Hybrid model integration#1254

Merged
jlarson4 merged 2 commits intodev-3.x-canaryfrom
feature/hybrid-model-prep
Apr 15, 2026
Merged

Prep work for Hybrid model integration#1254
jlarson4 merged 2 commits intodev-3.x-canaryfrom
feature/hybrid-model-prep

Conversation

@jlarson4
Copy link
Copy Markdown
Collaborator

Description

  • Update benchmark system to handle models that do not have MLP and/or Attention on every block
  • Significant base Bridge overhaul to add methods for handling hybrid model situations
  • Update component_setup to allow skipping of components when set to optional
  • Created ComponentScores – this allows us to expose composition scores with attached meta data, so we can tell which layer they are coming from – the index is no longer implied
  • generalized_components/base passes through the optional parameter and whitelists it
  • _get_or_create_bias handles hybrid architectures
  • _get_device_dtype handles hybrid architectures
  • weight processing skips layers without attention weights

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

@jlarson4 jlarson4 merged commit e3d451a into dev-3.x-canary Apr 15, 2026
36 of 42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant