v4.51.3-GraniteMoeHybrid-preview
📦 transformersView on GitHub →
✨ 3 features🔧 4 symbols
Summary
This release introduces the GraniteMoeHybrid model architecture, a hybrid design combining state space layers and Mixture-of-Experts (MoE) attention, available as a stable preview ahead of the v4.52.0 minor release.
Migration Steps
- Install the preview version using: pip install git+https://github.com/huggingface/transformers@v4.51.3-GraniteMoeHybrid-preview
✨ New Features
- Added support for the GraniteMoeHybrid model architecture.
- GraniteMoeHybrid architecture features state space layers and MoE attention layers with shared experts.
- Support for models using GraniteMoeHybrid on the Hugging Face Hub (e.g., ibm-granite/granite-4.0-tiny-preview).
🔧 Affected Symbols
GraniteMoeHybridGraniteMoeSharedModelAutoModelForCausalLMAutoTokenizer