●Change8

Migrating to Transformers v4.54.0

Version v4.54.0 introduces 2 breaking changes. This guide details how to update your code.

Released: 7/25/2025

2
Breaking Changes
5
Migration Steps
9
Affected Symbols

⚠️ Check Your Code

If you use any of these symbols, you need to read this guide:

LlamaErnieVoxtralLFM2DeepSeekV2ModernBERTDecoderEoMTDogeMask2Former

Breaking Changes

●Issue #1

The modeling files for standard Llama models have been significantly refactored and reduced in size, focusing only on the core functionality.

✓Solution

If you were relying on specific internal structures or helper functions within the old Llama model files, you may need to update your code to use the new, streamlined implementation. Review any custom Llama modifications against the new, smaller model definition.

●Issue #2

Mixture-of-Experts (MoE) models now leverage efficient 'megablocks' kernels for performance improvements.

✓Solution

If you were using custom attention implementations or specific kernel configurations for MoE models, these might need updating to integrate correctly with the new kernel abstraction layer. Check documentation for configuring custom attention mechanisms.

Migration Steps

  1. 1
    Review the release notes for any specific configuration changes related to Llama models, as the core modeling files have been heavily optimized and reduced.
  2. 2
    If you are using Mixture-of-Experts (MoE) models, be aware that underlying kernel implementations have changed to use 'megablocks'. Verify that any custom attention logic still functions correctly.
  3. 3
    Test your existing workflows involving Llama models and MoE models to ensure performance and output consistency are maintained.
  4. 4
    Explore the new distributed training capabilities if you are working with very large models, as loading times have reportedly improved significantly.
  5. 5
    Install the latest version of the library to gain access to the new models: Ernie 4.5, Voxtral, LFM2, DeepSeek V2, and ModernBERT Decoder models.

Release Summary

This release focuses on reducing library bloat and increasing speed through refactored Llama models, megablocks kernel integration, and native distributed training. It also introduces several new model architectures including Ernie 4.5, Voxtral, DeepSeek-V2, and LFM2.

Need More Details?

View the full release notes and all changes for Transformers v4.54.0.

View Full Changelog