●Change8

Migrating to Transformers v4.53.2-modernbert-decoder-preview

Version v4.53.2-modernbert-decoder-preview introduces 2 breaking changes. This guide details how to update your code.

Released: 7/16/2025

2
Breaking Changes
5
Migration Steps
5
Affected Symbols

⚠️ Check Your Code

If you use any of these symbols, you need to read this guide:

ModernBertForCausalLMModernBertForSequenceClassificationAutoModelForCausalLMAutoModelForSequenceClassificationpipeline

Breaking Changes

●Issue #1

The release notes describe a preview tag (`v4.53.2-modernbert-decoder-preview`) which is not a standard semantic version bump. Migrating to this specific version requires installing directly from the Git tag, which bypasses standard PyPI releases.

✓Solution

If you need the preview features immediately, install using the provided Git command: `pip install git+https://github.com/huggingface/transformers@v4.53.2-modernbert-decoder-preview`. Otherwise, wait for the official minor release `v4.54.0`.

●Issue #2

The usage examples show that the ModernBERT Decoder model can be loaded using `AutoModelForCausalLM` for generation tasks, but also incorrectly suggests loading it via `AutoModelForSequenceClassification`.

✓Solution

For text generation tasks using the decoder model, use `AutoModelForCausalLM`. If you intend to use the decoder for classification (as shown in the example), ensure the model configuration supports it, but typically decoders are used for generation. For standard generation, stick to `AutoModelForCausalLM`.

Migration Steps

  1. 1
    If you require immediate access to the ModernBERT Decoder architecture before the official v4.54.0 release, update your installation by installing directly from the preview tag:
  2. 2
    Run the following command: `pip install git+https://github.com/huggingface/transformers@v4.53.2-modernbert-decoder-preview`
  3. 3
    When using the model for text generation, ensure you are loading it with `AutoModelForCausalLM` (as demonstrated in the usage examples).
  4. 4
    When generating text, ensure you set `pad_token_id=tokenizer.eos_token_id` if you are using batch generation or need to explicitly define padding behavior, as shown in the example.
  5. 5
    If you are relying on standard semantic versioning, plan to upgrade to version `v4.54.0` when it is officially released, which will include this model without requiring Git installation.

Release Summary

This release introduces a preview of the ModernBERT Decoder, a causal language model variant of the ModernBERT architecture designed for autoregressive generation and sequence classification.

Need More Details?

View the full release notes and all changes for Transformers v4.53.2-modernbert-decoder-preview.

View Full Changelog