December-2025
Breaking Changes📦 unslothView on GitHub →
⚠ 5 breaking✨ 13 features🐛 11 fixes🔧 11 symbols
Summary
This release introduces massive performance gains with 3x faster training via new Triton kernels and enables 500K context length fine-tuning. It also adds support for Transformers v5, preliminary multi-GPU training, and several new model guides.
⚠️ Breaking Changes
- The internal parameter `rope_theta` has been changed to `rope_parameters['rope_theta']`.Fix: Update code referencing `rope_theta` to use `rope_parameters['rope_theta']`.
- The argument `unsloth_tiled_mlp` must now be passed as a `from_pretrained` argument.Fix: Ensure `unsloth_tiled_mlp` is passed correctly during model loading via `from_pretrained`.
- The `load_in_fp8` kwarg is now prevented from reaching the `Qwen3MoeForCausalLM` constructor.Fix: If you were explicitly passing `load_in_fp8` to this constructor, you may need to adjust how you initialize the model, though this change is likely a fix for internal incompatibility.
- The `reload_weights` rpc call has been removed from the GRPO trainer.Fix: No action required unless you were relying on this specific internal RPC call.
- The `include_buffers` argument was removed from `init_empty_weights` in `unsloth_zoo`.Fix: Remove `include_buffers` argument when calling `init_empty_weights`.
Migration Steps
- Update Unsloth and Docker to the latest versions.
- To upgrade Unsloth: `pip install --upgrade --force-reinstall --no-cache-dir --no-deps unsloth unsloth_zoo`
- If you specifically want PyTorch 2.9: `pip install --upgrade unsloth unsloth_zoo`
- If you encounter issues related to `rope_theta`, change usages to `rope_parameters['rope_theta']`.
- Ensure `unsloth_tiled_mlp` is passed as an argument to `from_pretrained`.
✨ New Features
- Introduction of new Triton kernels enabling 3x faster training and 30% less VRAM usage through padding-free and packing techniques.
- Support for training models with 500K context length, enabling reinforcement learning on a single 80GB GPU.
- Ability to fine-tune and deploy LLMs on mobile phones using PyTorch and Unsloth.
- Support for Hugging Face Transformers v5 (not enabled by default).
- Preliminary multi-GPU support via DDP (guide available).
- Support for `rollout_func` enabling multi-turn Reinforcement Learning (RL).
- Support for `vllm>=0.12.0` and efficient GRPO implementation for it.
- Support for `transformers>=5.0.0`.
- Support for SFT sample packing.
- Auto-enabling of padding-free SFT.
- Mistral packing and training on completions only.
- New model support: FunctionGemma, Nemotron 3, Mistral coding & instruct VLMs, GLM-4.6V, Qwen3-Next, Mistral Large 3, FLUX.2-dev.
- Improvements to vision utilities for vision VLM finetuning.
🐛 Bug Fixes
- Fixed Llama tokenizer padding_side when using model.generate in inference mode.
- Fixed TorchAO and QAT not working during saving.
- Fixed DeepSeek OCR finetuning not loading finetuned models.
- Fixed HuggingFace token logins not working for private repos.
- Fixed gradient_checkpointing parameter not being passed to model.for_training().
- Fixed Vllm guided decoding parameters.
- Fixed fbgemm version check.
- Skipped xpu fbgemm fp8 for Intel.
- Fixed Deepseek OCR LoRA Model Load.
- Corrected packed sequence lengths token count in unsloth_zoo.
- Fixed vLLM compatibility issues with LoRA in unsloth_zoo.