February-2026
Breaking Changes📦 unslothView on GitHub →
⚠ 1 breaking✨ 7 features🐛 33 fixes🔧 16 symbols
Summary
This release introduces major performance enhancements, including 12x faster MoE training and ultra-long context RL capabilities, alongside support for several new state-of-the-art models. Numerous bug fixes address stability across various architectures and dependencies.
⚠️ Breaking Changes
- Gemma-3 now uses Flex-Attention by default, which resolves previous O(N^2) memory usage issues. Users relying on the old memory behavior might see changes, but this change is performance-enhancing and fixes OOM errors.Fix: No action required unless specific O(N^2) behavior was intentionally relied upon; the new O(N) memory usage is superior.
Migration Steps
- Update Unsloth via `pip install --upgrade --force-reinstall --no-cache-dir --no-deps unsloth unsloth_zoo`.
- If PyTorch 2.9 is desired, use: `pip install --upgrade unsloth unsloth_zoo`.
✨ New Features
- Introduction of significantly faster MoE training (12x faster, 35% less VRAM) using new Triton and math kernels.
- Support for training embedding, BERT, and classifier models 1.8-3.3x faster with 20% less VRAM.
- Ultra Long Context Reinforcement Learning (RL) training enabled via new batching algorithms, supporting up to 380K context on a single B200 GPU.
- Added support for running and fine-tuning new models: DeepSeek-OCR 2, GLM-4.7-Flash, and Kimi-2.5.
- Gemma-3 now uses Flex-Attention by default, resulting in O(N) memory usage and >3x faster training.
- Vision fine-tuning now accepts mixed data batches containing only images and text data.
- Improved compatibility with `trl==0.27.1` and `transformers==5.1.0`, increasing notebook coverage from 30% to over 80%.
🐛 Bug Fixes
- Fixed VLM input embeds for gradients.
- Silenced tma fbgemm warnings/output.
- Fixed token login issues related to hf_hub.
- Prevented overwriting of slots.
- Fixed VLM + DDP checkpointing issues.
- Enabled 4-bit quantization support on AMD Radeon GPUs.
- Added missing import of inspect.
- Clarified NotImplementedError for fast_inference when fast_inference=False.
- Fixed NameError in _prepare_dataset by importing psutil.
- Fixed fastrope issue for zero strided tensors.
- Fixed crash when trl.experimental.openenv is unavailable.
- Fixed Boolean value of Tensor ambiguity error in mistral.py.
- Added support for init_lora_weights="corda" in get_peft_model.
- Fixed correctness bugs in rl.py, rl_replacements.py, and vision.py.
- Fixed correctness bugs across multiple model files.
- Fixed 3D tensor support for bitsandbytes 8-bit matmul in the forward pass.
- Fixed weight tying for LoRA embeddings and lm_head.
- Fixed Gemma3 QAT training instability when using the int8-int4 scheme.
- Added helpful error messages for fast_generate when fast_inference=False.
- Made llama.cpp CURL dependency optional when building from source.
- Removed redundant code related to has_block.
- Fixed rl.py issues: buffer reset, safer attribute access, and typo.
- Ensured user quantization_config is respected.
- Fixed vLLM PDL bug specifically on Blackwell GPUs (B200/B100).
- Synchronized chat_template from tokenizer to vLLM.
- Removed unused variable BlockDiagonalCausalMask.
- Replaced GitHub API check with vLLM version check for PDL fix.
- Restored model mode after generate in GRPO (stacked on previous fix).
- Fixed model training state restoration in GRPO trainer.
- Unified Version usage and fixed TRL version handling.
- Disabled stats when modelscope is being used.
- Fixed FBGEMM/CUTLASS errors on SM100 (Blackwell) GPUs.
- Fixed Kaggle telemetry misclassification when COLAB_ keys exist.