●Change8

v0.1.48-beta

📦 unslothView on GitHub →
✨ 29 features🐛 29 fixes🔧 6 symbols

Summary

This release significantly enhances Unsloth Studio with new export formats (NVFP4, FP8, imatrix GGUF) and introduces an OpenAI-compatible API serving system with automatic model swapping. Core improvements include substantial speedups for GRPO and MoE training, alongside numerous reliability fixes across installation, RAG, and training workflows.

Migration Steps

  1. If reusing an existing local `llama.cpp` build, use the `--with-llama-cpp-dir` flag during export/build.
  2. API clients needing tool call healing should explicitly opt into the extra nudge retry when necessary.

✨ New Features

  • Unsloth Studio can now export to NVFP4, FP8, and imatrix GGUFs after training.
  • Unsloth Studio can act as a llama-swap API system.
  • Added Japanese and Brazilian Portuguese language support to Unsloth UI.
  • Integrated MLX support.
  • Added safetensors tool calling and healing support.
  • DeepSeek-V4-Flash model is now supported with Thinking toggles and improved chat template.
  • OpenAI-Compatible API Serving supports automatic model swapping based on request (local GGUF loading).
  • API serving: Model swap path is safe by default; unknown model names keep using the current model.
  • API serving: /v1/models now returns clean model IDs and local GGUF catalog, hiding .gguf file paths.
  • API serving: Idle auto-unload feature to free VRAM after inactivity.
  • API serving: Clients can control tool-call healing per request and opt into extra nudge retry for malformed tool markup.
  • Export improvements: Users can select multiple export formats at once.
  • Export improvements: Support for portable FP8/INT8 exports, GGUF LoRA, and source-matched exports.
  • Export improvements: `save_pretrained_merged` now supports compressed FP8/FP4 exports.
  • Export UI now includes imatrix GGUF and compressed export options.
  • Exporting multiple checkpoints now avoids more repeated base-model downloads.
  • FP8, INT8, and GGUF-LoRA exports now respect `trust_remote_code`.
  • GGUF export handles models with missing quantization settings more reliably.
  • RAG attachments can now use whole-document context.
  • File chat now reads more PDFs and Word documents correctly, including right-to-left text, Indic text, and DOCX tables.
  • Local RAG checks are less likely to fail due to proxy settings.
  • The embedding model is now customizable in RAG settings.
  • GRPO training now supports sequence packing by default for old/ref log-probability calculations.
  • GRPO now avoids repeating the same shared prompt across grouped completions.
  • Full fine-tuning now uses the correct precision on V100 and other GPUs without bf16 support.
  • MoE LoRA detection now targets the correct expert MLP layers.
  • Grouped MoE can now turn on automatically for loaded and PEFT models.
  • FP8 quantization now handles odd tensor shapes and scale formats more reliably.
  • Llama 3 RoPE scaling is preserved with Transformers v5.

🐛 Bug Fixes

  • Unsloth core makes GRPO 1.3x faster.
  • Unsloth core provides HTTP fallback for stalled downloads.
  • Unsloth core provides better offline mode.
  • Unsloth core makes MoE training 3 to 5x faster.
  • Fixed silent freezing of training and chat progress during long runs.
  • Compare mode starts more reliably.
  • Switching or cancelling models during loading is less likely to get stuck.
  • Hub browsing recovers better from invalid Hugging Face tokens and shows clearer local file paths.
  • Fix multimodal list content handling in inference text paths.
  • Guard empty CUDA arch detection in the source build.
  • Use EMPTY_LOGITS on the fused-CE not-return_dict path.
  • Fixed GRPOTrainer evaluate() crash without prior training.
  • Fixed misleading 'only for image models' error for Qwen3-VL when torchvision is missing.
  • Fixed FlashAttention fp32 crash with DoRA (use_dora=True).
  • macOS installs handle paths with spaces more reliably and size GGUF context for unified memory.
  • macOS installs work better behind corporate TLS-inspection proxies.
  • Linux and macOS path handling is more reliable when folders contain spaces.
  • Windows ROCm RAG embedding avoids a `torchao` import crash.
  • Windows Unsloth startup handles UTF-8 text more reliably.
  • ROCm-on-WSL now supports discrete Radeon RDNA 3/4 GPUs, not just Strix Halo.
  • Blackwell `sm_100` and `sm_103` data-center GPUs now use the correct `llama.cpp` prebuilt selection.
  • GGUF fit checks now reserve memory for CUDA context and vision/model overhead more accurately.
  • Fixed tensor parallelism for vision and `mmproj` GGUF models.
  • GRPO logit scaling now works correctly with DDP-wrapped models.
  • RL trainers no longer reject valid full fine-tuning precision settings.
  • Unsloth gradient checkpointing is no longer silently disabled by `TrainingArguments`.
  • DDP training no longer crashes from a CPU-resident RoPE buffer.
  • FP8 dequant kernels now use safer offset handling.
  • `fast_generate` now shows a clearer error when vLLM-style inputs are used without fast inference.

Affected Symbols

GRPOsave_pretrained_mergedtrust_remote_codeTrainingArgumentsfast_generatetorchao