●Change8

v3.5.0

📦 localaiView on GitHub →
✨ 14 features🐛 5 fixes🔧 4 symbols

Summary

LocalAI 3.5.0 expands backend support significantly with MLX, Purego rewrites for Whisper and Stablediffusion, and introduces an Alpha Launcher for easier management. This release also brings numerous WebUI enhancements and stability fixes across various hardware platforms.

Migration Steps

  1. If using AMD GPUs, ensure your ROCm environment aligns with the updated requirements mentioned in the fix for RX 9060XT errors.
  2. Users on macOS using Docker should no longer face libomp.so linking issues.
  3. If you encounter issues with model loading on macOS, ensure necessary libutf8 libraries are present or update to this version.
  4. Users interested in the new launcher should note it is Alpha and may require workarounds on macOS due to unsigned binaries (see provided discussion link).

✨ New Features

  • Added support for MLX backend, including mlx, mlx-audio, and mlx-vlm.
  • Added support for video generation using the WAN backend via the diffusers backend.
  • Added CPU and MPS versions of the diffusers backend for image generation on CPU or MacOS.
  • WebUI enhancement: Added a button to download model configuration files.
  • WebUI enhancement: Added a manual model refresh button for the on-disk YAML configuration.
  • WebUI enhancement: Streamlined error streaming during SSE events to display errors directly to the user.
  • WebUI enhancement: Added a stop button for running backends.
  • WebUI enhancement: Models can now be imported and edited directly via the WebUI.
  • WebUI enhancement: Displays a list of installed backends.
  • Whisper backend rewritten in Purego with integrated Voice Activity Detection (VAD).
  • Stablediffusion backend converted to Purego.
  • Introduced LocalAI Launcher (Alpha) for simplified installation, startup, updates, and WebUI access on Linux and MacOS.
  • Added a new "system" backend path (LOCALAI_BACKENDS_SYSTEM_PATH) defaulting to /usr/share/localai/backends for read-only backend specification.
  • Implemented automatic synchronization of installed models between LocalAI instances within a federation (limited to gallery-installed models).

🐛 Bug Fixes

  • Fixed AMD RX 9060XT ROCm errors ("ROCm error: invalid device function") by updating the ROCm image and specifying correct GPU targets.
  • Resolved missing libomp.so linking issue on macOS Docker containers.
  • Addressed model loading problems on macOS by bundling necessary libutf8 libraries.
  • Improved CUDA device detection.
  • Set flash_attention to auto in llama.cpp for performance optimization.

🔧 Affected Symbols

diffusers backendwhisper backendstablediffusion backendllama.cpp