●Change8

v4.6.0

📦 localaiView on GitHub →
✨ 8 features🐛 4 fixes🔧 10 symbols

Summary

LocalAI 4.6.0 is a reliability-focused release that brings full-speed GPU offload for AMD ROCm backends and predictable realtime session warmups. It also introduces conversation forking in the UI and hardens distributed worker lifecycle management.

Migration Steps

  1. If using ROCm, ensure you are on Python 3.12 for correct `rocm-vllm` installation, or be aware of the previous CUDA-only wheel installation.
  2. If relying on the old behavior where a dead worker could indefinitely hold a model lock, be aware that this is now bounded/fixed.
  3. Pipelines that relied on lazy loading for the first turn might now experience a slight delay at session start due to eager pipeline warmup; opt out per pipeline using `disable_warmup: true` if necessary.

✨ New Features

  • AMD ROCm backends now run on-GPU at full speed due to fixes in ggml audio backends offloading, hipBLASLt kernel-tuning data bundling, correct vLLM wheel installation for ROCm on Python 3.12, and ASIC ID table location.
  • Realtime sessions now eagerly warm up the entire pipeline (VAD, ASR, LLM, TTS) upfront to eliminate cold-start stalls on the first turn.
  • New API endpoints `POST /backend/load` (and `/v1/backend/load`) added to manually load models into memory, mirrored by a "Load into memory" UI button and `load_model` MCP tool.
  • Conversation forking added to the built-in chat UI, allowing users to retry any assistant answer, branch a new chat from any point, duplicate chats, or copy conversations as Markdown.
  • A Prometheus counter (`localai_pii_events_total`) is exported on `/metrics` to track PII detection, masking, and blocking events.
  • SSRF protection added to the model gallery by validating `POST /models/apply` config-URL fetches against private/loopback/metadata addresses.
  • Idempotent backend installs: `POST /backends/apply` and `LOCALAI_EXTERNAL_BACKENDS` boot loop will no longer re-pull an already-installed backend unless `force: true` is specified.
  • Tool-calling and reasoning fixes for vLLM (restored non-streaming tool calls) and Python/MLX backends (improved tool-call argument decoding and reasoning block splitting).

🐛 Bug Fixes

  • Distributed model loads no longer wedge for ~15 minutes when a worker dies, as the dead worker can no longer pin the per-model PostgreSQL advisory lock.
  • Orphaned backend workers self-terminate instead of holding VRAM when their parent process dies.
  • Fixed latent stale-closure bug in chat UI where mid-conversation retries sent downstream turns back to the model.
  • Watchdog no longer logs optional `Free()` RPC returns as errors.

Affected Symbols

ggml audio backends (rocm-qwen3-tts-cpp, rocm-omnivoice-cpp, acetep-cpp, vibevoice-cpp)llama-cpp (run.sh)turboquant (run.sh)vLLM (rocm-vllm)POST /backend/loadPOST /v1/backend/loadPOST /models/apply/metricsLOCALAI_BACKEND_PARENT_WATCHLOCALAI_BACKEND_PARENT_WATCH_INTERVAL