v4.5.6
📦 localaiView on GitHub →
✨ 5 features🐛 19 fixes🔧 9 symbols
Summary
This release focuses heavily on improving distributed system stability, enhancing model backend capabilities with new GGML detectors, and fixing numerous bugs related to configuration, imports, and streaming retries. Key features include cross-replica state synchronization and new voice/face detection backends.
Migration Steps
- If using distributed agents, ensure NATS permissions are correctly configured.
- If encountering issues with GGUF context size parsing, verify configuration or rely on the new default fallback.
✨ New Features
- Implemented SyncedMap component and migrated finetune/quant/agent-tasks to use cross-replica state in distributed mode.
- Added LOCALAI_DISTRIBUTED_SHARED_MODELS environment variable to skip staging on shared volumes in distributed mode.
- Added voice-detect and face-detect ggml backends, replacing Python insightface/speaker-recognition.
- Introduced Semantic VAD EOU token for real-time processing.
- Defaulted swa_full:true for models using sliding-window-attention.
🐛 Bug Fixes
- Fixed missing NATS permission for distributed agent nodes.
- Config now falls back to DefaultContextSize for unparseable GGUFs and pins NVFP4 gallery context_size.
- Fixed gallery matching of mmproj/model quant as a whole token, preventing F16 from selecting BF16.
- Distributed mode now returns an empty backend list for agent nodes instead of failing on backend.list.
- Fixed openresponses by bounding the resume-stream buffer and enforcing response ownership.
- Ported multimodal path in ik-llama to the mtmd API and bumped to f96eaddb.
- Added explicit click dependency for spacy CLI to work on intel builds (kokoro fix).
- Improved binary download/upgrade robustness (resume, rate-limit, UX) in the launcher.
- Fixed missing agent NATS permissions (second report).
- Allowed invalid_reference_casting in fish-speech to allow tokenizers builds on darwin.
- Retried layer downloads on transient network errors for OCI images.
- Fixed sglang to parse tool_call function arguments before applying the chat template.
- Stopped max_tokens streaming retry loop on reasoning models for OpenAI compatibility.
- Fixed import logic to derive model name from the selected GGUF for repo-root URIs.
- Avoided quadratic-time debug logging in CleanupLLMResult / ParseFunctionCall.
- Fixed import logic to strip file:// scheme from model path for local imports.
- Aligned openresponses test model name with GGUF-derived naming.
- Fixed macOS notarization by stapling the ticket to the .app instead of just the dmg.
- Ensured UI-saved Check Interval is persisted across watchdog restarts.