●Change8

v4.5.0

📦 localaiView on GitHub →
✨ 11 features🐛 4 fixes🔧 26 symbols

Summary

LocalAI 4.5.0 introduces major new capabilities including depth perception, sound event classification, on-device TTS, and NER-based PII filtering. The release also enables continuous batching by default and ships a fully redesigned React UI.

Migration Steps

  1. If using realtime voice, be aware that speaker identity is now surfaced to the client and LLM.
  2. If relying on older PII filtering mechanisms, note that filtering is now NER-centric and uses the new `privacy-filter.cpp` backend.
  3. Users benefiting from continuous batching should see performance improvements without configuration changes due to prefix caching and VRAM-scaled concurrency being enabled by default.

✨ New Features

  • Introduced the `depth-anything` backend for monocular metric depth and camera pose estimation using Depth Anything 3 (ggml port), exposed via a typed `Depth` gRPC RPC and `POST /v1/depth`.
  • Added the `ced` backend for 527-class AudioSet sound event classification (e.g., baby cry, glass breaking) via REST (`POST /v1/audio/classification`) and a VAD-decoupled realtime stream.
  • Integrated the `supertonic` ONNX TTS backend for multilingual, on-device speech synthesis, featuring fast cold start and eliminating dependency on espeak/G2P.
  • Implemented a new `privacy-filter.cpp` engine for PII filtering using Named Entity Recognition (NER) token classification, running alongside a regex secret detector.
  • Realtime voice sessions are now speaker-aware, surfacing speaker identity to both the client and the LLM.
  • Realtime voice now supports conversation compaction via summarize-then-drop mechanism to reduce costs on long calls.
  • Continuous batching (multi-user serving) is enabled by default through automatic configuration of prefix caching, Blackwell-tuned batch sizes, and VRAM-scaled concurrency.
  • Added support for model aliases, allowing redirection/renaming of model names to other configured models live.
  • Word-level ASR timestamps are now available for NeMo and CrispASR models via the gRPC transcription path.
  • A complete redesign of the React UI has landed, featuring a new design language, shell/navigation overhaul, and improved data visualization.
  • Distributed staging has been hardened: cold-load staging is detached from request context, progress is broadcast across replicas, and the downloader is resumable.

🐛 Bug Fixes

  • Fixed issues related to Depth Anything V2 gallery models and metric-large/nested metric entries.
  • Fixed issues related to PII filtering, including adding live NER end-to-end coverage and fixing review issues in the privacy filter implementation.
  • Fixed issues related to realtime voice handling, including implementation of OpenAI `item.delete`, `item.truncate`, and `input_audio_buffer.clear` functionality.
  • Fixed issues related to distributed staging, ensuring large models finish loading during cold-start.

Affected Symbols

Depth Anything 3 (new backend)depth-anything.cpp (dependency)Depth (new typed gRPC RPC)POST /v1/depth (new REST endpoint)ced (new backend)ced.cpp (dependency)POST /v1/audio/classification (new REST endpoint)pipeline.sound_detection (new event)conversation.item.sound_detection (new event)supertonic (new TTS backend)supertonic-3 (model)privacy-filter.cpp (new backend)TokenClassify (modified path)analyze (exposed API)redact (exposed API)completions (request filtering extended)embeddings (request filtering extended)edits (request filtering extended)Ollama (request filtering extended)OpenAI Privacy Filter NER modelsOpenAI item.delete (realtime API)OpenAI item.truncate (realtime API)OpenAI input_audio_buffer.clear (realtime API)NeMo (ASR timestamps)CrispASR (ASR timestamps)gRPC transcription path