v4.4.1
📦 localaiView on GitHub →
✨ 1 features🐛 1 fixes🔧 1 symbols
Summary
This release introduces streaming capabilities for the realtime pipeline stages and fixes a compatibility issue with recent vLLM versions.
Migration Steps
- If using vLLM >= 0.22, ensure imports for get_tokenizer are updated to reference vllm.tokenizers.
✨ New Features
- Stream the LLM / TTS / transcription pipeline stages in realtime mode.
🐛 Bug Fixes
- Restored compatibility with vLLM versions >= 0.22 by updating the location of get_tokenizer to vllm.tokenizers.