v4.1.2
📦 localaiView on GitHub →
✨ 1 features🐛 2 fixes
Summary
This release introduces speculative decoding settings for llama.cpp and includes fixes for logprobs passing and chat retries. It also updates underlying dependencies like llama.cpp and stable-diffusion.cpp.
✨ New Features
- Wire speculative decoding settings for llama.cpp.
🐛 Bug Fixes
- Correctly pass logprobs in autoparser.
- Prevent retrying if chatdeltas or tooldeltas were already received from the backend in chat operations.