b10707
📦 llama-cppView on GitHub →
🐛 1 fixes🔧 2 symbols
Summary
Optimized kv-cells sequence scan for improved generation performance on longer contexts. This release also provides pre-compiled binaries for various platforms and hardware configurations.
🐛 Bug Fixes
- Optimized kv-cells sequence scan to stop once all sequences for a cell are seen, improving generation speed for longer contexts. This change affects the n-gram path via the get_prev_tokens function.