b9908
Breaking Changes📦 llama-cppView on GitHub →
⚠ 2 breaking🔧 1 symbols
Summary
This release enforces a strict RAM limit for the prompt cache, ensuring entries exceeding the limit are skipped and aggressively evicting old entries to maintain the limit. It also updates the availability of pre-built binaries across multiple platforms.
⚠️ Breaking Changes
- The behavior of the prompt cache RAM limit (--cache-ram) has changed to be a strict limit. Previously, the cache might temporarily exceed limits or preserve one entry regardless of size. Now, entries exceeding the RAM limit alone are skipped, and older entries are evicted aggressively to make space for new entries.
- Token-limit cleanup may now evict the last remaining cache entry, whereas previously one entry was always preserved.
Migration Steps
- If you rely on the prompt cache always preserving at least one entry even if it exceeds the RAM limit, be aware that this behavior is removed. Adjust your --cache-ram settings or expected memory usage accordingly.