b9910
📦 llama-cppView on GitHub →
🐛 3 fixes🔧 5 symbols
Summary
This release focuses on server-side stability by fixing draft model fit/load inconsistencies and refactoring parameter initialization. It also provides numerous pre-compiled binaries for diverse operating systems and hardware configurations.
Migration Steps
- Code relying on the previous draft model/context loading mechanism may need updates due to refactoring that unifies initialization and moves speculative initialization to speculative.cpp.
- Code interacting with server_context_impl model_dft and ctx_dft should note they now use raw pointers.
🐛 Bug Fixes
- Fixed inconsistency between draft model fit and load operations.
- Prevented progress callback throttling during draft model loading.
- Fixed an issue related to vertical alignment (valign).