b9752
📦 llama-cppView on GitHub →
✨ 2 features🐛 2 fixes🔧 2 symbols
Summary
The server component underwent a significant refactoring of batch construction, introduced an abort_all_slots feature, and included several minor fixes and debugging enhancements.
✨ New Features
- Added ability to abort all slots in the server component.
- Improved handling when the batch becomes full.
🐛 Bug Fixes
- Fixed an assertion error.
- Removed a debug log.