b9901
📦 llama-cppView on GitHub →
✨ 2 features🐛 3 fixes🔧 5 symbols
Summary
This release focuses on SYCL performance enhancements, particularly for the DMMV path by setting K_QUANTS_PER_ITERATION to 1 and fixing gating logic. Several internal SYCL initialization and constant definitions were also updated.
✨ New Features
- SYCL DMMV path now sets K_QUANTS_PER_ITERATION to 1 to improve GPU utilization, resulting in a boost to tg t/s on B70 when combined with the reorder gate being opened.
- SYCL reordered feature is now correctly gated by ggml_sycl_supports_reorder_dmmv in ggml_sycl_op_dequantize_mul_mat_vec.
🐛 Bug Fixes
- Fixed the gate for the reordered feature in SYCL by ensuring ggml_sycl_supports_reorder_dmmv is correctly set.
- Added missing types to ggml_backend_sycl_buffer_init_tensor so the extra field is allocated, allowing the reorder path to take effect.
- Replaced QK_WARP_SIZE with WARP_SIZE for QK_5 in SYCL implementation.