b10672
📦 llama-cppView on GitHub →
✨ 5 features🐛 11 fixes🔧 18 symbols
Summary
This release updates the OpenVINO backend to version 2026.3.1, adds support for Whisper.cpp and Qwen3.5 on NPU, and introduces new operations. It also includes significant improvements to static shape handling, token-count independence, and chunked prefill for better performance and accuracy.
Migration Steps
- To configure NPU compilation parameters without using the generic property escape hatch, set the GGML_OPENVINO_NPU_COMPILE_CONFIG environment variable. For example, to enable optimization-level=3 for NPU compilation, set it to 'optimization-level=3'.
✨ New Features
- OpenVINO Backend: Fuse IM2COL + MatMul convolution into OpenVINO convolution.
- OpenVINO backend: Support Whisper.cpp.
- OpenVINO backend: Enable Qwen3.5 on NPU.
- ggml-openvino: Forward NPU compilation mode parameters using GGML_OPENVINO_NPU_COMPILE_CONFIG environment variable for NPU compiler configuration.
- ggml-openvino: Support RELU, POOL_2D, QUICK_GEGLU, and ROLL ops.
🐛 Bug Fixes
- Fix ggml_rope_set_offset case.
- Static shapes: Corrected get_graph_input_shape() to handle dynamic s_copy/s_copy-leaf inputs and propagate static slot dimensions correctly.
- Static shapes: Optimized handling of -np 1 by short-circuiting defrag remainder gathers to avoid degenerate Slice/Concat operations.
- Token-count independence: Anchored GDN state slice and dropped rs_src_begin runtime inputs to correctly derive offsets from captured cgraph in static mode.
- Token-count independence: Fixed CONT to identify its token axis when the graph was captured with a single token.
- Chunked prefill: Added chunk_valid_len runtime input to handle padded tokens and zero g/beta for padded steps, preventing incorrect recurrence folding.
- Chunked prefill: Corrected get_is_prefill() and chunk loop bound to account for IMROPE stacking position planes, preventing out-of-bounds chunks.
- Fixed code style.
- Exclude GPU/NPU failing POOL_2D case.
- Fix pool case.
- Remove unused static remote context branch.