b10660
📦 llama-cppView on GitHub →
✨ 6 features🐛 4 fixes🔧 5 symbols
Summary
This release introduces support for the Qwen3.8-Flash-Next (qwen4exp) model, including its GGUF implementation, tensor loading, and decode graph. Several bug fixes related to PLE conv state and context handling are also included.
✨ New Features
- Added support for Qwen3.8-Flash-Next (qwen4exp) model architecture.
- Implemented GGUF-side plumbing for qwen4exp, including model architecture and converter.
- Added LLM_ARCH_QWEN4EXP with its hparams and tensor loading.
- Implemented the decode graph for Qwen3.8-Flash-Next, including hyper-connection residual stream, gated delta net layers, MoE block, and dense full attention.
- Added per-layer embedding for Qwen3.8-Flash-Next using a custom I32 graph input for token hashing.
- Enabled carrying the qwen4exp PLE conv state across ubatches for decode and chunked prefill.
🐛 Bug Fixes
- Fixed an issue where the history was read and updated in the same pass for the PLE n-gram hash embedding, causing incorrect context.
- Corrected an issue where an EOS token was cutting its own context in the PLE n-gram hash embedding.
- Fixed the qwen4exp PLE conv state to correctly handle decode and chunked prefill by zero-padding on the left.
- Resolved an issue where build_rs was called twice for the same layer, causing the second call to clobber the first write-back for the PLE layer.