b10701
📦 llama-cppView on GitHub →
🐛 1 fixes🔧 2 symbols
Summary
This release addresses a critical bug in DFlash2 NVFP4 models by ensuring that Q, K, V, and output projection scales are correctly passed to attention operations, resolving issues with speculative token generation. Various pre-compiled binaries for different platforms and hardware are also provided.
🐛 Bug Fixes
- Fixed an issue where DFlash2 NVFP4 draft models produced almost no accepted speculative tokens due to missing Q, K, V, and output projection scales not being passed to attention operations.