●Change8

b10658

📦 llama-cppView on GitHub →
✨ 3 features🐛 6 fixes🔧 6 symbols

Summary

This release introduces DFlash2 support with local convolution and candidate selection, alongside various bug fixes and code refactoring. Key changes include optimizations for DFlash2 and fixes for mrope and graph number calculations.

Migration Steps

  1. Move llama_model_dflash_selector_top_k to llama-ext.h.
  2. Rename hid and unary.

✨ New Features

  • Added DFlash2 support, including local convolution and candidate selector.
  • Introduced p_min parameter for DFlash2.
  • Enabled output transforms to be gated on DFlash2.

🐛 Bug Fixes

  • Reverted unnecessary changes.
  • Reverted draft sampling in rejection sampling.
  • Applied patch to fix the mrope bug.
  • Fixed CI issues.
  • Corrected graph number calculation.
  • Reverted top-k.cu changes.

Affected Symbols

DFlash2llama_model_dflash_selector_top_khidunarymrope bugtop-k.cu