●Change8

v0.2.0

📦 llama-cppView on GitHub →
✨ 15 features🐛 35 fixes🔧 22 symbols

Summary

This release bumps the llama.cpp version to 0.2.0 and ggml to 0.21.0, incorporating numerous backend improvements, bug fixes, and new model support across various hardware accelerators like SYCL, OpenCL, Metal, and Vulkan.

Migration Steps

  1. Keep the vocab-scale K-quant lm_head on the CPU for Adreno A7X (compiler issue workaround) in OpenCL.
  2. Update OpenVINO to 2026.3, skip nemotron-h rollback test.

✨ New Features

  • Added SME2 F32 GEMV kernel support.
  • Added Q2_K reordered MMVQ and ESIMD kernels for SYCL.
  • Added Q5_K ESIMD kernel for SYCL.
  • Enabled tensor split for LFM2/LFM2MOE.
  • Added support for ggml_rope_set_offset on OpenCL, SYCL, WGPU, and Hexagon.
  • Added --mmproj-device argument to mtmd.
  • Added support for DSpark for LFM2 models.
  • Added switch points per HW and quant type to tune the mvq->MMQ decode crossover for CUDA.
  • Added ggml_rope_set_offset with metal support.
  • Added dedup-cache-models preset option to server.
  • Added mulmat with overlapping src0/src1 for ggml-webgpu.
  • Added size guards to GGUFReader.
  • Added support for fused ssm_scan kernel (Mamba-2, d_state in {128, 256}) to OpenCL.
  • Added GraniteSWAForCausalLM / GraniteMoeSWAForCausalLM models.
  • Added Windows ARM64 CUDA support to the manual CI workflow.

🐛 Bug Fixes

  • Reverted SYCL Q2_K reordered MMVQ and ESIMD kernels.
  • Fixed multiple warnings in compiling SYCL backend.
  • Fixed load model with mlock issue in SYCL.
  • Fixed typos in ET.md.
  • Fixed q6_K flat mul_mat for Adreno A6x/A7x GPUs with older E031 compilers.
  • Fixed local size for norm in OpenCL.
  • Fixed --docker-repo being treated as router mode in server.
  • Gracefully fallback on unsupported regex patterns in JSON schema.
  • Reverted tensor-split meta backend fixes.
  • Fixed backend split scheduler race condition in ggml.
  • Fixed get block count error for Nemotron 3 Ultra.
  • Created V as a view of K in the k_iswa build_attn.
  • Avoided binding reference to null pointer in spec.
  • Fixed FA HMX queue ordering and packed the rescale D matrices in Hexagon.
  • Gated __fp16 on __ARM_FP16_FORMAT_IEEE in ggml-cpu.
  • Refactored sleep handling in server, allowing access /metrics during sleep.
  • Made models endpoints private when authentication is enabled in server.
  • Dequant q8_0 KV once in coopmat1 for Vulkan.
  • Added null checks in ggml_vk_queue_command_pools_cleanup for Vulkan.
  • Reverted sharing thread pools when `n_threads` differ.
  • Removed duplicate metadata load in models.
  • Read persisted settings before the API key probe in UI.
  • Dequantize q8_0 using packed types in metal.
  • Fixed tiled transpose for 0<->2 permuted CONT in Vulkan.
  • Skipped test-unicode build on win32/BUILD_SHARED_LIBS.
  • Fixed WAR race in the generic FA tile kernels when the WG spans subgroups in OpenCL.
  • Fixed deepseek-ocr SAM ggml_conv_2d with the im2col kept in F32.
  • Fixed LFM2 image tiling threshold in mtmd.
  • Fixed vocab-scale K-quant lm_head on the CPU for Adreno A7X (compiler issue workaround) in OpenCL.
  • Fixed multiple warnings in compiling sycl backend.
  • Fixed load model with mlock issue in sycl.
  • Fixed get block count error for Nemotron 3 Ultra.
  • Fixed FA HMX queue ordering and packed the rescale D matrices.
  • Fixed WAR race in the generic FA tile kernels when the WG spans subgroups.
  • Fixed LFM2 image tiling threshold.

Affected Symbols

llama.cppggmlsyclkleidiaiopenclmetalTPmtmdmodelserverCUDAcommongraphspecvulkangguf-pyRPCuiquantunicodereadmeci