ggml

This release includes an update to ggml to add MUL_MAT to operations that may require additional memory for WebGPU. Various pre-compiled binaries for different platforms and configurations are also provided.

This release introduces SWIGLU_CLAMP to ggml and adds Vulkan shader support, enhancing computational capabilities.

This release introduces support for the Qwen3.8-Flash-Next (qwen4exp) model, including its GGUF implementation, tensor loading, and decode graph. Several bug fixes related to PLE conv state and context handling are also included.

This release includes improvements to ggml, specifically shortening virtual device naming in CUDA and Metal. It also updates the device description build process for ggml-metal and refines naming conventions.

This release bumps the llama.cpp version to 0.2.0 and ggml to 0.21.0, incorporating numerous backend improvements, bug fixes, and new model support across various hardware accelerators like SYCL, OpenCL, Metal, and Vulkan.

This release includes updates to ggml, various build and CI improvements, and enhancements to the server and mtmd components. It also introduces support for integer tokenizer scores.

Introduced recurrent state rollback for nemotron on CPU and CUDA, and improved ssm_scan to support K > 1 for all backends. This release also includes runtime checks for operation support and fallback mechanisms.

This release includes a fix for ggml, forcing single-thread execution on WASI. It also provides pre-compiled binaries for various platforms and configurations.

This release includes a fix for ARM builds in ggml, addressing an unused variable issue. It also provides updated pre-compiled binaries for various platforms and configurations.

This release introduces aarch64 HWCAP fallbacks and fixes fp16 variant detection in ggml. It also renames the `has_fp16_va` field to `has_fp16` for better clarity on the FEAT_FP16 extension.

This release introduces dynamic allocation for split graph inputs in ggml, resolving crashes with wide MoE models. It also includes various pre-compiled binaries for different platforms and hardware accelerators.

This release addresses a bug in ggml related to KleidiAI CI and stringop overflow warnings. It also provides updated binary releases for various platforms and hardware accelerators.

This release includes adjustments to the ggml logic for offloading operations to the weight's backend and fixes for llama graph issues.

This release enables PowerPC CPU backend variants on AIX and fixes a semicolon issue in the PowerPC Q0 matmul implementation.

This release relaxes tensor contiguity requirements for quantized concatenation on CUDA and GGML, improving flexibility. It also provides updated pre-built binaries for various platforms and hardware accelerators.

This release introduces new functionality to ggml for checking tensor contiguity. It also provides updated pre-compiled binaries for various platforms and hardware accelerators.

This release primarily focuses on fixing a bug in the ggml library related to the CPU concatenation of quantized types. It also provides numerous pre-compiled binaries for various operating systems and hardware configurations.

This release focuses on performance improvements in ggml by optimizing AMX operations, leading to speedups in quantization benchmarks on Intel Xeon CPUs. It also provides updated pre-built binaries for numerous platforms.

This release introduces automatic iGPU flag application for CUDA/HIP in ggml operations. It also provides numerous pre-built binaries across various platforms, noting several specific builds have been disabled.

This release includes a bug fix within the ggml library related to interface method checking. It also provides numerous pre-compiled binaries for different operating systems and hardware configurations.

This release includes an internal fix in ggml related to 2D tensor operations and provides updated pre-compiled binaries across macOS, Linux, Android, Windows, and openEuler platforms.

This release enhances memory estimation for OpenCL backends during model fitting by utilizing CL_DEVICE_GLOBAL_MEM_SIZE. It also provides a comprehensive set of pre-compiled binaries for diverse platforms.

This release introduces a performance enhancement in ggml by implementing the fast walsh-hadamard transform for kv rotation and provides updated pre-compiled binaries for various operating systems and hardware configurations.

This release primarily focuses on an internal fix within ggml to prevent registering duplicate backends and devices. It also provides numerous pre-compiled binaries for various operating systems and hardware configurations.

This release reverts the change to use `find_library` for math library linking in ggml back to standard `-lm` linking to resolve CUDA compilation issues when `GGML_STATIC` is enabled.

This release updates ggml to use 64 bytes aligned tile buffers, showing minor speedups across various Qwen 35 quantization tests. It also provides extensive pre-compiled binaries for numerous operating systems and hardware configurations.

This release primarily focuses on providing pre-built binaries across various operating systems and hardware configurations, including updates for macOS, Linux, Android, Windows, and openEuler. Minor coding style adjustments were made in ggml.

This release introduces the SOLVE_TRI operation to the Hexagon backend and includes various performance and cleanup improvements within the Hexagon and GGML layers. Numerous pre-built binaries are provided for broad platform compatibility.

This release introduces graph versioning in ggml to replace the old graph reuse flag, alongside various platform-specific binary updates and a fix for Windows builds.

This release introduces a new simd_gemm kernel optimized for the RISC-V vector extension within ggml. It also provides numerous pre-compiled binaries for different operating systems and hardware configurations.

This release addresses a specific bug in the ggml library concerning ARM NEON nvfp4 dot product calculations on non-dotprod targets.

This release addresses missing cases for GGML_TYPE_Q1_0 within the ggml library and provides a comprehensive set of updated pre-compiled binaries for diverse operating systems and hardware configurations.

This release includes a stability fix in ggml by checking the return values of CUB calls used in argsort and top-k operations. It also provides numerous pre-compiled binaries for various operating systems and hardware configurations.

This release addresses a bug fix related to RWKV operations thread assignment within ggml and provides numerous pre-compiled binaries for diverse operating systems and hardware configurations.

This release focuses on an optimization within ggml to prevent unnecessary CUDA context creation during device initialization. It also provides numerous pre-compiled binaries for diverse operating systems and hardware configurations.

This release includes a critical bug fix in the ggml library related to IQ4_NL quantization and provides updated pre-compiled binaries for various operating systems and hardware configurations.

This release introduces native AVX512-FP16 support for F16 operations within ggml, alongside the release of numerous pre-compiled binaries for various platforms and hardware configurations.

This release introduces significant backend enhancements, primarily adding OpenVINO support and initial NPU capabilities, alongside numerous performance optimizations and bug fixes across quantization and attention mechanisms.

This release addresses a typo within the ggml component and provides updated binary distributions across various operating systems and hardware configurations, including new CUDA 12.4 and 13.1 builds for Windows.

The ggml component has bumped its internal RPC version. This release primarily provides updated pre-compiled binaries for various operating systems and hardware configurations.

This release introduces the GATED_DELTA_NET operation in ggml, adds KDA and qwen35 dense support, and removes an existing transpose operation.

This release updates comments within ggml regarding backends without memory reporting capabilities and provides a comprehensive set of pre-built binaries for macOS, Linux, Windows, and openEuler platforms.

This release updates ggml to use std::thread instead of OpenMP for AMX builds, resulting in significant inference speedups at the cost of slightly slower loading. Numerous pre-built binaries for various platforms have also been provided.

This release enhances ggml performance by fixing AMX issues and introducing batched support, leading to faster perplexity calculation times.

This release focuses on stability by avoiding undefined behavior in the ggml gemm ukernel. It also provides updated pre-built binaries for numerous platforms and hardware configurations.

This release enhances ggml's unary operation support for non-contiguous source tensors and adds F16 unary operation support to the Metal backend, alongside fixing the ELU operation on Metal.

This release enhances ggml with extended binary broadcast support for permuted source 1 and stabilizes continuous tensor handling by ensuring s0 is always 1.

This release improves backend registration stability on Linux by using a noexcept overload for file system checks, preventing crashes related to broken symlinks. It also provides updated pre-compiled binaries for numerous platforms.

LocalAI 4.2.0 introduces major multimodal capabilities, including voice recognition, face biometrics with liveness detection, and video generation. This release also brings Ollama API compatibility, a fully redesigned, brandable UI, and significant hardening to the vLLM and Distributed Mode backends.

This release focuses on stability improvements, including fixing a memory leak in mlx and adjusting settings for the Grok model on ggml. It also updates VS Code documentation and hides the VS Code launch option.