mtmd

llama.cpp 0.3.0 introduces multimodal capabilities with the dots3-note model and MTP support for GLM-4.5-Air, alongside significant updates to ggml and core model handling.

The mtmd module has been updated to use a pillow-accurate resizing algorithm and includes speed optimizations. The resize_algo has been corrected for all models.

This release includes a minor update to the mtmd component, utilizing ggml_rope_set_offset for improved functionality. It also provides pre-compiled binaries for various platforms and configurations.

This release introduces support for WebP image format via FFmpeg. It also provides updated pre-built binaries for various platforms including macOS, Linux, Android, and Windows with different hardware acceleration options.

This release bumps the llama.cpp version to 0.2.0 and ggml to 0.21.0, incorporating numerous backend improvements, bug fixes, and new model support across various hardware accelerators like SYCL, OpenCL, Metal, and Vulkan.

This release introduces a new --mmproj-device argument and a corresponding environment variable for controlling the mmproj device backend, along with immediate loading of the backend and a shortflag option.

This release includes a fix for the LFM2 image tiling threshold in the mtmd module and provides updated pre-compiled binaries for various platforms and hardware configurations.

This release includes updates to ggml, various build and CI improvements, and enhancements to the server and mtmd components. It also introduces support for integer tokenizer scores.

This release includes CMake clean-up and fixes for xcframework builds. It also consolidates vendor library targets and updates include paths for consumers.

This release includes various fixes for the mtmd and common modules, along with a reverted GGUF fix.

This release introduces support for Pocket-TTS in mtmd, significantly optimizing generation times and adding per-pack configuration. It also includes various bug fixes and code cleanups.

This release includes a bug fix for the mtmd component, specifically addressing an issue with longest_edge ignoring min/max pixel values. Various pre-compiled binaries for different platforms and hardware configurations are also provided.

This release addresses a bug in Qwen3-TTS generation where the text stream was being fed twice, leading to repeated utterances. The fix ensures correct streaming behavior.

Introduced chunk save/load functionality for mtmd and updated a constant name. This release also provides pre-compiled binaries for various platforms and hardware configurations.

This release includes a fix for unlimited-OCR in the mtmd module, addressing issues with max_tiles and converter settings. Various pre-compiled binaries for different operating systems and hardware configurations are also provided.

This release introduces multi-row batching support for deepseek-ocr, allowing rows to be processed in a single shot. This enhances efficiency for OCR tasks.

This release corrects duplicate empty audio chunks for short inputs and restores the tests.sh code. It also provides updated binary downloads for various platforms and configurations.

This release introduces the minicpmv46 downsample functionality and integrates it within gguf. It also includes the build for mtmd_image_preprocessor_llava_uhd and a 4x ignore vit merger, along with code fixes.

This release introduces the `n_embd_head` parameter for `mtmd`. It also provides pre-compiled binaries for various platforms and hardware configurations, including macOS, Linux, Android, and Windows, with support for different backends like CPU, Vulkan, ROCm, OpenVINO, SYCL, CUDA, and HIP.

This release introduces support for MiMo-V2.5 audio input with a new RVQ-based model and includes a gguf converter for mimo audio. Several bug fixes and minor code improvements have also been implemented.

This release includes a fix for the Android build and introduces a wide range of new hardware acceleration and platform support, including Vulkan, ROCm, OpenVINO, SYCL, HIP, and updated CUDA versions.

This release updates the Qwen3-VL position embedding interpolation to use `align_corners=True`, aligning with the transformers reference. It also provides a comprehensive set of pre-compiled binaries for various platforms and hardware accelerators.

This release primarily focuses on fixing UTF-8 handling on Windows for mtmd and arg, resolving ggml_fopen issues, and addressing build failures across platforms.

This release focuses on stability with several bug fixes across the mtmd module, build system improvements, and specific fixes for gemma4ua and potential overflows. Pre-built binaries for numerous platforms are provided.

This release addresses a bug related to n_token miscounting in mtmd and provides updated pre-compiled binaries for numerous platforms and hardware acceleration backends.

This release introduces a new post-decode callback feature for the mtmd component and provides updated pre-compiled binaries for macOS, Linux, Android, Windows, and UI components.

This release primarily focuses on integrating video input support into the mtmd component, alongside various build fixes and CLI enhancements. It also introduces lazy bitmap API support.

This release primarily addresses a bug fix in the `mtmd` component related to Gemma 4 audio projector embeddings by removing the `projection_dim`. It also notes the disabling of several platform-specific builds.

This release enables non-causal vision support for Gemma 4 unified models within the mtmd backend and provides numerous pre-compiled binaries across various platforms and accelerators.

This release introduces an option to skip building the Vision Transformer (ViT) within the mtmd/model pipeline and provides numerous updated pre-compiled binaries for various operating systems and hardware configurations.

This release updates the default configuration for n_head_kv in the mtmd context and cleans up by removing an AI-generated comment. It also provides numerous pre-compiled binaries for various operating systems and hardware configurations.

This release introduces chunking support and preprocessing fixes for the qwen3a model within mtmd, alongside several minor internal adjustments and the distribution of new binaries for various platforms.

This release exposes model modalities via the /v1/models endpoint and includes a fix for the build process, alongside renaming an internal component to mtmd_caps.

This release introduces support for LLAMA_ROPE_TYPE_NONE within the mtmd component and provides updated binary distributions for numerous operating systems and hardware architectures.

This release includes a minor fix for a missing struct tag in the mtmd component and provides extensive pre-compiled binaries for numerous operating systems and hardware targets.

This release primarily addresses a stability issue in the mtmd module related to small image handling. Various pre-compiled binaries for different operating systems and hardware configurations are provided.

This release introduces support for the Gemma 4 audio conformer encoder, detailing its specific architecture and preprocessing steps. Several internal fixes were implemented related to tensor loading and mask matching.

This release introduces support for dots in model names within the mtmd feature and adds GGUF conversion capabilities, alongside various platform-specific binary updates.

This release primarily addresses a bug in the GGUF conversion process specifically affecting audio and vision mmproj components. It also provides numerous pre-compiled binaries for different operating systems and hardware configurations.

This release introduces additional sanity checks for the mtmd component and provides a comprehensive set of pre-compiled binaries for numerous operating systems and hardware targets.

This release introduces significant support for DeepSeek-OCR within the mtmd framework, alongside numerous fixes related to vision model processing, RoPE types, and tensor handling. Several deprecated CLI arguments and internal resize algorithms related to OCR preprocessing were removed.

This release includes a fix for LightOnOCR image preprocessing within the mtmd module and provides updated binary distributions for macOS, Linux, Windows, and openEuler targeting various hardware and acceleration backends.

This release adds support for Nemotron Nano 12B v2 VL models and simplifies related code. It also implements a change to pre-downsample position embeddings during GGUF conversion for fixed input size handling.

This release introduces support for the MiniCPM-o 4.5 vision model via mtmd and provides updated pre-compiled binaries across multiple operating systems and hardware configurations.

This release primarily addresses a bug fix related to ASR performance for the LFM2.5-Audio-1.5B model and provides updated pre-compiled binaries across multiple platforms.

This release fixes an issue where use_non_causal was reported incorrectly in mtmd and includes internal code cleanup and refactoring.

This release focuses on a major refactor of mtmd audio preprocessing, including fixes for padding and log logic, while announcing a change in Linux distribution formats from .zip to .tar.gz.