Mamba

vLLM v0.27.0 introduces Kimi K3 support, new model additions like Qwen3.5 and VaultGemma, and a significant upgrade to PyTorch 2.13.0. The release also enhances performance and features across various areas including FlashAttention 4, Model Runner V2, KV offloading, and hardware enablement.

v0.24.0 introduces extensive support and performance optimizations for new models like MiniMax-M3 and DeepSeek-V4, matures the Model Runner V2 with default quantization support, and overhauls device selection by removing internal use of CUDA_VISIBLE_DEVICES.

This release focuses heavily on DeepSeek V4 maturity with new kernel support and packaging, significant advancements in Model Runner V2, and the introduction of an experimental Rust frontend. Performance saw notable gains from batch-invariant inference with Cutlass FP8 and the rollout of multi-tier KV cache offloading.

vLLM v0.17.0 introduces a major upgrade to PyTorch 2.10, integrates FlashAttention 4, and significantly matures Model Runner V2 with features like Pipeline Parallelism. This release also adds full support for the Qwen3.5 model family and introduces new performance tuning flags.

This release introduces several new models including Meta Muse Glimmer, GraniteMoeSWA, A.X-K1/K2, and Cosmos3 Edge. It also includes significant updates to attention mechanisms, vision processing, and generation capabilities, alongside important breaking changes in kernel opt-in and cache cropping.

Patch release 4.55.3 focuses on stability improvements for FlashAttention-2 on Ascend NPU, FSDP sharding fixes, and critical bug fixes for GPT-OSS and Mamba models.

A maintenance patch release focused on fixing regressions in cache inheritance, device placement, and distributed training (TP/device-mesh) across various model architectures like ModernBERT, GPT2, and Mamba.

LocalAI v2.26.0 introduces significant backend consolidation, dropping deprecated TTS and LLM backends in favor of modern alternatives like GGUF and new TTS options. This release also adds support for Nvidia L4T devices and introduces lazy grammar features for llama.cpp.