torch.compile

This release introduces significant performance optimizations for Kimi-K3 and DeepSeek V4, alongside advancements in speculative decoding and Model Runner V2 maturation. It also features a new Rust frontend and gRPC capabilities, tiered KV cache offloading, and expanded model support.

vLLM v0.26.0 introduces the Inkling model family, significant performance boosts for DeepSeek-V4, flexible attention backends, and enhanced KV offloading. The release also includes a Rust frontend with multimodal capabilities and deeper integration with Transformers 5.13.0.

vLLM v0.25.0 introduces Model Runner V2 as the default for dense models, significantly improving performance and adding support for new features like EVS and realtime embeddings. The release also deprecates PagedAttention and enhances the Transformers backend to match native vLLM speed, alongside numerous model additions and performance optimizations across various hardware platforms.

v0.23.0 brings significant hardening and optimization for DeepSeek-V4, expands Model Runner V2 to Llama/Mistral models, and advances the experimental Rust frontend. This release also mandates compatibility with Transformers v5.

vLLM v0.20.2 is a small patch release focused on bug fixes for DeepSeek V4, gpt-oss, and Qwen3-VL models.

v0.20.0 introduces major infrastructure upgrades, including a default switch to CUDA 13.0 and PyTorch 2.11, alongside significant performance enhancements like TurboQuant 2-bit KV cache and the re-enabling of FlashAttention 4 as default prefill backend.

v0.18.0 introduces major features like gRPC serving, GPU-less render serving, and significant improvements to KV cache offloading and Elastic Expert Parallelism. Ray is now an optional dependency, and numerous model-specific fixes and kernel optimizations have been integrated.

vLLM v0.16.0 introduces full support for Async scheduling with Pipeline Parallelism, a new Realtime WebSocket API, and a major overhaul of XPU platform support by deprecating IPEX in favor of vllm-xpu-kernels. This release also includes extensive model support additions and performance optimizations across various hardware platforms.

This release introduces extensive model support, significant performance enhancements across NVIDIA and AMD hardware (especially for MoE and FP4), and new API features like session-based streaming input. Several deprecated metrics and quantization methods have been removed.

This release updates vLLM to PyTorch 2.9.0 and CUDA 12.9.1, introduces Anthropic API compatibility, and significantly improves the stability of async scheduling and torch.compile integration.

This release introduces support for Qwen2.5-VL and a new transformers backend for arbitrary model support. It significantly improves DeepSeek model performance through KV cache memory alignment and torch.compile optimizations.

This release introduces significant MLA and FP8 kernel optimizations for DeepSeek models, resulting in 3x throughput and 10x memory capacity improvements. It also expands hardware support for Neuron and AMD, adds MiniCPM-o, and enhances the V1 engine with new metrics and prefix caching.

This release introduces the V1 engine alpha for improved performance and architectural simplicity, alongside full torch.compile integration. It adds support for several new models including Deepseek-VL2 and Whisper, while expanding hardware compatibility for Apple Silicon, AMD, and TPU.

This release includes numerous Studio and Desktop improvements, bug fixes, and feature enhancements. Key updates involve better handling of cached pipelines, improved UI elements in Studio, and more robust desktop application behavior.

Unsloth introduces extensive customization for its Studio, including new color palettes, custom fonts, and seven new display languages. Agents are now safer with enhanced tool-call permissions and workspace isolation, while Intel GPUs gain GPU-accelerated inference via Vulkan `llama.cpp` support.

This release introduces broad support for the new gpt-oss model, enabling low-VRAM fine-tuning, alongside significant algorithmic updates that improve performance across all models. It also adds support for Qwen3 models and expands compatibility to include newer NVIDIA hardware like RTX 50 series and Blackwell GPUs.

This release focuses on correctness and performance, rebuilding gradient-cached losses on a shared engine for faster training and fixing several gradient bugs. It also significantly improves inference speed when using `model.compile()`.

This release introduces official support for AWS Neuron devices and brings significant performance and stability improvements across FSDP2, DeepSpeed Sequence Parallelism, and XPU handling. The library also features faster imports by deferring heavy dependency loading.

This release introduces major refactoring for FSDPv2 setup, adding FP8 support, and significantly enhancing performance and stability for Intel CPU/XPU users. It also deprecates `ipex.optimize` and integrates SwanLab as a new experiment tracker.

This release introduces significant performance improvements through regional compilation for torch.compile and adds layerwise casting hooks for memory optimization. It also brings substantial enhancements to FSDP2 support, including enabling `FULL_STATE_DICT` and fixing memory issues.

This patch introduces method chaining for freezing/unfreezing modules and adds litlogger integration. It also removes support for Python 3.9 and fixes several bugs related to checkpointing, hyperparameter saving, and distributed sampling.

PyTorch 2.10 introduces Python 3.14 support for torch.compile, new features like combo-kernels fusion and LocalTensor for distributed debugging, and removes several deprecated or legacy functionalities across ONNX, Dataloader, and nn modules.

This maintenance release addresses critical regressions in PyTorch 2.9.0, specifically fixing memory issues in 3D convolutions, Inductor compilation bugs for Gemma/vLLM, and various distributed and numeric stability fixes.

PyTorch 2.9.0 introduces Python 3.10 as the minimum requirement, defaults the ONNX exporter to the Dynamo-based pipeline, and adds support for symmetric memory and FlexAttention on new hardware.

PyTorch 2.8.0 introduces high-performance quantized LLM inference on Intel CPUs, SYCL support for CPP extensions, and stricter validation for autograd and torch.compile. It includes significant breaking changes regarding CUDA architecture support and internal configuration renames.

This maintenance release focuses on fixing regressions and silent correctness issues across torch.compile, Distributed, and Flex Attention, while also improving wheel sizes and platform-specific compatibility for MacOS, Windows, and XPU.

PyTorch 2.6 introduces Python 3.13 support for torch.compile, FP16 support for X86 CPUs, and new AOTInductor packaging APIs. It includes a significant security change making torch.load use weights_only=True by default and deprecates the official Anaconda channel.

This release focuses heavily on memory optimization, especially around Z Image and VAE loading, alongside introducing several new video and image generation nodes (Kling, Tripo 3.0, new samplers). Several old API nodes (Kling, Pika) were dropped, and internal API type handling was cleaned up.

This release introduces a new Mixed Precision Quantization System and significant performance improvements, especially around memory offloading and torch.compile optimizations for fp8 operations. Several API nodes were updated to use a new client structure, and various stability fixes were applied to caching and execution flow.

This release focuses on performance improvements, especially for queued prompts and torch.compile compatibility, alongside adding new features like stackable VACE conditionings and fp8 compute support.

This release introduces major new pipelines (Wan 2.2, Flux-Kontext, Qwen-Image), significant performance optimizations via regional compilation and GGUF CUDA kernels, and an experimental modular pipeline system.

This release introduces several major video and image generation pipelines including Wan VACE, Cosmos Predict2, and Chroma, while significantly improving torch.compile performance and memory optimization techniques.

This release introduces significant improvements for `torch.compile` compatibility with both LLM.int8() and 4bit quantization, alongside a major refactoring to integrate with PyTorch Custom Operators. Support for Python 3.8 and older PyTorch versions has been dropped.