●Change8

Migrating to vLLM v0.8.2

Version v0.8.2 introduces 2 breaking changes. This guide details how to update your code.

Released: 3/23/2025

2
Breaking Changes
4
Migration Steps
11
Affected Symbols

⚠️ Check Your Code

If you use any of these symbols, you need to read this guide:

V1 EngineTransformersModelSchedulerxgrammarguidanceMamba2InternVLLLaVA-OnevisionQwen2.5-VLTele-FLMRayExecutor

Breaking Changes

●Issue #1

OpenVINO support has been removed from the core repository. Users must now use the external plugin for OpenVINO support.

●Issue #2

TP > 1 is no longer supported for Mamba2 models when Quantization is enabled.

Migration Steps

  1. 1
    If using OpenVINO, migrate to the external OpenVINO plugin.
  2. 2
    Update PyTorch to 2.6.0 if running on macOS or ROCm (AMD).
  3. 3
    If using Mamba2 with quantization, ensure Tensor Parallelism (TP) is set to 1.
  4. 4
    Optionally enable the new fastsafetensors loader for improved weight loading performance.

Release Summary

This release focuses on V1 engine stability, including a critical memory fix and crash prevention for duplicate request IDs. It introduces FP8 KV Cache support, a new fastsafetensors loader, and expands hardware support for TPU and ROCm.

Need More Details?

View the full release notes and all changes for vLLM v0.8.2.

View Full Changelog