●Change8

Migrating to vLLM v0.8.1

Version v0.8.1 introduces 2 breaking changes. This guide details how to update your code.

Released: 3/19/2025

2
Breaking Changes
4
Migration Steps
11
Affected Symbols

⚠️ Check Your Code

If you use any of these symbols, you need to read this guide:

ChatCompletionRequestAutoModelForImageTextToTextSGMVBGMVcustom_cache_managerfused_moe_kernelGemma3Olmo2Zamba2PixtralCALL_RESHAPE_AND_CACHE_XX

Breaking Changes

●Issue #1

Retired SGMV and BGMV Kernels for LoRA; users relying on these specific kernels for custom implementations may need to transition to newer kernel paths.

●Issue #2

Removed custom_cache_manager from the frontend, which may affect custom engine implementations.

Migration Steps

  1. 1
    Upgrade to v0.8.1 to receive critical bug fixes for the v0.8.0 release.
  2. 2
    If using custom Dockerfiles, ensure the latest version of the transformers library is installed as per updated documentation.
  3. 3
    Update any code referencing SGMV/BGMV kernels to use the standard LoRA kernel path.
  4. 4
    Remove references to custom_cache_manager if previously used in frontend integrations.

Release Summary

v0.8.1 is a maintenance release focusing on V1 engine stability, adding Zamba2 support, and enabling LoRA for embedding models. It includes critical fixes for sampling dtypes, quantization, and TPU performance.

Need More Details?

View the full release notes and all changes for vLLM v0.8.1.

View Full Changelog