●Change8

Migrating to vLLM v0.8.0rc2

Version v0.8.0rc2 introduces 1 breaking change. This guide details how to update your code.

Released: 3/17/2025

1
Breaking Changes
2
Migration Steps
8
Affected Symbols

⚠️ Check Your Code

If you use any of these symbols, you need to read this guide:

input cache clientMLAUltravoxMixtralGemma3pixtral.pysetup.pyXPU

Breaking Changes

●Issue #1

The input cache client has been removed from the V1 engine. Users relying on this specific client for caching will need to transition to alternative caching mechanisms provided by the engine.

Migration Steps

  1. 1
    Remove any references to the 'input cache client' in V1 engine configurations.
  2. 2
    If using TPU, ensure your environment is compatible with the updated ragged paged attention kernel which no longer requires padding.

Release Summary

This release focuses on V1 engine refinements, including making MLA the default and removing the input cache client. It also includes critical bug fixes for Ultravox, Mixtral, and ROCm testing environments.

Need More Details?

View the full release notes and all changes for vLLM v0.8.0rc2.

View Full Changelog