●Change8

Migrating to vLLM v0.7.3

Version v0.7.3 introduces 2 breaking changes. This guide details how to update your code.

Released: 2/20/2025

2
Breaking Changes
3
Migration Steps
10
Affected Symbols

⚠️ Check Your Code

If you use any of these symbols, you need to read this guide:

TransformersModelQwen2_5_VLForConditionalGenerationMllamaGLM4VMolmoPhi3KVCacheManagerModelInputForGPUrotary_embeddingv1.Sampler

Breaking Changes

●Issue #1

Separate text-only and vision variants of the same model architecture, which may require updating model loading logic for specific VLM architectures.

●Issue #2

V1 Engine now uses msgpack for core request serialization, potentially breaking custom integrations that rely on previous serialization formats.

Migration Steps

  1. 1
    Ensure PyTorch 2.6 or nightly is used for full compatibility with recent torch.compile enhancements.
  2. 2
    Update VLM implementations to use the new merged multimodal processors for Mllama, GLM4V, and Molmo.
  3. 3
    If using V1 Engine, verify that any custom request handling is compatible with msgpack serialization.

Release Summary

This release introduces significant DeepSeek optimizations including Multi-Token Prediction and MLA FlashAttention3 support, alongside major V1 Engine updates like LoRA and Pipeline Parallelism. It expands hardware support for TPU, ROCm, and Gaudi while adding several new model architectures and quantization methods.

Need More Details?

View the full release notes and all changes for vLLM v0.7.3.

View Full Changelog