●Change8

Migrating to vLLM v0.7.2

Version v0.7.2 introduces 1 breaking change. This guide details how to update your code.

Released: 2/6/2025

1
Breaking Changes
4
Migration Steps
9
Affected Symbols

⚠️ Check Your Code

If you use any of these symbols, you need to read this guide:

fused_moegrouped_topktorch.compileTransformersModelVLLM_LOGITS_PROCESSOR_THREADSAttention.forwardFinishReasonConstantListpynvml

Breaking Changes

●Issue #1

Qwen2.5-VL support currently requires a source installation of the Hugging Face transformers library rather than the stable release.

Migration Steps

  1. 1
    Install Hugging Face transformers from source to use Qwen2.5-VL.
  2. 2
    Set VLLM_LOGITS_PROCESSOR_THREADS environment variable to optimize high batch size structured decoding.
  3. 3
    Update compressed-tensors dependency to the latest version.
  4. 4
    Use --model-impl=transformers if you need to run text models not natively optimized by vLLM.

Release Summary

This release introduces support for Qwen2.5-VL and a new transformers backend for arbitrary model support. It significantly improves DeepSeek model performance through KV cache memory alignment and torch.compile optimizations.

Need More Details?

View the full release notes and all changes for vLLM v0.7.2.

View Full Changelog