●Change8

Migrating to vLLM v0.7.0

Version v0.7.0 introduces 2 breaking changes. This guide details how to update your code.

Released: 1/27/2025

2
Breaking Changes
3
Migration Steps
9
Affected Symbols

⚠️ Check Your Code

If you use any of these symbols, you need to read this guide:

LLM.sleepLLM.wake_upLLM.collective_rpcLLM.reset_prefix_cachetorch.compileDeepseekScalingRotaryEmbeddingMiniCPMVBaseModelMolmoForCausalLMVLLM_USE_V1

Breaking Changes

●Issue #1

Deepseekv3 integration initially broke quantization for other methods; ensure you are on the latest patch if using mixed quantization.

●Issue #2

V1 engine is a complete rewrite; while optional, it may have different performance characteristics or edge-case behaviors compared to V0.

Migration Steps

  1. 1
    To test the new V1 engine, set the environment variable VLLM_USE_V1=1.
  2. 2
    To enable torch.compile optimizations, use the -O3 engine parameter.
  3. 3
    For VLM developers, implement the merged multi-modal processor and get_*_embeddings methods to support the V1 engine.

Release Summary

This release introduces the V1 engine alpha for improved performance and architectural simplicity, alongside full torch.compile integration. It adds support for several new models including Deepseek-VL2 and Whisper, while expanding hardware compatibility for Apple Silicon, AMD, and TPU.

Need More Details?

View the full release notes and all changes for vLLM v0.7.0.

View Full Changelog