●Change8

v3.1.0

📦 tgiView on GitHub →
✨ 3 features🐛 4 fixes🔧 3 symbols

Summary

This release introduces full hardware support for Deepseek R1 on AMD and Nvidia, adds fp8 support for MoE models, and includes several stability fixes and dependency updates.

Migration Steps

  1. If using Deepseek R1, ensure you are using the latest Docker image (3.1.0) or newer.
  2. Users running Deepseek R1 on AMD or Nvidia should now see full support.

✨ New Features

  • Full support for Deepseek R1 on both AMD and Nvidia hardware.
  • Added fp8 support for Mixture-of-Experts (MoE) models.
  • Added support for deepseekv3 models.

🐛 Bug Fixes

  • Attempted to remove flaky AWS S3 cache for sccache.
  • Fixed telemetry reporting issues.
  • Addressed potential Out-Of-Memory (OOM) issues possibly introduced by the 2.5.1 change.
  • Hotfixed issues related to Intel CPU support.

🔧 Affected Symbols

attention-kernels 0.2.0moe-kernels 0.8.0moe-kernel 0.8.2