●Change8

v3.2.1

📦 tgiView on GitHub →
✨ 2 features🐛 2 fixes🔧 2 symbols

Summary

This release introduces support for the Gemma 3 text model type and the official release of the Gaudi Backend. It also includes necessary updates for Triton kernel compilation and various bug fixes.

Migration Steps

  1. If using Triton kernels, ensure `gcc` is available during runtime for kernel compilation.

✨ New Features

  • Router now supports the `gemma3-text` model type.
  • Release of Gaudi Backend for TGI.

🐛 Bug Fixes

  • Fixed an issue related to Sliding Window Attention.
  • Launcher now correctly gets the head dimension for VLMs.

🔧 Affected Symbols

kernelslauncher