●Change8

v1.28.0

Breaking Changes
📦 huggingface-hubView on GitHub →
⚠ 1 breaking✨ 7 features🐛 7 fixes🔧 16 symbols

Summary

This release introduces enhanced capabilities for deploying and managing Inference Endpoints, including hardware discovery via CLI and SDK, and support for managed engine images with multi-accelerator parallelism. Several bug fixes and documentation improvements are also included.

⚠️ Breaking Changes

  • The `huggingface_hub.constants.INFERENCE_ENDPOINT_IMAGE_KEYS` constant has been removed. Code directly accessing this constant will now raise an `AttributeError`.

Migration Steps

  1. When using `custom_image` for Inference Endpoints, key the dictionary with the engine name (e.g., `vLLM`, `sGLang`) instead of leaving it flat.
  2. Use the `--tensor-parallel-size` and `--data-parallel-size` flags with `hf endpoints deploy` and `hf endpoints update` to correctly configure parallelism for engines like vLLM and SGLang, especially when an endpoint is allocated multiple accelerators.

✨ New Features

  • Introduced `hf endpoints hardware` command to discover deployable hardware for Inference Endpoints, listing valid combinations, prices, and quotas.
  • The `list_inference_endpoints_hardware()` SDK function now provides the same hardware discovery data as the CLI command.
  • Managed engine images for Inference Endpoints now support engine-specific container types (e.g., vLLM, SGLang, TGI) via the `custom_image` parameter, keyed by engine name.
  • The CLI commands `hf endpoints deploy` and `hf endpoints update` now support `--engine`, `--tensor-parallel-size`, and `--data-parallel-size` flags for configuring multi-accelerator parallelism.
  • Added text-to-speech support for the DeepInfra inference provider.
  • Added feature-extraction support for the DeepInfra inference provider.
  • Ensured `hf extensions` commands do not use the metered GitHub REST API.

🐛 Bug Fixes

  • Fixed a bug where the scheduled upload of a single file in a subfolder was not working correctly.
  • Corrected an issue where the `tqdm_class` was ignored by the Xet transfer bar during downloads.
  • Resolved duplicated GPU rows appearing in `hf jobs stats`.
  • Prevented reporting a 429 status code as a window rate limit when the window was not exhausted.
  • Fixed the `ResolvedRevision` string value after pickling or copying.
  • Ensured that a redirect's `Content-Length` is not used as the file size in `get_hf_file_metadata`.
  • Modified Inference Endpoints creation to omit `model.task` instead of sending a null value.

Affected Symbols

`hf endpoints hardware``list_inference_endpoints_hardware()``InferenceEndpointHardware``custom_image``update_inference_endpoint``hf endpoints deploy``hf endpoints update``--engine``--tensor-parallel-size``--data-parallel-size``--custom-image``--health-route``--port``huggingface_hub.constants.INFERENCE_ENDPOINT_IMAGE_KEYS``hf endpoints stats``get_hf_file_metadata`