●Change8

v1.79.1.dev6

📦 litellmView on GitHub →
✨ 18 features🐛 20 fixes🔧 26 symbols

Summary

This release introduces significant feature enhancements, including expanded vector store support (Milvus, Azure AI), new OCR providers (VertexAI, Azure AI Doc Intelligence), and various UI improvements. Numerous bug fixes address issues across providers like AWS Bedrock, Azure, Gemini, and Anthropic, alongside memory leak resolutions related to Pydantic warnings.

Migration Steps

  1. If you were relying on Pydantic behavior that triggered deprecation warnings in Pydantic 2.11+, ensure your usage aligns with current Pydantic standards to resolve potential memory accumulation issues.

✨ New Features

  • Milvus vector store search support added.
  • Azure AI Vector Stores now support "virtual" indexes and vector store creation via the passthrough API.
  • Added support for `custom_llm_provider` for non-generation video endpoints.
  • E2E Container API Support added.
  • Guardrail information structure changed to use a list type for `guardrail_information`.
  • Milvus Passthrough API now supports create and read vector store operations.
  • Added support for S3 logger with `ssl_verify` when using minio logger.
  • Added VertexAI OCR provider support and cost tracking for the /ocr endpoint.
  • Added Azure AI Doc Intelligence OCR support.
  • Enabled automated prompt caching message format for Claude on Databricks.
  • Generalized tiered pricing in the generic cost calculator.
  • Added Litellm test key audio feature to the UI.
  • Added support for tags and descriptions in AWS Secrets Manager.
  • Added Bedrock Agentcore as a provider on LiteLLM Python SDK and LiteLLM AI Gateway.
  • Added Firecrawl search API support to the /search API.
  • Added serxng search API provider.
  • Support for `reasoning_effort` added for watsonx chat models.
  • Added `shared_session` support to the responses API.

🐛 Bug Fixes

  • Resolved memory accumulation caused by Pydantic 2.11+ deprecation warnings.
  • Fixed empty assistant message handling in AWS Bedrock Converse API to prevent 400 Bad Request errors.
  • Fixed Azure not accepting extra body parameters.
  • Fixed Anthropic token counting for VertexAI.
  • Fixed regression where Guardrail Entity Could not be selected and entity was not displayed in the UI.
  • Stripped base64 in S3 operations.
  • Fixed dot notation support on UI SSO configuration.
  • Fixed index field not being populated in streaming mode when n>1 and tool calls are present.
  • Updated perplexity cost tracking.
  • Fixed `image_config.aspect_ratio` not working for gemini-2.5-flash-image.
  • Fixed broken link on model_management.md documentation.
  • Properly translated Anthropic image format to OpenAI in the Anthropic adapter.
  • Fixed translation problem with Gemini parallel tool calls.
  • Fixed typo of the word 'original'.
  • Removed automatic summary from `reasoning_effort` transformation for OpenAI.
  • Fixed handling of float `redis_version` from AWS ElastiCache Valkey.
  • Fixed Langfuse input tokens logic for cached tokens.
  • Fixed Gemini API key being sent via x-goog-api-key header when using a custom `api_base`.
  • Fixed OpenAI Responses API streaming tests usage field names and cost calculation.
  • Fixed handling of None values in daily spend sort key in the proxy.

🔧 Affected Symbols

Passthrough EndpointsAWS Bedrock Converse APIAzureGuardrail EntityAnthropic token countingVertexAIs3 loggerui_sso.pylangfuse otelstreaming modetool callsperplexity cost trackinggemini-2.5-flash-imagemodel_management.mdAnthropic adapterGemini parallel tool callsgeneric cost calculatorguardrail_informationOpenAI Responses APIredisAWS ElastiCache ValkeyLangfuseGemini APIwatsonx chat modelsresponses APIaws secrets manager