●Change8

v1.79.2-nightly

📦 litellm
✨ 35 features🐛 20 fixes🔧 26 symbols

Summary

This release introduces significant feature enhancements across vector stores (Milvus, Azure AI), OCR providers (VertexAI, Azure AI Doc Intelligence), and new integrations like Bedrock Agentcore and CyberArk Secrets Manager. Numerous bug fixes address issues related to Pydantic warnings, API parameter handling, and cost calculation across various providers.

Migration Steps

  1. If you were relying on Pydantic behavior that is now deprecated in v2.11+, you may need to adjust your code to resolve memory accumulation issues.
  2. If using Azure AI Doc Intelligence OCR, note the new provider support.
  3. If using /ocr, note the new VertexAI OCR provider support.
  4. If using Bedrock Agentcore, note the new provider support.
  5. If using search APIs, note the addition of firecrawl and serxng providers.
  6. If using secrets management, note the addition of CyberArk integration.

✨ New Features

  • Added Milvus vector store search support.
  • Added support for multi-part form data on Passthrough Endpoints.
  • Added support for "virtual" indexes in Azure AI Vector Stores and creation of vector stores on the passthrough API.
  • Added custom_llm_provider support for non-generation video endpoints.
  • Added E2E Container API Support.
  • Added LLM provider response headers to Responses API.
  • Added support for Guardrail Entity selection and display in UI.
  • Added UI feature to delete Team Members with friction.
  • Added UI feature for Model Existing Credentials Improvement.
  • Added UI feature to show PII Config on Guardrail Info Page.
  • Added support for streaming responses API in langfuse otel.
  • Changed guardrail_information to be a list type.
  • Added Milvus Passthrough API support, including create and read vector store support.
  • Added S3 logger, including support for ssl_verify when using minio logger.
  • Added VertexAI OCR provider support and cost tracking for /ocr endpoint.
  • Added Azure AI Doc Intelligence OCR support.
  • Added Prometheus metric to track callback logging failures in S3.
  • Enabled automated prompt caching message format for Claude on Databricks.
  • Generalized tiered pricing in the generic cost calculator.
  • Added support for configuring static MCP header.
  • Added UI feature for Litellm test key audio.
  • Added UI feature to prevent trailing slash in sso proxy base url input.
  • Added UI feature for Tag Usage Top Model Table View and Label Fix.
  • Added UI feature for initial changes supporting prompts to multiple models.
  • Added support for tags and descriptions to aws secrets manager.
  • Added Bedrock Agentcore as a provider on LiteLLM Python SDK and LiteLLM AI Gateway.
  • Added firecrawl search API support for the /search API.
  • Added serxng search API provider.
  • Added support for reasoning_effort for watsonx chat models.
  • Added shared_session support to responses API.
  • Added gpt-image-1-mini pricing for Azure.
  • Added CyberArk Secrets Manager Integration.
  • Added Key Rotations support for CyberArk.
  • Added allowing Key based prefix to S3 path.
  • Added benchmark comparison with other AI gateways.

🐛 Bug Fixes

  • Resolved memory accumulation caused by Pydantic 2.11+ deprecation warnings.
  • Fixed empty assistant message handling in AWS Bedrock Converse API to prevent 400 Bad Request errors.
  • Fixed Azure not accepting extra body parameters.
  • Fixed Anthropic token counting for VertexAI.
  • Fixed stripping of base64 in S3.
  • Fixed dot notation support on ui sso.
  • Fixed returning the diarized transcript when it's required in the request.
  • Fixed index field not populated in streaming mode with n>1 and tool calls.
  • Updated perplexity cost tracking.
  • Fixed image_config.aspect_ratio not working for gemini-2.5-flash-image.
  • Fixed broken link on model_management.md.
  • Fixed translation problem with Gemini parallel tool calls.
  • Fixed typo of the word original.
  • Removed automatic summary from reasoning_effort transformation for openai.
  • Fixed handling of float redis_version from AWS ElastiCache Valkey in redis.
  • Fixed langfuse input tokens logic for cached tokens.
  • Fixed sending Gemini API key via x-goog-api-key header with custom api_base.
  • Enabled Noma guardrail support for images.
  • Fixed handling of None values in daily spend sort key in proxy.
  • Fixed OpenAI Responses API streaming tests usage field names and cost calculation.

🔧 Affected Symbols

Passthrough EndpointsAzure AI Vector StoresAWS Bedrock Converse APIResponses APIAzureAnthropicVertexAIGuardrail Entitys3 loggerminio loggerlangfuse otelguardrail_informationgemini-2.5-flash-imagemodel_management.mdClaude on Databricksaws secrets managerBedrock Agentcorefirecrawl search APIserxng search APIwatsonx chat modelsredisAWS ElastiCache ValkeyGemini APINoma guardrailopenaiCyberArk Secrets Manager