●Change8

v1.80.15-nightly

📦 litellmView on GitHub →
✨ 18 features🐛 32 fixes🔧 12 symbols

Summary

This release introduces numerous new features, including provider additions (abliteration.ai, Bedrock for token counting), enhanced Prometheus metrics, and UI improvements for the Playground. Numerous bug fixes address issues across proxy streaming, provider configurations (Gemini, Azure), security, and internal workflows.

Migration Steps

  1. If using wildcard routing with multiple credentials, review error messages and validation logic due to improvements in handling these scenarios.

✨ New Features

  • Calculate `total_tokens` manually if missing but derivable from input and output tokens.
  • Add built-in migration lock to prevent concurrent Prisma migrate deploy (Note: This feature was immediately reverted).
  • Add Prometheus metrics for request queue time and guardrails.
  • Add Anthropic cache control option to image tool call results.
  • Add Prometheus caching metrics for cache hits, misses, and tokens.
  • Add abliteration.ai provider support.
  • OpenRouter embeddings API support.
  • Support toggling tag matching between ANY and ALL for tag-routing.
  • Add custom proxy base URL support to Playground UI.
  • Add Key and Team Router Setting functionality.
  • Add Focus export support.
  • Add qualifire eval webhook.
  • Add support for Vertex AI API keys.
  • Add mcp registry.
  • Add Bedrock as a backend API for token counting.
  • Add memory leak detection tests with CI integration.
  • Add Endpoint Activity in Usage to the UI.
  • Update prices json for novita provider.

🐛 Bug Fixes

  • Revert the addition of the built-in migration lock due to issues.
  • Proxy: return JSON error response instead of SSE format for initial streaming errors.
  • Fix Litellm embeddings calltype for guardrail precallhook.
  • Prevent duplicate User-Agent tags in request_tags.
  • Make `base_connection_pool_limit` default value consistent.
  • Fix braintrust logging: pass span_attributes in async logging and skip tags on non-root spans.
  • Fix Gemini: support snake_case for google_search tool parameters.
  • Fix proxy: use async anthropic client to prevent event loop blocking.
  • Fix properly use litellm api keys.
  • Add index on LOWER(user_email) for faster duplicate email checks.
  • Mask extra header secrets in model info.
  • Add xiaomi_mimo to LlmProviders enum to fix router support.
  • Fix workflow issue with labeling using a working regex pattern.
  • Proactive RDS IAM token refresh to prevent 15-min connection failures.
  • Normalize Proxy Config Callback.
  • Fix how to execute cloudzero sql.
  • Improve error messages and validation for wildcard routing with multiple credentials.
  • Fix: Add thought_signatures to VertexGeminiConfig and test.
  • Fix Nova model detection for Bedrock provider.
  • Security fix: prevent expired key plaintext leak in error response.
  • Fix Prisma migration workflow from running in forks.
  • Add logprobs support for Azure OpenAI GPT-5.2 model.
  • Fix: response_format leaking into extra_body.
  • Fix litellm sdk embedding headers missing field.
  • Fix: google_genai streaming adapter provider handling.
  • Fix: align max_tokens with max_output_tokens for consistency.
  • Fix Watsonx Audio Transcription - filter model field.
  • Fix: Enforce org level max budget.
  • Fix test_count_tokens_caching.
  • Fix mcp error in multiple servers.
  • Add Custom CA certificates to boto3 clients.
  • Fix: turn_off_message_logging Does Not Redact Request Messages in proxy_server_request Field When Stored to Database.

🔧 Affected Symbols

LlmProvidersVertexGeminiConfigbase_connection_pool_limitgoogle_searchrequest_tagsUser-Agenttotal_tokensmax_tokensmax_output_tokensresponse_formatextra_bodyproxy_server_request