●Change8

v1.81.6-nightly

📦 litellmView on GitHub →
✨ 22 features🐛 33 fixes🔧 37 symbols

Summary

This release focuses heavily on bug fixes across various providers like Gemini, AWS Bedrock, and Vertex AI, alongside significant feature additions to the UI, Search API, and LiteLLM Vector Stores, including new model support.

Migration Steps

  1. If using Vertex AI with Anthropic models, ensure image URLs are correctly converted to base64 in tool messages.

✨ New Features

  • Added /list endpoint to Search API to list existing search tools in the router.
  • Introduced a reusable Table Sort Component for the UI.
  • Added Error message search functionality to UI spend logs.
  • Implemented permission management for users and teams in LiteLLM Vector Stores.
  • Added new OpenRouter models: `xiaomi/mimo-v2-flash` and `z-ai/glm-4.…`.
  • Added support for OpenRouter Kimi K2.5.
  • Implemented Usage Export breakdown by Teams and Keys in the UI.
  • Added validation of model cost map as a job.
  • Added custom_llm_provider as a Gemini translation mechanism.
  • Implemented Bulk Update Keys Endpoint.
  • Added support for showing Current Store and Retention Status in UI Spend Logs.
  • Added New Badge Dot Render feature to the UI.
  • Added event-driven coordination for global spend query to prevent cache stampede.
  • Added new model: `amazon.nova-2-pro-preview-20251202-v1:0`.
  • Integrated LiteLLM x Claude Agent SDK.
  • Added /openai_passthrough route for OpenAI passthrough requests.
  • Added /delete endpoint support for Gemini.
  • Added cost tracking and usage object in aretrieve_batch call type.
  • Added routing of xai chat completions to responses when web search options are present.
  • Added mock client factory pattern and mock support for PostHog, Helicone, and Braintrust integrations.
  • Implemented Realtime API benchmarks.
  • Added async_post_call_response_headers_hook to CustomLogger.

🐛 Bug Fixes

  • Fixed guardrails issues related to streaming-response regex.
  • Fixed migration issue and stabilized image build.
  • Filtered unsupported beta headers for AWS Bedrock Invoke API.
  • Allowed tool_choice for Azure GPT-5 chat models.
  • Fixed tool usage issue related to Anthropic (#19800).
  • Ensured BadRequestError is inspected after all other policy types.
  • Used local tiktoken cache during lazy loading.
  • Subtracted implicit cached tokens from text_tokens for correct Gemini cost calculation.
  • Fixed Prompt Studio history to correctly load tools and system messages.
  • Fixed CI/CD issues and changed related processes.
  • Fixed Gemini entry for `gemini-robotics-er-1.5-preview`.
  • Converted image URLs to base64 in tool messages for Anthropic when using vertex_ai.
  • Fixed router search tools v2 implementation.
  • Fixed stream_chunk_builder to preserve images from streaming chunks.
  • Added libsndfile to the main Dockerfile for ARM64 audio processing.
  • Added datadog_llm_observability to /health/services allowed list in proxy.
  • Prevented provider-prefixed model leaks in the proxy.
  • Routed hosted_vllm through base_llm_http_handler to support ssl_verify.
  • Added test to check if model map is correctly formatted.
  • Fixed model map path in validation test.
  • Fixed `litellm_fix_robotic_model_map_entry`.
  • Fixed sorting for /v2/model/info endpoint.
  • Fixed error_code in Spend Logs metadata.
  • Fixed issue where prisma generate ran as root user in non-root containers.
  • Supported file retrieval in GoogleAIStudioFilesHandle for Gemini.
  • Fixed extraction of input tokens details as a dictionary in ResponseAPILoggingUtils.
  • Fixed `max_input_tokens` for `gpt-5.2-codex`.
  • Fixed Batch and File user level permissions.
  • Fixed aspectRatio mapping in image edit.
  • Fixed vllm embedding format.
  • Removed unsupported header `prompt-caching-scope-2026-01-05` for vertex ai.
  • Fixed CI pipeline router coverage failure.
  • Resolved high CPU usage when router_settings is in DB by avoiding REGISTRY.collect() in PrometheusServicesLogger.

Affected Symbols

vertex passthrough model namesstreaming-response regexAWS Bedrock Invoke APIAzure GPT-5 chat modelstool_choiceantropicBadRequestErrortiktoken cachetext_tokensPrompt Studio historygemini-robotics-er-1.5-previewvertex_aiAnthropic tool messagesrouter search tools v2stream_chunk_builderlibsndfiledatadog_llm_observability/health/serviceshosted_vllmssl_verifymodel map validationlitellm_fix_robotic_model_map_entry/v2/model/infoSpend Logs metadataprisma generateGoogleAIStudioFilesHandleResponseAPILoggingUtilsmax_input_tokensgpt-5.2-codexBatch and File user level permissionsaspectRatio mappingvllm embedding formatprompt-caching-scope-2026-01-05REGISTRY.collect()PrometheusServicesLoggerCustomLoggerasync_post_call_response_headers_hook