●Change8

v1.81.7.dev1

📦 litellmView on GitHub →
✨ 22 features🐛 33 fixes🔧 42 symbols

Summary

This release focuses heavily on bug fixes across various providers like Gemini, Vertex AI, and AWS Bedrock, alongside significant feature additions to the UI, Search API, and LiteLLM Vector Stores, including new model support.

Migration Steps

  1. If using Vertex AI with Anthropic models, ensure image URLs in tool messages are correctly converted to base64.
  2. If encountering high CPU when router_settings are in DB, ensure PrometheusServicesLogger avoids REGISTRY.collect().

✨ New Features

  • Added /list endpoint to Search API to list existing search tools in the router.
  • Introduced a reusable Table Sort Component for the UI.
  • Added Error message search functionality to UI spend logs.
  • Implemented permission management for users and teams in LiteLLM Vector Stores.
  • Added new OpenRouter models: `xiaomi/mimo-v2-flash` and `z-ai/glm-4.…`.
  • Added support for OpenRouter Kimi K2.5.
  • Implemented Usage Export breakdown by Teams and Keys in the UI.
  • Added validation of model cost map as a job.
  • Added `custom_llm_provider` as Gemini translation.
  • Implemented Bulk Update Keys Endpoint.
  • Added support for showing Current Store and Retention Status in UI Spend Logs.
  • Added New Badge Dot Render feature to the UI.
  • Added event-driven coordination for global spend query to prevent cache stampede.
  • Added new model: `amazon.nova-2-pro-preview-20251202-v1:0`.
  • Added LiteLLM x Claude Agent SDK Integration.
  • Added /openai_passthrough route for OpenAI passthrough requests.
  • Added /delete endpoint support for Gemini.
  • Added cost tracking and usage object in aretrieve_batch call type.
  • Added routing of xai chat completions to responses when web search options is present.
  • Added mock client factory pattern and mock support for PostHog, Helicone, and Braintrust integrations.
  • Added Realtime API benchmarks.
  • Added async_post_call_response_headers_hook to CustomLogger.

🐛 Bug Fixes

  • Fixed guardrails issues related to streaming-response regex.
  • Fixed migration issue and added stable image.
  • Filtered unsupported beta headers for AWS Bedrock Invoke API.
  • Allowed tool_choice for Azure GPT-5 chat models.
  • Fixed tool usage with Anthropic.
  • Ensured BadRequestError is inspected after all other policy types.
  • Used local tiktoken cache in lazy loading.
  • Subtracted implicit cached tokens from text_tokens for correct Gemini cost calculation.
  • Fixed Prompt Studio history to load tools and system messages correctly.
  • Fixed Gemini entry for `gemini-robotics-er-1.5-preview`.
  • Converted image URLs to base64 in tool messages for Anthropic on vertex_ai.
  • Fixed router search tools v2.
  • Fixed stream_chunk_builder to preserve images from streaming chunks.
  • Added libsndfile to main Dockerfile for ARM64 audio processing.
  • Added datadog_llm_observability to /health/services allowed list in proxy.
  • Prevented provider-prefixed model leaks in proxy.
  • Routed hosted_vllm through base_llm_http_handler to support ssl_verify.
  • Added test to check if model map is correctly formatted.
  • Fixed model map path in validation test.
  • Fixed `litellm_fix_robotic_model_map_entry`.
  • Fixed sorting for /v2/model/info.
  • Fixed error_code in Spend Logs metadata.
  • Fixed CI/CD issues & changed processes during release day.
  • Fixed prisma generate to run as nobody user in non-root container.
  • Supported file retrieval in GoogleAIStudioFilesHandle for Gemini.
  • Fixed input tokens details extraction as dict in ResponseAPILoggingUtils.
  • Fixed `max_input_tokens` for `gpt-5.2-codex`.
  • Fixed Batch and File user level permissions.
  • Fixed aspectRatio mapping in image edit.
  • Fixed vllm embedding format.
  • Removed unsupported header `prompt-caching-scope-2026-01-05` for vertex ai.
  • Fixed CI pipeline router coverage failure.
  • Resolved high CPU when router_settings in DB by avoiding REGISTRY.collect() in PrometheusServicesLogger.

Affected Symbols

vertex passthrough model namesstreaming-response regexAWS Bedrock Invoke API headersAzure GPT-5 chat modelstool_choiceantropic toolBadRequestErrortiktoken cachetext_tokensPrompt Studio historySearch API /list endpointLiteLLM Vector Stores permissionsOpenRouter modelsgemini-robotics-er-1.5-previewvertex_ai tool messagesrouter search tools v2stream_chunk_builderDockerfile (ARM64)/health/servicesdatadog_llm_observabilityprovider-prefixed model leakshosted_vllmssl_verifymodel map validationlitellm_fix_robotic_model_map_entry/v2/model/info sortingSpend Logs metadataprisma generateGoogleAIStudioFilesHandleResponseAPILoggingUtilsmax_input_tokensgpt-5.2-codexBatch and File user level permissionsaretrieve_batch call typexai chat completionsweb search optionsanthropic gemini cache translationaspectRatio mappingvllm embedding formatprompt-caching-scope-2026-01-05 headerCustomLogger async_post_call_response_headers_hookPrometheusServicesLogger REGISTRY.collect()