●Change8

v1.82.dev2

📦 litellmView on GitHub →
✨ 10 features🐛 43 fixes⚡ 1 deprecations🔧 7 symbols

Summary

This release focuses heavily on bug fixes across numerous providers (Gemini, Anthropic, OpenAI, Azure, etc.), enhances model tracking and cost mapping, and introduces performance improvements for streaming latency. It also adds new UI features for project management and access control.

Migration Steps

  1. If you were relying on the structure of `ModelResponse` containing `StreamingChoices`, update your code to use `ModelResponseStream` instead.

✨ New Features

  • Added gemini-3.1-flash-image-preview model to the Vertex AI database.
  • Added v1 for anthropic responses transformation.
  • Enabled local file support for OCR.
  • Added Access group CRUD functionality with bidirectional team/key synchronization.
  • Added gpt-audio-1.5 to the model cost map.
  • Added gpt-realtime-1.5 to the model cost map.
  • Added OpenRouter native models to the model cost map.
  • Added Projects page to the UI with list and create flows.
  • Added in_flight_requests metric to /health/backlog endpoint and Prometheus.
  • Added Agent RBAC Permission Fix: Ensure Internal Users cannot create agents.

🐛 Bug Fixes

  • Preserved $ref in JSON Schema for Gemini 2.0+.
  • Moved duration to _hidden_params to match OpenAI response spec for transcription.
  • Mapped reasoning_effort to output_config for Claude 4.6 models.
  • Fixed converse handling for parallel_tool_calls.
  • Preserved forwarding server side called tools.
  • Fixed free models working from UI.
  • Added ChatCompletionImageObject in OpenAIChatCompletionAssistantMessage.
  • Fixed poetry lock.
  • Stripped stale mcp-session-id to prevent 400 errors across proxy workers.
  • Forwarded extra_headers on OpenAI code path in image_generation().
  • Passed model_info/metadata in image_edit for custom pricing.
  • Fixed tool_calls streaming indexes for chatgpt.
  • Corrected supported_openai_params for GPT-5 model family.
  • Corrected supported params for gpt-5-search models.
  • Resolved api_base from env var in Document Intelligence OCR for Azure AI.
  • Enabled function calling for PublicAI Apertus models.
  • Passed through native Gemini imageConfig params for image generation in vertex_ai.
  • Used provider-reported usage in streaming without stream_options for openrouter.
  • Preserved image_url blocks in multimodal messages for moonshot.
  • Removed StreamingChoices from ModelResponse, using ModelResponseStream instead.
  • Used output_index for parallel tool call streaming indices in responses.
  • Fixed Python 3.10 compatibility for OpenAIContainerConfig.
  • Included system and tools in token counting API requests.
  • Added Gemini and Vertex AI support to HeliconeLogger.
  • Handled openrouter models without '/' in name during register_model.
  • Fixed double-stripping of model names with provider-matching prefixes in adapter.
  • Included timestamps in /project/list response.
  • Fixed Claude code plugin schema.
  • Updated realtime guardrail test assertions for voice violation behavior.
  • Updated Azure pass-through test after Responses API routing change.
  • Added missing migration for LiteLLM_ClaudeCodePluginTable.
  • Restored parallel_tool_calls mapping in map_openai_params for bedrock.
  • Updated test mocks for renamed filter_server_ids_by_ip_with_info in mcp.
  • Added PROXY_ADMIN role to system user for key rotation.
  • Populated user_id and user_info for admin users in /user/info.
  • Stored task references in LLMClientCache._remove_key.
  • Propagated extra_headers to Upstream in image_generation.
  • Passed MCP auth headers from request into tool fetch for /v1/responses and chat completions.
  • Shortened guardrail benchmark result filenames for Windows long path support.
  • Defaulted available_on_public_internet to true in mcp.
  • Filtered internal json_tool_call when mixed with real tools in bedrock.
  • Fixed OIDC discovery URLs, roles array handling, and dot-notation error hints in jwt.
  • Implemented 4 targeted hot-path fixes for streaming latency improvements.

Affected Symbols

ChatCompletionImageObjectModelResponseModelResponseStreamStreamingChoicesLLMClientCacheOpenAIContainerConfigfilter_server_ids_by_ip_with_info

⚡ Deprecations

  • Deprecation dates added for xAI grok-2-vision-1212 and grok-3-mini models.
litellm v1.82.dev2 - Change8