●Change8

v1.80.9.dev6

📦 litellmView on GitHub →
✨ 22 features🐛 29 fixes🔧 24 symbols

Summary

This release adds extensive model support, new UI and agent gateway features, and numerous bug fixes and performance improvements across routing, authentication, and API handling.

Migration Steps

  1. Update docker‑compose.yml healthcheck command to use python instead of wget.
  2. Adjust credential encryption configuration so only database credentials are encrypted.
  3. If using AI Gateway Auth, update public route definitions to use wildcard patterns where needed.
  4. Switch server startup to Gunicorn when max_requests_before_restart is configured.
  5. Modify any custom tool_calls payloads to include the required structure expected by the updated responses handling.
  6. Review SSO integration and ensure role upsert logic is compatible with the new UI SSO endpoint.

✨ New Features

  • Add Cerebras zai-glm-4.6 model support
  • Add Bedrock writer models
  • Add 227 new Fireworks AI models
  • Add UI Settings feature
  • Propagate Langfuse trace_id
  • Add new container API file management with UI Interface
  • Add Voyage rerank API support
  • Add Arize Phoenix OSS – Prompt Management Integration
  • Add nested field removal support to additional_drop_params
  • Add computer use support for Gemini
  • Add DeepSeek-V3.2 model support
  • Allow root path to redirect when docs are not on root path
  • Expose team alias when authenticating
  • Agent Gateway – add cost per query for agent invocations
  • Agent Gateway – add token counting for non‑streaming and streaming
  • Add new provider – LangGraph
  • A2a gateway – add cost‑per‑token pricing
  • A2a gateway – support Bedrock AgentCore and LangGraph agents
  • UI SSO – allow fetching role from generic SSO provider (Keycloak)
  • Agent Gateway – track agent_id in SpendLogs
  • Agent Gateway – allow adding LangGraph and Bedrock AgentCore agents
  • Add Mistral Codestral 2508 and Devstral 2512 models

🐛 Bug Fixes

  • Swap URL Input and Display Name inputs
  • AI Gateway Auth – allow wildcard patterns for public routes
  • Router – handle tools=None in completion requests
  • Azure AI Anthropic – fix api‑key header and passthrough cost calculation
  • Responses – prevent streaming tool_calls from being dropped when text + tool_calls are present
  • OpenAI – include 'user' param for responses API models
  • Docker‑compose healthcheck – use python instead of wget
  • Cerebras – fix context window errors not recognized
  • DashScope – fix default base_url error
  • UI session – resolve MCP permissions across real teams
  • Deprecation banner – show only on /sso/key/generate
  • Azure – fix encoding video id error
  • SpendLogs – reduce memory accumulation
  • Credential encryption – affect only DB credentials
  • Unauthorized routes – return 403 instead of 503
  • Containers API – allow LIST and Create containers using custom‑llm‑provider
  • OpenAI flex pricing – extract service_tier from response/usage
  • Videos API – use litellm params for all video endpoints
  • Gemini – handle context errors
  • Anthropic – preserve server_tool_use and web_search_tool_result in multi‑turn conversations
  • Docs – remove streaming logging documentation
  • Supported Endpoints page – documentation fix
  • Guardrails – mask all regex pattern matches in content filter
  • UI SSO – allow upserting user role when SSO provider role changes
  • Tool calls – avoid error when only tool_calls are present in input
  • Nvidia and Gemini tests – fix failures
  • Server process – suggest Gunicorn instead of uvicorn when using max_requests_before_restart
  • Videos content – respect DB credentials
  • Responses API – apply TPM rate limit correctly

🔧 Affected Symbols

router.handleresponses.streamingopenai.user_paramazure.ai_anthropicdashscope.base_urlui.session_mcpcredential_encryptionhealthcheckcontainers_apiservice_tier_extractorvideos_apigemini.context_handleranthropic.server_tool_useguardrails.content_filterui_sso.role_upsertmax_requests_before_restartagent_gateway.cost_per_queryagent_gateway.token_countinga2a_gateway.pricinglanggraph.integrationcerebras.context_windowdeepseek_v3_2mistral_codestraldevstral