●Change8

v1.89.0

📦 litellmView on GitHub →
✨ 8 features🐛 33 fixes🔧 20 symbols

Summary

This release introduces new agent providers (Watsonx, LangFlow), adds per-MCP-server rate limiting, and includes numerous fixes across integrations like Datadog, Vertex AI, and Azure AI, alongside improved Docker image signing verification.

Migration Steps

  1. If you rely on the `gemini-3-pro-preview` model name, update your configuration to use `gemini-3.1-pro-preview` instead.

✨ New Features

  • Added support for Watsonx Orchestrate agent provider (A2A).
  • Added LangFlow agent provider with A2A session bridging.
  • Added per-MCP-server RPM rate limiting for keys and teams in the proxy.
  • Added native /health/drain preStop hook for graceful shutdown in the proxy.
  • Added support for forwarding per-request parameters to Vertex AI Search.
  • Added support for MCP OAuth passthrough and issuer-scoped JWT auth.
  • Route future Claude models to Anthropic provider via pattern matching.
  • Added Arize/Phoenix OpenInference rendering parity.

🐛 Bug Fixes

  • Bumped deprecated gemini-3-pro-preview to gemini-3.1-pro-preview in tests.
  • Mapped mistral/ministral-8b-latest in the model price map.
  • Fixed Datadog integration: split oversized batches on 413 instead of re-queueing forever.
  • Stopped use_chat_completions_api flag from leaking into provider request body.
  • Inlined legacy $ref defs in tool schemas for Anthropic and Fireworks.
  • Omitted OpenAI [DONE] marker on google-genai streamGenerateContent responses.
  • Enabled support for engines URL for Vertex AI Search vector stores.
  • Rendered caller-supplied filter options in caller order in the UI.
  • Skipped unnecessary batch input file reads.
  • Cleared allowed_tools and tool overrides on MCP server edit.
  • Fixed CircleCI rerun-failed-tests to collect tests when 2+ test files fail.
  • Stripped tool-level extra fields on 400 errors and retried for Azure AI.
  • Fixed small nits in CLAUDE.md documentation.
  • Added MCP semantic conventions to otelv2.
  • Emitted otel guardrail span when a guardrail blocks a passthrough request.
  • Stripped NUL bytes from spend log payloads to prevent PostgreSQL 22P05 errors.
  • Preserved 401 status for expired JWTs in OTel traces.
  • Captured 401 error details in management endpoint spans.
  • Fixed missing span for guardrail passthrough.
  • Allowed internal users to view search tools.
  • Fixed missing mcp otel attributes.
  • Resolved managed video model ids for authentication in the proxy.
  • Allowed team members to create keys on org-scoped teams.
  • Moved Gemini pass-through tests to use gemini-3.1-flash-lite.
  • Emitted thinking block for reasoning_content-only streaming chunks in Anthropic adapter.
  • Stripped output_config.effort for Vertex AI Claude models that reject it (Haiku 4.5).
  • Fixed A2A bugs.
  • Improved LiteLLM websocket functionality.
  • Fixed import-time completion call in test_register_model to prevent test collection issues.
  • Dropped module-level test calls that break local_testing collection.
  • Stabilized batch VCR coverage and stopped live upload/network leaks in tests.
  • Used user-supplied api_base as is for Model Garden OpenAI-compat path in Vertex AI.
  • Fixed A2A skill tags to be enterable and validated in the UI.

Affected Symbols

gemini-3-pro-previewgemini-3.1-pro-previewmistral/ministral-8b-latestuse_chat_completions_apitool schemas (Anthropic, Fireworks)streamGenerateContent (google-genai)Vertex AI Search (vector-stores)allowed_tools (MCP)tool overrides (MCP)watsonx Orchestrate (A2A)azure_aiCLAUDE.mdotel (MCP semantic conventions)guardrail spanspend log payloads (PostgreSQL)Vertex AI (Model Garden OpenAI-compat path)Claude models (Vertex AI)LangFlow (A2A)A2A skill tagstest_register_model