●Change8

v1.89.0-rc.1

📦 litellmView on GitHub →
✨ 6 features🐛 33 fixes🔧 37 symbols

Summary

This release introduces new agent providers (watsonx Orchestrate, LangFlow), enhances proxy features with rate limiting and graceful shutdown, and includes numerous fixes across providers, testing infrastructure, and OpenTelemetry tracing.

Migration Steps

  1. If using Datadog integration, be aware that oversized batches will now result in a 413 error instead of infinite re-queueing.

✨ New Features

  • Added support for watsonx Orchestrate agent provider (A2A).
  • Added LangFlow agent provider with A2A session bridging.
  • Added per-MCP-server RPM rate limiting for keys and teams in the proxy.
  • Added native /health/drain preStop hook for graceful shutdown in the proxy.
  • Implemented routing for future Claude models to the Anthropic provider via pattern matching.
  • Added Arize/Phoenix OpenInference rendering parity for tool_calls and content.

🐛 Bug Fixes

  • Bumped deprecated gemini-3-pro-preview to gemini-3.1-pro-preview in test responses.
  • Mapped mistral/ministral-8b-latest in the model price map.
  • Split oversized Datadog batches on HTTP 413 instead of re-queueing indefinitely.
  • Stopped the use_chat_completions_api flag from leaking into the provider request body.
  • Inlined legacy $ref definitions in Anthropic and Fireworks tool schemas.
  • Omitted OpenAI [DONE] marker on google-genai streamGenerateContent responses.
  • Fixed vector store support for Vertex AI Search by allowing engines URL.
  • Ensured caller-supplied filter options are rendered in caller order in the UI.
  • Skipped unnecessary batch input file reads.
  • Cleared allowed_tools and tool overrides on MCP server edit.
  • Fixed CircleCI rerun-failed-tests to correctly collect tests when 2+ test files fail.
  • Stripped tool-level extra fields on HTTP 400 errors for Azure AI and retried.
  • Removed fixed dimensions from the README hero image.
  • Fixed small nits in CLAUDE.md documentation.
  • Added MCP semantic conventions to otelv2.
  • Emitted otel guardrail span when a guardrail blocks a passthrough request.
  • Stripped NUL bytes from spend log payloads to prevent PostgreSQL 22P05 errors.
  • Supported MCP OAuth passthrough and issuer-scoped JWT auth.
  • Forwarded per-request parameters to Vertex AI Search for vector stores.
  • Dropped module-level test calls that broke local_testing collection.
  • Dropped import-time completion call in test_register_model.
  • Used user-supplied api_base as is for Vertex AI Model Garden OpenAI-compat path.
  • Preserved HTTP 401 status for expired JWTs in OTel traces.
  • Captured 401 error details in management endpoint spans for OTel.
  • Fixed missing span for guardrail passthrough.
  • Fixed missing MCP OpenTelemetry attributes.
  • Resolved managed video model IDs for authentication in the proxy.
  • Allowed team members to create keys on org-scoped teams.
  • Moved Gemini pass-through tests to use gemini-3.1-flash-lite.
  • Emitted thinking block for reasoning_content-only streaming chunks in Anthropic adapter.
  • Stripped output_config.effort for Vertex AI Claude models that reject it (Haiku 4.5).
  • Fixed A2A bugs identified in staging.
  • Stabilized batch VCR coverage and stopped live upload/network leaks in tests.

Affected Symbols

gemini-3-pro-previewgemini-3.1-pro-previewmistral/ministral-8b-latestteam_metadatause_chat_completions_apitool schemas (Anthropic, Fireworks)OpenAI [DONE]google-genai streamGenerateContentVertex AI Searchallowed_toolstool overridesCircleCI rerun-failed-testsAzure AIREADME hero imageCLAUDE.mdMCP semantic conventionsguardrail spanspend log payloadsNUL bytesMCP OAuthJWT authVertex AI Search paramsLangFlow agent providerA2A session bridgingA2A skill tagsClaude models routingAnthropic providertest_register_modelVertex AI Model Garden OpenAI-compat path/health/drainJWTsmanagement endpoint spansVertex Claude modelsoutput_config.effortgemini-3.1-flash-litereasoning_content-only streaming chunksMAX_PATH