●Change8

v1.88.0

📦 litellmView on GitHub →
✨ 8 features🐛 23 fixes🔧 23 symbols

Summary

This release focuses heavily on stability, bug fixes across various providers (Bedrock, Cohere, Deepseek, Vertex AI), and significant feature additions including Gemini 3.5 Flash support and enhanced Prometheus/OTEL metrics. It also introduces new Docker image signing verification procedures.

Migration Steps

  1. If using bedrock/cohere, ensure embedding_types are sent as a JSON array, not a string.
  2. If using Vertex AI with Gemini 3.5+, ensure function_call id is omitted on tool turns.

✨ New Features

  • Added support for Gemini 3.5 Flash.
  • Added support for Gemini managed agents.
  • Added gemini-3.1-flash-lite model cost map.
  • Propagate team_id and team_alias to all child OTEL spans.
  • Added Interactions API endpoint to playground with SSE streaming.
  • Added team passthrough routes create parity + edit load fix in UI.
  • Allow native MCP OAuth support for cursor.
  • Add granian as an ASGI compliant web server for better throughput stability.

🐛 Bug Fixes

  • Gate team allowed_passthrough_routes to proxy admins.
  • Stabilize image-edit VCR cassettes to stop live gpt-image-1 spend.
  • Fix bedrock/cohere to send embedding_types as JSON array, not string.
  • Migrate realtime + rerank tests off shut-down upstream models.
  • Replay openai/responses bridge cache hits as chat streams.
  • Sanitize batch metadata in bedrock to prevent Pydantic ValidationError.
  • Fix deepseek to use native /anthropic/v1/messages endpoint and sanitize tools.
  • Decode bytes and pass-through SSE for Google-native streamGenerateContent.
  • Replace shut-down gpt-4o-audio-preview with gpt-audio-1.5 in tests.
  • Seed Redis counter via SET NX to prevent cross-pod double-seed in spend_counter.
  • Normalize batch file IDs before ManagedObjectTable write in proxy.
  • Use forwarded model_id for native Azure container IDs in router.
  • Restore log filter loading indicator in UI.
  • Fix JWT on tools/list and REST tools/call server resolution in mcp.
  • Fix vertex_ai to omit function_call id on Vertex Gemini 3.5+ tool turns.
  • Never drop streamed text deltas; always emit terminal completion in interactions.
  • Expose Prisma idle/connect timeout + extra DB URL params in proxy.
  • Serialize guardrail_response to JSON in OTEL traces.
  • Hydrate wildcard discovery credentials in proxy.
  • Fix vertex_gemma to strip `context_management` from request body.
  • Recalculate cost after router retry failures.
  • Emit guardrail span on violation, surface status + categories in otel.
  • Tolerate transient 500 in google maps grounding test in vertex_ai.

Affected Symbols

bedrock/coheregpt-image-1realtimererankopenai/responsesbedrocksagemakergpt-4o-audio-previewgpt-audio-1.5deepseek/anthropic/v1/messagesGoogle-native streamGenerateContentRedis counterManagedObjectTableAzure container IDsMCPVertex Gemini 3.5+PrismaOTEL tracesVertex Gemmagoogle maps grounding testnpmblack