●Change8

v1.88.0-dev.1

📦 litellmView on GitHub →
✨ 10 features🐛 24 fixes🔧 25 symbols

Summary

This release focuses heavily on stability, security, and feature parity across various providers, including Day 0 support for Gemini 3.5 Flash and enhanced proxy/UI capabilities. It also introduces Docker image signature verification using cosign.

Migration Steps

  1. Users running Docker images should verify image signatures using the provided cosign commands (pinned commit hash recommended).

✨ New Features

  • Added support for Gemini 3.5 Flash (Day 0 support).
  • Added support for Gemini managed agents.
  • Added gemini-3.1-flash-lite model cost map.
  • Added user_email and user_alias to Prometheus user budget metrics.
  • Added Interactions API endpoint to playground with SSE streaming.
  • Propagate team_id and team_alias to all child OTEL spans.
  • Added team passthrough routes creation parity and edit load fix in UI.
  • Added native MCP OAuth support for cursor.
  • Migrated Interactions to Google Interactions API steps schema (May 2026).
  • Added granian as an ASGI compliant web server for better throughput stability.

🐛 Bug Fixes

  • Proxy admins can now gate team allowed_passthrough_routes.
  • Stabilized image-edit VCR cassettes to stop live gpt-image-1 spend.
  • Fixed Bedrock/Cohere to send embedding_types as JSON array, not string.
  • Migrated realtime + rerank tests off shut-down upstream models.
  • Fixed caching to replay openai/responses bridge cache hits as chat streams.
  • Sanitized batch metadata in Bedrock to prevent Pydantic ValidationError.
  • Fixed Deepseek to use native /anthropic/v1/messages endpoint and sanitize tools.
  • Fixed proxy to decode bytes and pass-through SSE for Google-native streamGenerateContent.
  • Replaced shut-down gpt-4o-audio-preview with gpt-audio-1.5 in tests.
  • Seeded Redis counter via SET NX in spend_counter to prevent cross-pod double-seed.
  • Normalized batch file IDs before ManagedObjectTable write in proxy.
  • Fixed router to use forwarded model_id for native Azure container IDs.
  • Restored log filter loading indicator in UI.
  • Fixed JWT on tools/list and REST tools/call server resolution in MCP.
  • Fixed Vertex AI to omit function_call id on Vertex Gemini 3.5+ tool turns.
  • Fixed Interactions to never drop streamed text deltas; always emit terminal completion.
  • Exposed Prisma idle/connect timeout + extra DB URL params in proxy.
  • Serialized guardrail_response to JSON in OTEL traces.
  • Expected session.created as xAI realtime initial event in tests.
  • Fixed proxy to hydrate wildcard discovery credentials.
  • Fixed Vertex Gemma to strip `context_management` from request body.
  • Recalculated cost after router retry failures in logging.
  • Fixed OTEL to emit guardrail span on violation, surface status + categories.
  • Fixed Docker to restore npm to non_root builder image.

Affected Symbols

bedrock/coheregpt-image-1realtimererankopenai/responsesbedrocksagemakergpt-4o-audio-previewgpt-audio-1.5deepseek/anthropic/v1/messagesGoogle-native streamGenerateContentRedis counterManagedObjectTableAzure container IDsMCPVertex Gemini 3.5+PrismaOTEL tracesxAI realtimevertex_gemmaroutergoogle maps grounding testnpmgranian