●Change8

v1.78.0-nightly

📦 litellm
✨ 13 features🐛 15 fixes🔧 2 symbols

Summary

This release introduces support for new Azure AI, Vertex AI Gemma, and OCI Cohere models, alongside EnkryptAI Guardrails integration. Numerous bug fixes focus on session management, rate limiting, and UI stability.

Migration Steps

  1. If you rely on ResponsesAPIResponse parsing behavior that is now relaxed, review your custom backend integration.
  2. If you were using shared sessions, verify correct parsing and usage after session handling fixes.

✨ New Features

  • Added new Azure AI models with pricing details.
  • Added EnkryptAI Guardrails integration.
  • Support for Vertex AI Gemma Models on Custom Endpoints.
  • Optimized SSL/TLS handshake performance with prioritized cipher.
  • Added new together models.
  • Implemented posthog per request api key.
  • Implemented Shared Health Check State Across Pods.
  • Added W&B Inference documentation.
  • Added OCI Cohere support with tool calling and streaming capabilities.
  • Added Bedrock AU Cross-Region Inference for Claude Sonnet 4.5.
  • Added VertexAI Gemma model family streaming support and MedGemma support.
  • Included server_name in /v1/mcp/server/health endpoint response.
  • Added support for setting tag based budgets.

🐛 Bug Fixes

  • Fixed dynamic Rate limiter v3 - inserting litellm_model_saturation.
  • Fixed sessions not being shared.
  • Prevents DB from accidentally overriding config file values if they are empty in DB.
  • Temporarily relaxed ResponsesAPIResponse parsing to support custom backends (e.g., vLLM).
  • Fixed OpenRouter cache_control to only apply to last content block.
  • Removed panic from hot path.
  • Ensured MCP client stays open during tool call.
  • Minimal fix: gpt5 models should not go on cooldown when called with temperature!=1.
  • Fixed usage page >> Model Activity >> spend per day graph: y-axis clipping on large spend values.
  • Fixed VertexAI - gemma model family support (custom endpoints).
  • Fixed SensitiveDataMasker converting lists to string.
  • Removed buggy comment in docker-compose that caused config.yaml based startup failure.
  • Fixed shared session parsing and usage issue.
  • Handled closed aiohttp sessions.
  • Prevented session leaks when recreating aiohttp sessions.

🔧 Affected Symbols

ResponsesAPIResponse/v1/mcp/server/health