v1.96.0-dev.1
Breaking Changes📦 litellmView on GitHub →
⚠ 1 breaking✨ 10 features🐛 31 fixes🔧 19 symbols
Summary
This release introduces Docker image signature verification using cosign and includes numerous bug fixes and feature enhancements across various components like Prometheus, aiohttp, Helm, Vertex AI, Anthropic adapter, and the UI. Notably, the ai-gateway handler has been removed in favor of litellm-core.
⚠️ Breaking Changes
- The `ai-gateway` handler has been removed. The `litellm-core` module is now the callable messages() SDK. Users previously relying on the `ai-gateway` handler will need to migrate to using `litellm-core` directly.
Migration Steps
- Migrate from the `ai-gateway` handler to using the `litellm-core` module directly for messages() SDK calls.
✨ New Features
- Added `service_tier` label to Prometheus latency and spend metrics.
- Published a generated JSON schema for `model_prices_and_context_window.json`.
- Added UI feature to split failed requests into their own series on the cache dashboard.
- Added UI feature for shareable log links via `log_id` query parameter on the logs page.
- Added UI feature to deep link team detail page via `?team=` query parameter.
- Added UI feature to deep link organization detail page via `?org=` query parameter.
- Added UI feature to link organization teams to their team detail pages.
- Added `qwen3.7-plus` and `qwen3.7-max` to the Dashscope model cost map.
- Added a Grafana dashboard for the OTel GenAI metrics.
- CLI can now read `base_url` from a persistent config file.
🐛 Bug Fixes
- Fixed `aiohttp` to keep keep-alive connector config when a session is rebuilt.
- Pinned bundled postgres and redis to `bitnamilegacy` images in Helm charts.
- Fixed `vertex` adapter to decide `rawPredict` passthrough streaming from the request body.
- Fixed `anthropic-adapter` to translate `stop_sequences` and `disabled thinking` for non-Claude targets.
- Fixed `jwt_auth` to allow `/v1/messages` for JWT teams by default.
- Fixed proxy to skip team model aliases that point at deleted deployments.
- Fixed proxy to report when a model write does not survive the post-write reload.
- Fixed proxy to reject model writes that corrupt an auto-router pseudo-model.
- Fixed router strategy to serialize latency for non-chat responses in lowest-latency routing.
- Fixed logging to preserve callback order in `get_combined_callback_list`.
- Fixed router to honor explicit `priority=0` in `acompletion`.
- Fixed `anthropic-adapter` to open the first content block with the real upstream type so reasoning-first streams start with thinking.
- Fixed SCIM to stop provisioning nested group IDs as internal users.
- Fixed UI to size Object Permissions card grid by container width.
- Fixed proxy to allow `/key/update` to identify the key by `key_alias`.
- Fixed UI to show public model names in usage breakdowns.
- Fixed UI to point the navbar and sidebar logos at the dashboard home route.
- Fixed proxy to stop serving stale team model allowlist after `/team/update`.
- Fixed proxy to resolve named credentials on provider-only batch and files calls.
- Fixed router to drop duplicate `Mapping` import that fails ruff F811.
- Fixed `otel` to make OTLP export work against Grafana Cloud.
- Fixed tests to assert `Content` variants are identified by type, not by the discriminator keyword.
- Fixed `mcp` to never write discovery results to the row, heal already-stamped rows, and retry failed discovery with backoff.
- Fixed `vertex_ai` to honor per-model `gcs_bucket_name` on managed-file read path.
- Fixed auth to resolve managed batch/file deployment `model_id` to model name for team access checks.
- Fixed `vertex_ai` to forward `function_call` ID on Vertex Gemini 3+ tool turns.
- Fixed complexity router to capture the classifier request body in spend logs.
- Fixed `managed-files` tests to lock in `store_unified_file_id` idempotency on batch retrieve.
- Fixed `vertex_ai` to skip context caching when the cached block ends on a model turn.
- Fixed bedrock to remove the dead `BedrockLLM` invoke code path.
- Fixed pricing to correct `gpt-5.4-mini` and `gpt-5.4-nano` token limits.