v1.98.0
📦 litellmView on GitHub →
✨ 11 features🐛 52 fixes🔧 21 symbols
Summary
This release introduces enhanced Docker image security with cosign signature verification and adds several new features including structured logging, per-deployment overrides, and vector store index management. It also includes numerous bug fixes across various components like Bedrock, proxy, and UI.
Migration Steps
- Verify Docker image signature using cosign and the pinned commit hash or release tag.
✨ New Features
- Added opt-in session_id and trace_id correlation to JSON log records via contextvars.
- Introduced per-deployment allowed_fails_policy and cooldown_time override support in the router.
- Enabled configuration of provisioned-throughput flat cost on a model deployment.
- Added daily rollup writes for per-model PTU flat cost based on active hour.
- Surfaced PTU flat cost on the daily activity read path.
- Added PTU inputs to the model form and flat cost to the Usage page.
- Gated PTU flat-cost attribution behind an opt-in environment variable.
- Implemented configurable estimated output tokens per key, team, and model for rate limiting.
- Added GET /v1/indexes endpoint to list vector store indexes.
- Added UI display for vector store indexes on the Vector Stores page.
- Added per-deployment keepalive_seconds SSE heartbeat to prevent load-balancer timeout on long streams.
🐛 Bug Fixes
- Dropped toolSpec.strict for Claude Sonnet 5 on Converse.
- Attribute Vertex passthrough batch cost to key/team/tags.
- Scoped file list pagination cursors to the caller.
- Skipped prisma-dependent hooks when no database is attached.
- Reported has_more false on caller-scoped file list pages.
- Restored management_v1 query-param validation under fastapi>=0.140.7.
- Stopped /{provider}/v1/files from capturing /openai_passthrough.
- Kept batch fallbacks inside the model group that owns the file.
- Corrected price dict-shaped image input token details at the image rate.
- Refreshed deprecation dates, corrected xAI pricing, and added missing provider models.
- Hid admin-only Logs tabs from roles that cannot call their endpoints.
- Guarded management_v1 against fastapi names removed in supported releases.
- Gated policy and prompt lookups on an admin capability.
- Isolated guardrail load failures per row.
- Gated organization and agent usage views behind capabilities.
- Performed atomic budget cascade with chunked reset scans.
- Treated SAML as configured in UI SSO detection.
- Rejected Anthropic server-side web_search tool with actionable error.
- Opened the classifier prompt editor above the edit auto-router form.
- Traced MCP tool calls instead of crashing on CallToolResult.
- Scoped Virtual Keys and Logs team lists to the caller.
- Gated the Old Usage page behind a proxy-admin capability.
- Cooled down failed fallback deployments and corrected cooldown TTL after Redis backfill.
- Wrote each daily spend batch in one upsert statement.
- Gated four sidebar pages on the roles their endpoints allow.
- Restored the Logs Deleted Teams tab for organization admins.
- Stopped leaking interception control fields to providers.
- Enabled native structured output for GLM 5 and DeepSeek V3.2.
- Scoped file list pagination cursors to the caller.
- Reported has_more false on caller-scoped file list pages.
- Restored management_v1 query-param validation under fastapi>=0.140.7.
- Stopped /{provider}/v1/files from capturing /openai_passthrough.
- Kept batch fallbacks inside the model group that owns the file.
- Corrected price dict-shaped image input token details at the image rate.
- Refreshed deprecation dates, corrected xAI pricing, and added missing provider models.
- Hid admin-only Logs tabs from roles that cannot call their endpoints.
- Guarded management_v1 against fastapi names removed in supported releases.
- Gated policy and prompt lookups on an admin capability.
- Isolated guardrail load failures per row.
- Gated organization and agent usage views behind capabilities.
- Performed atomic budget cascade with chunked reset scans.
- Treated SAML as configured in UI SSO detection.
- Rejected Anthropic server-side web_search tool with actionable error.
- Opened the classifier prompt editor above the edit auto-router form.
- Traced MCP tool calls instead of crashing on CallToolResult.
- Scoped Virtual Keys and Logs team lists to the caller.
- Gated the Old Usage page behind a proxy-admin capability.
- Cooled down failed fallback deployments and corrected cooldown TTL after Redis backfill.
- Wrote each daily spend batch in one upsert statement.
- Gated four sidebar pages on the roles their endpoints allow.
- Restored the Logs Deleted Teams tab for organization admins.
- Stopped leaking interception control fields to providers.