v1.98.0-dev.2
📦 litellmView on GitHub →
✨ 11 features🐛 28 fixes🔧 15 symbols
Summary
This release introduces enhanced Docker image verification using cosign and includes numerous bug fixes and new features related to proxy, PTU (Provisioned Throughput Units), and UI improvements. Key updates focus on cost attribution, rate limiting, and better error handling.
Migration Steps
- Verify Docker image signature using cosign with either the pinned commit hash or the release tag.
- Ensure FastAPI version is >= 0.140.7 if using management_v1 query-param validation.
- Opt-in to PTU flat-cost attribution using an environment variable if desired.
✨ New Features
- Configure provisioned-throughput flat cost on a model deployment.
- Daily rollup writes per-model PTU flat cost by active hour.
- Add opt-in session_id and trace_id correlation to JSON log records via contextvars.
- Surface PTU flat cost on the daily activity read path.
- Add per-deployment allowed_fails_policy and cooldown_time override support.
- Add PTU inputs to the model form and flat cost to the Usage page.
- Gate PTU flat-cost attribution behind an opt-in env var.
- Configurable estimated output tokens per key, team and model.
- Add GET /v1/indexes to list vector store indexes.
- Show vector store indexes on the Vector Stores page.
- Add per-deployment keepalive_seconds SSE heartbeat to prevent load-balancer timeout on long streams.
🐛 Bug Fixes
- Attribute Vertex passthrough batch cost to key/team/tags.
- Scope file list pagination cursors to the caller.
- Skip prisma-dependent hooks when no database is attached.
- Report has_more false on caller-scoped file list pages.
- Restore management_v1 query-param validation under fastapi>=0.140.7.
- Stop /{provider}/v1/files from capturing /openai_passthrough.
- Keep batch fallbacks inside the model group that owns the file.
- Enable native structured output for GLM 5 and DeepSeek V3.2 on Bedrock.
- Price dict-shaped image input token details at the image rate.
- Refresh deprecation dates, correct xAI pricing and add missing provider models.
- Hide admin-only Logs tabs from roles that cannot call their endpoints.
- Guard management_v1 against fastapi names removed in supported releases.
- Gate policy and prompt lookups on an admin capability.
- Isolate guardrail load failures per row.
- Gate organization and agent usage views behind capabilities.
- Atomic budget cascade with chunked reset scans.
- Treat SAML as configured in UI SSO detection.
- Reject Anthropic server-side web_search tool with actionable error on Bedrock.
- Open the classifier prompt editor above the edit auto-router form.
- Trace MCP tool calls instead of crashing on CallToolResult.
- Scope Virtual Keys and Logs team lists to the caller.
- Gate the Old Usage page behind a proxy-admin capability.
- Add per-deployment keepalive_seconds SSE heartbeat to prevent load-balancer timeout on long streams.
- Cool down failed fallback deployments and correct cooldown TTL after Redis backfill.
- Gate four sidebar pages on the roles their endpoints allow.
- Restore the Logs Deleted Teams tab for organization admins.
- Stop leaking interception control fields to providers.
- Warn when a deployment's credentials contradict its provider.