●Change8

v1.80.7.dev.4

Breaking Changes
📦 litellmView on GitHub →
⚠ 1 breaking✨ 18 features🐛 25 fixes🔧 9 symbols

Summary

This release adds extensive new features—including Guardrail tool‑call checks, OTEL integration, new model support, and UI enhancements—while fixing numerous bugs and introducing a breaking change that renames `output_tokens_details` to `completion_tokens_details`.

⚠️ Breaking Changes

  • The responses API field `output_tokens_details` is now mapped to `completion_tokens_details`, breaking code that expects the old field name.

Migration Steps

  1. Update any code that reads `output_tokens_details` from the responses API to use the new `completion_tokens_details` field instead.

✨ New Features

  • Guardrail API now supports tool call checks on OpenAI `/chat/completions`, OpenAI `/responses`, and Anthropic `/v1/messages`.
  • Basic Weave OTEL integration added.
  • Auto redirect to SSO on the new login page.
  • UI credentials handling switched to React Query.
  • Support added for Bedrock Qwen 2 imported model.
  • File content download support for Bedrock batches.
  • New model `amazon.titan-image-generator-v2:0` added.
  • Routing support limited to deployments that support websearch.
  • Admin viewers can now access global tag usage via management endpoints.
  • Agent Access Control: enforce allowed agents by key/team and add agent access groups on backend.
  • UI Agent Gateway: set allowed agents by key/team.
  • Ability to add OpenAI‑compatible chat providers using a .json file and inclusion of a public AI provider.
  • Guardrails API introduces a new `structured_messages` parameter.
  • VertexAI now supports BGE embeddings.
  • Cursor BYOK support with its own configuration added.
  • Background health checks added to the database layer.
  • Detailed debugging for reference enabled.
  • All callbacks are now displayed on the UI.

🐛 Bug Fixes

  • Added clear error message for empty LLM endpoint responses (OpenAI).
  • Skipped empty text blocks in Anthropic system messages.
  • Added missing `ocr` and `aocr` to `CallTypes` enum.
  • Checked `supports_response_schema` before using json_tool_call workaround for Groq.
  • Added missing Moonshot turbo models and corrected pricing.
  • Bedrock converse messages now respect `modify_params` as expected.
  • Fixed default API base test for ragfow.
  • Fixed Aim guardrail tests.
  • Resolved Ruff lint errors.
  • Handled empty response and VLLM streaming edge cases.
  • Mask all matching keywords in content filter for guardrails.
  • Optimized SpendLogs queries to use timestamp filtering for index usage.
  • Handled partial JSON chunks after the first valid chunk for Gemini.
  • Supported `during_call` event type in unified guardrails.
  • Fixed selection issue in Edit Membership modal.
  • Addressed code quality issues from Ruff linter.
  • Fixed Presidio guardrail test TypeError and license base64 decoding error.
  • Parsed `<budget:thinking>` blocks for Opus 4.5.
  • Fixed vector store configuration synchronization failure.
  • Added None value checks and updated public AI chat transformation tests.
  • Removed hardcoded API key from test file.
  • WatsonX audio transcriptions no longer force content type in request headers.
  • Budget update now allows null `max_budget`.
  • Fixed failing Vertex tests.
  • Ensured media resolution is applied only for Gemini 3 model.

🔧 Affected Symbols

CallTypesunified_guardrail.pyoutput_tokens_detailscompletion_tokens_detailsGuardrails APISpendLogscursor BYOK configurationAgent Access ControlWeave OTEL integration