●Change8

v1.80.8.rc.1

📦 litellm
✨ 23 features🐛 34 fixes⚡ 1 deprecations🔧 25 symbols

Summary

This release adds extensive new features—including Guardrail API enhancements, OTEL integration, new model support, and UI improvements—while delivering numerous bug fixes and minor deprecations.

✨ New Features

  • Guardrail API – support tool call checks on OpenAI `/chat/completions`, OpenAI `/responses`, Anthropic `/v1/messages`
  • Basic Weave OTEL integration
  • Auto redirect to SSO on new login page
  • UI – change credentials handling to use React Query
  • Map `output_tokens_details` of responses API to `completion_tokens_details`
  • Add support for Bedrock Qwen 2 imported model
  • Add support for file content download for Bedrock batches
  • New model `amazon.titan-image-generator-v2:0`
  • Support routing to only websearch‑supported deployments
  • Management endpoints – allow admin viewer to access global tag usage
  • Agent Access Control – enforce allowed agents by key/team and add agent access groups on backend
  • UI – Agent Gateway – set allowed agents by key/team
  • Allow adding OpenAI‑compatible chat providers via .json and add public AI provider
  • Enable detailed debugging for reference
  • Guardrails API – new `structured_messages` parameter
  • VertexAI – add BGE embeddings support
  • Add support for cursor BYOK with its own configuration
  • Add background health checks to the database
  • Show all callbacks on UI
  • Add `gpt-5.1-codex-max` model pricing and configuration
  • Customer (end‑user) usage tracking
  • Allow using dynamic rate‑limit/priority reservation on teams
  • Add Mistral Large 3 model support

🐛 Bug Fixes

  • Add clear error message for empty LLM endpoint responses (OpenAI)
  • Skip empty text blocks in Anthropic system messages
  • Add missing `ocr` and `aocr` to `CallTypes` enum
  • Check `supports_response_schema` before using json_tool_call workaround (Groq)
  • Add missing Moonshot turbo models and fix incorrect pricing
  • Make Bedrock converse messages respect `modify_params` as expected
  • Fix default API base test for RAGFlow
  • Fix Aim guardrail tests
  • Resolve Ruff lint errors
  • Handle empty response and VLLM streaming edge case
  • Mask all matching keywords in content filter (guardrails)
  • Optimize SpendLogs queries to use timestamp filtering for index usage
  • Handle partial JSON chunks after first valid chunk (Gemini)
  • Support `during_call` event type for unified guardrails
  • Fix selection handling in Edit Membership modal
  • Address code‑quality issues from Ruff linter
  • Fix Presidio guardrail test TypeError and license base64 decoding error
  • Parse `<budget:thinking>` blocks for Opus 4.5
  • Fix vector store configuration synchronization failure
  • Ensure media resolution is applied only for Gemini 3 model
  • WatsonX audio transcriptions – don’t force content‑type header
  • Fix budget update to allow `null` `max_budget`
  • Fix failing Vertex tests
  • Preserve `encrypted_content` in reasoning items for multi‑turn GitHub Copilot conversations
  • Fix `apply_guardrail` method and improve test isolation
  • Fix MLflow streaming spans for Anthropic passthrough
  • Dynamic rate limiter – token count now reflects actual usage and Redis TTL fixed
  • Secret managers integration – make email and secret‑manager operations independent in key‑management hooks
  • Use `organization.members` instead of deprecated `organization.organizations`
  • Handle empty content and error‑dict responses in Presidio guardrails
  • Anthropic beta headers fix for Qwen in Bedrock
  • Fix duplicate `reasoning_content` in Gemini streaming responses
  • Add missing None‑value checks and update public‑AI chat transformation tests
  • Remove hard‑coded API key from test file

🔧 Affected Symbols

CallTypesapply_guardrailunified_guardraildynamic_rate_limiterbudget.updateorganization.memberspresidiomlflowvertexbedrockgpt_5_1_codex_maxMistralLarge3amazon.titan_image_generator_v2cursor_BYOKWeaveOTELReactQueryCredentialsAgentAccessControlAgentGatewayOpenAIChatProviderstructured_messagesBGEEmbeddingsSpendLogsGeminiWatsonXSecretManagerHooks

⚡ Deprecations

  • Deprecated organization.organizations attribute – use organization.members instead