●Change8

v1.81.13.dev1

📦 litellmView on GitHub →
✨ 15 features🐛 32 fixes🔧 21 symbols

Summary

This release introduces new features like IBM watsonx.ai rerank support, Bedrock structured output API, and performance improvements in routing logic. Numerous bug fixes address issues across proxy headers, metrics counting, model parameter handling, and CI/testing stability.

Migration Steps

  1. If using PodLockManager, note that the atomic compare-and-delete in release_lock was reverted (PR #21469), implying the previous implementation might have caused issues or was incomplete.

✨ New Features

  • Added configurable fallback for generic guardrail endpoint connection failures.
  • Support native structured outputs API (outputConfig.textFormat) for Bedrock.
  • Add IBM watsonx.ai rerank support.
  • Add deployment affinity routing callback.
  • Add mapping for websearch from v1/messages to chat/completions.
  • Add 'reasoning' field to 'reasoning_content' field in delta.
  • [Feat] Add duckduckgo as search tool.
  • Add File deletion criteria with batch references.
  • [Feat]Add day 0 claude sonnet 4.6 feat support.
  • Add 'team' tag to logs, metrics, and cost management for Datadog integration.
  • Add version in claude-code-beta-headers-incident.
  • Add aviation and UAE policy templates with tag-based filtering.
  • [Feat] UI - Add CSV dataset upload to compliance playground.
  • Add supoort for context-1m-2025-08-07.
  • [Feature] UI - Models & Endpoints: Add Model Settings Modal.

🐛 Bug Fixes

  • Preserve metadata for custom callbacks on codex/responses path.
  • Prevent double-counting of litellm_proxy_total_requests_metric.
  • Return finish_reason='tool_calls' when response.completed contains function_call items for responses-api.
  • Fix master key rotation Prisma validation errors in proxy.
  • Make vector stores migration idempotent.
  • Preserve and forward OAuth Authorization headers through proxy layer.
  • Make PodLockManager.release_lock atomic compare-and-delete (Note: This was reverted later).
  • Preserve provider_specific_fields from proxy responses.
  • Add missing parameters to OPENAI_CHAT_COMPLETION_PARAMS.
  • Fix Bedrock service_tier cost propagation.
  • Add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS.
  • Avoid O(n) alias scan for non-alias get_model_list lookups.
  • Fix /v1/models returning wildcard instead of expanded models for BYOK team keys.
  • Broaden Nova 2 model detection to support all nova-2-* variants for Bedrock.
  • Fix mock test.
  • Restore proxy_server module attrs after test_proxy_admin_expired_key_from_cache.
  • Add prisma generate step to matrix CI workflow.
  • Resolve merge conflict in test_vertex_ai_rerank_transformation.py.
  • Use prisma.Json for JSON fields in _rotate_master_key create_many().
  • Add inference_geo to model prices JSON schema validator.
  • Guard against None metadata in prometheus metrics.
  • Restore litellm.model_cost after reload endpoint test.
  • Force-reinstall enterprise package to override PyPI version in CI.
  • Resolve MCP test isolation failures in parallel execution.
  • Restore default_internal_user_params instead of delattr-ing it.
  • Fix middleware and logging bottlenecks to improve streaming proxy throughput.
  • Install enterprise package into main project venv, not enterprise's own venv.
  • Allow internal_user_viewer to access RAG endpoints; restrict ingest to existing vector stores.
  • Preserve SSO role regardless of role_mappings config.
  • Fix CI failures - missing env key doc + streaming test.
  • Prevent sys.modules["langfuse"] import failures in langfuse unit tests.
  • Add = None defaults to Optional[str] fields in managed table models.

Affected Symbols

PodLockManagerOPENAI_CHAT_COMPLETION_PARAMSlitellm_proxy_total_requests_metricresponse.completedPrisma validationvector stores migrationOAuth Authorization headersbedrock structured outputs API (outputConfig.textFormat)IBM watsonx.ai rerankprovider_specific_fieldsusage-based routing v2team deployment filtermodel list lookupsnova-2-* variantsnova/ and nova-2/ spec prefixeswebsearch mappingreasoning_contentvLLM Embeddings encoding_formatlangfuse unit testssys.modules["langfuse"]managed table models Optional[str] fields