litellm_pro-mgmnt-dev-v1.81.13
📦 litellmView on GitHub →
✨ 13 features🐛 29 fixes🔧 6 symbols
Summary
This release introduces a new Prompt Management API, adds support for several new models and compliance features like EU AI Act templates, and includes numerous performance improvements and bug fixes across routing, proxy, and testing infrastructure.
Migration Steps
- If using tests that rely on logging capture in parallel execution contexts, note that caplog has been replaced with a custom handler.
✨ New Features
- Introduced a new Prompt Management API to interact with Prompt Management integrations.
- Added a configurable fallback mechanism for generic Guardrails to handle endpoint connection failures.
- Implemented guardrail tracing UI showing policy, detection method, and match details.
- Added support for Bedrock native structured outputs API using outputConfig.textFormat.
- Added new model aliases: github_copilot/gpt-5.3-codex and github_copilot/claude-opus-4.6-fast.
- Added support for IBM watsonx.ai rerank.
- Added support for OpenAI Evals API.
- Added Databricks to supported LLM providers for response schema.
- Added support for Claude Sonnet 4.6 pricing and Day 0 support for Claude Sonnet 4.6.
- Added EU AI Act Article 5 policy template for prohibited practices detection.
- Added GDPR Art. 32 EU PII Protection Policy Template.
- UI update: Content Filters now allow editing/viewing categories, 1-click adding categories, and navigating to the next page.
- UI update: Added filtering by User in the Usage section.
🐛 Bug Fixes
- Fixed exclusion of tool parameters for models that do not support function calling.
- Preserved metadata for custom callbacks on codex/responses path.
- Fixed handling of missing DATABASE_URL in proxy's append_query_params function.
- Reverted StreamableHTTPSessionManager to stateless mode in MCP.
- Prevented double-counting of the litellm_proxy_total_requests_metric.
- Fixed finish_reason='tool_calls' when response.completed contains function_call items in responses-api.
- Corrected OCI Grok output pricing.
- Fixed au.anthropic.claude opus 4 6 v1 issues.
- Fixed Prisma validation errors related to master key rotation in the proxy.
- Made vector stores migration idempotent.
- Fixed preservation and forwarding of OAuth Authorization headers through the proxy layer.
- Fixed preservation of provider_specific_fields from proxy responses.
- Fixed Bedrock service_tier cost propagation.
- Added missing OpenAI chat completion parameters to OPENAI_CHAT_COMPLETION_PARAMS.
- Fixed key expiry default duration.
- Fixed /v1/models returning wildcard instead of expanded models for BYOK team keys.
- Normalized encode() return type and handled Hugging Face tokenizer fallback in token-counter.
- Avoided KeyError on missing LAKERA_API_KEY during Lakera guardrail initialization.
- Fixed EU AI Act template by adding the missing category_file path.
- Fixed flaky failure in test_log_langfuse_v2_handles_null_usage_values.
- Fixed test isolation issue in http_handler tests.
- Fixed linting error PLR0915 (too many statements) in route_request.
- Fixed test issues related to parallel execution by replacing caplog with a custom handler.
- Corrected async mock for video generation logging test.
- Added cleanup fixture and no_parallel mark for MCP tests.
- Fixed PodLockManager.release_lock to use atomic compare-and-delete.
- Fixed router performance by removing quadratic deployment scan in usage-based routing v2.
- Fixed router performance by avoiding O(n^2) membership scans in team deployment filter.
- Fixed router performance by avoiding O(n) alias scan for non-alias get_model_list lookups.