●Change8

v1.81.3.rc.5

📦 litellmView on GitHub →
✨ 17 features🐛 32 fixes🔧 2 symbols

Summary

This release introduces significant enhancements to observability (Datadog, Prometheus metrics), adds support for Vercel AI Gateway embeddings, and includes numerous bug fixes across model integrations and UI functionality.

Migration Steps

  1. Add sensitive keywords "vertex_credentials" for Vertex AI LLM credentials configuration.

✨ New Features

  • Added Datadog agent support for LLM Observability.
  • Added Datadog cost management support and fixed a startup callback issue.
  • Added embeddings support for vercel_ai_gateway.
  • Support for role_mappings from environment variables.
  • Added Header-Based Context Propagation for a2a agent.
  • Added guardrail model argument support.
  • Implemented Virtual Keys with Auto Truncation of Table Values in the UI.
  • Added callbacks and labels to Prometheus metrics.
  • Added clientip and user agent in metrics.
  • Added TPM-RPM limit in Prometheus metrics.
  • Added timeout to onyx guardrail.
  • Messages API now supports bedrock converse caching and PDF support.
  • Enabled progress notifications for MCP tool calls.
  • Added configurable CLI JWT expiration via environment variable for CLI Auth.
  • Added UI for /rag/ingest API to upload documents (PDFs, etc.) to create vector stores.
  • Added Light/Dark Mode Switch for Development in the UI.
  • Added model_id label to Prometheus metrics.

🐛 Bug Fixes

  • Fixed Nova grounding issues.
  • Fixed issue with num retries environment variable type.
  • Fixed GPT-4o image-1.5 cost calculation not including output image tokens.
  • Fixed recursive pydantic issue.
  • Changed default key type from 'Default' to 'LLM API' in the UI.
  • Fixed RAG query pipeline to use router for completion.
  • Fixed password comparison with non-ASCII characters.
  • Propagated JWT auth metadata to OTEL spans.
  • Fixed correct schema path for prisma generation in Docker setup.
  • Fixed arguments issue and refactored into a helper function to reduce bloat.
  • Fixed proxy support for slashes in Google generateContent model names.
  • Fixed non-standard MCP URL pattern.
  • Optimized logo fetching and resolved MCP import blockers.
  • Fixed proxy to use return value from CustomLogger.async_post_call_success_hook.
  • Fixed server root path.
  • Fixed issue where user max budget could not be reset to unlimited.
  • Fixed UI issue preventing clearing content filter patterns when editing a guardrail.
  • Fixed UI issue with 'Create Key: Expire Key Input Duration'.
  • Fixed UI issue with 'Internal User: Bulk Add'.
  • Fixed model configuration for gpt-5.2-codex mode to use responses for Azure and OpenRouter.
  • Fixed custom_llm_provider for provider-specific parameters in responses.
  • Fixed Azure audio token pricing using generic cost calculator.
  • Corrected cached token cost calculation for xAI models.
  • Fixed token calculations and refactored code.
  • Safely handled None metadata in Prometheus logging to prevent AttributeError.
  • Resolved 'does not exist' migration errors as applied in setup_data.
  • Fixed /batches endpoint to return encoded IDs (from managed objects table).
  • Fixed function content to be valid JSON for GigaChat.
  • Fixed A2A Python SDK URL.
  • Fixed flaky tests in OIDC and Email CI/CD pipeline.
  • Fixed broken mocks in 6 flaky tests to prevent real API calls.
  • Fixed proxy support for slashes in Google generateContent model names.

Affected Symbols

RouterCustomLogger.async_post_call_success_hook