LiteLLM
AI & LLMsPython SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]
Release History
v1.90.4v1.92.0v1.93.0-rc.1v1.91.3v1.91.2v1.93.0-dev.3v1.93.0-dev.2v1.92.0-rc.21 featureThis release introduces Docker image signing using cosign for improved security verification. Users are provided with instructions on how to verify the image signatures.
v1.91.11 featureThis patch release (v1.91.1) primarily introduces Docker image signing via cosign for improved supply chain security and backports several previous fixes.
v1.93.0-dev.128 fixes9 featuresThis release introduces new features like entra_obo token exchange profile and enhanced Docker image signing verification via cosign. Numerous bug fixes address issues across streaming, guardrails, MCP operations, and spend reporting.
v1.92.0-rc.112 fixes8 featuresThis release introduces Docker image signature verification using cosign and adds several features to MCP, guardrails, and the UI, alongside numerous bug fixes across Anthropic integration, proxy security, and CI stability.
v1.91.024 fixes12 featuresThis release introduces Docker image signature verification via cosign and adds Day 0 support for Gemini 3.5 Flash and Gemini managed agents. Numerous bug fixes address issues across proxying, Bedrock, Deepseek, caching, and UI stability.
v1.90.31 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to enhance supply chain security. It also backports several fixes from development branches to the stable 1.90.x line.
v1.92.0-dev.220 fixes15 featuresThis release introduces Docker image signature verification via cosign and brings numerous fixes and feature enhancements across the proxy, UI, and various model providers like Bedrock, Vertex AI, and MCP.
v1.90.21 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to enhance supply chain security. Users are encouraged to verify image signatures upon pulling.
v1.90.11 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to enhance supply chain security. It also backports several fixes to the stable 1.90.x branch.
v1.89.51 featureThis release introduces Docker image signing via cosign for enhanced security and integrity verification of LiteLLM images.
e2e-video-31300saml-sso-e2e-demo-v1v1.92.0-dev.122 fixes10 featuresThis release introduces Docker image signature verification using cosign for enhanced security and adds features like declarative fallbacks and AES-256-GCM encryption for proxy credentials. Numerous bug fixes address issues across realtime processing, proxy permissions, Databricks tool calls, and UI stability.
v1.91.0-rc.119 fixes7 featuresThis release introduces Docker image signing verification via cosign and significant improvements to MCP OAuth handling and Prometheus metrics. Numerous bug fixes address issues across guardrails, Bedrock model listing, logging, and proxy performance.
v1.90.024 fixes14 featuresThis release introduces Docker image signing via cosign for enhanced security and adds support for the new Claude Fable 5 model across multiple providers. Numerous fixes and improvements were made across the UI, MCP handling, and rate limit error standardization.
v1.91.0-dev.222 fixes12 featuresThis release introduces significant infrastructure improvements, including a new Rust router/AI-Gateway, enhanced Docker image security via cosign verification, and numerous fixes across MCP, Anthropic, and Vertex integrations. Key features include Rust OCR support and new model pricing.
v1.87.5This release (v1.87.5) is a maintenance patch for the stable/1.87.x branch, backporting several fixes and rebuilding UI artifacts.
v1.85.7Version 1.85.7 is a maintenance release that backports several pull requests from the main branch to the stable/1.85.x branch.
v1.85.6This release backports the 1.84.8 patch set and dependency bumps to the stable/1.85.x branch, resulting in version 1.85.6.
v1.84.9Version 1.84.9 was released as a maintenance update, backporting a specific pull request (#30480) to the stable 1.84.x branch.
v1.84.2This release primarily consists of a chore update, cherry-picking a specific fix related to the proxy onto the v1.84.1 patch branch.
v1.84.10Version 1.84.10 is a maintenance release that backports several fixes from the main branch to the stable 1.84.x line.
v1.89.41 featureThis release introduces Docker image signing via cosign for enhanced security and includes several backported maintenance changes to the stable branch.
v1.88.51 featureThis release introduces Docker image signing via cosign for improved security verification and includes several minor maintenance backports to the stable branch.
v1.86.71 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to enhance supply chain security. It also includes several minor backported chore updates.
v1.91.0-dev.123 fixes17 featuresThis release introduces Docker image signing for security, implements LiteLLM plugin architecture v2, and includes numerous fixes across proxy, UI, MCP, and provider integrations, alongside Rust workspace scaffolding.
v1.90.0-rc.123 fixes14 featuresThis release introduces Docker image signing via cosign for enhanced security and adds significant features like standardized rate limit errors and Claude Fable 5 support. Numerous bug fixes were applied across the UI, MCP handling, and various provider integrations.
v1.89.31 featureVersion 1.89.3 introduces cryptographic signing for all LiteLLM Docker images using cosign to enhance supply chain security. This release also backports several fixes from development branches.
v1.88.41 fix1 featureThis release introduces Docker image signing via cosign for improved security verification and backports several bug fixes from development into the stable 1.88.x branch.
v1.87.41 featureThis release introduces Docker image signing via cosign for enhanced security and integrity verification. It backports several fixes from development branches to the stable 1.87.x line.
v1.89.21 featureThis release introduces Docker image signing using cosign for improved security verification. It also includes several minor chore updates and backports from previous development branches.
v1.88.31 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to enhance supply chain security. Users are encouraged to verify image signatures upon pulling.
v1.89.11 fix1 featureThis release introduces Docker image signing via cosign for enhanced security and backports several fixes related to MCP, model-info, and DB to the stable branch.
v1.88.22 fixes1 featureVersion 1.88.2 introduces cryptographic signing for all LiteLLM Docker images using cosign and backports several stability and feature fixes to the 1.88.x branch.
v1.87.31 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to enhance supply chain security. Users are provided with instructions on how to verify these signatures.
v1.86.61 fix1 featureThis release introduces Docker image signing via cosign for enhanced security and backports necessary patches and dependency updates from previous versions to the 1.86.x stable branch.
v1.89.033 fixes8 featuresThis release introduces new agent providers (Watsonx, LangFlow), adds per-MCP-server rate limiting, and includes numerous fixes across integrations like Datadog, Vertex AI, and Azure AI, alongside improved Docker image signing verification.
v1.84.81 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to improve supply chain security. It also backports several fixes from development branches into the stable 1.84.x series.
v1.87.21 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to enhance supply chain security. It also backports several features including Fable 5, batch-file auth, CrowdStrike AIDR, and Mantle Responses SigV4 to the stable branch.
v1.86.51 featureVersion 1.86.5 introduces Docker image signing via cosign for enhanced security and integrity verification. This release also backports several fixes from development branches.
v1.85.51 featureThis release introduces cryptographic signing for LiteLLM Docker images using cosign to enhance supply chain security. It also backports several fixes from main to the stable branch.
v1.84.72 fixesLiteLLM v1.84.7 introduces cryptographic signing for all Docker images using cosign and backports two specific fixes to the stable branch.
v1.89.0-rc.21 fix1 featureThis release introduces Docker image signing via cosign for enhanced security and includes a patch for Claude Fable 5 support.
v1.84.62 fixes1 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign and includes backported fixes for CrowdStrike AIDR metadata capture and an identity issue.
v1.88.11 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign for enhanced security and verifies dependency updates for pyjwt and ws.
v1.88.023 fixes8 featuresThis release focuses heavily on stability, bug fixes across various providers (Bedrock, Cohere, Deepseek, Vertex AI), and significant feature additions including Gemini 3.5 Flash support and enhanced Prometheus/OTEL metrics. It also introduces new Docker image signing verification procedures.
v1.89.0-rc.133 fixes6 featuresThis release introduces new agent providers (watsonx Orchestrate, LangFlow), enhances proxy features with rate limiting and graceful shutdown, and includes numerous fixes across providers, testing infrastructure, and OpenTelemetry tracing.
v1.88.0-rc.32 featuresThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to enhance supply chain security. Instructions are provided for verifying these signatures.
v1.87.11 fix1 featureThis release focuses on security by introducing cosign signing for all LiteLLM Docker images and backports a critical fix related to session token budget exemptions.
v1.88.0-rc.21 fix1 featureThis release introduces Docker image signing via cosign for improved security verification and includes a fix hardening the session-token exemption against default key generation parameters.
v1.86.41 fix1 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign for enhanced security verification. It also backports a fix concerning session-token budget-ceiling exemptions.
v1.85.41 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to enhance supply chain security. It primarily backports several staged fixes into the stable 1.85.x branch.
v1.84.51 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to enhance supply chain security. It backports several staged fixes into the stable branch.
v1.86.31 fix1 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign for enhanced security verification. It also incorporates backported bug fixes from previous development branches.
v1.87.023 fixes8 featuresThis release focuses heavily on stability, bug fixes across various providers (Bedrock, Cohere, Deepseek, Vertex AI), and introduces new features like Gemini 3.5 Flash support and enhanced UI/Proxy capabilities. It also mandates new steps for verifying Docker image authenticity via cosign.
v1.85.31 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to improve supply chain security. It also incorporates several cherry-picked fixes from recent pull requests.
v1.88.0-rc.122 fixes8 featuresThis release introduces Docker image signature verification via cosign and adds significant features for OpenTelemetry instrumentation, MCP routing, and enterprise context management. Numerous bug fixes address issues across Bedrock, Anthropic, guardrails, and proxy authentication flows.
v1.84.41 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to enhance supply chain security. The version has been bumped from v1.84.3 to v1.84.4.
v1.88.0-dev.124 fixes10 featuresThis release focuses heavily on stability, security, and feature parity across various providers, including Day 0 support for Gemini 3.5 Flash and enhanced proxy/UI capabilities. It also introduces Docker image signature verification using cosign.
v1.84.31 fix1 featureThis release introduces Docker image signing via cosign for improved security and includes a fix to restore npm to the non_root builder in the Docker setup.
v1.86.21 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign for enhanced security and integrity verification. It also includes minor maintenance updates via cherry-picks and uv locking.
v1.85.21 fix1 featureThis release introduces Docker image signing via cosign for enhanced security and includes a bug fix to restore npm usage in the non_root Docker builder.
v1.87.0-rc.221 fixes10 featuresThis release focuses heavily on stability, bug fixes across various providers (Bedrock, Cohere, Deepseek, Vertex AI), and significant feature additions including Gemini 3.5 Flash support and enhanced Prometheus metrics. It also introduces instructions for verifying Docker image signatures.
v1.86.11 fixThis patch release introduces cryptographic signing for all LiteLLM Docker images using cosign and includes a fix for a non-root npm package issue.
v1.86.011 fixes13 featuresThis release introduces significant infrastructure updates, including componentization of gateway/UI services and new Weighted-Routing Failover capabilities. It also addresses several bugs across providers and enhances OpenTelemetry tracing support.
v1.87.0-rc.122 fixes14 featuresThis release introduces Docker image signing via cosign for enhanced security and adds numerous fixes across providers like Gemini, Vertex AI, and Bedrock. Key features include new model cost maps, UI improvements, and support for the Microsoft Purview DLP guardrail.
v1.85.11 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to enhance supply chain security and provides instructions for signature verification.
v1.84.11 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to improve supply chain security. Users are provided with instructions on how to verify these signatures.
v1.87.0-dev.16 fixes5 featuresThis release introduces Docker image signature verification via cosign and adds support for Gemini 3.5 Flash and Gemini managed agents. It also includes numerous fixes across Bedrock, Deepseek, and proxy streaming.
v1.86.0-rc.112 fixes13 featuresThis release introduces Docker image signing via cosign for enhanced security and adds significant features like Weighted-Routing Failover and componentized service architecture. Numerous bug fixes address validation, rate limiting, and provider integrations across Vertex AI, Bedrock, and MCP.
v1.85.032 fixes5 featuresThis release focuses heavily on security hardening, including fixing multiple SSRF vulnerabilities and tightening access controls across various components. New features include support for the Z.AI GLM-5 model and enhancements to Gemini multimodal embeddings.
v1.84.0Breaking23 fixes11 featuresThis release introduces new features like support for gpt-image-2 and AIHubMix provider, alongside numerous bug fixes across caching, logging, and provider integrations. It also contains a breaking change: the removal of the /ui/chat page.
v1.85.0-rc.232 fixes5 featuresThis release focuses heavily on security hardening, fixing multiple SSRF vulnerabilities and tightening access controls across proxy, guardrails, and various clients. It also introduces new model support for Z.AI on Bedrock and enhances Gemini multimodal embedding capabilities.
v1.83.14-stable.patch.31 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to improve supply chain security. Users are provided with instructions on how to verify image signatures.
v1.83.10-stable.patch.11 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to improve supply chain security. Instructions are provided for verifying these signatures.
v1.83.14-stable.patch.22 featuresThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to improve supply chain security. Instructions are provided for verifying these signatures.
v1.84.0-rc.126 fixes11 featuresThis release introduces Docker image signature verification via cosign and adds several new features, including support for gpt-image-2 and AIHubMix provider. Numerous bug fixes address issues related to caching, logging, Vertex AI, and proxy stability.
v1.83.14-stable.patch.11 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to improve supply chain security. Instructions are provided for verifying image authenticity.
v1.83.14-stable25 fixes9 featuresThis release focuses heavily on security by introducing Docker image signing verification via cosign and includes numerous fixes across various providers (Anthropic, Azure, Gemini, Bedrock) and proxy features, alongside new model support like GPT-5.5.
1.84.0-dev.220 fixes9 featuresThis release introduces Docker image signing verification via cosign and adds support for new models like gpt-image-2 and providers like AIHubMix. Numerous security hardening measures and bug fixes were applied across proxy, authentication, and vector store integrations.
1.84.0-dev.113 fixes4 featuresThis release introduces Docker image signing verification via cosign and includes numerous fixes across caching, logging, pricing, and provider integrations. A feature to lazy-load optional routers was introduced and subsequently reverted.
v1.83.14.rc.127 fixes8 featuresThis release introduces Docker image signing via cosign for enhanced security and adds support for new models like GPT-5.5/5.4 snapshots and Bedrock GLM-5. Numerous fixes address streaming issues, model pricing accuracy, and security hardening across proxy and authentication layers.
v1.83.10-stable1 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to improve supply chain security. Instructions are provided for verifying image authenticity.
v1.83.13-nightly14 fixes5 featuresThis release introduces Docker image signature verification via cosign and adds features like Dashscope image generation support and multi-region Vertex hosts. Numerous bug fixes address issues across providers, UI filtering, and budget tracking.
v1.83.7-stable.patch.11 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to allow users to verify image authenticity. Instructions for verification using commit hash or release tag are provided.
v1.83.12-nightly6 fixes2 featuresThis release introduces Docker image signing via Cosign for enhanced security and adds support for the moonshot/kimi-k2.6 model. It also includes several infrastructure updates and bug fixes related to spend tracking and Docker builds.
v1.83.11-nightly8 fixes6 featuresThis release introduces Docker image signing verification via cosign and adds several features including proxy hot-reloading and audio support for Scaleway. It also includes numerous infrastructure improvements and bug fixes across Bedrock integration and CI stability.
v1.83.7-stable21 fixes9 featuresThis release introduces enhanced Docker image security via cosign signature verification and adds several new features, including AWS Gov Cloud support and new proxy management endpoints. Numerous bug fixes address issues related to database cleanup, routing logic, and Dockerfile consistency.
v1.83.10-nightlyBreaking17 fixes8 featuresThis release introduces mandatory Python 3.10+ support and enhances security by implementing Docker image signing verification via Cosign. New features include support for grok-4.20-0309-reasoning and improved budget management capabilities.
v1.83.9-nightly22 fixes4 featuresThis release introduces Docker image signature verification via cosign and includes numerous bug fixes across proxy security, caching, Bedrock handling, and UI components. Infrastructure updates also stabilized CI/CD processes.
v1.83.8-nightly20 fixes10 featuresThis release introduces Docker image signing via cosign for verification and adds new features like BM25-based prompt compression (litellm.compress()) and advisor tool orchestration. Numerous bug fixes address issues across S3 signing, caching, UI redirects, and provider integrations like Vertex AI and Dashscope.
v1.83.3-stable29 fixes5 featuresThis release focuses heavily on bug fixes across various providers (Anthropic, Gemini, Vertex AI) and internal proxy/UI stability, alongside introducing Docker image signing verification via cosign.
v1.83.7.rc.1Breaking14 fixes7 featuresThis release introduces Docker image signing verification instructions and includes numerous bug fixes across proxy, S3, logging, and guardrails. It also features a breaking change reducing the default Prometheus latency histogram buckets.
Common Errors
ServiceUnavailableError7 reportsServiceUnavailableError usually indicates the LLM provider is overloaded or temporarily unavailable. Implement retry logic with exponential backoff using `retry` decorator in `litellm` around the failing function calls, and/or check the LLM provider's status page to confirm an outage before retrying. You might also need to increase your rate limits or switch to a different model if the issue persists.
BadRequestError5 reportsThis error typically occurs in litellm. Check the example issues for common solutions.
ContentPolicyViolationError3 reportsContentPolicyViolationError usually arises from AI models flagging input or output text as violating their content policies. To fix it, carefully review your prompt and the model's response for potentially harmful or sensitive content. Redraft prompts to be less suggestive or controversial and implement output filtering to sanitize model responses, ensuring compliance with content safety guidelines.
InternalServerError3 reportsThis error typically occurs in litellm. Check the example issues for common solutions.
MidStreamFallbackError2 reportsThe MidStreamFallbackError in litellm often arises when a streaming response from an initial model fails (e.g., due to content policy or network issues) and the fallback mechanism within `content_policy_fallbacks` or similar configurations encounters a problem during the stream continuation. To fix this, ensure your fallback models are also capable of streaming and that their configurations are valid; additionally, robust error handling should be implemented within the `streaming_handler` to gracefully manage exceptions during fallback streaming, preventing premature stream termination.
ModuleNotFoundError2 reportsThis error typically occurs in litellm. Check the example issues for common solutions.
Related AI & LLMs Packages
AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.
Get up and running with OpenAI gpt-oss, DeepSeek-R1, Gemma 3 and other models.
🦜🔗 The platform for reliable agents.
The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
LLM inference in C/C++
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
Subscribe to Updates
Get notified when new versions are released