LiteLLM
AI & LLMsPython SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]
Release History
View all versions →v1.99.0-rc.25 fixesThis release introduces signature verification for LiteLLM Docker images using cosign, enhancing security. It also includes several backported bug fixes for UI, e2e tests, and Anthropic integration.
v1.100.0-rc.132 fixes13 featuresThis release introduces enhanced Docker image verification using cosign for improved security. It also includes numerous bug fixes across various components like Anthropic integration, proxy, and routing, alongside new features for spend reporting, UI improvements, and Terraform provider parity.
v1.100.0-dev.245 fixes2 featuresThis release introduces Docker image signature verification using cosign for enhanced security. It also includes numerous bug fixes across various components like Prometheus, OpenTelemetry, proxy, and provider integrations, alongside UI refactors and documentation updates.
v1.100.0-dev.1Breaking32 fixes4 featuresThis release introduces Docker image signing with cosign for enhanced security and includes numerous bug fixes across various integrations and the UI. Key features include per-team New Relic trace routing and improved Azure AI Foundry authentication.
v1.98.052 fixes11 featuresThis release introduces enhanced Docker image security with cosign signature verification and adds several new features including structured logging, per-deployment overrides, and vector store index management. It also includes numerous bug fixes across various components like Bedrock, proxy, and UI.
v1.99.0-rc.133 fixes6 featuresThis release introduces enhanced Docker image signature verification using cosign and includes numerous bug fixes across various modules like SCIM, cost tracking, streaming, and proxy functionalities. It also adds new features for auto-routed responses, savings visualization, and support for new models and providers.
v1.99.0-dev.2Breaking24 fixes7 featuresThis release introduces security enhancements with signed Docker images and includes numerous bug fixes across proxy, router, and UI components. It also features improvements to spend logging, end-to-end testing, and adds a new CLI command for configuring Claude Code.
v1.99.0-dev.134 fixes7 featuresThis release introduces enhanced Docker image security through cosign signing and includes numerous bug fixes across various components like proxy, Bedrock, and UI. New features include an Async Rust OCR Bridge and improved UI routing presets.
v1.98.0-rc.126 fixes9 featuresThis release introduces Docker image signature verification using cosign and enhances the auto-router with shadow evaluation capabilities. It also includes numerous bug fixes and UI improvements, with many components migrated to shadcn.
v1.97.023 fixes11 featuresThis release introduces Docker image signature verification using cosign for enhanced security. It also includes numerous features and bug fixes across various components like the proxy, UI, and integrations.
v1.98.0-dev.228 fixes11 featuresThis release introduces enhanced Docker image verification using cosign and includes numerous bug fixes and new features related to proxy, PTU (Provisioned Throughput Units), and UI improvements. Key updates focus on cost attribution, rate limiting, and better error handling.
v1.96.21 fixLiteLLM Docker images are now signed with cosign for enhanced security. This release also addresses a PyPI storage failure that affected the previous version.
v1.95.11 fixThis release introduces signature verification for LiteLLM Docker images using cosign. It also includes backported proxy request-handling maintenance and refreshed runtime dependencies.
v1.94.3This release focuses on enhancing security by introducing signature verification for Docker images using cosign. Users can now verify image integrity using either a pinned commit hash or the release tag.
v1.93.21 fixThis release backports proxy request-handling fixes and refreshes runtime dependencies for version 1.93.2. It also introduces enhanced security by verifying Docker image signatures using cosign.
v1.92.2This release focuses on verifying Docker image signatures using cosign and includes backported proxy request-handling maintenance along with refreshed runtime dependencies for version 1.92.2.
v1.91.51 fixThis release introduces cosign signature verification for all LiteLLM Docker images and includes backported maintenance for proxy request handling.
v1.90.71 fixThis release introduces Docker image signing with cosign for enhanced security and backports proxy request-handling fixes. Runtime dependencies have also been refreshed.
v1.89.7LiteLLM Docker images are now signed with cosign, providing enhanced security and verifiability. The release includes backported fixes from various pull requests to the stable branch.
v1.88.61 fixThis release introduces the ability to verify LiteLLM Docker image signatures using cosign and includes backported proxy request-handling fixes.
v1.96.032 fixes10 featuresThis release introduces Docker image signature verification using cosign for enhanced security and includes numerous bug fixes and feature enhancements across various components like Prometheus, aiohttp, Vertex AI, Anthropic adapter, and the UI.
v1.97.0-rc.123 fixes4 featuresThis release focuses on numerous bug fixes across various integrations like Azure Sentinel, Bedrock, and the proxy. It also introduces new features for the auto-router and UI, alongside enhanced Docker image signature verification.
v1.94.2This release focuses on verifying LiteLLM Docker image signatures using cosign. It provides instructions for verifying images using both pinned commit hashes and release tags, ensuring image integrity.
v1.97.0-dev.227 fixes7 featuresThis release introduces enhanced Docker image signature verification using cosign and includes numerous bug fixes and performance improvements across various components like the UI, auto-router, and proxy. Several new features enhance configurability and control for operators.
v1.97.0-dev.122 fixes11 featuresThis release introduces Docker image signature verification using cosign for enhanced security. It also includes numerous features, bug fixes, and performance improvements across various components like the proxy, UI, and integrations.
Common Errors
BadRequestError16 reportsThis error typically occurs in litellm. Check the example issues for common solutions.
ServiceUnavailableError7 reportsServiceUnavailableError usually indicates the LLM provider is overloaded or temporarily unavailable. Implement retry logic with exponential backoff using `retry` decorator in `litellm` around the failing function calls, and/or check the LLM provider's status page to confirm an outage before retrying. You might also need to increase your rate limits or switch to a different model if the issue persists.
UnsupportedParamsError3 reportsThis error typically occurs in litellm. Check the example issues for common solutions.
NotFoundError3 reportsThe "NotFoundError" in litellm usually stems from an incorrect or inaccessible API endpoint specified for a given model provider. To fix this, carefully verify the `api_base` URL in your litellm configuration, ensuring it precisely matches the provider's API documentation and that the endpoint is reachable from your environment. If using URL templating, confirm the variables are correctly substituted with valid values.
InternalServerError3 reportsThis error typically occurs in litellm. Check the example issues for common solutions.
ContentPolicyViolationError3 reportsContentPolicyViolationError usually arises from AI models flagging input or output text as violating their content policies. To fix it, carefully review your prompt and the model's response for potentially harmful or sensitive content. Redraft prompts to be less suggestive or controversial and implement output filtering to sanitize model responses, ensuring compliance with content safety guidelines.
Related AI & LLMs Packages
AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.
Get up and running with OpenAI gpt-oss, DeepSeek-R1, Gemma 3 and other models.
🦜🔗 The platform for reliable agents.
The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
LLM inference in C/C++
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
Subscribe to Updates
Get notified when new versions are released