LiteLLM
AI & LLMsPython SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]
Release History
v1.83.3-stable29 fixes5 featuresThis release focuses heavily on bug fixes across various providers (Anthropic, Gemini, Vertex AI) and internal proxy/UI stability, alongside introducing Docker image signing verification via cosign.
v1.83.7.rc.1Breaking14 fixes7 featuresThis release introduces Docker image signing verification instructions and includes numerous bug fixes across proxy, S3, logging, and guardrails. It also features a breaking change reducing the default Prometheus latency histogram buckets.
v1.83.6-nightly13 fixes7 featuresThis release introduces Docker image signature verification using cosign, adds new API endpoints and features like Ramp callbacks, and includes numerous bug fixes across routing, cost calculation, and Docker builds.
v1.82.3-stable.patch.41 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to improve supply chain security. Users are encouraged to verify image authenticity using the provided signing instructions.
v1.82.3.dev.91 featureThis release introduces cryptographic signing for all LiteLLM Docker images using cosign to improve supply chain security. Instructions are provided for verifying image signatures.
v1.83.3.rc.111 fixes6 featuresThis release introduces Docker image signature verification via cosign and adds significant features for team management, including project-level guardrails and enhanced UI controls for keys and rate limits. Numerous bug fixes address issues across the UI, proxy, and agent integrations.
v1.82.3-stable.patch.36 fixes3 featuresThis patch release introduces new UI features for rate limiting and credential exposure, adds embedding usage tracking fallback, and includes several UI and Docker build bug fixes. It also details how to verify the integrity of LiteLLM Docker images using cosign.
v1.82.6.rc.49 fixes4 featuresThis release introduces Docker image signing verification using cosign and adds significant new features to the router, including order-based fallback and health-check-driven routing. Several routing and Docker build fixes were also implemented.
v1.83.2-nightly1 featureThis release introduces Docker image signing using cosign for improved security verification. It also includes minor version bumps for internal components.
v1.83.5-nightly17 fixes9 featuresThis release introduces Docker image signing verification via cosign and adds significant features to the UI and Proxy, including project-level guardrails and Azure Entra ID credential support. Several bug fixes address issues related to UI filtering, team management, and Docker image loading.
v1.82.3.dev.71 featureThis release introduces Docker image signing using cosign to ensure image integrity. Users can now verify the authenticity of LiteLLM Docker images before deployment.
v1.82.3.dev.61 featureThis release introduces Docker image signing using cosign to ensure image integrity. Users can now verify the authenticity of LiteLLM Docker images before deployment.
v1.83.1-nightlyBreakingThis was a test release from the LiteLLM team to test a new signing process. Co-sign verification is temporarily non-functional.
v1.83.0-nightly14 fixes11 featuresThis release focuses heavily on infrastructure hardening, security improvements (like OpenSSF Scorecard adoption and dependency pinning), and numerous bug fixes across providers like Gemini, Anthropic, and Azure OpenAI. New features include Prometheus metrics for batch operations and budget enforcement across multi-pod deployments.
v1.82.3-stable.patch.29 fixes5 featuresThis release focuses heavily on UI enhancements, including new organizational and team management features, alongside critical bug fixes related to client handling, logging redaction, and database migration stability. A new provider for Amazon Nova models on SageMaker was also introduced.
v1.82.6.dev21 fix2 featuresThis release introduces enhancements to Prometheus metrics and proxy callback tracking, alongside a fix for missing post-call guardrail logs in the proxy.
v1.82.6.rc.2This release appears to be a release candidate (v1.82.6.rc.2) with no detailed changes provided in the snippet, linking only to the comparison view.
v1.82.3-stable.patch.18 fixes5 featuresThis release introduces support for the sagemaker_nova provider and includes numerous UI enhancements related to key management and team settings. Several critical bug fixes address issues with client eviction, logging redaction, and database migration stability.
v1.82.3.dev.5This release contains internal development changes between versions v1.82.5.dev.1 and v1.82.3.dev.5.
v1.82.5.dev.133 fixes11 featuresThis release focuses heavily on bug fixes and stability improvements across various providers like Anthropic, Gemini, and Vertex AI, alongside introducing new features such as Akto Guardrails integration and prompt management support for the Responses API.
v1.82.6.dev1This release appears to be a development release (v1.82.6.dev1) following a release candidate (v1.82.6.rc.1), with the full changelog available via the provided comparison link.
v1.82.6.rc.127 fixes10 featuresThis release introduces new features like the sagemaker_nova provider and various UI enhancements, alongside numerous bug fixes across providers (Anthropic, Gemini, Vertex AI) and infrastructure improvements, including security updates and better logging redaction.
v1.82.6.dev.118 fixes8 featuresThis release focuses heavily on UI modernization, especially around Teams and MCP management, alongside numerous bug fixes for proxy guardrails, logging, and key management endpoints. It also introduces infrastructure improvements and new environment variable support for Anthropic.
v1.82.6-nightly34 fixes9 featuresThis release focuses heavily on stability and feature parity across various providers, including significant fixes for Anthropic reasoning summaries, Gemini image handling, and Vertex AI pricing/streaming. It also introduces new features like Akto Guardrails integration and control plane management.
v1.82.1.dev.12 fixesThis development release primarily focuses on fixing bugs related to parameter passing for Azure OpenAI and improving streaming response handling across various providers.
v1.81.14.dev.39 fixes5 featuresThis release focuses heavily on UI modernization, infrastructure improvements including a control plane for worker management, and numerous fixes across various providers like Bedrock, Gemini, and Moonshot. Several documentation updates regarding pricing and caching were also included.
v1.81.14.dev.29 fixes5 featuresThis release focuses heavily on UI modernization, especially around Teams features, and includes numerous fixes across various providers like Bedrock, Gemini, and Anthropic. A new control plane for multi-proxy worker management was also introduced.
v1.82.3.dev.49 fixes5 featuresThis release focuses heavily on UI enhancements, including new organizational and team management features, alongside critical bug fixes related to logging redaction, database migrations, and dependency security updates.
v1.82.5-nightly15 fixes9 featuresThis release focuses heavily on bug fixes across the proxy, UI, and logging systems, alongside infrastructure improvements like black formatting and dependency updates. Key features include enhanced Anthropic integration and new control plane capabilities for multi-proxy management.
v1.82.3.dev.319 fixes7 featuresThis release focuses heavily on UI enhancements, including new organizational and team management features, alongside numerous bug fixes addressing security, logging redaction, and database migration stability. A new provider, sagemaker_nova, was also introduced.
v1.81.14.dev.117 fixes4 featuresThis release focuses heavily on bug fixes across the proxy, UI, and logging systems, alongside infrastructure updates like code formatting and dependency regeneration. New features include enhanced team endpoint access control and improved Anthropic environment variable support.
v1.82.3.dev.221 fixes4 featuresThis release focuses heavily on bug fixes across proxy functionality, UI components, and provider integrations, alongside infrastructure updates like dependency regeneration and code formatting. New features include enhanced team listing access control and proxy-wide rate limits.
v1.82.3.dev.12 fixesThis development release primarily focuses on fixing parameter passing issues for the Azure OpenAI provider, specifically for max_tokens and temperature.
v1.82.4-nightly22 fixes7 featuresThis release focuses heavily on UI enhancements, including new organizational controls and modernized settings pages. It also includes numerous bug fixes addressing security issues, database migration stability, and provider integration reliability.
v1.82.0.dev.7This release is a stable patch for Litellm version 1820, implemented by @shivamrawat1.
v1.82.3-stable10 fixes5 featuresThis release focuses heavily on UI enhancements, including organization and team management features, alongside critical bug fixes related to client handling, logging redaction, and database migration stability. A new provider for Amazon Nova models on SageMaker was also introduced.
v1.82.3.rc.19 fixes5 featuresThis release focuses heavily on UI enhancements, including organization and team management features, alongside critical bug fixes related to logging redaction, database migrations, and dependency security updates.
v1.82.0.patch520 fixes4 featuresThis patch release focuses heavily on bug fixes across various integrations (Anthropic, Azure, HuggingFace) and infrastructure stability, alongside introducing a new User Info V2 Endpoint and allowing organization_id setting on key updates.
v1.82.2-dynamoai.dev28 fixes9 featuresThis release focuses heavily on bug fixes, infrastructure improvements, and significant updates to the UI, including new endpoints and administrative features. Key fixes address security vulnerabilities, proxy stability, and various provider integrations.
v1.82.2-focus-export-2.dev21 fixes12 featuresThis release introduces several new UI features, including user info endpoints and team management tools, alongside numerous bug fixes addressing issues in proxy stability, cost tracking, and API integrations. Dependency updates for security were also performed.
v1.82.2-nightly.dev1This release primarily consists of internal development updates, as indicated by the nightly/dev tag, with a link provided to the detailed comparison.
v1.82.2-silent.dev24 fixes20 featuresThis release focuses heavily on enhancing Multi-Cloud Proxy (MCP) functionality, including new authentication methods, BYOM support, and UI improvements. It also brings numerous bug fixes across providers like Gemini, Anthropic, and OpenAI, alongside new feature support for transcription and image editing.
v1.82.2-nightly27 fixes19 featuresThis release focuses heavily on expanding support for various models (Gemini, Mistral Voxtral, Qwen3.5) and enhancing the Multi-Cloud Proxy (MCP) features, including new authentication and UI workflows. Numerous bug fixes address streaming issues, provider specific mappings, and security concerns.
v1.82.0.dev6This release appears to be a development build (v1.82.0.dev6) referencing the comparison against the previous stable release (v1.82.0-stable), but the specific changes are not detailed in the provided notes.
v1.82.0.dev527 fixes19 featuresThis release introduces significant feature enhancements, particularly around MCP server support, new model integrations (Voxtral, Qwen3.5), and expanded guardrail capabilities. Numerous bug fixes address issues across streaming, Anthropic integration, and UI stability.
v1.82.0.dev4This release primarily points to the full changelog for version v1.82.0.dev4 compared to v1.82.0-stable.
v1.82.0.dev3This release appears to be a development build (v1.82.0.dev3) following v1.82.0-stable, with the full changelog available via the provided comparison link.
v1.82.1-focus-dev14 fixesThis release focuses primarily on bug fixes across various components, including routing, caching, provider integrations (Fireworks, Sagemaker, Vertex AI, Bedrock), and the Responses API streaming bridge. Several provider-specific URL and request body issues were resolved.
v1.82.1-nightly.1-reThis release corresponds to a nightly update identified by the comparison link between v1.82.1-nightly.1 and v1.82.1-nightly.1-re.
v1.82.1.rc.143 fixes10 featuresThis release focuses heavily on bug fixes across numerous providers (Gemini, Anthropic, OpenAI, Azure, etc.), improves streaming performance, and introduces new features like UI Projects and enhanced model cost tracking. A key structural change involves updating response types to use ModelResponseStream instead of StreamingChoices.
v1.82.0-stable1 fixThis stable release primarily addresses a bug where HTTP/SDK clients were incorrectly closed during LLMClientCache eviction.
v1.82.1-nightly.2This release primarily consists of internal updates and bug fixes, referenced by the full changelog link.
v1.82.1-azure-dev43 fixes10 featuresThis release focuses heavily on bug fixes across numerous providers (Gemini, Anthropic, OpenAI, Azure, etc.), improves streaming performance, and introduces new features like UI Projects and enhanced model cost tracking. It also deprecates older xAI models.
v1.82.1-nightly.1This release primarily consists of internal updates and links to the detailed comparison between nightly builds.
v1.82.1-silent-dev214 fixes2 featuresThis release focuses heavily on bug fixes across various integrations, including completion bridges, routing, proxy endpoints, caching, and specific provider integrations like Vertex AI, Fireworks, Sagemaker, and Bedrock. New features include silent metric recording and helmchart deployment strategy support.
v1.82.0.patch42 fixesThis patch release primarily focuses on bug fixes, specifically addressing incorrect parameter passing for Azure OpenAI and improving streaming response handling across providers.
v1.82.1-dev13 fixesThis release focuses heavily on bug fixes across various components including routing, caching, proxy endpoints, and specific provider integrations like Fireworks, Sagemaker, and Vertex AI. Key improvements involve error handling during retries and data preservation in streaming responses.
1.82.1-dev-22 fixesThis development release focuses on fixing bugs related to parameter passing for Azure OpenAI and improving streaming response handling across various providers.
v1.82.1-nightly43 fixes10 featuresThis release focuses heavily on bug fixes across numerous providers (Gemini, Anthropic, OpenAI, Azure, etc.), improves streaming performance, and introduces new features like UI Projects and enhanced model cost tracking. A key change involves updating response types to use ModelResponseStream instead of StreamingChoices.
v1.82.dev43 fixes10 featuresThis release focuses heavily on bug fixes across various providers (Gemini, Anthropic, OpenAI, Azure AI, etc.), enhances model cost mapping, and introduces new features like UI Projects and improved streaming latency. It also deprecates older xAI models.
v1.82.rc.32 fixesThis release focuses on fixing critical parameter passing issues for the Azure OpenAI provider, specifically for max_tokens and timeout settings.
v1.81.14-stable.gpt-5.4-patch7This release contains a patch update for the v1.81.14-stable.gpt-5.4 branch, with details available in the full changelog link.
v1.82.rc.2This release corresponds to release candidate 2 (v1.82.rc.2) following the v1.82.0-nightly build.
v1.82.0.patch32 fixesThis patch release primarily focuses on fixing an issue related to Azure OpenAI API key passing and resolving a streaming max_tokens bug.
v1.81.14-stable.gpt-5.4-patch62 fixesThis patch release focuses on fixing critical bugs related to tool call handling for the Gemini 1.5 Flash model on the completion endpoint.
v1.82.dev243 fixes10 featuresThis release focuses heavily on bug fixes across numerous providers (Gemini, Anthropic, OpenAI, Azure, etc.), enhances model tracking and cost mapping, and introduces performance improvements for streaming latency. It also adds new UI features for project management and access control.
v1.82.dev141 fixes11 featuresThis release focuses heavily on bug fixes across various providers (Gemini, Anthropic, OpenAI, Azure, OpenRouter) and internal systems like MCP and caching. It also introduces new model support, performance optimizations for spendlogs and streaming latency, and new UI features for project management.
v1.81.14-stable.gpt-5.4-patch5This release corresponds to a patch update in the v1.81.14-stable.gpt-5.4 series, detailed in the linked comparison.
v1.81.14-stable.gpt-5.4_patch21 fixThis patch release primarily addresses a bug related to the incorrect passing of the `max_tokens` parameter to the OpenAI API for specific models.
1.82.143 fixes10 featuresThis release focuses heavily on bug fixes across numerous providers (Gemini, Anthropic, OpenAI, Azure, etc.), improves streaming performance, and introduces new features like UI Projects and enhanced model cost tracking.
v1.81.14-stable.gpt-5.4_patch2 fixesThis patch release primarily addresses bugs related to the Azure OpenAI provider, ensuring correct handling of completion and streaming responses.
v1.82.0.patch22 fixesThis patch release primarily focuses on fixing bugs related to parameter passing for Azure OpenAI and improving streaming response handling.
v1.82.0.patch12 fixesThis patch release addresses critical bugs related to Azure OpenAI API key passing and streaming response handling across various providers.
v1.81.14.gemini-batchesThis release corresponds to version v1.81.14.gemini-batches, with detailed changes available via the comparison link.
v1.81.14-redirect-alphaThis release contains a full changelog available via the provided comparison link between v1.81.14-stable and v1.81.14-redirect-alpha.
v1.82.0-nightly29 fixes18 featuresThis release introduces significant UI features related to Projects and Access Groups, adds several new models and cost map entries, and includes numerous performance optimizations and bug fixes across proxy, caching, and model handling.
v1.81.14-stableThis release includes a full changelog available via the provided comparison link, marking the stable release v1.81.14.
v1.82.rc.129 fixes18 featuresThis release focuses heavily on UI enhancements, particularly around Projects management and access control (RBAC), alongside numerous performance optimizations and bug fixes across model integrations and proxy functionality. Key additions include new model support and improved tracing capabilities.
v1.81.6.rc.9This release contains minor updates and bug fixes, detailed in the linked comparison between v1.81.6.rc.6 and v1.81.6.rc.9.
v1.81.12-stable_gpt-5.3.patch12 featuresThis patch updates the Anthropic beta headers configuration to enable the 1M context window feature for Anthropic and Vertex AI, and adds extensive beta header mappings for the Databricks provider.
litellm-trace-dev-v1.81.1620 fixes24 featuresThis release introduces significant feature enhancements, particularly around UI projects, access control, and model support, alongside numerous bug fixes and performance optimizations across various integrations and internal systems.
v1.81.12-stable.22 fixesThis release primarily focuses on bug fixes, specifically addressing issues with passing `max_tokens` to Azure OpenAI and improving streaming response handling across providers.
v1.81.6.rc.6This release includes minor updates and fixes between release candidates v1.81.6.rc.4 and v1.81.6.rc.6.
v1.81.14.rc.2This release contains a series of minor updates and fixes between release candidates v1.81.6.rc.5 and v1.81.14.rc.2.
litellm_presidio-dev-v1.81.168 fixes4 featuresThis release focuses on stability and UI improvements, introducing pagination for key aliases, fixing OOM issues, and enhancing real-time WebSocket error handling. A previously added /public/supported_endpoints feature was reverted.
v1.81.16.custm-auth.dev23 fixes17 featuresThis release focuses heavily on security fixes, performance optimizations across various components, and significant feature additions to the proxy and UI, particularly around tool policies, agent key assignment, and real-time guardrails. Critical security vulnerabilities were also addressed.
1.81.16-nightly16 fixes17 featuresThis release focuses heavily on security fixes, performance optimizations across various components, and significant feature additions to the proxy, guardrails, and UI, including support for new models and enhanced logging/tracing.
v1.81.6.rc.52 fixesThis release focuses on bug fixes, specifically removing a duplicate schema definition and introducing spend logging within the Release Candidate build.
v1.81.6.rc.43 fixesLiteLLM v1.81.6.rc.4 focuses on stability by fixing several bugs related to parameter passing (like max_tokens for Azure) and response streaming across various providers.
v1.81.12-stable.11 fixThis patch release backports a stateless fix for StreamableHTTPSessionManager to the v1.81.12-stable branch.
v1.81.12-stable_gpt-5.3This release appears to be a maintenance or internal version bump, indicated by the comparison range v1.81.12-stable to v1.81.12-stable_gpt-5.3, with no detailed changes provided in the snippet.
v1.81.15-nightlyThis release provides an update to the nightly build, linking to the full comparison between the previous development version and the new nightly version.
CVEv1.81.59 fixesThis release focuses primarily on bug fixes across various integrations, including recovery from zombie processes, improved pricing application, and fixes related to rate limiting and authentication flows. Several recent fixes were also reverted.
litellm-CVE-v1.81.152 fixesThis release addresses a critical CVE by fixing issues related to the identification and handling of the 'presidio' model integration.
RCE-v1.81.15This release primarily contains internal updates and links to the full comparison view for detailed changes.
litellm-v1.81.14.pre-call-hook-fix.dev16 fixes12 featuresThis release introduces new features like custom favicon support, configurable semantic cache dimensions, and guardrail policy versioning. Numerous bug fixes address issues across Bedrock, budget configuration, testing stability, and model cost mapping.
litellmv1.81.15.presidio.dev1 fix3 featuresThis release introduces new model pricing, enhanced content filtering with topic blockers, and a new guardrail hook for voice transcription, alongside a fix for Replicate server communication.
litellm-precall-dev-v1.81.1417 fixes12 featuresThis release introduces several new features, including custom favicon support, configurable semantic cache dimensions, and new guardrail policies. Numerous bug fixes address issues across Bedrock integration, budget calculation, testing stability, and data normalization.
litellm-pres-dev-v1.81.1521 fixes12 featuresThis release introduces new features like custom proxy favicons, configurable semantic cache dimensions, and guardrail policy versioning. Numerous bug fixes address issues across Bedrock integration, budget tracking, testing stability, and data normalization.
litellm_presidio-dev-v1.81.1516 fixes8 featuresThis release introduces new features like custom favicon support for the proxy, configurable semantic cache dimensions, and guardrail monitoring. It also includes numerous bug fixes across providers like Bedrock and Anthropic, and utility improvements like parameter key normalization.
Common Errors
ServiceUnavailableError7 reportsServiceUnavailableError usually indicates the LLM provider is overloaded or temporarily unavailable. Implement retry logic with exponential backoff using `retry` decorator in `litellm` around the failing function calls, and/or check the LLM provider's status page to confirm an outage before retrying. You might also need to increase your rate limits or switch to a different model if the issue persists.
ContentPolicyViolationError3 reportsContentPolicyViolationError usually arises from AI models flagging input or output text as violating their content policies. To fix it, carefully review your prompt and the model's response for potentially harmful or sensitive content. Redraft prompts to be less suggestive or controversial and implement output filtering to sanitize model responses, ensuring compliance with content safety guidelines.
ModuleNotFoundError3 reports"ModuleNotFoundError" usually arises when a required Python package isn't installed, or the Python environment is misconfigured. Resolve this by first ensuring the necessary package is installed using `pip install <package_name>` (e.g., `pip install fastapi`). If the package is installed yet still causing problems, verify your Python environment and that the package version is compatible.
BadRequestError2 reportsBadRequestError in litellm often arises from incorrect formatting of the messages array, especially when using tool calls or custom LLM providers. Ensure the `messages` array adheres strictly to the expected format for the specific model, including correct roles (e.g., "system", "user", "assistant", "tool") and content structures for tool use. Validate that `custom_llm_provider` settings are correctly configured for the intended LLM, with no configuration conflicts in place. Double-check your code against the official documentation and example usages to address format and configuration inconsistencies.
MidStreamFallbackError2 reportsMidStreamFallbackError in litellm usually arises when a streaming response fails mid-flight during a fallback scenario, obscuring the original error code. To fix this, ensure that the `MidStreamFallbackError` preserves the original exception's status code and message, allowing litellm to correctly handle errors like rate limits or other service issues further upstream - log this internal exception and its trace! Also ensure that any stream that throws an exception is immediately closed to prevent resource leaks.
NotFoundError2 reports"NotFoundError" in litellm usually means the model name you're using is invalid or the specified provider isn't supported for that model. Double-check your model name in the litellm.completion() call against the available models for your chosen provider (e.g., OpenAI, Azure). Ensure the provider setting in your code correctly reflects where the model is hosted; a typo can cause this error, so verify the spelling.
Related AI & LLMs Packages
AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.
Get up and running with OpenAI gpt-oss, DeepSeek-R1, Gemma 3 and other models.
🦜🔗 The platform for reliable agents.
The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
LLM inference in C/C++
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
Subscribe to Updates
Get notified when new versions are released