●Change8

v1.91.0-dev.1

📦 litellmView on GitHub →
✨ 17 features🐛 23 fixes🔧 34 symbols

Summary

This release introduces Docker image signing for security, implements LiteLLM plugin architecture v2, and includes numerous fixes across proxy, UI, MCP, and provider integrations, alongside Rust workspace scaffolding.

Migration Steps

  1. Users running Docker images should verify image integrity using the provided cosign commands, preferably using the pinned commit hash.
  2. If using Google provider interactions, note that the 'role' field has been dropped from Interaction output fields to match the Google specification.

✨ New Features

  • Introduced Docker image signing using cosign for enhanced security.
  • Resolved caller identity once into a Principal at the auth seam.
  • Synchronized Fireworks AI chat completions endpoint with the full API surface.
  • Implemented LiteLLM plugin architecture v2.
  • Added code interpreter interception on the Responses API within the sandbox.
  • Drove global proxy role from a SCIM admin group.
  • Ingested enterprise extension attributes into user metadata via SCIM.
  • Scoped a key to zero MCP servers using the no-mcp-servers sentinel.
  • Added Amazon Bedrock Mantle to the Add Model provider dropdown in the UI.
  • Allowed llm_api_routes virtual keys to list MCP tools via /v1/mcp/tools.
  • Scaffolded the outbound_credentials package with a typed Result for MCP.
  • Added opensandbox sandbox provider.
  • Added resolve_credentials dispatch skeleton for MCP.
  • Added current Workers AI text-generation models to the Cloudflare cost map.
  • Added chat completions code interpreter loop.
  • Added LiteLLM Rust workspace with Mistral OCR bridge.
  • Added OpenAI realtime translation layer to litellm-rust (Part 1 of 2).

🐛 Bug Fixes

  • Stopped exposing MCP server URLs on the AI Hub and public hub API.
  • Stopped per-model usage export from duplicating user spend across models in the UI.
  • Rejected model_list in proxy body and gated advisor client credentials.
  • Resolved user_id to email in the Spend Per User usage chart in the UI.
  • Prevented key-level metadata.tags from leaking into the Bedrock passthrough body.
  • Scoped team BYOK models by key team_id in /model/info for proxy.
  • Only expanded config-sourced AWS credential references for Bedrock.
  • Labeled request logs column "Key Alias" to match filter in the UI.
  • Recovered cost on interrupted and agentic Anthropic streams for passthrough/streaming.
  • Resolved user_id to email in Spend Per User usage chart.
  • Serialized team budget_limits to JSON in jsonify_team_object for the proxy.
  • Stopped revalidating realtime events at the logging boundary.
  • Isolated all per-deployment pricing overrides from sibling deployments in the router.
  • Stopped auth failures on the /mcp path from surfacing as cancelled tool calls.
  • Brought reportReturnType back under the basedpyright budget.
  • Fixed omission of function_response id post-tool-call in realtime events.
  • Guarded num_retries=None in async_function_with_retries in the router.
  • Kept team Organization optional for proxy admins in single-org setups.
  • Routed native Workers AI provider through the OpenAI-compatible endpoint for Cloudflare.
  • Blocked server credential leak to caller-supplied api_base during search.
  • Honored litellm_settings.request_timeout as an independent per-attempt timeout in the router.
  • Tightened role-based visibility of config and MCP fields.
  • Corrected regional processing uplift to apply only to gpt-5.4/5.5 series in model prices.

Affected Symbols

Principalfireworks_ai chat completions endpointlitellm plugin architectureResponses API (sandbox)exception_type provider dispatchchunk_creator dispatchInteraction output fieldsMCP server URLsLITELLM_OTEL_V2 flagSpend Per User usage chartmetadata.tags (Bedrock passthrough)/model/info (proxy)AWS credential references (Bedrock)no-mcp-servers sentinel/v1/mcp/toolsrequest logs columnreturn-url cookieAnthropic streamsSpend Per User usage chartteam budget_limitsrealtime eventsoutbound_credentials packageper-deployment pricing overrides (router)/mcp pathprovider dispatch (completion)reportReturnTypefunction_response id (realtime)async_function_with_retries (router)team Organization fieldWorkers AI text-generation modelslitellm-rust realtime translation layerosv-flagged dependenciesmodel_prices (gpt-5.4/5.5 series)litellm_settings.request_timeout