●Change8

v1.99.0-rc.1

📦 litellmView on GitHub →
✨ 6 features🐛 33 fixes🔧 4 symbols

Summary

This release introduces enhanced Docker image signature verification using cosign and includes numerous bug fixes across various modules like SCIM, cost tracking, streaming, and proxy functionalities. It also adds new features for auto-routed responses, savings visualization, and support for new models and providers.

Migration Steps

  1. Verify Docker images using cosign with the pinned commit hash or release tag.
  2. Update pytest configurations to enforce PT012, PT011, PT014, F811, and PT017 to prevent dead assertions and duplicate definitions.
  3. Ensure operator-declared IDs are present on config.yaml reservations for PTU configurations.
  4. Update to use the new cosign verification method for Docker images.

✨ New Features

  • Add router_model_name to auto-routed response bodies.
  • Add per-key Savings tab to key detail page.
  • Surface per-request auto-router savings to logging callbacks.
  • Add moonshot/kimi-k3 to model prices and context window map.
  • Add SCX.ai as a JSON-configured OpenAI-compatible provider.
  • Serve gpt-5.6 cross-region inference profiles on bedrock runtime.

🐛 Bug Fixes

  • Match group members by SSO identity or email before creating a placeholder.
  • Match streamed Messages usage cost to the recorded spend.
  • Price partial-stream spend rows at the real model and keep prompt and cache fields.
  • Price gpt-image-2 per size and quality from request params.
  • Populate team member emails missing from the roster snapshot.
  • Forward provider response headers on chat completions.
  • Price swe-1.7 at the standard tier, add swe-1.7-lightning.
  • Send the inference component header and honor hf_model_name.
  • Hash raw api keys before persisting to spend logs.
  • Make per-model budgets track spend, enforce, and report the same counter.
  • Bound each spend-log write statement by row count as well as bytes.
  • Preserve Bedrock Mantle validation errors.
  • Omit thinking.type=disabled for always-on thinking Claude models.
  • Read batch records the same way the upload validation does.
  • Scan batch records with the content hooks that are not guardrails.
  • Require an operator-declared id on a config.yaml reservation.
  • Never retract a flat charge for a deployment the run cannot see.
  • Deny the interactive dcr_bridge authorize for a user without server access.
  • Map Bedrock Mantle context overflow to ContextWindowExceededError.
  • Run policy pipelines when the caller sends its own metadata (/v1/messages, Claude Code).
  • Convert messages to contents in gemini count_tokens.
  • Don't log 'Could not identify azure model' when the deployment name resolves from the cost map.
  • Drive auto-router usage from the shared cost-optimization time picker.
  • Reset only the failed node on a cluster client timeout, not the whole client.
  • Strip root_path before matching the per-server MCP route spelling.
  • Group Codex turns under one session id.
  • Preserve uvicorn color_message args during secret redaction.
  • Serve gpt-5.6 cross-region inference profiles on bedrock runtime.
  • Decode model-encoded output file id so completed batches book spend.
  • Correct gemini-3.1-flash-lite-image capabilities and dedupe its entries.
  • Resolve admin OAuth sessions to the same server set the connect page shows.
  • Gate sampling params on /v1/messages like /chat/completions.
  • Apply GPT-5.6 Sol promotional pricing cut.

Affected Symbols

cosignghcr.io/berriai/litellmpytestlitellm.docker