●Change8

v1.97.0-dev.2

📦 litellmView on GitHub →
✨ 7 features🐛 27 fixes🔧 22 symbols

Summary

This release introduces enhanced Docker image signature verification using cosign and includes numerous bug fixes and performance improvements across various components like the UI, auto-router, and proxy. Several new features enhance configurability and control for operators.

Migration Steps

  1. Verify Docker images using cosign with the pinned commit hash (recommended) or the release tag.

✨ New Features

  • Add role capability gating to the UI.
  • Make reminder marker pair configurable for the auto-router.
  • Derive a default auto-router savings baseline from the hardest tier.
  • Allow operators to rename the four complexity tiers in the complexity router.
  • Make the gateway middleware the source of truth for successful requests.
  • Allow operators to replace the LLM classifier's system prompt.
  • Rebuild the auto-router benchmarks backend as a per-session rollup.

🐛 Bug Fixes

  • Name enable_jwt_auth when a JWT-shaped key is rejected.
  • Update anthropic model presets in the UI.
  • Switch to the dashboard node floor via nvm or fnm during bootstrap.
  • Self-heal handler clients closed after cache eviction.
  • Keep OpenAI prompt cache token details through usage reassembly.
  • Bill gpt-5.6 prompt cache reads at the cache read rate.
  • Account for Responses API usage in batches.
  • Skip rows without file objects in managed-files.
  • Redact fallback tracebacks at the call site and cover the sync deferred stream.
  • Recover from an interrupted Prisma toolchain install.
  • Bring basedpyright rule counts back under their budget limits.
  • Give proxy_admin_viewer read parity with proxy_admin.
  • Make the env-key doc gate see get_secret_bool reads.
  • Re-land evicted LLM client closing atop self-healing handlers.
  • Keep the connected DB client when a startup health check fails.
  • Return 403 from the OAuth2 enterprise gate.
  • Keep custom model_info across a price data reload.
  • Resolve pass-through credentials live from router deployments.
  • Match CJK keyword_tier_rules that regex word boundaries miss.
  • Bake the pip image's prisma engines at a world-readable path.
  • Bake the componentized prisma engines at /opt/prisma so any uid can start.
  • Keep the toolchain heal from raising on an unreadable nodeenv cache.
  • Sign Bedrock managed-file S3 requests with S3SigV4Auth.
  • Resolve the documented AI21_API_KEY instead of a misspelled name.
  • Fail the image build when the generated prisma engine paths drift off /opt/prisma.
  • Resolve the documented JINA_API_KEY as a fallback.
  • Only treat a recoverable database outage as grounds to serve without one.

Affected Symbols

enable_jwt_authfetch client's base urlTool Policies routeanthropic model presetsdashboard node floorhttp_handler clientsOpenAI prompt cacheResponses APIcomplexity tiersgateway middlewareLLM classifier's system promptauto-router benchmarksquery-param routingprisma enginesOAuth2 enterprise gatecustom model_infopass-through credentialsCJK keyword_tier_rulesBedrock managed-file S3 requestsAI21_API_KEYJINA_API_KEYdatabase outage handling