v1.95.0-dev.1
📦 litellmView on GitHub →
✨ 14 features🐛 8 fixes🔧 9 symbols
Summary
This release introduces enhanced Docker image signature verification using cosign and includes numerous features and bug fixes, particularly focusing on Rust integration for Anthropic messages, AWS LLM authentication, and performance improvements in streaming and caching.
Migration Steps
- To verify Docker image signatures, use the provided `cosign verify` commands. The recommended method uses a pinned commit hash for stronger cryptographic assurance.
- Users can optionally route native Anthropic /messages through Rust by setting the `LITELLM_RUST` environment variable.
✨ New Features
- Track prompt compression saved tokens in daily spend aggregates.
- Route native Anthropic /messages through Rust behind LITELLM_RUST env var.
- Expose Anthropic Messages route (POST /v1/messages) on the axum gateway.
- 1:1 port of OpenAI Responses API WebSockets to litellm-rust.
- Port BaseAWSLLM auth (credential resolution + SigV4) to litellm-core as a base provider.
- Add return_raw_model_name toggle for response model field.
- Always-on aggregate gateway DCR discovery front door.
- Add configuration tabs to the Cost Optimization page.
- Raise /spend/logs/v2 page_size cap to 1000.
- Identity-only session tokens for the gateway DCR front door.
- Audio transcription via rust core (py->rust bridge) for Bedrock.
- Forward bedrock_tags to CreateModelInvocationJob for batch jobs.
- Support MCP servers on the Anthropic /v1/messages API.
- Migrate client_credentials (M2M) onto the v2 resolver arm.
🐛 Bug Fixes
- Treat malformed cost-map token limits as absent on /v1/models.
- Send v4 ingestion header for otel callback.
- Restore Content-Type application/json header for fireworks_ai (fixes 415).
- Hide guardrail group headers when only one group has entries.
- Make in-memory and disk cache increments atomic.
- Only inject cache_control when the request carries none for Anthropic.
- Stop custom model_info leaking onto shared backend cost map key.
- Reset users/teams whose budget_reset_at is NULL.