v3.9.0rc0
📦 mlflowView on GitHub →
✨ 7 features🔧 1 symbols
Summary
MLflow 3.9.0rc0 is a pre-release introducing significant GenAI features like the MLflow Assistant, Trace Overview Dashboard, integrated AI Gateway, and LLM Judge capabilities.
Migration Steps
- To use the new AI Gateway features, note that the server is now located directly in the tracking server process.
✨ New Features
- Introduction of MLflow Assistant, an in-product chatbot backed by Claude Code to help diagnose and fix issues in the MLflow UI.
- New Trace Overview Dashboard in GenAI experiments providing pre-built statistics like latency, request count, quality metrics, and tool call summaries.
- Revamped AI Gateway feature: the Gateway server is now integrated directly into the tracking server, eliminating the need for a separate process. Includes passthrough endpoints, traffic splits, and fallback models.
- Support for Online Monitoring with LLM Judges, allowing automatic evaluation of traces using pre-defined or custom judges without writing code.
- New Judge Builder UI within the GenAI Experiment UI to define, iterate, and test custom LLM judge prompts.
- Implementation of Distributed Tracing, enabling context propagation across different services and processes for end-to-end request lifecycle tracking.
- Introduction of MemAlignOptimizer, a new judge optimization algorithm that learns from past feedback and retrieves relevant examples for more accurate evaluations.