●Change8

v3.14.0

Breaking Changes
📦 mlflowView on GitHub →
⚠ 3 breaking✨ 30 features🐛 26 fixes🔧 21 symbols

Summary

MLflow 3.14.0 introduces major GenAI features like one-command agent setup, durable tracing for Claude Code, Review Queues, and an LLM Playground. This release also updates default serialization formats for several model flavors, resulting in breaking changes.

⚠️ Breaking Changes

  • The default serialization_format for `mlflow.sklearn` log_model/save_model has changed from `cloudpickle` to `skops`.
  • The default serialization_format for `mlflow.pytorch.log_model` and `mlflow.pytorch.save_model` has changed to `"pt2"`.
  • The default serialization_format for `mlflow.lightgbm` log_model/save_model has changed to `"skops"`.

Migration Steps

  1. If using `mlflow.sklearn`, be aware that the default `serialization_format` is now `skops` instead of `cloudpickle`. Update code if explicit serialization format control is required.
  2. If using `mlflow.pytorch.log_model` or `mlflow.pytorch.save_model`, be aware that the default `serialization_format` is now `"pt2"`.
  3. If using `mlflow.lightgbm.log_model` or `mlflow.lightgbm.save_model`, be aware that the default `serialization_format` is now `"skops"`.

✨ New Features

  • Introduced one-command agent onboarding with `mlflow agent setup` for installing MLflow, setting up tracing, and configuring coding agents.
  • Added durable, low-latency tracing for Claude Code using a write-ahead-log.
  • Implemented Review Queues in the UI for assigning traces to reviewers and collecting structured feedback/annotations.
  • Revamped evaluation dataset UI allowing browsing, inspection, editing, and bulk management of records.
  • Added Pytest integration for GenAI regression testing using the `@mlflow.test` marker.
  • Launched LLM Playground in the UI for iterating on prompts against AI Gateway endpoints and Prompt Registry versions.
  • Added "Save prompt to registry" action to the Prompt Playground.
  • Added `EvaluationResult.passed`/`.reason` for `@mlflow.test` assertions.
  • Added shareable review queue URLs with a `startReview` deep link.
  • Allowed editing a completed review in place in focus mode.
  • Added `x-mlflow-run-id` support to OTLP trace ingestion.
  • Added `@mlflow.test` pytest marker and assertion framework for evaluation/tracing.
  • Added `mlflow skills view/list` CLI.
  • Improved review queue list layout (flat, sortable columns, status filter).
  • Added `MLFLOW_WORKSPACE` support to OSS auth provider for tracing.
  • Added cached token pricing to Databricks model catalog in Gateway.
  • Added `MLFLOW_GENAI_JUDGE_DEFAULT_MODEL` environment variable for evaluation.
  • Added handlers and jobs for UI-triggered evaluation runs via POST /mlflow/genai/evaluate/invoke.
  • Added support for UC trace location via `MLFLOW_TRACE_LOCATION` for Claude Code and Codex tracing.
  • Added label schemas support (handlers, SDK, REST client).
  • Added `run_id` support for trace APIs.
  • Added Dataset v2 port for evaluation datasets.
  • Added OSS-native label schema entity, validation, and SQL store.
  • Added support for mapping gen_ai.conversation.id to MLflow trace session.
  • Added polling logic to refresh traces tab empty state upon first trace ingestion.
  • Added MlflowWalSpanExporter to hand traces off to the WAL daemon.
  • Added support for native UC trace ingestion from TypeScript SDK.
  • Added Google ADK LLM judge scorers (`Hallucination`, `Safety`, `ResponseEvaluation`).
  • Added OpenAI `/responses/compact` passthrough route to AI Gateway.
  • Added 21 new models to Databricks model catalog in Gateway.

🐛 Bug Fixes

  • Fixed `ChrfScore` RAGAS scorer instantiation due to class name mismatch.
  • Mapped OpenAI Agents SDK guardrail spans to `SpanType.GUARDRAIL`.
  • Surfaced review-question modal failures as toasts.
  • Prevented review queues from shadowing usernames.
  • Made review-queue names unique case-insensitively (defined at table creation).
  • Normalized review-queue add-items ids before the trace-existence check.
  • Scoped review-queue permission UX gate to the active workspace.
  • Surfaced review-queue trace-removal failures and kept the selection on error.
  • Surfaced assignable-users load error in review-queue pickers.
  • Prefilled review answers from the most recent assessment by timestamp.
  • Surfaced review-queue self-assign failures with an error toast.
  • Fixed genai.evaluate() dropping dataset expectations and tags when scorers=[] was provided.
  • Required at least one question when saving review queue settings.
  • Sent review-queue schema_ids only when the questions actually change.
  • Blocked saving a review when a previously-answered question is cleared.
  • Compared review-queue picker usernames case-insensitively.
  • Bound review-queue completed_by to the authenticated caller.
  • Fixed non-functional JSON/Table toggle in the review queue full-trace explorer.
  • Surfaced review-queue deletion failures instead of swallowing them.
  • Fixed TS SDK traces storage when MLflow server uses a local FS artifact root without mlflow-artifacts:// uri schema.
  • Showed minute fidelity in the review-queue "Date added" column.
  • Showed the optional rationale box in the question preview for review queues.
  • Set model provider in Anthropic autolog so LLM cost is computed.
  • Added missing `ContextUtilization` RAGAS scorer class.
  • Refreshed per-trace queue membership after adding/removing review-queue items.
  • Fixed JSON response format for Gemini and Anthropic gateway providers.

Affected Symbols

mlflow.sklearnmlflow.pytorch.log_modelmlflow.pytorch.save_modelmlflow.lightgbm.log_modelmlflow.lightgbm.save_modelmlflow agent setup@mlflow.testEvaluationResultmlflow skills view/listOTLP trace ingestionMLFLOW_WORKSPACEPOST /mlflow/genai/evaluate/invokeMLFLOW_TRACE_LOCATIONgenai.evaluate()MlflowWalSpanExporterTypeScript SDK trace ingestionSpanType.GUARDRAILChrfScoreContextUtilizationGemini gateway providerAnthropic gateway provider