●Change8

v4.12.0rc1

Breaking Changes
📦 datadog-sdkView on GitHub →
⚠ 1 breaking✨ 24 features🐛 1 fixes🔧 7 symbols

Summary

This release introduces significant enhancements to AI Guard, LLM Observability (including audio support and sampling configuration), and adds tracing support for Ray Serve and Mistral AI client calls. It also includes bug fixes and infrastructure updates like a new normalized route tag for Tornado.

⚠️ Breaking Changes

  • The APM trace writer now uses the `v0.4` trace API version when both APM and LLMObs are enabled because `v0.5` cannot carry the LLMObs span data. Setting `DD_TRACE_API_VERSION=v0.5` with LLMObs enabled will log a warning and downgrade to `v0.4`. No user action is required unless you explicitly set `DD_TRACE_API_VERSION=v0.5` and rely on v0.5 features.

Migration Steps

  1. If you rely on `DD_TRACE_API_VERSION=v0.5` while using LLMObs, be aware it will be downgraded to `v0.4` and log a warning.
  2. To enable buffered stream response evaluation for OpenAI or Anthropic streaming calls, set `DD_AI_GUARD_ANALYZE_STREAM_RESPONSES_ENABLED=true`.
  3. To disable AI Guard auto-instrumentation for OpenAI specifically, set `DD_AI_GUARD_OPENAI_ENABLED=false`.
  4. To enable PyTorch distributed training tracing, set `DD_PATCH_MODULES=pytorch:true`.
  5. To ensure the injected library takes precedence over user-installed ddtrace packages during Single Step Instrumentation, set `DD_INJECT_EXPERIMENTAL_OVERRIDE_USER_DDTRACE=true`.

✨ New Features

  • Introduced `DD_AI_GUARD_OPENAI_ENABLED` environment variable as a per-provider kill switch for OpenAI SDK auto-instrumentation.
  • Added AI Guard support for LangChain 1.0+ agents built with `create_agent`.
  • Added opt-in evaluation of streamed OpenAI Chat Completions and Responses via `DD_AI_GUARD_ANALYZE_STREAM_RESPONSES_ENABLED`.
  • Introduced AI Guard standalone mode: traces are sent to Datadog even when APM tracing is disabled, attributed to AI Guard without triggering APM host billing.
  • Added opt-in buffered stream response evaluation for AI Guard on Anthropic streaming calls.
  • Added the normalized HTTP route span tag (`_dd.appsec.normalized_route`) for Tornado.
  • AppSec now detects OpenAI LLM usage within API endpoints, emitting the WAF address `server.business_logic.llm.event` to enable WAF rules to tag traces.
  • Added `aws.durable.operation_attempt` tag to `aws.durable.step` and `aws.durable.wait_for_condition` spans for AWS Durable Execution SDK.
  • Added instrumentation support for `ray[serve]>=2.47.1`, tracing Serve HTTP/gRPC requests, routing, method execution, and `DeploymentHandle.remote()` calls.
  • LLM Observability now supports attaching audio to messages via an optional `audio_parts` field in `LLMObs.annotate`.
  • Added a `sample_rate` argument to `LLMObs.enable()` to configure LLMObs trace sampling rate in code.
  • Added `SyncExperiment.rerun_evaluators()` to re-run evaluators on stored task outputs without re-executing the task function.
  • LLM span data can now be exported to the Datadog Agent over the APM trace and is reliably delivered even if the APM trace is not sampled.
  • OpenAI integration now captures input and output audio on audio chat completions as `audio_parts` on LLM span messages.
  • Added support for `DD_LLMOBS_SAMPLE_RATE` environment variable to control LLM Observability span retention.
  • Added span `metadata` to data exposed to span processors registered with `LLMObs.enable` or `LLMObs.register_processor`.
  • Added support for submitting LLM Observability spans for `mistralai` client chat and embedding calls.
  • Profiling implemented heap live count data collection.
  • Profiling revamped the adaptive sampling mechanism.
  • PyTorch profiler now properly shows nested frames in flame graphs.
  • Added tracing support for the `mistralai` client chat and embedding calls.
  • Added a `pytorch.rank` lifetime span for PyTorch distributed training, including relevant distributed tags.
  • Added ability to override a manually installed tracer for Single Step Instrumentation using `DD_INJECT_EXPERIMENTAL_OVERRIDE_USER_DDTRACE=true`.
  • The `DD_TEST_MANAGEMENT_ATTEMPT_TO_FIX_RETRIES` environment variable is now respected by the `ddtrace.testing` pytest plugin to override API-returned retry counts.

🐛 Bug Fixes

  • Fixed agentless export dropping for LLM Observability.

Affected Symbols

LLMObs.annotateLLMObs.enableLLMObs.register_processorSyncExperiment.rerun_evaluatorsDeploymentHandle.remoteinit_process_groupdestroy_process_group