Change8

v4.13.0rc1

Breaking Changes
📦 datadog-sdkView on GitHub →
1 breaking25 features🐛 1 fixes5 deprecations🔧 19 symbols

Summary

This release introduces significant enhancements to LLM Observability, including prompt management, image attachment, and OpenAI Realtime API instrumentation. It also brings improvements to tracing safety, OpenTelemetry integration, AI Guard features, and crash tracking.

⚠️ Breaking Changes

  • The `ml_app` argument and `DD_LLMOBS_ML_APP` environment variable for LLM Observability are deprecated and will be removed in the next major version. Use `agent_service` and `DD_SERVICE` instead to group traces belonging to a root agentic service.

Migration Steps

  1. Update imports from `ddtrace.appsec.ai_guard` to `ddtrace.aiguard`.
  2. For Strands Agents classes, import them from `ddtrace.aiguard.integrations.strands`.
  3. Remove the `DD_PYTEST_USE_NEW_PLUGIN` environment variable from your configuration.
  4. Update imports such as `from ddtrace.appsec.ai_guard import AIGuardClient` to `from ddtrace.aiguard import AIGuardClient`.
  5. Set `DD_ENV` instead of using the `label` parameter for `LLMObs.get_prompt()` and `LLMObs.refresh_prompt()`.
  6. Use `agent_service` and `DD_SERVICE` instead of the `ml_app` argument and `DD_LLMOBS_ML_APP` environment variable for LLM Observability.
  7. When using Bedrock calls with an application-inference-profile ARN as `modelId`, ensure `DD_LLMOBS_ENABLED=true` and you have `bedrock:GetInferenceProfile` IAM permission if you want to resolve the underlying foundation model.
  8. Set `DD_ENV` to select the prompt version for a specific environment when using `LLMObs.get_prompt()`.
  9. Set `DD_OPENAI_REALTIME_ENABLED=false` to disable Realtime API instrumentation for the OpenAI integration.
  10. Set `DD_EXPERIMENTAL_FLAGGING_PROVIDER_SPAN_ENRICHMENT_ENABLED=true` to enable experimental APM span enrichment for the OpenFeature provider.

✨ New Features

  • Trace and span IDs are now generated more safely in AWS Lambda MicroVM environments, reducing the risk of duplicate IDs.
  • Adds OpenTelemetry thread-context publication on Linux, enabling compatible profiling tools to correlate samples with the active trace and span.
  • Adds `DD_TRACE_OTEL_SEMANTICS_ENABLED` to opt in to OpenTelemetry trace semantics for spans created through OpenTelemetry APIs and for OpenTelemetry Protocol (OTLP) trace export.
  • Introduces per-integration kill-switch environment variables for AI Guard: `DD_AI_GUARD_ANTHROPIC_ENABLED` and `DD_AI_GUARD_LANGCHAIN_ENABLED` (both `true` by default).
  • Add support for the `DD_APPSEC_AGENTIC_ONBOARDING` environment variable for AAP.
  • Added `DD_BOTOCORE_BEDROCK_RESOLVE_INFERENCE_PROFILE` (default `false`) to resolve underlying foundation models for Bedrock calls using an application-inference-profile ARN as `modelId`.
  • Introduces capture of session metadata (session id, user id, and app name) from the Google ADK `Session` object on LLM Observability spans.
  • Spans generated under an agent are now automatically attributed to their nearest agent ancestor.
  • Added `agent_service` as a preferred alias for the `ml_app` argument across LLMObs SDK methods.
  • Auto-instrumented LLM integrations now automatically link Datadog Managed Prompts retrieved with `LLMObs.get_prompt()` and rendered with `.format()` to the resulting LLM span.
  • `LLMObs.get_prompt()` now resolves environment-scoped prompt versions and supports targeting for A/B testing. Set `DD_ENV` to select the version for that environment. The signature is now keyword-only after `prompt_id` and accepts `targeting_key` and `**attributes`.
  • Adds a `version` argument to `LLMObs.get_prompt()` for retrieving an exact prompt version.
  • Introduces support for attaching images to messages on LLM spans. Input and output messages annotated via `LLMObs.annotate` now accept an optional `image_parts` field.
  • The OpenAI integration now automatically instruments the Realtime API (`client.realtime.connect(...)`, both sync and async).
  • Adds prompt management methods to `LLMObs` for creating, updating, deleting, and listing prompts and prompt versions in the prompt registry.
  • Adds the `env_ids` argument to prompt creation and version update methods, allowing prompt versions to be deployed to feature-flag environments.
  • `session_id` now propagates to following spans in a trace.
  • Adds support for submitting LLM Observability spans for synchronous and asynchronous streaming chat completion calls using the `mistralai` client.
  • Adds `DD_CODE_COVERAGE_FLAGS` to group and filter uploaded code coverage reports for CI visibility.
  • Unhandled Python exceptions are now reported through the crashtracker, providing crash reports with exception type, message, and Python stack trace.
  • Enables obfuscation by default for stats computation.
  • Exception profiling is now enabled by default when the profiler is active on Python 3.12+.
  • Exception profiling now captures the full Python call stack at the point of the raise.
  • Adds experimental APM span enrichment for the Feature Flagging and Experimentation (FFE) OpenFeature provider when `DD_EXPERIMENTAL_FLAGGING_PROVIDER_SPAN_ENRICHMENT_ENABLED=true`.
  • The experimental OpenFeature provider now reports aggregated server-side flag evaluations to Datadog.

🐛 Bug Fixes

  • Managed prompts no longer expose deprecated deployment label metadata. #19047

Affected Symbols

⚡ Deprecations

  • The `ml_app` argument and `DD_LLMOBS_ML_APP` environment variable for LLM Observability are deprecated and will be removed in the next major version. Use `agent_service` and `DD_SERVICE` instead.
  • The public SDK for AI Guard has moved from the `ddtrace.appsec.ai_guard` package to the top-level `ddtrace.aiguard` package. Importing from `ddtrace.appsec.ai_guard` is deprecated and will be removed in 5.0.0. Update imports from `ddtrace.appsec.ai_guard` to `ddtrace.aiguard`. Strands Agents classes should be imported from `ddtrace.aiguard.integrations.strands`.
  • The legacy pytest plugin (`ddtrace/contrib/internal/pytest`) is deprecated and will be removed in ddtrace 5.0.0. This plugin is active only when `DD_PYTEST_USE_NEW_PLUGIN` is explicitly set to `false` or `0`. The new pytest plugin (`ddtrace/testing/internal/pytest`) is the default. To migrate, remove the `DD_PYTEST_USE_NEW_PLUGIN` environment variable.
  • The `label` parameter of `LLMObs.get_prompt()` and `LLMObs.refresh_prompt()` is deprecated. Set `DD_ENV` instead.
  • Managed prompts no longer expose deprecated deployment label metadata.