●Change8

v4.8.0

Breaking Changes
📦 datadog-sdkView on GitHub →
⚠ 2 breaking✨ 17 features🐛 1 fixes⚡ 3 deprecations🔧 30 symbols

Summary

This release introduces significant new features across AI Observability (LLM, AI Guard, ASM), CI Visibility (Bazel offline support), and various integrations like Ray, Pub/Sub, and LlamaIndex. It also includes several breaking changes related to resource naming in claude_agent_sdk and Ray tracing configuration.

⚠️ Breaking Changes

  • Tool span resource names for claude_agent_sdk have changed from the tool name (e.g. "Read", "Bash") to "claude_agent_sdk.tool". Users relying on tool resource names must update them to include the specific tool name as a suffix (e.g. "claude_agent_sdk.tool.Read").
  • Ray submission tracing configuration has changed: `ray.job.submit` spans are removed. Ray job submission outcome is now reported on the existing `ray.job` span via `ray.job.submit_status`.

Migration Steps

  1. If you rely on tool resource names for `claude_agent_sdk` spans, update your resource name matching logic to expect the format `claude_agent_sdk.tool.<tool_name>` instead of just `<tool_name>`.
  2. If you were using `DD_TRACE_INFERRED_PROXY_SERVICES_ENABLED`, rename it to `DD_TRACE_INFERRED_SPANS_ENABLED` to avoid deprecation warnings.
  3. If you were using the `pin` parameter in DBAPI traced connections/cursors, remove it and manage configuration via integration configuration or environment variables.
  4. If using RAGAS evaluations, update your code to manually submit evaluation results.
  5. To enable tracing of Ray submission spans (`task.submit`, `actor_method.submit`), set the environment variable `DD_TRACE_RAY_SUBMISSION_SPANS_ENABLED=true`.
  6. To control whether inferred Google Cloud Pub/Sub push spans link to the producer trace or start a new trace, use `DD_GOOGLE_CLOUD_PUBSUB_PROPAGATION_AS_SPAN_LINKS`.
  7. To enable AI Guard blocking behavior locally based on Datadog UI settings, ensure `DD_AI_GUARD_BLOCK` is set to `True` (default) or unset. Set to `False` for monitor-only mode.
  8. To enable tracing via `@tracer.wrap` to include the class name in the span name, set `DD_TRACE_WRAP_SPAN_NAME_INCLUDE_CLASS=true`.
  9. If using LLM Observability span retrieval, ensure `DD_API_KEY` and `DD_APP_KEY` are configured for API access.
  10. If using CI Visibility log submission, set `DD_AGENTLESS_LOG_SUBMISSION_ENABLED=true` for agentless setups or `DD_LOGS_INJECTION=true` when using the Datadog Agent.
  11. If you require OTLP HTTP/JSON export, set the environment variable `OTEL_TRACES_EXPORTER=otlp`.

✨ New Features

  • Added support for Bazel offline execution in CI Visibility using manifest mode (`DD_TEST_OPTIMIZATION_MANIFEST_FILE`) and payload-files mode (`DD_TEST_OPTIMIZATION_PAYLOADS_IN_FILES`).
  • Added tracing for individual Claude model turns within a Claude Agent SDK session, capturing input/output messages, model name, and token usage metrics (requires claude_agent_sdk >= 0.1.49).
  • Added Application Security support for FastAPI/Starlette applications using `app.mount()`, ensuring correct WAF evaluation, path extraction, and endpoint reporting relative to mount prefixes.
  • Added tracing for Google Cloud Pub/Sub admin operations (topic, subscription, snapshot, schema management).
  • Added tracing support for Google Cloud Pub/Sub push subscriptions, creating an inferred `gcp.pubsub.receive` span upon message delivery.
  • Added `ExperimentRun.as_dataframe()` method to convert experiment run results into a `pandas.DataFrame`.
  • Added an `eval_scope` parameter to `LLMObs.submit_evaluation()` to associate evaluations with the entire trace (`"trace"`) or a specific span (`"span"`).
  • Added `LLMObs.get_spans()` to retrieve LLM Observability span events from the Datadog platform API, supporting various filters and auto-pagination.
  • Profiles generated from fork-based servers now include a `process_type` tag (`main` or `worker`).
  • Added support for thread sub-sampling in profiling to limit the number of threads captured at each sampling interval, reducing CPU overhead.
  • Added APM tracing and LLM Observability support for `llama-index-core>=0.11.0`, tracing LLM calls, retrievers, embeddings, and agents.
  • Added a LiteLLM proxy guardrail integration (`DatadogAIGuardGuardrail`) for Datadog AI Guard, requiring `litellm>=1.46.1`.
  • Added tracing support for Azure CosmosDB CRUD operations on databases, containers, and items.
  • Introduced a `decorator` tag on LLM Observability spans traced via a function decorator.
  • Added automatic log correlation and submission for CI Visibility test runs; enable via `DD_AGENTLESS_LOG_SUBMISSION_ENABLED` or `DD_LOGS_INJECTION`.
  • Added support for exporting traces in OTLP HTTP/JSON format via libdatadog by setting `OTEL_TRACES_EXPORTER=otlp`.
  • LLM Observability experiments now accept a pydantic `ReportEvaluator` returning `ScalarResult` as a summary evaluator.

🐛 Bug Fixes

  • Ray submission tracing configuration now defaults to disabling submission spans (`DD_TRACE_RAY_SUBMISSION_SPANS_ENABLED=False`) unless explicitly enabled, ensuring only execution spans are traced by default.

Affected Symbols

claude_agent_sdkDD_TRACE_RAY_SUBMISSION_SPANS_ENABLEDray.job.submitray.jobDD_TRACE_INFERRED_PROXY_SERVICES_ENABLEDDD_TRACE_INFERRED_SPANS_ENABLEDddtrace.contrib.dbapi.TracedConnectionddtrace.contrib.dbapi.TracedCursorddtrace.contrib.dbapi_async.TracedAsyncConnectionDD_AI_GUARD_BLOCKAIGuardAbortErrorDD_TEST_OPTIMIZATION_MANIFEST_FILEDD_TEST_OPTIMIZATION_PAYLOADS_IN_FILESExperimentRun.as_dataframeLLMObs.submit_evaluationLLMObs.get_spansDD_API_KEYDD_APP_KEYDD_TRACE_WRAP_SPAN_NAME_INCLUDE_CLASSDD_WRAP_SPAN_NAME_INCLUDE_CLASSDD_AI_GUARD_BLOCKgoogle_cloud_pubsubDD_GOOGLE_CLOUD_PUBSUB_PROPAGATION_AS_SPAN_LINKSOTEL_TRACES_EXPORTERddtrace.appsec.ai_guard.integrations.litellm.DatadogAIGuardGuardraillitellmazure_cosmosllama-index-coreDD_AGENTLESS_LOG_SUBMISSION_ENABLEDDD_LOGS_INJECTION

⚡ Deprecations

  • The environment variable `DD_TRACE_INFERRED_PROXY_SERVICES_ENABLED` is deprecated and will be removed in 5.0.0; use `DD_TRACE_INFERRED_SPANS_ENABLED` instead.
  • The `pin` parameter in `ddtrace.contrib.dbapi.TracedConnection`, `ddtrace.contrib.dbapi.TracedCursor`, and `ddtrace.contrib.dbapi_async.TracedAsyncConnection` is deprecated and will be removed in version 5.0.0. Configuration should now use integration configuration and environment variables.
  • Support for the RAGAS integration in LLM Observability is removed. Users should manually submit RAGAS evaluation results as an alternative.