v4.7.0rc4
📦 datadog-sdkView on GitHub →
✨ 23 features🐛 1 fixes🔧 10 symbols
Summary
This release significantly enhances LLM Observability with new features like Pydantic evaluation support, remote evaluators, and reasoning capture. It also introduces process tag propagation across several components and updates dependencies for OpenFeature.
Migration Steps
- If using OpenFeature, ensure `openfeature-sdk` is updated to version 0.8.0 or higher.
- If using AWS Strands Agents, the recommended integration path is the Plugin API, requiring `strands-agents>=1.29.0`. The legacy HookProvider API is also supported.
- To disable the new process tag propagation feature across various components (profiling, runtime metrics, RC, etc.), set the environment variable `DD_EXPERIMENTAL_PROPAGATE_PROCESS_TAGS_ENABLED=false`.
✨ New Features
- Adds a request header provider (auth plugin) for MLFlow, including DD-API-KEY and DD-APPLICATION-KEY headers if environment variables are set.
- AI Guard evaluation calls now block by default if blocking is enabled in the UI; this can be disabled with `block=False`.
- AI Guard SDK response now includes Sensitive Data Scanner (SDS) results.
- Introduces AI Guard support for the AWS Strands Agents SDK via Plugin API (recommended, requires strands-agents >= 1.29.0) and HookProvider API (legacy).
- Adds tracing support for Azure Durable Functions, tracing durable activity and entity functions.
- Adds process tags to profiler payloads (can be deactivated via DD_EXPERIMENTAL_PROPAGATE_PROCESS_TAGS_ENABLED).
- Adds process tags to runtime metrics tags (can be deactivated via DD_EXPERIMENTAL_PROPAGATE_PROCESS_TAGS_ENABLED).
- Adds process tags to remote configuration payloads (can be deactivated via DD_EXPERIMENTAL_PROPAGATE_PROCESS_TAGS_ENABLED).
- Adds process tags to debugger payloads (can be deactivated via DD_EXPERIMENTAL_PROPAGATE_PROCESS_TAGS_ENABLED).
- Adds process tags to crash tracking payloads (can be deactivated via DD_EXPERIMENTAL_PROPAGATE_PROCESS_TAGS_ENABLED).
- Adds process tags to Data Streams Monitoring payloads (can be deactivated via DD_EXPERIMENTAL_PROPAGATE_PROCESS_TAGS_ENABLED).
- Adds process tags to Database Monitoring SQL service hash propagation (can be deactivated via DD_EXPERIMENTAL_PROPAGATE_PROCESS_TAGS_ENABLED).
- Adds process tags to stats computation payloads (can be deactivated via DD_EXPERIMENTAL_PROPAGATE_PROCESS_TAGS_ENABLED).
- LLM Observability: Adds support for capturing `stop_reason` and `structured_output` from the Claude Agent SDK integration.
- LLM Observability: Adds support for user-defined dataset record IDs via an optional `id` field in dataset creation/appending methods.
- LLM Observability: Experiment tasks can now optionally receive dataset record metadata as a third `metadata` parameter.
- LLM Observability: Introduces `RemoteEvaluator` to reference LLM-as-Judge evaluations configured in the Datadog UI by name during local experiments.
- LLM Observability: Adds cache creation breakdown metrics for the Anthropic integration (`ephemeral_5m_input_tokens` and `ephemeral_1h_input_tokens`).
- LLM Observability: Adds support for reasoning and extended thinking content in Anthropic, LiteLLM, and OpenAI-compatible integrations.
- LLM Observability: `LLMJudge` now forwards extra `client_options` to the underlying provider client constructor.
- LLM Observability: Adds three new `Dataset` methods for tag management: `dataset.add_tags`, `dataset.remove_tags`, and `dataset.replace_tags`.
- LLM Observability: Changes experiment execution to run evaluators immediately after each record's task completes instead of batching, posting spans incrementally.
- LLM Observability: Adds support for Pydantic AI evaluations in LLM Observability Experiments.
🐛 Bug Fixes
- Flag evaluations for non-existent flags now return `Reason.ERROR` with `ErrorCode.FLAG_NOT_FOUND` instead of `Reason.DEFAULT` when configuration is available but the flag is not found (preserves `Reason.DEFAULT` if no configuration is loaded).