v4.4.0
📦 datadog-sdkView on GitHub →
✨ 6 features🐛 20 fixes🔧 16 symbols
Summary
This release introduces significant enhancements to LLM Observability with class-based evaluators and concurrent synchronous experiment execution. It also adds support for LFI detection and Tornado framework integration for AAP, alongside numerous bug fixes across profiling, exception replay, and various integrations.
Migration Steps
- If using Tornado, explicitly enable AAP support via `DD_TRACE_TORNADO_ENABLED=true` or `DD_PATCH_MODULES=tornado:true`.
✨ New Features
- Adds support for class-based evaluators in LLM Observability by subclassing `BaseEvaluator`. The `EvaluatorContext` stores evaluation context, and `BaseSummaryEvaluator` supports summary evaluations.
- Adds support for running synchronous evaluators concurrently in experiments (async evaluators are not supported).
- Adds a new environment variable `DD_TRACE_LOG_LEVEL` to control the ddtrace logger level.
- Added support for capture expressions in log probes for dynamic instrumentation.
- Introduces support for Local File Inclusion (LFI) detection in `pathlib.Path.open()` for App and API Protection Exploit Prevention.
- Introduces AAP support for tornado web framework, enabled via `DD_TRACE_TORNADO_ENABLED=true` or `DD_PATCH_MODULES=tornado:true`.
🐛 Bug Fixes
- Fixes an issue where agent-based samplers could interfere with Standalone App and API Protection by rejecting traces before the custom AAP sampler was evaluated.
- Resolves an issue where user-defined SIGALRM handlers were not restored after TimeoutChannel cleanup in aws_lambda, causing custom timeout handlers to stop working after the first invocation.
- Fixes a gevent support issue in exception replay that could cause an exception when determining if a frame belongs to user code.
- Fixes errors while capturing exception replay snapshots.
- Resolves an issue where litellm>=1.74.15 wrapped router streaming responses in `FallbackStreamWrapper`, causing an `AttributeError` when accessing `.handler`. The integration now handles both original and wrapped responses.
- Fixed a profiling bug where non-pushed samples could leak data to subsequent samples.
- Fixed a profiling bug where `asyncio` task stacks could contain duplicated frames when the task was on-CPU.
- The stack Profiler now correctly resets thread, task, and greenlet information after a fork, preventing stale data from the parent process.
- Fixed crash in lock profiler when stack traces are too shallow (less than 4 frames); locks may now show location "unknown:0" instead of crashing.
- Fixed an issue that causes greenlets to misbehave when `gevent.joinall` is called.
- Resolves a crash occurring when forking while using the memory profiler.
- The Profiler now always reports CPU time for threads, regardless of their running state during sampling.
- Ensures the memory profiler clears its internal state immediately after fork in child processes via pthread_atfork.
- Resolves an issue where the Pydantic AI integration was not properly tracing `StreamedRunResult.stream_responses()` (introduced in `pydantic-ai==0.8.1`), leading to unfinished agent spans.
- Addresses an issue where the evaluators argument type for `LLMObs.experiment` was overly constrained; it now uses the covariant Sequence type.
- Fixes an issue where OpenAI spans showed model_name: "None" instead of falling back to the request model if the API response returned None for the model field.
- Resolves panics in the NativeWriter caused by celery closing file descriptors when starting beat.
- Remove noisy warning messages for non-callable view objects that cannot be instrumented for code origin.
- Fixed an issue that prevented instrumented probes from being removed correctly when Dynamic Instrumentation is disabled remotely from the Datadog UI.
- Fixes a regression introduced in v4.3.0 that prevented DSM from being automatically enabled for Kafka, AioKafka, Kombu, and Botocore integrations.