v3.10.0-rc.1
Breaking Changes📦 prometheusView on GitHub →
⚠ 2 breaking✨ 9 features🐛 13 fixes🔧 10 symbols
Summary
This release introduces a secure distroless Docker image variant and significant PromQL enhancements, including new `fill()` modifiers and performance improvements across the query engine and TSDB. Several bugs related to OTLP handling, PromQL functions, and memory leaks have also been resolved.
⚠️ Breaking Changes
- Users migrating existing **named** volumes from the busybox image to the distroless variant must manually adjust ownership of the volume data to UID/GID 65532 using `docker run --rm -v <volume_name>:/prometheus alpine chown -R 65532:65532 /prometheus` before starting the container with the distroless image.
- The distroless Docker image variant removes the VOLUME declaration, meaning named volumes or bind mounts must be explicitly configured when running the container.
Migration Steps
- If using named volumes with the busybox image, run `docker run --rm -v <volume_name>:/prometheus alpine chown -R 65532:65532 /prometheus` to adjust ownership before using the volume with the distroless image.
- When using the new distroless image, ensure any required volumes or bind mounts are explicitly specified in the `docker run` command, as the `VOLUME` declaration has been removed.
✨ New Features
- Introduction of a distroless Docker image variant for enhanced security, using UID/GID 65532 (nonroot) and omitting the VOLUME declaration.
- Addition of the `alertmanager` dimension to metrics: `prometheus_notifications_dropped_total`, `prometheus_notifications_queue_capacity`, and `prometheus_notifications_queue_length`.
- AWS Service Discovery now supports MSK Role.
- Addition of `fill()`, `fill_left()`, and `fill_right()` binop modifiers in PromQL for specifying default values for missing series.
- Addition of OpenAPI 3.2 specification for the HTTP API available at `/api/v1/openapi.yaml`.
- Addition of on-demand wall time profiling accessible at `<URL>/debug/pprof/fgprof`.
- Experimental support for early compaction of stale series in the TSDB memory, configurable via `stale_series_compaction_threshold` in the configuration file.
- Service discoveries can now be conditionally included or excluded during Prometheus binary compilation using Go build tags (`remove_all_sd`, `enable_<sd name>_sd`).
- Promtool now supports PromQL syntax features `promql-duration-expr` and `promql-extended-range-selectors`.
🐛 Bug Fixes
- Prevented Prometheus from crashing when encountering query strings containing only UTF-8 continuation bytes.
- Fixed missing `X-Prometheus-Stopping` header for the `/-/ready` endpoint when Prometheus is in the `NotReady` state.
- Fixed the PromQL `info()` function returning empty results when filtering by a label present on both the input metric and `target_info`.
- Fixed a bug during exemplar buffer grow/shrink operations that could lead to incorrect discarding of exemplars.
- Fixed broken graph display after page reload caused by incorrect Y axis min encoding/decoding in the UI.
- Fixed memory leaks in TSDB buffer pools by ensuring reference fields (Labels, Histogram pointers, metadata strings) are cleared before buffers are returned to pools.
- Fixed the PromQL info function so that series without identifying labels are now correctly returned.
- Filtered the `__name__` label from OTLP attributes during conversion to prevent duplicate labels.
- Fixed division by zero error when computing the stale series ratio if the TSDB head was empty.
- Fixed potential silent data loss for sum metrics during OTLP processing.
- Fixed smoothed interpolation calculation across counter resets in PromQL.
- Fixed a panic occurring in PromQL when using the `@` modifier on empty ranges.
- Fixed incorrect calculation of `avg_over_time` for a single native histogram.