●Change8

v3.13.0

Breaking Changes
📦 prometheusView on GitHub →
⚠ 2 breaking✨ 10 features🐛 19 fixes🔧 19 symbols

Summary

This LTS release introduces experimental API search endpoints, significant PromQL enhancements including new scalar functions and native histogram support, and several performance improvements across the TSDB and label matching. It also contains critical security fixes and changes how HTTP credentials are handled during redirects.

⚠️ Breaking Changes

  • HTTP clients no longer forward credentials (Authorization header, basic auth, bearer token, OAuth2, configured headers) when following a redirect to a different host. This affects scraping, remote read/write, alerting, and service discovery. Review configurations that rely on cross-host redirects retaining credentials.
  • Relative file paths passed to `promtool --http.config.file` are now resolved relative to that config file's directory instead of its parent directory. Update any scripts using relative paths in this configuration file.

Migration Steps

  1. If relying on credentials being forwarded across host redirects for scraping, remote read/write, alerting, or service discovery, reconfigure those services or use explicit hostnames/IPs.
  2. If using `promtool --http.config.file` with relative paths, ensure paths are relative to the config file itself, not its parent directory.

✨ New Features

  • Added experimental search endpoints to search metric names, label names, and label values via the API.
  • Added ability to filter RDS instances in AWS service discovery.
  • Added experimental scalar functions `min_of(a, b)` and `max_of(a, b)` to return the smaller or larger of two scalar values.
  • Added support for smoothed/anchored rate with native histograms in PromQL.
  • Exposed per-query `samplesRead` (and `samplesReadPerStep` with `stats=all` and the `promql-per-step-stats` feature flag) in the query stats response, and added the `prometheus_engine_query_samples_read_total` engine counter.
  • Added internal label `__convert_classic_histograms_to_nhcb__` to allow per-target override of `convert_classic_histograms_to_nhcb` scrape configuration via relabeling.
  • Added `storage.tsdb.chunk_encoding.floats` configuration field to select float chunk encoding (`xor` or `xor2`) at runtime.
  • Added Certificate support for ingesting data into an Azure Monitor Workspace via remote_write.
  • Added internal labels `__always_scrape_classic_histograms__` and `__scrape_native_histograms__` to allow per-target override of corresponding scrape configuration via relabeling.
  • Container images are now also published to the GitHub Container Registry (ghcr.io).

🐛 Bug Fixes

  • Fixed a cross-site scripting vulnerability in the UI by bumping `sanitize-html` (CVE-2026-44990).
  • Fixed failure when processing an AWS RDS cluster without instances.
  • Fixed race condition in initTime that could cause ErrOutOfBounds.
  • Fixed PromQL range query behavior where unaligned `end` caused subqueries to evaluate past the last actual step, inflating stats and potentially hitting limits.
  • Fixed PromQL issue where range queries with an at-modifier-unsafe function over a range-vector with an `@` modifier silently under-counted `totalQueryableSamples` for steps after step 0.
  • Fixed `fill_left`/`fill_right` producing missing samples in range queries when using `group_left`/`group_right` in PromQL.
  • Fixed `resets()` and `changes()` behavior in anchored range extenders with histograms.
  • Fixed panic on `1[5m] smoothed` and similar expressions when extended range selectors are enabled.
  • Fixed panic when a `smoothed` instant vector selector produces no samples for a series.
  • Fixed panic when using a parenthesised plain number as an offset (e.g. `foo offset -(5)`).
  • Fixed panic in `promtool` when parsing exposition text containing empty braces `{}`.
  • Fixed `promtool check healthy` and `check ready` when `--url` ends with a trailing slash.
  • Closed PromQL query after each rule evaluation to ensure resources are released.
  • Resolved VPC/IPAM-only Scaleway SD instances that lack legacy IP fields but have private NICs attached.
  • Prevented leaking head series when an integer histogram append is rejected (e.g. out-of-order) in TSDB.
  • Escaped label values offered by PromQL autocomplete in the UI.
  • Fixed chunk snapshot encoding for EncXOR2 chunks, preventing corruption on TSDB restart.
  • Stored a millisecond timestamp instead of a WAL segment number in `walExpiries` when a series is evicted, ensuring correct replay after WAL checkpointing.
  • Prevented loss of samples at the chunk-range boundary during series eviction via compaction by ensuring the boundary timestamp is captured before in-memory copy removal.

Affected Symbols

min()max()min_of()max_of()prometheus_engine_query_samples_read_total__convert_classic_histograms_to_nhcb____always_scrape_classic_histograms____scrape_native_histograms__storage.tsdb.chunk_encoding.floatsfill_left(x)fill_right(x)fill(x)predict_linear(metric[60s] @ T, X)resets()changes()smoothedpromtoolcheck healthycheck ready