Change8

py-1.43.0

📦 polarsView on GitHub →
20 features🐛 43 fixes7 deprecations🔧 19 symbols

Summary

This release introduces performance improvements, new features like `ewm_sum` and `scan_arrow_c_stream`, and numerous bug fixes. Several features and methods are deprecated, including casting numeric types to categoricals and `LazyFrame.profile()`.

Migration Steps

  1. Rename `missing_utf8_is_empty_string` to `empty_string_is_null` in your code.

✨ New Features

  • Pre-partition group-by on hive keys.
  • Added `ewm_sum` and `ewm_sum_by`.
  • Serve stale records from object_store DNS cache.
  • Make `partition_hive` an opt flag.
  • Added `scan_arrow_c_stream`.
  • When displaying IRExprs, only add parentheses when needed for disambiguation.
  • Support Iceberg field `initial-default` in native `scan_iceberg`.
  • Show only differing dtypes in `assert_frame_equal` error.
  • Expose ewm and rolling *_by expressions in Python visitor.
  • More accurate rolling moments.
  • Removed hard block on native scan for Iceberg V3 tables.
  • Added `Series.degrees`/`radians` and clarified trig functions use radians.
  • Merge sorted with multikey.
  • Added `list` expression which consistently packs elements together into new List type.
  • Added `read_database` Arrow fast-path for "python-oracledb".
  • Optimize joins with redundant equi-join keys.
  • Implemented missing datetime options for `CastColumnsPolicy`.
  • Added `Expr.cat.to` and `Expr.cat.physical`.
  • Added `POLARS_OOC_DISK_BUDGET_MB`.
  • Added build side preference parameter for join.

🐛 Bug Fixes

  • Avoid panic when union slice skips all rows.
  • Solve panic in `dt.replace` when there were multiple chunks.
  • Propagate `is_scalar` from the input to the output of `.sort()` and `.sort_by()`.
  • Resolve CSV column names overwrite in DSL->IR conversion.
  • Avoid IEJoin rewrite for Categorical comparisons.
  • Do not rewrite `sort().reverse()` to `sort(descending=True)` when `maintain_order=True`.
  • Ensure BinaryView offset+len does not exceed i32::MAX where possible.
  • Invalid offset in strptime.
  • Incorrect schema type for decimal <-> primitive division.
  • Panic in in-memory CSEE handling.
  • Fix offset in arrow ffi export of sliced struct arrays.
  • Fix cross filter not applied with sink and CSE.
  • Fix panic on projection pushdown with caches.
  • Fix `write_json()` null values in `Array` columns being written incorrectly as `null`.
  • Fix EntityTooSmall on sink_ipc to S3.
  • Raise error instead of silent wrapping for `select(len())`.
  • Float16 groupby aggregates.
  • Respect lexical ordering of Categorical in `top_k`/`bottom_k`.
  • Support dtype-list schema overrides in scan_csv.
  • Resolve unknown type in fused multiply add.
  • Honor `aws_checksum_algorithm` when provided.
  • Remove usage of deprecated `pyarrow.feather.read_table`.
  • Fix regression in sourcing HF_TOKEN.
  • Raise on addition/subtraction between temporal and non-temporal Series.
  • Avoid panic constructing Series from unaligned numpy arrays.
  • Rewrite with_columns on empty df to select during DSL -> IR.
  • Fix sink deadlock regression.
  • Avoid panic when consumer of vertical concat is done.
  • Harden IPC mmap API.
  • Classify S3 region request failures as IO errors.
  • Incorrect inner dtype in ArrayChunked::from_aligned_values.
  • Raise on implode of `Object` dtype instead of creating invalid `List(Object)`.
  • Return null not 0 for the first unbiased `ewm_var`/`ewm_std` value.
  • Database appends with `ADBC` shouldn't require "CREATE" privileges unless the table is confirmed not to exist.
  • Avoid slice pushdown panic on shared cache inputs.
  • Incorrect dtype for Unknown right literal in arithmetic.
  • Remove unnecessary unsafe Python object downcast.
  • Normalize non-native byte order in numpy array construction.
  • Use-after-free in numpy ufunc FFI.
  • Unpickling for pl.when().then().
  • Reduce unknown casts.
  • Consider a `rolling_rank` position invalid if `ost.len() < min_samples`.
  • Incorrect broadcast with empty len in pl.concat_list.

Affected Symbols

⚡ Deprecations

  • Casting numeric types to categoricals is deprecated.
  • `cat.get_categories()` and `cat.to_local()` are deprecated.
  • `LazyFrame.profile()` is deprecated.
  • Casts from non-nested dtypes into Lists are deprecated.
  • Bitwise operations between integers and booleans are deprecated.
  • `list/arr.to_struct()` calls that do not pass field names are deprecated.
  • The `missing_utf8_is_empty_string` parameter has been renamed to `empty_string_is_null`.