py-1.43.0
📦 polarsView on GitHub →
✨ 20 features🐛 43 fixes⚡ 7 deprecations🔧 19 symbols
Summary
This release introduces performance improvements, new features like `ewm_sum` and `scan_arrow_c_stream`, and numerous bug fixes. Several features and methods are deprecated, including casting numeric types to categoricals and `LazyFrame.profile()`.
Migration Steps
- Rename `missing_utf8_is_empty_string` to `empty_string_is_null` in your code.
✨ New Features
- Pre-partition group-by on hive keys.
- Added `ewm_sum` and `ewm_sum_by`.
- Serve stale records from object_store DNS cache.
- Make `partition_hive` an opt flag.
- Added `scan_arrow_c_stream`.
- When displaying IRExprs, only add parentheses when needed for disambiguation.
- Support Iceberg field `initial-default` in native `scan_iceberg`.
- Show only differing dtypes in `assert_frame_equal` error.
- Expose ewm and rolling *_by expressions in Python visitor.
- More accurate rolling moments.
- Removed hard block on native scan for Iceberg V3 tables.
- Added `Series.degrees`/`radians` and clarified trig functions use radians.
- Merge sorted with multikey.
- Added `list` expression which consistently packs elements together into new List type.
- Added `read_database` Arrow fast-path for "python-oracledb".
- Optimize joins with redundant equi-join keys.
- Implemented missing datetime options for `CastColumnsPolicy`.
- Added `Expr.cat.to` and `Expr.cat.physical`.
- Added `POLARS_OOC_DISK_BUDGET_MB`.
- Added build side preference parameter for join.
🐛 Bug Fixes
- Avoid panic when union slice skips all rows.
- Solve panic in `dt.replace` when there were multiple chunks.
- Propagate `is_scalar` from the input to the output of `.sort()` and `.sort_by()`.
- Resolve CSV column names overwrite in DSL->IR conversion.
- Avoid IEJoin rewrite for Categorical comparisons.
- Do not rewrite `sort().reverse()` to `sort(descending=True)` when `maintain_order=True`.
- Ensure BinaryView offset+len does not exceed i32::MAX where possible.
- Invalid offset in strptime.
- Incorrect schema type for decimal <-> primitive division.
- Panic in in-memory CSEE handling.
- Fix offset in arrow ffi export of sliced struct arrays.
- Fix cross filter not applied with sink and CSE.
- Fix panic on projection pushdown with caches.
- Fix `write_json()` null values in `Array` columns being written incorrectly as `null`.
- Fix EntityTooSmall on sink_ipc to S3.
- Raise error instead of silent wrapping for `select(len())`.
- Float16 groupby aggregates.
- Respect lexical ordering of Categorical in `top_k`/`bottom_k`.
- Support dtype-list schema overrides in scan_csv.
- Resolve unknown type in fused multiply add.
- Honor `aws_checksum_algorithm` when provided.
- Remove usage of deprecated `pyarrow.feather.read_table`.
- Fix regression in sourcing HF_TOKEN.
- Raise on addition/subtraction between temporal and non-temporal Series.
- Avoid panic constructing Series from unaligned numpy arrays.
- Rewrite with_columns on empty df to select during DSL -> IR.
- Fix sink deadlock regression.
- Avoid panic when consumer of vertical concat is done.
- Harden IPC mmap API.
- Classify S3 region request failures as IO errors.
- Incorrect inner dtype in ArrayChunked::from_aligned_values.
- Raise on implode of `Object` dtype instead of creating invalid `List(Object)`.
- Return null not 0 for the first unbiased `ewm_var`/`ewm_std` value.
- Database appends with `ADBC` shouldn't require "CREATE" privileges unless the table is confirmed not to exist.
- Avoid slice pushdown panic on shared cache inputs.
- Incorrect dtype for Unknown right literal in arithmetic.
- Remove unnecessary unsafe Python object downcast.
- Normalize non-native byte order in numpy array construction.
- Use-after-free in numpy ufunc FFI.
- Unpickling for pl.when().then().
- Reduce unknown casts.
- Consider a `rolling_rank` position invalid if `ost.len() < min_samples`.
- Incorrect broadcast with empty len in pl.concat_list.
Affected Symbols
⚡ Deprecations
- Casting numeric types to categoricals is deprecated.
- `cat.get_categories()` and `cat.to_local()` are deprecated.
- `LazyFrame.profile()` is deprecated.
- Casts from non-nested dtypes into Lists are deprecated.
- Bitwise operations between integers and booleans are deprecated.
- `list/arr.to_struct()` calls that do not pass field names are deprecated.
- The `missing_utf8_is_empty_string` parameter has been renamed to `empty_string_is_null`.