●Change8

py-1.24.0

📦 polarsView on GitHub →
✨ 10 features🐛 14 fixes🔧 13 symbols

Summary

This release focuses heavily on performance improvements, particularly in the new streaming engine, and introduces several enhancements like lossy decoding for CSVs and DataFrame.write_iceberg. Numerous bug fixes address stability issues across various operations including rolling statistics and sink phases.

Migration Steps

  1. If you were relying on the old parameter name for `is_in`, update your code to use `nulls_equal` instead of `join_nulls` (as `join_nulls` was renamed in joins, and `is_in` adopted the new name).

✨ New Features

  • Added lossy decoding to `read_csv` for non-utf8 encodings.
  • Added `DataFrame.write_iceberg` functionality.
  • Added 'nulls_equal' parameter to `is_in`.
  • Improved numeric stability for `rolling_{std, var, cov, corr}` operations.
  • Implemented IR Serde cross-filter.
  • Gave priority to pycapsule interface in `from_dataframe`.
  • Support writing `Time` type in
  • Activated all optimizations in sinks.
  • Added `AssertionError` variant to `PolarsError` in `polars-error`.
  • Pass filter to inner readers in multiscan new streaming.

🐛 Bug Fixes

  • Fixed Categorical min/max panicking when string cache is enabled.
  • Prevented encoding IPC record batch twice.
  • Ensured rewriting flag is respected in Node rewriter.
  • Corrected skip batch predicate for partial statistics.
  • Made the Parquet Sink properly phase aware.
  • Avoided division by zero in partitioned group-by.
  • Created new linearizer between rowwise new streaming sink phases.
  • Prevented dropping rows in sinks between new streaming phases.
  • Fixed incorrect lazy schema for `Expr.list.diff`.
  • Fixed Duration Series arithmetic operations.
  • Fixed unwrap None panic when filtering delta with missing columns.
  • Used stable sort for rolling-groupby.
  • Threw exception if dataframe is too large to be compatible with Excel.
  • Addressed regression where `read_excel` did not handle URL paths correctly.

🔧 Affected Symbols

read_csvDataFrame.write_icebergis_inrolling_stdrolling_varrolling_covrolling_corrfrom_dataframePolarsErrorExpr.list.diffsink_parquetmerge_sortedread_excel