●Change8

py-1.44.0

📦 polarsView on GitHub →
✨ 21 features🐛 43 fixes⚡ 3 deprecations🔧 39 symbols

Summary

This release introduces several performance improvements, new features like Iceberg schema evolution and correlated subqueries, and numerous bug fixes. It also deprecates the `rechunk` parameter and related functions.

✨ New Features

  • Support integer fixed-array dot products (#28829)
  • SQL correlated in (#28927)
  • Support more join_types in `join_where` (#28880)
  • Support Iceberg schema evolution (#28794)
  • SQL conformance (#28494)
  • Introduce `RemoteEngine` and a common base class for all engines (#28800)
  • Support Iceberg table with V3 deletion vectors in native `scan_iceberg` (#28772)
  • Support Iceberg snapshot properties (#28793)
  • Expose `sinked_paths_callback` on `sink_parquet` as unstable parameter (#28814)
  • Add `infer_schema_files` to CSV inference hint (#28809)
  • Add native fixed-array dot (#28504)
  • Export query metrics to polars_cloud (#28757)
  • Support Iceberg object storage paths (#28634)
  • More CSEE simplification (#28731)
  • Improve error in Series constructor with name in context (#28743)
  • Add adaptive HTTP rate-limiter for cloud IO (#28591)
  • Add `struct.drop()` (#28666)
  • Improve error message when CSV name de-duplication fails (#28658)
  • Always keep first metadata per source for Parquet (#28661)
  • Improve plan-time row estimates for multi-file parquet scans (#28380)
  • Expose array and plugin function views in Python visitor (#28635)

🐛 Bug Fixes

  • SQL handle quantified comparisons against a subquery (#28929)
  • Fix SQL subquery qualified lowering and cache row-index (#28926)
  • Clippy-stable failures (#28919)
  • Use uuidv7 for cloud observed queries (#28910)
  • Unsoundness in AnyValue::into_static for structs (#28889)
  • Invalidate suborders on projection (#28866)
  • Ensure SQL function registry is available in isolated subqueries (#28881)
  • Handle null in Expr.is_not_nan (#28883)
  • Ensure eager `unpivot` column selection aligns with the lazy engine (#28846)
  • Fix assertion error / incorrect data load from parquet written by fastparquet (#28656)
  • Fix projection pushdown incorrect `select()` height (#28751)
  • Restore `monitoring` as an engine-level parameter (#28821)
  • Bincode serialization of IR hints (#28837)
  • Retain datetime inference across CSV fallback (#28663)
  • Avoid chunk fragmentation on bool agg `min` and `max` (#28789)
  • Merge join with coalesce and empty suffix (#28799)
  • Wrong function name call in test (#28801)
  • Preserve row mapping in nested over expressions (#28767)
  • Restore CI compatibility with NumPy 2.5.2 (#28782)
  • Do not deduplicate inherently nondeterministic plan nodes (#28768)
  • Fix OOC spilling of categoricals (#28761)
  • Correct partitionable filter handling for pivot (#28749)
  • Small typo in docstring (#28763)
  • Avoid chunk fragmentation on bool agg `any` and `all` (#28754)
  • Ignore nans in (min|max)_horizontal (#28710)
  • Correct function display for str.replace_all (#28742)
  • Exclude debug tests from nodebug streaming run (#28747)
  • Convert predicate series dtype during prefiltered parquet reading (#28711)
  • Do not remove nested caches when performing predicate pushdown past a cache node (#28740)
  • Correct handling of nulls at the start of business day functions (#28703)
  • Do not panic when exceeding Parquet metadata scan cache (#28721)
  • Remove cryptographic hashing from CSPE (#28716)
  • Unsoundness in rayon block_on (#28709)
  • Check for (shallow) IR node equality on top of hashing in CSPE (#28506)
  • Don't hand IO plugins a predicate they cannot evaluate (#28643)
  • Raise on overflow in Decimal sum instead of silently wrapping (#28688)
  • Preserve nulls when importing Arrow maps (#28680)
  • Propagate null `by` column values in `rolling_*_by` (#27367)
  • Fix self-referencing `field` in `struct.with_fields` with `over` (#28678)
  • Fix Arrow buffer offset for `Utf8` and `Binary` (#28662)
  • Clamp group-by slice offset (#28579)
  • Release the GIL while resolving the schema in `__arrow_c_stream__` (#28642)
  • Fix data corruption on Series from nested Arrow `LargeList` (#28632)

Affected Symbols

rechunkExpr.rechunk()struct.rename_fields()scan_icebergsink_parquetCSV inferenceSeries constructorstruct.drop()ParquetSQLNumPyArrowDecimalrayonIO pluginsAnyValue::into_staticunpivotfastparquetExpr.is_not_nanbincodemin/max aggregationmerge joinover expressionscategoricalspivotstr.replace_allparquet readingcache nodebusiness day functionsParquet metadata scan cacheCSPEDecimal sumArrow mapsrolling_*_bystruct.with_fieldsArrow buffergroup-by slice__arrow_c_stream__Arrow LargeList

⚡ Deprecations

  • Deprecate `rechunk` parameter for all read/scan functions (#28063)
  • Deprecate `Expr.rechunk()` (#28692)
  • Deprecate `struct.rename_fields()` with an incorrect number of fields (#28672)