py-1.34.0-beta.1
📦 polars
✨ 12 features🐛 19 fixes🔧 7 symbols
Summary
This release introduces new batch collection methods for LazyFrames and significant performance improvements across various scan types. It also includes numerous bug fixes, especially around cloud storage interactions and data type handling.
✨ New Features
- Added LazyFrame.{sink,collect}_batches methods.
- Ensured deterministic import order for Python Polars package variants.
- Added support to display lazy query plan in marimo notebooks without needing to install matplotlib or mermaid.
- Added unstable `hidden_file_prefix` parameter to `scan_parquet`.
- Switched to using fixed-scale Decimals.
- Added support for unsigned 128-bit integers.
- Added unstable `pl.Config.set_default_credential_provider` configuration option.
- Ensured roundtrip compatibility for `BinaryOffset` type through Parquet.
- Added opt-in unstable functionality to load interval types as `Struct`.
- Added support for reading parquet metadata from cloud storage.
- Added user guide section on AWS role assumption.
- Added support for `unique`, `n_unique`, and `arg_unique` operations on `array` columns.
🐛 Bug Fixes
- Added support for float inputs for duration types.
- Fixed roundtrip of empty string through hive partitioning.
- Fixed potential Out-Of-Bounds writes in unaligned IPC read.
- Fixed regression error when scanning AWS presigned URL.
- Made `PlPath::join` for cloud paths replace on absolute paths.
- Corrected dtype for cum_agg in the streaming engine.
- Restored support for `np.datetime64()` in `pl.lit()`.
- Ignored Iceberg list element ID if missing.
- Fixed panic on streaming full join with coalesce.
- Fixed `AggState` on `all_literal` in `BinaryExpr`.
- Showed IR sort options in `explain` output.
- Fixed benchmark CI import.
- Fixed schema on `ApplyExpr` with single row `literal` in aggregation context.
- Fixed planner schema for dividing `pl.Float32` by an integer.
- Fixed panic when scanning from AWS legacy global endpoint URL.
- Fixed `iterable_to_pydf(..., infer_schema_length=None)` to scan all data.
- Ensured struct of nulls is not propagated with null.
- Became stricter with invalid NDJSON input when `ignore_errors=False`.
- Implemented `approx_n_unique` for temporal dtypes and Null.
🔧 Affected Symbols
LazyFrame.{sink,collect}_batchesscan_icebergscan_parquetpl.Config.set_default_credential_providerpl.litPlPath::joiniterable_to_pydf