Polars
Data & MLExtremely fast Query Engine for DataFrames, written in Rust
Release History
View all versions →py-1.44.1Thank you to all our contributors for making this release possible!
py-1.44.043 fixes21 featuresThis release introduces several performance improvements, new features like Iceberg schema evolution and correlated subqueries, and numerous bug fixes. It also deprecates the `rechunk` parameter and related functions.
rs-0.55.22 fixes1 featureThis release introduces an adaptive HTTP rate-limiter for cloud IO and addresses soundness issues in rayon block_on and IR node equality checks in CSPE. Documentation has also been updated regarding arg_extremum functions.
rs-0.55.160 fixes26 featuresThis release introduces significant performance improvements across various operations, including pushdown optimizations, enhanced join strategies, and more efficient memory handling. It also brings a host of new features like `struct.drop()`, improved CSV scanning, and experimental out-of-core spilling, alongside numerous bug fixes for enhanced stability and correctness.
py-1.43.216 fixes4 featuresThis release introduces performance improvements by pushing down `len()` operations and enhances data handling with fixes for enum metadata, parquet field IDs, and various slicing and join operations. It also deprecates certain casting and argument usage patterns.
py-1.43.115 fixes1 featureThis release includes performance improvements by optimizing boolean operations and bug fixes across various query execution scenarios, including joins, aggregates, and predicate pushdown. It also enhances cloud callback functionality and updates documentation.
py-1.43.043 fixes20 featuresThis release introduces performance improvements, new features like `ewm_sum` and `scan_arrow_c_stream`, and numerous bug fixes. Several features and methods are deprecated, including casting numeric types to categoricals and `LazyFrame.profile()`.
py-1.42.110 fixes3 featuresThis release introduces performance improvements, including faster small dtype sums and non-blocking path expansion, while deprecating the `strict` parameter in `pl.concat`. Numerous bugs related to panics, data casting, and resource leaks have also been resolved.
py-1.42.0Breaking25 fixes12 featuresThis release introduces performance improvements across cloud IO and expression evaluation, adds experimental features like out-of-core spilling and strict mode, and fixes numerous bugs related to SQL, projection pushdown, and joins. Notably, it deprecates implicit string-to-temporal casts.
rs-0.54.411 fixes44 featuresThis release focuses heavily on performance improvements across various operations, significant stabilization and expansion of the streaming engine capabilities, and numerous enhancements to SQL support and data source handling (like Parquet and cloud storage).
py-1.41.21 fixThis release focuses on performance improvements, including updates to jemalloc and optimizations in column and array operations. It also includes documentation updates and general hardening against async deadlocks.
py-1.41.111 fixes1 featureThis release focuses on performance improvements, particularly in agg_n_unique, and addresses numerous bugs related to sorting, null handling, precision loss, and expression evaluation pushdown. A new environment variable enables nested CSPE functionality.
py-1.41.049 fixes10 featuresThis release introduces new features like LazyFrame.gather and stabilizes the streaming engine, alongside numerous performance improvements and bug fixes across data reading and expression evaluation.
py-1.40.17 fixes1 featureThis release focuses on performance improvements, bug fixes across various operations like grouping, list sampling, and concatenation, and adds a new parameter to `merge_sorted`.
py-1.40.052 fixes9 featuresThis release introduces streaming support for grouped AsOf joins and numerous performance optimizations across the engine, particularly in streaming operations. Several bugs related to joins, aggregations, and data reading have been fixed, and support for the dataframe interchange protocol has been deprecated.
py-1.39.3This release contains no functional changes.
py-1.39.2This release contains no functional changes.
py-1.39.11 fixThis release primarily focuses on fixing an issue in the streaming engine related to empty rolling windows and includes documentation updates.
py-1.39.053 fixes45 featuresThis release focuses heavily on performance improvements across the streaming engine, I/O operations (CSV, NDJSON, Parquet), and various expression evaluations. Numerous bugs related to type handling, panics in streaming operations, and SQL compatibility have also been addressed.
rs-0.53.01 fix60 featuresThis release focuses heavily on performance improvements across various operations, especially in streaming contexts, and introduces significant enhancements to SQL support and data type handling, including new Extension types.
py-1.38.113 fixes1 featureThis release introduces the ability to retrieve a byte from binary data using get() and includes numerous bug fixes across query optimization, schema inference, and data handling. Several internal cleanups and documentation updates were also performed.
py-1.38.040 fixes27 featuresThis release focuses heavily on performance improvements across streaming, I/O, and core computations, alongside numerous bug fixes for stability and correctness. A key change is the deprecation of the `retries` argument in favor of using `storage_options`.
py-1.37.14 fixesThis release focuses on performance improvements, particularly for SQL UNION clauses, and includes several bug fixes related to IPC slicing and error handling. Dependency updates were also performed.
py-1.37.0Breaking38 fixes26 featuresThis release introduces significant performance improvements across various operations, including SQL ORDER BY clauses, NaN conversions, and streaming data processing. It also brings new features like `pl.PartitionBy`, enhanced SQL JOIN handling, and expanded data type support, alongside numerous bug fixes and documentation updates.
py-1.36.16 fixes1 featureThis release focuses on performance improvements, new features like Object literal creation, and numerous bug fixes across various components including binary operations and DataFrame methods.
Common Errors
InvalidOperationError7 reportsThis error typically occurs in polars. Check the example issues for common solutions.
ColumnNotFoundError6 reportsThis error typically occurs in polars. Check the example issues for common solutions.
ComputeError3 reportsComputeError in Polars often arises from operations on empty data structures like empty lists or structs, especially during type casting or when applied to nested structures within `list.eval`. To fix this, ensure your Polars expressions gracefully handle empty data by using `pl.when(pl.len() > 0).then(...)` conditions before performing operations that might fail on empty data or by explicitly casting empty structures to the expected type using `.fill_null(pl.lit(...))` with a suitable default value. This prevents the computation from encountering null values and potentially crashing during execution.
ArrowNotImplementedError2 reportsArrowNotImplementedError in Polars usually arises from compatibility issues between the Arrow library used by Polars and specific data types or features in your Parquet files or schemas. To fix this error, ensure your pyarrow version is compatible with the polars version and that pyarrow supports all datatypes present in your Parquet file; consider updating pyarrow or casting questionable columns (e.g., using `pl.col("my_col").cast(pl.Utf8)`) to more standard types before writing or reading your data. You may also need to explicitly specify the Arrow schema using `use_pyarrow=True` to control data type handling.
ShapeError2 reportsShapeError in Polars arises when operations expect data of compatible shapes (row/column counts) but encounter mismatches, like attempting element-wise math between series of differing lengths or incompatible aggregations. To fix it, ensure input series have the expected dimensions by using `explode()`, `reshape()`, or aggregation functions like `groupby()` before the operation causing ShapeError. Verify the operation and input data shapes align with Polars' documented expectations for the function used.
NoDataError2 reportsNoDataError in Polars often arises when operations, like CSV parsing or data filtering, result in an empty DataFrame, and `raise_if_empty=True` (the default). Fix this by setting `raise_if_empty=False` during operations like `scan_csv` or `read_csv` to return an empty DataFrame gracefully, or ensure the input data isn't empty if an empty frame is not desired, and make sure to properly infer the schema or provide one.
Related Data & ML Packages
An Open Source Machine Learning Framework for Everyone
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Tensors and Dynamic neural networks in Python with strong GPU acceleration
scikit-learn: machine learning in Python
Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more
Streamlit — A faster way to build and share data apps.
Subscribe to Updates
Get notified when new versions are released