●Change8

Migrating to Datasets 5.0.0

Version 5.0.0 introduces 1 breaking change. This guide details how to update your code.

Released: 6/5/2026

1
Breaking Changes
1
Migration Steps
4
Affected Symbols

⚠️ Check Your Code

If you use any of these symbols, you need to read this guide:

IterableDataset.shuffleDataset.batchDataset.to_sqlJson

Breaking Changes

●Issue #1

The default shuffling mechanism in streaming mode now uses multiple input shards, which is a change from the previous single-shard approach. To revert to the old mechanism, pass `max_buffer_input_shards=1` to `IterableDataset.shuffle()`.

Migration Steps

  1. 1
    If you rely on the exact behavior of shuffling in streaming mode before this release, update calls to `IterableDataset.shuffle()` to include `max_buffer_input_shards=1`.

Release Summary

This release introduces significant features including parsing Agent traces, enhanced streaming shuffle using multiple shards, and new batching capabilities. It also adds support for several new data formats like Iceberg, TsFile, 3D Mesh, and CoNLL.

Need More Details?

View the full release notes and all changes for Datasets 5.0.0.

View Full Changelog