●Change8

ray-2.55.0

📦 rayView on GitHub →
✨ 25 features🐛 27 fixes⚡ 2 deprecations🔧 24 symbols

Summary

This release introduces the new `DataSourceV2` API, GPU shuffle support, and significant enhancements to Ray Serve including gRPC streaming and HAProxy ingress. Numerous fixes address RCE vulnerabilities, scheduling races, and resource management issues across Ray Data.

Migration Steps

  1. If you relied on the old callback API, update your code as it has been removed.
  2. If you were using `on_exit` hooks for UDF cleanup, migrate to using `__ray_shutdown__`.

✨ New Features

  • Added `DataSourceV2` API with scanner/reader framework, file listing, and file partitioning.
  • Support GPU shuffle with `rapidsmpf` 26.2.
  • Added Kafka datasink, migrated to `confluent-kafka`, and support for `datetime` offsets.
  • Added Turbopuffer datasink.
  • Added 2-phase commit checkpointing with trie recovery and load method.
  • Queue-based autoscaling policy integrated with task consumers.
  • Enabled autoscaling for GPU stages.
  • Expressions: added `random()`, `uuid()`, `cast`, and map namespace support.
  • Added support for Arrow native fixed-shape tensor type.
  • Support writing tensors to tfrecords.
  • Added `pathlib.Path` support to `read_*` functions.
  • Added `cudf` as a `batch_format`.
  • Allow `ActorPoolStrategy` for `read_datasource()` via `compute` parameter.
  • Introduce `ExecutionCache` for streamlined caching.
  • Support `strict=False` mode for `StreamingRepartition`.
  • Ported changes from lance-ray into Ray Data.
  • Enabled PyArrow compute-to-expression conversion for predicate pushdown.
  • Add vLLM metrics export and Data LLM Grafana dashboard.
  • Include logical memory in resource manager scheduling decisions.
  • Add monotonically increasing ID support.
  • Added end-to-end gRPC client and bidirectional streaming support for Ray Serve.
  • Introduced HAProxy-based serving for Ray Serve with fallback proxy support.
  • Added queue-based autoscaling for async inference and Taskiq-backed workloads in Ray Serve.
  • Rolled out gang scheduling support across Ray Serve components.
  • Introduced deployment-scoped actors in Ray Serve with config/schema and lifecycle management.

🐛 Bug Fixes

  • Fix RCE in Arrow extension type deserialization from Parquet.
  • Fix `StreamingSplitDataIterator.schema()`.
  • Fix `ParquetDatasource` handling of `FileSystemFactory.inspect`.
  • Fix `read_parquet` file-extension filtering for versioned object-store URIs.
  • Fix `wide_schema_pipeline_tensors` cloudpickle deserialization.
  • Fix `OpBufferQueue` race condition.
  • Fix scheduling metrics computation.
  • Fix `OneHotEncoder` `max_categories` to use global top-k instead of per-partition.
  • Fix `ReservationOpResourceAllocator` resource borrowing for `ActorPoolMapOperator`.
  • Fix `DatabricksUCDatasource` `schema()` shadowing by schema string attribute.
  • Fix `AliasExpr` structural equality to respect rename flag.
  • Fix `_align_struct_fields` failure with unaligned scalar fields.
  • Fix `min_scheduling_resources` fallback to `incremental_resource_usage`.
  • Fix output backpressure unblocking sequence for terminal ops.
  • Fix multi-input operator object store memory attribution.
  • Fix reference cycle by moving to module scope.
  • Fix autoscaler logging: reduce verbose output and move traceback to debug.
  • Fix double counting `ref_bundle` + `input_files`.
  • Prevent `Limit` from getting pushed past `map_groups`.
  • Propagate schema in empty `_shuffle_block` to fix `ColumnNotFound` in chained left joins.
  • Fix unclear metadata warning and incorrect operator name logging.
  • Clamp rolling utilization averages to zero.
  • Fix floating point errors in `TimeWindowAverageCalculator`.
  • Remove default task-level timeout and clamp `end_offset` in Kafka datasource.
  • Avoid redundant reads in `train_test_split`.
  • Return `None` when no outputs have been produced.
  • Replace bare `raise` with `TypeError` in string concatenation.

Affected Symbols

DataSourceV2rapidsmpfconfluent-kafkaActorPoolStrategyread_datasourceStreamingRepartitionlance-raycudfStreamingSplitDataIteratorParquetDatasourceFileSystemFactoryread_parquetOpBufferQueueOneHotEncoderReservationOpResourceAllocatorActorPoolMapOperatorDatabricksUCDatasourceAliasExprLimitmap_groupsKafkatrain_test_splitray.data.DatasourceMOD

⚡ Deprecations

  • Removed legacy `BlockList`, `locality_with_output`, old callback API, and PyArrow 9.0 checks.
  • Replaced `on_exit` hook with `__ray_shutdown__` to fix UDF cleanup race.