●Change8

ray-2.58.0

📦 rayView on GitHub →
✨ 25 features🐛 17 fixes⚡ 2 deprecations🔧 26 symbols

Summary

This release introduces significant enhancements to Ray Serve LLM with KV cache and token-aware routing, and Ray Core with task event offloading. It also brings Databricks DeltaLake integration and a new shuffle v2 backend to Ray Data, alongside experimental Ray Sandbox and expanded TPU support across Ray Train and Core.

Migration Steps

  1. Rename reports_custom_op_stats to should_report_custom_op_stats across all MapTransformFn variants.
  2. Rename allocated to reserved and add type aliases in the autoscaling coordinator.

✨ New Features

  • Ray Serve LLM: Completed KV cache and token aware request routing, with tokenization happening in-process on the LLMRouter ingress replica. KV cache and token aware routing is also aware of CPU KV caches.
  • Ray Core: Enabled offloading task events from the GCS hot path by replacing the task event buffer with the ray event recorder. State APIs and ray.timeline now read from an in-memory store on the task events head.
  • Ray Data: Added Databricks integrations for writing to DeltaLake with Catalog support.
  • Ray Data: Shipped a new shuffle v2 backend, featuring improved performance for joins and aggregations.
  • Ray Data: Added Dataset.with_columns for multi-column expression projection.
  • Ray Data: Added write_delta for Delta Lake, with catalog support.
  • Ray Data: Added Torch inference API.
  • Ray Data: Promoted hash shuffle v2 to a selectable shuffle strategy, with aggregation support, vectorized aggregation, and block splitting during aggregation.
  • Ray Data: Added ignore_missing_paths and skip_paths to read_parquet on DatasourceV2.
  • Ray Data: Added delta_timestamps (temporal windows) to read_lerobot.
  • Ray Data: Added cluster usage metrics collection by sampling in background threads during execution.
  • Ray Data: Tolerate actor deaths during init via DataContext.max_consecutive_actor_init_deaths.
  • Ray Data: Expose RAY_DATA_HASH_SHUFFLE_MAP_TASK_TARGET_INPUT_BYTES in DataContext.
  • Ray Serve: Configurable status code and Retry-After header for backpressure rejections.
  • Ray Serve: Added dependency-ordered shutdown for deployments.
  • Ray Serve: Added an optional tie-break key to best-fit node scheduling.
  • Ray Serve: Scale ingress request router replicas per proxy node.
  • Ray Train: Integrates with TorchTPU backend.
  • Ray Train: Added a public preemption API and a controller PreemptingState.
  • Ray Train: Added data ingest metrics to the Train dashboard.
  • Ray LLM: KV-cache-aware routing: move tokenization into the LLMRouter ingress replica, decide KV/token routing there, broadcast KV lifecycle events to all ingress replicas, and make selection and reservation atomic.
  • Ray LLM: Enable KV cache offloading, make KV routing aware of CPU KV caches, and transmit tokens out-of-band so the engine skips tokenization.
  • Ray LLM: Add a KV cache offload/reload dashboard and a Ray Serve LLM SGLang metrics dashboard.
  • Ray Core: Added experimental Ray Sandbox, which runs task and actor code under gVisor and can run Docker-built images directly.
  • Ray Core: Added TPU Support: Ray Train adds support for TorchTPU backend, and Ray Core adds SubslicePlacementGroup for gang scheduling on TPU subslices, single-host TPU support in SlicePlacementGroup, and resource accounting for tpu7x and multi-core chips.

🐛 Bug Fixes

  • Ray Data: Fix an RCE where read_lance or nested pickle objects could execute arbitrary code.
  • Ray Data: Stop converting Arrow null columns to null[pyarrow] in to_pandas.
  • Ray Data: Fix iter_torch_batches device resolution and typing.
  • Ray Data: Propagate isolate_read_workers to DatasourceV2.
  • Ray Data: Avoid signalling epoch end for failed attempts.
  • Ray Data: Add tf-keras to the text_embedding pip packages.
  • Ray Serve: Fix the Serve replica ASGIService bypassing token authentication.
  • Ray Serve: Fix the proxy update loop getting stuck when a proxy's node is removed.
  • Ray Train: Fix Torch environment setup for V1 worker groups.
  • Ray Tune: Fix HyperOptSearch dropping tune.choice categories that are constant dicts.
  • Ray Tune: Deflake test_multi_trial_reuse_with_failing and decide test_experiment_restore completion from measured progress.
  • Ray RLlib: Fix TQC critic divergence by stopping actor-loss gradients from leaking into the critics.
  • Ray RLlib: Fix squashed-Gaussian log-prob corruption for saturated policies.
  • Ray RLlib: Use the target_qf_twin head in IQL target prediction.
  • Ray RLlib: Properly enforce use_kl_loss in the PPO Torch and TF policies.
  • Ray RLlib: Fix a KeyError in the multi-agent module-to-env connector.
  • Ray RLlib: Fix the API-doc consistency check for the new reverse/dedup policies, and deflake the test_env_runner callback-count tests.

Affected Symbols

LLMRouterray.timelineDataset.with_columnsDataset.write_deltaDataset.read_parquetDatasourceV2Dataset.read_lerobotDataContext.max_consecutive_actor_init_deathsDataContextRAY_DATA_HASH_SHUFFLE_MAP_TASK_TARGET_INPUT_BYTESBatchIteratoriter_torch_batchesMapTransformFnray_remote_args_fnDataset.zipASGIServiceTorchTPUSubslicePlacementGroupSlicePlacementGrouptpu7xHyperOptSearchtune.choiceTQCPPOgVisorDocker

⚡ Deprecations

  • ray_remote_args_fn
  • Dataset.zip