●Change8

v0.37.1

Breaking Changes
📦 lancedbView on GitHub →
⚠ 2 breaking✨ 13 features🐛 35 fixes🔧 8 symbols

Summary

This release introduces several new features including FTS enhancements, batched blob reads, and connection-level job operations. It also includes numerous bug fixes across Python, Rust, and Node.js, along with breaking changes to the `add_columns` method in both Rust and Python.

⚠️ Breaking Changes

  • The `add_columns` method in Rust has been refactored into a builder pattern. This may require changes to how columns are added in Rust code.
  • The `add_columns` method in Python has been refactored into a builder pattern. This may require changes to how columns are added in Python code.

Migration Steps

  1. Update Rust code to use the builder pattern for `add_columns`.
  2. Update Python code to use the builder pattern for `add_columns`.

✨ New Features

  • Added block size configuration for Full-Text Search (FTS).
  • Added `use_lsm` option to read MemWAL LSM data.
  • Support for batched blob range reads.
  • Exposed `AsyncTable.to_lance` in Python.
  • Configurable streaming transform parallelism in Python.
  • Support for custom stop-word lists in FTS.
  • Added `RemoteTable.fetch_blobs` HTTP client.
  • Added namespace/table existence check support in Python.
  • `create_index` now returns a Job handle.
  • Introduced connection-level job operations.
  • Added seekable blob range reads for remote tables.
  • Added `checkpoint_lsm`, `flush_lsm`, `compact_lsm`, and `get_lsm_stats` for LSM tables.
  • Inferred maintained indexes when an `LsmWriteSpec` omits them.

🐛 Bug Fixes

  • Fixed a crash in `AnswerdotaiRerankers`/`ColbertReranker` when `return_score="all"`.
  • Preserved row count in `MetadataEraserExec` for zero-column batches.
  • Accepted either timestamp or timestamp_millis for versions.
  • Rejected non-string Arrow metadata in Node.js.
  • Deflaked the `test_read_consistency_interval` test.
  • Avoided async work in sync representations in Python.
  • Surfaced masked merge_insert stream errors under HTTP2 in remote operations.
  • Distinguished corrupt tables from missing tables in `open_table`.
  • Logged when `storage_options` is ignored in `RemoteDBConnection.open_table` in Python.
  • Fixed hybrid search where `minimum_nprobes(0)` silently no-oped instead of raising an error.
  • Corrected `when_not_matched_by_source_delete()` to reset a previously-set condition.
  • Avoided debugger deadlock during connection inspection in Python.
  • Guarded concurrent table deletes in Python.
  • Reported unsplittable IVF partition errors in Python.
  • Aligned wheel ABI with supported versions in Python.
  • Prevented OpenSSL linkage in Linux wheels in Python.
  • Instructed dimension probe for instructor embeddings in Python.
  • Supported nullable pandas merge input in Python.
  • Covered nullable list v2.2 decoding in Python.
  • Preserved repeated indexed merge inserts in Python.
  • Prevented vector optimize regression after deletes in Rust.
  • Validated cloud DNS hostnames in remote operations.
  • Handled missing mirrored copy sources in Rust.
  • Percent-encoded index names in per-index remote REST paths.
  • Accepted Arrow scalars in table updates in Python.
  • Clarified bare `Vector` annotations in Python.
  • Exposed inline types to downstream checkers in Python.
  • Required compatible Node.js types.
  • Supported Polars 1.32 table scans in Python.
  • Covered non-nullable embedding schema append in Node.js.
  • Explained unsupported object storage mounts.
  • Covered offset pagination in search in Node.js.
  • Avoided `AttributeError` in `JinaEmbeddings` image input for str/Path.
  • Retrieved table size from metadata, not files.
  • Preserved nested Arrow data across versions in Node.js.

Affected Symbols

add_columns (Rust)add_columns (Python)AsyncTable.to_lanceRemoteTable.fetch_blobscreate_indexMetadataEraserExecRemoteDBConnection.open_tableJinaEmbeddings