●Change8

v0.37.1-beta.1

Breaking Changes
📦 lancedbView on GitHub →
⚠ 2 breaking✨ 8 features🐛 29 fixes🔧 41 symbols

Summary

This release introduces several new features including custom stop-word lists for FTS, a RemoteTable fetch_blobs client, and connection-level job operations. It also includes numerous bug fixes across Python, Rust, and Node.js, along with breaking changes to the `add_columns` function in Rust and Python.

⚠️ Breaking Changes

  • The `add_columns` function in Rust has been refactored into a builder pattern. This may require changes to how `add_columns` is called in existing Rust code.
  • The `add_columns` function in Python has been refactored into a builder pattern. This may require changes to how `add_columns` is called in existing Python code.

Migration Steps

  1. If using the `add_columns` function in Rust or Python, refactor your code to use the new builder pattern.

✨ New Features

  • Support for custom stop-word lists in Full-Text Search (FTS).
  • Added a RemoteTable `fetch_blobs` HTTP client for fetching blobs from remote storage.
  • Added support for checking if a namespace or table exists in Python.
  • The `create_index` function now returns a Job handle, allowing for asynchronous index creation monitoring.
  • Introduced connection-level job operations for managing jobs associated with a connection.
  • Added seekable blob range reads for RemoteTable, enabling partial blob retrieval.
  • Added `checkpoint_lsm`, `flush_lsm`, `compact_lsm`, and `get_lsm_stats` functions for managing LSM-tree tables.
  • The system now infers maintained indexes when an `LsmWriteSpec` omits them.

🐛 Bug Fixes

  • Deflake test_read_consistency_interval.
  • Avoided asynchronous work in synchronous representations to prevent potential deadlocks.
  • Surfaced masked merge_insert stream errors under HTTP2.
  • Distinguished between corrupt and missing tables when opening a table.
  • Added logging when `storage_options` is ignored in `RemoteDBConnection.open_table`.
  • Fixed an issue where `hybrid search minimum_nprobes(0)` would silently no-op instead of raising an error.
  • Corrected `when_not_matched_by_source_delete()` to not reset a previously-set condition.
  • Avoided debugger deadlock during connection inspection in Python.
  • Guarded against concurrent table deletes in Python.
  • Reported unsplittable IVF partition errors in Python.
  • Aligned Python wheel ABI with supported versions.
  • Prevented OpenSSL linkage in Linux wheels.
  • Instructed dimension probe for instructor embeddings in Python.
  • Supported nullable pandas merge input in Python.
  • Covered nullable list v2.2 decoding in Python.
  • Preserved repeated indexed merge inserts in Python.
  • Prevented vector optimize regression after deletes in Rust.
  • Validated cloud DNS hostnames in remote operations.
  • Handled missing mirrored copy sources in Rust.
  • Percent-encoded index names in per-index remote REST paths.
  • Accepted Arrow scalars in table updates in Python.
  • Clarified bare Vector annotations in Python.
  • Exposed inline types to downstream checkers in Python.
  • Required compatible Node.js types in Node.js builds.
  • Supported Polars 1.32 table scans in Python.
  • Covered non-nullable embedding schema append in Node.js.
  • Explained unsupported object storage mounts.
  • Covered offset pagination in search for Node.js.
  • Avoided `AttributeError` in JinaEmbeddings image input for `str`/`Path`.

Affected Symbols

add_columns (Rust)add_columns (Python)RemoteTable.fetch_blobsnamespace/table exist (Python)create_indexconnection-level job operationsRemoteTable seekable blob range readscheckpoint_lsmflush_lsmcompact_lsmget_lsm_statsLsmWriteSpectest_read_consistency_intervalsynchronous representations (Python)merge_insert stream errors (remote)open_tableRemoteDBConnection.open_tablehybrid search minimum_nprobeswhen_not_matched_by_source_delete()connection inspection (Python)concurrent table deletes (Python)IVF partition errors (Python)Python wheel ABILinux wheels (OpenSSL linkage)instructor embeddings (Python)pandas merge input (Python)nullable list v2.2 decoding (Python)repeated indexed merge inserts (Python)vector optimize (Rust)cloud DNS hostnames (remote)mirrored copy sources (Rust)per-index remote REST pathsArrow scalars (Python)bare Vector annotations (Python)inline types (Python)Node.js typesPolars 1.32 table scans (Python)embedding schema append (Node.js)object storage mountsoffset pagination (Node.js search)JinaEmbeddings image input