●Change8

v0.27.0

Breaking Changes
📦 lancedbView on GitHub →
⚠ 1 breaking✨ 21 features🐛 15 fixes⚡ 1 deprecations🔧 6 symbols

Summary

This release introduces major features like parallel inserts, expression builder API, and upgrades the core Lance dependency to 3.0.0-rc.3. It also contains breaking changes in Rust APIs for table creation and data addition.

⚠️ Breaking Changes

  • The Rust implementation of `create_table()` and `Table.add()` now accepts `RecordBatch` or `Vec<RecordBatch>` as input, which may break existing calls expecting different types.

Migration Steps

  1. If you were relying on the previous input types for `create_table()` or `Table.add()` in Rust, update them to use `RecordBatch` or `Vec<RecordBatch>`.

✨ New Features

  • Added third-party licenses lists.
  • Reranker information is now shown in the hybrid search explain plan.
  • Added a `getitem` implementation for the permutation.
  • Improved Permutation PyTorch integration.
  • Updated Lance dependency to v2.0.1.
  • Added background checking for dataset updates.
  • Upgraded napi-rs from v2 to v3 in the Node.js bindings.
  • Hooked up a new writer for insertion operations.
  • Enabled parallel inserts for local tables.
  • Added expression builder API for type-safe query filters in Rust.
  • Upgraded Lance to 3.0.0-rc.2 and added bindings for fast_search.
  • Added `num_deleted_rows` to the delete result.
  • Added support for remote index parameters.
  • Added parity between the `fast_search` keyword argument for vector and FTS searches.
  • Python: Support for dictionary to SQL struct conversion in `table.update()`.
  • Allow passing Azure client/tenant ID through the remote SDK.
  • Infer JavaScript native arrays.
  • Upgraded Lance to 3.0.0-rc.3.
  • Support for `prewarm_index` and `prewarm_data` on remote tables.
  • Rust: Support for `Expr` in projection queries.
  • Node.js: Support for field/data type input in the `add_columns()` method.

🐛 Bug Fixes

  • Improved error message for multi-field FTS index creation.
  • Fixed support for dynamic projection on remote tables.
  • Fixed non-stopping dataset version check after passing the first consistency check interval.
  • Python: Graceful handling of empty result sets in hybrid search.
  • Allowed permutation reader to be used with remote tables.
  • Allowed permutation reader to work with remote tables as well.
  • Stopped retrying OpenAI 401 authentication errors.
  • Updated `DatasetConsistencyWrapper` to accept same-version updates.
  • The permutation reader no longer expects all offsets to fit in one batch.
  • Python: Pinned `pylance` to match DataFusion table provider version.
  • Fixed CI: Only run npm publish on release tags.
  • Propagated cast errors in `add()`.
  • Propagated managed versioning for namespace connection.
  • Fixed CI: Upgraded LocalStack to 4.0 for S3 integration tests.
  • Restored storage options on copy in localstack tests.

Affected Symbols

create_table()Table.add()PermutationDatasetConsistencyWrappertable.update()add_columns()

⚡ Deprecations

  • Outdated files for the embedding registry have been deprecated.