●Change8

python-v0.34.0

Breaking Changes
📦 lancedbView on GitHub →
⚠ 2 breaking✨ 28 features🐛 26 fixes⚡ 1 deprecations🔧 17 symbols

Summary

This release introduces significant new features including table branching, Polars integration, and OAuth support, alongside breaking changes that adjust filter logic and remove unused fields.

⚠️ Breaking Changes

  • The 'loss' field has been removed from IndexStatistics.
  • Repeated 'where' filters in queries are now combined using AND logic instead of the previous behavior of replacing them.

Migration Steps

  1. If you relied on repeated 'where' filters being replaced, update your query logic as they are now combined with AND.

✨ New Features

  • Unified sync 'create_index' API in Python to match the async API.
  • Routed 'merge_insert' through the MemWAL LSM write path.
  • Support for remote tables in PyTorch dataloaders.
  • Added 'update_field_metadata' to edit per-field metadata.
  • Support for blob modes in query 'to_pandas'.
  • Support for datafusion expressions for merge insert predicates in Rust.
  • Implemented set/unset_lsm_write_spec REST variant for remote operations.
  • Added table branch support.
  • Support for checking out a specific version on a branch.
  • Support for FM-Index scalar index for substring search.
  • Added 'isin' support to the 'Expr' builder.
  • Expanded IndexConfig with rich per-index metadata.
  • Accept 'Expr' in Table.delete and 'when_not_matched_by_source_delete' in merge operations in Python.
  • Support 'rename_table' on LanceNamespaceDatabase.
  • Exposed new IndexConfig fields in Python and Node.js bindings.
  • Added skills to connect and update column metadata.
  • Added table branch support to remote tables and Python/TS bindings.
  • Added approximate mode to vector queries.
  • Sent read-freshness signal on the lance-namespace path.
  • Added blob v2 schema declaration and write path in Rust.
  • Added blob read and materialization APIs in Rust.
  • Added OAuth header provider in Rust.
  • Exposed OAuth connection config in Python.
  • Exposed OAuth connection config in Node.js.
  • Added skill to work with branches better.
  • Added Polars dataframe integration.
  • Implemented monotonic reads via x-lancedb-min-read-version watermark for remote operations.
  • Re-exported arrow and datafusion crates from lancedb in Rust.

🐛 Bug Fixes

  • Fixed an issue where an empty 'vector_results' list caused a ValueError in rerankers (MRR).
  • Guarded against empty 'vector_results' in RRFReranker.rerank_multivector.
  • Routed blob query pandas through the scanner in Python.
  • Made LanceDBClientError pickleable in Python.
  • Ran AsyncTable.search embeddings on a dedicated executor in Python.
  • Pushed down namespace full reads in Python.
  • Converted numpy scalars in 'value_to_sql'.
  • Treated NAPI_RS_FORCE_WASI as truthy only when set to 'true' in Node.js.
  • Fixed support for LargeList label list indexes.
  • Raised clear TypeError for bare List/Tuple in pydantic schema conversion in Python.
  • Reported local write progress bytes from Lance in Rust.
  • Escaped single quotes in struct field names in 'value_to_sql'.
  • Clarified single dictionary input error in Python.
  • Returned typed errors instead of panicking in the Bedrock embedding path in Rust.
  • Allowed shell pre-commit hooks in bumpversion configs in CI.
  • Fixed overflow when using sys.maxsize for k in query with namespace connection.
  • Parsed RFC 3339 created_at and improved IndexConfig repr.
  • Added missing stacklevel=2 to warnings.warn() and fixed broken message concatenation.
  • Omitted empty api key header in remote operations.
  • Supported namespace clients with dynamic headers in remote operations.
  • Ensured read freshness provider is built into the namespace client.
  • Raised clear error when permutation API is used on remote tables in Python.
  • Added missing stacklevel=2 to warnings.warn() calls.
  • Routed sync namespace connections through Rust in Python.
  • Routed async namespace connections through Rust in Python.
  • Averaged MRR reciprocal ranks over all rankings in Python.

Affected Symbols

IndexStatisticscreate_index (sync API)merge_insertPyTorch dataloadersupdate_field_metadatareplace_field_metadatato_pandas (blob modes)Table.deletemerge (when_not_matched_by_source_delete)rename_tableLanceNamespaceDatabaseIndexConfigTable.search (Async)value_to_sqlRRFReranker.rerank_multivectorLanceDBClientErrorpermutation API (remote tables)

⚡ Deprecations

  • The 'replace_field_metadata' function is deprecated in favor of 'update_field_metadata'.