v0.33.0
Breaking Changes📦 lancedbView on GitHub →
⚠ 1 breaking✨ 14 features🐛 15 fixes🔧 5 symbols
Summary
This release introduces an elastic dataloader, support for various data types in the expression builder, and enhanced object store capabilities. It also includes numerous bug fixes and performance improvements, alongside updates to Python and Node.js integrations.
⚠️ Breaking Changes
- The behavior of `Permutation.with_format("torch")` in Python has been aligned with HuggingFace's `set_format("torch")`. This may require adjustments if your code relies on the previous, non-aligned behavior.
Migration Steps
- If using `Permutation.with_format("torch")`, review its behavior to ensure compatibility with the new alignment to HuggingFace `set_format("torch")`.
✨ New Features
- Added an elastic dataloader as an iterable dataset.
- Support for date, datetime, bytes, and Decimal literals in the expression builder.
- Alignment of `Permutation.with_format("torch")` with HuggingFace `set_format("torch")`.
- Added `get_lsm_write_spec` to read the installed LSM write spec.
- Support for Tencent COS and GooseFS object stores via new feature flags.
- Exposed Lance metrics via OpenTelemetry in Python and Node.
- Introduced blob v2 fetch API in Python.
- Added support for the WatsonxReranker component in Python.
- Added table FTS query tokenization.
- Support for distributed analyze plan metrics in clients.
- Published `lancedb-compat` for pre-Haswell x86_64 hosts.
- Skill references now work with jobs, including server connections.
- Added remote branch diff and merge client APIs.
- Branch skill updates for merge.
🐛 Bug Fixes
- Handled empty or wrong-length vectors returned by embedding functions.
- Resolved Ollama embedding serialization error in `create_table` in Python.
- Fixed `flatten_columns` raising an error when `flatten=False` in Python.
- Avoided manifest writes for read-only directory namespace opens.
- Preserved zero distance bounds in hybrid search in Python.
- Addressed Gemini batching, user agent, and variable dimensions in Python.
- Preserved phrase semantics in sync queries in Python.
- Filled bad vector values element-wise in Python.
- Skipped embedding functions for empty batches in Rust.
- Routed local sync namespace operations through Rust in Python.
- Bound scanner memory for wide-row bulk ingestion in Python.
- Raised a clear `ValueError` when a vector column cannot be inferred in Python.
- Bound remote insert request size to avoid ingestion timeouts in Rust.
- Sanitized Map fields across Arrow versions in Node.
- Allowed selection of `_rowid` in Permutation in Python.