●Change8

v1.31.0

📦 weaviateView on GitHub →
✨ 18 features🐛 7 fixes🔧 3 symbols

Summary

This release introduces significant features including Muvera encoding, HNSW periodic snapshotting, enhanced keyword search operators (AND/OR), and major advancements in shard replica movement and replication engine stability. The team focused on integrating file copy for replication and improving monitoring and error handling within the replication FSM.

Migration Steps

  1. Review changes related to named vectors if you rely on specific auto-schema behavior, although compatibility adjustments were made.
  2. If using keyword search, note the change to lowercase 'minimumOrTokensMatch' if configuring manually.
  3. Monitor replication operations closely after upgrading, especially if using advanced replication features.

✨ New Features

  • Introduced backward compatible named vectors, with initial changes to auto-schema behavior which were later reverted/adjusted to maintain compatibility.
  • Enabled adding new named vectors to existing collections by default.
  • Added Muvera encoding support.
  • Updated default Muvera repetition count.
  • Implemented HNSW periodic snapshots for improved data durability and recovery.
  • Added support for minimum should match and AND operators in BM25 keyword search.
  • Refactored BM25 arguments and changed minimumOrTokensMatch to lowercase.
  • Integrated Replica Copy via File Copy with FSM/Engine for shard replication.
  • Implemented the GET /v1/replication/replicate/{id} API endpoint.
  • Added FSM state Prometheus metrics for replica replication operations.
  • Implemented shard copy with integrity checking.
  • Added snapshot/restore mechanism to the replication FSM.
  • Introduced user-facing UUID for each replication operation.
  • Introduced a 'transferType' parameter to distinguish copy or move replication operations.
  • Introduced replication cancellation and deletion capabilities.
  • Added GET ops with parameters and get sharding state API operations.
  • Added DELETE /replications/replicate endpoint.
  • Implemented logic to delete relevant replication operations when classes/shards are deleted.

🐛 Bug Fixes

  • Fixed direct candidate resolving in query routing.
  • Fixed snapshot restore handling cases where RBAC is nil.
  • Skipped replication operations that are already running or completed.
  • Fixed replication to forbid multiple move operations on the same source.
  • Fixed copy of docidcounter and prop tracker files.
  • Fixed engine shutdown to cancel if error max is reached.
  • Fixed query consumer semaphore to query the leader to avoid racey state changes.

🔧 Affected Symbols

ShardReplicationEngineGET /v1/replication/replicate/{id}DELETE /replications/replicate