●Change8

v1.33.1

📦 weaviateView on GitHub →
✨ 10 features🐛 30 fixes🔧 7 symbols

Summary

This release focuses heavily on stability and performance improvements, particularly around RAFT consensus, memberlist configuration, and extensive additions to metrics collection across various components like LSM, compaction, and replication. It also introduces image support for the generative-cohere module.

Migration Steps

  1. refact(memberlist-config): re-name env var to MEMBERLIST_FAST_FAILURE_DETECTION by @moogacs in https://github.com/weaviate/weaviate/pull/9260 (If you were using the old environment variable name for fast failure detection, update it to MEMBERLIST_FAST_FAILURE_DETECTION.)

✨ New Features

  • feat(modules): add support for images in generative-cohere module by @antas-marcin in https://github.com/weaviate/weaviate/pull/9253
  • feature: RoaringSet as default strategy for dimensions bucket by @aliszka in https://github.com/weaviate/weaviate/pull/9283
  • feat: :loud_sound: add 1% sampled queries to slow log by @amourao in https://github.com/weaviate/weaviate/pull/9315
  • feat(metrics): lsm wal recovery metrics by @jeroiraz in https://github.com/weaviate/weaviate/pull/9312
  • feat(metrics): memtable flushing metrics by @jeroiraz in https://github.com/weaviate/weaviate/pull/9294
  • feat bucket lifecycle metrics by @jeroiraz in https://github.com/weaviate/weaviate/pull/9329
  • feat(metrics): segment metrics by @jeroiraz in https://github.com/weaviate/weaviate/pull/9321
  • feat(metrics): lsm cursor metrics by @jeroiraz in https://github.com/weaviate/weaviate/pull/9336
  • feat(metrics): bucket read/write ops metrics by @jeroiraz in https://github.com/weaviate/weaviate/pull/9337
  • feat(debug): add debug endpoints for shard and lock status monitoring by @amourao in https://github.com/weaviate/weaviate/pull/9402

🐛 Bug Fixes

  • fix(raft-transport): avoid invalid ports to avoid rollout errors and tweak memberlist config by @moogacs in https://github.com/weaviate/weaviate/pull/9064
  • fix(raft-rpc): client polic retry and unit test by @moogacs in https://github.com/weaviate/weaviate/pull/9010
  • fix(raft): avoid raft node discovery for local node by @moogacs in https://github.com/weaviate/weaviate/pull/9230
  • fix(modules): remove model validation in generative-cohere module by @antas-marcin in https://github.com/weaviate/weaviate/pull/9231
  • fix(raft): pass local node name and address to store to use it in discovery by @moogacs in https://github.com/weaviate/weaviate/pull/9235
  • fix(raft): joiner avoid self join dials by @moogacs in https://github.com/weaviate/weaviate/pull/8997
  • Fix appending to broken .wal files by @dirkkul in https://github.com/weaviate/weaviate/pull/9238
  • Fix case of a missing user prefix by @dirkkul in https://github.com/weaviate/weaviate/pull/9349
  • Avoid collisions on compressed named vector buckets by @abdelr in https://github.com/weaviate/weaviate/pull/9240
  • hnsw: prevent race between UpdateUserConfig and ShouldUpgrade by @asdine in https://github.com/weaviate/weaviate/pull/9358
  • Wait for cache prefill before async indexing can trigger an upgrade by @trengrj in https://github.com/weaviate/weaviate/pull/9345
  • fix: panic caused by dropping store before queues by @asdine in https://github.com/weaviate/weaviate/pull/9367
  • Avoid collisions with vectors compressed folder and RQ quantization by @antas-marcin in https://github.com/weaviate/weaviate/pull/9361
  • Requantize api should skip uncompressed vectors by @trengrj in https://github.com/weaviate/weaviate/pull/9365
  • Add info PQ compression log if fails by @robbespo00 in https://github.com/weaviate/weaviate/pull/9373
  • ensure localResults has enough space by @asdine in https://github.com/weaviate/weaviate/pull/9370
  • Ensure error returned by `getOrInitShard` is passed to the error response by @tsmith023 in https://github.com/weaviate/weaviate/pull/9374
  • fix TestTotalDimensionTrackingMetrics race by @asdine in https://github.com/weaviate/weaviate/pull/9380
  • Fix fetchObject with certainty for multi-named-vectors by @dirkkul in https://github.com/weaviate/weaviate/pull/9376
  • Ensure error returned by `getOrInitShard` is passed to the error response by @tsmith023 in https://github.com/weaviate/weaviate/pull/9385
  • Refactor usage module and properly lock index and shards during read by @dirkkul in https://github.com/weaviate/weaviate/pull/9323
  • Fix object and vector storage calculations by @dirkkul in https://github.com/weaviate/weaviate/pull/9392
  • skip large ids when little-endian and big-endian ids are mixed in the same bucket by @asdine in https://github.com/weaviate/weaviate/pull/9397
  • Make bucket min reuse default in line with weaviate default by @dirkkul in https://github.com/weaviate/weaviate/pull/9257
  • use correct boundary for mix endianness detection by @asdine in https://github.com/weaviate/weaviate/pull/9414
  • Add strategy check for map cursor by @dirkkul in https://github.com/weaviate/weaviate/pull/9410
  • fix: remove all possible weaviate_lsm_bucket_cursor_duration_seconds related metrics by @antas-marcin in https://github.com/weaviate/weaviate/pull/9421
  • change limits between big endian and little endian detection by @rlmanrique in https://github.com/weaviate/weaviate/pull/9417
  • fix: don't cause cluster crash when runtime config is malformed by @gkampitakis in https://github.com/weaviate/weaviate/pull/9411
  • Remove model validation from generative+rerank modules by @dirkkul in https://github.com/weaviate/weavia

🔧 Affected Symbols

generative-cohere moduleraft-transportraft-rpcraftgenerative+rerank moduleshnswMEMBERLIST_FAST_FAILURE_DETECTION