●Change8

1.0.0

📦 chromaView on GitHub →
✨ 48 features🐛 8 fixes⚡ 3 deprecations🔧 8 symbols

Summary

Version 1.0.0 introduces major architectural improvements, including moving the sysdb client to its own crate, enhancing distributed system capabilities with better routing and assignment, and improving compaction and garbage collection mechanisms. This release also features updates to Python and JS clients, including switching to the official Ollama client.

Migration Steps

  1. If you rely on the internal structure of the worker crate, note that the segment, log, memberlist, and assignment modules have been moved out.
  2. If you were using the old installation command for JS client setup, update it to use 'yarn/pnpm add' instead of 'yarn/pnpm install' based on documentation updates.

✨ New Features

  • Added tracing to intermediate methods between sysdb and fastapi.
  • Improved gRPC communication with knn.
  • Increased Sysdb GRPC max concurrent stream.
  • Added node name to memberlist.
  • Implemented node name usage for routing.
  • Implemented round robin gRPC connections amongst N query nodes.
  • Moved sysdb client into its own crate and added a background poller for GC.
  • Added num_records_last_compaction metric to sysdb.
  • Updated compactor to flush total records upon compaction.
  • Updated DBConfig to take read address.
  • Added collection_id parameter to QuotaEnforcer enforce function.
  • Implemented GetCollectionSize on SysDB read replica.
  • Updated Python SysDB to support get_collection_size.
  • Switched to the official Ollama python client.
  • Fixed typings on the JS client.
  • Implemented Round Robin across tiny-stories data sets.
  • Implemented Get collections to GC and added to rust sysdb client.
  • Implemented compactor server interface.
  • Implemented handling for one-off compaction messages in the compaction manager.
  • Implemented compaction client.
  • Fixed dynamic span names in Jaeger.
  • Added collection_id to quota_enforcer calls.
  • Created VersionFiles in S3 from Sysdb.
  • Switched Ollama JS EF to use the ollama js client.
  • Implemented GC orchestrator and Fetch version file operator.
  • Added basic rust frontend stub.
  • Bumped helm chart for rust fe.
  • Implemented multiple member assignment.
  • Added basic rust bindings stub.
  • Created database flow e2e tests.
  • Defined query operators and plan in rust types.
  • Implemented Get database e2e tests.
  • Added Rust mypy stubs for precommit.
  • Balanced gRPC channels.
  • Serialized Where clause to protobuf.
  • Serialized/Deserialized query plan to/from ProtoBuf.
  • Added GRPCs in SysDB to help with garbage collection.
  • Implemented Rust<>Python proxy calls back into python.
  • Implemented distributed executor for Rust frontend.
  • Implemented SysDB get_collection_with_segments endpoint.
  • Added query route.
  • Updated rust ml with extra fields.
  • Implemented multi-dimensional admission control.
  • Changed dispatcher from LIFO to FIFO.
  • Added metrics for circuit breaker in mdac.
  • Added better error handling for Axum routes.
  • Added parsers.
  • Added cache for FE with no invalidation.

🐛 Bug Fixes

  • Fixed BlockManager::cached() which was incorrectly always returning true.
  • Fixed binary search implementation of LimitOperator.
  • Fixed bad OpenAPI documentation.
  • Removed numpy pinning from the python client.
  • Fixed FastAPI operations id to be unique.
  • Fixed failing JS CI due to ollama import.
  • Cleaned up delete_segments logic.
  • Kept track of the current span if a task must be queued.

🔧 Affected Symbols

BlockManager::cachedLimitOperatorQuotaEnforcerSysDBOllama python clientOllama JS EFJaegerAxum routes

⚡ Deprecations

  • The segment module in the worker crate is deprecated.
  • The log module has been moved out of the worker crate.
  • The memberlist and assignment module have been moved out of the worker crate.