●Change8

v0.3.8

Breaking Changes
📦 ragasView on GitHub →
⚠ 2 breaking✨ 6 features🐛 5 fixes⚡ 1 deprecations🔧 16 symbols

Summary

This release focuses heavily on internal refactoring, migrating core functionalities like semantic similarity and simple criteria to collections, and merging LLM factory methods. Several bugs related to async handling and specific synthesizers were also fixed.

⚠️ Breaking Changes

  • The internal structure for semantic similarity has been migrated to collections. If you were accessing internal components related to semantic similarity directly, you may need to update paths.
  • The `instructor_llm_factory` has been merged into `llm_factory`. Code using `instructor_llm_factory` must be updated to use `llm_factory`.

Migration Steps

  1. Update any code referencing `instructor_llm_factory` to use `llm_factory` instead.
  2. Review usage of semantic similarity components due to migration to collections.
  3. If encountering issues related to async patching, note that uvloop detection now correctly handles nest_asyncio skipping.

✨ New Features

  • Semantic similarity functionality has been migrated to collections.
  • Reusable testing infrastructure added for metrics migration.
  • Console scripts for `ragas_examples` added.
  • Quickstart command added with templates to run.
  • Aspect critic metric added supporting coherence, harmfulness, maliciousness, and correctness.
  • Simple criteria functionality has been migrated to collections.

🐛 Bug Fixes

  • Fixed an issue where uvloop detection would incorrectly skip nest_asyncio patching, preventing patching errors.
  • Resolved a NameError encountered during the evaluation of llamaindex query engine.
  • Removed error suppressor from `async_utils.py` and `engine.py`.
  • Fixed handling of tuple-formatted entities in `SingleHopSpecificQuerySynthesizer`.
  • Added missing metrics (ToolCallF1, ChrfScore) to the sidebar and documented the deprecation of ContextUtilization.

🔧 Affected Symbols

semantic similarity (internal components)answer_correctnesscontext_entity_recallContextRelevanceToolCallF1ChrfScoreContextUtilizationinstructor_llm_factoryllm_factorySingleHopSpecificQuerySynthesizersimple criteria (internal components)LangchainLLMWrapperLlamaIndexLLMWrapperembedding wrappersasync_utils.pyengine.py

⚡ Deprecations

  • ContextUtilization metric is now deprecated (mentioned in sidebar/docs update).