●Change8

v0.3.3rc1

Breaking Changes
📦 ragas
⚠ 2 breaking✨ 11 features🐛 20 fixes⚡ 2 deprecations🔧 7 symbols

Summary

This release focuses heavily on internal restructuring, migrating modules like `tracing`, `prompts`, `dataset`, and experimental metrics out of experimental namespaces and into the main package structure. It also includes numerous bug fixes, performance optimizations (like 50% speedup for factual correctness), and improved LLM compatibility.

⚠️ Breaking Changes

  • The `ragas.experimental` namespace has been retired. Users must update imports from this namespace to the main package structure.
  • The structure of `ragas.embeddings` has been refactored; while backward compatibility is maintained, users relying on internal structures might need review.

Migration Steps

  1. Update all imports that referenced modules or functions under `ragas.experimental` to their new locations in the main `ragas` package.
  2. Review usage of `ragas.embeddings` if internal structures were relied upon, due to refactoring.
  3. If using OpenAI O1 series models, ensure you are aware of the temperature setting change or explicitly set bypass temperature if needed.

✨ New Features

  • Refactored `embeddings` to maintain backward compatibility.
  • Moved `tracing` functionality from experimental to `ragas/integrations`.
  • Refactored LLM structure and added `InstructorLLM`.
  • Moved `prompts` module to the main `ragas` package.
  • Moved `dataset` module to the main package and updated imports.
  • Moved experimental metrics to the main package.
  • Added support for cancellable tasks.
  • Knowledge graph flexibility has been improved.
  • Improved relationship builders for better async performance and reduced memory utilization.
  • Added support for installing examples as an installable package.
  • Added support for both langchain LLM and ragas LLM objects.

🐛 Bug Fixes

  • Fixed setup and local CI issues.
  • Fixed issue where model fields were accessed incorrectly; now accessed via class.
  • Fixed an issue where the answer was not matching the split sentences.
  • Fixed shields badge for downloads to use pepy.
  • Fixed type checking issues for pyright.
  • Fixed OIDC error in Claude code.
  • Fixed git ignore for `_version.py`.
  • Fixed repeating template bug in `_nv_metrics.py`.
  • Fixed `ragas` directory structure paths.
  • Fixed an issue with cache hits.
  • Used `generate_multiple` instead of looping `generate` in `ResponseRelevancy`.
  • Updated required columns in `RefusalRate` documentation.
  • Fixed e2e test failures related to `datasets>=4.0.0` and missing dependencies.
  • Fixed broken link in FAQ (metrics tracing) documentation.
  • Fixed bugs related to temperature changes.
  • Fixed `BadRequestError` when using OpenAI O1 series models; added option to bypass temperature.
  • Returned `False` when `pred_sequence` length and `ref_sequence` length differ.
  • Fixed the mermaid diagram rendering issue on navigation.
  • Fixed typos and cleaned up style in documentation/code.
  • Fixed temperature setting to 0.01 for better LLM compatibility (and subsequent bugs related to this change).

🔧 Affected Symbols

ragas.experimentalragas.embeddingsInstructorLLM_nv_metrics.pyRefusalRateResponseRelevancy_version.py

⚡ Deprecations

  • Support for v0.2 security releases has been stopped.
  • Deprecation warnings have been added for existing `embeddings` usage due to refactoring.