●Change8

v0.3.3

Breaking Changes
📦 ragas
⚠ 2 breaking✨ 11 features🐛 19 fixes⚡ 2 deprecations🔧 7 symbols

Summary

This release focuses heavily on internal restructuring, moving modules like `tracing`, `prompts`, `dataset`, and experimental features into the main package structure while retiring the `ragas.experimental` namespace. Numerous bug fixes address CI, LLM compatibility (especially OpenAI O1 series), and metric stability.

⚠️ Breaking Changes

  • The `ragas.experimental` namespace has been retired. Users must update imports from this namespace to the main `ragas` package locations.
  • The `dataset` module has been moved from an external location to the main `ragas` package. Imports referencing the old location must be updated.

Migration Steps

  1. Update any imports referencing the retired `ragas.experimental` namespace.
  2. Update imports for the `dataset` module to reflect its move to the main `ragas` package.
  3. Review usage of `embeddings` due to added deprecation warnings and refactoring.

✨ New Features

  • Refactored Ragas `embeddings` to maintain backward compatibility.
  • Moved `tracing` functionality from experimental to `ragas/integrations`.
  • Refactored LLM structure and added `InstructorLLM`.
  • Moved `prompts` module to the main `ragas` package.
  • Added support for cancellable tasks.
  • Knowledge graph flexibility has been improved.
  • Improved relationship builders for better async support and reduced memory utilization.
  • Added support for `make install-minimal` to speed up CI and development.
  • Moved experimental metrics to the main package.
  • Examples are now available as an installable package.
  • Added support for both LangChain LLM and Ragas LLM interfaces.

🐛 Bug Fixes

  • Fixed setup and local CI issues.
  • Fixed issue where model fields were accessed incorrectly; now accessed via class.
  • Fixed an issue where the answer was not matching the split sentences.
  • Fixed shields badge for downloads to use pepy.
  • Fixed type checking issues reported by pyright.
  • Fixed OIDC error in Claude code.
  • Fixed git ignore for `_version.py`.
  • Fixed repeating template bug in `_nv_metrics.py`.
  • Fixed various `ragas` directory structure path issues.
  • Fixed e2e test failures related to `datasets>=4.0.0` and missing dependencies.
  • Fixed issue with cache hits.
  • Replaced looping `generate` with `generate_multiple` in `ResponseRelevancy` metric.
  • Updated required columns in `RefusalRate` documentation.
  • Fixed broken mermaid diagrams on navigation.
  • Fixed typos and cleaned up style in various places.
  • Fixed `BadRequestError` when using OpenAI O1 series models; added option to bypass temperature.
  • Fixed logic where `False` was not returned when `pred_sequence` length and `ref_sequence` length differed.
  • Fixed temperature change to 0.01 related bugs.
  • Fixed broken link in FAQ regarding metrics tracing.

🔧 Affected Symbols

ragas.experimentalragas.datasetragas.embeddingsResponseRelevancy_nv_metrics.pyRefusalRateInstructorLLM

⚡ Deprecations

  • Deprecation warnings have been added for the old `embeddings` structure/usage.
  • Support for v0.2 security releases has been stopped.