●Change8

v0.3.6

📦 ragas
✨ 10 features🐛 15 fixes⚡ 1 deprecations🔧 6 symbols

Summary

This release introduces several new features, including CHRF score support, enhanced input flexibility for metrics, and OCI Gen AI integration. Numerous bug fixes address issues related to asyncio, metric calculations, and dependency compatibility.

Migration Steps

  1. If you rely on LLMs or Prompts, be aware of the new deprecation warnings and plan to update usage according to future releases.
  2. If encountering issues with HuggingfaceEmbeddings and empty arrays, ensure your environment is compatible or check for updates related to Numpy 3.13 fixes.

✨ New Features

  • Added support for CHRF score calculation.
  • Metrics now support greater input flexibility.
  • Implemented tokens counting functionality.
  • Added support for tool call F1 metric.
  • Added Azure token usage extraction.
  • Enabled parallel tool call evaluation.
  • Introduced quoted spans metric.
  • Added LlamaIndex agentic evals for Gemini.
  • Added OCI Gen AI Integration for direct LLM support.
  • Added save/load functionality and improved repr for LLM-based metrics.

🐛 Bug Fixes

  • Fixed asyncio related issues.
  • Updated simple RAG initialization to use embed_text(s) correctly.
  • Ensured old_temperature is set correctly, fixing issues #1937 and #2110.
  • Fixed coroutine warning for bleu score calculation.
  • Added disallowed_special parameter when using tiktoken encode.
  • Improved metric decorators with better validation and error handling.
  • Fixed noise sensitivity computation.
  • Fixed generate_multiple caching issue (#1980).
  • Fixed concurrent ResponseRelevancy evaluation.
  • Corrected answer_relevancy scoring logic to prevent false zeros.
  • Fixed Numpy 3.13 compatibility issue.
  • Fixed type hints in tests.
  • Fixed metric inheritance patterns by separating factory-created metrics from class-instantiated metrics.
  • Fixed ambiguous truth value for empty numpy array in HuggingfaceEmbeddings (fixes #2080).
  • Fixed various typos in Markdown files.

🔧 Affected Symbols

EmbeddingExtractorHuggingfaceEmbeddings_bleu_score.pysimple RAG initLLMsPrompts

⚡ Deprecations

  • Deprecation warnings for LLMs and Prompts have been introduced.