v0.1.21

Full Changelog: v0.1.16...v0.1.21

• Pins to langchain<0.3
• based on v0.1.16 which was the last working version for 0.1
• hopefully last release before v0.2

v0.1.20

What's Changed

• fix: pin langchain_core to <3 by @jjmachan in #1329

Full Changelog: v0.1.19...v0.1.20

v0.1.19

What's Changed

• feat: add non llm based context recall by @shahules786 in #1266
• feat: Non LLM based context precision by @shahules786 in #1264
• feat: added n and temperature passthrough to prompt by @jjmachan in #1281
• Add option to disable progress bar in evaluate by @freddiev4 in #1279
• feat: new aspect critic metrics by @shahules786 in #1286
• feat: add simple scorer metric by @shahules786 in #1291
• docs: add docs for new metrics by @shahules786 in #1292
• Fix wrong input key name by @ruanhao in #1300
• chore: move rubrics based metric into single file by @shahules786 in #1287
• feat: Instance specifc rubrics metrics by @shahules786 in #1304
• feat: agent goal accuracy metric by @shahules786 in #1303
• docs: agent metrics by @shahules786 in #1314
• Update Opik notebook example and integration by @jverre in #1295
• fix: column rename by @shahules786 in #1324

New Contributors

• @freddiev4 made their first contribution in #1279
• @ruanhao made their first contribution in #1300

Full Changelog: v0.1.18...v0.1.19

v0.1.18

What's Changed

• fix: inverse distance by @shahules786 in #1267
• fix: non llm based metrics by @shahules786 in #1268
• fix: v1 to v2 dataset by @shahules786 in #1275
• fix: make score nested if loop_is_running by @jjmachan in #1276

Full Changelog: v0.1.17...v0.1.18

v0.1.17

What's Changed

• update documentation for faithfulness with HHEM by @Miaoranmmm in #1242
• New data representation by @shahules786 in #1222
• Feat: Opik Integration by @jverre in #1256
• Add support to MultiTurn Metrics by @shahules786 in #1249
• fix: add single_turn_ascore by @shahules786 in #1261
• fix: handle v0.1 dataset by @shahules786 in #1262
• Non LLM based metrics by @shahules786 in #1260
• feat: set and get prompts for metrics by @shahules786 in #1259

New Contributors

• @jverre made their first contribution in #1256

Full Changelog: v0.1.16...v0.1.17

v0.1.16

What's Changed

• load HHEM on specified device by @Miaoranmmm in #1235
• feat: new Prompt object and faithfulness metric by @jjmachan in #1232
• feat (experimental) added new prompt and metric into ragas.experimental by @jjmachan in #1240

New Contributors

• @Miaoranmmm made their first contribution in #1235

Full Changelog: v0.1.15...v0.1.16

v0.1.15

What's Changed

• fix: typo in model name by @shahules786 in #1198
• metrics: add domain specific rubrics based scoring by @vaishakhRaveendran in #1189
• docs: rubrics based metrics by @shahules786 in #1200
• docs: add context utilisation to the index by @shahules786 in #1202
• Docs: fix typo in metrics by @shahules786 in #1203
• Implementation of Noise sensitivity metrics from RAGChecker by @sahusiddharth in #1190
• Tiny typo fix: it's -> its by @peterkaminski in #1219
• fix : solves #1215 - Inconsistency in the Value of EVALMODE_TO_COLUMNS[EvaluationMode.ca] by @jucgonzalezes in #1216
• chore: removed extra of requirements by @jjmachan in #1225
• Bugfix of evaluate function in llama_index.py by @vcim in #1223
• [FIX] - Fix for summarization edge case by @sky-2002 in #1201

New Contributors

• @peterkaminski made their first contribution in #1219
• @jucgonzalezes made their first contribution in #1216
• @vcim made their first contribution in #1223

Full Changelog: v0.1.14...v0.1.15

v0.1.14

What's Changed

• Add question to context recall doc by @shahules786 in #1173
• Add context utilization by @shahules786 in #1174
• doc: fix output for testset generation and missing files by @jjmachan in #1176
• Helicone Integration by @chitalian in #1178
• docs: minor fixes by @shahules786 in #1183
• solved issue that adaptation crashes when havig multiple nested promp… by @ssoima in #1181
• Bugfix Summarization Score by @emreds in #1164
• [DOCS] - Fix summarization score expression by @sky-2002 in #1184
• metrics: add faithfulness with HHEM model by @shahules786 in #1191
• Fix test command in development.md to match Makefile configuration by @sahusiddharth in #1195
• feat: deeper helicon integration by @jjmachan in #1196
• fix: added a test for prompts by @jjmachan in #1197

New Contributors

• @chitalian made their first contribution in #1178
• @ssoima made their first contribution in #1181
• @emreds made their first contribution in #1164
• @sahusiddharth made their first contribution in #1195

Full Changelog: v0.1.13...v0.1.14

v0.1.13

What's Changed

• Fix "global flags not at the start" raised in extract_text function by @vaishakhRaveendran in #1113
• Random seed fixed. by @Gwenn-LR in #1142
• feat: Usage tracking for evaluations by @jjmachan in #1129
• Fixing Demo Code - Compare Embeddings for retriever Docs by @matheusft in #1153
• fix: run_config post_init issue by @jjmachan in #1158
• Add save function for statement_prompt by @miyaketomoya in #1157
• Fix: adapt function to translate cached prompts in place. by @HerrIvan in #889
• fix: RunConfig documentation and improvements by @jjmachan in #1159
• Docs/run config by @jjmachan in #1160
• change segmenter language when language adaptaion by @miyaketomoya in #1163
• change default model to gpt4o-mini by @shahules786 in #1166
• Add tests to ragas.run_config.RunConfig rng attribute. by @Gwenn-LR in #1169
• feat: added track_completed decorator to track completed functions by @jjmachan in #1171
• docs: added development.md file by @jjmachan in #1115
• Allow Metric.score to work within an existing asyncio loop by @abramowskia in #1161

New Contributors

• @Gwenn-LR made their first contribution in #1142
• @matheusft made their first contribution in #1153
• @miyaketomoya made their first contribution in #1157
• @abramowskia made their first contribution in #1161

Full Changelog: v0.1.12...v0.1.13

v0.1.12

What's Changed

• [Fix bug] The adaptation for rewrite_invalid_question_prompt was not saved by @psydok in #1071
• [Fix bug] utils.py by @jrredondo in #1083
• Llama Index integration: optional RunConfig for evaluate() by @antono in #1085
• Create a custom directory loader for RAGAS by @vaishakhRaveendran in #1122
• Fix "sentence" -> "claim" the context recall metric docs by @artmatsak in #1126
• fix: temperature parameter in generate_text not ignored. by @HerrIvan in #887

New Contributors

• @psydok made their first contribution in #1071
• @jrredondo made their first contribution in #1083
• @antono made their first contribution in #1085
• @vaishakhRaveendran made their first contribution in #1122
• @artmatsak made their first contribution in #1126
• @HerrIvan made their first contribution in #887

Full Changelog: v0.1.11...v0.1.12