Skip to content

Issues: confident-ai/deepeval

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

106 Open 194 Closed
106 Open 194 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

ContextualPrecision endless generation
#1205 opened Dec 3, 2024 by enrico-stauss
Make TELEMETRY_DATA_FILE located into temp folder or configurable by env var
#1200 opened Nov 29, 2024 by cisarpavel
AssertionError: n_contexts_per_doc must be a positive integer.
#1199 opened Nov 29, 2024 by tenchanhe
Issue regarding consistent loading of source_files data (from goldens original values)
#1173 opened Nov 21, 2024 by CAW-nz
3
How to integrate llama3 to deepeval?
#1163 opened Nov 15, 2024 by drbhushan
1
GEval not focusing on expected_output & Relying on OpenAI instead
#1149 opened Nov 11, 2024 by pavan-growexxer
2
Asynchronous test runs are sometimes not completed correctly
#1147 opened Nov 9, 2024 by jmaczan
14
Hallucination metric score is set to 0.0 in one run and 1.0 in another run, despite having the same input values
#1146 opened Nov 9, 2024 by jmaczan
1
Hallucination metric assigns only either 0.0 or 1.0 score
#1145 opened Nov 9, 2024 by jmaczan
1
EvaluationDataset in deepeval suddenly requires an OpenAi key since update
#1140 opened Nov 4, 2024 by kbarendrecht
5
Error while calculating Knowledge retention ; Evaluation LLM outputted an invalid JSON. Please use a better evaluation model.
#1139 opened Nov 4, 2024 by jaysudhakaran
4
Improving Answer Relevancy Template
#1137 opened Nov 3, 2024 by dipanjanS
4
Get tokens for each request
#1114 opened Oct 22, 2024 by luqianjiang
1
TypeError while evaluating a test case for Role Adherence metric using CustomLlama3_8B.
#1105 opened Oct 21, 2024 by Qca-2023
HumanEval doesn't actually runs test
#1102 opened Oct 19, 2024 by cfytrok
The latest versions of deepeval and Instructor can't be installed together
#1100 opened Oct 18, 2024 by milest
5
APIConnectionError: Connection error.
#1094 opened Oct 17, 2024 by paulacanva
3
Accuracy Issue - accuracy is 0.00
#1090 opened Oct 17, 2024 by Pitambartech
Incompatibility with ragas v0.2.x causing TypeError in measure function
#1089 opened Oct 17, 2024 by lagom0327
Typo in scorer/scorer.py
#1087 opened Oct 16, 2024 by jtquach1
Missing API key and endpoint configuration...(etc) in OpenAI() and support for Azure OpenAI
#1086 opened Oct 16, 2024 by NaraSyuhei
1
Question About The Syntesizer
#1068 opened Oct 10, 2024 by Miriam2040
Results are not in the same order as input test cases
#1034 opened Sep 25, 2024 by joaopcm1996
1
Typo in docs/docs/benchmarks-introduction.mdx
#1032 opened Sep 24, 2024 by jtquach1
g_eval metric implicitly falls back to top score instead of sum of weighted top probabilities
#1029 opened Sep 23, 2024 by mattany
1
ProTip! Add no:assignee to see everything that’s not assigned.