-
Notifications
You must be signed in to change notification settings - Fork 231
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feat - custom code evaluation in UI #589
Conversation
…re code safe execution
…ds in custom evaluation db
…t in evaluations component
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-
Getting this error.
https://github.com/Agenta-AI/agenta/assets/6608260/b1220883-9137-4d42-a3cb-3f46991abea0 -
@mmabrouk The button to create a custom evaluation is distracting the flow of the other evaluations.
My suggestions are: -
Add it as a first item of the dropdown like:
[List Custom evaluations]
|_ Create a new custom evaluation (This is a button to redirect to the new evaluation form)
|_ The items of list.. -
Have it as "Text Button" under the last dropdown.
The error has been fixed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@aybruhm There are bugs.
- Scores are not saved.
- When we run it again we get errors
For theses two bugs see this video
Screen.Recording.2023-09-13.at.17.55.23.mov
- In this new evaluation we can't see the code we added.
Please propose a solution the see the code. (editing the code after running the evaluation is not needed) - The Button to create a new custom evaluation is distracting the flow to run evaluations.
Try to put it within the dropdown or as an underlined text button under the dropdown - In the evaluations list we see NaN
- Minor but it looks like the the dropdown to list the evaluations is a different from the dropdown group. when we select another dropdown and then the custom evaluation, both got selected. Check this screenshot:
…ios logic to make backend call
Closed in favor of #610 |
All Fixed in #610 |
Description
This PR addresses #569 and introduces the following:
Workflow Summary:
Additional Notes:
Making API call(s):
If you plan to make an API call from the code to your local machine, please use Ngrok to tunnel it. Making an API call directly to your local machine from the custom evaluation code will not work due to the security restrictions in place.
Related Issue
Closes #569