-
Notifications
You must be signed in to change notification settings - Fork 220
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fixes argument order for custom code evaluator #1690
Conversation
@LucasTrg is attempting to deploy a commit to the mmabrouk's projects Team on Vercel. A member of the Team first needs to authorize it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're right! Thanks for catching that, @LucasTrg!
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
@all-contributors please add @LucasTrg for code and bug fix |
I've put up a pull request to add @LucasTrg! 🎉 |
Thanks @LucasTrg ! |
Hey, I noticed arguments were fed out of order for custom code evaluators.
This is the template for the proposed custom evaluator
But currently evaluators are invoked with
result = environment["evaluate"](app_params, inputs, correct_answer, output)
To me the argument order in the template makes more sense, but it might not be backward compatible for people that worked around it by modifying the signature of the "evaluate" method.
Let me know which options sounds best to you and I can expand the PR in the desired direction :)