Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow pluggable judging schemes / implement some different judging schemes #43

Open
anishathalye opened this issue Feb 8, 2017 · 0 comments

Comments

@anishathalye
Copy link
Owner

Gavel has multiple components - the ranking algorithm is just one part of it, but the score collection / admin panel is another important part of the system. It would be neat if these were decoupled enough so that the system could support different judging schemes.

Right now, we only support pairwise comparison as a judging model with Crowd-BT as the algorithm, but maybe it would be cool to support other algorithms (e.g. Elo) and maybe even different judging models (e.g. normalization). It would be nice to write accompanying documentation that describes the tradeoffs of the different models and algorithms (e.g. with very few judges, Crowd-BT might be hard, and absolute scores + normalization may be an okay approach).

This would likely be a very substantial change to Gavel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant