You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Gavel has multiple components - the ranking algorithm is just one part of it, but the score collection / admin panel is another important part of the system. It would be neat if these were decoupled enough so that the system could support different judging schemes.
Right now, we only support pairwise comparison as a judging model with Crowd-BT as the algorithm, but maybe it would be cool to support other algorithms (e.g. Elo) and maybe even different judging models (e.g. normalization). It would be nice to write accompanying documentation that describes the tradeoffs of the different models and algorithms (e.g. with very few judges, Crowd-BT might be hard, and absolute scores + normalization may be an okay approach).
This would likely be a very substantial change to Gavel.
The text was updated successfully, but these errors were encountered:
Gavel has multiple components - the ranking algorithm is just one part of it, but the score collection / admin panel is another important part of the system. It would be neat if these were decoupled enough so that the system could support different judging schemes.
Right now, we only support pairwise comparison as a judging model with Crowd-BT as the algorithm, but maybe it would be cool to support other algorithms (e.g. Elo) and maybe even different judging models (e.g. normalization). It would be nice to write accompanying documentation that describes the tradeoffs of the different models and algorithms (e.g. with very few judges, Crowd-BT might be hard, and absolute scores + normalization may be an okay approach).
This would likely be a very substantial change to Gavel.
The text was updated successfully, but these errors were encountered: