You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I would like to compute alpha values for different subsets of my annotated data. But computing alpha for each subset separately needs different data for the random_coincidences matrix. Since the random_coincidences matrix is computed based on the reliability_data, the expected values get skewed if annotations in subsets are unbalanced.
It would be nice if one could either add all_reliability_data as parameter or if one could compute random_coincidences by oneself and add this matrix as parameter to the alpha-function.
The text was updated successfully, but these errors were encountered:
johannabi
changed the title
Support for document-wise computation of alpha
Support computation of alpha for subsamples
Oct 24, 2024
I would like to compute alpha values for different subsets of my annotated data. But computing alpha for each subset separately needs different data for the random_coincidences matrix. Since the random_coincidences matrix is computed based on the reliability_data, the expected values get skewed if annotations in subsets are unbalanced.
It would be nice if one could either add all_reliability_data as parameter or if one could compute random_coincidences by oneself and add this matrix as parameter to the alpha-function.
The text was updated successfully, but these errors were encountered: