You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The following aspects of the output and questions of the recording and preparing process were a bit confusing to me, would be great if they could be improved :)
prepare
Allow to specify multiple models at the same time. Vulnerabilties are more valuable the more generalizable they are, so encouraging to report the biggest amount of models possible is better.
record
Ask extra questions for parameters such as temperature, max_tokens, top_p (leave empty for default). I have no idea how I would go about changing them right now.
Do not just say model failed the test: at first, I thought this meant the instance was not a good example of the vulnerability (I actually ran the ToxicityChecker with threshold 0 before realizing it was the other way around, since I just though the Toxicity Model was bad). Rename passed to something more appropriate such as vulnerability present. Also, might be good to add the exact toxicity/bias score to the outputs. This gives the user the possibility to do some checks.
The text was updated successfully, but these errors were encountered:
The following aspects of the output and questions of the recording and preparing process were a bit confusing to me, would be great if they could be improved :)
prepare
record
model failed the test
: at first, I thought this meant the instance was not a good example of the vulnerability (I actually ran the ToxicityChecker with threshold 0 before realizing it was the other way around, since I just though the Toxicity Model was bad). Renamepassed
to something more appropriate such asvulnerability present
. Also, might be good to add the exact toxicity/bias score to the outputs. This gives the user the possibility to do some checks.The text was updated successfully, but these errors were encountered: