You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
However, with #396 we're changing what datasets look like. So far, a lot of metadata lived on the dataset level and will be moved to the prompt level. This makes sense because every prompt may have different harm categories.
While not in scope for #396 we want the accurate harm_categories to be reflected for each prompt.
Describe the solution you'd like
Go through all the datasets and see if there are indications per prompt on what harm_categories the prompts belong to. For example, the original dataset may have a column for that. Note that this can be multiple values (harm categories is plural, i.e., a list).
This can be done one dataset at a time. If you volunteer to take on one dataset please comment below to avoid having redundant work.
Is your feature request related to a problem? Please describe.
We have lots of fetch methods for datasets in
pyrit.datasets
, see https://github.com/Azure/PyRIT/blob/main/pyrit/datasets/fetch_example_datasets.pyHowever, with #396 we're changing what datasets look like. So far, a lot of metadata lived on the dataset level and will be moved to the prompt level. This makes sense because every prompt may have different harm categories.
While not in scope for #396 we want the accurate
harm_categories
to be reflected for each prompt.Describe the solution you'd like
Go through all the datasets and see if there are indications per prompt on what
harm_categories
the prompts belong to. For example, the original dataset may have a column for that. Note that this can be multiple values (harm categories is plural, i.e., a list).This can be done one dataset at a time. If you volunteer to take on one dataset please comment below to avoid having redundant work.
CC @rdheekonda
The text was updated successfully, but these errors were encountered: