probe: add Arabic donotanswer #1017
Conversation
Signed-off-by: Emad Alghamdi <[email protected]>
|
DCO Assistant Lite bot All contributors have signed the DCO ✍️ ✅ |
|
I have read the DCO Document and I hereby sign the DCO |
|
recheck |
There was a problem hiding this comment.
Thank you for this! Additional language support is currently evolving and this give some insight into how others are thinking about it.
I made a couple comments mostly related to location of code, long term form for this may change further, however this is great step towards enabling testing on targets in more languages.
garak/detectors/mitigation_ar.py
Outdated
There was a problem hiding this comment.
I think language specific detectors are likely better contained in the same module. In this case adding this class to garak/detectors/mitigation.py as it's own class is a preferred approach to make this available in the near term possibly as garak.detectors.mitigation.MitigationBypassArabic or MitigationBypassAR to use the bcp47 value as an indicator of language.
In a future iteration I would like to see garak.detectors.mitigation.MitigationBypass load the language specific values needed from a data file based on the language compatible with the probe that generated the attempt.
There was a problem hiding this comment.
Language specific implementations are likely better contained in the same module. In this case adding these classes to garak/probes/donotanswer.py as it's own set of classes, based on a prefix or suffix for the language, is a preferred approach to make this available in the near term.
In a future iteration I would like to see garak.probes.donotanswer.* load the language specific values needed from a data file based on the language the run requires to be compatible with the target application.
There was a problem hiding this comment.
I modified the original code to support Arabic (and potentially other languages). Now language specific implementation is included within the targeted probe and detector module. Finally, Arabic data path is changed to /data/ar/*
There was a problem hiding this comment.
A minor file structure item, consider using bcp47 values as a standard already adopted by the project to determine the storage path for language specific values, in this case garak/data/ar/* might provide a more programatic access friendly pattern locating files for a specific language.
Signed-off-by: Emad Alghamdi [email protected]
I added Arabic translation of all donotanswer probes which were quality checked by human to ensure suitability to the Arabic language, added Arabic detector for the probe. The new probes and detectors passed the test during development.