Skip to content

Issues: NVIDIA/garak

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

306 Open 296 Closed
306 Open 296 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

tap.PAIR getting "run_tap() got an unexpected keyword argument 'evaluator_model'" bug Something isn't working
#1044 opened Dec 15, 2024 by rafaelsandroni
FEAT Add a custom REST LLM API template to Garak Config yaml file
#1041 opened Dec 10, 2024 by EricXQiu
generator feature: add deprefix prefix generators Interfaces with LLMs
#1038 opened Dec 6, 2024 by leondz 24.12
@leondz
package garak for kali housekeeping Repository and project maintenance
#1036 opened Dec 5, 2024 by leondz
@jmartin-tech
3
error: 'latin-1' codec can't encode character '\u2026' bug Something isn't working
#1035 opened Dec 4, 2024 by wiktorw767
2
probes: fileformats probe fails when targeting a local modal path bug Something isn't working
#1034 opened Dec 3, 2024 by jmartin-tech
Fall back when online services can't be reached. bug Something isn't working
#1033 opened Dec 2, 2024 by erickgalinkin 24.12
1
align prompt count per probe architecture Architectural upgrades probes Content & activity of LLM probes
#1032 opened Dec 2, 2024 by leondz
Allow applying prompt template to the input for huggingface models.
#1029 opened Dec 1, 2024 by zijiachen95 24.12
@jmartin-tech
2
reporting: enable logging & skipping of unrecognised probes, detectors in digest generation reporting Reporting, analysis, and other per-run result functions
#1024 opened Nov 25, 2024 by leondz
probe: Image Hijack new plugin Describes an entirely new probe, detector, generator or harness probes Content & activity of LLM probes
#1023 opened Nov 25, 2024 by leondz
probe: gritty pixy new plugin Describes an entirely new probe, detector, generator or harness probes Content & activity of LLM probes
#1020 opened Nov 22, 2024 by leondz
2
probe: more xss exfil probes probes Content & activity of LLM probes
#996 opened Nov 11, 2024 by leondz
4 tasks
24.12
atkgen: disinfo probe probes Content & activity of LLM probes
#994 opened Nov 11, 2024 by leondz
atkgen: illegal content probe probes Content & activity of LLM probes
#993 opened Nov 11, 2024 by leondz
detector: Use LLMaaJ to see if models are following payload instructions architecture Architectural upgrades detectors work on code that inherits from or manages Detector
#992 opened Nov 11, 2024 by leondz 24.12
feature: Enable use_safetensors/True for HF models and expose as config var architecture Architectural upgrades
#990 opened Nov 11, 2024 by leondz 24.12
@leondz
feature: Block pickle loading unless garak’s configured to do unsafe things architecture Architectural upgrades
#989 opened Nov 11, 2024 by leondz
5 tasks
24.12
feature: add plugin file date/hash to plugin load messages & logs for auditing architecture Architectural upgrades
#988 opened Nov 11, 2024 by leondz 24.12
@jmartin-tech
test: run tests automatically at fixed time intervals, to catch shifts in dependencies housekeeping Repository and project maintenance tests Testing-related
#987 opened Nov 11, 2024 by leondz 24.12
@jmartin-tech
feature: support audit trails, where asset IDs can be plugged in architecture Architectural upgrades
#985 opened Nov 11, 2024 by leondz 24.12
feature: auto-eval architecture Architectural upgrades
#984 opened Nov 11, 2024 by leondz 24.12
@erickgalinkin
feature: Enable skipping of tests for policy points target is permissive over architecture Architectural upgrades policy Related to policy scanning
#983 opened Nov 11, 2024 by leondz
feature: report target’s apparent policy & match to a predefined policies architecture Architectural upgrades policy Related to policy scanning
#982 opened Nov 11, 2024 by leondz
11 tasks
atkgen: flexible probe probes Content & activity of LLM probes
#981 opened Nov 11, 2024 by leondz
7 tasks
ProTip! Exclude everything labeled bug with -label:bug.