Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

payloads: web-related, incl. probes.encoding migration #933

Merged
merged 19 commits into from
Oct 11, 2024

Conversation

leondz
Copy link
Collaborator

@leondz leondz commented Sep 26, 2024

Factor out web-related payloads, and complete the transition to garak.payloads use in probes.encoding

Verification

  • garak -m test -g 1 -p encoding,latentinjection,xss and check the report.jsonls include appropriate payload text
  • python -m pytest tests/test_payloads.py
  • python -m pytest tests/probes/test_probes_encoding.py
  • python -m pytest tests/probes/test_probes_latentinjection.py

@leondz leondz added architecture Architectural upgrades probes Content & activity of LLM probes labels Sep 26, 2024
@leondz leondz requested a review from jmartin-tech September 26, 2024 15:02
@leondz leondz changed the title payloads: web-related, and probes.encoding migration payloads: web-related, incl. probes.encoding migration Sep 28, 2024
@jmartin-tech
Copy link
Collaborator

jmartin-tech commented Oct 2, 2024

Lets consider making the exposed entry point for load_payload be the interface to the Director.

Copy link
Collaborator

@jmartin-tech jmartin-tech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to have consistent DEFAULT_PARAMS naming as payload_name? Examples of impact noted in the review.

Also looking at this payload_trigger being populated directly with the payloads set seems non-intuitive, but I don't have a better alternative to offer at this time so reasonable to live with for now.

garak/probes/latentinjection.py Outdated Show resolved Hide resolved
garak/probes/latentinjection.py Outdated Show resolved Hide resolved
garak/probes/latentinjection.py Outdated Show resolved Hide resolved
garak/probes/xss.py Outdated Show resolved Hide resolved
garak/probes/xss.py Outdated Show resolved Hide resolved
garak/payloads.py Show resolved Hide resolved
garak/payloads.py Show resolved Hide resolved
@leondz leondz merged commit 90ed571 into main Oct 11, 2024
10 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Oct 11, 2024
@jmartin-tech jmartin-tech deleted the payload/web_related branch October 11, 2024 13:00
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
architecture Architectural upgrades probes Content & activity of LLM probes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants