Skip to content

Commit

Permalink
per-probe tags now adjustable based on payload selection (#1031)
Browse files Browse the repository at this point in the history
fix bug where `encoding` probes with payloads set in config would fail
to load

## Verification

try this config

encodingpayloads.yaml
```
run:
  generations: 1

plugins:
  probe_spec: encoding
  extended_detectors: true
  probes:
    encoding:
      payloads:
        - default
        - xss
        - slur_terms
  model_type: test
```

Before fix:

```
failed to load probe probes.encoding.InjectAscii85
failed to load probe probes.encoding.InjectBase16
failed to load probe probes.encoding.InjectBase2048
failed to load probe probes.encoding.InjectBase32
failed to load probe probes.encoding.InjectBase64
failed to load probe probes.encoding.InjectBraille
failed to load probe probes.encoding.InjectEcoji
failed to load probe probes.encoding.InjectHex
failed to load probe probes.encoding.InjectMorse
failed to load probe probes.encoding.InjectNato
failed to load probe probes.encoding.InjectROT13
failed to load probe probes.encoding.InjectUU
failed to load probe probes.encoding.InjectZalgo
```

After fix:

```
encoding.InjectAscii85                                                          encoding.DecodeMatch: PASS  ok on 1140/1140                                                         
encoding.InjectAscii85                                                         encoding.DecodeApprox: PASS  ok on 1140/1140
encoding.InjectBase16                                                           encoding.DecodeMatch: PASS  ok on  570/ 570                                                         
encoding.InjectBase16                                                          encoding.DecodeApprox: PASS  ok on  570/ 570
encoding.InjectBase2048                                                         encoding.DecodeMatch: PASS  ok on  570/ 570                                                         
encoding.InjectBase2048                                                        encoding.DecodeApprox: PASS  ok on  570/ 570
encoding.InjectBase32                                                           encoding.DecodeMatch: PASS  ok on  570/ 570                                                         
encoding.InjectBase32                                                          encoding.DecodeApprox: PASS  ok on  570/ 570
encoding.InjectBase64                                                           encoding.DecodeMatch: PASS  ok on 1105/1105                                                         
encoding.InjectBase64                                                          encoding.DecodeApprox: PASS  ok on 1105/1105
encoding.InjectBraille                                                          encoding.DecodeMatch: PASS  ok on  570/ 570                                                         
encoding.InjectBraille                                                         encoding.DecodeApprox: PASS  ok on  570/ 570
encoding.InjectEcoji                                                            encoding.DecodeMatch: PASS  ok on  570/ 570                                                         
encoding.InjectEcoji                                                           encoding.DecodeApprox: PASS  ok on  570/ 570
encoding.InjectHex                                                              encoding.DecodeMatch: PASS  ok on  570/ 570                                                         
encoding.InjectHex                                                             encoding.DecodeApprox: PASS  ok on  570/ 570
encoding.InjectMorse                                                            encoding.DecodeMatch: PASS  ok on  570/ 570                                                         
encoding.InjectMorse                                                           encoding.DecodeApprox: PASS  ok on  570/ 570
encoding.InjectNato                                                             encoding.DecodeMatch: PASS  ok on  570/ 570                                                         
encoding.InjectNato                                                            encoding.DecodeApprox: PASS  ok on  570/ 570
encoding.InjectROT13                                                            encoding.DecodeMatch: PASS  ok on  570/ 570                                                         
encoding.InjectROT13                                                           encoding.DecodeApprox: PASS  ok on  570/ 570
encoding.InjectUU                                                               encoding.DecodeMatch: PASS  ok on  570/ 570                                                         
encoding.InjectUU                                                              encoding.DecodeApprox: PASS  ok on  570/ 570
encoding.InjectZalgo                                                            encoding.DecodeMatch: PASS  ok on  588/ 588                                                         
encoding.InjectZalgo                                                           encoding.DecodeApprox: PASS  ok on  588/ 588
```

side note - `encoding.py` is getting close to the point where it needs a
tidy
  • Loading branch information
leondz authored Dec 2, 2024
2 parents 5495536 + 0f678af commit c236753
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion garak/probes/encoding.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@


def _load_payloads():
global payloads
global payloads, extra_tags
payloads = []

extra_payload_tags = {
Expand Down

0 comments on commit c236753

Please sign in to comment.