Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support JSON config as well as YAML #913

Open
leondz opened this issue Sep 13, 2024 · 0 comments
Open

support JSON config as well as YAML #913

leondz opened this issue Sep 13, 2024 · 0 comments
Labels
architecture Architectural upgrades

Comments

@leondz
Copy link
Collaborator

leondz commented Sep 13, 2024

Summary

YAML isn't a universally powerful and flawless format. Consider also supporting JSON for standard configs.

Basic example

E.g. fast.yaml would become:

{
  "system": {
    "parallel_attempts": 20,
    "lite": true
  },
  "run": {
    "generations": 5
  },
  "plugins": {
    "probe_spec": "continuation,dan,encoding.InjectBase64,encoding.InjectHex,goodside,av_spam_scanning,leakreplay,lmrc,malwaregen.SubFunctions,malwaregen.TopLevel,packagehallucination,realtoxicityprompts.RTPIdentity_Attack,realtoxicityprompts.RTPProfanity,realtoxicityprompts.RTPSexually_Explicit,realtoxicityprompts.RTPThreat,snowball,xss",
    "extended_detectors": false
  }
}

Motivation

Why are we doing this? What use cases does it support? What is the expected outcome?

  1. YAML lacks some definitional strength, making certain values difficult to express, and an alternative is useful
  2. The internal config structure is agnostic to which text format the config is stored in, hopefully keeping the scope of code changes relatively limited
  3. It's good to provide us with flexibility around something as core and powerful as configs
  4. JSON schema validation is trivial and built-in to python, enabling strong config checking pre-run
  5. We will gain a quick, reasonable answer to the various gripes about yaml that I (and presumably also the world) have, like how the official YAML implementation in Python doesn't follow the official YAML spec
@leondz leondz added the architecture Architectural upgrades label Sep 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
architecture Architectural upgrades
Projects
None yet
Development

No branches or pull requests

1 participant