Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fall back when online services can't be reached. #1033

Open
erickgalinkin opened this issue Dec 2, 2024 · 1 comment
Open

Fall back when online services can't be reached. #1033

erickgalinkin opened this issue Dec 2, 2024 · 1 comment
Labels
bug Something isn't working
Milestone

Comments

@erickgalinkin
Copy link
Collaborator

If a remote resource e.g. Huggingface model used in a detector cannot be reached, we should either fail gracefully or throw an error and offer a substitution. This will be important for running garak in airgapped/offline systems.

Steps to reproduce

Run garak -m test without an internet connection or with SSL intercept.

Expected behavior

When a website e.g. huggingface.co cannot be reached, gracefully pass to another detector/probe instead of failing with a raised ValueError.

Current behavior

Run terminates.

garak version

0.10.0

@erickgalinkin erickgalinkin added the bug Something isn't working label Dec 2, 2024
@leondz leondz added this to the 24.12 milestone Dec 2, 2024
@leondz
Copy link
Collaborator

leondz commented Dec 2, 2024

Run terminates.

not good or the correct behaviour. should in the very worst case skip that detector. thanks for flagging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants