Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating rule: link_microsoft_low_reputation.yml #778

Closed
wants to merge 5 commits into from

Conversation

aidenmitchell
Copy link
Member

@aidenmitchell aidenmitchell commented Sep 7, 2023

Removing link check and editing NLU logic. Holds up well in hunts.

Removing link check and editing NLU logic. Holds up well in hunts.
@aidenmitchell aidenmitchell requested a review from a team September 7, 2023 23:22
@notion-workspace
Copy link

1693620022

@aidenmitchell aidenmitchell marked this pull request as draft September 8, 2023 00:39
@aidenmitchell aidenmitchell marked this pull request as ready for review September 12, 2023 16:02
@aidenmitchell
Copy link
Member Author

Leaving the Tranco check in... because it works. We should investigate a better solution later, but the rule preforms well in this state.

)
)
or (
any(ml.nlu_classifier(body.html.inner_text).entities, .name == "urgency")
and not any(ml.nlu_classifier(body.current_thread.text).intents,
.name == "benign" and .confidence == "high"
and any(ml.nlu_classifier(body.current_thread.text).intents, .name not in~ ("benign", "unknown")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aidenmitchell does this need to be an all to match our previous behavior? as written, we could get a benign:high and cred_theft:low and still flag here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we could get a benign and cred_theft intent, because we're only evaluating one thing, body.current_thread.text?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.intents is an array because i think we can return multiple with different confidence levels, but @bfilar can confirm

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so if the model returns benign that would be the only intent we return. If benign is not present in intent array would could return something like: (cred_theft,steal_pii)

@aidenmitchell
Copy link
Member Author

Closing in favour of #791

@aidenmitchell aidenmitchell deleted the patch-50 branch September 18, 2023 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants