Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send sloth slos to grafana cloud #1348

Merged
merged 7 commits into from
Sep 16, 2024

Conversation

QuentinBisson
Copy link
Contributor

Before adding a new alerting rule into this repository you should consider creating an SLO rules instead.
SLO helps you both increase the quality of your monitoring and reduce the alert noise.


Towards: https://github.com/giantswarm/giantswarm/issues/31098
This PR creates aggretation for sloth slos so we do not have too high cardinality based on pod, node and instance labels for instance. I will try them out on a test installation first to see the load

Checklist

@QuentinBisson QuentinBisson self-assigned this Sep 4, 2024
@QuentinBisson QuentinBisson requested a review from a team as a code owner September 4, 2024 14:17
@QuentinBisson
Copy link
Contributor Author

@giantswarm/team-atlas I would think we can do without the 3d , 1d and so on and only use the 5m ones?

@QuentinBisson QuentinBisson force-pushed the send-slo-aggregations-to-grafana-cloud branch from a1f5c3b to 0c218a7 Compare September 4, 2024 14:22
Copy link
Contributor

@lyind lyind left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't review the queries in detail, but the recorded series LGTM.

I don't know for sure, why other sli_error:ratio_rate* series except aggregation:slo:sli_error:ratio_rate5m would be needed.

@QuentinBisson
Copy link
Contributor Author

It actually makes sensé, i'll remove them

Copy link
Contributor

@QuantumEnigmaa QuantumEnigmaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@QuentinBisson QuentinBisson merged commit 5f17372 into main Sep 16, 2024
7 checks passed
@QuentinBisson QuentinBisson deleted the send-slo-aggregations-to-grafana-cloud branch September 16, 2024 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants