Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Severity is not calculated correctly, when metrics are absent #146

Open
err404r opened this issue Jul 2, 2024 · 1 comment
Open

Severity is not calculated correctly, when metrics are absent #146

err404r opened this issue Jul 2, 2024 · 1 comment

Comments

@err404r
Copy link

err404r commented Jul 2, 2024

Bug Description

In #98 severity was introduced to metrics rules.

However there is a small bug in them. When metrics is absent over time. the value became equal to one. Which means severity became warning.

And this shouldn't be like that. We could easily miss that node goes down. Because usually we are not sending warnigs to pagerduty

To Reproduce

  1. Deploy cos proxy
  2. Put unit down
  3. Alert in AlertManager will have warning severity

Environment

Relevant log output

-

Additional context

No response

@lucabello
Copy link
Contributor

We should adjust the way we calculate the severity for that rule.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants