Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add default partition when no matching labelset liimits #6435

Merged
merged 6 commits into from
Dec 18, 2024

Conversation

yeya24
Copy link
Contributor

@yeya24 yeya24 commented Dec 17, 2024

What this PR does:

We added per labelset limits in Per LabelSet, which allows users to define subtenants within a user by defining labelsets and we can apply limits such as active series to throttle the usage of each subtenant.

There is a scenario where team label is used to define subtenants for a user. However, there might be a lot of teams and it is not easy to set limits for every team explicitly. Instead, users can define labelsets limits for some of the large teams and use put rest of teams using a catch all limit. This is called the default partition, which means it matches series that doesn't match any of the existing labelset limits.

Default partition doesn't have any label defined. In the example below, the second limit is the default partition and it should match all series that with name != prometheus_build_info.

    limits_per_label_set:
      - limits:
          max_series: 10000
        label_set:
          __name__: prometheus_build_info
      - limits:
          max_series: 10000
        label_set:

Metrics for default partition looks like below.

cortex_ingester_limits_per_labelset{labelset="{}",limit="max_series",user="1"} 2
cortex_ingester_usage_per_labelset{labelset="{}",limit="max_series",user="1"} 2

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

CHANGELOG.md Show resolved Hide resolved
totalCount++
func getCardinalityForLimitsPerLabelSet(ctx context.Context, numSeries uint64, ir tsdb.IndexReader, allLimits []validation.LimitsPerLabelSet, limit validation.LimitsPerLabelSet) (int, error) {
// Easy path with explicit labels.
if limit.LabelSet.Len() > 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we document in the label set config that a empty labelset is the "default" and will match all other series?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in the latest commit

Signed-off-by: Ben Ye <[email protected]>
Signed-off-by: Ben Ye <[email protected]>
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Dec 18, 2024
@yeya24 yeya24 merged commit 27d68c2 into cortexproject:master Dec 18, 2024
17 checks passed
@yeya24 yeya24 deleted the default-partition branch December 18, 2024 01:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/ingester lgtm This PR has been approved by a maintainer size/XL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants