Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Metrics][Perf] Move BucketLookup tree to AggregatorStore to reduce memory usage #5976

Open
utpilla opened this issue Nov 13, 2024 · 0 comments
Labels
enhancement New feature or request needs-triage New issues which have not been classified or triaged by a community member pkg:OpenTelemetry Issues related to OpenTelemetry NuGet package

Comments

@utpilla
Copy link
Contributor

utpilla commented Nov 13, 2024

Package

OpenTelemetry

Is your feature request related to a problem?

No response

What is the expected behavior?

Currently, we switch to using binary search for Histogram measurements if the bounds count >= 50.

Binary search has been implemented using a custom binary search tree. Here's the code for reference. This leads to allocating memory for the tree for each MetricPoint. This could be moved to a more central place such as AggregatorStore as the Histogram bounds would always stay the same for a given Metric. That way all MetricPoints could share the same tree structure.

We have cases where people use very high cardinality Histogram metrics. This would greatly benefit such users. Check #5416 for reference. This user has a Histogram metric with a cardinality of around 600M! We could greatly reduce memory usage in such scenarios by avoiding the repeated creation of these trees.

Which alternative solutions or features have you considered?

N/A

Additional context

No response

@utpilla utpilla added enhancement New feature or request needs-triage New issues which have not been classified or triaged by a community member labels Nov 13, 2024
@github-actions github-actions bot added the pkg:OpenTelemetry Issues related to OpenTelemetry NuGet package label Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request needs-triage New issues which have not been classified or triaged by a community member pkg:OpenTelemetry Issues related to OpenTelemetry NuGet package
Projects
None yet
Development

No branches or pull requests

1 participant