Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modifying logic to provide metrics per MINIO endpoint #83

Merged
merged 5 commits into from
Nov 28, 2023
Merged

Modifying logic to provide metrics per MINIO endpoint #83

Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
59a7a34
Modifying logic to provide metrics per MINIO endpoint
nsarras Oct 30, 2023
ebe061c
Adding logging with rotation after reaching 100MB. Improving exceptio…
nsarras Nov 7, 2023
c3101bf
Added logging, refactored agent to be multi-threaded vs multi-process…
nsarras Nov 21, 2023
c1ebfaa
Switching logging level
nsarras Nov 21, 2023
1bddd1f
Refactoring to remove duplicate lines of code
nsarras Nov 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 22 additions & 18 deletions dss_metrics/minio_rest_collector.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,24 +64,28 @@ def poll_statistics(self, metrics_data_buffer, exit_flag):
if not endpts or len(endpts) == 0:
nsarras marked this conversation as resolved.
Show resolved Hide resolved
continue

minio_endpoint = endpts.pop()
miniocluster_id = self.get_minio_cluster_uuid(minio_endpoint)
minio_metrics = self.get_minio_metrics_from_endpoint(
minio_endpoint)

tags = {}
tags['cluster_id'] = self.cluster_id
tags['target_id'] = socket.gethostname()
tags['minio_id'] = miniocluster_id
tags['type'] = self.TYPE

for metric in minio_metrics:
if self.filter and not self.check_whitelist_key(metric[0]):
continue
metrics_data_buffer.append(
metrics.MetricInfo(
metric[0], metric[0], metric[1], tags, time.time())
)
for minio_endpoint in endpts:
miniocluster_id = self.get_minio_cluster_uuid(
minio_endpoint)
minio_metrics = self.get_minio_metrics_from_endpoint(
minio_endpoint)

tags = {}
tags['cluster_id'] = self.cluster_id
tags['target_id'] = socket.gethostname()
nsarras marked this conversation as resolved.
Show resolved Hide resolved
tags['minio_id'] = miniocluster_id
tags['minio_endpoint'] = minio_endpoint
tags['type'] = self.TYPE

for metric in minio_metrics:
if (self.filter and not
self.check_whitelist_key(metric[0])):
continue
metrics_data_buffer.append(
metrics.MetricInfo(
metric[0], metric[0], metric[1], tags,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why pass metrics selectively?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is because we may not want to expose all metrics to the end user all of the time, hence why we filter using a whitelist if the filter flag is given

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, why not metrics.MetricInfo(metric, tags..) ? Let MetricInfo filter out.
metrics.MetricInfo(metric[0], metric[0], metric[1], tags, doen't really help readability, else its better to use named params.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See new commits on PR, I made a lot of changes and not using this approach anymore

time.time())
)

nsarras marked this conversation as resolved.
Show resolved Hide resolved
def check_whitelist_key(self, key):
for regex in self.whitelist_patterns:
Expand Down