Record memory usage per scraping job (#1156)

* record memory usage per scraping job Signed-off-by: QuentinBisson <[email protected]> * Rename --------- Signed-off-by: QuentinBisson <[email protected]>
giantswarm · May 14, 2024 · 9707e0e · 9707e0e
1 parent a929cf5
commit 9707e0e
Show file tree

Hide file tree

Showing 2 changed files with 16 additions and 0 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- Add recording rules to show prometheus scraping job memory usage.
 - Add `cluster_control_plane_unhealthy` inhibition.
 - Added inhibitions expressions for CAPI clusters.
 - make targets for pint linter

diff --git a/...rometheus-rules/templates/recording-rules/monitoring.resource-usage-estimation.rules.yaml b/...rometheus-rules/templates/recording-rules/monitoring.resource-usage-estimation.rules.yaml
@@ -0,0 +1,15 @@
+apiVersion: monitoring.coreos.com/v1
+kind: PrometheusRule
+metadata:
+  labels:
+    {{- include "labels.common" . | nindent 4 }}
+  name: monitoring.resource-usage-estimation.recording.rules
+  namespace: {{ .Values.namespace  }}
+spec:
+  groups:
+  - name: monitoring.resource-usage-estimation.recording
+    rules:
+    - expr: (count({__name__=~".+"}) by (cluster_id, job) / on(cluster_id) group_left prometheus_tsdb_head_series) * on(cluster_id) group_left sum(container_memory_usage_bytes{container="prometheus"}) by (cluster_id)
+      record: giantswarm:observability:monitoring:resource_usage_estimation:memory_usage_bytes
+    - expr: (count({__name__=~".+"}) by (cluster_id, job) / on(cluster_id) group_left prometheus_tsdb_head_series) * on(cluster_id) group_left sum(container_memory_working_set_bytes{container="prometheus"}) by (cluster_id)
+      record: giantswarm:observability:monitoring:resource_usage_estimation:memory_working_set_bytes