Target Labels are "dropped"

[orelhinhas]

Hi,

after deployed this Prometheus, I tried to monitor my web apps and rabbitmq, but after following all documentation when I open Prometheus UI - Service Discovery all my "Target Labels" are dropped.

This scenario occurs only when I set up other apps, the k8s cluster monitoring is OK.

I tried even the "example-app" in "Getting Started" documentation but the symptom is the same.

Thanks in advance

[camilb]

Hi @orelhinhas, did you create a ServiceMonitor to monitor additional apps?

Should look like this:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: rabbitmq
  namespace: monitoring
  labels:
    k8s-app: rabbitmq
spec:
  selector:
    matchLabels:
      k8s-app: rabbitmq
  endpoints:
  - port: exporter
    interval: 10s
  namespaceSelector:
    matchNames:
    - default

Here is an example for RabbitMQ deployment

apiVersion: v1
kind: Service
metadata:
  name: rabbitmq
  labels:
    k8s-app: rabbitmq
spec:
  ports:
  - name: rabbitmq
    port: 5672
    #targetPort: 15672
    protocol: TCP
  - name: rabbitmq1
    port: 15672
    protocol: TCP
  - name: exporter
    port: 9090
    targetPort: exporter
    protocol: TCP
  selector:
    k8s-app: rabbitmq
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
 name: rezoomex-rabbitmq-deployment
 labels:
  app: queue
spec:
 replicas: 1
 template:
  metadata:
   labels:
    k8s-app: rabbitmq
  spec:
   containers:
    - name: rabbitmq
      image: rabbitmq:3.6.11-management-alpine
      env:
        - name: RABBITMQ_DEFAULT_USER
          value: user
        - name: RABBITMQ_DEFAULT_PASS
          value: password
      ports:
        - name: rabbitmq
          containerPort: 5672
          protocol: TCP
        - name: rabbitmq1
          containerPort: 15672
          protocol: TCP
    - name: rabbitmq-exporter
      image: kbudde/rabbitmq-exporter
      ports:
        - name: exporter
          containerPort: 9090
      env:
        - name: RABBIT_USER
          value: user
        - name: RABBIT_PASSWORD
          value: password
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: rabbitmq
  namespace: monitoring
  labels:
    k8s-app: rabbitmq
spec:
  selector:
    matchLabels:
      k8s-app: rabbitmq
  endpoints:
  - port: exporter
    interval: 10s
  namespaceSelector:
    matchNames:
    - default