Getting StatsD to work in OnlyOffice Kubernetes/Docker 8.2

[torsten-simon]

Do you want to request a feature or report a bug?

Bug

What is the current behavior?

StatsD / Metrics not working

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem.

• installing OnlyOffice Docserver 8.2 in a Kubernetes based enviroment following the steps from https://github.com/ONLYOFFICE/Kubernetes-Docs?tab=readme-ov-file
• Add helm config values

metrics:
  enabled: true
log:
  level: ALL

In addition to that, since I had issues with the METRICS_ENABLED env in previous versions (7.5):
Add a secret for local-production-linux.json with contents of

    {
        "statsd": {
                "useMetrics": true,
                "host": "statsd-exporter-prometheus-statsd-exporter",
                "port": "8125",
                "prefix": "ds."
        }
    }

(I can confirm that the docserver is parsing this file since it fails with an error on startup if I put in a syntax error on purpose.)

What is the expected behavior?

Pushing metrics to statsd in cluster, seeing log messages like Flushing stats at

Did this work in previous versions of DocumentServer?
Yes, last tested on a docker based install with onlyoffice/documentserver-ee:7.5

DocumentServer Docker tag:
onlyoffice/docs-docservice-ee:8.2.0-1 (also tried with "de" variant)
Helm chart version 4.3.0

Host Operating System:
Kubernetes Rancher / Ubuntu Worker

[igwyd]

Hello @torsten-simon, metrics generally work, I checked it on the latest Helm release v4.4.0 with Grafana, but there is a issue #71746 with some of them - no data in Authirizations, Get lock and Saving Changes, do you have the same?

[torsten-simon]

Hi @igwyd thanks for taking a look!

Unfortunately, the only stats I'm able to retrieve come from the statsd exporter metrics itself.

Could you confirm that there should be log messages when stats are flushed by the docserver (I've set the log level to ALL)? Because, I can't see any in 8.2 but could see them in 7.5

We're mostly interested in the editing & viewing stats (ds_expireDoc_connections_edit & ds_expireDoc_connections_view) to analyze the current usage to verify if the license is "large" enough.

[igwyd]

Maybe it becase you add your local-productin-linux.json, I didn't do it, only change metrics.enabled: true and install statsd.

[torsten-simon]

Hi @igwyd , I tried both with metrics.enabled: true as well as local-production-linux.json, in both cases I don't see any log messages regarding pushed stats. (metrics.enabled: true also dont has an affect on the value in the regular json config files, altough I'm not sure if this is correct behaviour)

[igwyd]

Can you show env for docservice process? cat /proc/7/environ (7 is PID docservice)

There will be a statsd object and what is specified in it will have priority over what is passed in the config.
The values ​​in env are taken from here.

PS i dont know about metrics logging in k8s yet, but i will let you know.

[torsten-simon]

Hi @igwyd, thanks for comming back and sorry for the dealy.

In my case, the metrics config shows up as expected as well

Hope that helps to further investigate it

[igwyd]

This is fine. let's see kubectl get svc, is there statsd-exporter-prometheus-statsd-exporter here?

kubectl get pods, is there statsd-exporter-prometheus-statsd-exporter-xxxxx-xxxxx and prometheus-server-xxxxx-xxxxx, are they running?

PS attach screenshots please.

[torsten-simon]

Hi @igwyd ,

kubectl svc output:

NAME                                         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                 AGE
prometheus-kube-state-metrics                ClusterIP   10.43.47.156    <none>        8080/TCP                                11m
prometheus-prometheus-node-exporter          ClusterIP   10.43.189.117   <none>        9100/TCP                                11m
prometheus-prometheus-pushgateway            ClusterIP   10.43.26.11     <none>        9091/TCP                                11m
prometheus-server                            ClusterIP   10.43.149.57    <none>        80/TCP                                  11m
redis-headless                               ClusterIP   None            <none>        6379/TCP                                28d
redis-master                                 ClusterIP   10.43.100.110   <none>        6379/TCP                                28d
statsd-exporter-prometheus-statsd-exporter   ClusterIP   10.43.164.118   <none>        9102/TCP,8126/TCP,8125/UDP              27d

kubectl get pods:

NAME                                                          READY   STATUS    RESTARTS   AGE
prometheus-kube-state-metrics-7df7665df9-8pbcz                1/1     Running   0          12m
prometheus-prometheus-node-exporter-2g8jk                     1/1     Running   0          12m
prometheus-prometheus-node-exporter-47q5t                     1/1     Running   0          12m
prometheus-prometheus-node-exporter-6crpd                     1/1     Running   0          12m
prometheus-prometheus-node-exporter-74xj6                     1/1     Running   0          12m
prometheus-prometheus-node-exporter-8x5tl                     1/1     Running   0          12m
prometheus-prometheus-node-exporter-fs99w                     1/1     Running   0          12m
prometheus-prometheus-node-exporter-fz44q                     1/1     Running   0          12m
prometheus-prometheus-node-exporter-hzwcw                     1/1     Running   0          12m
prometheus-prometheus-node-exporter-jgtdr                     1/1     Running   0          12m
prometheus-prometheus-node-exporter-kgp4q                     1/1     Running   0          12m
prometheus-prometheus-node-exporter-lvfqk                     1/1     Running   0          12m
prometheus-prometheus-node-exporter-plwfl                     1/1     Running   0          12m
prometheus-prometheus-node-exporter-qhgpp                     1/1     Running   0          12m
prometheus-prometheus-node-exporter-w5sjq                     1/1     Running   0          12m
prometheus-prometheus-node-exporter-w7w8m                     1/1     Running   0          12m
prometheus-prometheus-pushgateway-6f8f9dd848-2l7kq            1/1     Running   0          12m
prometheus-server-6977d97578-rhv76                            2/2     Running   0          2m2s
redis-master-0                                                1/1     Running   0          28d
statsd-exporter-prometheus-statsd-exporter-6c8587bf54-ptzhg   1/1     Running   0          27d

Everything is running
Rancher Screenshot (Pods)

[igwyd]

Open the document, edit it and after a while go to some pod where there is curl and execute curl statsd-exporter-prometheus-statsd-exporter:9102/metrics | grep -i ds_. Are there any ds metrics? For example like these