Permissions required to collect metrics?

[ehrenfeu]

Hi all,

following a conversation with Claire Stoffel we were trying to incorporate more metrics into our Prometheus/Grafana setup, especially regarding data size.

Claire was referring to the image.sc thread on OMERO storage reports that gives some starting points on how to query OMERO for the necessary information.

Only when trying to add these queries to etc/prometheus-omero-counts.yml it is when I realized that my previous approach of simply adding a new user and group (that doesn't have any special permissions or other group memberships) for running the OMERO prometheus exporter doesn't work.

Digging deeper and using that said user on the command line to run an HQL query:

/opt/omero/server/venv3/bin/omero hql --all "
SELECT
    details.group.name, details.owner.omeName, SUM(size)
FROM
    OriginalFile
WHERE
    size > 0
GROUP BY
    details.group.name, details.owner.omeName
ORDER BY
    details.group.name, details.owner.omeName
"

results in something like this:

 #  | Col1 | Col2     | Col3       
----+------+----------+------------
 0  | user | root     | 15247652   
 1  | user | <hidden> | 82258      
 2  | user | <hidden> | 4776772955 
 3  | user | <hidden> | 193684     
 4  | user | <hidden> | 695107     
 5  | user | <hidden> | 65259      
 6  | user | <hidden> | 14505      
 7  | user | <hidden> | 176237     
 8  | user | <hidden> | 163619     
 9  | user | <hidden> | 135735     
 10 | user | <hidden> | 188946     
 11 | user | <hidden> | 184515     
 12 | user | <hidden> | 186883     
 13 | user | <hidden> | 10557

Adjusting the user to be an "Administrator with restricted privileges" results in the group and username column being filled correctly.

The question is now: is this the way to go? My feeling is that having something more "read-only" would be a cleaner solution, but I'm not sure how to achieve this.

Cheers,
Niko

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/omero-storage-reports/41819/11

[sbesson]

@ehrenfeu thanks for opening this issue. Yes, upgrading the user permissions is required as the moment for some of the queries. This is also something we have in our production deployments where the monitoring user needs to be either a restricted or full admin to make cross-group queries and access information in compliance with OMERO's permissions system.

I agree with you that it would be great to work towards a more native/read-only integration between the server and Prometheus. Looking quickly at the existing integrations, one technical solution might be to develop and deploy a OMERO Prometheus micro-service that would expose metrics in a Prometheus compatible format.

[ehrenfeu]

Thanks @sbesson - having a microservice for this sounds like the way of choice. Now we only need to wait until it manifests itself on github somehow I guess 😝

[ehrenfeu]

Turning around this question, is there a template / example somewhere on how to create an OMERO microservice?