Harvest customized polling interval of 1 second using REST Collectors

[bheem23reddy]

For one of our largest clients, we have installed the Harvest (NABOX) for monitoring the storage environment. Currently this tool provides graphs for monitoring with the lowest sampling interval of 10 seconds. Since this is used to monitor a customer facing financial application which has a high impact even if there is a 1 second glitch on the application due to high transaction volume, we need the Harvest tool to provide the monitoring of the environment with a sampling/polling rate of per second.

If we are able to monitor the metrics at 1 sec polling interval for a period of 72 hours (Usually issues are reported during this window for RCA) then we will be able to save lot of unnecessary trouble shooting of huge performance archives which still does not provide the real cause of the issue due to large polling interval. This enables customer to look for potential storage issues themselves and saves a lot of tickets to backend support.

Kindly let us know if you need any additional details on this.

Thanks in advance,
Bheem Reddy

[rahulguptajss]

@bheem23reddy In Harvest, a customized polling interval is effective provided it's shorter than the combined time of an API call and the processing of it by Harvest. If the polling interval exceeds this combined time, it may result in a delay in metrics collection. The duration of API calls can fluctuate based on factors such as cluster response times, load on cluster and the number of objects involved.

Identifying the optimal polling interva involves some experimentation. If you notice any lags or delays in data collection, these should be investigated by examining your logs. For further assistance, please feel free to share your logs with us. You can send them at [email protected]

[cgrinds]

@bheem23reddy is this still needed?

[bheem23reddy]

@rahulguptajss we have upgraded the harvest to the latest as recommended. We need the config changes to be done on harvest and Prometheus level to poll every 5 sec interval.

please share the links to update the config files.

Thanks in advance.

regards
Bheem

[rahulguptajss]

@rahulguptajss we have upgraded the harvest to the latest as recommended. We need the config changes to be done on harvest and Prometheus level to poll every 5 sec interval.

please share the links to update the config files.

Thanks in advance.

regards Bheem

@bheem23reddy Steps are documented @ #2556

[rahulguptajss]

This issue is identical to #2559. Therefore, closing this thread and will continue the discussion there.