Add support for podman metrics in docker module

[MichaelKatsoulis]

Proposed commit message

• WHAT: Enhance docker module so that podman metrics are collected and calculated correctly
• WHY: support for podman metrics in docker module

Details

Podman offers a docker compatible API for metrics collection. So using docker module to collect podman metrics should be working out of the box.
In reality the memory metrics were not published at all and cpu usage percentage calculation was incorrect.
The reason is that the podman api returns zero values for precpu_stats. These stats refer to the latest's read cpu statistics, needed for cpu percentage calculation. Also due to these stats being zero, the memory metrics were not populated because of a sanity check in the code.

beats/metricbeat/module/docker/memory/helper.go

Line 55 in 3f51793

if containerStats.Stats.MemoryStats.Limit == 0 || containerStats.Stats.PreCPUStats.CPUUsage.TotalUsage == 0 {

For docker, the precpu_stats are returned every time.
The solution in the Podman's case is to stream the api response. By default we had set the stream option to false as there was no need in case of docker.

As part of this PR we introduce a new configuration parameter named podman (by default false). If set to true, indicating that podman is used, then the stream parameter is set to true, only for the collection of cpu and memory stats.
The reason is that for cpu and memory stats the precpu_stats are needed.
From the streamed response, we get the second response as it was noticed during testing that the precpu_stats of the first response were incorrect and the cpu was miscalculated.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

There is no disruptive user impact.

How to test this PR locally

1. start podman machine (In macOS use podman desktop)
2. start an nginx test container podman run -d -p 8080:80 --name mynginx nginx
3. exec into mynginx pod and start a cpu intensive task:
   a. podman exec -ti mynginx bash
   b. while true; do :; done &
4. run metricbeat locally with docker module enabled and podman parameter set to true
5. watch the docker.* metrics being published to ES with correct values. Compare with podman stats command results.

Related issues

• Closes https://github.com/elastic/enhancements/issues/22801

Use cases

Screenshots

Podman Stats command (check for mynginx container)

[botelastic]

This pull request doesn't have a Team:<team> label.

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @MichaelKatsoulis? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit

[mergify]

backport-8.x has been added to help with the transition to the new branch 8.x.
If you don't need it please use backport-skip label and remove the backport-8.x label.

[gizas]

Suggested change

	# Docker module supports metrics collection from podman's docker compatible API. In case of podman set to true.
	# Docker module supports metrics collection from podman's restful API. In case of podman set to true.

[gizas]

Wondering if we should protect ourselves and verify somewhere the minimum version of podman we support or at least we tested with?

I see https://podman.io/blogs/2020/07/01/rest-versioning.html that api introduced in versions > v2.0
and probably in our tests we used the the v2 API version(inn versions >4.0)

[gizas]

I guess this compatibility section needs to be updated: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-module-docker.html#_compatibility_16

[gizas]

The relevant docker integration should also be updated with the new variable podman, right?