[8.16](backport #6011) Emit Pod data only for running Pods in the Kubernetes provider

[mergify]

What does this PR do?

The kubernetes provider emits data each time a Pod gets updated, even if the Pod is not yet running. As a result, every time a new Pod is spawned, we get multiple updates as it is created, scheduled, containers are created, and so on. Instead, emit the data only if the Pod is actually running.

Why is it important?

Configuration reloading can be quite expensive when there are a lot of Pods on the Node. We should avoid doing so unnecessarily. This change should help #5835 and #5991.

Note that in principle this exposes us to a new failure mode. It's possible for the Pod to successfully finish running and be removed before we push the new configuration to beats. This was much less likely when we included the Pod when it was scheduled, but still possible with our 100ms debounce timer in the coordinator. I think the tradeoff is worth it, considering the issues with config reloading on large Nodes.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool

How to test this PR locally

Deploy the agent in a local kind cluster using either the default manifests or the Helm Chart.

Related issues

• Relates Elastic agent uses too much memory per Pod in k8s #5835
• Relates Memory Consumption Issue with Elastic Agent on Kubernetes with high number of resources #5991

---

This is an automatic backport of pull request #6011 done by [Mergify](https://mergify.com).

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[elastic-sonarqube]

Quality Gate failed

Failed conditions
0.0% Coverage on New Code (required ≥ 40%)

See analysis details on SonarQube