add a local run tool for dev purposes

[hervenicol]

What this PR does / why we need it

This should improve dev experience, so we don't have to wait for the CI build.

Checklist

• Update changelog in CHANGELOG.md.
• Follow deployment test procedure in the tests/manual_e2e directory and have a working branch.

[TheoBrigitte]

I love it <3 <3

[TheoBrigitte]

I like the idea but running outside the cluster makes internal network not reachable, for instance I can't test my Alertmanager controller because it needs to talk to mimir-alertmanager-headless.mimir.svc

[TheoBrigitte]

Would you be fine to try a different approach where we replace the docker image in the deployment in order to run it in the cluster ?

[hervenicol]

Would you be fine to try a different approach where we replace the docker image in the deployment in order to run it in the cluster ?

That requires extra steps:

• build the image
• tag the image
• login on the registry
• push the image to a registry
• update the image tag in the deployment

I'm not sure these extra steps are worth it (especially authenticating on the registry, and making sure we change the image tag - which then implies doing some cleanup for old images).

If your use case is the alertmanager access, why not doing it with a port forward like grafana access?

[TheoBrigitte]

We could use the $USER name to tag the image which make it easy to clean it up afterward.
But this solution might not work because the app would still be reconciled by chart operator eventually and override the docker image "maybe".

Port forward would not work as I cannot resolve mimir-alertmanager-headless.mimir.svc locally; I had a deeper look into this yesterday and there's no way to reach out to CoreDNS as port forward can't expose UDP protocol. Even then k8s network would not be reachable locally, that just does not work.

[hervenicol]

All right, I see we have the same behaviour / issue with grafana. Somehow I just checked that the operator was starting and running, but didn't check its actual behaviour and error messages 🤦

I think we should have these parameters (dependencies URLs/ports) as user-definable parameters.
Not only to solve the "how to run the operator locally" question, but also because it's very "us-and-today-specific" to say things like "grafana will run in the monitoring namespace" and "alertmanager will run in the mimir namespace". These should not be hardcoded.

[hervenicol]

So, now it's working, and you can add the alertmanager setup when available @TheoBrigitte .