Add ability to reload cert and key from disk.

[george-angel]

Enables frequent rotation of certificates

What and why?

This change will reload certificate + key on the listener when the files change on-disk.

Allows use of cert-manager and avoiding having hardcoded infinite TTL certificates.

Testing Steps

Started controller:

...
k8s-sidecar-injector-5c8bb965cc-98jbt k8s-sidecar-injector I0812 09:25:07.618723       1 main.go:153] configuration replaced

delete the TLS secret, let cert-manager recreate it, and try to recreate injected Pod:

k8s-sidecar-injector-5c8bb965cc-98jbt k8s-sidecar-injector 2020/08/12 09:28:29 http: TLS handshake error from 10.2.3.1:20124: remote error: tls: bad certificate

A minute later, injector controller is able to handle requests again (without restarting):

k8s-sidecar-injector-5c8bb965cc-98jbt k8s-sidecar-injector I0812 09:30:00.085404       1 webhook.go:510] AdmissionReview for Kind=/v1, Kind=Pod, Namespace=sys-prom Name=thanos-store-0 (thanos-store-0) UID=7efecd19-0f72-4422-add3-511a6308345e patchOperation=CREATE UserInfo={system:serviceaccount:kube-system:statefulset
-controller 34071325-acfd-11e7-97e1-0afa8bad3a0a [system:serviceaccounts system:serviceaccounts:kube-system system:authenticated] map[]}

Reviewers

Required reviewers: @byxorna
Request reviews from other people you want to review this PR in the "Reviewers" section on the right.

⚠️ this PR must have at least 2 thumbs from the MAINTAINERS.md of the project before merging!

[byxorna]

This looks great, thanks for this change!

Does this obsolete the changes in #46?

[george-angel]

No, this is a stand-alone change for cert reload, ignore #46 for now (although thanks for comments). I will clean it up and raise a PR with general version updates there.

[george-angel]

If you are happy with this change, we can merge it in and I'll update the other PR with just changes to modules + Docker image? Thank you.

[byxorna]

This LGTM - I cant merge it though, so @alex-laties @kirooshu @defect want to take it from here?