Simplify k3s vm-based deployment by enabling write-only ssh-less 1st server setup. (super-token?)

[cameronelliott]

So, I'm working with Terraform to create and launch k3s clusters on VMs. Really digging k3s, by the way.

Anyway, because the kubeconfig is created by the first server, a back-channel from the first k3s server
created to the cluster-launching-computer must be made to retrieve the kubeconfig for usage.
This is usually done with ssh.

These types of clusters are often stood up & destroyed without any end-user ssh usage or connections, if possible.

So, in this scenario, the sole purpose of creating an SSH key, and configuring providing an ssh login method is merely to retrieve the kubeconfig for kubectl usage,etc.

If it was possible to provide the initial kubeconfig rather than --token for the 1st server stood up,
the need for the ssh backchannel goes away for many users, and simple VM creation with cloud-init can make Terraform (and similar) style launching of k3s clusters quicker and easier. Since configuration would be push-only.

cloud-init style installation of k3s may also be faster than ssh based installs, as install happens right when server is ready after boot, whereas ssh type installs against a new, booting server are effectively polling to attempt ssh login against a booting server. If ssh retry delays are using a back-off delay, the ssh login for k3s install could be delayed much later than a cloud-init install.

A tool or flag (--generate-kubeconfig) would be needed to craft the initial kubeconfig prior to launching
the k3s cluster.

Anyway, I wanted to throw this idea out there and get feedback on whether others see the merit in this.
I see others #3487 have questions about how to get the kubeconfig from outside the primary server, or from the token also.

@brandond

[cameronelliott]

It may be the case that a kubeconfig cannot be the entropy or data needed to initialize the 1st server, for practical reasons.
But if there was a "super-token" from which a kubeconfig could be generated and the 1st server configured, this would mean creating k3s clusters would be a write-only operation to a set of virtual machines on a cloud-provider. Along with a tool or special flag (--super-token-to-kubeconfig) that could convert the "super-token" to a kubeconfig.

I believe this would dramatically simplify the lives of dev-ops persons using k3s in the field, by not needing to setup ssh on their deployed k3s server sets, in order to retrieve the kubeconfig.

[cameronelliott]

I wonder if I could effect the same push-only initial-server type of model (without needing to retrieve the kubeconfig) by creating and pushing the certificates to the initial, 1st server?

[brandond]

The kubeconfig contains a certificate signed by the cluster CA that it uses to authenticate to the cluster. You can't pre-create this, because the cluster CAs don't exist until you start the cluster the first time.

If you wanted, you could probably create your own cluster CAs, place those on the host before starting K3s the first time, and then sign and generate your own admin kubeconfig certificates without having to pull anything off the host.

See the discussions at:

• How to configure my own CA for k3s ? #1868 (comment)
• Certificate and access management for edge computing #2342 (comment)