From 1e1985e9a732a2a5d522254454da32709ddfb78b Mon Sep 17 00:00:00 2001 From: Derek Nola Date: Thu, 12 Dec 2024 09:48:59 -0800 Subject: [PATCH] Update Server and Agent CLI pages (#362) * Update agent help page Signed-off-by: Derek Nola * Update server help page Signed-off-by: Derek Nola --------- Signed-off-by: Derek Nola --- docs/cli/agent.md | 28 ++++++++++++++++++++-- docs/cli/server.md | 59 ++++++++++++++++++++++++++++++---------------- 2 files changed, 65 insertions(+), 22 deletions(-) diff --git a/docs/cli/agent.md b/docs/cli/agent.md index 2b4bb7b28..7ac434322 100644 --- a/docs/cli/agent.md +++ b/docs/cli/agent.md @@ -27,6 +27,12 @@ Options are documented on this page as CLI flags, but can also be passed as conf | `--token-file` value | `K3S_TOKEN_FILE` | Token file to use for authentication | | `--server value, -s` value | `K3S_URL` | Server to connect to | +### Listener + +| Flag | Default | Description | +| --- | --- | --- | +| `--bind-address` | 0.0.0.0 | k3s bind address | + ### Data | Flag | Default | Description | @@ -49,7 +55,9 @@ Options are documented on this page as CLI flags, but can also be passed as conf | Flag | Default | Description | | ------------------------------------ | ---------------------------------- | ------------------------------------------------------------------ | -| `--container-runtime-endpoint` value | N/A | Disable embedded containerd and use the CRI socket at the given path; when used with --docker this sets the cri-docker socket path | +| `--container-runtime-endpoint` | N/A | Disable embedded containerd and use the CRI socket at the given path; when used with --docker this sets the docker socket path | +| `--default-runtime` | N/A | Set the default runtime in containerd | +| `--image-service-endpoint` | N/A | Disable embedded containerd image service and use remote image service socket at the given path. If not specified, defaults to --container-runtime-endpoint. | | `--pause-image` value | "docker.io/rancher/pause:3.1" | Customized pause image for containerd or docker sandbox | | `--private-registry` value | "/etc/rancher/k3s/registries.yaml" | Private registry configuration file | @@ -59,6 +67,8 @@ Options are documented on this page as CLI flags, but can also be passed as conf | --------------------------- | -------------------- | ----------------------------------------- | | `--node-ip value, -i` value | N/A | IP address to advertise for node | | `--node-external-ip` value | N/A | External IP address to advertise for node | +| `--node-internal-dns` | N/A | internal DNS addresses to advertise for node | +| `--node-external-dns` | N/A | external DNS addresses to advertise for node | | `--resolv-conf` value | `K3S_RESOLV_CONF` | Kubelet resolv.conf file | | `--flannel-iface` value | N/A | Override default flannel interface | | `--flannel-conf` value | N/A | Override default flannel config file | @@ -77,8 +87,11 @@ Options are documented on this page as CLI flags, but can also be passed as conf | ------------ | ------------------------------------- | | `--rootless` | Run rootless | | `--docker` | Use cri-dockerd instead of containerd | +| `--enable-pprof` | Enable pprof endpoint on supervisor port | | `--prefer-bundled-bin` | Prefer bundled userspace binaries over host binaries | | `--disable-default-registry-endpoint` | See "[Default Endpoint Fallback](../installation/private-registry.md#default-endpoint-fallback)" +| `--vpn-auth` | See "[Integration with the Tailscale VPN provider](../networking/distributed-multicloud.md#integration-with-the-tailscale-vpn-provider-experimental)" | +| `--vpn-auth-file` | See "[Integration with the Tailscale VPN provider](../networking/distributed-multicloud.md#integration-with-the-tailscale-vpn-provider-experimental)" | ### Deprecated @@ -122,7 +135,7 @@ OPTIONS: --token value, -t value (cluster) Token to use for authentication [$K3S_TOKEN] --token-file value (cluster) Token file to use for authentication [$K3S_TOKEN_FILE] --server value, -s value (cluster) Server to connect to [$K3S_URL] - --data-dir value, -d value (agent/data) Folder to hold state (default: "/var/lib/rancher/k3s") + --data-dir value, -d value (agent/data) Folder to hold state (default: "/var/lib/rancher/k3s") [$K3S_DATA_DIR] --node-name value (agent/node) Node name [$K3S_NODE_NAME] --with-node-id (agent/node) Append id to node name --node-label value (agent/node) Registering and starting kubelet with set of labels @@ -133,18 +146,29 @@ OPTIONS: --lb-server-port value (agent/node) Local port for supervisor client load-balancer. If the supervisor and apiserver are not colocated an additional port 1 less than this port will also be used for the apiserver client load-balancer. (default: 6444) [$K3S_LB_SERVER_PORT] --protect-kernel-defaults (agent/node) Kernel tuning behavior. If set, error if kernel tunables are different than kubelet defaults. --container-runtime-endpoint value (agent/runtime) Disable embedded containerd and use the CRI socket at the given path; when used with --docker this sets the docker socket path + --default-runtime value (agent/runtime) Set the default runtime in containerd + --image-service-endpoint value (agent/runtime) Disable embedded containerd image service and use remote image service socket at the given path. If not specified, defaults to --container-runtime-endpoint. --pause-image value (agent/runtime) Customized pause image for containerd or docker sandbox (default: "rancher/mirrored-pause:3.6") --snapshotter value (agent/runtime) Override default containerd snapshotter (default: "overlayfs") --private-registry value (agent/runtime) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml") + --disable-default-registry-endpoint (agent/containerd) Disables containerd fallback default registry endpoint when a mirror is configured for that registry + --nonroot-devices (agent/containerd) Allows non-root pods to access devices by setting device_ownership_from_security_context=true in the containerd CRI config --node-ip value, -i value (agent/networking) IPv4/IPv6 addresses to advertise for node + --bind-address value (listener) k3s bind address (default: 0.0.0.0) --node-external-ip value (agent/networking) IPv4/IPv6 external IP addresses to advertise for node + --node-internal-dns value (agent/networking) internal DNS addresses to advertise for node + --node-external-dns value (agent/networking) external DNS addresses to advertise for node --resolv-conf value (agent/networking) Kubelet resolv.conf file [$K3S_RESOLV_CONF] --flannel-iface value (agent/networking) Override default flannel interface --flannel-conf value (agent/networking) Override default flannel config file --flannel-cni-conf value (agent/networking) Override default flannel cni config file --kubelet-arg value (agent/flags) Customized flag for kubelet process --kube-proxy-arg value (agent/flags) Customized flag for kube-proxy process + --enable-pprof (experimental) Enable pprof endpoint on supervisor port --rootless (experimental) Run rootless --prefer-bundled-bin (experimental) Prefer bundled userspace binaries over host binaries --docker (agent/runtime) (experimental) Use cri-dockerd instead of containerd + --vpn-auth value (agent/networking) (experimental) Credentials for the VPN provider. It must include the provider name and join key in the format name=,joinKey=[,controlServerURL=][,extraArgs=] [$K3S_VPN_AUTH] + --vpn-auth-file value (agent/networking) (experimental) File containing credentials for the VPN provider. It must include the provider name and join key in the format name=,joinKey=[,controlServerURL=][,extraArgs=] [$K3S_VPN_AUTH_FILE] + --disable-apiserver-lb (agent/networking) (experimental) Disable the agent client-side load-balancer and connect directly to the configured server address ``` diff --git a/docs/cli/server.md b/docs/cli/server.md index 76ef29124..3e41a5864 100644 --- a/docs/cli/server.md +++ b/docs/cli/server.md @@ -36,7 +36,7 @@ The following options must be set to the same value on all servers in the cluste | Flag | Environment Variable | Default | Description | |---------------------------------------|--------------------------|----------------------------------------|-------------------------------------------------------------------------------| -| `--datastore-endpoint` value | `K3S_DATASTORE_ENDPOINT` | | Specify etcd, Mysql, Postgres, or Sqlite data source name | +| `--datastore-endpoint` value | `K3S_DATASTORE_ENDPOINT` | | Specify etcd, NATS, MySQL, Postgres, or SQLite data source name | | `--datastore-cafile` value | `K3S_DATASTORE_CAFILE` | | TLS Certificate Authority file used to secure datastore backend communication | | `--datastore-certfile` value | `K3S_DATASTORE_CERTFILE` | | TLS certification file used to secure datastore backend communication | | `--datastore-keyfile` value | `K3S_DATASTORE_KEYFILE` | | TLS key file used to secure datastore backend communication | @@ -55,6 +55,8 @@ The following options must be set to the same value on all servers in the cluste | `--etcd-s3-bucket` value | | | S3 bucket name | | `--etcd-s3-region` value | | "us-east-1" | S3 region / bucket location (optional) | | `--etcd-s3-folder` value | | | S3 folder | +| `--etcd-s3-proxy` | | | Proxy server to use when connecting to S3, overriding any proxy-releated environment variables | +| `--etcd-s3-config-secret` | | | Name of secret in the kube-system namespace used to configure S3, if etcd-s3 is enabled and no other etcd-s3 options are set | | `--etcd-s3-insecure` | | | Disables S3 over HTTPS | | `--etcd-s3-timeout` value | | 5m0s | S3 timeout (default: 5m0s) | @@ -102,6 +104,7 @@ The following options must be set to the same value on all servers in the cluste | `--advertise-address` value | node-external-ip/node-ip | IPv4/IPv6 address that apiserver advertises for its service endpoint
Note that the primary `service-cidr` IP range must be of the same address family as the advertised address | | `--advertise-port` value | listen-port/0 | Port that apiserver uses to advertise to members of the cluster | | `--tls-san` value | N/A | Add additional hostnames or IPv4/IPv6 addresses as Subject Alternative Names on the TLS cert | +| `--tls-san-security` | true | Protect the server TLS cert by refusing to add Subject Alternative Names not associated with the kubernetes apiserver service, server nodes, or values of the tls-san option | ### Data @@ -118,17 +121,17 @@ The following options must be set to the same value on all servers in the cluste ### Networking -| Flag | Default | Description | -|-----------------------------------|-----------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `--cluster-cidr` value | "10.42.0.0/16" | IPv4/IPv6 network CIDRs to use for pod IPs | -| `--service-cidr` value | "10.43.0.0/16" | IPv4/IPv6 network CIDRs to use for service IPs | -| `--service-node-port-range` value | "30000-32767" | Port range to reserve for services with NodePort visibility | -| `--cluster-dns` value | "10.43.0.10" | IPv4 Cluster IP for coredns service. Should be in your service-cidr range | -| `--cluster-domain` value | "cluster.local" | Cluster Domain | -| `--flannel-backend` value | "vxlan" | One of 'none', 'vxlan', 'ipsec'(deprecated), 'host-gw', 'wireguard-native', or 'wireguard'(deprecated) | -| `--flannel-ipv6-masq` | "N/A" | Enable IPv6 masquerading for pod | -| `--flannel-external-ip` | "N/A" | Use node external IP addresses for Flannel traffic | -| `--servicelb-namespace` value | "kube-system" | Namespace of the pods for the servicelb component | +| Flag | Default | Description | +|-----------------------------------|-----------------|--------------| +| `--cluster-cidr` value | "10.42.0.0/16" | IPv4/IPv6 network CIDRs to use for pod IPs | +| `--service-cidr` value | "10.43.0.0/16" | IPv4/IPv6 network CIDRs to use for service | +| `--service-node-port-range` value | "30000-32767" | Port range to reserve for services with NodePort visibility | +| `--cluster-dns` value | "10.43.0.10" | IPv4 Cluster IP for coredns service. Should be in your service-cidr range | +| `--cluster-domain` value | "cluster.local" | Cluster Domain | +| `--flannel-backend` value | "vxlan" | One of 'none', 'vxlan', 'ipsec'(deprecated), 'host-gw', 'wireguard-native', or 'wireguard'(deprecated) | +| `--flannel-ipv6-masq` | "N/A" | Enable IPv6 masquerading for pod | +| `--flannel-external-ip` | "N/A" | Use node external IP addresses for Flannel traffic | +| `--servicelb-namespace` value | "kube-system" | Namespace of the pods for the servicelb component | | `--egress-selector-mode` value | "agent" | Must be one of the following:
  • disabled: The apiserver does not use agent tunnels to communicate with nodes. Requires that servers run agents, and have direct connectivity to the kubelet on agents, or the apiserver will not be able to function access service endpoints or perform kubectl exec and kubectl logs.
  • agent: The apiserver uses agent tunnels to communicate with nodes. Nodes allow the tunnel connection from loopback addresses. Requires that servers also run agents, or the apiserver will not be able to access service endpoints. The historical default for k3s.
  • pod: The apiserver uses agent tunnels to communicate with nodes and service endpoints, routing endpoint connections to the correct agent by watching Nodes. Nodes allow the tunnel connection from loopback addresses, or a CIDR assigned to their node.
  • cluster: The apiserver uses agent tunnels to communicate with nodes and service endpoints, routing endpoint connections to the correct agent by watching Endpoints. Nodes allow the tunnel connection from loopback addresses, or the configured cluster CIDR range.
| @@ -170,9 +173,10 @@ The following options must be set to the same value on all servers in the cluste | `--enable-pprof` | Enable pprof endpoint on supervisor port | | `--docker` | Use cri-dockerd instead of containerd | | `--prefer-bundled-bin` | Prefer bundled userspace binaries over host binaries | -| `--disable-agent` | See "[Running Agentless Servers (Experimental)](../advanced.md#running-agentless-servers-experimental)" | +| `--disable-agent` | See "[Running Agentless Servers](../advanced.md#running-agentless-servers-experimental)" | | `--embedded-registry` | See "[Embedded Registry Mirror](../installation/registry-mirror.md)" | - +| `--vpn-auth` | See "[Integration with the Tailscale VPN provider](../networking/distributed-multicloud.md#integration-with-the-tailscale-vpn-provider-experimental)" | +| `--vpn-auth-file` | See "[Integration with the Tailscale VPN provider](../networking/distributed-multicloud.md#integration-with-the-tailscale-vpn-provider-experimental)" | ### Deprecated Options @@ -205,22 +209,25 @@ OPTIONS: --alsologtostderr (logging) Log to standard error as well as file (if set) --bind-address value (listener) k3s bind address (default: 0.0.0.0) --https-listen-port value (listener) HTTPS listen port (default: 6443) - --advertise-address value (listener) IPv4 address that apiserver uses to advertise to members of the cluster (default: node-external-ip/node-ip) + --advertise-address value (listener) IPv4/IPv6 address that apiserver uses to advertise to members of the cluster (default: node-external-ip/node-ip) --advertise-port value (listener) Port that apiserver uses to advertise to members of the cluster (default: listen-port) (default: 0) --tls-san value (listener) Add additional hostnames or IPv4/IPv6 addresses as Subject Alternative Names on the server TLS cert - --data-dir value, -d value (data) Folder to hold state (default: /var/lib/rancher/k3s or ${HOME}/.rancher/k3s if not root) + --tls-san-security (listener) Protect the server TLS cert by refusing to add Subject Alternative Names not associated with the kubernetes apiserver service, server nodes, or values of the tls-san option (default: true) + --data-dir value, -d value (data) Folder to hold state default /var/lib/rancher/k3s or ${HOME}/.rancher/k3s if not root [$K3S_DATA_DIR] --cluster-cidr value (networking) IPv4/IPv6 network CIDRs to use for pod IPs (default: 10.42.0.0/16) --service-cidr value (networking) IPv4/IPv6 network CIDRs to use for service IPs (default: 10.43.0.0/16) --service-node-port-range value (networking) Port range to reserve for services with NodePort visibility (default: "30000-32767") --cluster-dns value (networking) IPv4 Cluster IP for coredns service. Should be in your service-cidr range (default: 10.43.0.10) --cluster-domain value (networking) Cluster Domain (default: "cluster.local") - --flannel-backend value (networking) backend<=option1=val1,option2=val2> where backend is one of 'none', 'vxlan', 'ipsec' (deprecated), 'host-gw', 'wireguard-native', 'wireguard' (deprecated) (default: "vxlan") + --flannel-backend value (networking) Backend (valid values: 'none', 'vxlan', 'host-gw', 'wireguard-native' (default: "vxlan") --flannel-ipv6-masq (networking) Enable IPv6 masquerading for pod --flannel-external-ip (networking) Use node external IP addresses for Flannel traffic --egress-selector-mode value (networking) One of 'agent', 'cluster', 'pod', 'disabled' (default: "agent") --servicelb-namespace value (networking) Namespace of the pods for the servicelb component (default: "kube-system") --write-kubeconfig value, -o value (client) Write kubeconfig for admin client to this file [$K3S_KUBECONFIG_OUTPUT] --write-kubeconfig-mode value (client) Write kubeconfig with this mode [$K3S_KUBECONFIG_MODE] + --write-kubeconfig-group value (client) Write kubeconfig with this group [$K3S_KUBECONFIG_GROUP] + --helm-job-image value (helm) Default image to use for helm jobs --token value, -t value (cluster) Shared secret used to join a server or agent to a cluster [$K3S_TOKEN] --token-file value (cluster) File containing the token [$K3S_TOKEN_FILE] --agent-token value (cluster) Shared secret used to join agents to the cluster, but not servers [$K3S_AGENT_TOKEN] @@ -234,14 +241,14 @@ OPTIONS: --kube-controller-manager-arg value (flags) Customized flag for kube-controller-manager process --kube-scheduler-arg value (flags) Customized flag for kube-scheduler process --kube-cloud-controller-manager-arg value (flags) Customized flag for kube-cloud-controller-manager process - --datastore-endpoint value (db) Specify etcd, Mysql, Postgres, or Sqlite (default) data source name [$K3S_DATASTORE_ENDPOINT] + --datastore-endpoint value (db) Specify etcd, NATS, MySQL, Postgres, or SQLite (default) data source name [$K3S_DATASTORE_ENDPOINT] --datastore-cafile value (db) TLS Certificate Authority file used to secure datastore backend communication [$K3S_DATASTORE_CAFILE] --datastore-certfile value (db) TLS certification file used to secure datastore backend communication [$K3S_DATASTORE_CERTFILE] --datastore-keyfile value (db) TLS key file used to secure datastore backend communication [$K3S_DATASTORE_KEYFILE] --etcd-expose-metrics (db) Expose etcd metrics to client interface. (default: false) --etcd-disable-snapshots (db) Disable automatic etcd snapshots --etcd-snapshot-name value (db) Set the base name of etcd snapshots (default: etcd-snapshot-) (default: "etcd-snapshot") - --etcd-snapshot-schedule-cron value (db) Snapshot interval time in cron spec. eg. every 5 hours '* */5 * * *' (default: "0 */12 * * *") + --etcd-snapshot-schedule-cron value (db) Snapshot interval time in cron spec. eg. every 5 hours '0 */5 * * *' (default: "0 */12 * * *") --etcd-snapshot-retention value (db) Number of snapshots to retain (default: 5) --etcd-snapshot-dir value (db) Directory to save db snapshots. (default: ${data-dir}/db/snapshots) --etcd-snapshot-compress (db) Compress etcd snapshot @@ -254,15 +261,19 @@ OPTIONS: --etcd-s3-bucket value (db) S3 bucket name --etcd-s3-region value (db) S3 region / bucket location (optional) (default: "us-east-1") --etcd-s3-folder value (db) S3 folder + --etcd-s3-proxy value (db) Proxy server to use when connecting to S3, overriding any proxy-releated environment variables + --etcd-s3-config-secret value (db) Name of secret in the kube-system namespace used to configure S3, if etcd-s3 is enabled and no other etcd-s3 options are set --etcd-s3-insecure (db) Disables S3 over HTTPS --etcd-s3-timeout value (db) S3 timeout (default: 5m0s) --default-local-storage-path value (storage) Default local storage path for local provisioner storage class - --disable value (components) Do not deploy packaged components and delete any deployed components (valid items: coredns, servicelb, traefik, local-storage, metrics-server) + --disable value (components) Do not deploy packaged components and delete any deployed components (valid items: coredns, servicelb, traefik, local-storage, metrics-server, runtimes) --disable-scheduler (components) Disable Kubernetes default scheduler --disable-cloud-controller (components) Disable k3s default cloud controller manager --disable-kube-proxy (components) Disable running kube-proxy --disable-network-policy (components) Disable k3s default network policy controller --disable-helm-controller (components) Disable Helm controller + --embedded-registry (experimental/components) Enable embedded distributed container registry; requires use of embedded containerd; when enabled agents will also listen on the supervisor port + --supervisor-metrics (experimental/components) Enable serving k3s internal metrics on the supervisor port; when enabled agents will also listen on the supervisor port --node-name value (agent/node) Node name [$K3S_NODE_NAME] --with-node-id (agent/node) Append id to node name --node-label value (agent/node) Registering and starting kubelet with set of labels @@ -271,16 +282,24 @@ OPTIONS: --image-credential-provider-config value (agent/node) The path to the credential provider plugin config file (default: "/var/lib/rancher/credentialprovider/config.yaml") --docker (agent/runtime) (experimental) Use cri-dockerd instead of containerd --container-runtime-endpoint value (agent/runtime) Disable embedded containerd and use the CRI socket at the given path; when used with --docker this sets the docker socket path + --default-runtime value (agent/runtime) Set the default runtime in containerd + --image-service-endpoint value (agent/runtime) Disable embedded containerd image service and use remote image service socket at the given path. If not specified, defaults to --container-runtime-endpoint. + --disable-default-registry-endpoint (agent/containerd) Disables containerd fallback default registry endpoint when a mirror is configured for that registry + --nonroot-devices (agent/containerd) Allows non-root pods to access devices by setting device_ownership_from_security_context=true in the containerd CRI config --pause-image value (agent/runtime) Customized pause image for containerd or docker sandbox (default: "rancher/mirrored-pause:3.6") --snapshotter value (agent/runtime) Override default containerd snapshotter (default: "overlayfs") --private-registry value (agent/runtime) Private registry configuration file (default: "/etc/rancher/k3s/registries.yaml") --system-default-registry value (agent/runtime) Private registry to be used for all system images [$K3S_SYSTEM_DEFAULT_REGISTRY] --node-ip value, -i value (agent/networking) IPv4/IPv6 addresses to advertise for node --node-external-ip value (agent/networking) IPv4/IPv6 external IP addresses to advertise for node + --node-internal-dns value (agent/networking) internal DNS addresses to advertise for node + --node-external-dns value (agent/networking) external DNS addresses to advertise for node --resolv-conf value (agent/networking) Kubelet resolv.conf file [$K3S_RESOLV_CONF] --flannel-iface value (agent/networking) Override default flannel interface --flannel-conf value (agent/networking) Override default flannel config file --flannel-cni-conf value (agent/networking) Override default flannel cni config file + --vpn-auth value (agent/networking) (experimental) Credentials for the VPN provider. It must include the provider name and join key in the format name=,joinKey=[,controlServerURL=][,extraArgs=] [$K3S_VPN_AUTH] + --vpn-auth-file value (agent/networking) (experimental) File containing credentials for the VPN provider. It must include the provider name and join key in the format name=,joinKey=[,controlServerURL=][,extraArgs=] [$K3S_VPN_AUTH_FILE] --kubelet-arg value (agent/flags) Customized flag for kubelet process --kube-proxy-arg value (agent/flags) Customized flag for kube-proxy process --protect-kernel-defaults (agent/node) Kernel tuning behavior. If set, error if kernel tunables are different than kubelet defaults.