Add kine and runtimes search docs

Signed-off-by: Vitor Savian <[email protected]>

Address comments

Signed-off-by: Vitor Savian <[email protected]>

README.md

@@ -28,4 +28,4 @@ This command generates static content into the `build` directory and can be serv
 
 ### Deployment
 
-An automated GH action will deploy the website to [GitHub Pages](https://github.com/rancher/rke2-docs/tree/gh-pages) once a PR has been merged to `main`.
+An automated GH action will deploy the website to [GitHub Pages](https://github.com/rancher/rke2-docs/tree/gh-pages) once a PR has been merged to `main`.

docs/advanced.md

@@ -405,3 +405,26 @@ spec:
     - name: NVIDIA_DRIVER_CAPABILITIES
       value: all
 ```
+
+:::info Version Gate
+Available as of October 2024 releases: v1.28.15+rke2r1, v1.29.10+rke2r1, v1.30.6+rke2r1, v1.31.2+rke2r1.
+:::
+
+RKE2 will now use `PATH` to find alternative container runtimes, in addition to checking the default paths used by the container runtime packages. In order to use this feature, you must modify the RKE2 service's PATH environment variable to add the directories containing the container runtime binaries.
+
+It's recommended that you modify one of this two environment files:
+
+- /etc/default/rke2-server # or rke2-agent
+- /etc/sysconfig/rke2-server # or rke2-agent
+
+This example will add the `PATH` in `/etc/default/rke2-server`:
+
+```bash
+echo PATH=$PATH >> /etc/default/rke2-server
+```
+
+:::warning
+`PATH` changes should be done with care to avoid placing untrusted binaries in the path of services that run as root.
+:::
+
+

docs/backup_restore.md → docs/datastore/backup_restore.md

@@ -1,10 +1,10 @@
 ---
-title: "Etcd Backup and Restore"
+title: Backup and Restore
 ---
 
-In this section, you'll learn how to create backups of the rke2 cluster data and to restore the cluster from backup.
-
-**Note:** /var/lib/rancher/rke2 is the default data directory for rke2, it is configurable however via `data-dir` parameter.
+:::warning Backup and restore for external databases or sqlite
+Snapshots are for embedded etcd only, if you use another datastore with `datastore-endpoint` config go to [Experimental](backup_restore.md#external-db-backups-experimental)
+:::
 
 ## Creating Snapshots
 
@@ -93,7 +93,7 @@ rke2 server \
 systemctl start rke2-server
 ```
 
-6. You can continue to add new server and worker nodes to cluster per standard [RKE2 HA installation documentation](install/ha.md#3-launch-additional-server-nodes).
+6. You can continue to add new server and worker nodes to cluster per standard [RKE2 HA installation documentation](../install/ha.md#3-launch-additional-server-nodes).
 
 
 ### Other Notes on Restoring a Snapshot
@@ -170,7 +170,7 @@ rke2 server \
 
 ### Options
 
-These options can be set in the [configuration file](./install/configuration.md):
+These options can be set in the [configuration file](../install/configuration.md):
 
 | Options | Description |
 | ----------- | --------------- |
@@ -199,3 +199,31 @@ You can manually prune "scheduled" snapshots down to a smaller amount using the
 ```
 rke2 etcd-snapshot prune --name etcd-snapshot --etcd-snapshot-retention <NUM-OF-SNAPSHOTS-TO-RETAIN>
 ```
+
+## External DB Backups (Experimental)
+
+:::warning
+In addition to backing up the datastore itself, you must also back up the server token file at `/var/lib/rancher/rke2/server/token`.
+You must restore this file, or pass its value into the `token` option, when restoring from backup.
+If you do not use the same token value when restoring, the snapshot will be unusable, as the token is used to encrypt confidential data within the datastore itself.
+:::
+
+### Backup and Restore with SQLite
+
+No special commands are required to back up or restore the SQLite datastore.
+
+* To back up the SQLite datastore, take a copy of `/var/lib/rancher/rke2/server/db/`.
+* To restore the SQLite datastore, restore the contents of `/var/lib/rancher/rke2/server/db` (and the token, as discussed above).
+
+### Backup and Restore with External Datastore
+
+When an external datastore is used, backup and restore operations are handled outside of RKE2. The database administrator will need to back up the external database, or restore it from a snapshot or dump.
+
+We recommend configuring the database to take recurring snapshots.
+
+For details on taking database snapshots and restoring your database from them, refer to the official database documentation:
+
+- [Official MySQL documentation](https://dev.mysql.com/doc/refman/8.0/en/replication-snapshot-method.html)
+- [Official PostgreSQL documentation](https://www.postgresql.org/docs/8.3/backup-dump.html)
+- [Official etcd documentation](https://etcd.io/docs/latest/op-guide/recovery/)
+

docs/datastore/external.md

@@ -0,0 +1,189 @@
+---
+title: External DB
+---
+
+# Datastore options
+
+:::warning Experimental
+RKE2 officially supports Embedded etcd, other external datastores are experimental.
+:::
+
+* **Embedded [Etcd](https://etcd.io/)**  
+  Embedded Etcd is the default datastore, and will be used if no other datastore configuration is present.
+* **Embedded [SQLite](https://www.sqlite.org/index.html)**  
+  SQLite cannot be used on clusters with multiple servers.
+* **External Database**  
+  The following external datastores:
+  * [etcd](https://etcd.io/) (certified against version 3.5.4)
+  * [MySQL](https://www.mysql.com) (certified against versions 5.7 and 8.0)
+  * [MariaDB](https://mariadb.org/) (certified against version 10.6.8)
+  * [PostgreSQL](https://www.postgresql.org/) (certified against versions 12.16, 13.12, 14.9 and 15.4)
+
+:::warning Prepared Statement Support
+RKE2 requires prepared statements support from the DB. This means that connection poolers such as [PgBouncer](https://www.pgbouncer.org/faq.html#how-to-use-prepared-statements-with-transaction-pooling) may require additional configuration to work with RKE2.
+:::
+
+### SQLite Configuration Parameters
+If you wish to use SQLite as the database for your RKE2 server, you must set `disable-etcd` option without a `server` option for RKE2 to knows that it needs to disable etcd and run with SQLite.
+
+### External Datastore Configuration Parameters
+If you wish to use an external datastore such as PostgreSQL, MySQL, or etcd you must set the `datastore-endpoint` config so that RKE2 knows how to connect to it. You may also specify parameters to configure the authentication and encryption of the connection. The below table summarizes these options:
+
+| Options | Environment Variable | Description
+|---------|----------------------|------------
+| `datastore-endpoint` | `RKE2_DATASTORE_ENDPOINT` | Specify a PostgreSQL, MySQL, or etcd connection string. This is a string used to describe the connection to the datastore. The structure of this string is specific to each backend and is detailed below. |
+| `datastore-cafile` | `RKE2_DATASTORE_CAFILE` | TLS Certificate Authority (CA) file used to help secure communication with the datastore. If your datastore serves requests over TLS using a certificate signed by a custom certificate authority, you can specify that CA using this parameter so that the RKE2 client can properly verify the certificate. |
+| `datastore-certfile` | `RKE2_DATASTORE_CERTFILE` | TLS certificate file used for client certificate based authentication to your datastore. To use this feature, your datastore must be configured to support client certificate based authentication. If you specify this parameter, you must also specify the `datastore-keyfile` parameter. |
+| `datastore-keyfile` | `RKE2_DATASTORE_KEYFILE` | TLS key file used for client certificate based authentication to your datastore. See the previous `datastore-certfile` parameter for more details. |
+
+### Datastore Endpoint Format and Functionality
+As mentioned, the format of the value passed to the `datastore-endpoint` parameter is dependent upon the datastore backend. The following details this format and functionality for each supported external datastore.
+
+<Tabs queryString="ext-db">
+<TabItem value="PostgreSQL">
+
+
+  A typical `datastore-endpoint` datastore-endpoint option for PostgreSQL has the following format:
+
+  `postgres://username:password@hostname:port/database-name`
+
+  More advanced configuration parameters are available. For more information on these, please see https://godoc.org/github.com/lib/pq.
+
+  If you specify a database name and it does not exist, the server will attempt to create it.
+
+  If you only supply `postgres://` as the endpoint, RKE2 will attempt to do the following:
+
+  - Connect to localhost using `postgres` as the username and password
+  - Create a database named `kubernetes`
+
+</TabItem>
+<TabItem value="MySQL / MariaDB">
+
+  A typical `datastore-endpoint` option for MySQL and MariaDB has the following format:
+
+  `mysql://username:password@tcp(hostname:3306)/database-name`
+
+  More advanced configuration parameters are available. For more information on these, please see https://github.com/go-sql-driver/mysql#dsn-data-source-name
+
+  If you specify a database name and it does not exist, the server will attempt to create it.
+
+  If you only supply `mysql://` as the endpoint, RKE2 will attempt to do the following:
+
+  - Connect to the MySQL socket at `/var/run/mysqld/mysqld.sock` using the `root` user and no password
+  - Create a database with the name `kubernetes`
+
+</TabItem>
+
+<TabItem value="etcd">
+
+  A typical `datastore-endpoint` option for etcd has the following format:
+
+  `https://etcd-host-1:2379,https://etcd-host-2:2379,https://etcd-host-3:2379`
+
+  The above assumes a typical three node etcd cluster. The parameter can accept one more comma separated etcd URLs.
+
+</TabItem>
+</Tabs>
+
+## Single Server with SQLite
+
+### 1. Set `disable-etcd` without the `server` parameter in the config file
+
+```yaml
+disable-etcd: true
+```
+
+### 2. Install RKE2 
+```bash
+curl -sfL https://get.rke2.io | sh -
+```
+
+### 3. Enable rke2-server service
+```sh
+systemctl enable rke2-server.service
+```
+
+### 4. start the rke2-server service
+
+```sh
+systemctl start rke2-server.service
+```
+
+You can follow the server starting by `kubectl get nodes` to see the server get the `Ready` status. See [Cluster access](../cluster_access.md) for more info about how to access RKE2.
+
+## External database
+
+### 1. Create an External Datastore
+
+You will first need to create an external datastore for the cluster. See the [Datastore options](external.md#datastore-options) documentation for more details.
+
+### 2. Launch Server Nodes
+
+RKE2 requires two or more server nodes for this HA configuration. See the [Requirements](../install/requirements.md) guide for minimum machine requirements.
+
+When starting the `rke2-server` service on these nodes, you must set the `datastore-endpoint` option in the config so that RKE2 knows how to connect to the external datastore. The `token` option can also be used to set a deterministic token when adding nodes. When empty, this token will be generated automatically for further use.
+
+For example, a `config.yaml` like the following could be used to config RKE2 with a MySQL database as the external datastore and set a token:
+
+:::note 
+The RKE2 config file needs to be created manually. You can do that by running touch /etc/rancher/rke2/config.yaml as a privileged user. 
+:::
+
+```yaml
+datastore-endpoint: "mysql://username:password@tcp(hostname:3306)/database-name"
+token: SECRET
+```
+
+The datastore endpoint format differs based on the database type. For details, refer to the section on [datastore endpoint formats.](external.md#datastore-endpoint-format-and-functionality)
+
+To configure TLS certificates when launching server nodes, refer to the [datastore configuration guide.](external.md#external-datastore-configuration-parameters)
+
+By default, server nodes will be schedulable and thus your workloads can get launched on them. If you wish to have a dedicated control plane where no user workloads will run, you can use [taints](../advanced.md#node-labels-and-taints).
+
+Once you've started the `rke2-server` process on all server nodes, ensure that the cluster has come up properly with `kubectl get nodes`. You should see your server nodes in the `Ready` state.
+
+### 3. Optional: Join Additional Server Nodes
+
+The same example config in Step 2 can be used to join additional server nodes, where the token from the first node needs to be used.
+
+If the first server node was started without the `token` option, the token value can be retrieved from any server already joined to the cluster:
+
+```bash
+cat /var/lib/rancher/rke2/server/token
+```
+
+then you can install the second server with the `server` address in the config with the step 2:
+
+```yaml
+server: https://you-first-server-node-address:9345
+datastore-endpoint: "mysql://username:password@tcp(hostname:3306)/database-name"
+token: SECRET
+```
+
+There are a few config flags that must be the same in all server nodes:
+
+- Network related flags: `cluster-dns`, `cluster-domain`, `cluster-cidr`, `service-cidr`
+- Flags controlling the deployment of certain components: `disable-helm-controller` and any component passed to `disable`
+- Feature related flags: `secrets-encryption`
+
+:::note
+Ensure that you retain a copy of this token as it is required when restoring from backup and adding nodes.
+:::
+
+### 4. Optional: Join Agent Nodes
+
+Because RKE2 server nodes are schedulable by default, agent nodes are not required for a RKE2 cluster. However, you may wish to have dedicated agent nodes to run your apps and services.
+
+You just need to specify the URL the agent should register to (either one of the server IPs or a fixed registration address) and the token it should use in the `config` file.
+
+```yaml
+server: https://you-first-server-node-address:9345
+token: SECRET
+```
+
+and then you can install the agent:
+
+```bash
+curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE="agent" sh -
+```
+

docs/install/ha.md

@@ -106,14 +106,14 @@ Once you've launched the `rke2 server` process on all server nodes, ensure that
 You should see your server nodes in the Ready state.
 
 :::note
-By default, any `kubectl` command will require root user access, unless `RKE2_KUBECONFIG_MODE` override is provided. Read more about it in [cluster access page](https://docs.rke2.io/cluster_access).
+By default, any `kubectl` command will require root user access, unless `RKE2_KUBECONFIG_MODE` override is provided. Read more about it in [cluster access page](../cluster_access.md).
 :::
 
 ### 5. Optional: Join Agent Nodes
 
 Because RKE2 server nodes are schedulable by default, the minimum number of nodes for an HA RKE2 server cluster is three server nodes and zero agent nodes. To add nodes designated to run your apps and services, join agent nodes to your cluster.
 
-Joining agent nodes in an HA cluster is the same as [joining agent nodes in a single server cluster](quickstart.md#linux-agent-worker-node-installation). You just need to specify the URL the agent should register to and the token it should use.
+Joining agent nodes in an HA cluster is the same as [joining agent nodes in a single server cluster](../install/quickstart.md#linux-agent-worker-node-installation). You just need to specify the URL the agent should register to and the token it should use.
 
 ```yaml
 server: https://my-kubernetes-domain.com:9345

docs/install/quickstart.md

@@ -57,7 +57,7 @@ After running this installation:
 * A token that can be used to register other server or agent nodes will be created at `/var/lib/rancher/rke2/server/node-token`
 
 :::note
-If you are adding additional server nodes, you must have an odd number in total. An odd number is needed to maintain quorum. See the [High Availability documentation](./ha.md) for more details.
+If you are adding additional server nodes, you must have an odd number in total. An odd number is needed to maintain quorum. See the [High Availability documentation](ha.md) for more details.
 :::
 
 ### Linux Agent (Worker) Node Installation

docs/security/secrets_encryption.md

@@ -55,7 +55,7 @@ RKE2 contains a subcommand `secrets-encrypt`, which allows administrators to per
 - Reencrypting secrets
 
 :::danger
-Failure to follow proper procedure when rotating secrets encryption keys can cause permanent data loss. [Creating a snapshot](../backup_restore.md) before rotating is recommended. Proceed with caution.
+Failure to follow proper procedure when rotating secrets encryption keys can cause permanent data loss. [Creating a snapshot](../datastore/backup_restore.md) before rotating is recommended. Proceed with caution.
 :::
 
 <!-- ### New Encryption Key Rotation

i18n/zh/docusaurus-plugin-content-docs/current/backup_restore.md

@@ -95,7 +95,7 @@ rke2 server \
 systemctl start rke2-server
 ```
 
-6. 你可以根据标准 [RKE2 HA 安装文档](install/ha.md#3-启动其他-server-节点)继续向集群添加新的 Server 和 Worker 节点。
+6. 你可以根据标准 [RKE2 HA 安装文档](../install/ha.md#3-启动其他-server-节点)继续向集群添加新的 Server 和 Worker 节点。
 
 
 ### 恢复快照的其他注意事项

i18n/zh/docusaurus-plugin-content-docs/current/install/quickstart.md

@@ -64,7 +64,7 @@ journalctl -u rke2-server -f
 * 可用于注册其他 Server 或 Agent 节点的令牌将在 `/var/lib/rancher/rke2/server/node-token` 中创建。
 
 :::note
-如果要添加其他 Server 节点，则总数必须为奇数。仲裁要求节点数为奇数。有关详细信息，请参阅[高可用文档](./ha.md)。
+如果要添加其他 Server 节点，则总数必须为奇数。仲裁要求节点数为奇数。有关详细信息，请参阅[高可用文档](ha.md)。
 :::
 
 ### Linux Agent（Worker）节点安装

i18n/zh/docusaurus-plugin-content-docs/current/security/secrets_encryption.md

@@ -55,7 +55,7 @@ RKE2 包含一个实用的子命令 `secrets-encrypt`，它允许管理员执行
 - 重新加密 Secret
 
 :::danger
-如果你在轮换 Secret 加密密钥时没有遵循正确的程序，数据可能永久丢失。建议在轮换之前[创建快照](../backup_restore.md)。因此，请谨慎操作。
+如果你在轮换 Secret 加密密钥时没有遵循正确的程序，数据可能永久丢失。建议在轮换之前[创建快照](../datastore/backup_restore.md)。因此，请谨慎操作。
 :::
 
 ### 单 Server 加密密钥轮换

sidebars.js

@@ -8,10 +8,10 @@ module.exports = {
         'install/quickstart',
         'install/requirements',
         'install/configuration',
-        'install/ha',
         'install/methods',
         'install/containerd_registry_configuration',
         'install/airgap',
+        'install/ha',
         'install/windows_airgap',
         'install/server_roles',
         'install/uninstall',
@@ -44,9 +44,16 @@ module.exports = {
         'security/token',
       ],
     },
+    {
+      type: 'category',
+      label: 'Datastore',
+      items: [
+        'datastore/external',
+        'datastore/backup_restore',
+      ],
+    },
     'architecture',
     'cluster_access',
-    'backup_restore',
     {
       type: 'category',
       label: 'Networking',
@@ -80,4 +87,4 @@ module.exports = {
     },
     'known_issues',
   ],
-};
+};