-
Notifications
You must be signed in to change notification settings - Fork 248
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Dumps and snapshots on Meilisearch Cloud (#2729)
--------- Co-authored-by: Tamo <[email protected]>
- Loading branch information
1 parent
e23ad8e
commit d1b8b9f
Showing
8 changed files
with
85 additions
and
100 deletions.
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,86 +1,56 @@ | ||
| --- | ||
| title: Snapshots — Meilisearch documentation | ||
| title: Exporting and using Snapshots — Meilisearch documentation | ||
| description: Snapshots are exact copies of Meilisearch databases. They are often useful for periodical backups. | ||
| --- | ||
|
|
||
| # Snapshots | ||
| # Exporting and using snapshots | ||
|
|
||
| A snapshot is an exact copy of the Meilisearch database, located by default in `./data.ms`. Snapshots are useful as quick backups. | ||
| A [snapshot](/learn/advanced/snapshots_vs_dumps#snapshots) is an exact copy of the Meilisearch database. Snapshots are useful as quick backups, but cannot be used to migrate to a new Meilisearch release. | ||
|
|
||
| Since databases are bound to the Meilisearch version that created them and snapshots are direct copies of the database, you cannot use snapshots to migrate data when upgrading Meilisearch. | ||
| This tutorial shows you how to schedule snapshot creation to ensure you always have a recent backup of your instance ready to use. You will also see how to start Meilisearch from this snapshot. | ||
|
|
||
| ## Creating snapshots | ||
| <Capsule intent="warning" title="Meilisearch Cloud and snapshots"> | ||
| Meilisearch Cloud does not support snapshots. | ||
| </Capsule> | ||
|
|
||
| ### Creating a single snapshot | ||
| ## Scheduling periodic snapshots | ||
|
|
||
| Use [the `/snapshots` route](/reference/api/snapshots) to generate a single snapshot: | ||
|
|
||
| <CodeSamples id="create_snapshot_1" /> | ||
|
|
||
| This will return a summarized task object. Use [the `/tasks` route](/reference/api/tasks) to monitor the status of this operation. | ||
|
|
||
| ### Scheduling periodic snapshots | ||
| It is good practice to create regular backups of your Meilisearch data. This ensures that you can recover from critical failures quickly in case your Meilisearch instance becomes compromised. | ||
|
|
||
| Use the [`--schedule-snapshot` configuration option](/learn/configuration/instance_options#schedule-snapshot-creation) to create snapshots at regular time intervals: | ||
|
|
||
| ```bash | ||
| meilisearch --schedule-snapshot | ||
| ``` | ||
|
|
||
| The first snapshot is created on launching Meilisearch. After that, snapshots are created on a set interval until you deactivate snapshots by ending the Meilisearch instance. By default, one snapshot is taken every 24 hours. | ||
| The first snapshot is created on launch. You will find it in the [snapshot directory](/learn/configuration/instance_options#snapshot-destination), `/snapshots`. Meilisearch will then create a new snapshot every 24 hours until you terminate your instance. | ||
|
|
||
| Modify the interval between each new snapshot by providing an integer to [`--schedule-snapshot`](/learn/configuration/instance_options#schedule-snapshot-creation): | ||
| Meilisearch **automatically overwrites** old snapshots during snapshot creation. Only the most recent snapshot will be present in the folder at any given time. | ||
|
|
||
| In cases where your database is updated several times a day, it might be better to modify the interval between each new snapshot: | ||
|
|
||
| ```bash | ||
| meilisearch --schedule-snapshot=3600 | ||
| ``` | ||
|
|
||
| [Learn more about snapshots flags and environment variables.](/learn/configuration/instance_options#schedule-snapshot-creation) | ||
|
|
||
| ### Snapshot directory | ||
|
|
||
| By default, Meilisearch creates snapshots in a directory called `snapshots/` at the root of your project. | ||
|
|
||
| The destination can be modified with [`--snapshot-dir`](/learn/configuration/instance_options#snapshot-destination): | ||
|
|
||
| ```bash | ||
| meilisearch --schedule-snapshot --snapshot-dir mySnapShots/ | ||
| ``` | ||
| This instructs Meilisearch to create a new snapshot once every hour. | ||
|
|
||
| Old snapshots are **automatically overwritten** during snapshot creation. Only the most recent snapshot will be present in the folder at any given time. | ||
| <Capsule intent="tip"> | ||
| If you need to generate a single snapshot without relaunching your instance, use [the `/snapshots` route](/reference/api/snapshots). | ||
| </Capsule> | ||
|
|
||
| ## Starting from a snapshot | ||
|
|
||
| Because snapshots are exact copies of your database that haven't gone through any processing besides compression, starting a Meilisearch instance from a snapshot is significantly faster than adding documents manually or starting from a dump. | ||
|
|
||
| Using the CLI flag `--import-snapshot`, Meilisearch will start the server using the provided snapshot. | ||
| To import snapshot data into your instance, launch Meilisearch using `--import-snapshot`: | ||
|
|
||
| ```bash | ||
| meilisearch --import-snapshot mySnapShots/data.ms.snapshot | ||
| ``` | ||
|
|
||
| ## Common problems | ||
|
|
||
| Take note that whenever you launch Meilisearch from a snapshot, it will **stop processing and throw an error** if it encounters either of the two following situations: | ||
|
|
||
| 1. A database already exists, meaning you have a non-empty `data.ms` folder in the same directory as your Meilisearch binary | ||
| 2. No snapshot is found at the given path | ||
|
|
||
| In both cases, **this behavior is [configurable](/learn/configuration/instance_options#ignore-missing-snapshot)**. | ||
|
|
||
| If you don't want Meilisearch to throw an error when finding that a database already exists, you can add the following flag: `--ignore-snapshot-if-db-exists=true`. When using this flag, Meilisearch will use the existing database to start an instance instead of throwing an error. The snapshot will be ignored. | ||
|
|
||
| If you do not want Meilisearch to throw an error when there is no snapshot at the given path, you can add the following flag: `--ignore-missing-snapshot`. Meilisearch will then continue its process and not import any snapshot. | ||
|
|
||
| When starting from a snapshot, chances are that you already have an existing database. **For security reasons, a database is never overwritten**. To load a snapshot when an existing database is present, you will have to manually delete the existing database. By default, this is the contents of the `data.ms` folder (unless you [changed the path](/learn/configuration/instance_options#database-path)) which is located in the same folder as your Meilisearch binary. | ||
| The simplest way to delete your database is with the terminal command `rm -rf data.ms`, after which you should be able to start Meilisearch with a snapshot. | ||
|
|
||
| [[More about snapshots flags and environment variables]](/learn/configuration/instance_options#schedule-snapshot-creation) | ||
|
|
||
| ## Use cases | ||
|
|
||
| **Snapshots are safeguards in case of problems**. If your Meilisearch instance encounters a problem or if you make a mistake while manipulating your database, restarting your instance with the latest snapshot is an easy way to recover your data. | ||
| Because snapshots are exact copies of your database, starting a Meilisearch instance from a snapshot is much faster than adding documents manually or starting from a dump. | ||
|
|
||
| ### Version compatibility | ||
| <Capsule intent="warning"> | ||
| For security reasons, Meilisearch will never overwrite an existing database. By default, Meilisearch will throw an error when importing a snapshot if there is any data in your instance. | ||
|
|
||
| Since a snapshot is an exact replica of your database, it can only be opened by the same version of Meilisearch that created it. | ||
| You can change this behavior by specifying [`--ignore-snapshot-if-db-exists=true`](/learn/configuration/instance_options#ignore-dump-if-db-exists). This will cause Meilisearch to launch with the existing database and ignore the dump without throwing an error. | ||
| </Capsule> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,37 +1,38 @@ | ||
| --- | ||
| title: Snapshots vs dumps — Meilisearch documentation | ||
| title: Snapshots and dumps — Meilisearch documentation | ||
| description: "Meilisearch offers two types of backups: snapshots and dumps. Snapshots are mainly intended as a safeguard, while dumps are useful when migrating Meilisearch." | ||
| --- | ||
|
|
||
| # Data backup: Snapshots vs dumps | ||
| # Snapshots and dumps | ||
|
|
||
| Meilisearch has two ways to backup its data: `snapshots` and `dumps`. | ||
| This article explains Meilisearch's two backup methods: snapshots and dumps. | ||
|
|
||
| ## Snapshots | ||
|
|
||
| Snapshots make it possible to schedule the creation of hard copies of your database. These copies are bound to a specific Meilisearch version. | ||
| A snapshot is an exact copy of the Meilisearch database, located by default in `./data.ms`. [Use snapshots for quick and efficient backups of your instance](/learn/advanced/snapshots). | ||
|
|
||
| This feature is **intended mainly as a safeguard**: if something goes wrong in an instance, you're able to relaunch your database quickly and efficiently. | ||
| The documents in a snapshot are already indexed and ready to go, greatly increasing import speed. However, snapshots are not compatible between different versions of Meilisearch. Snapshots are also significantly bigger than dumps. | ||
|
|
||
| The documents in a snapshot are already indexed and ready to go, greatly increasing import speed. However, **snapshots are not compatible between different versions of Meilisearch**. | ||
| In short, snapshots are a safeguard: if something goes wrong in an instance, you're able to recover and relaunch your database quickly. You can also schedule periodic snapshot creation. | ||
|
|
||
| ## Dumps | ||
|
|
||
| Dumps export Meilisearch data in a way that is not bound to a specific Meilisearch version. This means **dumps are ideal for migrating your data when you upgrade Meilisearch.** | ||
| A dump isn't an exact copy of your database like a snapshot. Instead, it is closer to a blueprint which Meilisearch can later use to recreate a whole instance from scratch. | ||
|
|
||
| **Importing a dump requires Meilisearch to re-index all your documents.** This process requires a significant amount of time and memory proportional to the size of the database. | ||
| Importing a dump requires Meilisearch to re-index all documents. This process uses a significant amount of time and memory proportional to the size of the database. Compared to the snapshots, importing a dump is a slow and inefficient operation. | ||
|
|
||
| At the same time, dumps are not bound to a specific Meilisearch version. This means dumps are ideal for migrating your data when you upgrade Meilisearch. | ||
|
|
||
| <Capsule intent="note"> | ||
| We do not recommend using dumps from a new Meilisearch version to import an older version. | ||
| Use dumps to transfer data from an old Meilisearch version into a more recent release. Do not transfer data from a new release into a legacy Meilisearch version. | ||
|
|
||
| For example, you can import a dump from Meilisearch v0.21 into v0.22 without any problems. Importing a dump generated in v0.22 into a v0.21 instance, however, can lead to unexpected behavior. | ||
| For example, you can import a dump from Meilisearch v1.2 into v1.6 without any problems. Importing a dump generated in v1.7 into a v1.2 instance, however, can lead to unexpected behavior. | ||
| </Capsule> | ||
|
|
||
| ## Conclusion | ||
| ## Snapshots VS dumps | ||
|
|
||
| To summarize: | ||
| Both snapshots and dumps are data backups, but they serve different purposes. | ||
|
|
||
| - Snapshots are highly efficient, but not portable between different versions of Meilisearch | ||
| - Dumps are portable between different Meilisearch versions, but not very efficient | ||
| Snapshots are highly efficient, but not portable between different versions of Meilisearch. **Use snapshots for periodic data backups.** | ||
|
|
||
| For more information, have a look at the documentation guides for [snapshots](/learn/advanced/snapshots) and [dumps](/learn/advanced/dumps). | ||
| Dumps are portable between different Meilisearch versions, but not very efficient. **Use dumps when updating to a new Meilisearch release.** |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.