From 2f5be480eaa088028e2c5de2a8d54c3224118d5d Mon Sep 17 00:00:00 2001 From: Jay Clifford <45856600+Jayclifford345@users.noreply.github.com> Date: Tue, 15 Oct 2024 14:49:39 +0100 Subject: [PATCH] docs: Sandbox play with Mimir (#9247) * Updated to use admonition * sandbox play with mimir * added image back. Removed banner * fixed formating issues * Update docs/sources/mimir/get-started/play-with-grafana-mimir/index.md Co-authored-by: Taylor C <41653732+tacole02@users.noreply.github.com> * Update docs/sources/mimir/get-started/play-with-grafana-mimir/index.md Co-authored-by: Taylor C <41653732+tacole02@users.noreply.github.com> * Update docs/sources/mimir/get-started/play-with-grafana-mimir/index.md Co-authored-by: Taylor C <41653732+tacole02@users.noreply.github.com> * Update docs/sources/mimir/get-started/play-with-grafana-mimir/index.md Co-authored-by: Jack Baldry --------- Co-authored-by: Taylor C <41653732+tacole02@users.noreply.github.com> Co-authored-by: Jack Baldry --- .../play-with-grafana-mimir/index.md | 96 +++++++++++++++---- 1 file changed, 80 insertions(+), 16 deletions(-) diff --git a/docs/sources/mimir/get-started/play-with-grafana-mimir/index.md b/docs/sources/mimir/get-started/play-with-grafana-mimir/index.md index 2b3b7201d57..15c8e74324c 100644 --- a/docs/sources/mimir/get-started/play-with-grafana-mimir/index.md +++ b/docs/sources/mimir/get-started/play-with-grafana-mimir/index.md @@ -24,8 +24,22 @@ keywords: menuTitle: Play with Mimir title: Play with Mimir weight: 10 +killercoda: + title: Play with Mimir + description: Learn about Grafana Mimir, which provides distributed, horizontally scalable, and highly available long term storage for Prometheus. + details: + intro: + foreground: "docker-compose-update.sh" + preprocessing: + substitutions: + - regexp: "tutorial-architecture\\.png" + replacement: https://grafana.com/media/docs/mimir/tutorial-architecture.png + backend: + imageid: ubuntu --- + + # Play with Mimir Grafana Mimir is a distributed, horizontally scalable, and highly available long term storage for [Prometheus](https://prometheus.io). @@ -37,12 +51,27 @@ In this tutorial, you'll: - Run Grafana to explore Grafana Mimir dashboards - Configure a testing recording rule and alert in Grafana Mimir + + ## Prerequisites - Git - [Docker](https://docs.docker.com/) and [Docker Compose](https://docs.docker.com/compose/) - Availability of both ports `9000` and `9009` on your host machine +{{< admonition type="tip" >}} +Alternatively, you can try out this example in our interactive learning environment: [Play with Mimir](https://killercoda.com/grafana-labs/course/mimir/play-with-mimir). + +It's a fully configured environment with all dependencies already installed. + +{{< /admonition >}} + + + + + + + ## Download tutorial configuration 1. Create a copy of the Grafana Mimir repository using the Git command line: @@ -55,14 +84,20 @@ In this tutorial, you'll: cd docs/sources/mimir/get-started/play-with-grafana-mimir/ ``` -**Note**: the instructions in this tutorial assume that your working directory is `docs/sources/mimir/get-started/play-with-grafana-mimir/`. +{{< admonition type="note" >}} +The instructions in this tutorial assume that your working directory is `docs/sources/mimir/get-started/play-with-grafana-mimir/`. +{{< /admonition >}} + + + + ## Start Grafana Mimir and dependencies Start running your local setup with the following Docker command: ```bash -docker-compose up -d +docker compose up -d ``` This command starts: @@ -90,8 +125,18 @@ The following ports will be exposed on the host: To learn more about the Grafana Mimir configuration, you can review the configuration file `config/mimir.yaml`. + + + + ## Explore Grafana Mimir dashboards +{{< docs/ignore >}} +{{< admonition type="note" >}} +Sandbox users: If you're using the interactive learning environment, you can access all links directly by clicking on them. This will redirect you to the VM's localhost where the services are running. +{{< /admonition >}} +{{< /docs/ignore >}} + Open Grafana on your local host [`http://localhost:9000`](http://localhost:9000) and view dashboards showing the status and health of your Grafana Mimir cluster. The dashboards query Grafana Mimir for the metrics they display. @@ -109,6 +154,10 @@ A couple of caveats: The dashboards installed in the Grafana are taken from the Grafana Mimir mixin which packages up Grafana Labs' best practice dashboards, recording rules, and alerts for monitoring Grafana Mimir. To learn more about the mixin, check out the Grafana Mimir mixin documentation. To learn more about how Grafana is connecting to Grafana Mimir, review the [Mimir datasource](http://localhost:9000/datasources). + + + + ## Configure your first recording rule Recording rules allow you to precompute frequently needed or computationally expensive expressions and save their result @@ -118,9 +167,8 @@ offered by Grafana. 1. Open [Grafana Alerting](http://localhost:9000/alerting/list). 2. Click **New recording rule**, which also allows you to configure recording rules. 3. Configure the recording rule: - 1. Select **Mimir or Loki recording rule** in the top selector. + 1. Give the rule a name, such as `sum:up`. 1. Choose **Mimir** in the **Select data source** field. - 1. Type `sum:up` in the **Rule name** field. 1. Choose **Code** in the **Builder | Code** field on the right. 1. Type `sum(up)` in the **Metrics browser** query field. 1. Type `example-namespace` in the **Namespace** field. @@ -137,6 +185,10 @@ rule is now being created in Grafana Mimir ruler and will be soon available for ``` 1. Confirm the query returns a value of `3` which is the number of Mimir instances currently running in your local setup. + + + + ## Configure your first alert rule Alerting rules allow you to define alert conditions based on PromQL expressions and to send notifications about firing @@ -144,15 +196,17 @@ alerts to Grafana Mimir Alertmanager. In this section you're going to configure tooling offered by Grafana. 1. Open [Grafana Alerting](http://localhost:9000/alerting/list). -2. Click **New alert rule**. -3. Configure the alert rule: - 1. Select **Mimir or Loki alert** in the top selector. - 2. Choose **Mimir** in the **Select data source** field. - 3. Type `up == 0` in the **Metrics browser** query field. - 4. Type `MimirNotRunning` in the **Rule name** field. - 5. Select `example-namespace` in the **Namespace** field. - 6. Select `example-group` in the **Group** field. - 7. From the upper-right corner, click the **Save and exit** button. +1. Click **New alert rule**. +1. Configure the alert rule: + 1. Type `MimirNotRunning` in the **Rule name** field. + 1. Choose **Mimir** in the **Select data source** field. + 1. Type `count(up == 0)` in the **Metrics browser** query field. This currently shows `no data` since all instances are running. +1. Scroll down to **Set evaluation behavior**: + 1. Select `New folder` and type `example-folder` in the **Folder name** field. + 1. Select `New evaluation group` and type `example-group` in the **Group name** field. Set the evaluation interval to `30s`. +1. Scroll down to **Configure labels and notifications**: + 1. Select the `Contract point` dropdown and choose `grafana-default-email`. +1. Click the **Save rule and exit** button. Your `MimirNotRunning` alert rule is now being created in Grafana Mimir ruler and is expected to fire when the number of Grafana Mimir instances is less than three. You can check its status by opening the [Grafana Alerting](http://localhost:9000/alerting/list) @@ -162,9 +216,9 @@ To see the alert firing we can introduce an outage in the Grafana Mimir cluster: 1. Abruptly terminate one of the three Grafana Mimir instances: ```bash - docker-compose kill mimir-3 + docker compose kill mimir-3 ``` -1. Open [Grafana Alerting](http://localhost:9000/alerting/list) and check out the state of the alert `MimirNotRunning`, +2. Open [Grafana Alerting](http://localhost:9000/alerting/list) and check out the state of the alert `MimirNotRunning`, which should switch to "Pending" state in about one minute and to "Firing" state after another minute. _Note: since we abruptly terminated a Mimir instance, Grafana Alerting UI may temporarily show an error when querying rules: the error will auto resolve shortly, as soon as Grafana Mimir internal health checking detects the terminated instance as unhealthy._ @@ -186,7 +240,11 @@ To resolve the alert and recover from the outage, restart the Grafana Mimir inst docker-compose start mimir-3 ``` 2. Open [Grafana Alerting](http://localhost:9000/alerting/list) and check out the state of the alert `MimirNotRunning`, - which should switch to "Normal" state in about one minute. + which should switch to "Normal" state in about 30 seconds. + + + + ## Summary @@ -194,8 +252,14 @@ In this tutorial you started Grafana Mimir locally in a high-available setup as some metrics to Grafana Mimir. You then queried those metrics stored in Mimir using Grafana, and visualized them in some Grafana dashboards. Lastly, you configured a recording rule and an alert via the Grafana Alerting UI and verified that the alert fired as expected when the condition was met. + + Once you've completed the tutorial, release all Docker resources by running this Docker command: ```bash docker-compose down -v ``` + + + +