Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Remove references to mmap in store-gateway #8155

Merged
merged 2 commits into from
May 20, 2024
Merged

docs: Remove references to mmap in store-gateway #8155

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
006268e
docs: Remove references to mmap in store-gateway
56quarters May 16, 2024
b1ed439
Code review
56quarters May 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 8 additions & 15 deletions docs/sources/mimir/references/architecture/components/store-gateway.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,7 @@ On the read path, the [querier]({{< relref "./querier" >}}) and the [ruler]({{<
## Bucket index

To find the right blocks to look up at query time, the store-gateway requires a view of the bucket in long-term storage.
The store-gateway keeps the bucket view updated using one of the following options:

- Periodically downloading the [bucket index]({{< relref "../bucket-index" >}}) (default)
- Periodically scanning the bucket

### Bucket index enabled (default)
The store-gateway keeps the bucket view updated by periodically downloading the [bucket index]({{< relref "../bucket-index" >}}).

To discover each tenant's blocks and block deletion marks, at startup, store-gateways fetch the [bucket index]({{< relref "../bucket-index" >}}) from long-term storage for each tenant that belongs to their [shard](#blocks-sharding-and-replication).

Expand All @@ -41,12 +36,6 @@ For example, if you're running the Grafana Mimir cluster in Kubernetes, you can

For more information about the index-header, refer to [Binary index-header documentation]({{< relref "../binary-index-header" >}}).

### Bucket index disabled

When the bucket index is disabled, the overall workflow is still nearly the same.
The difference occurs during the discovery phase of the blocks at startup, and during the periodic checks.
Iterations over the entire long-term storage download `meta.json` metadata files while filtering out blocks that don't belong to tenants in their shard.

## Blocks sharding and replication

The store-gateway uses blocks sharding to horizontally scale blocks in a large cluster.
Expand Down Expand Up @@ -125,11 +114,15 @@ Keeping the index-header on the local disk makes query execution faster.
### Index-header lazy loading

By default, a store-gateway downloads the index-headers to disk and doesn't load them to memory until required.
When required by a query, index-headers are memory-mapped and automatically released by the store-gateway after the amount of inactivity time you specify in `-blocks-storage.bucket-store.index-header.lazy-loading-idle-timeout` has passed.
When required by a query, index-headers are loaded and automatically released by the store-gateway after the amount of inactivity time you specify in `-blocks-storage.bucket-store.index-header.lazy-loading-idle-timeout` has passed.

Grafana Mimir provides a configuration flag `-blocks-storage.bucket-store.index-header.lazy-loading-enabled=false` to disable index-header lazy loading.
When disabled, the store-gateway memory-maps all index-headers, which provides faster access to the data in the index-header.
However, in a cluster with a large number of blocks, each store-gateway might have a large amount of memory-mapped index-headers, regardless of how frequently they are used at query time.
When disabled, the store-gateway loads all index-headers, which provides faster access to the data in the index-header.
56quarters marked this conversation as resolved.
Show resolved Hide resolved
However, in a cluster with a large number of blocks, each store-gateway might have a large amount of loaded index-headers, regardless of how frequently they are used at query time.

When the index-header is loaded, only a portion of it is kept in memory to reduce memory usage.
The rest of the index-header is read from disk as required.
This requires that store-gateways have memory available to be used by the operating system for caching disk accesses.

## Caching

Expand Down
Loading