This repository has been archived by the owner on Jun 7, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 292
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #1106 from zalando/repartitioning-docs
Repartitioning documentation
- Loading branch information
Showing
1 changed file
with
21 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
--- | ||
title: Repartitioning | ||
position: 12 | ||
--- | ||
|
||
## Repartitioning | ||
|
||
Throughput of event type is defined by default statistic, which basically sets number of partitions for the event | ||
type (although it does not represent it clearly). Number of partitions is a scaling unit for Nakadi publishing and | ||
consumption. In order to change number of partitions one have to repartition event type by changing | ||
read_parallelism and write_parallelism. Nakadi takes for repartitioning the maximum of read_parallelism and | ||
write_parallelism. | ||
|
||
### Important caveats | ||
|
||
- Publishing events to event type with hash partition strategy will change the partitions in which they were appearing | ||
before | ||
- Nakadi guarantees ordering per partition per batch, repartitioning event types from 1 partitions to more will break | ||
total order of events | ||
- Repartitioning allows to only increase number of partitions | ||
- Consuming subscriptions are disconnected once repartitioning is finished |