diff --git a/best-practices/massive-regions-best-practices.md b/best-practices/massive-regions-best-practices.md index 9619fe5cffcb7..6e041c02094c2 100644 --- a/best-practices/massive-regions-best-practices.md +++ b/best-practices/massive-regions-best-practices.md @@ -51,6 +51,18 @@ You can check the following monitoring metrics in Grafana's **TiKV Dashboard**: ![Check Propose wait duration](/media/best-practices/propose-wait-duration.png) ++ `Commit log duration` in the **Raft IO** panel + + `Commit log duration` is the time Raftstore takes to commit Raft logs to the majority of members in the respective Region. The possible reasons for a high value of this metric with significant fluctuations include the following: + + - The workload on Raftstore is heavy. + - The append log operation is slow. + - Raft logs cannot be committed timely due to network congestion. + + Reference value: lower than 200-500 ms. + + ![Check Commit log duration](/media/best-practices/commit-log-duration.png) + ## Performance tuning methods After finding out the cause of a performance problem, try to solve it from the following two aspects: @@ -136,6 +148,14 @@ The default size of a Region is 96 MiB, and you can reduce the number of Regions > + The query performance, especially for queries that deal with a large range of data, might decrease. > + The Region scheduling slows down. +### Method 7: Increase the maximum number of connections for Raft communication + +By default, the maximum number of connections used for Raft communication between TiKV nodes is 1. Increasing this number can help alleviate blockage issues caused by heavy communication workloads of a large number of Regions. For detailed instructions, see [`grpc-raft-conn-num`](/tikv-configuration-file.md#grpc-raft-conn-num). + +> **Note:** +> +> To reduce unnecessary thread switching overhead and mitigate potential negative impacts from batch processing, it is recommended to set the number within the range of `[1, 4]`. + ## Other problems and solutions This section describes some other problems and solutions. diff --git a/media/best-practices/commit-log-duration.png b/media/best-practices/commit-log-duration.png new file mode 100644 index 0000000000000..de889199aeb62 Binary files /dev/null and b/media/best-practices/commit-log-duration.png differ diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index e6b8cc60766ac..71911e16ffaf9 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -163,7 +163,7 @@ This document only describes the parameters that are not included in command-lin ### `grpc-raft-conn-num` -+ The maximum number of links among TiKV nodes for Raft communication ++ The maximum number of connections between TiKV nodes for Raft communication + Default value: `1` + Minimum value: `1`