HDDS-11773. Prevent frequent DataNode Ratis snapshotting. #7473
Conversation
Change-Id: I2baf863c537cc3f5b0e2905c2fb1ca88d05c0ff2
jojochuang
changed the title
There was a problem hiding this comment.
Thanks @jojochuang .
But this also changes OM's default. It could be a concern when Ratis snapshot interval is set to a value too high for followers to catch up, thus failing OM bootstrapping? Is there an existing mechanism to tune this for Datanodes only?
What do you think? @szetszwo
|
isn't it DataNode only? |
Right. Pls amend the config tag |
| @@ -279,15 +279,15 @@ | |||
| </property> | |||
| <property> | |||
| <name>hdds.ratis.snapshot.threshold</name> | |||
| <value>10000</value> | |||
| <value>100000</value> | |||
| <tag>OZONE, RATIS</tag> | |||
There was a problem hiding this comment.
| <tag>OZONE, RATIS</tag> | |
| <tag>OZONE, CONTAINER, RATIS</tag> |
There was a problem hiding this comment.
Actually I'm not entire sure if it should be tagged CONTAINER or DATANODE
| <tag>OZONE, RATIS</tag> | ||
| <description>Number of transactions after which a ratis snapshot should be | ||
| taken. | ||
| </description> | ||
| </property> | ||
| <property> | ||
| <name>hdds.container.ratis.statemachine.max.pending.apply-transactions</name> | ||
| <value>10000</value> | ||
| <value>100000</value> | ||
| <tag>OZONE, RATIS</tag> |
There was a problem hiding this comment.
| <tag>OZONE, RATIS</tag> | |
| <tag>OZONE, CONTAINER, RATIS</tag> |
What changes were proposed in this pull request?
HDDS-11773. Bump hdds.ratis.snapshot.threshold and hdds.container.ratis.statemachine.max.pending.apply-transactions to 100k
Please describe your PR in detail:
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-11773
How was this patch tested?
Applied the change to a HBase cluster. Previously it was snapshotting every 4-5 seconds, and now it is doing it about every minute.