Skip to content
This repository has been archived by the owner on Feb 1, 2024. It is now read-only.

Add RFC for State Pruning #8

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

peterschwarz
Copy link
Contributor

This RFC describes State Pruning, a necessary feature for long-term
stability of the Sawtooth platform. It is a method for controlling
growth of the state database by providing a mechanism for pruning older
state roots in the Merkle-Radix state database.

Signed-off-by: Peter Schwarz [email protected]

This RFC describes State Pruning, a necessary feature for long-term
stability of the Sawtooth platform.  It is a method for controlling
growth of the state database by providing a mechanism for pruning older
state roots in the Merkle-Radix state database.

Signed-off-by: Peter Schwarz <[email protected]>
Signed-off-by: Peter Schwarz <[email protected]>

Along side the serialized node, a change log for a particular state root is
stored. This change log includes the keys of values that are added, and keys of
deleted values by replaced by future state roots. The change log is written to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"This change log includes the keys of values that are added, and keys of deleted values by replaced by future state roots." Im sure what you mean by "by replace by future state roots", is this a typo?

Ethereum Geth has also discussed the use of state pruning for their Merkle trie
implemenation. It has not yet implemented this for its disk-based storage, but
it has implemented reference counting for its in-memory storage of the trie. See
this [blog post](https://blog.ethereum.org/2015/06/26/state-trie-pruning/) for
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This link does not work. I get a "Page not found"


## Configuration

Maximum block depth is a local configuration option. It is set via the command
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this be added to the validator.toml file as well? or only on the command line?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added some text for this.

- Addressed some typos.
- Added a reference to the local configuration in the toml
configuration, as well as CLI arguments.

Signed-off-by: Peter Schwarz <[email protected]>

Along side the serialized node, a change log for a particular state root is
stored. This change log includes the keys of values that are added, and keys of
deleted values by replaced by future state roots. The change log is written to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo? "...by replaced by..."

Copy link

@danintel danintel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A valuable feature--I am frequently asked about handling long-term blockchain growth and this will be useful.

There's some spelling errors that should be fixed.

around when to switch from the pre-compacted to the post-compacted database.

See the
[documenation](http://www.lmdb.tech/doc/group__mdb.html#ga3bf50d7793b36aaddf6b481a44e24244)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/documenation/documentation/

[prior-art]: #prior-art

Ethereum Geth has also discussed the use of state pruning for their Merkle trie
implemenation. It has not yet implemented this for its disk-based storage, but

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/implemenation/implementation/

end of the copy, switch the reference to the underlying database for use in
normal operation and discard the old database.

This particular solution is simple in it's implementation, and requires no

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/it's/its/

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Development

Successfully merging this pull request may close these issues.

5 participants