Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Table.Upsert based on pebble.Merge #108

Open
marino39 opened this issue Aug 23, 2023 · 0 comments
Open

Table.Upsert based on pebble.Merge #108

marino39 opened this issue Aug 23, 2023 · 0 comments

Comments

@marino39
Copy link
Collaborator

The idea is for Table.Upsert to use pebble.Merge function. This function writes to SST entry marked as a merge. The entry is merged with old version of the key on key get, when key is encountered in the iteration and during compactions.

This way we could limit amount of reads during indexing TokenBalances in Sequence Indexer to 0. The keys would be merged as they are read or compacted.

Estimated impact:

  • ~50% gain in speed of indexing
  • Unknown probably minor loses in read speed as now keys will need to be merged on read. However, the more frequently address is written to probably it will be read more frequently too. So I believe impact on read speed should spread and be really low.

Merger interfaces:
https://github.com/cockroachdb/pebble/blob/1798fbf5956c16664b6707c51947b65c9e1a10d4/internal/base/merger.go#L88

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant