Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use barriers when flushing #25

Open
g2p opened this issue Jan 15, 2019 · 5 comments
Open

Use barriers when flushing #25

g2p opened this issue Jan 15, 2019 · 5 comments
Assignees

Comments

@g2p
Copy link
Collaborator

g2p commented Jan 15, 2019

Follow-up to #14.
Cc @fondation451 and @samoht.

We should issue a barrier between after flushing all non-root nodes and before flushing the root node.
Then issue a second barrier after writing the root node.
If durability is desired (not just ordering), make the second barrier a durable one.
When using the Unix backend, we also need to prevent it from adding an extra layer of buffering, by passing ~buffered:false when instanciating ( https://github.com/mirage/mirage-block-unix/blob/master/lib/block.mli#L53 ).

@g2p
Copy link
Collaborator Author

g2p commented Jan 15, 2019

In the LMDB-like case when we don't write or compute CRCs, the first barrier should be issued after writing everything but the first sector of the root block. That way, we can guarantee that the block with the newest generation is indeed a complete root block.

In fact, do this unconditionally.

@g2p
Copy link
Collaborator Author

g2p commented Jan 29, 2019

Add per-flush parameters for durability. Use it in Irmin.

@g2p
Copy link
Collaborator Author

g2p commented Jan 29, 2019

Also upstream the mirage-block stuff

@g2p
Copy link
Collaborator Author

g2p commented Jan 29, 2019

Control open semantics, because some can be broken. Unix's buffering should be bypassed for example.

@g2p
Copy link
Collaborator Author

g2p commented Jan 29, 2019

The idea behind writing the header last is that the sector containing the new, highest generation number should only be written after the rest of the block is ready. Possible outcomes with an uncooperative device and an unlucky sequence of operations:

  • everything is written, the new root will be used, fine
  • some plausible data was written but the header was not written, at worst an already valid header already exists but it won't have the highest generation, because that will belong to the previous root and be elsewhere on the disk

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants