Data Consistency

Required reading: Blocks

Since StormDB addresses key-value pairs using an offset based strategy, the file contents must be perfectly aligned. If this is not the case (for example, partial writes, force kill) the entire database would be irrecoverable.

To ensure that data is readable even in the face of all odds, StormDB implements two strategies:

1. A CRC32 checksum added at the end of each block
2. Appends a dummy key-value pair (aka sync marker) before the start of every block

Block Frame

The following frame applies for both, the WAL and the data file:

+---------------------|--------------|----------+
| sync marker kv pair | 128 kv pairs | checksum |
+---------------------|--------------|----------+