Skip to content

Commit

Permalink
Fix format and language issues in the specification
Browse files Browse the repository at this point in the history
  • Loading branch information
minborg committed Aug 21, 2020
1 parent 9fb2a06 commit 1382e08
Show file tree
Hide file tree
Showing 9 changed files with 33 additions and 25 deletions.
Binary file added C:\Users\buddy\dev\shm\OPERAND_CHRONICLE_MAP
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
8 changes: 4 additions & 4 deletions spec/1-design-goals.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
arbitrary sequences of bytes.
- Chronicle Map optionally persists to a *single file* via the memory-mapping facility, present in
Windows and POSIX-compatible operating systems (`mmap`). The file shouldn't necessarily materialize
on disk. It's OK for Chronicle Map if the file resides a memory-mounted file system. The whole
on disk. It's OK for Chronicle Map if the file resides on a memory-mounted file system. The whole
Chronicle Map state is contained in the file contents i. e. it is possible to move, copy or send
the file to another machine, access it and observe exactly the same Chronicle Map state. Chronicle
Map doesn't use the metadata of the file.
Expand All @@ -16,7 +16,7 @@
https://en.wikipedia.org/wiki/ACID) here; see the detailed definition below in
[Guarantees](#guarantees-1) section).
- Chronicle Map runs only on little-endian architectures.
- Chronicle Map is *not* distributed: the whole data store resides the memory of a single machine
- Chronicle Map is *not* distributed: the whole data store resides in the memory of a single machine
(a single file, if persisted).

## Guarantees
Expand All @@ -35,7 +35,7 @@ Chronicle Map implementation. It might be an OS thread or a "greener" thread.

#### CPU

- CPU supports atomic 64-bit compare-and-swap operations with aligned memory, i. e. if several
- The CPU supports atomic 64-bit compare-and-swap operations with aligned memory, i. e. if several
threads, possibly belonging to different processes, try to perform compare-and-swap operation on
the same 4-byte block, at most one thread will succeed and all the rest will fail.
- Aligned 32-bit or 64-bit writes are atomic, that means either all 4(8) bytes are written to
Expand All @@ -48,7 +48,7 @@ The two above points are true for CPUs with x86 and x86_64 architectures.

- If some values X<sub>i</sub> are written at addresses A<sub>i</sub>, then 32- or 64-bit value Y
is written at address B, there is a way to ensure some memory order. In particular, if a
concurrent thread reads value Y from address B, after that it will read X<sub>i</sub values from
concurrent thread reads value Y from address B, after that it will read X<sub>i</sub> values from
A<sub>i</sub> addresses (but not the old values, stored at those addresses).

> In terms of the Java Memory model, this assumption means "writes to A<sub>i</sub> addresses
Expand Down
1 change: 0 additions & 1 deletion spec/2-design-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,6 @@ after the previous tier.* All tiers, either first in their segments or chained,
<ol>
<li>If the keys are identical, the entry for the queried key within the segment is found, exit
the procedure.</li>

<li>If the keys are not identical (because of a full collision of parts of their hash codes for
storing in hash lookup tables), continue the search in the current tier's hash lookup table
(step #2), without resetting the hash lookup slot index.</li>
Expand Down
35 changes: 20 additions & 15 deletions spec/3-memory-layout.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,15 @@ one big continuous block of memory. Its structure, from lower addresses to highe
persisted to disk.

3. [The global mutable state](#global-mutable-state).
4. <a name="segment-headers-alignment" />
4. [Segment header alignment](#segment-headers-alignment")
The alignment to the next page boundary by addresses (page size of the current memory mapping),
if a new Chronicle Map is created, if an existing one (i. e. persisted) is loaded, the alignment
is read from the global mutable state (specifically this field of the global mutable state is
actually immutable, once written).

> The purpose of this alignment is to minimize the number of pages spanned by the following
> *segment headers area*. The segment headers area is frequently accessed and updated, so the pages
> it spans almost always reside in TLB cache and always need to be flushed to the disk.
> it spans almost always reside in the TLB cache and always need to be flushed to the disk.
> The alignment (more precisely, the offset to the next area, the segment headers area) is stored
> in the global mutable state, rather then computed each time a mapped Chronicle Map store is
Expand All @@ -30,18 +30,18 @@ one big continuous block of memory. Its structure, from lower addresses to highe
5. [The segment headers area](#segment-headers-area).
6. [The main segments area](#main-segments-area).
7. Zero or several [extra tier bulks](#extra-tier-bulks).
7. Zero or more [extra tier bulks](#extra-tier-bulks).

The process of initialization of the Chronicle Map's memory is described on [Initialization Order](
5-initialization.md) page.

## Self-bootstrapping header

The concept is described in [Self Bootstrapping Data](
The concept is described in the [Self Bootstrapping Data](
https://github.com/OpenHFT/RFC/blob/master/Self-Bootstrapping-Data/Self-Bootstraping-Data-0.1.md)
specification.

The structure of this area is described in [Size Prefixed Blob](
The structure of this area is described in the [Size Prefixed Blob](
https://github.com/OpenHFT/RFC/blob/master/Size-Prefixed-Blob/Size-Prefixed-Blob-0.1.md)
specification. The first 8 bytes contains the hash value of bytes sequence from 9th byte to the end
of this size-prefixed blob, computed by [xxHash](https://github.com/Cyan4973/xxHash/) algorithm
Expand All @@ -63,12 +63,13 @@ The global mutable state is 33 bytes long.
> global mutable state and segment headers (see below).
2. Bytes 8..10 - the number of allocated [extra tier bulks](#extra-tier-bulks). An unsigned 24-bit
value, stored in the little-endian order.
value, stored in little-endian order.

See also the [extra tier bulk allocation](5-initialization.md#entra-tier-bulk-allocation)
operation.

3. Bytes 11..15 - the index of the first *free* segment tier. An unsigned 40-bit value, stored
in the little-endian order. Extra segment tiers are allocated in bulks, so there is usually a chain
in little-endian order. Extra segment tiers are allocated in bulks, so there is usually a chain
of allocated, but unused yet segment tiers. This field points to the head of this chain, or has
value `0`, if there are no free segment tiers.

Expand All @@ -79,15 +80,18 @@ The global mutable state is 33 bytes long.
[`actualSegments`](3_1-header-fields.md#actualsegments) &minus; 1) has tier index `actualSegments`,
the first tier of the first extra tier bulk has tier index `actualSegments` + 1, etc. Tier indexes
are 1-counted, because value 0 has some special meaning.

4. Bytes 16..20 - the number of used extra segment tiers. An unsigned 40-bit value, stored in
the little-endian order.
little-endian order.

5. Bytes 21..24 - the offset of the segment headers area from the beginning of the memory of this
Chronicle Map store. An unsigned 32-bit value, stored in the little-endian ordered. This field
Chronicle Map store. An unsigned 32-bit value, stored in little-endian ordered. This field
determines the size of [the 4th area of the general Chronicle Map structure](
#segment-headers-alignment).

6. Bytes 25..32 - the Chronicle Map data store size, the offset to the end of the [main segments
area](#main-segments area) or the last [extra tier bulk](#extra-tier-bulks). A non-negative 64-bit
value, stored in the little-endian order.
value, stored in little-endian order.

> The reference Java implementation: [`VanillaGlobalMutableState`
> ](../src/main/java/net/openhft/chronicle/hash/VanillaGlobalMutableState.java).
Expand All @@ -97,7 +101,7 @@ The global mutable state is 33 bytes long.
The offset to this area is stored in the 5th field of the [global mutable state
](#global-mutable-state).

The size (in bytes) of a segment headers area is [`actualSegments`](
The size (in bytes) of a segment header's area is [`actualSegments`](
3_1-header-fields.md#actualsegments) * [`segmentHeaderSize`](3_1-header-fields.md#segmentheadersize
). Each segment header starts at offsets from the start of the segment headers area, that are
multiples of `segmentHeaderSize`. Each segment header is 32 bytes long. `segmentHeaderSize` &minus;
Expand Down Expand Up @@ -126,28 +130,29 @@ multiples of `segmentHeaderSize`. Each segment header is 32 bytes long. `segment
It is a 64-bit value, stored in the little-endian order. If the value of this field is 0, this
means there is no chained segment tier in this segment yet after the first tier, in other words,
the first tier is the only one in the chain for the current segment.

5. Bytes 24..31 - reserved for use by extensions.

> The reference Java implementation: [`BigSegmentHeader`
> ](../src/main/java/net/openhft/chronicle/hash/impl/BigSegmentHeader.java)
## Main segments area

This area contains first tiers of the Chronicle Map's segments.
This area contains the first tiers of the Chronicle Map's segments.

A main segments area starts immediately after the end of a segment headers area without extra
offsets and alignments.

The size of a main segments area is [`actualSegments`](3_1-header-fields.md#actualsegments) *
[`tierSize`](3_1-header-fields.md#tiersize). Segment tiers in a main segments area doesn't have
[`tierSize`](3_1-header-fields.md#tiersize). Segment tiers in a main segments area don't have
extra gaps in memory between each other.

### Segment tier structure

See also [segment tiers design overview](2-design-overview.md#segment-tier) for more explanations
about this structure.

Segment tier is [`tierSize`](3_1-header-fields.md#tiersize) bytes long.
A segment tier is [`tierSize`](3_1-header-fields.md#tiersize) bytes long.

The segment tier structure:

Expand All @@ -163,7 +168,7 @@ A hash lookup area starts at the same address as a segment tier, containing it.
A hash lookup consists of [`tierHashLookupCapacity`](3_1-header-fields.md#tierhashlookupcapacity)
slots each of [`tierHashLookupSlotSize`](3_1-header-fields.md#tierhashlookupslotsize) bytes. In each
slot a 32-bit or 64-bit (if the `tierHashLookupSlotSize` is 4 or 8, respectively) value is stored in
little-endian order. The slot value of 0 designates an empty slot.
little-endian order. A slot value of 0 designates an empty slot.

##### Hash lookup key

Expand Down
10 changes: 7 additions & 3 deletions spec/3_2-lock-structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@

The lock structure is 8 bytes (64 bits) long.

1. *Count word*:
*Count word*:
1. Bits 0..29 - read lock count (little-endian)
2. Bit 30 - update lock flag
3. Bit 31 - write lock flag
2. *Wait word*: bits 32..63 - wait count (little-endian)

*Wait word*:
1. bits 32..63 - wait count (little-endian)

## Try acquire read lock

Expand Down Expand Up @@ -107,14 +109,15 @@ procedure and call one depending on the context.
<a name="release-write-lock" />
<a name="write-to-update-lock-downgrade" />
<a name="write-to-read-lock-downgrade" />

## Release write lock, or write to update lock downgrade, or write to read lock downgrade

Perform a CAS operation on the count word of the lock state, comparing 0x80000000 (i. e. a count
word with the write lock flag set) and swapping with 0 (in case of releasing write lock), or
0x40000000 (in case of write to update lock downgrade), or 1 (in case of write to read lock
downgrade). The result of the CAS operation is the result of the procedure.

## Try upgrade update to write lock
## Try upgrade to write lock

Perform a CAS operation on the count word of the lock state, comparing 0x40000000 (i. e. a count
word with the update lock flag set, the write lock flag not set, the read lock count of zero) and
Expand All @@ -140,6 +143,7 @@ procedure.

<a name="time-limited-write-lock-acquisition" />
<a name="time-limited-update-to-write-lock-upgrade" />

## Time-limited write lock acquisition or update to write upgrade

1. Perform the corresponding *try acquire* procedure ([write lock](#try-acquire-write-lock) or
Expand Down
4 changes: 2 additions & 2 deletions spec/5-initialization.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ existing Map, persisted to some file).
[wait until the Chronicle Map is ready](#wait-until-chronicle-map-is-ready). Otherwise, acquire
an exclusive file lock, and check the file size again. If the file size is non-zero now, release
the file lock and [wait until the Chronicle Map is ready](#wait-until-chronicle-map-is-ready). If
the file is still empty, [write self bootstrapping header](#write-self-bootstrapping-header) to the
the file is still empty, [write the self bootstrapping header](#write-self-bootstrapping-header) to the
beginning of the file, then release the file lock.

> The purpose of acquiring file lock is to ensure that only one concurrent process (if any)
Expand Down Expand Up @@ -49,7 +49,7 @@ existing Map, persisted to some file).
out memory from the beginning of the tier to the start of the [entry space](
3-memory-layout.md#entry-space), i. e. zero out this tier's hash lookup, segment tier counters area
and free list.
6. White the segment headers offset into the 5th field of the global mutable state.
6. Write the segment headers offset into the 5th field of the global mutable state.
7. Write the offset to the end of the main segments area into the 6th field of the global mutable
state.
8. If the Chronicle Map is persisted, ensure all data written to the file is flushed to the disk.
Expand Down

0 comments on commit 1382e08

Please sign in to comment.