KV Cache

C++ Implementation of a Key-Value cache.

The cache is composed by a primary cache (RAM Cache) and a secondary cache (Storage Cache).

The RAM Cache has a maximum size. When the maximum capacity has been reached, KV Pairs will be evicted based on the selected cache eviction strategies to make place to new elements. Evicted elements will then be moved to the secondary cache.

Tools and Required Software

Before the project can be used, you should check that all the necessary softwares and tools are installed. Mandatory are the following:

- C++ compiler compatible with C++20

gcc 7+

Install command

Debian/Ubuntu:
```
  sudo apt install build-essential
```
Windows:
```
  choco install mingw -y
```
MacOS:
```
  brew install gcc
```

clang 6+
Install command
- Debian/Ubuntu:
```
  bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"
```
- Windows:
  
  Visual Studio 2019 ships with LLVM (see the Visual Studio section). However, to install LLVM separately:
```
  choco install llvm -y
```
  llvm-utils for using external LLVM with Visual Studio generator:
```
  git clone https://github.com/zufuliu/llvm-utils.git
  cd llvm-utils/VS2017
  .\install.bat
```
- MacOS:
```
  brew install llvm
```

Visual Studio 2019 or higher

Install command + Environment setup

On Windows, you need to install Visual Studio 2019 because of the SDK and libraries that ship with it.

Visual Studio IDE - 2019 Community (installs Clang too):

  choco install -y visualstudio2019community --package-parameters "add Microsoft.VisualStudio.Workload.NativeDesktop --includeRecommended --includeOptional --passive --locale en-US"

Put MSVC compiler, Clang compiler, and vcvarsall.bat on the path:

  	choco install vswhere -y
  	refreshenv

  	# change to x86 for 32bit
  	$clpath = vswhere -products * -latest -prerelease -find **/Hostx64/x64/*
  	$clangpath = vswhere -products * -latest -prerelease -find **/Llvm/bin/*
  	$vcvarsallpath =  vswhere -products * -latest -prerelease -find **/Auxiliary/Build/*

  	$path = [System.Environment]::GetEnvironmentVariable("PATH", "User")
  	[Environment]::SetEnvironmentVariable("Path", $path + ";$clpath" + ";$clangpath" + ";$vcvarsallpath", "User")
  	refreshenv

- CMake at least version 3.15 or higher

Install Command

Debian/Ubuntu:
```
  sudo apt-get install cmake
```
Windows:
```
  choco install cmake -y
```
MacOS:
```
  brew install cmake
```

- Conan (Also Vcpkg could be used but isn't tested)

Install Command

Via pip - https://docs.conan.io/en/latest/installation.html#install-with-pip-recommended
```
  pip install --user conan
```

Note: Check that pip is installed with python3, otherwise there will be errors when configuring the cmake project

Windows:
```
  choco install conan -y
```
MacOS:
```
  brew install conan
```

C++20 is required since the following features are used in the library or in the client:

Concepts
Designated Initializer
Map.contains
Using enum construct

To setup the environment on linux ubuntu, there are two scripts that can be used:

For installing conan

scripts/conan_install.sh

For Installing the required Cpp tools

scripts/required_cpp_tools_install.sh

Optional Tools

This project may also run with other software for static code analysis, documentation and related things. These are optional but they are strongly suggested for development since they improve code quality.

Furthermore, a clang-format file is present the root folder to have a standard formatting for the codebase.

Doxygen

Install Command

Debian/Ubuntu:

  sudo apt-get install doxygen
  sudo apt-get install graphviz

Windows:

  choco install doxygen.install -y
  choco install graphviz -y

MacOS:

  brew install doxygen
  brew install graphviz

ccache

Install Command

Debian/Ubuntu:
```
  sudo apt-get install ccache
```
Windows:
```
  choco install ccache -y
```
MacOS:
```
  brew install ccache
```

Cppcheck

Install Command

Debian/Ubuntu:
```
  sudo apt-get install cppcheck
```
Windows:
```
  choco install cppcheck -y
```
MacOS:
```
  brew install cppcheck
```

include-what-you-use

Install Command

Follow instructions here: https://github.com/include-what-you-use/include-what-you-use#how-to-install

To install these optional tools environment on linux ubuntu the following script can be run:

For Installing the required Cpp tools

scripts/optional_cpp_tools_install.sh

Build

This software uses cmake for generating the build system configuration.

In particular, starting from the root directory, the command to generate the build configuration is:

Single Configuration Builders (Make, Ninja)

  Single configuration builders won't know about configurations, so cmake has to create different build directories for each one of them.

  - Release
  ```Bash
  cmake -S . -B ./build/Release -DCMAKE_BUILD_TYPE:STRING=Release
  ```

  - Debug
  ```Bash
  cmake -S . -B ./build/Debug -DCMAKE_BUILD_TYPE:STRING=Debug
  ```

  - Release with Debug Symbols
  ```Bash
  cmake -S . -B ./build/RelWithDebInfo
  ```

  - Release with Minimum Size
  ```Bash
  cmake -S . -B ./build/MinSizeRel
  ```

Multiconfiguration Builders (Visual Studio, Ninja Multi-Configuration)

  Multiconfiguration builders will work directly in a single build folder and the actual configuration has to be specified at compile time.
  ```Bash
  cmake -S . -B ./build
  ```

Then, to actually compile the code, you can use directly cmake via the following commands launched from the root directory:

Single Configuration Builders (Make, Ninja)

Single configuration builders won't know about configurations, so the build process has to be launched separately from each build folder.
- Release
```
 cmake --build ./build/Release
```
- Debug
```
 cmake --build ./build/Debug
```
- Release with Debug Symbols
```
 cmake --build ./build/RelWithDebInfo
```
- Release with Minimum Size
```
 cmake --build ./build/MinSizeRel
```
Multiconfiguration Builders (Visual Studio, Ninja Multi-Configuration)

Multiconfiguration builders will work directly in a single build folder and the actual configuration has to be specified at build time.
- Release
```
 cmake --build ./build -- /p:configuration=release
```
- Debug
```
 cmake --build ./build -- /p:configuration=Debug
```
- Release with Debug Symbols
```
 cmake --build ./build -- /p:configuration=RelWithDebInfo
```
- Release with Minimum Size
```
 cmake --build ./build -- /p:configuration=MinSizeRel
```

CMake will automatically pick the predefined C++ compiler. On Linux systems you can change it via the "CXX" system variable.

Automatic Tests can be run from the build directly via CMake.

Single Configuration Builders (Make, Ninja)
```Bash cd build/Debug ctest -C Debug ```
Multiconfiguration Builders (Visual Studio, Ninja Multi-Configuration)
```Bash cd build ctest -C Debug ```

This library uses the Catch2 framework for automatic testing.

Continuous Integration

This project uses appveyor for running continuous integration. After every commit, the source code is compiled and tested on the following environments:

Linux Ubuntu 20.04 with GCC 9.3.0
Windows Server 2022 with MSBuild VC142

Static Code Analysis and Sanitizers

The cmake configuration supports various tools to be run automatically post build

What You See is What You Get
Clang-Tidy
Cppcheck

To add them to the build process, uncomment the relative options in the root's cmake file from line 22 to 43. These tools must be installed on the develompent system and cmake must be able to find them on the current system path. See the section on optional tools for installation instructions.

Library Documentation

The library supports doxygen for automatically generating code documentation. See the section on optional tools for instructions on installing doxigen on your system. The "doxygen-docs" target will be automatically generated by cmake if the option is uncommented in the root's cmake file on line 33. For example, if you generate a make configuration, the command to create the documentation after the build has been completed would be

cd build/Release
make doxygen-docs

If the process is successfull, the produced documentation will be found in the following directory:

build/Release/html

Test Client

With the library a small sample client is deployed. The following are the possible options:

Usage:
    KVCacheClient [--memCacheMaxSizeMB=<mb>] 
				  [--memoryCacheType=<memType>] 
				  [--storageCacheType=<storageType>] 
				  [--cacheEvictionStrategy=<cacheEviction>] 
				  [--storagePath=<path>] 
				  [--fileLoad=<filePath>] 
    KVCacheClient (-h | --help)
    KVCacheClient --version
 
 Options:
          --memCacheMaxSizeMB=<mb>                  Max size as Integer of the Memory Cache in MB. [Default 1 MB].
          --memoryCacheType=<memType>               Type of MemoryCache: (UNORDERED_MAP | ORDERED_MAP). [Default is UNORDERED_MAP].
          --storageCacheType=<storageType>          Type of StorageCache: (NONE | LINEAR_FILE). [Default is NONE].
          --cacheEvictionStrategy=<cacheEviction>   Algorithm for Cache Eviction when the MemoryCache is full: (FIFO | LIFO | LRU | MRU). [Default is FIFO].
          --storagePath=<path>                      Path of the StorageCache. Default is current directory.
          --fileLoad=<filePath>                     Path of a file to load into the cache. The file will need to have the KV Pairs on consecutive lines
          -h --help                                 Show this screen.
          --version                                 Show version.

In the example folder there are two usefull files. The first is a text file with a set of kv pairs (for the fileLoad option) and the second small bash script for launching the application. In general the full path of the compiled binary will be:

Single Configuration Builders (Make, Ninja)
- Release ```Bash /build/Release/src/KVCacheClient ```
- Debug
```
 /build/Debug/src/KVCacheClient
```
- Release with Debug Symbols
```
 /build/RelWithDebInfo/src/KVCacheClient
```
- Release with Minimum Size
```
 /build/MinSizeRel/src/KVCacheClient
```
Multiconfiguration Builders (Visual Studio, Ninja Multi-Configuration)
```Bash /build/src/KVCacheClient ```

Name		Name	Last commit message	Last commit date
Latest commit History 36 Commits
.vscode		.vscode
exampleData		exampleData
scripts		scripts
src		src
test		test
.clang-format		.clang-format
.clang-tidy		.clang-tidy
.gitignore		.gitignore
CMakeLists.txt		CMakeLists.txt
LICENSE		LICENSE
README.md		README.md
appveyor.yml		appveyor.yml
conanfile.py		conanfile.py

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

KV Cache

Tools and Required Software

- C++ compiler compatible with C++20

- CMake at least version 3.15 or higher

- Conan (Also Vcpkg could be used but isn't tested)

Optional Tools

Build

Continuous Integration

Static Code Analysis and Sanitizers

Library Documentation

Test Client

About

Releases

Packages

Languages

License

fsaporito/KVCache

Folders and files

Latest commit

History

Repository files navigation

KV Cache

Tools and Required Software

- C++ compiler compatible with C++20

- CMake at least version 3.15 or higher

- Conan (Also Vcpkg could be used but isn't tested)

Optional Tools

Build

Continuous Integration

Static Code Analysis and Sanitizers

Library Documentation

Test Client

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages