-
Notifications
You must be signed in to change notification settings - Fork 8
HMSDK v2.0 User Guide
Important
Please read Capacity Expansion instead because the current document is deprecated and no longer valid.
This document describes how to install and use HMSDK 2.0.
This section describes the HMSDK components.
linux: linux kernel for HMSDK support
- page demotion and promotion implementation in DAMON(Data Access MONitor).
damo: userspace tool for 2-tier memory management
- user space tool to enable migration across fast and slow memory tiers.
tools: HMSDK tools contains
-
gen-config.pyto generate a damo config for 2-tier memory management scheme.
You can download HMSDK repository from GitHub. Make sure you attach
--recursive since HMSDK includes additional repositories as submodules.
$ git clone --recursive https://github.com/skhynix/hmsdk.git
$ cd hmsdk
This includes downloading the entire linux git history, so git cloning with
--shallow-submodules will significantly reduce the download time.
Please read this link for the general linux kernel build.
Since HMSDK 2.0 takes advantages of DAMON in linux kernel, additional DAMON related build configurations have to be enabled as follows.
$ cd hmsdk/linux
$ cp /boot/config-$(uname -r) .config
$ make menuconfig
$ echo 'CONFIG_DAMON=y' >> /.config
$ echo 'CONFIG_DAMON_VADDR=y' >> /.config
$ echo 'CONFIG_DAMON_PADDR=y' >> /.config
$ echo 'CONFIG_DAMON_SYSFS=y' >> /.config
$ echo 'CONFIG_MEMCG=y' >> /.config
$ make -j$(nproc)
$ sudo make INSTALL_MOD_STRIP=1 modules_install
$ sudo make install
You can run uname -r to verify if the kernel has been installed correctly.
$ uname -r
6.6.0-hmsdk2.0+
HMSDK 2.0 offers support for hot/cold memory tiering through the use of DAMON and its userspace tool, damo. To begin using HMSDK 2.0, ensure that you have installed the HMSDK kernel along with DAMON. To check if DMAON is properly installed, enter the following command:
$ if grep CONFIG_DAMON /boot/config-$(uname -r); then echo "installed"; fi
To start HMSDK 2.0, you can use the damo tool. Before starting, ensure that you have a proper configuration file.
# The -d and -c options can receive multiple DRAM or CXL NUMA nodes.
$ sudo ./tools/gen_config.py -d DRAM_NODES [DRAM_NODES ...] -c CXL_NODES [CXL_NODES ...] -o hmsdk.json
# make sure cgroup2 is mounted under /sys/fs/cgroup, then create "hmsdk" directory below.
$ sudo mount -t cgroup2 none /sys/fs/cgroup
$ echo '+memory' | sudo tee /sys/fs/cgroup/cgroup.subtree_control
$ sudo mkdir -p /sys/fs/cgroup/hmsdk
# Start HMSDK 2.0 based on hmsdk.json.
$ sudo ./damo/damo start hmsdk.jsonPlease refer to the detail usage of gen_config.py at Tools section below.
$ sudo ./damo/damo stop
Regarding the damo usage, please refer to damo usage.
gen_config.py is a python script that can generate a proper damo config file
for HMSDK 2-tier memory management scheme.
For example, if the system contains a CXL memory recognized as NUMA node 2 as the hardware topology shown in the following example.
$ numactl --hardware
available: 3 nodes (0-2)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
node 0 size: 31956 MB
node 0 free: 29028 MB
node 1 cpus: 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55
node 1 size: 32242 MB
node 1 free: 31032 MB
node 2 cpus:
node 2 size: 96761 MB
node 2 free: 96659 MB
node distances:
node 0 1 2
0: 10 21 14
1: 21 10 24
2: 14 24 10
Then a config file can be generated providing node 0, 1 for dram nodes and node 2 for CXL node.
$ sudo ./tools/gen_config.py --dram_nodes 0 1 --cxl_nodes 2 -o hmsdk.json
Please note that the generated hmsdk.json contains cgroup filter under the
cgroup name as "hmsdk" by default.
If you want to apply HMSDK 2.0 globally, use -g/--global option when running
gen_config.py as follows.
$ sudo ./tools/gen_config.py -d 0 1 -c 2 -g -o hmsdk.json
Please note that -d and -c are short options of --dram_nodes and --cxl_nodes.