Skip to content

hmalloc

Jump to bottom
Honggyu Kim edited this page Dec 13, 2024 · 5 revisions

hmalloc ‐ heterogeneous memory allocator

The HMSDK provides heterogeneous memory allocation APIs so that users can explicitly specify some memory area to be allocated at desired NUMA nodes. This can be done by linking libhmalloc.so library. The supported APIs are defined at hmalloc.h, which can be simply included to use hmalloc APIs that is explained below.

It would be better to start with glossary for simpler explanation afterwards.

GLOSSARY

HMALLOC APIs

  • The hmalloc APIs are heterogeneous memory allocation APIs provided by libhmalloc.so such as hmalloc(3), hcalloc(3), hposix_memalign(3), hmmap(3), etc. All the APIs defined in hmalloc.h are hmalloc APIs.

HMALLOC POOL

  • The hmalloc pool is specially managed memory areas that can be optionally controlled by hmctl(8) tool. If target programs allocate memory using hmalloc APIs, then this area is mapped as hmalloc pool. This hmalloc pool has no effect if the target program runs without hmctl(8), but if it runs with hmctl(8) attached, then the memory policy of this area can be changed based on the usage of hmctl(8).

HMCTL

  • The hmctl(8) is a tool that controls heterogeneous memory allocation policy. That means it can change the memory policy of hmalloc pool allocated by hmalloc APIs internally using mmap(2) and mbind(2). If hmctl(8) is attached and -m/--membind or -p/--preferred option is given with a valid NUMA node ID, then the hmalloc pool memory is allocated from the target node with the given memory policy based on the usage of hmctl(8).

HMALLOC COMPONENTS

HMALLOC HEADER

Users have to include the hmalloc header hmalloc.h before using hmalloc APIs including:

  • hmalloc, hcalloc, hposix_memalign, hmmap, etc.

HMALLOC LIBRARY

  • The hmalloc library, libhmalloc.so, provides heterogeneous memory allocation APIs provided by hmalloc.h.

HMCTL

  • The hmctl is a user level tool that controls heterogeneous memory allocation policy. Please see HMCTL at GLOSSARY section.

INSTALLATION

Download

$ git clone https://github.com/skhynix/hmsdk.git
$ cd hmsdk/hmalloc

Build

In order to build hmalloc, the following packages are required.

  • cmake, make and gcc
  • libjemalloc-dev ,libnuma-dev
  • pandoc for manual (optional)
$ cmake -B build
$ make -j -C build
$ sudo make install -C build

Then the required files are installed at the following location.

/usr/local/bin/hmctl
/usr/local/lib/libhmalloc.so
/usr/local/include/hmalloc.h
/usr/local/share/man/man8/hmctl.8
/usr/local/share/man/man3/hmalloc.3
/usr/local/share/man/man3/hmalloc_usable_size.3
/usr/local/share/man/man3/hposix_memalign.3
/usr/local/share/man/man3/hmmap.3

If you want to change the install path, then CMAKE_INSTALL_PREFIX can be changed as follows.

$ cmake -B build -DCMAKE_INSTALL_PREFIX=/path/to/install

HMALLOC APIs

The hmalloc APIs are heterogeneous memory allocation APIs provided by libhmalloc.so. It provides following memory allocation and free APIs and please refer to their man pages for the usage.

hmctl (heterogeneous memory controller)

The hmctl tool is to control heterogeneous memory allocation policy for hmalloc APIs provided by libhmalloc.so library. hmctl controls memory policy with:

  • -m, --membind=nodes: allocate memory only from nodes for hmalloc APIs. This allocates hmalloc area to nodes with MPOL_BIND policy.
  • -p, --preferred=node: Preferably allocate memory on node for hmalloc APIs. This allocates hmalloc area to node with MPOL_PREFERRED policy.
  • -P, --preferred-many=nodes: Preferably allocate memory on nodes for hmalloc APIs. This allocates hmalloc area to nodes with MPOL_PREFERRED_MANY policy.
  • -i, --interleave=nodes: Set a memory interleave policy. Memory will be allocated using round robin on nodes.
  • -w, --weighted-interleave=nodes: Set a weighted memory interleave policy. Memory will be allocated using the weighted ratio for each node, which can be read from /sys/kernel/mm/mempolicy/weighted_interleave/node*.

If there is a simple program example that allocates 256 MiB using malloc() and 512 MiB using hmalloc(), then users can set the memory policy for both global scope and the hmalloc pool separately. The following usage is to set memory policy with MPOL_PREFERRED to node 1 in global scope and MPOL_BIND to node 2 in hmalloc pool as follows.

$ numactl -p 1 hmctl -m 2 ./example &
[1] 49288
255 MiB is allocated by malloc().
511 MiB is allocated by hmalloc().
Press enter to stop.

The numastat output shows that hmctl correctly makes the 512 MiB of hmalloc area to be allocated at node 2.

$ numastat -c -p $!
Per-node process memory usage (in MBs) for PID 49288 (example)
         Node 0 Node 1 Node 2 Node 3 Total
         ------ ------ ------ ------ -----
Total         3    258    512      0   773

Please refer to hmctl man page for the usage in details.

HMALLOC INTERNALS

graph TD

subgraph hmalloc_APIs
    hmalloc
    hrealloc
    etc1[...]
end
subgraph normal_allocation_APIs
    malloc
    realloc
    etc2[...]
end
subgraph NUMA
    node0
    node1
    etc3[...]
    nodeN
end

p --> normal_allocation_APIs
p("program<br/> (libhmalloc.so linked)") --> hmalloc_APIs

hmalloc_APIs --> |without hmctl| a("use system default allocator<br/>(ptmalloc by default)")
hmalloc_APIs --> |hmctl| b(hmalloc pool)
normal_allocation_APIs --> a

b --> |"hmctl mempolicy option<br\>(e.g. --membind)"|c((mbind)) 
c -.-> node0
c -.-> node1
c -.-> etc3
c -.-> nodeN
Loading

If a target program does use hmalloc APIs and linked with libhmalloc.so, then its normal memory allocation such as malloc will allocate dynamic memory using its system default memory allocator, ptmalloc in glibc by default. In terms of its internal allocation calls using hmalloc APIs such as hmalloc, its behavior depends whether the program runs with hmctl or not. If it runs without hmctl, then it just simply redirect hmalloc to malloc calls and that makes it also uses the system default allocator for dynamic memory allocation.

flowchart TD
b("$ ./program") --> d2(("malloc"))--> f2("normal allocation")
b --> e(("hmalloc")) --> d2
Loading

However, if the target program runs with hmctl, then hmalloc API calls use hmalloc pool for dynamic memory allocation. This area is affected by the memory policy options of hmctl such as -m, -p, -P, -i or -w so that users can apply a specific memory policy only to the hmalloc pool area while the rest of the area remains unaffected.

flowchart TD
c("$ hmctl [-m|-p|-P|-i|-w] $NODE_ID ./program") --> d3(("malloc")) -->  f4("normal allocation<br/>(default memory policy applied)")
c --> e2(("hmalloc"))  --> g("allocation at numa $NODE_ID<br/>(managed as hmalloc pool)")
Loading
Clone this wiki locally