NAME
SYNOPSIS
DESCRIPTION
ENVIRONMENT
TIER PARAMETERS
POLICIES
THRESHOLD PARAMETERS
DRAM FALLBACK POLICY
EXAMPLES
NOTES
COPYRIGHT
SEE ALSO
libmemtier.so - interposer library which enables the memkind memory tiering.
In order to use the memkind memory tiering with pre-built binaries, pass the following environment variables along with the command:
LD_PRELOAD=libmemtier.so MEMKIND_MEM_TIERS="..." command {arguments ...}
This library enables the memkind memory tiering mechanism. With this funcionality, allocations will be split between different types of memory automatically. The library allows making allocations with the usage of multiple kinds keeping a specified ratio between them. This ratio determines how much of the total allocated memory should be allocated with each kind. Both LD_PRELOAD and MEMKIND_MEM_TIERS environment variables are mandatory for enabling the memkind memory tiering.
memkind_create_pmem()
function.memkind_create_pmem()
function.NOTE: The application will fail when provided environment variable string is not in the correct format.
Default values for a threshold between first two tiers in the MEMKIND_MEM_TIERS environment variable are:
INIT_VAL = 1024, MIN_VAL = 513, MAX_VAL = 1536.
If there are more tiers defined, each next undefined threshold will have all parameters increased by 1024, so the next undefined threshold between the next two tiers will have:
INIT_VAL = 2048, MIN_VAL = 1537, MAX_VAL = 2560.
NOTE: Because setting the above parameters is optional, they will be set to default values in case they are not defined.
With the usage of both DRAM and KMEM_DAX tiers, if there is not enough memory to satisfy the DRAM tier memory allocation request, the allocation will fall back to PMEM memory.
With the usage of both DRAM and FS_DAX tiers, if there is not enough memory to satisfy the DRAM tier memory allocation request, the allocation will fail.
If there is not enough memory to satisfy the FS_DAX or KMEM_DAX tier memory allocation request, the allocation will fail.
The following example will run ls with the memkind memory tiering library. Make sure that paths to both libmemtier.so and libmemkind.so are included in LD_LIBRARY_PATH. During the application run, 20% of the allocated memory will come from PMEM memory and 80% will come from DRAM (FS_DAX:DRAM ratio is 1:4):
LD_PRELOAD=libmemtier.so MEMKIND_MEM_TIERS="KIND:FS_DAX,PATH:/mnt/pmem0,PMEM_SIZE_LIMIT:10G,RATIO:1;KIND:DRAM,RATIO:4;POLICY:STATIC_RATIO" /bin/ls -l
The example value of MEMKIND_MEM_TIERS environment variable where all allocations will come from PMEM memory with filesystem created with the path /mnt/pmem0 (PMEM file size is limited only by the specified filesystem):
LD_PRELOAD=libmemtier.so MEMKIND_MEM_TIERS="KIND:FS_DAX,PATH:/mnt/pmem0,RATIO:1;POLICY:STATIC_RATIO"
The example value of MEMKIND_MEM_THRESHOLDS environment variable. With INIT_VAL=64, on the application start all allocations lower than 64 bytes threshold will come from DRAM and equal to or greater than this value will come from PMEM memory NUMA nodes. The threshold value changes during the runtime in order to maintain the ratio. MIN_VAL=1 and MAX_VAL=10000 set the lower and upper limits of the threshold value. Note that the DYNAMIC_THRESHOLD policy has to be set in MEMKIND_MEM_TIERS environment variable:
LD_PRELOAD=libmemtier.so MEMKIND_MEM_TIERS="KIND:DRAM,RATIO:1;KIND:KMEM_DAX,RATIO:4;POLICY:DYNAMIC_THRESHOLD" MEMKIND_MEM_THRESHOLDS="INIT_VAL:64,MIN_VAL:1,MAX_VAL:10000"
libmemtier works for applications that do not statically link a malloc implementation. When libmemtier is loaded with LD_PRELOAD, allocations with size zero, like malloc(0), have the same result as the system’s standard library call. Most notably, a valid pointer may be returned in such calls, contrary to the default memkind behavior of returning NULL when size zero is passed to malloc-like functions.
Copyright (C) 2021 - 2022 Intel Corporation. All rights reserved.
memkind(3), malloc(3)
The contents of this web site and the associated GitHub repositories are BSD-licensed open source.