CODEWITHSUNDEEP
linuxnumahuge-pages
ByteMe95
ByteMe95

Reputation: 876

linux mbind call fails on HUGETLB memory

I'm trying to use mbind to bind my memory to a specific NUMA node. I noticed it fails if my memory is allocated with mmap using MAP_HUGETLB, but mbind succeeds if I dont try to MAP_HUGETLB.

I definitely want both HUGE pages and to map the memory to a specific NUMA node. AmI doing something wrong, or is this not supported?

Running on CentOS7

//mbind fails on this
void * buf = mmap( NULL, sz, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE | MAP_HUGETLB, 0, 0 );
//works on this
void * buf2 = mmap( NULL, sz, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, 0, 0 );

int numaNode = 1;
unsigned long nodemask = 0;
nodemask |= 1 << numaNode;
if( mbind( buf, sz, MPOL_BIND, &nodemask, sizeof(nodemask) * 8, 0 ) < 0 )
    printf( "mbind buf failed: %s\n", strerror(errno) );

if( mbind( buf2, sz, MPOL_BIND, &nodemask, sizeof(nodemask) * 8, 0 ) < 0 )
    printf( "mbind buf2 failed: %s\n", strerror(errno) );

Edit: to be clear the mmap works fine and my machine is configured with huge pages. I didn’t check the mmap return code here for brevity

Upvotes: 0

Views: 371

Answers (2)

Tiborcz Kiss
Tiborcz Kiss

Reputation: 65

I think this artilcle arlready solved under, please check for it.

link

Upvotes: -1

Maxim Egorushkin
Maxim Egorushkin

Reputation: 136495

You don't check mmap return value for error. It fails to allocate huge pages unless you reserve them first.

Reserve huge pages with hugeadm, for example:

sudo hugeadm --pool-pages-min 2MB:4 --pool-pages-max 2MB:8 # 4-8 2MB pages
sudo hugeadm --pool-pages-min 1GB:2 --pool-pages-max 1GB:4 # 2-4 1GB pages

Another option is to use transparent huge pages without having to pass any extra flags to mmap. See Transparent Hugepage Support for full details.

Upvotes: -1

Related Questions