CODEWITHSUNDEEP
memory-managementlinux-kernelcentos7huge-pagesdpdk
Ofir Hermesh
Ofir Hermesh

Reputation: 145

CentOs7: HugePages_Rsvd equals 18446744073709551615

I have an application which uses a large amount of huge pages, for the purpose of DPDK. I allocate the pages at system start and then load/unload the application several times. After some reloads, the program cannot allocate huge pages anymore. When I look at meminfo, I see:

HugePages_Total:    2656
HugePages_Free:     1504
HugePages_Rsvd:     18446744073709551615   
HugePages_Surp:        0

This will remains this way, and will not let any application allocate huge pages, until I reboot the machine.

Any idea?

Upvotes: 1

Views: 409

Answers (1)

cosinus0
cosinus0

Reputation: 613

The decrement_hugepage_resv_vma function attempts -1 for resv_huge_pages, but unsigned arithmetic causes it to instead ULONG_MAX (unsigned long resv_huge_pages), which is 18446744073709551615 on 64-bit systems.

alloc_huge_page()
    page = dequeue_huge_page_vma(h, vma, addr, avoid_reserve)
        decrement_hugepage_resv_vma() {
            h->resv_huge_pages--;
        }

static void return_unused_surplus_pages(struct hstate *h,
                    unsigned long unused_resv_pages)
{
    h->resv_huge_pages -= unused_resv_pages;
}

Other less odds reason is gather_surplus_pages() function can overflow resv_huge_pages.

Try next during you test:

while [  1  ] ; do date | awk '{print $4}' >> HugePages_Rsvd.log; cat /proc/meminfo | grep HugePages_Rsvd >> HugePages_Rsvd.log; sleep 1;  done

I gues if resv_huge_pages will increment slowly so problem in h->resv_huge_pages += delta;

But if resv_huge_pages suddenly become -1 (unsigned long == 18446744073709551615 ) so problem in h->resv_huge_pages--; (resv_huge_pages was 0 and aftre decrement == -1)

Depend from you kernel you can check patch mm: numa: disable change protection for vma(VM_HUGETLB) 6b79c57b92cdd90853002980609af516d14c4f9c and BUG large value for HugePages_Rsvd

Upvotes: 3

Related Questions