CODEWITHSUNDEEP
linuxperformancenuma
jack malkovick
jack malkovick

Reputation: 533

NUMA information in /proc/vmstat

I need to get some NUMA related information about my application (I can't use numatop tool for example, but I can use numastat). Therefore, I have some questions regarding NUMA related fields in /proc/vmstat, not being sure that I correctly understand their meaning.

These two are clearly related to allocation of (new) pages.

What about access of already allocated pages? I'm especially interested in pages allocated on one node and accessed from another. Are these two the ones I'm looking for?

And in the end,

Is this useful for me if I'm using custom calls from libnuma, like numa_bind in order to forcefully bind a process to a node? Without auto balancing are there any pages migrated to increment this counter?

Upvotes: 4

Views: 1519

Answers (1)

Margaret Bloom
Margaret Bloom

Reputation: 44066

Those are metrics used to profile the Automatic NUMA Balancing.

The balancer works as follow:

  1. When the process under inspection is not scheduled a portion of its address space is scanned and each page marked as not present.
    This will generate a fault the very next time the process accesses an address in those pages.
    These falts are intended and called NUMA Hinting Faults (NHF).
  2. When a NHF happens the kernel migrates the page to the memory local to faulting thread.

Now, a process may have multiple threads and when the balancer pickups a portion of the address space it cannot know which thread will access which page and so it cannot rule out pages that are already local to a node where one of the thread is executing.

For example, if the process has two threads, A e B, in nodes N1 and N2, the balancer cannot skip page X even if it is already in the local memory of node N1 (or N2).
So sometimes, the balancer will find itself in the situation where the NHF page is already in the memory closest to the thread, this is called a local NHF.

The percentage of local NHF over the total NHF is the esteem of how much optimal is the topology of the allocated memory.

Upvotes: 2

Related Questions