CODEWITHSUNDEEP
cmemory-managementlinux-kernelnuma
Bao Bui
Bao Bui

Reputation: 305

Is it possible for a user process to tell the OS to relocate the mapping done by mmap to other NUMA node?

Consider this scenario: a user process running on a NUMA machine calls mmap to creates a new mapping in the virtual address space. It then uses the memory returned by mmap for its processing (storing its data,...). Now for some reasons, the user process is scheduled to a different NUMA node. Is it possible for the user process to tell the OS to relocate the already mapped memory (to a different NUMA node) while preserving the data?

Upvotes: 6

Views: 890

Answers (1)

osgx
osgx

Reputation: 94245

Migrating of physical memory is possible with migrate_pages call from libnuma (-lnuma): http://man7.org/linux/man-pages/man2/migrate_pages.2.html

 long migrate_pages(int pid, unsigned long maxnode,
                      const unsigned long *old_nodes,
                      const unsigned long *new_nodes);

Link with -lnuma.

migrate_pages() attempts to move all pages of the process pid that are in memory nodes old_nodes to the memory nodes in new_nodes. Pages not located in any node in old_nodes will not be migrated. As far as possible, the kernel maintains the relative topology relationship inside old_nodes during the migration to new_nodes.

There is also tool migratepages in numactl package to migrate all pages of pid: http://man7.org/linux/man-pages/man8/migratepages.8.html

You also can change memory policy with set_mempolicy: http://man7.org/linux/man-pages/man2/set_mempolicy.2.html

mbind syscall can be used to migrate subset of pages to some NUMA node:

https://www.kernel.org/doc/Documentation/vm/page_migration

...allows a process to manually relocate the node on which its pages are located through the MF_MOVE and MF_MOVE_ALL options while setting a new memory policy via mbind()

http://man7.org/linux/man-pages/man2/mbind.2.html

  If MPOL_MF_MOVE is specified in flags, then the kernel will attempt
   to move all the existing pages in the memory range so that they
   follow the policy.  Pages that are shared with other processes will
   not be moved.  If MPOL_MF_STRICT is also specified, then the call
   will fail with the error EIO if some pages could not be moved.

Upvotes: 4

Related Questions