A library for easy use of huge page support in the LINUX Kernel

A cooperation
between

and
.

Funded by
.

libhugepagealloc

Most modern architectures support multiple page sizes. For example, IA-32 architecture supports 4K and 4M page sizes. There is a huge page support in the Linux Kernel using either the mmap system call or shared memory system calls (shmget, shmat). For the common programmer this a problem, because he has to change his program code to get huge pages for his memory. This library provides a wrapper to common dynamic memory allocation calls as malloc and new. It uses hugetlbfs to allocate memory backed by huge pages. For the user this is transparent since it is not necessary to recompile his code.

INSTALL

• Kernel:

  Take care that your kernel is configured with CONFIG_HUGETLB_PAGE (present under Processor types and feature) and CONFIG_HUGETLBFS (present under file system option on config menu) config options.
  The output of "cat /proc/meminfo" should have output like:

  HugePages_Total: xxx HugePages_Free: yyy Hugepagesize: zzz KB

  A cat /proc/filesystems should also show a filesystem of type "hugetlbfs" configured in the kernel.
  

• Allocating huge pages after booting:

  Now you have huge page support in the kernel you need to reserve contiguous memory for the huge pages. Since memory will fragment with time this is best done in a init script or just after booting. To reserve a certain number of huge pages type for example:

  echo 20 > /proc/sys/vm/nr_hugepages

  to reserve 20 huge pages. Be careful: memory reserved for huge pages is not available for normal programs with standard standard page size. Now you have to mount a hugetlbfs directory to make the hugepages available for the mmap system call. As root type:

  mount none /mnt/huge -t hugetlbfs

  /mnt/huge is the mountpoint. Now adjust the permissons on the mountpoint directory and you are done.

• Build the library:

  Just type make.

• Testing:

  Go to the test directory and type make. Run the ./test executable. If you execute it you should get something like:

  got page 0x40165008 freed page

  OK, that was with normal pages. If you call any program like that:

  LD_PRELOAD=./libhugepagealloc.so ./test

  your memory allocation should be backed by large pages. For the test program it will report:

  large page allocation active got page 0xb8400000 freed page

  If something goes wrong, the library will give a hopefully meaningful error message.

Usage

To replace the standard libc calls you have to preload the library.Type

If everything runs smooth, you should get the message

Download

• libhugepagealloc
  Download