-
Notifications
You must be signed in to change notification settings - Fork 0
Large Page Support
Philippe Marschall edited this page Apr 14, 2020
·
10 revisions
If you allocate large amounts of contiguous memory it may beneficial to use larger pages to consume fewer page table entries and reduce pressure on the TLB. For example a page size of 2 MiB uses 500 times fewer page table entries than the default page size of 4 KiB.
Large pages differ by operating system but is usually supported through mmap
.
Linux supports large pages through HugeTLB Pages
int size = 2 * 1024 * 1024;
int flags = MmapFlags.MAP_SHARED | MmapFlags.MAP_ANONYMOUS | MmapFlags.MAP_HUGETLB | MmapFlags.MAP_HUGE_2MB;
ByteBuffer buffer = Mman.mmap(size, flags);
try {
// use buffer
} finally {
Mman.munmap(buffer);
}
Setting up a large page pool is a bit more involved.
First you need to create a group for shared memory usage. Here we call it hugetlb_shm
.
groupadd hugetlb_shm
adduser $USER hugetlb_shm
hugeadm --set-shm-group=hugetlb_shm
use
getent group hugetlb_shm
to get the gid. Then configure the gid under vm.hugetlb_shm_group
in /etc/sysctl.conf
, here our gid is 1001.
vm.hugetlb_shm_group = 1001
# minimum of 8 2MB pages with a total size of 16MB
hugeadm --pool-pages-min=2M:8
# maximum of 1024 2MB pages with a total size of 2GB
hugeadm --pool-pages-max=2M:1024
- Huge pages part 2: Interfaces
- Huge pages part 3: Administration
- Hugepages und the Debian Wiki
- hugeadm(8) - Linux man page
macOS calls their large pages "superpages", it should work out of the box.
int size = 2 * 1024 * 1024;
int flags = MmapFlagsMacOs.MAP_SHARED | MmapFlagsMacOs.MAP_ANONYMOUS | MmapFlagsMacOs.VM_FLAGS_SUPERPAGE_SIZE_2MB;
ByteBuffer buffer = Mman.mmap(size, flags);
try {
// use buffer
} finally {
Mman.munmap(buffer);
}