The x86-32 Fedora kernel comes in two flavours. ‘normal’ and ‘PAE’. (There are -debug flavours of both of these two, but ignore those for the purpose of this blog post). The common wisdom surrounding ‘which kernel do I use’ has for the longest time been “How much RAM do you have?”

The kernel offers several configuration options to satisy this question. If you have less than 1G of ram, you don’t need anything special. After 1G, you need to enable CONFIG_HIGHMEM4G. (In actuality, it’s something like 960MB). As the name suggests, this allows you to run kernels that can see up to 4G of RAM. It does this by mapping in/out ‘highmem’ pages (those after the 1G mark) as they are needed. The ‘normal’ kernel for Fedora is HIGHMEM4G.

This sounds like all you’d ever need. Who has more than 4G of RAM ? Even today, typically only servers. However, that mapping/unmapping comes with some overhead, and if you’re using a lot of highmem, you may be better off with the third choice..

Enter PAE (Page Address Extensions). This option is enabled by the kernel config option CONFIG_HIGHMEM64G. Its name tellingly indicates it can support up to 64G of RAM in theory. I say in theory, because there are still a number of really nasty cases that need to be taken care of (Like: the pagetables need to fit in non-highmem memory, and by filling up lowmem with pagetables, you don’t leave much for other purposes). After a point, it just makes more sense to go to x86-64. PAE also gives us the ability to do hardware NX due to it having a wider PTE (page table entry) format.

But there’s another case where PAE is desirable which seems to catch out some people.
If you have exactly 4G of RAM, you may not be able to address it all, even with a CONFIG_HIGHMEM4G kernel.

PCI devices need some physical address space. How much, depends on what PCI devices are present, and how they are configured. This address space needs to live in the lower 4G of memory. So the BIOS carves out a hole in the memory map and says “PCI lives here”. If you had 4G of RAM, you may find the memory map now looks like..

3G RAM : 1G PCI ‘hole’ : 1G RAM

See the problem? To address that top 1G of RAM, we need to use addresses past 4G. And to use those, we need PAE.