It seems to me that 64-bit computing is the wave of the future with very little effort to adopt, yet from where I stand, not as many companies are going the 64-bit route as I would have thought.
Nearly every processor sold, if not every processor sold, is of the AMD64 or EM64T (IA-32E) architectures. Memory prices are continuing to drop making it more and more common to see x86 (or technically x86_64) based systems with 32GB, 64GB, or even 96GB of RAM.
Through Physical Address Extensions, (PAE), 32-bit processors have long been able to address more than 4GB of physical memory, given proper OS support. The addition of just 4 bits of memory additional addressing allows a 32-bit processor to support up to 64GB of RAM.
In Red Hat Enterprise Linux (hereafter referred to as RHEL), support was added for up to 64GB of memory in RHEL4 via the hugemem kernel. The RHEL4 Release Notes state that the hugemem kernel provides a 4GB per-process address space and a 4GB kernel space. It is also noted, though, that running the hugemem kernel will have a performance impact as the kernel needs to move from one address lookup table to another when switching from kernel to user space and vice-versa. It is not stated in the release notes, but I have heard conjecture that the performance impact could be up to 30%.
RHEL5, the latest major release of Red Hat Enterprise Linux actually removes support for the hugemem kernel. 32-bit RHEL5 will support at most 16GB of memory. See the RHEL Comparison Chart for details. I cannot find specific references as to why hugemem was removed in RHEL5 but I have heard that the performance impact of hugemem was a hassle to deal with from a support perspective. (Not to mention the assertion that 32-bit is dead!).
So, if a user is running 32-bit RHEL4 with greater than 16GB of memory their upgrade path to 32-bit RHEL5 is limited by the 16GB maximum in RHEL5. One would have to do a fresh 64-bit install of RHEL5 to take advantage of the increased memory.
64-bit on the other hand, can address a full 2TB of memory. There is also no longer the distinction between LOWMEM and HIGHMEM for the kernel. The elimination of the 1GB/3GB split (or the 4GB/4GB translation with hugemem) increases the stability of the kernel when dealing with memory intensive loads. I’ve seen many cases where 32-bit systems with 8, 12, or even 16GB of RAM have fallen over because the kernel can no longer assemble contiguous blocks of memory fast enough from its limited 1GB address space, while HIGHMEM sits with many GB of useable memory pages.
In cases like this, a migration to 64-bit has nearly always resolved the issue with kernel memory starvation. (In a couple of cases there was a runaway app that consumed every available page of memory on the system, so there was no difference between 32 or 64 bit.
The moral of the story? Go 64-bit with any new server implementations. Begin putting plans into place now to migrate legacy 32-bit systems to 64-bit in the near future. Having a solid, actionable plan will go a long way to ensure a smooth transition.