The 4Gb Memory Map Limit

I've numerous times been asked about the '4Gb memory limit' with regards to Windows operating systems (Well, more commonly for Windows, obviously it's applicable to other OS' too on a 32 bit system)....So, I thought I'd take the time-out to answer it once that way I can just send out a link, rather than regurgitating or pointing people at Google.

So, what is the 4Gb limit? We all know what RAM is right, so I won't go into that. First, where does the limit come from? A 32 bit operating system addresses memory using 32 bits - this means that it can address 2^32 bytes of information. 2^32 is 4,294,967,296 bytes or 4Gb. This is where the hard limit of 4Gb comes from - the 32 bit address space.

In the early 90s when the first 32 bit OS' were coming about the 'big memory' capability was considered huge - 4Gb memory space when the average machine had 4Mb - well I'm sure you can see the point.

Of course now days desktop machines, server etc. with 4Gb of RAM are not that uncommon (hell my Macbook has 4Gb in it) and we're starting to hit this addressing limit.

Why can't my machine see 4Gb of RAM? 

One of the most common questions I get is that of somebody who's just upgraded their 32 bit Windows XP32 to 4Gb and they cannot 'see' the 4Gb. Typically they'll see 3.25Gb or 3.5Gb. Why is this?

Well - the limit of addressing for a 32 bit OS is 4Gb.. What's often forgotten is that this 4Gb of space may actually contain other items except memory. Typically this is where hardware uses something called 'Memory Mapped I/O'. This is where a device maps itself into the main address space for performance purposes - it's significantly faster than using the traditional bus method.

This can be misleading - the system STILL can access a total of 4Gb address space but you may not have the full 4Gb available to you.

If you want to have a look at any memory mapped I/O on your machine you can do it from Device Manager. Open it and select 'Resources by  Connection' on the view menu. You can then expand the 'Memory' tab and see what hardware devices are mapped into your memory space.

You'll see the address space used...Remembering that the system can ONLY address 4Gb you can see why some of your 4Gb just appears to disappear.

You'll always see for example 000A0000-000Bffff mapped - this is a long held convention mapping for memory cards.

You can if you want change this mapping by changing the 'Aperture' setting of your video card (assuming it supports it)...128Mb is a smaller 'hole' than 256Mb after all.

So, that's why you rarely get ALL of your 4Gb of RAM available when you upgrade & use a 32 bit OS. Brain hurting?

What about Terminal Services and Citrix?

Again, another common one. I often come across machines that have had more than 4Gb put in them, and often 'tuned' to allegedly 'support' this configuration...the client then complains of poorer performance than before the upgrade. To understand this, you need to understand how Windows utilizes memory.

Firstly, to be clear we're now talking about virtual memory mapping, not the physical we were previously! This is important, try and keep up.

Each application effectively has it's own 4Gb virtual memory map - this is divided into a Kernel area of 2Gb and 2Gb for the application. Now this is where you start to get scaling problems with Citrix/TS on a 32 bit server - essentially all of the shared session information has to be put into the 2Gb kernel mode area. This pretty much limits the scaling capability of a Citrix/TS server.

So, why is it that different versions of Windows 2003 server say they can address more than 4Gb of RAM? Well they can, but they use buggery to do it. Consider the options:

/3GB (4Gb Tuning)

This is a switch that can be added to the BOOT.INI file. This is astonishingly bad for Citrix/TS environments - for full details of how it works see here:  http://technet2.microsoft.com/windowsserver/en/library/edc9f27d-76fb-4139-9555-20acc684c3af1033.mspx?mfr=true

In summary /3GB switches the virtual memory layout to 1Gb for the Kernel, and 3Gb for the application. This is bad because it will actually limit the scalability of your server as you'll be able to house less sessions before you run out of kernel space.

/PAE (Physical Address Extension)

This is the method often cited for allowing certain editions of Windows to support up to 64Gb (I think, could be less. I'll check). Now, this again is bad in Citrix/TS environments as it again limits your kernel space. Or rather more correctly put increases the loading in your kernel space. The reason for this is that memory above the 4Gb hard-limit is mapped into tables within the 4Gb limit. In essence the address space is increased to 24 bits (2^24 = 64Gb).

Now, the problem with this is that each time memory is mapped above the limit you require an additional Page Table Entries (PTE) are required in the kernel space - reducing your kernel space and therefore reducing the scalability of your Citrix/TS farm.

I've been to sites where they've enabled PAE for more than 4Gb RAM where removing the extra RAM and disabling the PAE resulted in a positive performance increase for the users and additional scaling for the server. Go figure.

So what's the answer?

Well the obvious one is to move to 64-bit for larger Citrix/TS implementations. I'm not sure why there's been such a slow take up of 64 bit in this arena. I know there was confusion around the Itanium set, driver compatibility rumours etc. but it just makes sense in larger environments to be running with x64.

MS are pushing this more and more now with their enterprise products - Exchange 2007 only runs operationally on x64 for example. Yes, I'm aware there's a 32 bit however that's for support/testing/evaluation and is not for live environments.

So, there you have it. I'll review this and possibly put a bit more detail in at a later date. I'm off for a lay down.