I was recently involved in investigating a problem that turns out to be a complete SNAFU which nicely illustrates the chaos that is the PC platform. It’s about the NX/XD bit. Let’s start with a bit of history.
It turns out to be useful, and in fact required for improved security, to separate code and data. On the 8086, no such separation existed. On the 80286, protected-mode segmentation clearly distinguished between code and data segments. Writing to a code segment required creating a data-segment alias. The same segmentation rules carried over to 32-bit protected mode on the 80386, but lost any meaning in flat-model operating systems (OS/2 2.0, Windows NT, 32-bit UNIX). Because the data (DS) and code (CS) segments mapped the entire memory, any data page could be written to and executed. In the age of Internet, that turned out to be a terrible idea because any buffer overflow on the stack could be immediately exploited to inject and execute code in a vulnerable process.
In 2003, AMD introduced the NX (No-eXecute) bit as part of the AMD64 architecture. When enabled in the EFER MSR, 64-bit and PAE page tables support an additional bit which makes a page non-executable. As a result, it is again possible to separate between code and data, and make data pages non-executable. That makes exploits harder (far from impossible!) because there is no longer a ready supply of writable and executable pages and most importantly, the stack is no longer executable.
Microsoft saw NX as a Good Thing and added support for it in a Windows XP SP2 (2004), calling the feature DEP, or Data Execution Prevention. This was initially an opt-in feature because certain classes of legitimate software did not bother distinguishing between data and code and crashed with DEP enabled (typically software which generated code on the fly in one manner or another).
Microsoft presumably made it known to Intel that NX was a desirable feature. Intel had to support NX in all AMD64-compatible CPUs, but also added NX support to late-model 32-bit Prescott Pentium 4 CPUs, and even the later Pentium M processors.
Intel of course couldn’t leave well enough alone and called the bit XD (eXecute Disable), and more importantly, also added a way to completely disable the NX through the IA32_MISC_ENABLE (1A0h) MSR. That’s where things started going south. Continue reading →