Several years ago, after attempting to get a very old 286 version of Xenix running in a VM, I concluded that it was probably incompatible with any 386 and later processor. Recently I revisited this issue and examined the problem in detail.
The operating system in question is IBM Personal Computer XENIX 1.0. It’s historically significant, not so much because it was the second OS licensed by IBM from Microsoft, but rather because it was the first protected-mode OS available for a PC (the IBM PC/AT to be exact). IBM PC XENIX 1.0 was finalized around October 1984, just a few moths after the IBM PC/AT (“Salmon”) was introduced (August 1984). The PC/AT and PC XENIX 1.0 were in fact announced on the same day.
This flavor of Xenix is quite picky about the hardware it runs on. It was designed to run on the first-generation PC/AT with 20 MB fixed disk, and has trouble even on later IBM PC/AT models (different hard disks, EGA).
But the reason why IBM PC XENIX 1.0 can’t run on a 386 is different. It’s related to the way the OS manages the segment descriptor tables and it shows a lot about how it took Intel years to properly manage the x86 architecture in a forward-compatible manner.
Forward vs. Backward Compatibility
Everyone is familiar with backward compatibility—in this context, the ability of newer x86 processors to run software written for older x86 processors. Backward compatibility is sometimes difficult to achieve, but it is a well defined problem because the behavior of older processors (and software) is known.
Forward compatibility is in some ways the inverse and entails designing the x86 architecture in an extensible manner. It is far more difficult to achieve because the future is by definition unknown. However, over the years, certain practices and techniques emerged which make forward compatibility if not guaranteed then at least much more manageable .
On the CPU level, the key elements are CPUID bits which identify the availability (or not) of individual features, as well as control registers which must often be used to explicitly enable new features.
Another—perhaps less obvious—ingredient is enforcing correct usage of reserved bits. That is, reserved bits are typically MBZ (Must Be Zero) and attempts to set them will cause faults (usually general protection faults). In other words, even if a bit has no function in a given CPU, it must be left unset. This strategy ensures that when previously reserved bits are assigned a function, existing software will not suddenly trigger the new (and unanticipated) behavior.
Although this approach may seem obvious, it’s clearly not; it’s something Intel had to learn by mistake.
IBM PC XENIX 1.0 Descriptor Usage
The IBM PC XENIX 1.0 is an excellent case in point. Being a protected-mode OS, it manages a GDT (Global Descriptor Table) as well as a per-process LDT (Local Descriptor Table). Recall that descriptor tables contain the segment base address, limit, and attributes.
The descriptor format is very similar on the 286 and 386; the 386 version is an extension of the 286 one. In both cases, 8 bytes are reserved for a descriptor, but on the 286, only 6 bytes are used—the base address is only 24-bit, not 32-bit, the limit field is shorter, and there are fewer flag bits. The 286 documentation available at the time PC XENIX 1.0 was written (e.g. the 1983 iAPX 286 Operating System Writer’s Guide) very clearly states that the last word is “reserved for iAPX 386” and “must be zero”.
Unfortunately, the CPU did not enforce that, and allowed the Xenix developers to create a wonderful landmine bug, not detectable on a 286 without a great effort but ready to blow up spectacularly on a 386.
There at least are two basic issues. The Xenix kernel uses the
mmudescr routine to build a descriptor; this routine only writes 6 bytes, not 8. That wouldn’t be a problem per se. But various other routines (such as
getxfile which loads an executable) use the
copyseg routine to copy 8 bytes when moving descriptors around. In various places, the kernel builds a descriptor on the stack, writing only 6 bytes. Then it copies 8 bytes to the GDT/LDT, with the last word containing random garbage that just happened to be on the stack.
This does not even necessarily produce invalid descriptors. It can happen that the descriptor ends up being valid, but with a base which points far beyond the end of available RAM. That of course makes the segment unusable, although in a way that does not immediately trigger faults.
It’s difficult to say whether Intel was aware of this particular case; it’s entirely plausible that it was, or if not then at least of some analogous example. The 386 documentation says: “All the descriptors used by the 80286 are supported by the 80386 as long as the Intel-reserved word (last word) of the 80286 descriptor is zero.” That strongly suggests the problem was known.
It was of course far too late to fix the 286 design—that would break the existing software. On the other hand, it would have been much more difficult to blame Intel when software which explicitly broke the rules didn’t work on the 386.
It would seem that it took Intel surprisingly long to learn from these problems—for example, it wasn’t until the Pentium that the CPU prevented writing reserved bits in the CR4 register or setting reserved bits in page tables. It is also possible that Intel deliberately did not enforce the rules so as to not break some existing software, although that is no excuse for not forcing correct reserved page table bit usage on the 386, when the paging structures first appeared.
In the end, Intel no doubt realized that the biggest beneficiary of a forward-compatible x86 architecture is Intel. If software incorrectly sets reserved bits, Intel either can’t use those bits or existing software fails on new processors, sometimes in very unpleasant ways.
That’s not to say software developers are blameless. In some or perhaps most cases, they are simply sloppy (see the Xenix example above) and unintentionally set reserved bits. In other cases, they might be too clever for their own good, for example using reserved bits in page tables for their own purposes (I am currently not aware of any real-world example, but some likely exist).
Enforcing the rules ends up helping everybody. It doesn’t cause any difficulties for programmers, just helps them write correct code. It helps users because their software is more likely to run on future processors. And it avoids a few bad headaches for Intel.