To understand why the maddeningly complex world of DOS memory managers and extenders came to be, it’s necessary to understand the evolution of the PC platform. Even though memory managers and DOS extenders reached their peak on 32-bit 386 and later systems, the foundation for their existence was laid with the original IBM PC.
To recap, the IBM PC, released in 1981, was built around the Intel 8088 chip. The PC was designed as a short-lived stopgap product and was meant to be simple and cheap, rather than extensible and future-proof. Of course it was exactly the low price and relative simplicity that made the PC into a major force, taking everyone–including IBM–by surprise.
The Intel 8088 (and 8086) processor was a late 1970s design, introduced in 1979. It had a 20-bit address bus, which meant it could address up to 1MB RAM. That was far more than the 64KB which the typical 8-bit microprocessors of the day (Z80, 6502) could address. In fact the choice of a 16-bit CPU was a major factor in the IBM PC’s success. The first PCs were sold with 16, 32, or 64KB RAM. Memory was very expensive and IBM initially did not even offer any expansion options to fully populate the PC’s address space.
Because the 1MB address space of the PC was seemingly insanely large, the designers did not worry about conserving it. The first 640KB (addresses 00000h-9FFFFh) were reserved for regular (so-called base or conventional) memory, directly usable by applications. Again, it should be understood that for several years, PCs could not even be upgraded to have 640KB RAM. The next 128KB (A0000h-BFFFFh) were reserved for video memory, followed by 128KB (C0000h-DFFFFh) reserved for miscellaneous adapter cards. The last 128KB (E0000h-FFFFFh) were reserved for the system ROM, which included the BIOS and built-in BASIC (the latter often not included on clones).
For a while, the address space was big enough and additional memory could be accessed directly without any tricks. But that did not last long. By the mid-1980s, users of memory-hungry applications such as spreadsheets (Lotus 1-2-3) started running out of memory and out of address space, meaning that more memory could not be added to the system. A new solution was required.
In 1984, IBM released the PC/AT, built around the Intel 286 processor. The 286 had 24 address lines and could directly address up to 16MB RAM—although the early PC/AT models were sold with 256KB or 512KB RAM, still under the 640KB limit. However, the PC/AT could be expanded to 3MB RAM from the beginning. Unfortunately, there was a catch. To address more than 1MB RAM, the 286 had to run in so-called protected mode. But DOS could not run in protected mode and the CPU had to be in so-called real mode, compatible with the earlier 8088 processor—including the 1MB addressing limit. Thus the AT was not a solution.
DOS requiring real mode wasn’t the only problem. In 1985, 286-based systems were relatively rare compared to the millions of IBM PC and PC/XT systems and clones already in use at the time. Users of those systems wanted more memory but were not willing to replace the entire system. A successful solution had to be compatible with the vast installed base of existing systems.
In 1985, the LIM EMS (Lotus/Intel/Microsoft Expanded Memory Specification) was created. The initial release was called EMS 3.0, for reasons that may be lost to history. EMS, or expanded memory, required a special memory board and a software driver which provided a uniform interface to applications. EMS utilized the well known technique of bank switching, which used a fixed “window” in the processor’s address space to selectively map a portion of memory on the EMS board. The initial EMS specification allowed for up to 4MB of expanded memory, which would have been extremely expensive at the time. Before the end of 1985, and updated EMS 3.2 specification was released, with support for up to 8MB expanded memory.
The EMS technology had the virtue of being usable on both old PCs and the newer 286-based systems. In the late 1980s, some AT-compatible chipsets (notably the Chips & Technologies NEAT chipset) even provided EMS-compatible hardware without the need for a separate EMS board.
But expanded memory wasn’t without problems. Bank switching was relatively fast since memory did not need to be copied, but it significantly complicated software design because the large memory was not accessible all at once. In addition, running code out of EMS was difficult.
Meanwhile, the PC/AT and compatible systems could use more than 640KB RAM, but in a completely different manner. In the early 286-based systems, additional memory was also installed in the form of adapter boards, but those were much simpler than EMS boards. The AT memory boards simply presented RAM on the system bus, without any additional logic. The challenge was using that memory from DOS.
IBM’s PC DOS 3.0, shipped with the PC/AT, came with a VDISK.SYS driver which allowed the use of so-called extended memory (memory above 1MB, not accessible from real mode). At the time, that was about all DOS users could do with any memory above 1MB. Of course most of the early PC/ATs didn’t have any extended memory, so the issue was initially somewhat academic.
As mentioned earlier, the CPU had to be in protected mode before accessing memory above 1MB. That was a problem because the Intel 286 processor was only designed to switch from real to protected mode, but not back. After a reset, the CPU always started in real mode. IBM solved the mode switching challenge by adding special circuitry which could reset the CPU without disturbing the rest of the system state (such as memory contents or device states).
In Intel’s defense, it must be noted that the 286 was introduced in 1982 and wasn’t designed with DOS in mind. Intel could not imagine why anyone would run a 286 in the crippled real mode, rather than almost immediately switching to protected mode and staying there.
To use extended memory, DOS-based software like VDISK.SYS had to go through a number of steps. First, the BIOS would be called to perform a copy operation between extended and base memory. The BIOS would switch to protected mode, then perform the copy, up to 64KB. Next, the BIOS had to reset the CPU, perform a limited re-initialization of the processor, and finally return to the caller. This process obviously wasn’t particularly fast. However, it was still blazingly fast compared to the alternative—using a disk instead of RAM.
A worse problem was that extended memory was entirely unmanaged. The BIOS provided no mechanism to allocate or free extended memory, only to copy to/from it. The designers of the PC/AT assumed that an advanced OS (such as XENIX) would manage the memory… but almost all users ended up running DOS, and DOS definitely didn’t qualify as an advanced OS. That created problems when more than one application at the same time wanted to use a portion of extended memory.
For a while (1985-1986), the deficiencies of extended memory weren’t pressing issues. Expanded memory was much more useful than extended memory, avoiding the non-existent extended memory management. Many memory boards (such as Intel’s Above Board 286) could provide extended memory, but would be typically configured to present EMS to the system instead.
- Extending DOS, Duncan et al., Addison-Wesley, 1990; ISBN 0-201-55053-9
- DOS and Windows Protected Mode, Al Williams, Addison-Wesley, 1993; ISBN 0-201-63218-7