Installing 16-bit OS/2 in a virtual machine ranges between “tricky” and “impossible”, depending on the version of OS/2 and virtualization software used. In VirtualBox 4.0.8, things have moved further away from “impossible” and closer towards merely “tricky”. Version 4.0.8 fixed problems with floppy emulation which were making installation difficult, but plenty of hurdles still remain. Most of those have little to do with virtualization and are simply a consequence of OS/2’s age—the PCs of the late 1980s were not quite like today’s PCs.
When creating a VM for OS/2 1.x, the chosen OS type does not matter much. The defaults are generally not suitable. The following settings must or should be modified:
- The hard disk must be smaller than 512 MB. Sizes of 500 MB or less are usable (and 500 MB is still huge for 16-bit OS/2). Sizes greater than approximately 504 MB require geometry translation, which OS/2 1.x does not support.
- A floppy drive must be added to the VM—obviously, since the installation media are floppies.
- The emulated CD-ROM should be removed. There are no ATAPI CD-ROM drivers for 16-bit OS/2, and some versions of OS/2 take noticeably longer to boot when a CD-ROM is present.
- The memory size assigned to the VM generally does not matter. The minimum (4MB) is enough, and 16-bit OS/2 cannot use more than 16 MB RAM.
- Any virtualization mode should work—software, VT-x, or AMD-V.
In general, the difficulty of installing OS/2 increases with decreasing version number of OS/2—older versions are more picky. Let’s start from the easier to install versions and work our way back in time.
OS/2 1.30.1 or later, IBM or Microsoft
The following paragraph applies only to OS/2 1.30.1 and OS/2 1.30.2, not the original OS/2 1.3 release. Note that IBM often did not label the releases very clearly, and all 1.30.x releases display “Version 1.30” in the initial splash screen. A quick way to distinguish the original release is by the copyright date—the original 1.3 version shows 1990 copyright, while the updated versions show 1991.
Installation is straightforward as long as the above precautions are observed. For MS OS/2, either installation diskette A or B may be used. Either HPFS or FAT can be chosen for the system partition. Accepting defaults in the installer is recommended for users who do not have experience with OS/2 1.x, except that the mouse type must be manually set to PS/2. After feeding the requested floppies to the installation program, the initial setup is done and OS/2 happily boots from the virtual hard disk. Easy!
IBM OS/2 1.3
Installing the initial release of OS/2 1.3 is much harder because this and most earlier versions crash on machines faster than a 386:
To avoid confusing this issue with other potential problems, verify that it is in fact TRAP 0000 (not some other trap number), CX=0000, and AX=86A0. The contents of most other registers vary depending on host system configuration and the version of OS/2 in use.
The OS/2 keyboard driver contains timer calibration code which uses a simple software loop. On “fast” machines, the maximum number of iterations is reached before the timer ticks even once; note that due to this dependency, the OS does not always crash, and especially on slower host systems using software virtualization, the crash may not occur every time.
The crash is a division by zero (trap 0) on a DIV CX instruction where the CX register contains zero. The way around this problem is patching OS/2 and getting rid of the bad loop. The original code sequence looks like this:
MOV DX, 3 L1: XOR CX, CX L2: LOOP L2 ; loop 65,536 times DEC DX ; run the loop 3 times JNE L1 CLI ... ; read current tick STI SUB CX, BX MOV AX, 1F4h ; 500 MOV BX, 0C8h ; 200 MUL BX ; 500 * 200 = 100,000 DIV CX ; traps if CX = 0 SHR CX, 1
This obviously became a problem as far back as 1991, because OS/2 1.30.1 contains slightly different code:
MOV BX, 0C8h ; 200 MUL BX CMP CX, 0 JNE L3 MOV CX, 1Fh ; 31 L3: DIV CX
In other words, if CX is zero, it is forced to a non-zero value to avoid the division by zero trap.
As it turns out, the MOV/MOV/MUL/DIV instruction sequence is quite unique and occurs only in the BASEDD0x.SYS drivers. To avoid the crash, it is sufficient to remove the DIV instruction, replacing it by NOPs.
A simple Python script is available to do the hard work. The script can be run either on individual files (such as BASEDD01.SYS) or on floppy images. It is thus suitable for patching physical boot floppies as well as diskette images. The script creates backups of modified files, but should be used carefully.
OS/2 1.2, Microsoft or IBM
OS/2 1.2 requires patching—see above. Installation is not substantially different from OS/2 1.3.
Microsoft OS/2 1.1
MS OS/2 1.1 requires patching. In versions prior to OS/2 1.2, there was no merged BASEDD0x.SYS driver but rather several separate drivers. The driver which needs patching in OS/2 1.1 and 1.0 to avoid division by zero traps is KBD01.SYS.
The installation program was different, and HPFS was not an option. Note that some versions of OS/2 1.1 may still be unstable on fast host CPUs.
Microsoft OS/2 1.0
MS OS/2 1.0 requires patching to avoid the division by zero traps—see above. In addition, OS/2 1.0 crashes or hangs if the VM has more than 16 MB RAM. RAM sizes of 8 MB or less are safe, and more than sufficient.
FDISK must be run from the installer to create a partition for OS/2. Note that this partition cannot be greater than 32 MB, but FDISK can automatically create a primary partition of maximum allowed size. The installer must also format the partition.
MS OS/2 1.0 did not support PS/2 mice. OS/2 can be installed without mouse support (with barely noticeable loss of functionality), or a driver for PS/2 mice can be manually copied from a later OS/2 release.
IBM OS/2 1.0 and 1.1
Versions of IBM OS/2 prior to 1.2 use a 286-style method of switching from protected to real mode, at least on PC/AT-compatible systems. This method involves resetting the CPU and is not supported by VirtualBox (or most other virtualization products). These versions currently cannot be installed in a VM. Microsoft’s releases of OS/2 1.0 and 1.2 used a faster 386-specific method when a 386 or later CPU was detected. IBM’s releases probably did not use that method because at the time, IBM’s 386 systems were all based on the PS/2 architecture.