Investigating the rather odd behavior of the Microsoft OS/2 1.21 disk driver led me to Compaq and their EXTDISK.SYS driver. While experimenting with various setups, I realized that DOS versions older than 5.0 do not support more than two hard disks exposed by the system’s BIOS, and will in fact quite likely hang early during boot-up if there are “too many” hard disks.
This seems to have been one of the many things that “everyone knew” back in the day, similar to the fact that DOS versions older than 3.3 may hang while booting from disks with significantly more than 17 sectors per track.
As was the case with the “too many sectors per track” problem, the issue with “too many hard disks” was missed for years simply because no one had a PC with more than two hard disks. This was a technical rather than architectural limitation. While the IBM PC/XT and PC/AT BIOS implementations were limited to two hard disks, the INT 13h interface as such was not.
In the days of full-height 5¼” drives, it simply was not feasible to install more than two hard disks into a PC, especially when a 5¼” floppy drive was also required. Even the big IBM PS/2 Model 80 (1987) with a tower case could only house two full-height 5¼” drives. There might also be trouble with the power supply, as the PC hard disks of the time were not designed for staggered spin-up and a standard AT power supply might have trouble spinning up four drives at the same time.
Sure, there were half-height hard disks, but who wanted four drives in the first place? People who needed to maximize the storage capacity… and the most obvious way to do that was buying a large capacity drive, which in the 1980s was inevitably a full-height 5¼” monster. Like my 1988-model 650 MB ESDI drive, for example.
Yes, there were solutions like the NetWare DCB which supported many drives, but those were only usable by NetWare and did not expose the drives via INT 13h.
Two things happened circa 1988. One was Compaq releasing the Deskpro 386/25 with an expansion unit option, a system which supported up to four hard AT-style disks (that is, the expansion unit housed up to two ESDI drives accessible via the PC/AT hard disk style programming interface, which may be called WD1003 or WD1010 or several other things). The other development was Adaptec releasing the AHA-1540/1542 SCSI HBA, and there were perhaps other SCSI vendors as well.
Compaq supported up to four hard disks, Adaptec in theory up to seven. In any case, it is apparent that both companies ran into the same problem with DOS, and solved it in a very similar manner.
Compaq simply did not expose the drives in the expansion unit through the BIOS at all. DOS users needed the EXTDISK.SYS driver, and users of other operating systems (such as OS/2 or NetWare) needed a custom driver.
Adaptec was in a more complicated situation. The AHA-154x was an add-on card which could be installed in a PC/AT compatible machine (the AHA-154x did not work in older systems because it was a bus-mastering adapter) that already had one or two AT style drives. The AHA-154x BIOS keeps the total hard disk maximum to two. In practice that means that if there are two SCSI hard disks attached to an AHA-154x (which also includes AHA-154xA and AHA-154xB, but not necessarily newer models), the Adaptec BIOS may add zero, one, or two drives to the system, depending on how many hard disks there are already installed. In any case, the total won’t be greater than two.
For DOS users, Adaptec offered a combination of ASPI4DOS.SYS (ASPI driver for the AHA-154x) plus ASPIDISK.SYS (DOS hard disk device driver). Adaptec’s ASPIDISK.SYS was functionally very similar to Compaq’s EXTDISK.SYS and allowed DOS users (especially users of DOS 4.x and older) to utilize more than two hard disks.
DOS Bug
The bug is quite visible in the MS-DOS 4.0 source code. In MSINIT.ASM (IO.SYS/IBMBIO.COM module), DOS calls INT 13h/08h and stores the number of disks in the HNUM variable. No attempt is made to validate the value returned by INT 13h.
Further down in MSINIT.ASM, DOS sets up the hard disks, calling the SETHARD routine for each drive, but it will not set up more than two. Trouble will start near the SETIT label, where the DRVMAX variable may end up with a number much higher than the number of drives that SETHARD was run on.
Eventually, disaster strikes in the $SETDPB routine in the DOS kernel. The code near LOG2LOOP label attempts to calculate the cluster shift for the FAT file system, but gets stuck in an endless loop because the BPB for a drive was never initialized and contains zeros.
This bug is present in every DOS version with hard disk support before 5.0, that is, in DOS 2.0 up to and including DOS 4. In my experiments, all these DOS versions hang when booting on a machine that exposes four BIOS drives. MS-DOS 4.01 from April 1989 still hangs, and so does Russian MS-DOS 4.01 from February 1990.
It is clear that the bug went unnoticed or at least unfixed for a number of years simply because PCs with more than two hard disks were extremely rare to nonexistent.
DOS 5.0
It is likely that DOS 4.0 (1988) was released just before PCs with multiple hard disks became a thing. By the time Microsoft started working on DOS 5.0 in earnest in 1990, EXTDISK.SYS and ASPIDISK.SYS were certainly well established, and the problem must have been known.
MS-DOS 5.00.224 Beta from June 1990 (the oldest DOS 5.0 beta I could test) does not suffer from the bug described above, and shows four hard disks exposed by the BIOS in FDISK.
Further related work was done in August 1990 with the following comment:
M011 8/07/90 CAS msinit.asm rewrote lots of spaghetti code
msbio1.asm used for initializing hard drive
partitions. fixed bugs 2141, 2204,
1866 and 1809 and prepared for
zenith/etc. support
The above is an excerpt from an MS-DOS 5.0 OAK (OEM Adaptation Kit). The first entry in the relevant file (MSBIO.TAG) is dated 7/17/90 which leaves open the question of who actually fixed the problem with more than two hard disks and when, since it must have been fixed by June 1990.
There is another rather curious data point:
The above screenshot shows that Japanese IBM DOS 4.05/V does not hang and FDISK correctly shows four hard disks. Here’s the boot screen of said DOS version:
This shows that the fix made it into at least some DOS 4.x code base. However, the system files in IBM DOS J4.05/V are dated October 1990, decidedly newer than the MS-DOS 5.00.224 Beta.
SCSI HBAs
In any case, the fix was well known to SCSI HBA vendors. Starting with the AHA-154xC, Adaptec offered an option for “BIOS Support for More Than 2 Drives (MS-DOS(R) 5.0 and above)”. When this option was disabled, the BIOS keep the total number of hard disks to no more than two, just like AHA-154xB and earlier. When enabled, the Adaptec BIOS would expose all the hard disks it would find as BIOS drives 80h, 81h, 82h, 83h, 84h, etc.
BusLogic adapters offered a more or less identical setting to solve the identical problem.
When this setting was enabled, DOS 5.0 and later no longer needed ASPIDISK.SYS or any other vendor specific driver. DOS itself could directly use the BIOS to access all hard disks in the system (limited by the number of available drive letters).
I believe clone BIOSes with support for more than two IDE hard disks generally started appearing only since 1994 or so, and assumed (not unreasonably) that the user would be installing DOS 5.0 or later. In the worst case, the BIOS could usually be set up to not detect the 3rd and/or 4th hard disk. It was the SCSI HBAs that were prepared to deal with trouble.
APAR IR86346
Completely by accident, the puzzle of the DOS fix was solved while I was looking for something totally unrelated. In an IBM announcement letter from October 1990, the following sentence jumped out at me: DOS 3.3 and 4.0 support up to two fixed disks in a system. DOS 4.0 supports up to seven fixed disks when corrective service diskette (CSD) #UR29015 is installed.
I happened to have CSD UR29015 on hand, so I looked at the included documentation. The README file states: APAR IR86346 requires DOS 4.0 to be installed with NO MORE THAN two fixed disk drives before installing corrective service. Once corrective service is installed, you can attach the additional fixed disk drives.
In the APARLIST file there’s a table which includes the following entry:
CSD APAR KEYWORD COMPONENT ABSTRACT ------- ------- -------- --------- ----- ... UR27164 IR86346 ABEND IBMBIO DOS 4.0 hangs with more than 2 hardfiles ...
Yep, that’s exactly the observed problem! With more than hard disks, DOS 4.0 and older simply hangs.
There’s also a table of fix releases in the same file (excerpted):
...
CSD UR25066 05/10/89
IFD UR25788 06/07/89
IFD UR27164 09/25/89
IFD UR27749 10/11/89
...
CSD UR27164 which (was the first to include the fix for APAR IR86346) was released on September 25, 1989. The previous CSD from June 1989 did not include the fix.
The documentation does not lie and with CSD UR29015 applied, IBM DOS 4.0 has no trouble booting up and seeing four hard disks:
That clarifies the timeline a lot. MS-DOS 4.01 from April 1989 could not possibly contain the fix. IBM fixed the bug sometime in Summer 1989, which is why IBM DOS J4.05/V includes the fix. Microsoft’s Russian MS-DOS 4.01 was likely branched before mid-1989 and the fix was never applied.
And this also explains why the earliest MS-DOS 5.0 betas don’t have the problem with more than two hard disks, even though there is no record of Microsoft fixing it. Because Microsoft didn’t—IBM did, a few months before the work on MS-DOS 5.0 started.
The only minor remaining mystery is who opened APAR IR86346. It could have been an external customer, although both the Adaptec AHA-154x HBA and the Compaq Deskpro/25 were designed to protect against DOS hanging. Then again, perhaps some other SCSI HBA was not quite so careful and could trigger the hang with multiple hard disks.
It is also possible that the bug was discovered and internally reported when IBM was working on its own SCSI adapters, released in March 1990 together with the first wave of PS/2 machines with SCSI drives.
Fascinating find! I’m sure IBM discovered the bug when they were in the process of introducing SCSI adapters and hard disks as standard equipment. But that CSD is mentioned even earlier than the October 1990 round of PS/2 Model 80 refreshes. The announcement letter dated March 20, 1990 for the Model 80 -121 and -321 which used the Type 2 planar also announced an increase of available storage bays. Your corrective service diskette statement is also part of this letter! It can be found at https://ardent-tool.com/8580/190-054.txt
The letter states:
“The fixed disk controller is the newly introduced Small Computer System Interface (SCSI), the PS/2 Micro Channel SCSI Adapter. This bus master adapter provides additional expansion capability while providing an interface for the new 3.5-inch, half-high SCSI fixed disk drives of either 120MB (8580-121) or 320MB
(8580-321).”
Further,
“Design enhancements to these systems offer significant advantages in configuration flexibility and expansion. In addition to providing seven available adapter slots, the standard configuration will support up to five internal Direct Access Storage Devices (DASD) and removable media, such as diskette drives, fixed disk drives, tape backup and CD-ROM. An optional feature allows support of up to six internal DASD devices.
The standard fixed disk drive configuration of the Model 80 121 and 321 contains one SCSI fixed disk drive located in one of the two non-accessible bays. The other non-accessible bay contains the necessary hardware to install a second IBM SCSI fixed disk drive option. The 5.25-inch, full-high bay can be converted into two 3.5-inch, half-high bays through the use of the optional Fixed Disk Drive Kit A (#1053) (6451053). This conversion allows the installation of a third and fourth IBM SCSI fixed disk drive option.”
The five bays come from the two externally accessible 3.5 inch bays, the externally accessible 5 1/4 bay (position D), and one dual 3.5 bracket installed in the rearward drive bay slot (position C). Installing a second dual hard drive bracket in Position D brings it up to six bays. As you pointed out earlier, the towers previously used full height ESDI or MFM drives, and those controllers only supported one or two attached disks. The Type 1 planar (the 1987 models) only mention accommodating a second fixed disk in their announcement letter: “Additional positions for a second fixed disk drive and for a second 3.5-inch diskette drive.”
So this March letter announcing the PS/2 Model 80’s new drive capabilities and the corrective service diskette is probably the earliest mention of IBM supporting more than two hard drives. The advent of SCSI supporting many devices on one adapter along with denser 3.5 drives allowed IBM to stuff more disks into the tower and then bam, they hit the bug. At least, that’s a logical guess to me.
Yes. The October 1990 announcement was simply the one I found first. Logically the systems (and SCSI adapters) announced in March 1990 needed the same fix.
It is possible that someone external reported the problem to IBM, but it’s more likely that IBM ran into the problem internally when developing their own SCSI adapters.
I agree. I love whenever you post these deep dives into specific bugs. As a QA person it scratches every one of my itches.
IBM would’ve been developing the Tribble adapter in the spring-summer of 1989 since the earliest adapter BIOS dates from then. That lines up with the Japanese DOS release date and September release of the CSD, so I think your working theory of IBM discovering the bug internally has the most supporting evidence. I’m surprised Compaq didn’t bring the issue up with Microsoft, but you know how things get prioritized or how lack of source visibility can make an OEM misjudge a cause.
> In the days of full-height 5¼” drives, it simply was not feasible to install more than two hard disks into a PC, especially when a 5¼” floppy drive was also required.
I’m certainly sure than this was possible with the IBM 5161 Expansion Unit that was released before the PC/XT, so 1982? With PC or PC/XT + 5161 setup you had available four full height 5 1/4”, and while four HDs is unlikely because the reliance on disquettes, three seems perfectly feasible.
I had four hard drives. IBM DOS 3.3 would check each drive in turn, looking for a valid boot sector, and if it found one, it would boot from that drive. I regularly booted from drive E:
MS-DOS 5 – at least early versions – would only boot from C:. I mentioned it on the Microsoft DOS 5 CompuServe group, and the reply was, “Why would you want to boot from any other drive?” It turned out a lot of people did, but Microsoft’s official reply was ‘that’s the way it is now.’
I reconfigured my system to boot from C:, then switched to DR-DOS 5.0, which I ran (with DESQview and QEMM) until the late 90s when I got paid to write software to run under Windows 95.
I love these history posts.
I worked for small PC clone builder in the late 80′ and the first half of the 90’s, and I can’t remember ever having more than two MFM or RLL hard drives in a system other than a Novell Advanced Netware 386 server. That used an Adaptec 1542C. I know we updated at least one normal PC with a third IDE drive but that would have been a 486 and MS-DOS 5.0 so well supported.
I never gave it much thought, but the MFM and RLL controllers could be set to a primary address (1F0 h) and secondary (170 h). I just never though about how it would be used. Maybe something like Ontrack Disk Manager would help, the driver being loaded after the standard drives by MS-DOS.
Are you saying that you had 4 physical hard disks visible to the BIOS but only some of them had DOS-compatible file systems on them?
Yes the 5161 could hold up to two hard disks… but then what? What controller(s) would you use for more than two to be visible to the BIOS? I’m genuinely curious.
Yes, Compaq is a bit of a mystery. They clearly had a very good relationship with Microsoft. And even though DOS was managed by IBM at the time, I’m sure Microsoft could have done something.
One possibility is that Compaq found out too late to get it fixed in DOS 4.0, and at any rate they would have needed a fix in DOS 3.3 as well. It is conceivable that Compaq didn’t want to bother with that or Microsoft told them “we’re not doing that” (another DOS 3.x minor version), and/or “just use OS/2” (which did have the support built in!).
And once Compaq had a solution for DOS 3.x, it worked in DOS 4.0 just as well. It is also conceivable that they very much did not want to break booting from existing DOS floppies (which would happen if they exposed the extra drives to the BIOS).
About the only thing I can think of which could use hard disks on the secondary adapter was NetWare. In fact NetWare was the company which introduced the concept of tertiary and quaternary adapters.
Technically I am pretty sure you’re right and DM could have supported those, but I don’t know if it actually did.
DOS could use a hardcard assigned as a secondary controller if the Quantum documentation reflected reality. Needed the supplied hardcard driver to do it.
In the Digital Research world, DOS Plus 1.2 supports only one or two BIOS hard drives, with a single partition on each one, and doesn’t seem to probe for any others. Looking at the code DRDOS 3.40-5 seem to have a maximum of 16 mounted volumes but there’s no reason why each volume couldn’t be on its own BIOS hard drive. I haven’t put this to the test though.
Nice write up!
I think I wrote more or less this as a comment to the previous post, but:
IBM releasing the AT with a combined fixed+floppy disk inteface that could be configured as primary/secondary, but no software/firmware/drivers to support the secondary setting, is surprising. Did WD just add that setting without IBM requesting it?
But also, these bugs (both the amount of disks, and problems with more than 17 sectors) really shows that testing was bad at the time. It would probably not had been that hard to modify an AT bios to have the second hard disk show up multiple times, but write protected to ensure it won’t be corrupted, just to be able to test this. Also it would had been fairly easy to create a BIOS that would simulate more sectors, more heads or more tracks by some simple hard coded geometry translation.
No, this is not the fault of testing. QA tested products that existed, not products that didn’t. Sure they could have discovered these bugs, but at the costs of also spending a lot of time and resources varying all kinds of parameters only to find that there’s no problem there.
With the number of hard disks, it may be fair to blame the developers, they should have realized that more than two drives would be desirable. With the sectors per track… the code was written in what, 1982? At that point, hard disks with more than 17 spt were years off, and nobody could predict with any kind of certainty how hard disks would evolve.
It’s at least reasonable to test that code ignores unexpected input. In this case, if the number of hard disks were more than two, it would just round down to two.
While it would had been highly unusual, the large CDC drives had more than 17 tracks. The terrible Google AI summary says that a Hawk had 32 sectors per track while a Phoenix had 64 sectors per track.
What was predictable was that the BIOS could tell running code that there are a certain number of floppies, hard disks and that they have certain geometries, and the space for storing this information had a fixed size and thus it’s reasonable to test between zero and max of each of these.
My gut feeling is that Microsoft didn’t have any test setup, they just assembled and ran it on a few different PCs they had. On one hand I get why this would be the case, but on the other hand they had grown big enough to be able to afford to do full tests.
Or maybe it was an actual policy, to make sure that people had to upgrade in the future?
In the late 80s, IBM was obsessed with limiting the capability of the Personal Systems to protect the AS/400 market. Comparatively small hard drives with limited connections all played into it and not testing products not shipped from IBM made sure it was harder to go past the IBM designated limits. Remember DOS 4 was conspicuously not tested on software from outside of IBM which had a similar effect.
Yeah, but IBM wasn’t the only customer, and Microsoft wasn’t interested in protecting IBMs or other manufacturers larger systems.
I.E. Microsoft had incentives to actually make their products work on more expanded systems.
MS had reduced DOS development staff during the PC-DOS 4 era. Most OEMs were wanting an MS-DOS that was as close to IBM’s as possible to go with the nearly identical clones being produced. Compaq and a few other OEMs were willing to go beyond what IBM chose to limit themselves to. The specialized drivers were the responsibility of those OEMs as was any support burden caused by those drivers.
MS changed things with the run up to DOS 5 as following IBM’s lead became less important.
(note that early Compaq systems however did not have BIOS support for more than two drives)
(though note that it was IBM that decides to support more than two drives and not MS)
i hope that this fix is pushed on to the DOS 4.00 source code availible on github.
@Abrahan: I fixed this partially in lMS-DOS / lDOS, my fork of the 2024 April release, which is available from my website at pushbx.org
In particular, multiple hard disks are supported, but the kernel probably will only log in the first primary partition and a single chain of logical partitions in the first extended partition (type 5 or type 0Fh) per hard disk, where “first” relates to the order of the partition table entries.
I do plan to eventually support a wider array of configurations.
Don’t forget search engines and SSDs as well.
(Google was started in 1996 and DOS 5.0 was released in 1991)
Did you see the article “The Man Who Beat IBM” that claims that after a certain point, MS-DOS was actually being developed by Compaq? https://every.to/feeds/b0e329f3048258e8eeb7/the-man-who-beat-ibm
The article makes a number of unsourced and vague claims that sound quite wrong, but without knowing the details it’s difficult to be sure exactly what’s true and what’s not.
One example is claims that IBM didn’t care about backward compatibility and that the PC/XT and PC/AT were significantly incompatible with the original PC. That’s just not true; IBM cared very much about backward compatibility. Of course with any changes, something can break, and maybe it did. Without knowing the precise details, it’s difficult to judge such claims, but overall they sound untrue.
Another example is this quote: “Both men knew why: Microsoft DOS had never been a true copy of PC DOS, as Gates had admitted to Canion during the development of Compaq’s first machine. The differences had only increased over time, as Microsoft’s deal with IBM prohibited the same developers working on both versions.” I don’t know what this is supposed to be about. We know for a fact, because we have the source code, that the DOS kernel could be built “IBM style” or “Microsoft style”, but there weren’t two separate DOS versions. Early on, most clones used “Microsoft style” MS-DOS, while Compaq went for maximum compatibility. But there weren’t two separate versions.
Or here: “Even people who owned other PC clones were choosing to buy that [Compaq DOS] instead of Microsoft’s own public version.” What is this talking about? Until about 1986, there was no “Microsoft’s public version” of DOS! But I fully believe that clone owners might have preferred Compaq DOS (or PC DOS!) to whatever MS-DOS version came with their clone.
I assume this is what you’re mainly talking about: “From Gates’s perspective, this was an incredible deal. He was able to halt all internal development on Microsoft DOS, saving time and money. From this point onward, every version of Microsoft DOS he sold was, in fact, Compaq DOS, with the digital equivalent of its serial numbers filed off.” I will rate that claim as “bullshit”, plain and simple.
Of course Microsoft kept developing DOS. Are we supposed to believe that, for example, DOS support for the IBM PC/AT was secretly developed at Compaq, and IBM never noticed? That Microsoft shipped NDA-ed IBM prototypes to Compaq behind IBM’s back? Uh, nope, for one thing Compaq’s own lawyers would have killed anyone trying to do that (as the article makes plain)!
I think the article is based on stories that were told and retold over the years, and although there is some truth to them, the stories morphed so far that they’re quite detached from reality.
There is no doubt that Compaq did work on DOS, and that Microsoft integrated code developed by Compaq into DOS. One obvious example is EMM386 (originally CEMM at Compaq). It is quite possible that some DOS utilities also came from Compaq. However, we know for a fact (again, because the source code is available) that by the time DOS 4.0 came around, the utilities like KEYB or XCOPY or MODE were IBM’s from PC DOS.
There may well be other bits of DOS that had been developed at Compaq. It is known that Microsoft and Compaq had a good relationship, and if Compaq needed something in DOS, they were likely able to get it.
As an aside, it is actually true that a certain company took over the development of DOS and let Bill Gates charge royalties to other OEMs… but that company was IBM (in 1986), not Compaq.
That article is particularly silly since we have enough of the DOS 4.0 source code with comments and dates at the above of most of the files to see the revision history and can see that both IBM and Microsoft employees contributed to it.
COMPAQ’s 3.31 series was essentially a fork that wasn’t maintained after that point; if it worked, it worked, and people used it well into the 1990s; it was a sensible choice if you wanted perks like >32MB disks without running RAM-hungry 4.0.
We can see Microsoft put in a lot of (mostly wasted) effort into 4.0, as did IBM, much of which got thrown out for 5.0, which seemed to be chiefly a Microsoft undertaking.
It’s obvious Microsoft and COMPAQ worked together closely. DOS 4.0 pretty much had the feature set 3.31 had, for example. And COMPAQ and Microsoft weren’t competitors at all – COMPAQ was an OEM of MS-DOS back when Microsoft didn’t sell directly at all.
The article was kind of silly about claiming COMPAQ somehow “beat” IBM. COMPAQ simply got a 386 machine that worked and was AT compatible out the door faster. And their OEM MS-DOS worked better with large drives than IBM’s bloated 4.0. Yet at the same time, COMPAQ was putting a lot of effort into MS OS/2 support; machines like the Portable III (286) have full OS/2 1.1 support, for example.
Yes, I found the claims about DOS extremely dubious. It seems to refer to a period starting circa 1982, which is just not at all credible. It all kind of sounds like CEO talk, stories from people who were rather distant from the reality on the ground.
The thing that got thrown out in DOS 5.0 was the IBM Shell, and the IFS interface. Most of the rest of the OS was actually kept, but shoving a big chunk of DOS into the HMA was what made all the difference.
I think the claim about beating IBM referred to EISA killing Microchannel. Which it did. Of course Compaq itself didn’t outlast the MCA by that much…
It was the PS/2 and DOS 4 where IBM really didn’t care about supporting third party hardware or software that worked on earlier PC models. The EMS buffers are the easiest example to provide but other software also failed to work. Okay, IBM made a first attempt with the PC Jr to restrict what could run but that also failed badly.
IIRC, the published DOS 4 plan was IBM had the lead with MS only supplying a small team to correct for any IBM restrictive elements like IBM’s famous desire to only read hard disks formatted by IBM tools. Obviously, MS needed to put a larger staff together to get to a marketable 4.01.
There was one other change between DOS 4 and DOS 5. Share was split out of the large hard disk support freeing up some additional memory for anyone that didn’t need networking.
I agree that that article seems to build on hearsay that has been distorted a bit now and then.
Although HP bought Compaq, judging by the hardware it actually seems like HP bought Compaq and then discontinued the HP PC products and renamed the Compaq PC products to become HP.
It’s interesting to think about which prats of companies were of interest and not when a company bought another company. Like how Compaq sold the DEC Alpha architecture to Intel, while AMD since earlier held a license for the Alpha CPU bus EV6 that they used for their Athlon processors. Also HP/UX last release was in 2007 while DEC OSF/1 / Digital Unix / Tru64 had it’s last release in 2010. VMS still lives on in an independent company, while DECNET is more or less dead.
I wonder what Compaqs plan was when they bought DEC? Getting to sell Compaq PC hardware to the probably large amount of existing DEC customers, both for servers and workstations?
Could it had been a power move to leverage in negotiations with Intel and/or AMD and/or other chipset manufacturers? Those who created the Alpha architecture would for sure had been able to create their own chipsets for Intel and AMD CPUs, and possibly also an x86 compatible processor.
Maybe VMS and porting it to Itanium was a tactic to get better prices for Intel hardware by making Intel hope that VMS would continue to sell great? (It kind of feels like both Compaq and Intel long term shot themselves in the foot here – Itanium flopped and VMS not being available for common CPUs until recently ensured that it only ever became a niche.
While I’m veering far off the thread with this tangent, it’s also interesting to think about that at the time Compaq bought DEC, in mid 1998, Windows NT had started to become more and more common as a server OS and also common as a desktop OS for larger companies, but there might had been at least some room for an x86 VMS version for more reasonable priced workstations, competing in the fading Unix market. Also imagine a VMS subsystem for Windows NT, like the OS/2 subsystem but complete, and like the “Windows services for Linux/Android” today.
Compaq’s plans for DEC was to move back into the higher end market that the push for volume ceded. There were enough marketing releases at the time of the merger to show that.
Alpha was going to be a problem because the cost of upgrading fabs meant that one needed massive volumes to have affordable chips. Unfortunately, many believed Intel’s very optimistic schedules for Itanium and expected to have Itanium systems in the mid-90s. Keeping Alpha around for the 5 years of Itanium’s repeated delays would have made sense. Probably given how Itanium really wasn’t a good general purpose CPU, Alpha could have hung around a while longer until multi-core X86 closed the performance gap.
Compaq had no problem competing against Intel. They funded NexGen after all.
While those that grew up on VMS prefer it to Unix, there isn’t all that much to distinguish the functionality for a user which is what the subsystem would provide. Files-11 had some advantages but I am not sure that NT would be rewritten to use yet another file system.
@Richard:
Thanks!
I vaguely remember hearing about Nexgen, but I didn’t know anything about them. Compaq and Olivetti funded them. Interestingly Olivetti PCs were OEM manufactured for DEC. (I assume that DEC used Olivetti PCs in Europe and Tandy PCs in North America).
Re Itanium: It’s funny that Intel multiple times believed that they would create a new fancy thing that would eventually replace x86. First it was the iAPX 432, and then i860, then i960, and eventually Itanium. It must had been a huge blow to Intel that AMD won with x86. And then ARM ate up a lot of the market. This makes me wonder if ARM would had been a good competitor earlier on, or if it just happened to match the requirements and possibilities of today but wasn’t that good decades ago? Also in a contra factual reality it would had been interesting if Nexgen would had bought AMD rather than the other way around.
Re VMS: I have to admit that I don’t have that much experience, and it’s kind of a challenge to learn VMS since things like paths are way different and such. My hot take that Unix lovers would hate me for saying is that Unix, the actual distribution and not just the kernel, feels like something a bunch of leet script kiddies would had cobbled together, where each got their will to incorporate quirks, and the least skilled of the script kiddies got to package everything, including writing start scripts that makes autoexec.bat and config.sys seem like a dream.
Except for the installer and boot loader, afaik NT can use any file system that you write a driver for. Can’t remember if I’ve seen ext2 drivers for NT, but still. (I’m currently reading the old book Windows NT file system internals, which kind of shows that the only hurdle when writing a file system for NT at the time was the lack of easily obtainable information. Kind of, sort of.
I would think that a VMS subsystem for NT would had allowed customers to move over to NT more easy. Sure, there were ways to port application (IIRC DEC/Compaq was involved in the Fortran implementation for Visual Studio, that I think Intel inherited.
One of the flagship products for VMS was the database RDB. DEC sold it to Oracle before Compaq had bought DEC, so Compaq couldn’t really had done that much in that field. Or maybe they could? According to Wikipedia RDB is still a VMS only product, maybe the sales agreement did forbid Oracle to port it to other platforms (for a period of time)?
Re other filesystems for NT, there’s at least a btrfs driver and a bootloader that lets you boot Windows off it.
About leet script kiddies: I agree with you, but it’s really funny to see how how Windows NT went some kind of careful design in Windows NT 3.1 to a pile of “a bunch of leet script kiddies” (the most egregious is the fact that they are “moving more time and language settings from Control Panel to Settings” in the next Windows 11 release… come on, it’s 13 years after introduction of Control Panel replacement), even Linux script kiddies have managed to switch from X11 to Wayland in that time and Apple or Google do changes like these in a single release). While Linux Kernel (but not userspace!) managed to acquire all the interesting “advanced capabilities” that Windows NT had from the beginning.
The end result: Windows NT was revolutionary (truly revolutionary, no quotes!) when it was released, but today… it’s only interesting because there are millions of apps written for it.
I’m genuinely curious. What was so revolutionary about NT? What was so special, with no counterpart in Unix, VMS, OS/2, or Mach?
I will say that NT is the newest major OS that hasn’t died.
Consider the fact that NT was designed to properly handle multiple CPUs from the day one while Linux finally got rid of BKL (hack that’s needed to adapt SMP to non-SMP core) in year 2011. And NT had asyncronous file access since day one, too, while Linux got it year 2019 (and it’s still not enabled on Android and ChromeOS for security reasons). Containers were also in the original design (while OS/2 only got them briefly in version for PowerPC and only in experimental fashion). So for about 12 years Windows NT was the only OS that could manage background processes robustly – pretty damn basic need, don’t you think? Linux, of course, got all that in a year 2007 and corresponding change to userspace (systemd) needed three more years. Or event tracing. DTrace in Solaries and SystemTap in RHEL arrived in 2005, too (and work on propagating that to userspace is still ongoing) – while NT had it from the beginning.
As I have said: I don’t think there are anything big left worthwhile in NT to copy (most things are either copied, or in process or are impossible to copy without full redesign OS which would break backward compatibility), by now, but, well… when, after 30+ years we are “almost in position to finally offer things that NT users took for granted for all these years”… I would say that it’s pretty damn good indication that original was “revolutionary”.
Also: our shortcomings are a continuations of our virtues. The fact that NT offers faster execution for the CPU-bound processes that Linux (yes, even today, that’s difference in original design, you can not simply fix that with kernel upgrade) doesn’t matter when most CPU-bound applications, these days, ALSO need either network access or file access, where NT loses – and loses big (again: problems in the original design, pretty hard to fix). In fact NT had asynchronous file processing since day one as an attempt to handle that deficiency… it worked well enough to make MS SQL Server successful, but not everyone was happy to spend crazy amount of time doing all the required dances when on Linux everything “just worked”.
Sure, if you go to circa 1995 and compare NT against Linux, NT looks like absolutely amazing tech. But that’s because Linux was crap, and only looked good if you compared it to DOS. You’d need to compare NT against Solaris, NeXTSTEP, UnixWare, or at minimum OS/2 to get any kind of a sensible comparison.
And yes, NT I/O performance is by far the worst today. Both Linux and macOS run rings around it. I see that every time when trying to build some larger software project.
Re asynchronous file access: Sorry if I’m mixing things up, but I assume that this refers to having an API where you ask the OS to do file I/O, and the API returns immediately and later you get a signal/message that the I/O is done? If so then AmigaOS (in 1985) and it’s file system predecessor TripOS (even earlier on, so old that it was written in BCPL rather than C) had asynchronous I/O, but very few programs used it as the documentation was published by another company (Bantam) than the rest of the reference manuals (Addison Wesley), and also that you kind of had to bypass the actual DOS API and send messages directly to the file system driver(s). (The separate publishers were probably an existing contract in place for TripOS. The documentation is weird as the AmigaDOS manual just ignores everything not related to file I/O, while the rest of the documentation ignores all file API I/O, with the result that there were no examples of using asynchronous file I/O combined with the windowing GUI (where the general method was to wait for messages from each window, and you could easily wait for messages from multiple windows, so adding asynchronous file I/O to wait for messages/signals from seems logical).
I always thought it was weird that the socket network API was so similar to the file I/O API in *ix, yet one was asynchronous and the other not.
Btw a script kiddie problem is naming convetions: Using the term “inode” for different things in memory and on disk was a really bad choice.
Re revolutionary: I don’t know what various workstation OSes could do at the time. I have a hunch that if any of then did things that NT could do, it would had been VMS.
Re performance: I remember the “Microsoft scalability days” fiasco where they demonstrated that Exchange could send X mails in 24 hours. SUN redid it with sendmail and got done in IIRC 1½ hour. On the other hand, Microsoft might even had done this on purpose. For once for someone not following everything they might not know that the Exchang results were awful. But more importantly it might had tricked the competition to think that they were ahead, when in fact the selling argument for Exchange was the scheduling feature and also that you needed zero configuration on each Windows client, as the general Windows logon was used for accessing the Exchange server. With anything using sendmail you’d either need to configure smtp, pop3, imap and such or do mailing on a *ix computer. In theory you might had been able to share various folders using samba, connected to an NT domain for account validation, and used a non-existing windows mail client using unix style file I/O. Or you could had used a non-existing pop3/imap server that would use an NT domain for account validation, and a probably non-existing Windows pop3/imap client that could send windows account login information to the server. Also, while imap was technically a thing since the late 80’s, few if any used it in the 90’s, and thus in comparison Exchange was great as it had all messages on the server, while internet/sendmail style mailing usually meant that you had your mails stored locally and if you switched computers you lost access to existing mails.
Re performance: I also think that NT/Microsoft didn’t care that much about certain aspects of performance. For example if you make repeated calls with small strings to output to the console then NT is super slow. And the console is in general slow as long as the console window is visible. Like when running something that output several screens of text it’s worth pressing alt-tab to have something else cover the console, and alt-tab again after a moment to see the results. (Like the WSAEnum…. API example for listing winsock2 protocols/address families available takes longer time to run than to compile (at least with pre complied headers) using Visual Studio 6 + Windows 2000 + VirtualBox (with whatever CPU speed default setting VirtualBox has). Or at least it does if you have enabled/installed lots of protocols).
Well, if you talk about priority, then the first commercial jfs was in aix, the first big os with containers was vm/370, I don’t know when first asyncio appeared, but openvms already had it. About events tracing, I think the first was vm/370 again, btw, you can’t compare windows events with d(ynamic)tracing until at least win2k, before that, it has just a few fixed event logs.
About script kiddies. Both x11 and rcinit are badly needed redesign/upgrade. Unfortunately, instead of greybeards from the 70s-80s, all we got were Poettering and Hogsberg. 🙁
Even if you compare NT and Solaris, NeXTSTEP, UnixWare or OS/2… you would still have similar expression: solid, well-designed thing vs something cobbled together. And the same even with workstation OSes.
But if you pick one, single, OS… nope, not even close. None of these OSes have the ability to restrict the rogue process and ensure that it couldn’t fork-bomb the system into unsable state. I think only Solaris eventually got such ability – and that took more than a decade. Sure, IBM’s Mainframe OSes and VMS had that ability… but how fair is it to compare NT to grand sum total of everything else that existed at that point of time?
It’s like Rust: every single facet of it was invented elsewhere and if you take a look on any single feature in isolation – it has previous art somewhere. But combining them is not easy, while adding them to existing design is even harder – that’s why we are still not entirely there, 30 years later.
My point is just that putting together existing prior art is not a revolution.
Comparing NT against VMS and OS/2 is more than fair, since that’s where NT developers had come from. So NT had absolutely no excuse to not match and exceed those two OSes in particular.
Re fork-bombing, maybe NT’s best defense was that process creation was so slow that it was impractical? It’s still order of magnitude slower than competition in that regard (so much so that anything that can avoid rapid process creation is a win).
OS/2 had event tracing (because of IBM obviously), so not having that in NT would have been weird. Back then static traces only, although OS/2 did add dynamic tracing sometime in the late 1990s.
I’d say rcinit was a mess but a well understood mess. And systemd does lots more than just replace that, it also includes functionality of Solaris SMF and such things.
Wayland is a classic exercise in reinventing the wheel, poorly. After 10 years, it’s finally good enough to replace X11, just barely. Too many times when asked “what is the equivalent of this existing X11 functionality in Wayland?” the answer was “not our problem, that’s up to the Desktop Environment”. Because obviously developers want to deal with a bunch of different DEs.
What new concepts NT brought to the table was a staple of Advocacy debates 30 years ago. Not sure rehashing that would be beneficial to anyone. Simply doing a more effective design based on the lessons that others learned could be useful enough.
It’s fair to compare NT to VMS (and then NT is revolutionary because there are many things that VMS haven’t done, like support for multiple personalities), it’s fair to compare it to OS/2 (and then NT is revolutionary because it had proper support SMP and async programming) – but it’s not fair to compare NT to amaglamation of VMS, OS/2 and all other OSes that existed at that time.
As for “just that putting together existing prior art is not a revolution” – sure, but “putting together existing prior art” is a necessary component.
Every “revolutinary” thing that exists, be it GUI on desktop in Mac, finger-only manipulations in iPhone or even airplanes by Wright brothers, invariably, have dozen of predecessors that haven’t went anywhere – in all aspects of that these revolutionary creations have done.
Revolution happens when something NEW TO THE PUBLIC occurs… and takes hold. That “takes hold” part is the important part of the revolution, being “the first one to invent” is almost never a “revolutionary” achievment, being the “the first one to popularize” is.
Because revolution happens not when something is made for the first time, but when something changes the face of civilization (or just small part of civilization, like a single industry).
These two events can be separated by decades and even centuries.
One example. ECS is all the rage today and is seen as solution for many woes in OOP programming… but it was invented before OOP! Both Alan Kay and Bjarne Stroustrup knew about it! They have built OOP on the Ivan Sutherland’s Sketchpad’s ideas and were extremely proud to kick ECS out from their designs – thus we had to wait 35 years till “Thief: The Dark Project” pioneered the rediscovered concept. Does it mean that Looking Glass Studios are not revolutionaries? Simply because they recreated something that was “swept under the carpet” and forgotten decades earlier?
I’m sure we can all agree that without a company with almost unlimited money like Microsoft behind it, NT would never have happened.
Re SMP — that was simply a question of the timeline. When OS/2 was designed circa 1985-1986, multi-processor PCs were not even on the drawing board. How do you support something that doesn’t exist?
When NT came out, multi-processor systems were available from a number of vendors. Solaris supported SMP since 1992-1993, SVR 4.2MP came out in 1993, Novell had been talking about SMP since what, 1989? Microsoft simply could not afford to release NT without SMP support, because in the meantime it had moved from “future tech” to a requirement for a server OS.
Well, the talk is about smp with giant lock vs finer-grained locking. And windows nt is indeed the first major/popular os without the giant lock from the beginning, by virtue of being newer. The visionaries behind it predicted, in the late 80s, that there would be computers with more than 2 cpus.
It didn’t take a visionary to be told by Compaq in 1989 “hey, we’re putting out a two-processor PC later this year” (I know, the SystemPro wasn’t real SMP yet). Corollary was talking about 10-processor SMP 486s in 1990. Intel had the 82489DX in 1992, and they weren’t really a leader, that was when SMP was about to go mass market.
Which it didn’t because of Windows 9x, but that’s a different story…
I think the discussion mixes very different questions: whether NT was a revolutionary OS – yes, it brought many things that popular OSes before it lacked to the realm of computing (even if all things were explored in at least one other OS, these OSes were either niche or extremely exotic). And, once again, revolutions happen not when someone invents something (tablet computers were first shown in the Star Trek decades before Steve Jobs presented iPad), but when these inventions become affordable.
Another question is whether other OSes like OS/2 or Linux could have been developed with all these features since the beginning – and, sure enough, “unlimited pockets” and the desire to create OS for the next half-century were an important factors.
But remember that NT wasn’t the only OS with such premises, there were plenty of OSes that had the exact same mandate, from IBM’s “Grand Unifying Theory of Systems” to Apple’s Aquarius and Taligent projects, to Tanenbaum’s Amoeba and Wirth’s Oberon and also Plan 9 from Bell Labs… there were plenty of OSes that tried to be revolutionary, at the time… and they many ideas that NT had, too – but they were complete and utter failures in the market. Attempts to achieve the revolution with them have fizzled out.
I think that Microsoft didn’t aim for revolutionary, but rather to have everything that would make it as future proof as possible, and the result ended up being revoultionary.
As for SMP: Microsoft was big enough that they could likely just had told Intel that they are interested in SMB, and “we need 100 PC motherboards with two or more 386 CPUs” and Intel would had created that product if it didn’t already exist.
I would rather guess that Microsoft realized that they just lacked the skills to create an OS for the future, and hired David Cutler, and SMP was one of the results of this. Kind of, sort of.
================
As for who could had created something like that, at that time:
I would say that if there had been a hobbyist interest in something similar to NT, someone like Linus Torvalds could had made an NT like OS rather than a Unix like OS. The main obstacle would likely had been getting a modified version of Windows 3.x going under a enthusiast created OS. Something like ntvdm and running Windows 3.x application had been a must at the time.