The Secret History of ATAPI

The other day I asked myself a seemingly trivial question: What was the first ATAPI CD-ROM drive and when was it available? Given that ATAPI was a major technology which instantly obsoleted all proprietary CD-ROM interfaces and made SCSI much less desirable, one might expect that there would have been some press releases touting the advantages of the new technology, articles describing the whys and wherefores, but… nope. There is nothing.

A rather old IDE CD-ROM drive with curious jumper settings

In 1993, CD-ROM drives used either SCSI or one of several proprietary interfaces, the major amongst those being Matsushita/Panasonic, Mitsumi, Philips, and Sony. In 1995, the proprietary interfaces were effectively gone and most new CD-ROM drives used the ATAPI interface. Something clearly happened in 1994, but exactly what, when, and how—that’s something of a mystery.

There are reports that Philips showed their CDD-300 CD-ROM drive with “new IDE interface” (e.g. ATAPI) at the CeBIT ’94 expo in March 1994. When the drive started shipping is unclear, but it was certainly in 1994.

Further digging showed that NEC was another early vendor of ATAPI CD-ROM drives. Establishing the timeline is again very difficult, but in June 1994 there was a Linux kernel patch to support the NEC CDR-260. Its author seems to have been completely unaware of the existence of ATAPI, but that’s perhaps not surprising since the ATAPI 1.2 specification was only finalized on June 10, 1994, although a draft was available in February of that year.

Too Old to Work?

There is a possibly misleading Microsoft KB article (KB131499) claiming that the “NEC CDR-38, NEC CDR-250, CDR-260, CDR-260R, and CDR-260GW CD-ROM drives are incompatible with the ATAPI specification version 1.2”.

Another Microsoft KB article (Q121547) brings more detail: The NEC CDR-260 drive “follows the ATAPI A1.8 specification, which is a pre-release specification that has not been ratified by the ATAPI council”. The article further adds: “NEC also manufactures the CDR260R CD-ROM drive, which does conform to ATAPI 1.2 specification and works with Windows NT version 3.5. Starting in November 1994, Gateway 2000 began shipping the CDR260R rather than the CDR260 in its computers.”

Note that the two articles contradict each other: KB131499 claims that the NEC CDR-260R is not compatible with ATAPI 1.2, whereas Q121547 explicitly states that it is.

The NEC CDR-260 was apparently an OEM drive and no real documentation exists. What is interesting is that photos of an NEC CDR-260 made in July 1994 show a jumper selecting between “8-bit bus” and “16-bit bus”. ATAPI is inherently 16-bit just like ATA… then again there are specialized 8-bit ATA variants.

Now, if there were CDR-260 drives built the first half of 1994, that would explain why those drives weren’t compatible with the ATAPI 1.2 specification which was officially published in June 1994. It’s also no wonder that Windows NT wasn’t the only operating system that didn’t like those drives. NeXTSTEP 3.3 release notes explicitly list the NEC CDR-260 and CDR-273 as unsupported.

Linux source code offers some clues too. The NEC CDR-260 is clearly mostly a standard ATAPI drive but several relatively minor differences prevent it from working with normal drivers.

Examining NEC_IDE.SYS

Another indirect source of information is the NEC_IDE.SYS device driver for DOS (required for MSCDEX). The OS/2 Museum has an NEC_IDE.SYS file dated 2-18-94, size 21,669 bytes. This driver does not work with modern ATAPI drives, but shows something interesting:

NEC_IDE.SYS from February 1994

Rev A1.7 quite likely corresponds to a pre-release ATAPI specification, but notice how there is absolutely no mention of NEC, only Oak Technology OTI-011.

Analyzing the driver shows that it cannot work because it issues the ATAPI Soft Reset command (code 08h) and expects that when the command completes, the drive will set the DRDY and DSC bits in the ATA status register, just like an ATA hard disk would.

The ATAPI 1.2 or even ATAPI 2.6 specifications are not 100% clear on this point, but suggest that the bits should probably not be set. ATA/ATAPI-6 is unambiguous and says the DRDY and DSC bits will be clear, just like after ATAPI device power-up, in order to “hide” ATAPI devices from old ATAPI-unaware drivers for ATA hard disks.

A newer version of NEC_IDE.SYS from April 1995 works:

Updated NEC_IDE.SYS compliant with ATAPI 1.2

It’s notable that the newer driver handles the standard ATAPI behavior (status register bits clear after ATAPI Soft Reset) as well as the old NEC drive behavior (DRDY and DSC set). It is highly likely that the driver was designed to handle both old pre-standard drives and newer ATAPI 1.2 compliant CD-ROM drives.

But again… why does a more or less standard ATAPI driver for an NEC CD-ROM drive only mention Oak Technology OTI-011 and nothing else…

Oak OTI-011

So what exactly was the Oak Technology OTI-011 chip? There is not much information available. Unusually, in place of datasheets, bitsavers only has several patents related to the OTI-011 chip. That is not great, but does show fairly clearly what OTI-011 did.

Oak Technology OTI-011 chip made in January 1994

Note that in the late 1980s, Oak Technology developed OTI-012, a CD-ROM controller chip with no IDE/ATA support. The OTI-011 is a successor chip (no, I don’t know why the Oak numbering went backwards) developed in 1992-1993 which combined the functionality of OTI-012 with a built-in ATA interface, suitable for implementing an ATAPI CD-ROM.

The OTI-011 required an external microcontroller and DRAM and was expected to be connected to a DSP which delivered the raw bits from a CD-ROM. The OTI-011 decoded sub-channel data and raw sector data, and was capable of performing the additional error correction and detection needed for CD-ROMs. The host interface of OTI-011 implemented ATA task file registers in hardware and set up ATA/ATAPI command processing for the attached microcontroller. The OTI-011 chip itself implements very little of the ATA protocol itself; the external microcontroller’s firmware does the hard work.

The patents describe what the OTI-011 chip did in terrible mangled lingo, but don’t provide any background information. Normally there would be no public information about the development of a chip… but in this case there is, thanks to lawsuits that Oak Technology (later Zoran) initiated in the late 1990s and early 2000s.

There is an U.S. International Trade Commission document which supplies a surprising amount of historical information, and also establishes a fairly detailed timeline, since that was required for establishing the validity of US Patent 5,581,715 (aka ‘715 patent).

We learn that Oak Technology achieved “considerable commercial success” with the OTI-011 chip because it solved an important problem (attaching a CD-ROM drive to the ATA bus) before others.

Oak Technology, namely Phil Verinsky, conceived of the OTI-011 chip “no later than April 21, 1993” while “the date of the ATAPI specification is June 10, 1993. There is no evidence that the ATAPI specification or any earlier version of it was communicated to Oak before the April 21, 1993, conception date.”

Note that the official ATAPI 1.2 specification is dated June 10, 1994, but the above refers to Western Digital’s “ATAPI CD-ROM Standard: Proposal for Working Draft” document from June 10, 1993.

We learn that in early 1993, Mitsumi worked with Oak Technology on a CD-ROM drive attached to the ATA bus but not utilizing ATAPI. It appears that this project didn’t really go anywhere.

In April 1993, there was a sales agreement between NEC and Oak concerning the not-yet-completed OTI-011 chip. It is almost certain that NEC was Oak’s first customer and likely the first user of the OTI-011 chip. This fits with other known facts, namely that NEC shipped ATAPI CD-ROMs before the ATAPI 1.2 standard was finalized.

The OTI-011 chip taped out on July 22, 1993, was quickly corrected and taped out again on July 29, 1993. Given that information alone, it’s near certain that OTI-011 chips were made in 1993, but it’s much less clear if any CD-ROM drives based on those chips were shipped in 1993.

We also learn the versions and dates of several unpublished early ATAPI specification drafts:

  • Revision A1.4, dated Aug. 17, 1993, 141 pages
  • Revision A1.5, dated Aug. 28, 1993, 158 pages
  • Revision A1.6, dated Sep. 10, 1993, 174 pages
  • Revision A1.7, dated Oct. 18, 1993, 176 pages
  • Revision A1.7B, dated Nov. 2, 1993

Note that revision 1.7B was almost certainly the last one before SFF took over the ATAPI project.

Under the SFF heading, ATAPI 1.0 and 1.1 drafts were made available but very quickly replaced. Revision 1.2 was the first widely used ATAPI standard.

NEC CDR-260

The OS/2 Museum was able to obtain a functioning NEC CDR-260 drive with a February 1994 manufacturing sticker (February 1994 was when the draft of ATAPI 1.2 was made available). Sure enough, the drive is… interesting. That starts with the drive jumpers which according to the label provide an option to switch the IDE interface into an 8-bit mode (see photo at beginning of article). What good that might be… nobody knows, since there is no documentation.

NEC CDR-260 ATAPI drive made in Feb ’94

Looking inside the drive, one very unsurprisingly finds an Oak Technology OTI-011 chip (lower right corner):

NEC CDR-260 electronics

Linux detects the CDR-260 drive, but is a bit confused about the drive’s name. It turns out that the model name is not byte swapped in the data block returned by IDENTIFY PACKET DEVICE, but Linux expects that it will be, so it ends up getting scrambled. This is the exact same problem that certain old IDE drives had. Unsurprisingly, the problem has been noticed before. It is worth mentioning that the ATAPI specification makes no suggestion that the ASCII strings embedded in the IDENTIFY PACKET DEVICE response should be byte-swapped, but “everyone” expects them to be.

What’s much worse is that after inserting a CD-ROM, the drive “falls off the bus” with many errors and many reset attempts. Linux is effectively unable to read anything.

But the drive is not dead. In DOS, the NEC_IDE.SYS driver (dated 08/11/1994, size 24,824 bytes) works just fine. The drive can read CD-ROMs without trouble. Interestingly, the previously mentioned older NEC_IDE.SYS driver (dated 02/18/1994, size 21,669 bytes) appears to find the drive but then says that it’s not ready.

The working NEC_IDE.SYS driver appears to detect this particular model, or rather detects NEC (by definition ATAPI) drives with firmware version lower than two; the drive in question reports firmware revision 1.01. Such old NEC drives are then treated specially by the NEC_IDE.SYS driver.

A survey of various operating systems confirms that yes, the NEC CDR-260 is unusual. As mentioned above, Linux has workarounds for it, but clearly not sufficient. Solaris mentions the CDR-260 explicitly, and special-cases the CDR-260 and a few other drives to only transfer one sector at a time.

The ATAPI driver in OS/2 has by far the most complete support for these drives. The author clearly had the elusive ATAPI revision 1.7B specification in hand. The OS/2 driver translates a number of SCSI commands and converts the SCSI mode page information returned by the drive from the ATAPI 1.7B format to the published ATAPI 1.2 specification. There’s also the following comment:

The 1.7 spec indicates that the completion status only relies on the
DRQ bit being 0.  If we are in 1.7B compatibility mode and the DRQ
bit is 0, set the reason variable to IR_COMPLETE.

I suspect that might be the why Linux doesn’t work with this drive and why Solaris only reads one sector at a time. From reading the ATAPI mailing list, it is very clear that as late as December 1993, some important low-level details of ATAPI were still in flux. If the NEC CDR-260 drives really conform to the pre-SFF revision 1.7B of the ATAPI specification, it’s not surprising that software written for ATAPI 1.2 and later has trouble with them.

The NEC CDR-260 drives were also found in NEC’s own PC-98 machines. This blog post (Korean) shows a drive with a “CDR-260(PI)” designation on a Japanese label and much the same electronics as the one pictured above. The OTI-011 chip visible in the blog photos was made in week 46 of 1993; there is a high likelihood that NEC shipped the CDR-260 drives before the end of 1993, though perhaps only in Japan.

Other Early Entrants

As mentioned above, Philips/LMS was another early supplier of ATAPI drives. Examining DDATAPI.SYS driver from August 1994 that shipped with CM-207 drives (and likely others), it is apparent that the driver is very unlikely to work with generic ATAPI CD-ROMs. The DDATAPI.SYS driver expects that the Sector Number register (typically at I/O port 1F3h or 173h) changes its value in a certain way. Yet the ATAPI specification leaves said register more or less completely unused and does not define any behavior for it.

That is a hint Philips started implementing their ATAPI drives well before the ATAPI specification was finalized. It is possible that the Sector Number register has some vendor specific meaning on Philips drives, but it’s also possible that the register had some defined function in unpublished drafts of the ATAPI specification.

Note that Philips did also produce integrated ATAPI CD-ROM controllers (e.g. SAA7380), but there is no evidence that such chips existed before 1996.

Cirrus Logic was, together with Oak Technology, an early ATAPI chip vendor. The CL-CR3400 may have been available in 1994, but unlike the OAK OTI-011, it was only an ATAPI interface chip, not an integrated CD-ROM controller. Only the follow-up chip, the Cirrus Logic CL-CR3410 (1995) integrated both the ATAPI interface and a CD-ROM decoder.

Mitsumi was another early vendor of ATAPI CD-ROM drives with the FX001DE drive. So was Sony with CDU-55E and CDU-55D drives. If this document is to be believed, all those drives were available in 1994 and did not fully conform to ATAPI 1.2.

Secret Society

It’s not a coincidence that even industry insiders called ATAPI a “secret society”. According to Dal Allan, ATAPI “went public” in October 1993, and became an SFF project (before moving to X3T13 in 1996). But work on ATAPI probably started sometime in 1992 when, again according to Dal Allan, It was Compaq who told WD “You and the CD-ROM guys get together and give us a CD attached to the second ATA connector.”

ATAPI became sort of tied into Western Digital’s EIDE, although it is unclear if Western Digital sold any ATAPI related products. It is however clear that WD lent its significant expertise when figuring out how to make ATAPI devices coexist with IDE hard disk on a single cable without upsetting the BIOS or driver software (a puzzle that was still being solved just before ATAPI 1.2 was completed).

Until late 1993, ATAPI development was driven by Western Digital but done in secret, with confidential drafts of the ATAPI specification circulated amongst a handful of companies. From the Oak Technology related litigation we know that ATAPI A1.3 specification was dated June 10, 1993. We don’t know any dates of the earlier versions, but it is unlikely that A1.3 would have been the initial version; it might however been the first version seen outside of Western Digital.

Conclusion

We still don’t know who shipped the first ATAPI CD-ROM drive and when. The pre-1994 drafts of the ATAPI specification remain as secret as they were back in the day. That is a shame, because properly supporting certain old drives (such as the NEC CDR-260) would require the ATAPI 1.7B specification.

We did find out that Oak Technology was an important player in the early ATAPI CD-ROM drive evolution, and that ATAPI CD-ROM drives were made in early 1994, perhaps even in late 1993, before the earliest ATAPI standard (revision 1.2) was finalized. These drives tend to be incompatible with standard ATAPI driver software, but work with the DOS drivers that these drives shipped with.

This entry was posted in CD-ROM, PC history, Standards, Undocumented. Bookmark the permalink.

60 Responses to The Secret History of ATAPI

  1. Michal Necasek says:

    There can’t be CD-ROMs with 512-byte sectors because they wouldn’t be CD-ROMs. The relevant standards for recording very clearly specify 2048-byte sectors. But it was somewhat common for old SCSI CD-ROM drives to have a jumper that made them report 512-byte sectors to software, basically splitting each sector into four. The 2K sector size is pretty much baked into the physical CD-ROM format.

    LMS or LMSI is Philips. Very likely the drive used the “normal” Philips interface but there was a DEC specific interface board.

    I’m sure it was possible to stuff more than two 5.25″ HH drives into a system, but it certainly wasn’t common, and it required appropriate power supplies and such. And with FH drives… yeah, those monsters were big and two 5.25″ FH drives in one system is the most I’ve ever seen.

    Some Mac OS X boot DVDs have a HFS+ file system and also an ISO 9660 file system with completely different contents, namely Boot Camp drivers for Windows.

    ATAPI was (is) a real standard, it’s just that some of the early drives weren’t fully compliant. Newer drives (say 1995 and onward) don’t tend to cause trouble. I’d say ATAPI 1.2 was the minimum… but some of the earliest drivers weren’t quite that.

  2. Richard Wells says:

    ECMA-119 devotes too much space to the concept of sectors storing more or less data than 2048 bytes (but still a power of 2) for there not to have been a proposal in the works for CD-ROM derivatives that didn’t use the physical and logical sectors with 2048 data bytes.

  3. Michal Necasek says:

    ECMA-119 and ISO 9660 clearly distinguish between “logical sectors” (2K or larger, power of two size, up to the size of physical sector) and “logical blocks” (512 bytes or larger power of two, up to the size of logical sector). It is explicitly noted that logical sectors might comprise more than one physical sector on the medium if physical sectors are smaller than 2K.

    In practice, logical sectors larger than 2K would require a medium with 4K or larger sectors. I don’t know if such a thing exists (and is used with ISO 9660 file system). ISO 9660 file systems with logical blocks smaller than 2K exist, but are exceedingly rare and may not be well supported.

    The logical block size is recorded as part of an ISO 9660 file system; the logical sector size is not since it’s effectively a property of the underlying medium.

  4. Battler says:

    Now, after debugging that particular CD-ROM driver on my emulator, I can say for certain why it’s going “not ready” – it wants an IRQ to be raised after every sector read. I assume that by ATAPI revision 1.7B, which NEC CDR-260 firmware 1.01 used, they already removed that requirement. The driver must have been for NEC CDR-260 firmware 1.00 then, of which I’ve yet to see any in the wild, which I assume must have used an even earlier revision of the ATAPI specification – the driver’s splash string would indicate 1.7 without letter.

  5. Michal Necasek says:

    Thanks for the additional information. Yes, that sounds like the kind of thing they would have wanted to change before the ATAPI spec was finalized.

  6. I have some problems with ATAPI drives in 1997-2000s Toshiba laptops. The drives that were used in these laptops e.g. Toshiba XM-1502B, XM-1602B, Teac CD-220EA works, but if I use anything else laptop do not start… I have tried more than 10 ATAPI drives from 2000-2008..

    Could it be that ATAPI drives are locked or motherboard vendor could put some restrictions?

    I have describe my problem with list of drives that I have tested on Vogons. If you could help me, I would be very glad 🙂

    https://www.vogons.org/viewtopic.php?f=46&t=96203

  7. Michal Necasek says:

    If the machine doesn’t get past the BIOS POST, it’s likely that the BIOS is looking for something specific. It could be locked to a specific vendor string (returned by the IDENTIFY PACKET DEVICE command), or it might be using some non-standard ATAPI commands. Hard to say.

    IBM did that kind of thing with hard disks and I think other devices in their ThinkPads. So it’s not unheard of. The simplest solution probably is to only use supported drives.

  8. PHP says:

    Here are some images of an OTI011 that was produced in December 1993 inside of an NEC CDR-250 that was manufactured in August of 1994.
    https://imgur.com/a/wV7NsL0

  9. Michal Necasek says:

    Cool, thanks for the photos! That OTI-011 chip sat in a tray somewhere for a surprisingly long time.

  10. ISO 9660 file systems with logical blocks smaller than 2K exist, but are exceedingly rare and may not be well supported.

    Yeah. Linux support for them broke in the transition between 2.4 and 2.6 and the SHSUCDX filesystem driver used by FreeDOS (and MS-DOS users who want to support the DVD-sized ISO filesystems included on DVDs produced by Linux burning tools as a safety net) doesn’t recognize them either.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.