Everyone knows that the IBM PC established 512-byte sectors on floppies and hard disks as the standard, which survived for several decades until the advent of “native” 4K-sector drives.
Of course what “everyone knows” is not necessarily the whole story.
PC Floppy Sector Sizes
The original 1981 IBM PC Technical Reference says: The [IBM PC floppy] drives are soft sectored, single sided, with 40 tracks. They are Modified Frequency Modulation (MFM) coded in 512 byte sectors, giving a formatted capacity of 163,840 bytes per drive.
But that was never really true; while PC DOS 1.0 indeed used 8 sectors per track, resulting in 163,840 bytes (512 × 8 × 40 bytes) on a floppy disk, PC DOS 2.0 supported 9 sectors per track on the same hardware, with increased 184,320 bytes disk capacity. This was possible in large part because the BIOS in the IBM PC was fairly flexible when it came to floppy disk formats.
It should be noted that the IBM PC used the NEC μPD765 floppy disk controller (FDC), which was fairly agnostic about the disk layout (though it had very specific ideas about how sectors should be formatted). It supported more or less any number of sectors per track (up to 255, much more than the physical drive limits), and could handle sector sizes that were a power of two from 128 bytes to 16 kilobytes.
The PC BIOS exposed the floppy drive characteristics through the Disk Parameter Table (DPT), sometimes also Disk Base Table. The BIOS had a default table, and during POST, initialized interrupt vector 1Eh to point at that table, called DISK_BASE in the original PC BIOS. The user could reprogram vector 1Eh and point at a different table. A routine called GET_PARM in the BIOS took care to always access the table and not hardcode any parameters.
The boot sector of PC DOS 2.0 (and the BIOS component of DOS) took advantage of this flexibility and modified the DPT in order to extend the number of sectors per track from 8 to 9 on 180K/360K disks. Newer DOS versions also modify the DPT.
But back to sector sizes. The PC BIOS documentation (in practical terms, the BIOS listing published by IBM) is notable for not saying anywhere that disk sector size is 512 bytes. In fact the BIOS goes to some lengths to make sure that the sector size is not fixed.
The BIOS INT 13h interface is structured such that read/write/verify operations all determine the transfer size by the number of sectors, not by the number of bytes. The PC BIOS does have to calculate the byte length of a disk transfer in order to set up the DMA controller. The DMA_SETUP routine in the BIOS takes care to calculate the transfer length using the sector size code from the DPT.
The BIOS INT 13h documentation also clearly shows that the format service (INT 13h/05h) can accept different sector sizes, and explicitly lists 128, 256, 512, and 1024 bytes as possibilities.
In sum, the PC BIOS can work with different sector sizes and is not harcoded to 512-byte sectors… with one tiny but important exception explained later.
DOS Sector Sizes
DOS was never hardcoded for 512-byte sectors, for a good reason. In the early era of DOS (circa 1980-1985), 512-byte sectors were far from standard. DOS was originally developed for 8″ floppies, which typically used either 128 or 1024 bytes per sector. There were also MS-DOS systems with 256-byte sectors on floppies, such as the DEC Rainbow. In other words, DOS had to be flexible about sector sizes because there was no universal sector size in the early DOS era.
The DOS 1.0 programming interface was compatible with CP/M and used FCB style I/O based on 128-byte records. While this interface was derived from 128-byte sectors used on early floppy systems, in 1980 it was already common to use physical sector sizes that were a multiple of 128 with CP/M. The UNIX-style file I/O API added in DOS 2.0 was even less dependent on sector sizes.
The flexibility of DOS with regard to sector sizes was also exploited in the early days of hard disks, when larger logical sector sizes were sometimes used to get past the 32MB partition limit in 3rd-party add-on products.
PC Hard Disk Sector Sizes
By the time (1983) the IBM PC/XT came out, 512-byte sectors were established well enough that the XT’s hard disk BIOS did in fact hardcode 512 bytes per sector. It is not immediately clear if the hard disk controller could support any other sector size.
The PC/AT continued the tradition, and only supported 512 bytes per sector on hard disks. While the WD1010 Winchester Disk Controller chips did support sector sizes of 128, 256, 512, or 1024 bytes, the AT disk controllers quite possibly did not (at minimum the sector buffer could not hold more than 512 bytes). In any case, the WD1010 SDH register (SDH for Sector size, Drive, Head), normally mapped at I/O port 1F6h, had to be explicitly programmed for 512-byte sector size on the PC/AT disk controller.
Because the XT and AT BIOS only supported 512-byte sectors on hard disks, all operating systems only supported 512-byte sectors as well. When the ATA standard was created in the late 1980s, sector sizes other than 512 bytes were not even considered. The hard disk controller Drive/Head Register (formerly SDH), which theoretically allowed the user to select the sector size, was fixed to only support 512-byte sectors. Before the first official ATA standard was finalized in 1993, one of the sector size selection bits in the Drive/Head Register (the one that previously had to be set to zero) was repurposed to enable LBA (Logical Block Addressing). This was presumably a safe extension because setting the bit to one would have not produced working results on at least some PC/AT compatible hard disk controllers.
Needless to say, a PC hard disk controller can come with its own ROM, and many did. A custom hard disk BIOS ROM could use sector sizes other than 512 bytes.
Floppy Booting
The IBM PC BIOS assumes that a machine capable of booting from a floppy drive must have at least 32K RAM (cassette-only PCs might have had only 16K, sufficient to run ROM BASIC). It loads the boot sector at the top of the first 32K of RAM.
One might expect that if the BIOS assumed 512-byte sector sizes, it would load the boot sector at address 32K – 512, i.e. 7E00h. Only the BIOS loads the boot sector at 7C00h, which is 32K – 1024. As if the BIOS expected that floppy disk sectors could be up to 1024 bytes in size (the largest common floppy sector size at the time).
There’s of course a catch. When it is loading a boot sector, the PC BIOS (just like all its successors) uses the default DPT, and therefore expects 512-byte sectors. The boot sector of any floppy capable of booting on a PC compatible system must therefore be 512 bytes in size. The boot sector can install a different DPT, switch to a different sector size, and keep using the BIOS floppy services, but the boot sector itself must be 512 bytes big in order to actually boot.
That is one reason for PCs using 512-byte sector sizes. The other reason is that 512 bytes is a good size for maximizing storage capacity. Smaller sectors have more overhead, and larger sectors may end up leaving too much unused space at the end of a track.
There is every indication that throughout the 1980s, sector size of 512 bytes was a good compromise between keeping the sector overhead on the storage medium low and at the same time not wasting too much space on the file system level when storing many small files. In other words, 512-byte sectors were just the right size, not too big and not too small.
What Is a Valid Boot Sector?
Booting brings another question: How does the BIOS recognize a valid boot sector? On the IBM PC, the answer is very simple: It doesn’t. The original PC BIOS attempts to read sector 1 from track 0 (and head 0, originally the only head) from a floppy into memory at address 7C00h. If the read succeeds, the BIOS jumps to the first byte and the rest is up to the boot sector. If the read (with retries) fails, the BIOS brings up the ROM BASIC.
This behavior clearly caused some trouble with diskettes that were physically formatted but had no valid boot code on them; such floppies would likely hang the PC without giving any indication to the user as to what was going on. The PC/AT BIOS uses a simple heuristic to decide whether a sector that was read without error indeed looks like a valid boot sector. These heuristics did not exist in the PC/XT BIOS.
The AT BIOS first checks if the boot sector starts with byte 06h or greater. This happens to exclude ADD opcodes at the beginning of the 8086 opcode map, and will reject zeroed boot sectors.
The BIOS then loads the first word (not byte) of the boot sector and checks whether the following 8 words are all identical to it. If yes, the boot sector is again rejected. This catches freshly formatted disks with any repeating byte or word pattern. If either of the checks fails, the PC/AT BIOS stops with an error rather than attempting to boot from the hard disk.
Once the heuristics are satisfied, the PC/AT BIOS again jumps to the boot sector and lets it take over. Note that IBM BIOSes do not check for any kind of signature on boot floppies. The reason is obvious: DOS 1.x never had any such signature in the boot sector. Neither did numerous other operating systems.
Note that the heuristics described above are employed by the third (1986) revision of the PC/AT BIOS. The exact logic slightly changed across PC/AT BIOS revisions.
Hard Disk Boot Sectors
The PC/AT BIOS uses completely different logic for deciding whether a fixed disk boot sector is valid. Once a boot sector is read from hard disk without error, the BIOS checks whether the last word (i.e. the word at offset 510) is AA55h. If so, it jumps to the first byte of the boot sector. Note that no heuristics are applied, only the AA55h signature is checked.
The older PC/XT fixed disk BIOS resides in an add-on ROM, but uses the same logic, and requires the same AA55h signature at the end of a 512-byte hard disk boot sector.
What the BIOS most certainly does not do is anything related to partitioning. If the first sector of a hard disk is a typical MBR (Master Boot Record), the boot sector itself will examine the partition table stored within it and act accordingly. However, that is completely independent of what the BIOS does. An MBR must have the AA55h signature, or it will be rejected by the BIOS, but that is the only BIOS requirement. A hard disk bootable by a PC BIOS can utilize any partitioning scheme, or none at all; the BIOS does not care.
Non-512-byte BIOS Sectors
One widespread (if perhaps not well known) example of using sector sizes other than 512 bytes is no-emulation El Torito CD-ROM booting.
CD-ROMs always have 2048-byte sectors. The El Torito specification supports booting from floppy or hard disk images stored on the CD-ROM; in that case, each CD-ROM sector logically contains four 512-byte sectors accessed through the emulated drive.
El Torito also supports “no-emulation” aka native booting. This assumes the use of extended INT 13h services, which allow using LBA rather than CHS addressing. For no-emulation booting, El Torito uses 2048-byte sectors through INT 13h.
No-emulation booting has the advantage that BIOS-accessible sectors map 1:1 to sectors on CD-ROM, allowing straightforward access to the ISO 9660 (or other) file system on the CD-ROM.
4K Hard Disks
Sometime around 2005, the storage industry started pushing for hard disks with 4096-byte sectors rather than the traditional 512-byte sectors. The biggest reason was simple: Capacity. 4K sectors have less storage overhead than 512-byte sectors. Common file systems in that era were using 4K or larger blocks anyway, so 4K physical sectors are arguably more natural than any smaller size. In addition, ECC (Error Correction Code) is more capable when used with larger data blocks; that is to say, the same amount of total ECC data can correct more errors when used for a single 4K sector compared to eight 512-byte sectors.
The ATA/ATAPI-7 standard, published in 2005, was the first to allow any possibility of hard disks with sector sizes other than 512 bytes. Until then, one sector was defined as strictly 512 bytes.
Unfortunately for the disk vendors, 512-byte hard disk sectors were by that time quite ingrained not only in disk interfaces (ATA including SATA, covered by the above mentioned ATA/ATAPI standard) but also firmware, operating systems, and various system applications.
Operating systems could work with non-512-byte sectors (see CD-ROMs), but booting from such drives was a very different proposition. While computers were perfectly capable of booting from CD-ROMs specifically, the mechanism was unsuitable for hard disks.
The initial and very widespread solution to the problem was emulating 512-byte sectors (512e drives, often labeled AF for Advanced Format). This approach allows existing firmware and operating systems to continue working unchanged. There is a potential performance problem if file systems using 4K blocks are not aligned with the physical 4K sectors, forcing a write of one 4K block to rewrite two adjacent 4K sectors. System software must take care to align file system to avoid this problem.
Hard disks exposing 4K sectors through the disk interface are often called Advanced Format, but typically labeled 4Kn for “4K native”.
Windows 8 (2012) was the first Microsoft OS that supported booting from 4Kn hard disks out of the box; this long delay reflects just how unprepared the PC ecosystem was to deal with hard disks using non-512-byte sectors in the late 2000s, after a quarter century of 512-byte sectors being the only choice for hard disks.
Ironically, adding support for 4K sectors is in some ways going back to the roots. In the early 1980s, non-PC hard disks were often used with 128-byte, 256-byte, or 1024-byte sectors; that was one reason why e.g. Seagate designated their hard disks with unformatted capacity, because the formatted capacity was a function of sector size. For example a 1982 ST400 series brochure from Seagate shows that the drives were formatted at the factory with 256-byte sectors. Even a 1986 product manual for the ST212 drive still lists the formatted capacity assuming 256-byte sectors, rather than 512-byte sectors, despite the fact that most users almost certainly formatted the drives with 512-byte sectors. Only the ST251 series product manual from 1987 lists the formatted capacity in 512-byte sectors.
Addendum
A well informed reader pointed out the 8FORMAT utility which modifies the BIOS DPT in order to access 8″ drives, including support for media with sector sizes other than 512 bytes. Note that 8FORMAT is a recently developed utility.
There was also a somewhat similar tool called 22DISK from Sydex, but I believe 22DISK programmed the FDC directly, similar to other Sydex tools.
I think you got the progression backwards, DOS 1.1 supported double sided disks with 8 sector tracks while DOS 2 support 9 sector tracks (single and double sided) and 8 sector tracks. The DOS 2 8-sector format was the 9-sector format except the last sector wasn’t used. Amazingly, it still worked under DOS 1 despite the shorter gaps between sectors. There was also a brief period when IBM had a “driver” for DOS 1.0 that treated double sided drives as a pair of single sided drives.
IO.ASM shows the DOS sector sizes. 128 byte sectors were used with single density formats for both 8″ and 5.25″ with 512 byte sectors for 5.25″ double density formats and 1024 byte sectors for double density 8″ formats. Note that the single density formats implemented what would later be described as clusters with either 2 or 4 sectors each.
At least, magnetic disks were more coherent than the disk formats overlaid on flash with large blocks subdivided into numerous sectors which were then aggregated into much larger clusters with some incorrectly designed formats burning through the lifespan by making many changes to a given block. On the extreme other side of disk accessing is the DEC DF32 where all 32K words per drive were directly accessible so no sectors at all. Lots of programmers will be ecstatic to have missed the true craziness before the standard disk controllers were created.
You’re totally right, I must have gone by my memory of what Tim Paterson wrote about 9-sector formats in DOS—only he remembered it wrong. See e.g. https://retrocmp.de/fdd/general/floppy-formats.htm Corrected.
Indeed, Windows up to XP aligns the first partition at (emulated 512 byte) sector 63, which causes two (“real”) 4 KiB sector read-modify-write operations on 512e drives and the usual cluster size of 4 KiB. Vista upwards places the first partition at sector 2048 or 1 MiB, so this problem is mitigated.
PC98 systems with 1.2MB drives had 1024 byte sectors and booted MSDOS with no problem. PC98 quad density drives used the standard 512 byte sectors. I haven’t found if the PC98 DOS would boot off the earlier NEC hard drives formatted with 256 byte sectors. Just because IBM locked in one sector size in the BIOS didn’t mean DOS had to.
The weakness of MSDOS in regards to sectors is that DISKCOPY and FORMAT would only work if every sector was the exact same size. That limitation was exploited with some copy protection schemes that used 256 byte (MS Word) or 1024 byte (Sierra) sectors for some tracks. The bootable versions kept the first track with 512 byte sectors.
OSes for systems not derived from the IBM PC used different sector sizes with 520 bytes and 528 bytes being common. The IBM PC improved performance by having the error correction done by the controller instead of reading a larger sector that included the error correction to be processed by the computer.
Yes, I know that some Japanese systems used 1K sectors on floppies, but I don’t know them well. I believe Fujitsu FMR did that too, not only NEC. Quite possibly others as well. DOS can definitely handle it.
The weakness of DOS was really a weakness (if we can call it that) of the BIOS, it has no provisions to work with different sector sizes. Especially not with different sector sizes on the same track. It should be possible to get the BIOS to read/write such disks by dynamically modifying the DPT, but the copy utility has no generic way to find out which sector is how big. Sure, there are tools that handle it, but those are written to work with the FDC on a low level, not on the DOS level.
Addendum: Those Japanese systems were also not compatible with PCs on the BIOS level, so they’re only somewhat relevant. But they certainly show that DOS always could handle sector sizes other than 512 bytes.
Its been ages but I recall somewhere that HPFS and Japanese hard disks not playing nice as they didn’t use 512byte sectors. I can’t remember for the life of me where/how I saw this other than it’s something weird that sticks in my mind.
For the brief time I had a pc98, and the numerous trips I took to Japan when I was in Hong Kong I never could find a Japanese version of OS/2 1.x so I really have no idea if the 1k drives were really a thing.
I believe HPFS may have assumed 512-byte disk sectors. What drives did those Japanese machines use? Not ATA if they used 1K sectors.
I’m a bit fuzzy on the history of Japanese OS/2, because there was the whole thing with all the different hardware platforms. Japanese OS/2 2.1 existed, and ran on standard PC compatibles. Supposedly there was OS/2 1.2 for PC-98 but it’s impossible to find.
MS-DOS 1.25 for Commodore 710 uses 256 byte sectors with GCR encoding (not MFM!) because that’s the underlying format of the 8050 disk drive.
Makes sense, Commodore (and Apple) used GCR. That of course is something DOS didn’t really care about, typically only the floppy controller would care about the actual encoding.
Internet Archive does have a copy of OS/2 Warp for PC-9800 machines which might answer questions about HPFS. The information I have for NEC hard drives indicates that the PC-98 DOS compatible units that used sector sizes other than 512 bytes had limited capacities and wouldn’t make much sense as HPFS boot drives.
DOS would need to know the registers to get the controller into the correct mode but thankfully the number of Apple GCR capable PD72070 FDCs were very limited.
Also, the Commodore MS-DOS is not booted from the boot sector but from a Commodore BASIC program that loads the first boot stage into memory and executes it via SYS. This boot code (written in 6502 assembly!) then loads IO.SYS into memory and calls it. So 256-byte sectors are not a problem in this setup, as the bootloader does not need to fit in one sector.
I wonder if MS-DOS compatibles (not IBM PC compatibles) had the same restrictions in practice. I suspect that at least a few OEM versions of MS-DOS or other, DOS compatibles OSes (DOS Plus, PC-MOS etc ?) had made provisions to support non-512 Byte sectors.
See article. There were quite a few MS-DOS compatibles with non-512-byte sector sizes. Most of them died out quickly, although Japanese systems with 1K sectors survived into the 1990s.
Or to put it differently, before the IBM PC killed off the competition, 512-byte sector were quite unusual. 128-byte and 1024-byte sectors were very common, and 256-byte sectors were also used.
I have Fujitsu MS OS/2 1.1 and 1.21 for FMR
Oh? Archived anywhere?
And do you have anything to run it on? 🙂