As previously mentioned, the OS/2 Museum adapted the Phantom redirector example from the second edition of Undocumented DOS to demonstrate that the redirector interface was already fully implemented in the August, 1984 release of PC DOS 3.0, a fact apparently unknown to most authors of DOS literature.
The Phantom 3.0 redirector is now available for download in the form of a 360K floppy image. Kudos to the authors of Undocumented DOS for writing it in the first place. Needless to say, the code is provided as is, and although bug reports are welcome, no promises are being made as to if or when any issues might be fixed.
Bug Fixes
This project turned out to be a bit more involved than it should have been, because the Phantom redirector as delivered with Undocumented DOS contained a surprising number of bugs. Following is a list of the issues discovered and fixed:
- Crash on startup. In set_up_xms_disk(), Phantom called the dos_ftime() routine to get the current time in a format suitable for storing in a FAT directory entry. Unfortunately the dos_ftime() routine was only meant to be called from within the INT 2Fh handler and used uninitialized data during startup. This did not cause visible harm on DOS 5.0/6.x but caused hangs with DOS 3.x/4.0.
- Truncated directory searches. Phantom contained an embarrassing bug which caused directory searches to be limited to 32 entries (one 1,024-byte sector). This was trivially fixed in find_next_entry() by resetting the index variable after exhausting each directory sector.
- Problems with larger directories. Indirectly related to the previous bug, there was a really embarrassing blooper in get_dir_start_sector(). A spurious semicolon after an if statement caused Phantom to fail finding a subdirectory if it wasn’t located in the first sector of its parent directory.
- Inability to create directories on non-empty drive. A slightly less embarrassing bug in Phantom resulted in 16-bit instead of 32-bit arithmetic being used in get_dir_start_sector() to calculate sector offsets for directory searches. As a result, directory creation and searching worked well on an empty disk, but failed horribly after a few files had been copied onto the Phantom disk.
- Bad CHKDSK/LABEL behavior on DOS 4.0. Some DOS commands, including CHKDSK and LABEL, detect non-local drives via INT 21h/4409h. On DOS 4.0, that causes the ‘IFS header’ pointer in the CDS to be dereferenced. Because Phantom left this pointer as NULL, DOS was examining memory in the interrupt vector table (IVT) and returned unpredictable results. Worse yet, this could cause unpredictable memory corruption. This was fixed by providing an IFS header with contents crafted to placate DOS.
- Spurious file write errors. Phantom was not updating the open mode in the fill_sft() routine, which caused some files to be non-writable on DOS 3.x. When a file is created via INT 21h/3Ch, it should be opened for reading and writing (regardless of the attributes given). This was fixed by forcing the open mode in the SFT to allow writing when a file was created this way.
- Incorrect CD behavior on DOS 3.0. This problem is perhaps not so much a bug in Phantom as the documentation it was based on. On DOS 3.0, the symptom was ‘CD FOO’ resulting in the current directory being set to ‘FOO\FOO’. This was fixed by not updating the CDS in the redirector in function cd() and letting DOS do it. See below for details.
- Incorrect CD behavior with deep directories. Changing into directories more than two levels deep, such as ‘\A\B\C’, failed to work and ended up in \A instead. The problem was that Phantom modified the pathname buffer in the SDA, which clearly wasn’t supposed to happen. Fixed by restoring the buffer before returning to DOS.
- Incorrect findfirst error code. The TREE utility in DOS 4+ failed to traverse the entire directory tree, stopping after the first leaf directory. Fixed by adjusting error codes returned by findfirst (INT 2Fh/11Bh). More details below.
It is apparent that Phantom was never properly tested, especially not with older DOS versions (3.x/4.0), even though Phantom contained code to explicitly support those versions. Given that, it is likely that some bugs in Phantom still remain. Nevertheless, Phantom remains an invaluable example of DOS redirector programming, and the known bugs were fairly easy to fix once the problems were understood.
One of the more interesting misconceptions in Phantom was clearly caused by a lack of any official documentation. The INT 2Fh/1105h redirector callback is executed by DOS when changing directories. The Phantom authors (and RBIL contributors) clearly thought that it was the redirector’s job to update the CDS, but in fact the callback is only meant to validate that the target directory exists; DOS subsequently updates the CDS. This caused the most visible problems on DOS 3.0, but the fix (not updating the CDS in INT 2Fh/1105h callback) works well on all DOS version.
Another interesting issue was the handling of findfirst/findnext error codes. This was causing obvious errors with TREE (on DOS 4+) where the display stopped after traversing into the first leaf directory. The DOS findfirst function (INT 21h/4Eh) is documented to return three possible error codes, 2 (file not found), 3 (path not found), and 18 (no more files). Microsoft’s official documentation is poor and does not explain which error is returned when. TREE obviously expects error 18 (no more files) to be returned when searching for subdirectories and none are found; Phantom was returning error 2 (file not found), confusing TREE.
The problem was in part caused by Phantom not creating the ‘.’ and ‘..’ entries in each subdirectory; on a typical DOS disk, a findfirst for entries with the directory attribute bit set and a ‘*.*’ search pattern will always find at least those entries in each subdirectory. For Phantom, that was not the case. The solution was to return error 18 (no more files) from findfirst if the search pattern contains wildcards and no matching entry was found. If findfirst finds nothing but the search pattern did not contain wildcards, error 2 (file not found) is returned.
During testing, it was found that DOS 3+ does not appear to return error code 2 from findfirst (INT 21h/4Eh), although it could perhaps happen under unusual circumstances. Typically the returned code will be 3 (if search pattern is invalid, such as ‘C:\FOO\’) or 18 (if search pattern is valid but no matching entry was found).
Some DOS references claim that findfirst returns either error 2 or 18, which is demonstrably untrue with DOS 3+. However, it was probably true for DOS 2.x. The behavior documented for DR-DOS is not what was observed with MS-DOS/PC DOS; namely MS-DOS/PC DOS always returns error 18 on failed searches regardless of whether the search pattern included wild-cards or not. Microsoft’s documentation is, as usual, silent on the topic; findfirst is documented to return error codes 2, 3, and 18, but no hint whatsoever is given as to which error code might be returned when (the RBIL is, sadly, no better). Which would of course be fine if DOS utilities didn’t depend on specific error codes being returned from findfirst…
Modifications for DOS 3.0
Adapting Phantom to work with PC DOS 3.0 was a straightforward process. Almost all of the modifications needed to account for differences in the layout of internal DOS structures, but the redirector interface itself needed no changes.
The DOS data segment in DOS 3.0 was laid out differently from later versions, requiring changes to the List of Lists processing code and a different SDA layout. In addition, the SFT layout in DOS 3.0 was slightly different, with the filename being offset by one byte (DOS 3.0 stored a 16-bit offset into a directory sector rather than the 8-bit index of the relevant directory index).
As mentioned above, the change directory logic had to be modified to work correctly on DOS 3.0, but this was strictly speaking a fix to better conform to the DOS redirector interface rather than a version-specific modification.
Impressions
While working with Phantom, it was inevitable to become acquainted with certain aspects of internal DOS operation. It soon became apparent that as far as file and directory functionality is concerned, the differences between DOS 3.0 (1984) and DOS 6.0 (1993) are not nearly as big as one might be led to believe. While there are numerous minor differences, the basic structure remained constant across all versions. It certainly didn’t look like any kind of “rewrite” happened at any point between 3.0 and 6.x.
It may not be unjustified to say that DOS 3.0 was the last truly major version of DOS, with the following releases only bringing relatively minor evolutionary changes which did not fundamentally affect the core DOS operation and structure. In a way that should be expected: MS-DOS 4.0 (the near-mythical multi-tasking MS-DOS 4) was where significant changes occurred after 1984. And DOS 5.0, later released as OS/2, was where truly major changes in DOS occurred after 1985-1986, while DOS 3.x was receiving only very minor updates.
For all the hoopla surrounding the “regular” DOS 4.0 (1988) and 5.0 (1991), neither of those versions brought anything like the changes between DOS 1.x and 2.0 or 2.x and 3.0. MS-DOS 6.0 was widely known at the time it was released (1993) not to include any significant changes to the core of DOS, however important a release it may have been for other reasons. Again not surprisingly, since at that time DOS was so entrenched and simultaneously obsolete that major changes did not make sense. Windows 95 was a different story of course…