Is it so hard to document things?

A few weeks ago I spent a bit of time debugging a program which mysteriously failed under DOS 3.3, although it worked without any apparent problem on DOS 4.0 and later, and there was no indication that it required anything later than DOS 3.1 or so.

The problem turned out to be related to INT 21h, function 6300h, a curiously underdocumented DOS API. The call was first implemented in the Far East versions of MS-DOS 2.25 and returned a table of DBCS lead bytes. That is crucial information for software running on DBCS systems, required for correctly parsing pathnames etc. The RBIL also documents the call for Far East versions of DOS 3.2 and later. So what’s the problem?

The problem is that the call only exists in Far East version of DOS 3.x. But that’s not the whole story. Microsoft’s official documentation (e.g. the MS-DOS Encyclopedia) documents the INT 21h, function 6300h API to be only available in MS-DOS 2.25. The call is not documented in any of the official references for MS-DOS 3.3, 4.0, 5.0, or PC DOS 7.0.

So how are applications supposed to get the information? In DOS 3.3, the internationalization support was redesigned (by IBM?) and function 65h was used to return country-related information. However, perhaps because IBM did not support Japanese or other DBCS languages at the time, there was no documented way to obtain the DBCS lead byte table. In DOS 4.0, sub-function 7 was added to INT 21h, function 65h to return the table, in almost the same format as the one previously returned by INT 21h, function 6300h.

The official MS-DOS documentation does not clearly say whether INT 21h, function 65h, sub-function 7 is supported on US and other non-DBCS versions of DOS, but it is supported and returns an empty lead byte table.

What the official documentation doesn’t say is that DOS 4.0 and later also supports the old INT 21h, function 6300h API, in all national language versions. The upshot is that an application can call INT 21h, function 6300h on any Far East version of DOS at least since 3.2 (and 2.25) to obtain the DBCS lead byte table (verified with MS-DOS/V 6.2).

It is a mystery why Microsoft/IBM didn’t document this fact. In all official DOS 3.3 and later references, INT 21h, function 63h is marked as reserved. Did Microsoft want to force programmers to utilize  the newer INT 21h, function 65h API? But how were they supposed to get the information on DOS versions prior to 4.0? There seems to be no good reason not to document the API, as there’s nothing secret or potentially subversive about it.

It would seem that an application might then simply call INT 21h, function 6300h to obtain the DBCS lead byte table and just be done with it, documented or not. The catch is that on US versions of DOS 3.x, the INT 21h, function 6300h API simply does nothing. It does not indicate an error, it just returns without modifying any registers or flags.

So how to get the DBCS lead byte table without complicated DOS version checks and guesswork? It’s actually easy. The DS:SI registers should be set to 0:0 before calling the API. If a valid DBCS lead byte table is returned, the DS:SI registers will be modified (the table can’t possibly be stored at 0:0). If DS:SI are still 0:0, the API is not implemented, the DOS is not a Far East version, and hence there are no double-byte characters and no lead byte table.

Note that INT 21h, function 6300h is also implemented in the OS/2 2.0 DOS box and in NTVDM.  It is likewise implemented in DR DOS, at least versions 6.0 and later. Interestingly, the API is not documented in the DR DOS technical reference either, but the DEBUG.EXE utility in Novell DOS 7 displays an appropriate description (“DOS: Two byte chars”) when the INT 21h service is about to be executed.

Update: Last week I obtained a copy of Developing Applications Using DOS by Christopher, Feigenbaum, and Saliga. While it is not an official DOS reference book, it was written by the engineers who led the development of DOS 4.0 at IBM and it is clearly an unusually well informed book. INT 21h, function 6300h is documented in detail and marked as a DBCS-only function introduced in DOS 3.2. The function is labeled as “published”, that is supported by future DOS versions. There’s a general note that DBCS functions return an invalid function error on non-Asian DOS versions, but no specific remarks for INT 21h, function 6300h with regard to DOS versions.

This entry was posted in DOS. Bookmark the permalink.

66 Responses to Is it so hard to document things?

  1. Yuhong Bao says:

    Well, remember that the MS-DOS encyclopedia dates back to 1986. Maybe the DBCS version of MS-DOS 3.2 was not released at the exact time it was written.

  2. michaln says:

    Remember that there was a second edition published in 1988, which is what I was looking at. It describes DOS 3.3 (in an separate, fairly extensive appendix), so it can’t possibly be too old.

  3. Yuhong Bao says:

    I think the normal US DOS documentation do not describe it exactly because it was only applicable to DBCS versions of DOS.

  4. michaln says:

    Then why document the INT 21h, function 6300h API for the DOS 2.25 DBCS releases? If the US references ignored all DBCS related APIs, it would be somewhat understandable, but that’s clearly not the case.

  5. Yuhong Bao says:

    Well, notice only the MS-DOS encyclopedia document it.

  6. michaln says:

    And Advanced MS-DOS Programming. Both published by Microsoft Press.

  7. Yuhong Bao says:

    I think the question is that is it so hard to update documentation, because that is the problem.

  8. techfury90 says:

    I believe I have your answer.

    http://hp.vector.co.jp/authors/VA003720/lpproj/int10h/i216300.htm

    英語版の DOS 3.30 ではサポートされていませんが、NEC PC-9801 用の MS-DOS 3.3 ではサポートされています(たしかそんな記憶が…)。

    Rough translation:

    It is not supported by English DOS 3.30, but is supported by MS-DOS 3.3 for NEC PC-9801 (such memories…).

  9. Yuhong Bao says:

    Do you know what was the first version of DOS that supported it?

  10. techfury90 says:

    Unsure. I did find another site that said 3.2 supported it, though. Don’t have that link handy, though. I just went looking in all 3 dead tree books I have on PC-98 programming, dated 1989, 1992, and 1993. Neither mention it in their DOS API reference tables. I think I might have found the official NEC DOS programmer’s guide in PDF form for either DOS 5.0 or 6.2 in the past, though. Let me see if I can scare a copy of that up, it might have an answer.

  11. techfury90 says:

    PC-98 DOS 3.3C programmer’s guide from 1990 on archive.org makes no mention of it. Go figure.

  12. techfury90 says:

    http://island.geocities.jp/cklouch/column/pc98bas/pc98dosver_en.htm
    English reference of PC-98 DOS versions. The first publicly released version was 2.0 in November 1983. It states that this 2.0 is forked from MS-DOS 2.25. Maybe the PC-98 really is our match here? Too bad that the oldest PC-98 DOS that I have my hands on is “some version of 3.30″… maybe. Now that I think of it, I seem to remember that 2.0/2.25 may have been included with PC-98 Lotus 1-2-3? I have images of that, maybe they have what we’re looking for…

  13. Michal Necasek says:

    Interesting tidbit: In the DOS 5/6 days, DOS function 63h was called “ECS_Call”, ECS being “Extended Code System” (aka DBCS); it did something even in regular DOS versions, but did more in DBCS versions. In DOS 3.3 days though, it was called “Hongeul” (spelled that way) and did nothing in regular DOS versions. In DOS 2.11 days, it didn’t exist at all. It’s very likely that MS-DOS supported Korean first, then Japanese.

    The DOS 3.3 OEM adaptation guide (July 1987) documents DOS function 63h as a “MS-DOS 2.25 application level interface extension” and calls it “get_lead_tbl”.

  14. dosfan says:

    INT 21h function 63h is documented to be first supported in MS-DOS 2.25 which was only released in South Korea and possibly Japan. What’s interesting is that MS-DOS 2.11 only supports functions up to 58h though MS-DOS 3.0 supports functions up to 63h (function 63h is a NOP in U.S. versions until MS-DOS 3.3 and is basically a stub in later versions) so perhaps function 63h was defined during or after MS-DOS 3.0 development. It would be nice to get hold of MS-DOS 2.25 to verify what the INT 21h dispatch table does (presumably it checks for functions 00h-58h and 63h since functions 59h-62h don’t exist in MS-DOS 2.x).

  15. techfury90 says:

    2.25 was definitely released in Japan. The initial PC-98 non-beta DOS release in November 1983 was 2.25 branded by NEC as 2.0, yet shows as 2.11 when you run ver. I should also note that *many* PC-98 applications came with 2.25 included, as ASCII/Microsoft/NEC allowed developers to redistribute that release freely.

    Hell, I just went looking through some PC-98 disk images and I have a copy of it. Need to throw together something to test INT 21h AX=6300, though…

  16. Michal Necasek says:

    MS-DOS 2.25 was reportedly released in October 1985, that is after DOS 3.1 and when DOS 3.2 development was probably in full swing (says Ray Duncan in Advanced MS-DOS Programming). That would easily explain why a “2.25 only” function was numbered after functions appearing in 3.1.

    The MS-DOS Encyclopedia claims that Kanji support was added in MS-DOS 2.01. DOS 2.1 for IBM was different. MS-DOS 2.11 was merged PC DOS 2.1 plus MS-DOS 2.01 with internationalization. The Encyclopedia claims DOS function 63h is only supported in DOS 2.25. MS-DOS 4.0 Programmer’s Reference does not list function 63h, even though AFAIK MS-DOS 4.0 did support it (but it also had an alternative).

    In the MS-DOS 3.21 OAK (early 1987), there is again evidence of function 63h aka ECS_call, and in fact the PRINT.EXE and SYS.COM source code refers to it, but the function is not supported and calling code is compiled out. It’s properly documented (as a 2.25 extension) in the adaptation guide, too.

    The information about Far East versions of DOS in US-centric literature tends to be extremely sparse. That is likely why DOS function 63h is so poorly documented in the first place.

  17. Yuhong Bao says:

    @techfury90: You should be able to just use DEBUG.

  18. dosfan says:

    @techfury90

    Is it possible that you could upload MS-DOS 2.25 disk images somewhere ? I would really like to analyze it as it is the only version of PC DOS / MS-DOS I haven’t looked at.

  19. techfury90 says:

    That is interesting. I wonder why that reference claims that the PC-98 DOS release from 11/83 is 2.25, then? Maybe there was a later re-release that was 2.25 based?

    Yes, DEBUG should do it, although I’m not sure if I have a DEBUG version that’ll run on this DOS release. I only have DEBUG for PC-98 DOS 6.20 handy. On the other hand, I have Turbo C for PC-98, so I could just hack something up in there real fast. I’ll poke at it later today.

  20. techfury90 says:

    I unfortunately don’t have disk images of the distribution, nor am I sure it’s actually DOS 2.25. It claims to be 2.11 with a copyright of 1983, yet the PC-98 DOS version list says it’s 2.25, but as Michal says, this might be wrong. It was included with PC-98 Wordstar 6, which is completely left in English…

    Here’s where I found it: http://slt.retroprograms.com/Wordstar.html

    You’ll probably want this so you can extract the files: https://hp.vector.co.jp/authors/VA013937/editdisk/index_e.html

  21. dosfan says:

    I’ve seen both 11/83 and 10/85 release dates for MS-DOS 2.25.

    Note Advanced MS-DOS Programming claims that DOS 3.1 was released 11/84 but this is inconsistent with the info I have which says it was released 4/85 (the timestamps on the PC DOS 3.1 files are 3/7/85). As far as I can tell it was MS-DOS 3.05 that was released 11/84. Anyway original MS-DOS 2.25 disk images would definitively answer this question.

  22. Michal Necasek says:

    Heh, 1024-byte sectors. The whole thing is from late 1990s. The DOS system files are from May 1988. But the copyright dates in the system files are 1983 and the reported version is 2.11. If I had to guess, it is MS-DOS 2.11.

  23. Michal Necasek says:

    MSDOS.SYS function dispatcher supports functions up to 58h. That is consistent with MS-DOS 2.11.

    So… who the heck has the real MS-DOS 2.25?

  24. techfury90 says:

    Yeah, it didn’t do anything for me with a test program. I would guess that you’re right, it’s probably from some unknown Korean release judging by the references to Hangul. The conspicuous lack of it in several otherwise fairly complete third party PC-98 references makes me inclined to agree as well.

    Unfortunately, I can’t read Korean (just English, German, and Japanese), so I wouldn’t know where to keep digging.

    Incidentally, 1024 byte sector support was one of the things that pushed NEC to license MS-DOS. I read several Japanese blog posts a month or two ago (wish I had saved the link…) from one of the first engineers who developed the PC-98, actually. They had not really considered the IBM PC at all when designing the PC-98. It was simply meant as an upmarket somewhat compatible complement to their PC-88 8 bit machines. This changed when NEC management sent the team on a trip to the USA around Christmas 1982 to take a look at what 16 bit machines in the USA looked like and were used for. They detoured to Seattle before heading back to Japan to hash out a licensing agreement with Microsoft for MS-DOS on the PC-98 after seeing it in use on the IBM PC. At either rate, one of the compelling features to NEC was support for 1024 byte sectors, which CP/M-86 allegedly couldn’t do at the time, along with the nascent DBCS support Microsoft was working on for Toshiba’s PASOPIA16.

  25. techfury90 says:

    There’s an Amazon.co.jp seller that has ASCII’s “MS-DOS Programmer’s Handbook” for just 889 Japanese yen. The publication date is April 1, 1985. Think I should grab it and see if it mentions AX=6300h? I already have the necessary account and means to get it forwarded to myself in the USA…

  26. Yuhong Bao says:

    @dosfan: I don’t think that version will run on PC-98. I already have the idea to point INT 28h to an IRET so it can still be used to debug the DOS kernel itself.

  27. techfury90 says:

    I have Turbo Debugger from Turbo C++ 4.0J, CodeView from MSVC 1.52 PC-98, and SYMDEB and DEBUG as shipped in the PC-98 DOS 6.2 Extended Function Set (yeah, you had to pay for DEBUG, SYMDEB, LINK, and a few other things after 3.3B)

    The former two still have English UIs by default.

  28. Yuhong Bao says:

    (this is needed because INT 28h routines may use the DSKSTACK while debugging DOS kernel code that is already using it, while DEBUG would call DOS routines that use the IOSTACK where INT 28h is called from)

  29. Michal Necasek says:

    Another data point: SYSTEM3.DOC aka MS-DOS 3.05 Adaptation Guide addendum from 11/30/1984 (about midway between PC DOS 3.0 and PC DOS 3.1 releases) lists DOS function 63h as “Hongeul” without offering any further comments or actual documentation. (The same document also lists INT 24h error code 0Dh as “used by MT-DOS”, our good friend Multitasking DOS 4.0).

    The corresponding MSDOS.SYS dated 11/28/1984 and calling itself 3.06 indeed appears to accept DOS functions up to and including 63h, although I don’t think the function does anything.

  30. Michal Necasek says:

    That’s interesting. MS-DOS supported 128-byte and 1024 sectors probably before it was even called MS-DOS (i.e. when it was 86-DOS). IBM had 512-byte sectors in 1981, DEC (or was it HP) had 256-byte sectors in the early 1980s. So MS-DOS worked with 128-, 256-, 512-, and 1024-byte sectors pretty early on.

    I am getting more and more mystified by this MS-DOS 2.25 thing. I cannot find any solid evidence that it ever existed. It is mentioned in 1980s programming references, including documents from Microsoft, but I can’t find any mention of a vendor actually releasing MS-DOS 2.25, leave alone screenshots or disk images.

    The supposed timeline (release in October 1985) makes limited sense as that was over a year after DOS 3.0 release. In October 1985, NEC was already releasing Japanese DOS based on MS-DOS 3.1. Which I bet supported DOS function 63h.

    Japanese BASIC.EXE shipped with IBM DOS J4.05/V reportedly calls INT 21h/63h to verify that it’s running on a DBCS system.

  31. techfury90 says:

    There’s another Japanese book on Amazon circa 1988 that claims to cover undocumented aspects of the PC-98 BIOS and DOS. I’m going to buy it and see that it has to say.

    BTW, IBM Japan had DBCS DOS in 1983 on the 5550 Multistation series, which evolved to the PS/55 in 1987/1988. There was also the JX of 1984, which was the Japanese adaptation of the PCjr. DOS/V is actually descended from PS/55 DOS, just modified to use VGA instead of IBM Japan’s 1024×768 display adapter with built in kanji font ROM.

  32. Yuhong Bao says:

    The history of Korean DOS turned out to be interesting:
    https://www.betaarchive.com/forum/viewtopic.php?t=27393
    I wonder if MS-DOS 2.25 was for a specific DBCS OEM.

  33. techfury90 says:

    Yeah, I did a bit of research last night and discovered that Korean DOS typically ran on otherwise vanilla PCs that had DBCS display adapters. Interesting that they used a SJIS based encoding and then moved to an EUC based one. In Japan, EUC mainly only sees use on Unix based systems. DOS has always used CP932 in Japan.

    More or less, it appeared from my research that the most common PC vendors in Korea in the 80s were Samsung, GoldStar/LG, and TriGem. These 3 apparently had their own DBCS display adapters. I did see a hint that later on, it was implemented using drivers in conjunction with SVGA cards. Who knows what they had before.

  34. techfury90 says:

    http://forum.softhistory.org/showthread.php?tid=23

    That’s where I saw some of that.

  35. techfury90 says:

    Also, I forgot, there was the AX group in Japan circa 1986, which developed a DBCS capable spin on EGA. It was a consortium led by NEC’s competitors (mostly Hitachi, Mitsubishi, Matsushita, Oki, IIRC) that failed to gain traction in the marketplace.

  36. Michal Necasek says:

    OEM or OEMs. It seems to be another Multitasking DOS 4.0, technically released but so rarely used that it may never even have existed. With the difference that before the disks turned up, at least some of the MT-DOS 4.0 OEMs were mentioned in the press and on the web.

    The published timeline really really makes me wonder if the Japanese (and Korean?) OEMs didn’t just switch to MS-DOS 3.10 instead. That was already out by the time 2.25 supposedly came out.

    I have not been able to find any mention of anyone ever shipping MS-DOS 2.25 (which is certainly not a proof that no one did). Microsoft clearly said at some point that it came out in October 1985, although October 1984 would have made a lot more sense. The fact that the “Hongeul” INT 21h/63h was listed in a Nov ’84 document is also interesting. And 2.25 was very clearly mentioned and documented in the 1987 MS-DOS 3.21/3.3 OEM Adaptation Guides.

    The sheer lack of surviving Japanese/Korean disk images is vexing.

  37. Michal Necasek says:

    MS-DOS 2.11 had some DBCS support, but I don’t know how much. The DOS 2.0/2.11 source code on CHM contains quite a few references to ‘IBMJAPVER’ and ‘KANJI’, and that’s from roughly August 1983. What DOS 2.11 did not have is that funky 63h DOS function.

  38. techfury90 says:

    It sounds vexing at first, but it really boils down to “Japanese folks seem to respect the letter of copyright law”, unfortunately for us. Something tells me it’s a similar story in Korea.

    I don’t think I have PC-98 DOS 3.10. Oldest I have images of is 5.0, maybe some 3.3 release (NEC released several subversions of 3.3), unfortunately.

    What I have learned is that the usual method of piracy in Japan is a homegrown P2P network known as Perfect Dark. I have managed to find some PC-98 stuff from there, mostly H game compilations (I know, so shocking and surprising), but I did find a bunch of PC-98 disk utilities amongst other things the last time I looked. I’ll see if I can scare up 3.X that way.

  39. techfury90 says:

    I suspect that 63h wasn’t really necessary for Japanese users. If you look at CP932, which all Japanese versions of DOS (and Windows) use, only 81h-9Fh and E0-FCh are used as lead bytes for a two byte character. Realistically, you could simplify that to 80H-9Fh and E0-FFh.

    Remember, most of CP932’s upper half are 1 byte katakana nestled in between the two lead byte groups.

  40. Michal Necasek says:

    I would suspect a much simpler explanation, the software was far less widespread to begin with, and the mechanisms which tend to preserve data (warez CD-ROMs, BBSs, Internet) were not in place. Sneakernet is good for pirating but not for preserving software (and it is a huge irony that a lot of software survived to this day only thanks to piracy).

    Consider the difficulty of finding WordStar 3.02 or 3.20 for IBMs. It was one of the most popular software packages at the time. Nowhere to be found.

    I think winworldpc has some kind of Japanese DOS 3.1 for Toshibas, and also Japanese MS-DOS 3.2/3.3. But nothing 2.x.

    I have an incomplete copy of PC DOS J4.05/V — oldest DBCS PC DOS I could find, and that’s from 1990.

  41. techfury90 says:

    Also a very valid point. On the other hand, Japan *did* have BBSes in the day, along with several online services. I know they were “notorious” for piracy in the day, or so Japanese Wikipedia tells me.

    I always wondered why BBSes seemed to be a mostly North American (and Japanese) phenomena anyway.

    4.05/V is probably the earliest Japanese DOS that doesn’t need special hardware.

  42. Michal Necasek says:

    I don’t think BBSes were in any way specific to North America/Japan. I know BBSes were pretty common in the early to mid-1990s in Europe. I also know Fidonet was popular in Russia for a very long time. What the US had was the big commercial online services like CompuServe.

    Was 4.05/V actually the first DOS 4.0/V?

  43. techfury90 says:

    Ah yeah, the online services. Japan had PC-VAN from NEC and Nifty-Serve (JV with Fujitsu and CompuServe) as its main online services. They showed up around 1986, soon after Japan started privatizing their state-owned phone company in 1985, allowing cheap modems to be bought/used. Both actually survive to this day as broadband ISPs.

    Uncertain. I’d have to look at some stuff to see. I do know that IBM had DOS/V before MS sold it, despite the joint development. Word has it that MS dragged their feet on releasing it to OEMs because they didn’t want to anger NEC, as their NEC contract was larger than all the other Japanese OEMs combined. As a result, most early DOS/V PCs actually came with IBM PC-DOS.

  44. techfury90 says:

    http://island.geocities.jp/cklouch/column/dosvhist.html

    Says 4.05/V was the first, shipped November 5, 1990.

  45. Yuhong Bao says:

    “The published timeline really really makes me wonder if the Japanese (and Korean?) OEMs didn’t just switch to MS-DOS 3.10 instead. That was already out by the time 2.25 supposedly came out.”
    Which is what made me suspect that it was only for certain OEMs in the first place. I hope it is at least known which OEMs used it.

  46. techfury90 says:

    Well, if you feed the Korean Wikipedia article for MS-DOS into Google Translate, it has the following to say about versions of Korean DOS:

    MS-DOS in Korean

    MS-DOS 2.11 May 24, 1985 [5] – Combined Hangul code
    Microsoft MS-DOS 4.0 May 1990
    MS-DOS 5.0, July 1991
    MS-DOS 6.0, August 1993
    MS-DOS 6.2 September 1994

    So Korea skipped 3.x and went to 5.0? I can actually kind of believe it.

    If we look to the PC-98 as a point of comparison, the de facto DOS version was 2.11 for quite some time after 3.x was out. Most people didn’t buy DOS 2.11, as it came included on software disks. HDDs weren’t really standard on PC-98s until the DOS 5 era, so buying the DOS retail package was mainly only done by those who had an HDD.

  47. techfury90 says:

    There is also a more general overview of DOS versions on that article.

    2.11 1983 – Support for non-English languages ​​and date formats.
    2.25 1985 – Expanded support for Korean and Japanese kanji. Released only in East Asian countries.

  48. techfury90 says:

    Japanese Wikipedia claims that some version of 2.25 made it to Japan labeled as 2.11 (but allegedly contains “2.25” somewhere in MSDOS.SYS)…. except it’s labeled as citation needed. Great. Now we just need to figure out who licensed MS-DOS 2.11 and/or 2.25 in Japan and Korea.

    For Japan, it appears that NEC released two re-issues with different SKUs in 1986 for PC-98 DOS 2.11. Maybe one of these is the mysterious Japanese 2.25? The changes are listed as being improved versions of the NEC supplied Japanese input method editor, but maybe this prompted a behind the scenes change to 2.25 for some reason?

    Another possibility is Fujitsu. They licensed MS-DOS for their FM-16B (Greek letter beta), which was a PC-98 competitor.

    http://museum.ipsj.or.jp/en/computer/personal/0014.html

    Looks like they got MS-DOS “2.1” and 3.1 by April 1986, possibly (I’m guessing they took those versions out of a brochure), but didn’t have it at launch in December 1984. That would leave us with a date of some time in 1985 or early 1986 for FM-16B MS-DOS “2.1”. Another possible candidate for such a version, if it even exists.

    For Korea: I suspect the possible answers are TriGem, Samsung, and/or GoldStar (present day LG). I’ll try to see what I can find from Korean Wikipedia by way of Google Translate.

  49. Michal Necasek says:

    DOS 2.x survived rather long in some corners because the major difference from 3.x was the lack of networking support and larger hard disks. For those interested in neither, there was limited value in upgrading from 2.x to 3.x.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.