While working with a modified BIOS image in a virtual machine, I ran into mysterious hangs when trying to load an old version of EMM386.SYS from MS-DOS 4.01. A newer version of EMM386 did not exhibit the problem.
A quick debugging session showed that the culprit was an uninitialized interrupt vector 68h. The BIOS POST left the vector as zeros; newer EMM386 versions verify that the vector is non-zero before calling through it. That explains why only the old EMM386 locked up, but it doesn’t explain what EMM386 was trying to do in the first place.
Ralf Brown’s Interrupt List (RBIL) is uncharacteristically quiet. There are two incomplete entries saying that Novell’s EMM386.EXE uses INT 68h, but there is no detail and lots of question marks. There is also a note that the Windows debug kernel (Windows 3.1 perhaps?) uses INT 68h to output debug strings. It’s also worth noting that most of the interfaces provided by EMM386 use INT 67h. Googling around suggests that INT 68h might also have something to do with Soft-ICE.
As it turns out, the correct answer is “all of the above”. Not just Novell’s but also Microsoft’s EMM386 calls INT 68h. It is related to Windows 3.x kernel debugging, and Soft-ICE supports it as well.
INT 68h is a semi-undocumented debugger interface. Its origin is unclear; it is known to be supported by Microsoft’s WDEB386 debugger and Soft-ICE. However, that does not explain why EMM386 would use it, since WDEB386 is always loaded after EMM386.
It appears that a similar debugger called DEB386 existed, probably in the form of a loadable DOS device driver with a $DebugDD signature. It is not clear if Microsoft ever published the driver, but published EMM386 versions certainly look for the $DebugDD string.
When EMM386 loads, it can use the INT 68h interface to output strings to a debug console. But EMM386 also switches the CPU to protected mode, which would disable any real-mode debugging interface… and to that end, EMM386 calls INT 68h and passes the address of several internal structures to the debugger. Presumably the debugger is then able to operate in protected mode as well.
The protected-mode debugging interface uses INT 41h, which the RBIL documents in better detail. The INT 41h interface was used heavily for Windows 3.x kernel debugging, and again it was implemented by WDEB386 and Soft-ICE. The interface may have existed in earlier Windows/386 releases as well.