Several years ago, I noticed that in some virtual and physical environments, the DOS Shell in both IBM’s and Microsoft’s DOS 4.0x has a very odd glitch: the mouse pointer moves, but does not register any clicks… except on scroll bars, and even then only partially.
On and off, I tried to find the source of the problem. I suspected the keyboard controller (with PS/2 mice), keyboard BIOS, PS/2 mouse BIOS… but never found any bug. Not long ago I finally found the cause of the problem, and it was not at all what I had expected.
The problem didn’t seem to be in the mouse itself, or the mouse driver. In fact the same problem happened with different mouse drivers, and whether nor not the shell’s built-in PS/2 mouse driver (PCIBMDRV.MOS) was used. I tried disassembling the built-in driver but found it to be quite simple and highly unlikely to cause trouble.
Later I made another interesting observation: On one system where the DOS Shell didn’t work right with a mouse, running the shell in an OS/2 VDM magically fixed the problem. On the other hand, running the shell in a Windows 3.x DOS box did not!
That made me wonder about the emulated keyboard controller hardware and how it might differ between Windows and OS/2. Fortunately the virtual keyboard device source code for both systems is available in their respective DDKs.
Examining the code made me realize that Windows 3.x essentially passes through most keyboard controller commands, but OS/2 does not and emulates much of the controller state. Perhaps by luck I happened onto the culprit: The keyboard enable bit (or keyboard inhibit bit, depending on which reference one reads), or bit 4 in the keyboard controller status register which may be obtained by using keyboard controller command C0h. What does that have to do with mouse input? Well…
Some readers of this site may remember that IBM PC/AT machines came with a curious gadget: a key lock (or key switch) mechanism. Most users considered it useless, but it was no doubt required by some corporate customers.
What the key lock did was exactly what the name suggests: it locked the keyboard. When the keyboard was locked, the system wouldn’t boot. A true blue PC/AT would first say “302-System Unit Keylock is Locked”, then “-Unlock System Unit Keylock”, and finally “(RESUME = “F1″ KEY)”. And obviously until the keyboard was unlocked, pressing the F1 key didn’t do much.
When the keyboard was locked in a running system, this simply inhibited any keyboard input but the system remained running. This was perhaps intended for leaving systems unattended and prevent accidental or intentional interference (janitors, untrained personnel).
The writers of the DOS Shell at IBM clearly thought about this scenario and realized that if the mouse can control the shell, a malicious user can still do a lot of damage even when the keyboard is locked—for example by deleting system files (which can be done by just using the mouse). That’s a problem, at least if one cares about the key lock.
So the DOS Shell writers decided to be clever. The shell appears to employ a fairly standard event-driven user interface. At some point mouse clicks are converted to internal events, and at that time the shell checks the keyboard inhibit state (i.e. the key switch locked/unlocked position). If the keyboard is found to be locked, mouse clicks are simply discarded.
Ironically, by the time the DOS Shell came out (1988), the key lock mechanism was obsolete. The XT Model 286 did not come with a key lock at all. PS/2 machines replaced the mechanical switch with a password mechanism which acted like a keyboard lock but could not be easily engaged in a running system, at least to my knowledge.
The DOS Shell writers were well aware of that and checked the machine model and sub-model before reading the keyboard lock state. And that’s where things get very messy.
On an IBM XT Model 286, the keyboard inhibit status bit always indicates an unlocked keyboard. Clone PCs presumably behaved like either a PC/AT (real keyboard lock) or an XT Model 286 (permanently unlocked). This was probably the situation when DOS 4.01 was current.
DOS 5.0 (1991) used a different shell written by Microsoft and the input handling was different, presumably ignoring such archaisms as a keyboard lock. Clone makers, but even IBM (for example ThinkPad T23) forgot that the keyboard lock had ever existed, but pretended that their systems were PC/ATs.
The upshot is that on at least some (if not most) newer systems, keyboard controller command C0h will indicate that the keyboard lock is engaged. On such a system, the DOS shell in IBM/MS DOS 4.0x will ignore mouse clicks because it will think it’s a PC/AT with a key lock that’s in the locked position. Ironically, keyboard input will work just fine because the Shell doesn’t bother checking the inhibit switch and instead expects keyboard input to be prevented on the keyboard controller level (as it would be on a system with a real keyboard lock).
OS/2 2.0 was written long enough ago that the keyboard inhibit switch had not yet been forgotten, and OS/2 MVDM faithfully emulates PC/AT behavior (see PKBDCMD_QUERYINHIBIT and VDMQueryInhibit in the OS/2 DDK source code). Windows 3.x simply passes through the real hardware state, therefore DOS Shell on a given system behaves the same whether running under Enhanced mode Windows 3.x or not (at least with regard to the keyboard inhibit state bit).
All in all, quite a puzzle.
i like dosshell from pc-dos 4.01
Just a small correction: I seem to remember that in the 1990’s timeframe my 386 (sorry, I don’t know the exact manufacturer) still had the “key thingy”. Never knew that you could read its status through software!
Keep up the great work, very interesting blog!
Sorry if I wasn’t clear… some clones did come with key locks, maybe up until 1995 or so. Might be interesting to know when key locks finally died out 🙂
Normally the only software that would read the status bit was the BIOS, in order to display a warning or error message. Because the key lock worked on the keyboard controller level, typical software couldn’t care less about it — if the keyboard was locked, there was just no keyboard input. And that was the case whether software used the BIOS to read from the keyboard or not.
Another minor correction. On the IBM PC/AT, I seem to recall that the key lock also physically kept the top case from being opened. You could remove the screws on the back, but the top did not slide forward enough to remove it (without physically bending it, in which case you could tell there was a breach).
That I did not know — I’ve only seen key locks on clones where it strictly prevented keyboard input but had no other mechanical function. But it doesn’t surprise me that the PC/AT design was clever.
How did these clicks squeak through?
This is what (I’m guessing) happens: The scroll bars can be dragged around, but an input event is only generated when the mouse button is released and the scroll bar lands in its final position. That event will be blocked. But the scroll bar dragging is not blocked, because that does not produce any input.
Keyboard locks lasted through the early 2000s on Point of Sale systems.
More relevant to the BIOS discussion was that many systems had optional case/keyboard locks. NEC did that on 386 desktops and other companies removed the lock from the desktop case but kept it with the server case. Since the BIOS and motherboard is the same, the BIOS default of showing the keyboard as locked might make the optional key lock easier to add and it still will work 99+% of the time.
Note: Some NEC Powermate 386 systems described above included DOS 4.01. I don’t have a copy of it so I can’t know if they had this bug or if the DOSSHELL got patched to solve it or if NEC used a different hardware design that prevented the problem.
Aside: IBM had a $50 option (IBM PC Keylock Feature) released at the same time as the AT to give earlier systems similar protections. I have never seen one.
Second Aside: I think that Topview does not lock down the mouse in the way DOSSHELL did when the keyboard is locked. Not sure I tested it correctly though.
As long as the systems without a key lock behaved the same as the XT Model 286 (keyboard never “inhibited”), there would be no problem. Alternatively, the system may not have reported a BIOS model/submodel matching the PC/AT.
Looking at the shell again, it checks for model 0FCh and submodel 1 or 2 (i.e. PC/AT but not XT Model 286). This was presumably done because the shell accesses the keyboard controller directly and assumes that anything that’s not a PC/AT might have different keyboard hardware (PCs and XTs obviously did, though PS/2s not really). I do know that for example older ThinkPads (720, 750, probably other 700-series as well) reported a different model.
So the Powermate 386 might have reported a model other than PC/AT or it may have behaved as if the keyboard was never locked. I would be very surprised if it had the problem. The bug really only showed up when AT “compatibles” all reported to be one of the original PC/AT models but no longer behaved quite like either of them.
ThinkPad 720s were Micro Channel-based, so I would expect them to have a different machine type/subtype than the PC/AT.
I always noticed the DOS 4.0 DOSSHELL sure felt sluggish on a real PC/AT; if it was chit-chatting with the keyboard controller for every mouse click, that would explain some of it.
Sure, the 720s were MCA systems but the 750s were not.
And yes, on a PC/AT, every mouse click would involve a trip to the keyboard controller to check if the keyboard is locked or not. Fun stuff.
It’s been a while since I’ve touched a 750. What machine type and subtype does it report?
Mine reports model F8h, submodel 61h, revision 09. That’s with BIOS 91G0716 (03/31/95).
Sysinfo from Norton Utilities 8.0 thinks it’s a PS/2 machine, which raises the question “what is a PS/2 machine, really”?
What easily detectable (through software) differences are there between a PS/2 and non-PS/2 system? Thinkpad 750 has the Extended BIOS Data area in addition to the mouse port. Some of the things I have read suggest the 750 used the watchdog timer to identify the contents of the bays so presumably the related BIOS functions were available.
If MCA is present then it’s definitely a PS/2. Otherwise things get tricky – model byte F8h is supposed to indicate a PS/2 (which is what Norton Utilities is assuming) but the PS/1 also used that model byte as did a few ThinkPad and ValuePoint systems (foolish IBM) and the PS/2 model 25 and 30 systems used model byte FAh.
It seems that ultimately it’s the look of the system that determines if it’s a PS/2 or not but of course you can’t detect that programmatically.
You folks raised an excellent point here that it’s hard to know what is a PS/2 and what isn’t.
A PS/2 Model 25 (8086 version) or a PS/2 Model 35 SX are certainly PS/2s, despite having ISA buses and generally acting like a PC/AT.
Is a Reply or NCR Micro Channel-based machine a “PS/2”?
The EBDA and a PS/2 mouse port were standard features of all AT-compatibles roughly 1995 and later… and those invariably report themselves as model 0FCh, submodel 1. I honestly don’t know how to tell. I think the correct answer is “properly written software doesn’t care” 🙂
One difference that Model 30 had relative to the PC/AT was a slightly different floppy controller with additional registers. And I think also reversed disk change line polarity, which was so messed up that even IBM’s OS/2 drivers tried to detect the polarity rather than “know” what it is on IBM hardware 🙂
Did anyone ever figure out the rationale behind the Model 25/30’s oddball diskette controller? That thing couldn’t handle 1.44MB capacity drives, and was also wholly incompatible with anything from the PC/AT interface.
I’ve assumed it was a spiritual descendant of the PC Convertible, but my PC Convertible has been long gone since I actually checked this out.
I’m sure it had something to do with the fact that the PS/2 machines used significantly redesigned custom VLSI chips with much higher level of integration… and IBM’s illusion that it was setting standards with the PS/2 line. Maintaining old PS/2 floppy drives is Not Fun, everything is just different enough to be a hassle without adding any real value.
I know this comment is a good 6 months after the fact, but I just had to say that the investigative digging through PC history that you’ve done with figuring this out… Michal – never stop doing this, your work is awesome. It’s always a great read for me – I know this might be another slow clap from the choir, but just wanted to shout out that your work is greatly appreciated by this lurker =)
6 months after the fact is relatively early. The interesting thing about this blog is that sometimes there are good replies 2-3 years later 🙂
Thanks for the encouragement!
Recommended reading about KBCs: https://blogs.msdn.microsoft.com/oldnewthing/20040917-00/?p=37833#comment-211593
While I was reading about PS/1, I found out about UNIXBOOT.COM. OS/2 2.0 probably special cases the model/submodel bytes for the PS/1 model 2121.
There were HP Vectra (and perhaps other HP PC’s?) that had a keylock-ish thing even on for example Pentium Pro systems.
It didn’t use a physical key. You pushed the keylock button on the PC and then the keyboard were disabled until you typed in a password which you could set in bios.
It would be interesting to see how this would interact with dos 4 shell. 🙂
You actually managed to get MS\PC-DOS 4.0x installed in a VM?!?!?
I’ve never been able to get past the point where it asks you to insert a blank floppy for the SELECT COPY disk (it claims that the disk is write-protected, both when using a blank floppy image and when using an actual physical blank floppy disk, and refuses to continue); does it perhaps need the host system to have a built-in floppy drive (something my host machine doesn’t have)?
Also, keep up the great work! 😀
(Also also, were you able to patch the DOS 4.0x shell to work properly?)