ANOMALY: meaningless REX prefix used

Posted on June 20, 2018 by Michal Necasek

While trying to get work done, I was confronted by several disturbing messages printed on the console of a 64-bit Windows 7 system:

[0x7FEEFEFAAA0] ANOMALY: meaningless REX prefix used
[0x7FEEFEE48C0] ANOMALY: meaningless REX prefix used
[0x7FEEFEE54B0] ANOMALY: meaningless REX prefix used

This was disturbing because I pretty quickly established that the application I was running was not printing those messages. The expression “meaningless REX prefix” does actually convey comprehensible meaning to me, but I still don’t want to see it printed by some shadowy force, especially not without any clue as to why it might be printed.

A quick trip to a search engine only established the all-too familiar fact: there are lots of clueless people on the Internet. That includes people writing knowledge base articles. All in all, there are many avenues to ask the same question and get no useful answer.

There are all sorts of theories. It’s Windows 10. It’s anti-virus software. It’s Raptr. The winner is probably “my guess is that some non-Microsoft component (driver, software, utility) is installed and has “extended” the command processing environment with a REXX interpreter”. The old saying “better to remain silent and be thought a fool than to speak and to remove all doubt” certainly comes to mind.

Since I was not running Windows 10 or AVG, I could immediately dismiss some of those theories (and no, I didn’t take the REXX theory seriously for one second). But that didn’t help me much. So, where is that blasted message coming from? Continue reading →

Posted in Bugs, Debugging | 9 Comments

A Brief History of Unreal Mode

Posted on June 15, 2018 by Michal Necasek

After a run-in with a particularly crazy manifestation of unreal mode (Flat Assembler, or fasm), I decided to dig deeper into the history of this undocumented yet very widely used feature of 32-bit x86 processors.

For the purposes of this discussion, unreal mode is a variant of the x86 real mode with non-standard segment limits and/or attributes, different from the processor state at reset. To recap, real mode on the 286 and later CPUs has much more in common with protected mode than with the real (and only) mode of the 8086. Notably, undefined opcodes raise exceptions, segment limit overruns cause general protection or stack faults, and (on the 386 and later) 32-bit registers and 32-bit addressing can be used—subject to limit checks.

The origins of unreal mode are shrouded in the mists of time. But enough is known that certain outlines are quite clearly defined. Let’s present a rough timeline of unreal mode.

Continue reading →
Posted in 386, Corrections, Microsoft, PC history, Undocumented | 47 Comments

USB 0.9

Posted on June 2, 2018 by Michal Necasek

A couple of months ago I lamented the fact that historic USB documentation appears to have vanished from the face of the Earth. Today I finally found one such document, the USB 0.9 specification from April 13, 1995, published almost exactly nine months before the final USB 1.0 specification.

And where did I find the USB 0.9 specification, you may ask? Why of course, on my own hard drive. Not kidding. It’s been there the whole time. I really hate when that happens!

Anyway, back to USB. Back in December I complained that the Wikipedia article on USB claimed that USB 1.0 only specified the 1.5 Mbps low-speed transfer rate, and USB 1.1 added the 12 Mbps full-speed transfers. I also wrote that I had a vague memory of 12 Mbps being in fact defined first, and 1.5 Mbps added later, but I could find no evidence for that. Either way, USB 1.0 most certainly defined both speeds.

To recap, the March 20, 1995 issue of InfoWorld reported that “USB supports a 1.5MBps data transfer rate—compared to a standard serial port data transfer rate of approximately 400KBps”. A letter from the USB program manager at Intel published in the April 17, 1995 issue of InfoWorld stated that “USB supports a 12Mbps data transfer rate”. Note the capitalization.

In retrospect, it is apparent that InfoWorld simply screwed it up. When they wrote 1.5MBps for USB, they really meant 1.5 megabytes per second, which translates to 12 megabits per second. Except when they were talking about serial ports, they surely meant 400 kilobits per second.

After reviewing the USB 0.9 specification, I can say with certainty that the hints of low-speed USB transfers being a late addition are correct. The USB 0.9 specification defines only one transfer rate, 12 Mbps. There isn’t the slightest mention of any other transfer rate. That means the 1.5 Mbps low-speed transfers were probably added in the USB 0.99 specification in August 1995 (and yes, I checked, the USB 0.99 spec is not on my hard disk).

Posted in Documentation, PC history, USB | 30 Comments

Troubled Time

Posted on May 30, 2018 by Michal Necasek

This is not an article about current affairs

Over the last few weeks, I had several interesting run-ins with time, specifically how time is represented and processed by computers. Deep down it’s really all about a clash of human culture and history with physical reality.

At one extreme, there is local time, with noon exactly when the Sun is highest in the sky. Different depending on where you are, and exactly how humans worked with time throughout most of recorded history. That approach works very well as long as people and information can’t move much faster than about the speed of a horse. The 19th century introduced train travel and telegraph. If one sat on a train and started going eastwards or westwards, it didn’t take long for a pocket watch to get increasingly out of sync with local time. To solve that problem, and make it possible to maintain and publish usable schedules, time zones were introduced.

To solve a different problem, or perhaps cause more problems, the 20th century introduced daylight savings time. To cause maximum pain to computer scientists, daylight saving is not observed universally and is not constant. A real winner in this category is probably Egypt’s 2016 cancellation of daylight saving time three days before it was due to begin.

To communicate over longer distances, computers are forced to agree on a common definition of time. That is the other extreme: UTC, or Universal Coordinated Time, which conveniently doesn’t know any time zones or daylight saving and is the same everywhere on Earth (modulo relativity effects).

Sadly, computer users only care about local time, which means computers have to convert between local time and UTC all the time. That is merely complicated when that time is “now”, hideously difficult when the time is in the past, and impossible when the time is in the future. Continue reading →

Posted in Bugs, PC history, Random Thoughts | 18 Comments

A Word on the CALL 5 Spell

Posted on May 26, 2018 by Michal Necasek

After years of searching for some reasonably widespread DOS application which used the CP/M-style CALL 5 interface and coming up with absolutely nothing, Jeff Parsons of pcjs.org found one: None other than Microsoft Word, specifically the spell checker in the DOS-based versions of MS Word 2.x and 3.x. These versions were sold roughly from 1985 to 1987.

Microsoft Word spell checker

What’s significant is that Word 2.x/3.x was obviously Microsoft’s own product, and was sold and supported during the years when OS/2 was in development (and OS/2 needed to manipulate the A20 line) and high memory (HMA, managed by HIMEM.SYS) was on the drawing board. Much like EXEPACK, Microsoft Word 2.0 for DOS actually post-dates the IBM PC/AT with its A20 gate circuitry. It is highly probable that no one at Microsoft even realized the dependency of the MS Word 2.x/3.x spell checker on the A20 gate until years later. Continue reading →

Posted in DOS, Microsoft, PC history | 10 Comments

ICEBP Finally Documented

Posted on May 25, 2018 by Michal Necasek

After more than 30 years, Intel finally documented the INT1 instruction, also known as ICEBP (opcode F1h), in the latest (May 2018, -067) edition of the SDM. This was probably forced by security concerns, because from a security standpoint, having undocumented instructions which trigger special interrupts from user mode is insane, and Intel does not need more bad press than it already has.

The situation was fairly ridiculous, with most system software developers well aware of what ICEBP does, despite Intel’s abject failure to document what has been a part of the x86 architecture since the 386 (released in 1985). It helped that AMD’s 64-bit architecture manuals provided enough information about INT1/ICEBP for well over a decade.

Things got doubly ridiculous when Intel’s hardware virtualization (VT-x) documentation needed to talk about a “privileged software exception” without providing the slightest hint how such a mysterious thing might happen (why yes, executing ICEBP!). That, predictably, caused problems.

To recap, INT1 aka ICEBP (opcode F1h) is a single-byte breakpoint instruction similar to INT3 (opcode CCh), with the interesting property that it does not set any bits in DR6 and triggers a #DB exception without any privilege checks. It is intended for use with hardware debuggers (In-Circuit Emulators or ICEs) and conceptually behaves much like a software-triggered unmaskable hardware interrupt. That is why it needs to be handled specially by hypervisors, because it’s not quite like other exceptions and not quite like a hardware interrupt. It is however perfectly usable by software debuggers with no ICE in sight. System-level debuggers may even prefer using ICEBP to avoid conflicts with application-level debuggers.

Documenting an instruction after more than 30 years might, or at least should, be a world record. Intel is expected to break its own record once it fully documents the SALC instruction (opcode D6h), which has been with us since 1978 (introduced in the 8086). For decades Intel pretended SALC does not exist, in recent SDM editions the instruction is mentioned by name together with its opcode and a brief description, but not documented in the instruction reference and left out of the opcode map.

Posted in 386, Documentation, Intel, Undocumented | 14 Comments

SpaceMaker Update

Posted on May 17, 2018 by Michal Necasek

Jeff Parsons has been able to locate an executable compressed with Realia SpaceMaker which significantly pre-dates all hitherto known SpaceMaker or EXEPACK survivors. It’s an editor called DVED.COM version 6.02, found on disk 191 of the PC-SIG Library 8th Edition CD-ROM. The DVED.COM file is timestamped September 14, 1983 and its accompanying READ.ME file is dated September 18, 1983.

DOS timestamps are not necessarily trustworthy, and timestamps from before the PC/AT era (and built-in clocks) are notoriously unreliable. But in this case, the version history in the READ.ME file clearly talks about DVED “V6.02 (September 14th, 1983)”, which matches the COM file’s timestamp.

The 1983 DVED.COM file is certainly compressed with SpaceMaker. It has the same MEMORY$ signature, no long runs of zero bytes, as well as stub loader code at the beginning and end of the file that’s very similar (but not identical) to what SpaceMaker 1.06 (1986) produces.

Could DVED.COM have been compressed at some point well after 1983? Yes, but there’s an excellent reason to believe that’s not the case. DVED stands for Dewar Visual Screen Editor, and it was written by the same Robert B.K. Dewar who also wrote SpaceMaker. The  on-line help for DVED in fact explicitly mentions SpaceMaker as a product available from Realia (DVED itself was a freebie). If anything should have been compressed with SpaceMaker, it’s DVED.

The 1983 version of DVED.COM can be examined live on pcjs.org, just load PCSIG08:DISK0191 into the machine and run DVED.

Can anyone find an even older executable compressed with SpaceMaker (or anything else for that matter)?

Posted in PC history | 10 Comments

Realia SpaceMaker

Posted on April 30, 2018 by Michal Necasek

A recent exploration of Microsoft’s EXEPACK posed the question whether EXEPACK was the first executable compressor, at least in the world of PCs. It wasn’t.

That distinction almost certainly belongs to Realia SpaceMaker, which was probably released sometime in late 1982. Why “probably”? SpaceMaker is so old and so obscure that the oldest (and only) surviving version (1.06) is from 1985, in fact newer than the earliest known EXEPACK.

SpaceMaker was covered by the trade press, though not exactly extensively. Peter Norton mentioned it in his Hard Disk Housekeeping article in the January 8, 1985 issue of PC Magazine. That is the most solid piece of evidence that SpaceMaker is older than EXEPACK. SpaceMaker was written about in December 9, 1985 issue of Fidonews (The PC’s Space Program, by Gene Plantz). It was also mentioned in the January 1986 issue of Compute! magazine (page 119, Last Minute Gifts).

Note that the first known coverage is from 1985. So how do we know that SpaceMaker is from 1982? In the absence of surviving binaries, it’s difficult to prove. But SpaceMaker was advertised in the January 1983 issue of PC Magazine, which probably hit the newsstands in late 1982. And it was listed it as being developed in 1982-1983 (together with the Realia Termulator terminal emulator) in a legal document (page 20 in the PDF) which ought to be a reasonably trustworthy source of information.

If only we could ask the author to clarify the timeline. But alas, we cannot. The author of Realia SpaceMaker, Robert B.K. Dewar, died in 2015. Continue reading →

Posted in Compression, Development, PC history | 13 Comments

Undocumented RDTSC

Posted on April 27, 2018 by Michal Necasek

The other day I wrote a simple DOS program which used the RDTSC instruction in order to obtain precise time measurements (of how long it takes a PS/2 keyboard to send data; more about that some other time). The 16-bit DOS program worked fine in a VM running PC DOS 2000, as well as in NTVDM on Windows running on a system with a Core 2 class CPU. But when I tried running it on an old IBM ThinkPad 760XL laptop with a 166 MHz Pentium MMX processor running PC DOS 2000, it just hung when executing RDTSC. Unless I disabled EMM386, and then it worked fine.

That was a bit of a mystery. The Pentium MMX does obviously support the RDTSC instruction. The only troublesome possibility would be if perhaps someone set the CR4.TSD bit (the Time Stamp Disable bit) which would cause RDTSC (and RDTSCP on newer CPUs which support it) to #GP fault outside of ring 0. But old EMM386 does not even touch CR4, and at any rate the same program with the same EMM386 version worked on a newer CPU. Could the ThinkPad’s BIOS be setting CR4.TSD? Continue reading →

Posted in Documentation, Intel, Pentium | 15 Comments

The A20-Gate Fallout

Posted on April 13, 2018 by Michal Necasek

A recent post explored the motivation (i.e. backwards compatibility) to implement the A20 gate in the IBM PC/AT. To recap, the problem IBM solved was the fact that 1MB address wrap-around was an inherent feature of the Intel 8086/8088 CPU, but not the 80286 and later models—but a number of commercial software packages intentionally or unintentionally relied in the wrap-around.

Interestingly, it is obvious that the address wrap-around was much better known and understood in 1981 than it was in the 1990s. For example in 1994, the usually very well informed Frank van Gilluwe wrote in Undocumented PC (page 269): A quirk with the 8088 addressing scheme allowed a program to access the lowest 64KB area using any segment:offset pair that exceeded the 1MB limit. […] Although there is no reason for software to ever use this quirk, bugs in a few very old programs used segment:offset pairs that wrap the 1MB boundary. Since these programs seemed to work correctly, no actions were taken to correct the defects.

Yet it is known that Tim Paterson quite intentionally used the wrap-around to implement CALL 5 CP/M compatibility in QDOS around 1980, and Microsoft Pascal intentionally used it in 1981. In both cases there were arguably very good reasons for using the wrap-around.

Intentional or not, software relying on 8086 address wrap-around was out there and important enough that by the end of 1983, IBM had implemented the A20 gate in the upcoming PC/AT. But did they have to do that? Continue reading →

Posted in IBM, Microsoft, PC architecture, PC history | 93 Comments