SYSENTER, Where Are You?

It has only recently been brought to my attention that Intel’s SYSENTER/SYSEXIT instructions have rather unusual past, and their origin is shrouded in mystery and confusion. One facet of the usage of these instructions is also a little unorthodox.

Depending on who you ask, the SYSENTER/SYSEXIT instruction pair was introduced either in the Pentium Pro or in the Pentium II. But that makes absolutely no sense on the face of it, right? Either the Pentium Pro supported these instructions or it didn’t. Well…

The Pentium Pro (aka P6) was introduced in November 1995 and it was the first member of a tremendously successful family of processors which destroyed all of x86’s RISC rivals in the late 1990s, and quite possibly saved Intel after the Itanium and NetBurst debacles. The Pentium Pro and its immediate successors, Pentium II and Pentium III, turned out to be immensely scalable and between 1995 and 2001, raised the clock speed from 150 MHz (original Pentium Pro) all the way to 1.4 GHz (Pentium III-S and other Tualatin PIII variants).

Undocumented?

The Pentium Pro also introduced a few new instructions, such as CMOV/FCMOV (conditional moves) or RDPMC, and a number of architectural features (global pages, page address extensions, machine check exceptions, MTRRs). The Pentium Pro documentation from December 1995 (Pentium Pro Family Developer’s Manual, Volume 2: Programmer’s Reference Manual, Intel order no. 242691-001) certainly made no mention of SYSENTER or associated MSRs.

On the other hand, Intel’s U.S. Patent no. 5948097 (Method and apparatus for changing privilege levels in a computer system without use of a call gate, Glew et al.) filed in August 1996 describes SYSENTER in detail and says: “For one embodiment, SYSENTER and SYSEXIT are assembly-language instructions that may be executed on an Intel architecture processor, such as the Pentium Pro processor.”

The book Pentium Pro and Pentium II System Architecture (Shanley) claims that SYSENTER/SYSEXIT were introduced in the Pentium II but notes that “these instructions are not documented in the currently-released (as of 8/7/97) Intel manuals”. The book was published in December 1997. The book also says that SYSENTER/SYSEXIT capability is indicated by the SEP bit in the CPUID feature mask, but only if the processor is family 6, model 3, stepping 3 or greater. That is quite unusual and goes against the logic of CPUID.

The Pentium II Processor Developer’s Manual from October 1997 (Intel order no. 243502-001) contains absolutely no mention of SYSENTER or SYEXIT, and neither does the Intel Architecture Software Developer’s Manual from December 1997 (Intel order numbers 243190-001, 243191-001, 243192-001).

On the other hand Intel’s Application Note AP-485, Intel Processor Identification and the CPUID Instruction from June 1997 (Intel order no. 241618-007) is slightly older and does mention SYSENTER and SYSEXIT, but only explains how to detect the presence of the instructions, not how to use them. It is likely that AP-485 is the source of information in Pentium Pro and Pentium II System Architecture.

The second edition of the Intel Architecture Software Developer’s Manual is from April 1999 (Intel order numbers 243190-002, 243191-002, 243192-002) also covers the Pentium III and does properly document SYSENTER and SYSEXIT—as instructions introduced in the Pentium II. The instruction set reference gives the following pseudocode for detecting SYSENTER presence (already seen in the AP-485 CPUID note):

IF (CPUID SEP bit is set)
  IF (Family == 6) AND (Model < 3) AND (Stepping < 3)
  THEN
     Fast System Call NOT supported
  FI
  ELSE Fast System Call is supported
FI

It also says: “The Pentium Pro processor (Model = 1) returns a set SEP CPUID feature bit, but does not support the SYSENTER/SYSEXIT instructions.” Curiously, the SYSENTER MSRs are not listed in the MSR reference (Appendix B of Volume 3: System Programming) at all.

It is notable that the current (July 2017) Intel SDM lists the SYSENTER MSRs as architectural MSRs supported on the Pentium Pro and later (not Pentium II and later as one might think).

Documented Errata

In October 1998, the Pentium II Specification Update (Intel order no. 243337-019) added erratum A62, SYSENTER/SYSEXIT instructions can implicitly load “null segment selector” to SS and CS registers. The erratum does not affect a typical operating system and wouldn’t be very noteworthy on its own.

What is quite noteworthy is that the same erratum (SYSENTER/SYSEXIT instructions can implicitly load “null segment selector” to SS and CS registers) was also added as erratum 82 to the October 1998 edition of the Pentium Pro Processor Specification Update (Intel order no. 242689-032). The obvious question is how that is possible, if the Pentium Pro supposedly does not support SYSENTER/SYSEXIT?

Some have claimed that erratum 82 is in fact the reason why SYSENTER support should be disabled on Pentium Pros. That is extremely difficult to believe when the Pentium II has the exact same erratum.

Out in the Wild

It wasn’t until the early 2000s when SYSENTER/SYSEXIT started being used; given that public documentation may have only surfaced in 1999, that should not be surprising. Windows XP (2001) added optional support for SYSENTER and SYSEXIT. In 2003, Linux introduced SYSENTER support in the 2.6.0 kernel.

And immediately there was trouble. The initial Linux code caused system crashes on Pentium Pro CPUs. The reason? Linux implemented the model/stepping version check documented by Intel in the SDM instruction reference.

To this day (July 2017), the Intel SDM instruction reference claims that the check for CPUs which have the SEP bit but do not support SYSENTER is ‘IF (Family = 6) and (Model < 3) and (Stepping < 3)’.

But sometime in 2000 or so, the CPUID application note was changed to list the SYSENTER support condition to ‘IF (Family == 6) AND (ModelStepping < 0x33)’ which is certainly not the same! Notably for later-model Pentium Pros with model 1, stepping 9, the two conditions give different results.

As Linux users discovered, the condition given in the instruction reference is incorrect and SYSENTER (or is it SYSEXIT?) does not behave as expected even on later Pentium Pros.

An interesting question is the Pentium Pro OverDrive, in reality a Deschutes Pentium II. The PPro OverDrive identifies itself as model 3, stepping 2 (in reality it is a close relative of model 5 Pentium IIs). According to the CPUID test, it does not support SYSENTER. According to common sense, it really should.

So What Happened?

The unanswered question is what really happened (and even Andy Glew, the author of the SYSENTER patent and designer of the instruction doesn’t know). SYSENTER certainly was part of the Pentium Pro design. It seems likely that Intel discovered some problem with the SYSENTER instruction very shortly before the chip was released. And instead of documenting the problems and fixing them, Intel decided not to document the instruction at all.

Given that existing software had no expectations about SYSENTER, that didn’t cause trouble. Only after the Pentium II came out and SYSENTER was documented, it became apparent that it had been present on the Pentium Pro all along, but was either buggy or functioned differently. It is not obvious why Intel chose not to fix it in later Pentium Pro steppings.

So… what does SYSENTER and SYSEXIT actually do on Pentium Pro CPUs? How is it different from the later models? And what about the Pentium Pro OverDrive? So many questions.

This entry was posted in Intel, PC history, Undocumented. Bookmark the permalink.

10 Responses to SYSENTER, Where Are You?

  1. calvin says:

    > The Pentium Pro (aka P6) was introduced in November 1995 and it was the first member of a tremendously successful family of processors which destroyed all of x86’s RISC rivals in the late 1990s, and quite possibly saved Intel after the Itanium and NetBurst debacles. The Pentium Pro and its immediate successors, Pentium II and Pentium III, turned out to be immensely scalable and between 1995 and 2001, raised the clock speed from 150 MHz (original Pentium Pro) all the way to 1.4 GHz (Pentium III-S and other Tualatin PIII variants).

    And much more than that too – Pentium M was a lightly refreshed P6, Core 2 was a massively overhauled and improved design, but still P6 derived, and IIRC, everything after Core 2 is a descendant of that. So yeah, Kaby Lake is a (distant) descendent of the Pentium Pro. When you consider Intel’s last from-scratch x86 designs were Bonnell and NetBurst, that might be a good thing!

  2. Julien Oster says:

    This is now keeping me wondering almost as much as the question what the perpetually “reserved” CR1 ever did.

    At least with Pentium Pro’s SYSENTER implementation, we have a chance of finding out through experimentation (I consider it unlikely that they disabled the instruction, given both the timeframe and that it probably would have been easier to clear the flag in CPUID).

  3. Michal Necasek says:

    From what I read, the instruction exists and is not disabled on PPros (that’s what Linux users found out when 2.6 kernel enabled SYSENTER use). But it does not behave the same as on PII and later. So yes, it should be possible to discover what it does through experimentation.

  4. Michal Necasek says:

    Yes, the Pentium M was an updated/reworked P6, hybridized with the NetBurst front side bus and optimized for lower power usage. It is also my understanding that Core/Core 2 and the current crop of CPUs are still P6 derivatives, but I can’t judge if the similarity is still significant or not. But I think there’s no question that the Pentium Pro was a one of the most significant microprocessor designs.

  5. Richard Wells says:

    If it was an intermittent bug, it might be impossible to figure out what SYSENTER does on the Pentium Pro unless someone at Intel releases information describing a working case. I expect that there was a case that worked at least some of the time or Intel would have completely blocked the instructions.

    Note that Intel went through two Pentium II designs before getting a quad processor replacement for the Pentium Pro so it wouldn’t be completely impossible that the initial Pentium II fixes interfered with multi-socket designs.

  6. Michal Necasek says:

    In practice, not documenting the instructions was just as good as blocking them.

    The thing with late multi-socket PIIs was interesting. The black Pentium Pro with 1M L2 cache lasted quite a while.

  7. hobbes says:

    > As Linux users discovered, the condition given in the instruction reference is incorrect and SYSENTER (or is it SYSEXIT?) does not behave as expected even on later Pentium Pros.

    SYSEXIT. The LKML archive link states that SYSENTER worked fine and the system call executed successfully, only to double fault at SYSEXIT later. This suggests that SYSENTER to call and IRET to return could’ve worked and still be faster than INT + IRET.

    Anyone with a Pentium Pro machine and kernel debugging knowledge to test what’s actually going on with SYSEXIT around?

  8. Michal Necasek says:

    I have both, but currently not the time to set up an experiment. It should be possible to figure out just what exactly goes wrong on the Pentium Pros. I also want to check a Pentium Pro OverDrive because I’d be shocked if SYSENTER didn’t work there.

    I suppose back in the day, no one cared enough because when Linux introduced SYSENTER support, Pentium Pros were already quite outdated (there were probably ~3 GHz P4s on the market, and 1+ GHz PIII-based Xeons for servers were already old news).

  9. hobbes says:

    > there were probably ~3 GHz P4s on the market

    It was 2002, so Northwoods were brand new parts, and since it’s NetBurst, they weren’t much faster than an average abacus despite the clock speed. But yeah, point taken, PPro was already 7 years old technology and everyone was on PIII or P4, so nobody bothered.

  10. Michal Necasek says:

    The Linux 2.6 trouble post that I found is actually from August 2003. Either way, most of the world at the time was likely running at least a Pentium II, which itself was 5 years old in 2007 and far surpassed in performance in 2002/2003. That plus the Pentium Pro was never all that widespread to begin with. I bet there were lots more Pentium MMXs than Pentium Pros in operation in 2003.

Leave a Reply

Your email address will not be published. Required fields are marked *