So interestingly, processor makers are on the same page with you re: computations, and lots of processors can internally do computations in "extended precision", e.g. 80-bit floats, only converting to/from 64-bit doubles at the start and end of the computation.
IBM's new and rising supercomputer architecture, POWER9, supports hardware IEEE binary128 floats (quad precision). Their press claims the current fastest supercomputer in the world uses POWER9.
The ppc64 architecture (still produced by IBM) supports "double-double" precision for the long-double type, which is a bit hacky and software-defined, but has 106 bit mantissa.
And ARM's aarch64 architecture supports IEEE binary128 long-doubles as well, though it is implemented in software now (by compiler). Maybe they plan a hardware implementation in the future?
Essentially there are two different sets of floating point instructions on x86 and x86-64:
- the x87 instructions, which descend from the original 8087 coprocessor (and have 80-bit registers), and
- the SSE instructions, which descend from the Pentium MMX feature set, are faster, support SIMD operations, and can be fully pipelined.
The x87 instructions are basically for legacy compatibility, or if you manually use long doubles on some platforms.
The idea behind extended precision registers was good in theory, but ultimately caused too much hassle in practice.
Yep - and there are absolutely some cases where you do want to manually use it which is why the x86_64 ABI on SysV (used by Linux and OS X, still specifies the long double type as 80-bits, and why GCC and Clang will still emit these instructions when long doubles are used!
(Sorry, this is more for the folks who aren't familiar with this, since it seems like you are familiar, but I didn't want it to seem like this isn't widely supported when they read "legacy" or "some platforms")
Here is a good toy examples that runs into the same numbers shown in the parent, showing the two different instruction types, and that long double can give you the correct answer, while still being run in hardware, vs. going all the way to float128s which are currently emulated in software!
I agree that extended precision can be very useful, though I think the failing was on the software side: basically languages and compilers didn't provide useful constructs to control things like register spilling (which caused the truncation of the extended precision).
The current hardware trends seem to be providing instructions for compensated arithmetic, like FMA and "2sum" operations. I think this is ultimately a better solution, and will make it possible to give finer control of accuracy (though there will still be challenges on the software/language side of how to make use of them).
All the floating-point arithmetic that is natively supported these days is the 32- and 64-bit kind in SSE instruction sets and its extensions. The fact that something is "available" doesn't mean much in terms of actual support. As far as I know, long double means 128-bit floats in modern clang/gcc, and they are done in software.
Long doubles are typically 80-bit x87 "extended precision" doubles as far as I've seen. (Except on windows :-P ). It's part of the reason why LLVM has the 80 bit float type.
They are definitely still supported in modern Intel processors. That said, there can be some confusion because they end up being padded to 16 bytes for alignment reasons, so take 128 bits of memory, but they are still only 10 byte types.
They are a distinct type from the "quad" precision float128 type, which is software emulated as you mentioned.
All that being said, you are right that most of the time float math ends up in SSE style instructions, but as soon as you add long doubles to the mix, the compiler will emit x87 style float instructions to gain the extra precision.
And nobody uses this terrible mis-feature in practice, everything runs via 64 bit xmm registers.
Rightly so, because programmers want their optimizing compiler to decide when to put a variable on the stack and when to elide a store/load cycle by keeping it in a register. With 80 bit precision, this makes a semantic difference and you end up in volatile hell.
Yeah I agree that everything typically runs in XMM registers and that's what people want. I'm not sure what about the availability of extended precision makes it s a misfeature? For some cases it IS what you want, and it's nice to be able to opt in to using it..
EDIT: If I had some application where I needed the extended range, like maybe I was going to run into the exact numbers above, I'd appreciate the ability to opt-in to this. Totally agree I wouldn't want the compiler to surprise me with it, but also not terrible, or useless.
To be fair, the problem you describe isn't inherent to 80-bit floating point values. If you use 80-bit values in your ABI or language definition, it won't occur - it occurs when you try to user a wider type to implement a narrower type, e.g., implementing 64-bit floats (as specified in the ABI or language) with 80-bit operations.
In that case, the extra precision is present and "carried across" operation when registers or the dedicated floating point stack is used, but is discarded when values are stored to a narrower 64-bit location. So the problem is one really of mismatch between the language/ABI size and the supported hardware size. Of course, 80 bits isn't a popular floating point size any more in modern languages, so this happens a lot.
The x87 ISA does, yes, and they are supported for binary compatibility reasons. However the actual x87 registers are shadowed by the vector registers so you can only use one. Any modern vectorizing compiler uses the vector instructions for FPU arithmetic, even when scalar, with a max precision of 64-bit.
>The x87 ISA does, yes, and they are supported for binary compatibility reasons.
Well x86-64 is not binary compatible with x86 so that's not the reason. It is mostly for software relying on either the rounding quirks or the extended 80 bit precision I guess.
> However the actual x87 registers are shadowed by the vector registers so you can only use one
You are confusing with the legacy MMX registers which are deader than the x87 for stack. XMM registers do not shadow the for stack.
Fair point, I was a little fast and loose with my words there, which is definitely dangerous when it comes to things like C language / ABI standards! :-P
https://en.m.wikipedia.org/wiki/Extended_precision