Where Intel Processors Fail At Math (Again) 239
rastos1 writes: In a recent blog, software developer Bruce Dawson pointed out some issues with the way the FSIN instruction is described in the "Intel® 64 and IA-32 Architectures Software Developer's Manual," noting that the result of FSIN can be very inaccurate in some cases, if compared to the exact mathematical value of the sine function.
Dawson says, "I was shocked when I discovered this. Both the fsin instruction and Intel's documentation are hugely inaccurate, and the inaccurate documentation has led to poor decisions being made. ... Intel has known for years that these instructions are not as accurate as promised. They are now making updates to their documentation. Updating the instruction is not a realistic option."
Intel processors have had a problem with math in the past, too.
Dawson says, "I was shocked when I discovered this. Both the fsin instruction and Intel's documentation are hugely inaccurate, and the inaccurate documentation has led to poor decisions being made. ... Intel has known for years that these instructions are not as accurate as promised. They are now making updates to their documentation. Updating the instruction is not a realistic option."
Intel processors have had a problem with math in the past, too.
Intel Common Core i7 (Score:5, Funny)
with new maths
Re:Intel Common Core i7 (Score:4, Funny)
Re: (Score:3)
Thats true though. Using nearest integer rounding, 1.4 can be accurately represented as one and produce a sum of 2.8, which can be represented by three.
In other words, 1+1=3 for sufficiently large values of 1.
That would be 1.4 + 1.4 = 2.8. 2.8 could be then rounded to 3, at which point you could say 1.4 + 1.4 ~= 3,
Anything beyond that, though, is horseshit.
Re: (Score:2)
The idea of significant figures is to avoid grossly overstating the precision of a measurement (or a calculation involving measurements). For example, if substance A is measured as 25 g with spring-type kitchen scale and substance B is measured as 10.3682 g with a digital analytical balance, it would be misleading to report the result as 35.3682 g.
Taken out of context, without further instructions, the issue about 8 + 5 = 10 is unclear. As an instructor, I suspect that the intend was to see if the student c
Re: Intel Common Core i7 (Score:5, Insightful)
what's the problem, 8 + 5 = 10, absolutely true in base 13
Re: (Score:3, Funny)
"'8 + 5 = 10' is True or false"
Is True.
Re: (Score:2)
The kind of crap known as "significant figures".
Which don't apply here, because this is math. And even if it were a physics problem, "10" still has the same number of significant digits as "13".
Re: (Score:3)
> Writing a fancy word.
> And doing it incorrectly.
What this mean... (Score:5, Interesting)
Re:What this mean... (Score:4, Insightful)
I would test that theory first. I have a hunch some GPUs are going to take shortcuts with math that someone like the guy who wrote this article will object to.
Re:What this mean... (Score:5, Interesting)
Re:What this mean... (Score:5, Informative)
You might take a look at the article and at Intel's reply.
The issue is in sine, cosine, and similar trig functions, with an actual error of 4e-21. That error scales, of course.
Intel's documentation change basically says you should scale and reduce your numbers first before running the functions.
Consider what that level of error precision means. If you were measuring with a meter stick, you could be measuring the radius of electron charge radii with several precision bits left over. If you were measuring the distance between the Sun and Proxima Centari, you could do it in millimeters and have accuracy to spare.
Even though I've run HPC simulations most of my career, we've seldom needed more than around six decimal digits of precision; that's akin to variations of human hair width when working at the meter level. It's only a problem when someone throws some strange scale into the mix; we're running physics on the kg-m-s scale, and suddenly someone complains that their usage of microseconds and nanometers breaks the physics engine We answer simply, "Yes. Yes, it does." If you need to operate in both scales, you need a different library that handles it.
Finally, even the actual article admits this is mostly about documentation. "The absolute error in the range I was looking at was fairly constant at about 4e-21, which is quite small. For many purposes this will not matter. ... for the domains where that matters the misleading documentation can easily be a problem." He then points out that a bunch of existing math libraries know about it. He mentions that high precision libraries have different solutions and always have. He mentions that most scientists who need it use better, high precision libraries. And he details that is really just the rough approximations done on the FPU that already plays fast-and-loose by switching between 53-bit and 63-bit floating point values that have been documented as being only good for that kind of approximation since the 1980s. Everybody who works professionally with floating point for any amount of time already knows the entire x86 family (including AMD and Intel) dating back to the original coprocessor are all terrible if you need high precision.
Re:What this mean... (Score:5, Insightful)
AMD CPU:s reportedly return exactly the same values as Intel CPU:s. I'm guessing they do so for compatibility reasons, so that any workarounds that software developers have implemented work as expected.
Re: (Score:3)
AMD CPU:s reportedly return exactly the same values as Intel CPU:s.
What, for transcendental functions? That's both impractical and useless. No "compatiblity reasons" could potentially justify this. If you take the table maker's dilemma into consideration, there's absolutely no reason to standardize on specific implementations of transcendental function - there's no fundamental simple way of "doing them right". I don't believe that IEEE-754 even standardizes anything beyond basic operations, and for good reason; the thing you're proposing could easily lull numerical develop
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
How is there no "correct answer" to e.g cos(0.4) to some specified accuracy? It's right or it's not. Sure, there may be some undefined bits if the precision of the result exceeds the required accuracy, but to whatever accuracy was promised, the result is correct or not.
Re: (Score:3)
Re: (Score:2)
But you can declare, in your docs, what accuracy you promise, and then either succeed or fail at that. And that's the issue here, no? Intel fails at their own published accuracy.
Total rubbish article (Score:5, Informative)
HOWEVER, if your argument is an extended precision number close to pi = 3.14..., then the last bit in the mantissa of that number has a value of 2^-62. So if you calculate an argument close to pi, the unavoidable bounds for the rounding error are 2^-63. This error in the argument is about 10 times larger than the error caused by using an approximation for pi in the argument reduction. If you use double precision, the error in the argument is about 20,000 times larger than the error caused by the argument reduction.
All this has been known for years; posting it today and claiming there is any problem is just ridiculous.
Re: (Score:2)
Re: (Score:2)
Get a processor that isn't crippled in the Floating Point department, if you really care that much.
Intel sucks less then AMD does on that one (1/3 speed on Core i-series (1-3rd gen tested) vs 1/5 on FX). FP performance is much better on other types of processors - MIPS, Itanium, and POWER all show much better FP performance then x86-based processors (and SPARC runs faster then AMD).
Re: (Score:2)
Do you have any benchmarks to back that up? The ones I see from SPEC [specbench.org] pretty much has Intel dominating in FP (compared to what few Itanium and POWER results there are).
Re: (Score:2)
Re: (Score:2, Interesting)
Re: (Score:2)
Re: (Score:3, Interesting)
Re: (Score:3)
In an office environment it adds up fast when you have 100 computers. Also you need a more expensive power supply by going with AMD systems. True the first i7 sucked 200 watts of power but the 4770k is really efficient.
Re: (Score:2)
Actually, GPUs do maths very inaccurately. They trade non-standard floating point implementations for speed.
Re: (Score:3)
Re: (Score:2)
Perhaps, but keep in mind the standard doesn't specify everything. According to TFA, this Intel issue with fsin does not technically violate the IEEE-754 standard on how sin() should be calculated.
Re: (Score:2)
Re: (Score:2)
The AC's claim is wrong. The inaccurate math in GPUs is nothing to do with defects and everything to do with performance tuning. A simple Google search will tell you why and how, and more importantly what you can expect especially if you're using the GPU for general purpose computing tasks.
Mathworks published a long paper a while ago on what you can offload to a GPU accurately and what you shouldn't, and there's been many studies on the accuracy of the single precision floating point units in GPUs as well.
Re: (Score:2)
But generally only single-precision. GPUs are made for speed, not accuracy - gamers want their frame rate high, and don't care if a few pixels are shifted one space to the right.
Re:What this mean... (Score:5, Informative)
On the consumer side, AMD GPUs rock at double, while nVidia's suck.
Re:What this mean... (Score:4, Informative)
To be fair, almost no consumers have any use for double. And commercial entities who do usually don't mind the extra zero at the end of GPU's cost, because to them, that's just expenses to be written off on their taxes.
Exact mathematical value isn't the ideal (Score:5, Insightful)
The main goal for Floating Point coprocessor sine calculations is to get a good enough result in a set number of cycles.
Given that fully approximating sine takes about as many concrete operations as bits in the value, getting it exactly right isn't usually a trade off people want to make.
There's a reason the C standard specifies that mathematical trig functions are platform dependent. If you want it precise, do it yourself to the level of precision you need.
Re:Exact mathematical value isn't the ideal (Score:5, Insightful)
This won't matter to a gamer playing some first-person shooter.
Re: (Score:3)
Well they would if the shooter was designed to apply, say, the character rotation as a delta versus as an absolute. That operation uses a lot of sin/cos, most games are designed such that the angle is stored, the delta updates the angle, and the rotation reapplied on update. Versus rotating the vertices based on the delta from the update, and saving the result (until the next update). You do the latter too much and eventually your object looks like poo. Mathematically, it's perfectly acceptable, but practic
Re: (Score:3)
Anyone who deals with floating point math very quickly learns about error accumulation and how to deal with it.
Re: (Score:3)
Re:Exact mathematical value isn't the ideal (Score:5, Interesting)
Sorry, but anyone relying on this for scientific use where the answer matters should be using software that gives them the accuracy they want and - ultimately - are the only people who will realise whether the result is correct "enough" or not for their process.
Some idiot researcher who expects Excel or an FPU instruction to be accurate for sin to more than 10 decimal places is going to crop up SO MANY anomalies in their data that they'll stick out like a sore thumb.
Nobody serious would do this. Any serious calculation requires an error calculation to go with it. There's a reason that there are entire software suites and library for arbitrary precision arithmetic.
I'm a maths graduate. I'll tell you now that I wouldn't rely on a FPU instruction to be anywhere near accurate. If I was doing anything serious, I'd be plugging into Maple, Matlab, Mathematica and similar who DO NOT rely on hardware instructions. And just because two numbers "add up" on the computer, that's FAR from a formal proof or even a value you could pass to an engineer.
Nobody's doing that. That's why Intel have managed to "get away" with those instructions being like that for, what? Decades? If you want to rotate an object in 3D space for a game, you used to use the FPU. Now you use the GPU. And NEITHER are reliable except for where it really doesn't matter (i.e. whether you're at a bearing of 0.00001 degrees or 0.00002 degrees).
Fuck, within a handful of base processor floating point instructions you can lose all accuracy if you're not careful.
Re: (Score:2)
"I'll tell you now that I wouldn't rely on a FPU instruction to be anywhere near accurate. If I was doing anything serious, I'd be plugging into Maple, Matlab, Mathematica and similar who DO NOT rely on hardware instructions. And just because two numbers "add up" on the computer, that's FAR from a formal proof or even a value you could pass to an engineer."
That depends on what kind of FPU you are using. The Power 6/Power 7 Decimal Floating Point unit is sufficiently accurate for engineering use
Re: (Score:2)
Hello,
As a maths grad working with computers, you probably have to rely on documentation for any tool you're using, right? The article is claiming the documentation is inaccurate. If we can't rely on the documentation to be accurate, what can we rely on? Maple, Matlab, and Mathematica ALSO rely on the documentation being accurate. If they told you one precision, and you got another, might you not complain, and want that information widely spread so they're more apt to fix it?
Also, I've noticed that Math
Re: (Score:2)
www.smbc-comics.com/?id=2999
Re: (Score:2)
Re:did you even bother looking it up? (Score:2, Insightful)
They do:
https://docs.python.org/2/library/decimal.html
http://en.wikipedia.org/wiki/Arbitrary-precision_arithmetic
Just because your world is limited to .NET it does not mean there aren't other things out there... Did you even bother looking up?
Re: (Score:2)
Java also has a decimal type, though without operator overloading it isn't as pleasant to work with.
http://docs.oracle.com/javase/... [oracle.com]
Re: (Score:3)
Re: (Score:2)
The x86 instruction set has BCD instructions too. I don't think they see much use now - processors are fast enough that you can do decimal math without them. But they were needed at the time.
Re: (Score:2)
Read TFA. Not even a close approximation, and docu (Score:5, Informative)
The documentation says that the result will be correct until the last decimal place. So if the CPU says the answer is:
0.123 456 789 123 456 789
You have have a close approximation, accurate to the 17th decimal place, according to the documentation.
The problem is, the correct answer may be:
0.123 444 555 666 777 888
The documentation says it's fairly precise. In truth, it's only good to the fourth decimal place in some cases, whereas Intel documented the function to be accurate to 66 bits or so.
Re: (Score:2)
Well, no.
From TFA, the absolute error closely approximates 0.000000000000000000004.
So you'll only see a relative error as large as you're showing (off in the fifth decimal place), if the correct answer is something like 0.000000000000000012345, which might show up as 0.000000000000000012344.
example from TFA. try it (Score:4, Informative)
Here's an example from TFA:
tan(1.5707963267948966193)
actual -39867976298117107068
x87 fpu -36893488147419103232
error 743622037674500958.81 ulp
Re: (Score:2)
tan(1.5707963267948966193)
And the example is rubbish.
That number 1.5707963267948966193 is a decimal number. Before you get anywhere near calculating the tangent of that number, you have to convert it to a binary number. The error in that conversion will be about 2^-64. That means the argument that you pass to the FTAN instruction isn't actually 1.5707963267948966193, but a number that is different from this by up to 2^-64. The error that you get by not passing 1.5707963267948966193 but a slightly differ
Re:Exact mathematical value isn't the ideal (Score:5, Informative)
The error is not small. If you read the article, on certain very reasonable inputs (not pathological at all), you can sometimes wind up with only _four_ bits being correct.
Many scientific applications absolutely depend on fast hardware sine implementation. As you said, getting it exactly right isn't a tradeoff that people usually want to make.
This has nothing to do with the C standard. Intel's own documentation was incorrect, making 'YMMV' completely moot.
Re: (Score:3)
The issue here, is that any computed sine value outside the first quadrant (input values 0 to pi/2) is computed by reducing the input quantity. The function is periodic, so adding or subtracting any multiple of the period (2 pi) from the input value, is mathematically valid. So, the error is made to be small for each value in that fir
Re: (Score:3, Interesting)
The problem isn't caused by the actual sine calculation, but in the preparatory range reduction, where the input value is mapped to the +-PI/2 interval. It appears that a "hard coded" approximate value of PI is the culprit, because the approximation is only accurate to 66 bits, but for the correct result, it would have to be correct to 128 bits. AMD at one point made processors which used the full precision value of PI and returned correct results for fsin(x). It broke software, so AMD "fixed" it by breakin
Re: (Score:2)
Re: (Score:2)
It really would be for the greater good.
Re: (Score:2)
I recall working with numerical methods from about 40 years ago, and all of the calculations that required a call to sin were range reduced to the region of +/- pi/4 anyway. The reason is that the taylor series expansions for sine and cos are most accurate in the region of zero, and for values in excess of pi/4, it is more accurate to do a transformation and implement a different call.
It is likely that the serious numerical code already handle this condition inside the internal algorithms.
Re: (Score:3)
I recall working with numerical methods from about 40 years ago, and all of the calculations that required a call to sin were range reduced to the region of +/- pi/4 anyway. The reason is that the taylor series expansions for sine and cos are most accurate in the region of zero, and for values in excess of pi/4, it is more accurate to do a transformation and implement a different call.
If you read the article, that's what the Intel processor does. Instead of an infinitely precise value of pi, it uses pi rounded to 66 bits (which is 13 bits more than normal double precision arithmetic would use, and 2 bits more than extended precision arithmetic would use). So all these people getting all excited about an error in Intel's FPU most likely wouldn't be capable of implementing the sine function anywhere near as precise as FSIN does.
Re: (Score:2)
> AMD at one point made processors which used the full precision value of PI and returned correct results for fsin(x). It broke software, so AMD "fixed" it by breaking it in microcode.
Do you have a link for that please? Thanks.
Re: (Score:2)
So, manually subtract a more accurate value of pi from the argument and pass that to fsin()? Sounds trivial enough for anyone really concerned about accuracy and speed.
More or less accurate than math textbooks ... (Score:2)
FPUs are **inherently** approximations (Score:2)
If you want it precise, do it yourself to the level of precision you need.
People just don't realize that FPUs are **inherently** approximations, anyone's FPU, its not Intel specific. There are inaccuracies converting to and from binary, there are inaccuracies depending on the relative magnitude of operands, the are inaccuracies due to rounding, etc ...
Do you know one way to tell if a calculator app is implemented using the FPU. Try 0.5 - 0.4 - 0.1, you may not get zero if a FPU is used. That is why handheld calculators often implement calculations using decimal math rather tha
Re: (Score:2)
The stack of errata on any current modern CPU is quite large. It is not just math functions...
I'm not talking about errata, about bugs, I'm talking about the inherent lack of precision when using a FPU. Its a design compromise inherent into all FPUs, inherent into IEEE single and double precision.
Re: (Score:2)
Re: (Score:2)
So use hardware floating point operations when it doesn't matter, like for games. But if you're going to calculate the trajectory of a mission to Pluto or something, bring the math out into your software, take the time, do it right.
Forget it. _You_ can't write an implementation of the sine function that is more precise than FSIN. Apple's implementation on PowerPC was more accurate; Java VMs must have a more precise implementation to pass Sun's compatibility tests. But anyone who knows about "Taylor polynomials" or "Pade approximations" or "CORDIC" will end up with something less precise.
Re: (Score:2)
Given that fully approximating sine takes about as many concrete operations as bits in the value
I'm not sure that this is even true - the way I understand it, nobody even knows how many operations it takes in the general case, if by "fully approximating sine" you mean "returning a result no more than half-ulp from the correct value for every valid input".
Re: (Score:2)
I'm not sure that this is even true - the way I understand it, nobody even knows how many operations it takes in the general case, if by "fully approximating sine" you mean "returning a result no more than half-ulp from the correct value for every valid input".
That's actually solved (for double precision). Google for crlibm.
Re: (Score:2)
Re: (Score:2)
They can. But they make tradeoffs because computers are expected to do the same calculations much faster (or were in the past, and now have to keep making the same mistakes for compatibility reasons).
Re: (Score:2)
I know nothing of processing, at least at this level. You often post good stuff here. Can you explain why computer processors cannot do the functions in the same way that calculators do? Maybe I'm asking the question in the wrong way, but you know what I mean (I hope).
The whole problem is that you may have been reading an article written by a blogger who made some rather idiotic assumptions about the precision of the sine function.
Here's the reality: If you look at posts on stackoverflow.com you will find to your horror that many "programmers" apparently don't even know the difference between "float" and "double" and do calculations with six digits precision instead of 15, out of stupidity. Intel processors also support "extended precision" which gives you >18 digi
Re: (Score:2)
Re: (Score:2)
burrrrrnnn
Re: (Score:2)
You apparently cannot reed.
That makes me wonder if you made a very clever joke, or weather you can't right.
Re: (Score:2)
The FSIN instruction is actually accurate to better (quite a bit better) than one ULP ... for the first quadrant. So they're not actually saving any CPU cycles by this "approximation".
If you calculate y = FSIN (x), then within the first quadrant y is within one ULP from the exact value sin (x). However, for _all_ values x, y will be equal to sin (x') for some number x' that is within one ULP from x.
Bad intel (Score:3, Funny)
We already know it's a sin to eat pi.
Re: (Score:3)
I ought to tan your hide, cos that was terrible.
Stop going off on tangents.
Inaccurate headline (Score:5, Informative)
The headline is quite inaccurate. The processors are doing what they're designed to do; approximate the results of certain operations to a "good enough" value to achieve an optimal result:work ratio. Sort of like how the NFL measures first-downs with a stick, a chain, and some eyeballs rather than bringing in a research team armed with scanning electron microscopes to tell us how many Planck lengths short of the first down they were.
This is a documentation failure. They're fixing the documentation. For anyone who would actually care about perfect accuracy in these kinds of operations, there are any number of different solutions to achieve the desired, more accurate result. The headline and the summary make it seem as though there's a problem with the processor which is simply incorrect.
Re: (Score:2)
For anyone who would actually care about perfect accuracy in these kinds of operations, there are any number of different solutions to achieve the desired, more accurate result.
...like using an AMD K5? [computer.org]
;-)
Re: (Score:2)
Re: (Score:2)
Unfortunately, IEEE 754 doesn't provide a way to indicate the level of precision (number of significant figures) of the answer.
It's not a bug (Score:2)
The Intel engineers watched Superman III, and they have a plan.
Re: (Score:2)
The Intel engineers watched Superman III, and they have a plan.
They're going to override the security?
We are Pentium of Borg. (Score:2)
I've got 98.999999997 problems (Score:2)
Here we go... (Score:2)
Round off the usual suspects...
To be expected (Score:3)
There's nothing I find particularly alarming here and the behaviour is in fact pretty much what I would expect for computing sin(x). Sure, maybe the doc needs updating, but nobody would really expect fsin to do much better than what it does. And in fact, if you wanted to maintain good accuracy even for large values (up to the double-precision range), then you would need a 2048-bit subtraction just for the range reduction! As far as I can tell, the accuracy up to pi/2 is pretty good. If you want good accuracy beyond that, you better do the range reduction yourself. In general, I would also argue that if you have accuracy issues with fsin, then your code is probably broken to begin with.
Article is wrong - it is documented (Score:3)
This is what the documentation says: "The internal value of Ï that the x87 FPU uses for argument reduction and other computations is as follows: Ï = 0.f â-- 2^2 where: f = C90FDAA2 2168C234 C". A more precise approximation according to Wikipedia would have been f = C90FDAA2 2168C234 C4C6 4...; the difference between pi and the approximation used by Intel is about 0.0764 * 2^-64.
If you let x = pi, then people would ordinarily expect that sin (x) = 0. That, however, would be wrong. Storing pi into a floating-point number produces a rounding error. Rounded to extended precision (64 bit mantissa) instead of the usual double precision (53 bit mantissa) produces a result of 4 * 0.C90FDAA2 2168C235 instead of 4 x C90FDAA2 2168C234 C4C6 4...; this is too large by 4 * (1 - 0.C4C64...) * 2^-64. The sine of that number would also be 4 * (1 - 0.C4C64...) * 2^-64.
But FSIN doesn't subtract pi from that number x, instead it subtracts 4 * 0.C90FDAA2 2168C234 C. So we get a result of 4 * (1 - 0.C) * 2^-64 instead of 4 * (1 - 0.C4C64...) * 2^-64. That's what he complains about. The reality is that the correct result would have been zero, but we couldn't get that because trying to assign pi even to an extended precision number gives some rounding error.
Now in practice, if you calculate an argument for the sine function, and that argument is close to pi, even if you manage to get a correctly rounded extended precision result, you must expect a rounding error up to 2^-63, and therefore an error in the result up to 2^-63, even if the calculation of the result is perfect. FSIN gives a result that is about 0.0764 * 2^-64 away from that, so the inevitable error caused by rounding the argument is increased by a massive 3.82 percent. Doing the calculation in double precision, as almost everyone does, makes the rounding error 2048 times larger and FSIN is now 0.00185 percent worse than optimal.
/fp:strict (Score:2)
/fp:strict
Failing that, use a pen and paper.
Dawson found a bug in gcc 4.3 as well (Score:3)
Dawson points to an 'optimisation' in gcc 4.3: constant folding is done using the higher-precision MPFR library [gnu.org]. At least the gcc developers seem to think it's an optimisation, but unless it's disabled by default, it is actually a bug. In the absence of undefined behaviour, optimisations must not change observable behaviour. And, as Dawson demonstrates, this one does.
If you need MPFR precision, you should use MPFR explicitly.
Re: (Score:3)
http://www.shsmedia.com/pentiu... [shsmedia.com]
Remember... "Don't Divide, Intel Inside"
Re:Say what? (Score:5, Funny)
We are pentium of borg. Division is futile. You will be approximated.
From what I remember it was the first revision of the Pentium 1 aka the Pentium 0.999998163849
Re: (Score:2)
What do you mean? An African or European SQRT?
Re: (Score:2)
Re:My workaround (Score:5, Funny)
The positive or the negative root?
I just use the average of the two, for predictable output.
Re: (Score:2)
Re: (Score:2)
Are those differences consistent?, are there differences between runs on the same machine?
Monte-Carlo rendering : from the name, that's using a lot of random numbers (like throwing a million darts at a wall and seeing what they hit.). So the software or even hardware implementation of random number generators may be important. Newer CPUs may have a built-in random generator which uses thermal noise - but the software has to use that ; and I presume it's foremost for security applications.
I suppose 3D render