Wednesday, September 11, 2013

A quick lesson in floating point arithmetic


Consider the following code snippet:

double a = 15.95;
double b = 19.95;
double c = a + a; //evaluates to 31.9
c = c + b; //evaluates to 51.849999999999994

This is a classic case of where floating point arithmetic falls down. This is working as designed, but not as expected. This can be easily fixed by using decimal rather than double, but lets explore this issue.

The crux of the issue is that some base-10 numbers, like 0.9, cannot be represented in base-2 accurately. Since most of us don’t speak base-2 (well, if you’re reading this you probably do), you could compare this to a fraction like 1/3, which in base-10 would be “0.3333333….”. We of course just truncate the rest of the 3s when we’re in base-10. It’s close enough, but what about for practical use?

For large financial calculations, this can cause issues. Remember the movie Office Space? They exploited this floating point vulnerability by taking those fractional remainders. For example, when we truncate “0.333333….”, they would take the “0.000000333…” that gets truncated from the number.

So back to our earlier example with 0.9. In base-2, it’s represented as “0.111001100…” which, when converted back to base-10 (which is exactly what’s happening above), is essentially “0.8999…”, or in our case, 31.9 eventually becomes 31.89999... instead. That’s exactly where the issue presents itself and why you should use decimal instead in these kinds of situations.

For more on decimals, Jon Skeet has a great writeup about it. Read it here.








No comments:

Post a Comment